Navigation SearchNavigation ContactNavigation Products
Resources

Rethinking the Right Answers for Testing

As the hiring environment grows more competitive and complex, both the function and form of testing are changing.

Over the past few years, the media has devoted a great deal of attention to the dramatic ways in which the business world has become more complex, the speed of change has accelerated, and the war for talent has grown more fierce. Staffing managers and HR leaders, however, understand that these trends are more than media hype—they are the new realities that organizations must confront as they seek to identify and hire the best talent. As a result, the ways in which HR thinks about and approaches testing has changed.

Consider the following two examples:

Joe applied for a manufacturing job in person, filling out a paper application, providing all of his “vital” information and work history. Now, several days after submitting the application, he reports to the company’s HR department and is sequestered in a room with a group of other applicants and presented with a paper-and-pencil test that will measure his skill level.

The test, which has long been part of the hiring process at the company, is viewed as valuable because it has been validated and also proven to be legally defensible. Joe, however, finds it to be confusing, little related to what he perceives as the duties of the job, and not a very good use of his time. Joe thinks to himself, “What do many of these questions have to do with this job?” Still, he soldiers through, hands his completed answer sheet to the proctor, and walks out the door. Joe feels a little frustrated and, what’s more, now he’s not so sure that this is a job he really wants, regardless of how well he performed on the test.

Mary has recently applied for a position with a financial services firm. Surfing the Web late one night, she came across the posting on a company job board, thought it looked intriguing, and decided to apply. As part of the application process, she completed an online test that asked her questions about her work experience and preferences. She was impressed with the ease and convenience of the process and felt as if, through the questions she was asked, she was actually learning valuable information about the organization and the specific duties of the job. (Unbeknownst to Mary, these job duties had changed recently as the organization embarked on a new business strategy.) She was even more impressed when, upon submitting her responses, she received immediate feedback, learning that she would be progressing to the next step in the selection process, which was a phone interview. In addition, as a final step, Mary was asked to provide feedback on her experience with the online application process.

The contrast between Joe’s and Mary’s testing experiences is reflective of the differences between the “old approach” to testing, which tended to be on-site and low-tech, and the way in which testing is commonly delivered today—off-site, online, on the candidate’s own time, or on-site at a company location using the latest technology. But a closer examination and a broader view of the types and applications of testing reveal differences that underscore the radical changes that define the current state of, and current thinking about, testing.

In fact, as is evident by comparing the Joe and Mary examples, many of the changes HR has enacted are significant, such as connecting testing to the organization’s business strategy, relying more on technology, repositioning testing within the selection process, and acknowledging and addressing the preferences of candidates. And there are likely more changes coming, as the hypercompetitive global economy and the talent demands of organizations continue to exert more pressure on organizations and their leaders to make the best-possible hiring decisions.

What does this continued evolution of the testing realm mean for your organization? In this white paper we will explore the forces that are helping to shape organizations’ testing strategy, detail the changes defining the current and future state of testing, and offer best practices we have observed in our work with DDI clients. (See the sidebar blow for an explanation of the various terms that commonly fall within the testing realm.)

How Does Testing Differ from Assessment?

The terms “testing” and “assessment” are frequently used interchangeably. Many tools fit under the broad umbrella suggested by these terms. Common examples include:

Psychometric testing:
Statistically calibrated measures of abilities, skills, knowledge, and personality traits. Results for test takers are often expressed in reference to a normative group.

Behavioral simulations:
Measures that replicate important aspects of a target job or role. These assessments measure competencies—related sets of behaviors, such as planning and organizing, which are important aspects of job performance. Simulations are usually scored by trained assessors using carefully constructed scoring guidelines.

Multi-perspective surveys:
Multi-perspective or 360-degree surveys assess competencies and other aspects of on-the-job performance as viewed by the subject’s manager, subordinates, peers, and/or clients.

Interviews:
When well-designed and appropriately structured, the commonly used employment interview can also be used to assess important competencies.

Why Testing is Evolving

As they strive to meet the challenges associated with powerful external forces, such as globalization, technology, increased competition, and the ever-accelerating rate of change in general, organizations have had to be more flexible than ever before. They’ve needed to formulate and execute new strategies, adapt their business practices and processes, and make sure that they have the talent to keep up.

This has meant that they have had to look anew at both the type of people they hire and also the systems and processes they use to select them. As for the type of people they hire, organizations increasingly are looking closely at their strategies and business needs in order to determine the combination of skills, knowledge, experiences, and personal attributes required for success in a given position. At DDI, we refer to this combination as a Success ProfileSM. Once determined, the Success Profile becomes the foundation upon which a sound, effective, and efficient selection system is built. And as strategies and business needs change, the Success Profile needs to change, too.

The processes, then, that comprise the selection system, have to engender the same degree of flexibility and evolve accordingly to meet the organization’s changing talent requirements. This degree of flexibility and evolution must extend to the organization’s approach to testing.

How Technology is Shaping Testing and Assessment

As with most other activities in which businesses engage, testing has been shaped—and reshaped—by advances in technology. With testing, however, this impact is especially dramatic due to the fact that the underlying logic for how tests work is essentially mathematical and is thus shifted into hyper-drive when fueled by the huge volumes of data that can be collected via Internet-based applications.

Yesterday’s tests, such as the one taken by Joe in the example above, were expensive to develop. Many years of effort could be devoted to the collection of the normative and validation data that is required to show that the test is measuring what it is supposed to and can be used to make valid employment decisions. These efforts would then be codified into a paper version of the test that would rarely change—for fear of having to repeat the significant development effort.

Today’s Internet-based test delivery systems are able to deliver tests in massive volumes to job applicants in virtually any location. In fact, it is now feasible to collect tens of thousands of cases of test-taker responses in just a day or two. This enables rapid research and development cycles that were not possible even in the recent past. The implications for tests used in selection have been dramatic for the organizations that have participated in this testing revolution. Huge databases have allowed for sophisticated research to shape online test content that uses questions that maximize their predictive ability.

Because the Internet enables global business processes, testing has also grown to meet the demands of global corporate hiring systems. Again, facilitated by careful analysis of worldwide data, testing researchers have been able to design algorithms that allow for a common test to be used while at the same time accommodating local differences in culture and language. And the growth of these capabilities is just beginning.

Combining with these new opportunities and innovations are a variety of other factors that have helped to shape the modern age of testing. At the top of the list is the increasing complexity of the business environment.

A More Complex Environment

Testing has long been a critical component of an accurate, effective selection process. The problem is that tests have often been used to measure as few as one or two independent aspects or skills required for success on the job. For example, a skill test might be used to measure an individual’s ability to use a spreadsheet software package, or a personality test would be included to see if he or she is extroverted enough to succeed in sales. Tests such as these have, in many cases, served the relatively limited purpose of a check on individual characteristics, seemingly disconnected from the rest of the selection system and certainly not purposely aligned with the organization’s business needs.

Today, that view of testing is changing as organizations are beginning to think in terms of “selection strategy” and are looking to testing and assessment as an integrated component that supports that strategy. There are many reasons for this change.

Perhaps no reason is as important as the increasingly competitive market for talent in which employers today must compete. DDI’s global benchmark study, the Selection Forecast 2006-2007, found that staffing directors believe the “war for talent” continues to intensify, with nearly three quarters of respondents predicting that the competition for talent will increase. As a result, a more strategic, efficient, and altogether updated approach to hiring and promoting talent is required—one through which more relevant candidate data can be gathered and analyzed to make the best-possible hiring decisions. That is, hiring those individuals who are the best fit with the job and the organizational culture, and who are, therefore, more likely to stay in the job for the long term.

Other factors are combining to define a highly competitive market for talent, as well. Organizations that are looking to expand globally to penetrate new markets are also going global in search of new talent, encountering multiple challenges in the process, including those related to language and cultural differences. Beyond the use of the Internet, other technological advancements, meanwhile, have opened up a new world of possibilities for testing with regard to flexibility, efficiency, and speed. Probably the most important and impactful of these is the emergence of the applicant tracking system, which allows organizations to realize greater efficiencies by managing the progress of candidates through the selection process, and simplifying and automating important steps.

On the delivery end, new technologies and new types of tests are emerging that take advantage of multimedia presentation of content, complex scoring of candidate responses, and easier connectivity, allowing for new methods of distributing the testing content. Not only are tests covering more ground with regard to what they predict—things such as engagement or personal motivations—but they are also providing candidates with a realistic job preview, allowing candidates to take a quick peek into what life will be like in the job.

In addition, in some instances testing can now be delivered in a multimedia format that has a game-like feel. This rich presentation can be highly engaging and can help sell the job to top candidates. In short, new technologies are creating a win-win testing and assessment environment in which candidates are benefiting from an enhanced experience while companies are better able to gather the data they need to make more accurate selection decisions. (The same benefits also are allowing organizations to make better development decisions, as testing and assessment capabilities are often used in a diagnostic capacity, as well as in support of hiring and promotion decisions. See “The Role of Testing in Development,” for more on this.)
 

The True Costs of Hiring Systems

The true costs of hiring systemsIt is not surprising that 79% of staffing directors cite turnover as a key measure of the success of their selection systems. Every time a new position comes open—whether from an increase in head count or through turnover—a cost meter starts running.

It can take months or years to recoup the costs incurred from the time the meter starts running until a new hire becomes fully productive. These costs really fall into three main areas:

  1. Hiring Process—How much does it cost your organization to find and place a new hire, including advertising and posting costs, time spent sourcing and screening candidates, travel and relocation costs? How long does it take to fill open positions?
  2. Ramp-up—Once a candidate accepts your job offer, how much extra time does his or her manager or co-workers have to spend with them? What training investments are needed to educate the new hire on internal processes or to accelerate their development?
  3. Productivity Gaps—What is the performance gap between your star performers and the rest? A 2000 McKinsey study found that high performers generate 40% or more annually than an average performer. The same study found that high performers in sales functions generated 67% more in revenue than average performers.

Testing and Quality of Hire

Another trend that has served to reshape how organizations look at and apply testing is the growing awareness of the cost of poor or mediocre hires. In addition to the obvious costs related to hiring the wrong people—costs that typically are incurred when those individuals leave on their own or are terminated—there also are the costs related to lower productivity and lost opportunities tied to the substandard job performance of these hires.

This growing awareness, in addition to pressures from senior leadership for HR to be more accountable, has fueled an increased scrutiny of hiring decisions and, accordingly, a need for more, and more accurate, information to inform these decisions. In turn, organizations are taking a closer look at the tests they are including as part of their hiring and promotion processes, questioning their validity and relevance, and seeking out updated, new, or additional tests that can help to provide a more complete picture of a candidate’s true capabilities and fit with the job and organization. They also are looking at the efficiency of their testing and assessment processes, as there is ever-greater pressure to not only reach the best possible decisions but also to reach them—and hire the best talent—more quickly.

Also related to the costs associated with hiring are the costs incurred when an organization’s hiring decisions and practices face legal challenges. Organizations are wisely taking a fresh look at their tests, as well as their entire selection processes, to ensure that they maximize validity while minimizing adverse impact. In many cases, organizations are finding that in order to meet legal guidelines—and, no less important, to successfully hire a more diverse workforce that can infuse the organization with a wide array of valuable perspectives and skill sets—they need to make adjustments to their selection processes in general and their testing strategy in particular.

The Next Generation of Testing

Predictably, with all of the changes and pressures described above, testing is entering a new phase that is very different from the staid approach to testing that predominated for decades. We have seen signs of this transformation in our work with hundreds of clients who are devising new testing strategies and overhauling their entire selection processes and approaches to testing, to make them more efficient, more effective, and more candidate-focused.

From this work,we have seen four trends that characterize the “next generation of testing.” Some of these trends, such as the prevalence of technology, took root years ago and continue to gain momentum.

Others, meanwhile, reflect new thinking that promises to become more widespread moving forward.

1. Changes in business/strategy/jobs mean that tests and assessments have to keep pace.

Consider a bank that has changed its strategy and plans to aggressively sell financial services products to its existing customers. While in the past, the bank might have hired tellers who could efficiently and accurately handle customer transactions, the new strategy will require that tellers ask customers questions that uncover additional sales opportunities. This change in strategy makes obsolete the past teller job description, as well as the hiring process for identifying and selecting tellers. In this instance, different or additional tests would likely be in order to properly screen or assess candidates. (In a similar vein, we have worked with organizations ranging from a large manufacturer that wanted its line people to take on additional maintenance tasks, to a telecomm organization that was changing its sales strategy from product selling to consultative selling. We partnered with each of these organizations to create a testing strategy that aligned with and supported these changes.)

In addition to keeping up with changing strategies and changing jobs, tests are becoming more dynamic and adaptable. For example, some organizations have decided that testing needs to occur earlier in the selection process. Others have determined that they need to do a better job of screening for motivational fit, not just for skills.

Many changes in testing strategy also have come in specific response to the requirements of the global economy. These include changes to accommodate language requirements and various cultural norms, as well as the need to make tests available 24/7 (online) and provide technical support and guidance to improve the testing experience for candidates. In some organizations, these changes in norms and language offerings have even been made to accommodate the growing diversity of the U.S.workforce, especially in locations and industries where a large immigrant population defines the talent pool.

2. Testing enables a more complete view of a candidate.

For decades, testing systems have been based on narrow abilities or skills. More recently, competencies have been added to further define job success. While job-related abilities and skills remain valuable, it often is imperative that employers are able to ascertain a more complete picture of a candidate in order to determine his or her true fit with the position. This is especially true in the case of positions for which hiring the wrong person can prove extremely costly, such as a leadership position, a key sales job, or a hard-to-fill individual contributor position requiring highly specialized competencies or technical proficiency.

In these instances, and even in some instances where the hiring need is for high volume, hourly positions, organizations are increasingly constructing broader Success Profiles against which to assess candidates. As they seek to gather a wider range of information on candidates, organizations are updating old tests or replacing them with new instruments that can efficiently test for multiple areas. For example, a large retail organization with which we work uses a multifaceted test that measures for factors related to customer service, integrity, reliability, and other areas. It administers this test to millions of applicants each year.

One such example is related to organizations’ growing desire to hire engaged employees. DDI defines engaged employees as those who take pride in their work, are actively committed to the job and the organization, and are willing to give their best efforts to accomplish organizational goals. Our Career Battery tests evaluate candidates’ capabilities, motivations, and interests, but the results of these tests also can be expressed in the form of an Engagement Readiness Index that has been shown to predict employee engagement—an efficient application of a single instrument to gather multiple forms of relevant candidate data. Employee engagement, in turn, is related to business relevant outcomes such as productivity and retention.

3. Testing is evolving with technology.

Yes, the Internet has opened up the job application process to a larger number of individuals, making it possible to apply for a job—and take a test as part of the application process—pretty much any time and anywhere. But there are other ways in which technology has changed the face of testing.

One new capability that testing has enabled is adaptive delivery, in which the succeeding test questions change based on candidate responses to earlier questions. On a skills or knowledge test, succeeding test items can be more or less challenging based on whether or not the candidate provides the correct response to an earlier question. This type of testing reduces the overall amount of time required to take the test—a key strategy for increasing candidate satisfaction and keeping candidates engaged in the process. While this type of testing has been used in the past in large-scale testing programs (e.g., military and academic settings), the use of the Internet with this adaptive approach now makes this capability available in employment settings. In fact, we are now working with a large consumer products company to implement a state-of-the-art adaptive testing system on a global scale.

Technology also has accelerated the blending of tests and simulations. Virtual work environments can be created and realistic job tasks assigned to candidates. An example of this might include a test of the Success Profile match for a manufacturing team member position, in which candidates visit a secure web site and are presented with typical manufacturing tasks such as process control, work standards, and production activities, all presented in the form of a computer-operated interface. Interacting with the interface, the candidates must address, in real time, the production, quality, and safety issues they encounter.

There is tremendous value in this approach for both the candidate and the company. Simulations such as these provide candidates with realistic job previews, giving them the information they need to determine whether they want to continue on or opt out of the selection process. Candidates appreciate this information as it helps to either confirm their interest in the job or help them realize they should not pursue it, as it doesn’t fit their needs and expectations For the company, these simulations achieve significant incremental gains in the ability to identify candidates who can learn faster and perform at higher levels in the target job, while not significantly impacting the investment required to set up and maintain the assessments. New tests and simulations have the ability to stream candidates into the best-fitting jobs, increasing the chance for early career success in the new position and positively impacting retention rates.

One of the most significant results of the emergence of technology in the selection realm is the seamless integration of testing with other phases. Now, a candidate can apply online, be screened using one or more tests, and advance to the next step in the hiring process (and be informed of this progress instantly, if desired), all within a few minutes sitting in front of a computer—a level of integration that once would have been unimaginable.

From the employer’s perspective, perhaps the most revolutionary aspects of this technology-driven integration are the ability to manage candidates through the selection funnel (tracking where each candidate is in the process relative to a final decision) and the portability of candidate data. The ease with which candidate data can now be centrally collected and applied to other uses, such as employee on-boarding or development planning, serves not only to make the data more versatile but, it could be argued, it also makes the data more valuable.

4. Tests should address candidate needs and preferences.

Due to the new flexibilities offered by technology, employers are now making greater use of the testing process to make a positive impression on candidates. Candidates have needs during their job search—for information, for communication, for clarity—and addressing these needs means making sure tests are convenient, easy to understand, relevant to the job, and respectful of the candidate’s time. Research has shown that candidates’ views of an employer are related to their reactions to the various steps in the selection process, including testing. By designing and managing the testing process in a manner that appeals to candidates, employers are able to reinforce their brand as an employer.

One way that organizations can confirm that their approach to testing is sending the right message is to seek candidate feedback on the perceived fairness and ease of use of the tests (through brief surveys, follow-up interviews with new hires, etc.). This feedback can help to identify problem areas and reveal adjustments or major changes to the testing approach that might be required.

Best Practices for Getting the Most out of Testing

As the trends above illustrate, a testing strategy that made sense even a few years ago may not make sense now. Likewise, a testing strategy that works today could be out of date in the near future. Testing is a dynamic activity that should always support the organization’s hiring objectives and, by extension, its business needs and strategy. What’s more, a testing strategy that is the right one for a given organization, industry, or job type may not be best for every situation.

While this case-by-case nature of testing prevails, our work with clients has revealed a number of best practices that define the most effective testing strategies. They include the following:

Tie your testing strategy to your business strategy. Organizations are growing increasingly aware of the need to build hiring processes around specific strategic priorities and business drivers, whether they are focused on going global, sparking innovation, containing costs, or any number of others. For instance, if maintaining market share is a major business driver for your organization, hiring people who can deliver exceptional customer service must be a priority. Your hiring strategy and, by extension, your testing strategy, should be linked to the need to keep customers in the fold.

Find ways to make testing technology work for you. As explained above, technology has revolutionized testing, especially with regard to access, efficiency, integration, capability, and data portability. We have worked with several organizations that are realizing many if not all of these benefits.

In addition, organizations are finding that testing technology allows them to increase the size of their applicant pools. A large hotel chain, for instance, looks at applicant test results and identifies those who may not be a good fit with the job for which they applied but who might be a worthy candidate for another position. This allows them to expand the candidate pool for the other position with which the applicant is a better fit.

Leverage test results for multiple applications. Just as tests can reveal an individual’s personality traits, competencies, skills, or knowledge, they also can reveal his or her development needs. This means that, at the point an individual is hired, the tests administered as part of the hiring process will have provided useful baseline data that can be applied to build an on-boarding plan or even a year-one development plan. In addition, these same tests also can be used to assess the readiness and development needs of individual contributors as they are groomed for future leadership positions.

Define testing needs through Success Profiles. Properly designed and administered tests can efficiently capture data about an individual relative to any of the four areas of a Success Profile—i.e., competencies, knowledge, experience, and personal attributes. Important to keep in mind, however, is that tests can’t provide sufficient data to fully assess for all of these areas. Instead, tests are best used and deployed in concert with and to supplement the information from other tools, such as interviews, simulations, or assessment centers. A multifaceted selection process is necessary for constructing a complete picture of a candidate.

Revisit and revise your testing strategy as your business and jobs evolve. As their business needs and strategies change, organizations are recognizing that they no longer can rely indefinitely on their older tests. Today, jobs, and candidate pools, are changing too quickly for employers to use tests indefinitely without regularly re-examining their connection to the target job.

Put the candidate’s needs and impressions on an equal par with legal considerations. It’s not the lawyers who will ultimately take your tests—candidates will. While, yes, you need to make sure that the tests you are using are legally defensible, you need to make sure that the tests are easy to understand, easy to take, and elicit the candidate information you need to make the best possible hiring decisions.
 

The Role of Testing in Development

While testing is essential to helping organizations identify and hire the best people, testing, of course, also plays an  important diagnostic role in development, an important fact to keep in mind when formulating your testing strategy.

Some might point out the differences between testing for hiring and testing for development, but from DDI’s perspective, they actually differ very little. Our reason for this view is that testing, whether for hiring or development purposes, involves:
  • The same tools. Many of the same tools skill/knowledge tests, inventories, simulations that are used for a hiring application also can be used to identify the strengths and development needs of individuals.
  • The same data. As mentioned in this white paper, the same testing data gathered as part of the selection process can be applied to create an initial development plan for the individual if he or she is hired for the target position. This dual use of data promotes greater efficiency by, in many cases, eliminating the need to also test new employees for development purposes.
  • The same goal. Regardless of whether you are testing for hiring or development purposes, the ultimate goal is the same: to gather the important information about an individual to make the best possible decision concerning their fit—whether it’s a current or future job or role.

Use testing to help build your employment brand. Employers sometimes forget that a test is no different from an interview in that it is another point of contact with a candidate. Therefore, a test also presents another early opportunity to make a positive or negative impression.

The tests you use and how you administer them can send important signals to candidates about your organization as a place to work. A sophisticated online test, for example, may communicate to candidates that the organization is technologically savvy. An appropriately challenging test may also send the message that the organization seeks to hire only the best, and those who perform acceptably on the test, and are eventually offered the job, may feel that they are being invited to join a prestigious employer that “won’t just hire anyone.”

Does Your Testing Strategy Make the Grade?

When it comes to being at the leading edge of testing, how do you know where your organization stands?

To find out,we suggest that you pose and honestly answer the following questions:

  1. What are you trying to measure with your tests and how does this relate to the job of interest?
  2. How often do you need to revalidate or recalibrate your tests? Have your jobs evolved?
  3. How does your testing strategy tie to your business strategy?
  4. How regularly do you re-evaluate your testing strategy?
  5. How do you prove the value of your tests within your organization? Can you use the answers to the four previous questions to sell the value of testing to your organization?

How you answer these questions will reveal a great deal about your organization’s current testing strategy—whether you are making the grade or whether you need to carefully re-evaluate your approach to testing, if not your entire approach to selection.

DDI offers a wide array of testing solutions, including tests that are specifically designed for use in certain industries. To learn about our testing offerings, contact your DDI representative or call 1-800-933-4463.

Talk to an Expert: Rethinking the Right Answers for Testing

* Denotes required field
 *
 *
 *
 *
 *
 *
 *
 *
 Security code