1. Make
sure assessment results are being
interpreted by licensed/PhD PsychologistsPsychologists (particularly industrial/organizational, clinical and counseling) have training in test development, administration, interpretation and
experience providing feedback. It is
best to have licensed consultants. The
firm should have at least one on staff who knows how to work with questionnaires.
2. Confirm
that the assessments are valid and that the measures are reliable and have been
tested over timeThe testing industry is not really
regulated, so anyone can put 10 questions on a website and say it is a great
measure. Ask for the technical
reliability and validity research behind each test. You probably don’t want to read it, but at
least you will know whether it exists. There are many excellent questionnaires that
have many years of research behind them.
3. Verify
that candidates will be given an assessment battery, versus a one-test solutionMost tests measure one thing well. If the vendor tells you the test is
comprehensive and does everything, then it probably does not measure them all
very well. The key things that measure
success in a job are listed below in the order of how well they predict. At entry levels, there may be one trait
that is good enough (conscientiousness), but at higher levels, the jobs are
complex and multiple areas should be measured.
1. Problem-Solving
2. Work
Personality Traits
3. People/Influencing
Personality Traits
4. Motivation
5. People/Task
Management Style
6. Sales
Style
4. Be
wary of brief tests - generally longer =
more accurate
Most people do not like to even think
about statistics. But statistics tells
us that the more examples we have of something, the more reliable the averages
and other measures. Tests are built on
statistics, so the more questions, the more reliable the output becomes. As a rule of thumb, the test should have about
15 or more questions for every trait it claims to measure. So, if the test claims to measure 10 traits,
then it should be around 150 questions (or more). Some tests have even more questions, which
often leads to very high levels of reliability.
5. Match
test capabilities to organization/situation
It is critical to have a clear
understanding of what you want the test to tell you. In a selection assessment, it is critical to
have an understanding of the job the person will fill (job analysis). It is also important to understand the
culture of the facility or company.
6. Evaluate the metrics
Make sure the assessment firm is
collecting data to understand whether their process is giving accurate
results. This is critical to insure the
assessment is not biased against particular groups (minority groups, women,
etc.). On a practical level, they should
be invested in whether their assessment works.
There is a variety of data they can combine to understand this. They should be pushing a validation study,
rather than waiting for you to bring it up.
7. Determine
the security of your data
Have your IT people check the safety
of your data. There are security issues
in terms of access to your data. There
are also security issues in terms of back-up.
If you are a healthcare firm, the firm needs to meet additional HIPAA
requirements. Some assessment firms are
small and have the data stored on a server in their office.
8. Establish
that you will have access to your database
If the data is stored safely, do you
still have access when you want it? With
the right system, the firm can secure your data and allow you access. Also, if you decide to stop using an
assessment vendor, how long will it take you to get your data? Many assessment firms require several weeks
to provide data and reports to new assessment vendors. Remember, you own the data.
9. Be
wary of professionals representing only one test company
Think about the professional tools
your industry uses. Each of your vendors
tends to have one service/product that it does quite well. Some vendors try to be all things to all people. As a result, they may have one great product
and many mediocre ones. The assessment
world is the same. So, if your
assessment firm is representing only one test producer, they probably do not have
access to the best tests in the various areas that you want to measure. Also, assessment firms that represent only one product line are making their living through that product. So, they are not looking for the best
solution for you, they are looking to sell more.
10. Confirm
there are no hidden fees
Often, assessments may be priced lower
on the surface. Then the assessment
report may not be completely clear, so the assessment firm charges extra to
walk you through the results.
BONUS TIP - Make sure
the “test” answers are not available on the internet
This
may seem obvious, but since there are so many tests, many have been shared on
the internet.