Welcome to our first session in Psychometric Testing!
In this first session we will explore the following:
1. What is a psychometric test?
2. A brief background of psychometric testing.
What is a psychometric test? How are they developed?
Let’s start out by telling you what a psychometric test is not!
You may have come across various different online tests. Not all of them are psychometric. Whether they are or not will depend on how they were designed, for what purpose and what they intend to measure. For example, you may have come across the Thematic Apperception Test or the Rorshach Inkblot Test. These tests are used mainly in clinical settings. The client is asked to look at scenes or pictures and to articulate what they see. Such tests are supposed to be able to assess the unconscious mind. However, interpretation is not as objective as we would like to see in occupational testing and assessment. Furthermore, what the client “sees” may be based on external factors such as culture, upbringing or season of the year.
Psychometric Tests are different! They are defined as quantitative (numerical) assessments of one or more psychological (in the head) attributes. So, psychometric tests are used to assess in a quantitative way things like numerical reasoning skills, verbal aptitude, extroversion, conscientiousness and so on. That alone does not make a test psychometric of course. In order to be psychometric the test must have been designed to be:
1. Administered in a standardised manner
2. Scored in a standardised manner
3. Interpreted in a standardised manner
4. Constructed according to psychometric principles
You’ll see later how important standardisation is throughout the use of psychometric tools. In fact standardisation is a critical element of all scientific HR processes.
As for being constructed according the psychometric principles, what we mean here is that the test must have gone through each aspect of a very scientific development process. It is not good enough to simply write down a few questions, produce a flashy report and start selling a new test! This is the process it must go through:
a. Development of a rationale behind the test that is supported by research. For example – if I decide to develop a new test of extroversion, the scientific research literature and models must inform the design of my questions and the aspects of extroversion that I attempt to assess.
b. Next I need to write some experimental questions. At this stage I’ll write more questions than I intend to have in the final version of my test because I am prepared to throw some out based on feedback.
c. Now I’ll find a sample of people (who represent the group I am designing the test for) and I’ll ask this sample to complete my experimental test. In other words, I pilot my test.
d. Next I head back to my office and assess the responses and how they relate to each other statistically. At this stage I am running item analysis to test that similar questions (items) are indeed related to each other in the way I would expect. More on this later. The point to grasp now is that this is an iterative process. Things won’t be perfect the first time around. Based on the statistics I’ll need to remove some questions, refine others and then go back to point C. I’ll do this again and again until I am happy with the statistics I get at point D!
e. Now that my test questions are performing well I need to enter a standardisation phase. Here, a larger group of people will complete my test and that will show me where people tend to score on the test. This group will become my benchmark or norm group later and add meaning to the scores of future test-takers.
f. The next question is “Does my new test actually assess what it is supposed to assess and/or does it predict something meaningful?”. As an example, you would expect my numerical reasoning test predict success of accountant trainees. This stage is called Validation, we are assessing the validity of the test or whether it is fit for purpose.
g. Now, we all know from science classes at school that all good experiments end with a write-up! That’s exactly what we do at the final stage of psychometric test development. We need to write up all of the above stages in a long document which is called the test’s technical manual. It is this manual that prospective clients with reputable training in psychometrics will consult before purchasing a psychometric test. So, if your test publisher tells you they don’t have such a document, it might be wise to stay away from them. On the other hand – do expect to pay for the manual. Some publishers will offer them free of charge but others will require a fee.
Based on the above, hopefully you can see that developing tests well takes time and effort. It is for this reason that good tests are usually not cheap! Not only that. The test is not static. People change, norms change and so validities may even change. The publisher cannot put the test on the shelf and forget as if it were a book they wrote years ago. This is why usually clients will pay a fee per test report or per candidate whenever they use the test. If you are attracted by free or very cheap internet-based tests you could be making a costly mistake. Particularly if you plan using the test in candidate selection for your business.
What is the history of psychometric testing?
Let’s give you a very brief background at this stage. Something that I find interesting being based in Asia and often working with Western-developed tests is the role Asia has in the development of psychometric tests for assessment at work! If it had not been for the Chinese Cultural Revolution, China may actually be far more advanced in this field than the West because China certainly was testing for individual differences before such testing was recorded in Europe or the USA. The Chinese government believed it important to test for entrants to the Civil Service more than 4000 years ago! However, at the time of the Cultural Revolution, this sort of assessment was outlawed as being too bourgeoisie.
This paved the way for the West to develop individual difference assessment and one of the first names to crop up in textbooks is French Psychologist Binet who around 1905 coined the term IQ. His application of testing was of course more related to education. The World Wars saw an increase in and marked use of psychological assessment for selection decisions. Here people were forcibly recruited into the army through conscription. The arduous task for decision-makers was where best to place these newcomers. The Army Alpha and Army Beta tests assisted in answering this question. Although successful for war-time placements, the same tests used in peacetime by commercial organisations gained a poor reputation as they worked less well (they were not designed for this application).
Nowadays tests are used extensively in organisations to assist in selection and development decisions as well as team-building, career guidance and performance appraisal. Reputable test publishers go out of their way to assess their tests and ensure they are free from bias and that they work to do the job they were designed to do. Tests are useful decision-making tools although they should never be used on their own. You’ll find out why later. One of the biggest problems we face in Asia is an influx of poorly designed tests along with non-psychologist distributors who know little about psychology and psychometrics. In this course you’ll learn more about this, how to spot good from bad and above all you’ll gain the confidence to make competent decisions about using the best psychometric test for your purpose.
THIS ARTICLE IS CLEARED FOR PUBLISHING ON PSYCHOLOGY1 GROUP SITES ONLY. IT REMAINS COPYRIGHT AND INTELLECTUAL PROPERTY OF PSYASIA INTERNATIONAL PTE. LTD. YOU ARE NOT AUTHORIZED TO PUBLISH IT ON ANY OTHER SITE. YOU ARE NOT PERMITTED TO COPY/PASTE THIS ARTICLE OR TO SAVE IT TO YOUR LOCAL DRIVE. YOU ARE ONLY PERMITTED TO READ IT ONLINE AT OUR WEBSITE. VIOLATION OF THESE TERMS WILL RESULT IN BANNING OF OFFENDING IPS AND LEGAL ACTION FOR THOSE WHO REPUBLISH THIS ARTICLE WHETHER IT BE WITH OR WITHOUT A REFERENCE TO THE ORIGINAL AUTHOR.
