In this session we will explore the following:
1. Computer-based scoring of psychometric tests
2. Hand-scoring of psychometric tests
3. Norming of test results
4. The link between scoring of tests and reliability
Converting raw scores to standardised scores and using representative norms will be covered in a later session.
Once a psychometric test has been properly administered, it needs to be scored. Depending on the test chosen, you may have a few options.
a. You can opt for computer-based scoring.
This would work if you had administered the test using computer software or if you had asked your candidate to complete an online test. For online tests, this option is good because it is less likely to involve scoring errors! Your candidate completes the test online and then the system immediately and automatically scores the test. There is no additional input required and hence less chance for error. This pre-supposes the publisher has used the correct scoring algorithms of course. Whilst most reputable test publishers will, we do know of one who had an error in a test battery that was not spotted until one of their distributors pointed out that his partner had done poorly on a test for which she was a subject matter expert!!
If you administer the test to your candidate using desktop software, you should be able to automatically score it in the same way as above.
b. You can opt for hand-scoring or a bureau service or keyed input followed by computer-scoring. You are most likely to use this option if you administered the test to your candidate using hard-copy test booklets and answer sheets.
Firstly, you’ll need to double-check the answer sheets to ensure that there are no irregularities. Ensure that it’s obvious which answer the respondent selected. Be careful with any “blobs” that may have appeared from ink or pencil smudges etc. If a respondent has changed their mind after selecting a response and has crossed it out, ensure that you only use the most recent response in scoring.
For hand-scoring using a scoring key, you’ll next need to align the scoring key with the answer sheet. The exact requirements will vary based on the test you are using, so ensure that you read and fully understand the instructions provided by the test publisher.
Once you have scored the responses, double-check your scoring. You then need to record the score. The score you calculate at this point is called the RAW SCORE. On its own, a raw score means nothing. If I tell you that you scored 54 on a numerical reasoning test or 75 on the extraversion scale of a personality assessment, you’ll need to ask me more questions before you truly understand your score. The most important question to ask would be how your score compared to others. The comparison of your score with others is called norming.
It is called norming because we compare a candidate’s score to a group of others (called the norm group) who completed the test in the past. To undertake this comparison, you can do it by way of a simple calculation or through the use of norm tables either developed by yourself or, more usually, supplied by the test publisher.
Norm tables allow us to use a standard vocabulary for expressing a candidate’s score in relation to others who have taken the test and it is for this reason that we call your new score a standardised score. A standardised score is simply your candidate’s raw score, compared with the norm group and expressed in terms of how the candidate scored in relation to others. We’ll consider standardised scores in more detail in a later lesson. You’ll see by now that your objective is to calculate the candidate’s standard score as this is the way to achieve maximum meaning. If you opt for paper and pencil tests and hand-scoring, the process can be lengthy. So are there other options?
We have already seen above that we can simply have the candidate complete an online test. However, you may not wish to do this if there are many candidates. This is because you will need as many computers as candidates if you are going to supervise them. If you are using an unsupervised test, the candidate can complete on their own PC, but you may be concerned about possible cheating and so on. This is why you may end up using paper and pencil tests (in a supervised environment). However, there is an alternative to arduous hand-scoring if you have used paper and pencil tests.
You can use the bureau service of your psychometric test distributor. You just need to check that the answer sheet is properly completed, clear and free from any irregularities and then send the answer sheet to the distributor by fax or scanned email. The bureau service will then score the test for you and send you a report.
Furthermore, you may have another option yet. If you have access to a computer or online test system, you can probably also enter the candidate’s responses to each question into the system and have the system produce the report. This is essentially what the bureau service above does for you. Doing it yourself should work out cheaper. Do be careful when you transpose the responses though – accuracy is far more important than speed unless you want to invalidate the whole process!!
Self-scoring answer sheets: Some psychometric tests are supplied with self-scoring answer sheets. These are much easier to use than non-self-scoring answer sheets. In this case you usually need to open up the answer sheet by tearing off some perforated card. Inside the answer sheet, the candidate’s responses will have been duplicated via carbon or similar onto a scoring card. Usually, you add up the number of responses (often black circles) that appear inside a circle. Those outside of a circle represent incorrect answers so don’t get counted. Once you’ve added up correct responses, you have your raw score. Slightly different procedures obviously apply for personality assessments and fewer personality assessments provide self-scoring answer sheets due to their scoring complexity. When using self-scoring answer sheets you need to be especially careful to ensure that the candidate presses hard on the answer sheet when completing the test. If they are light-handed their responses may not come through onto the scoring card!
Finally, let’s consider the link between psychometric test scoring and reliability/validity. As you know, the test administrator can have a huge impact upon psychometric test reliability throughout the whole process. At the scoring stage you can affect reliability simply by scoring incorrectly. This might happen because you miss the fact that a candidate crossed out their answer and changed their mind. It may also happen because you try to score fast and just don’t add up correctly. Perhaps you use the scoring key incorrectly or perhaps the scoring is so arduous (often the case for personality assessments) that you simply get lost in the scoring or incorrectly use your calculator!
Ensure therefore that you fully understand how to score the test, use the scoring key as per the publisher’s instructions, score slowly and double check or have someone else double check your scoring. If possible, use computer based scoring or self-scoring answer sheets. Incorrect scoring reduces reliability and of course that means that a valid test can become invalid and a waste of time or money!
Interested in learning more about psychometric testing for HRM? Keep reading – your next free session is not far away! To ensure you don’t miss a single instalment, we suggest you follow-us on twitter as each new post will be announced there. You may also like to join our face-to-face psychometric training courses in Singapore or Hong Kong – these range from simple introductory courses through to Certification Courses such as the BPS Level A and BPS Level B Certificates of Competence in Occupational Testing. Not in Singapore or Hong Kong? No problem – we also offer both recorded and live online training in psychometrics! For full details please see here or email us.
DO NOT COPY OR SAVE THIS ARTICLE TO YOUR COMPUTER.
THIS ARTICLE IS CLEARED FOR PUBLISHING ON PSYCHOLOGY1 GROUP SITES ONLY. IT REMAINS COPYRIGHT AND INTELLECTUAL PROPERTY OF PSYASIA INTERNATIONAL PTE. LTD. YOU ARE NOT AUTHORIZED TO PUBLISH IT ON ANY OTHER SITE. YOU ARE NOT PERMITTED TO COPY/PASTE THIS ARTICLE OR TO SAVE IT TO YOUR LOCAL DRIVE. YOU ARE ONLY PERMITTED TO READ IT ONLINE AT OUR WEBSITE. VIOLATION OF THESE TERMS WILL RESULT IN BANNING OF OFFENDING IPS AND LEGAL ACTION FOR THOSE WHO REPUBLISH THIS ARTICLE WHETHER IT BE WITH OR WITHOUT A REFERENCE TO THE ORIGINAL AUTHOR.
