 |
|
 |
| |
Archive for the ‘Psychometric Tests’ Category
Monday, November 30th, 2009
To complement the BPS Level A and BPS Level B Certificates of Competence in Occupational Testing Training in Singapore, we have announced new dates for the same psychometric training course in Hong Kong. PsyAsia has been running this course in Singapore and Hong Kong since 2002 and we have collected many impressive delegate reviews. The course is run by a British Psychological Society award-winning resident and registered business psychologist with a PhD earned through work with validation of Psychometric Tests in Asia – there is nobody more qualified in Asia to run this course! For more details of the course and facilitator and to register online, please click here.
The Hong Kong Level A and B Psychometric Training Course will run over the following dates:
Level A: 9-11 March 2010
Level B: 16-18 March 2010
Psychometric Test Administration only: 9 March 2010
The Singapore Level A and B Psychometric Training Course will run over the following dates:
Level A: 27-29 January 2010 or 24-26 February 2010
Level B: 1-3 March 2010
Psychometric Test Administration only: 27 January 2010 or 24 February 2010
Delegates may opt to join Level A in Singapore and Level B in Hong Kong if they wish at no additional cost but should contact us before booking in order to ensure availability.
The day immediately following Level B is conversion training for the Saville Consulting Wave®. Level B delegates may register for this training at 50% off the regular fee by selecting that option during the booking process.
Tags: bps certificates of competence hong kong, bps level a hong kong, bps level a singapore, bps level b hong kong, bps level b singapore, hong kong psychometric tests, psychometric course hong kong, psychometric course singapore, psychometric test singapore, psychometric test training hong kong, psychometric test training singapore, Saville Consulting Wave, saville wave
Posted in Psychometric Tests | Comments Off
Friday, November 20th, 2009
The Market for Psychometrics in Singapore
There are so many Psychometric Tests on the market in Singapore now, the task of choosing the right one is not easy. Choice is always a good thing, however as humans we often look for easy or stereotypical ways of making those choices and they are not always the best ones to make. For example, a client of ours was preparing for an upcoming team-building session. He approached us asking if we had a certain test that he could use in that session. Our answer was that we don’t supply that test for various very good reasons. The client’s response was “but so many people use it”. This is a typical response. Another potential client had been looking around in Singapore for Psychometric Personality Tests to use in his training sessions as an added benefit. He categorically advised us that he was not interested in validity and was looking for something simple and cheap! The reality here is that at best he is wasting his time and the time of those who will complete his tests. At worst and most likely, his trainees will be led to believe things about themselves which frankly may not be true (reliable or valid!).
Science, Psychology, Psychometrics and the Real World of Business
As busy professionals we often assume that if lots of other people are using a test it must be a good one. This is a huge mistake. Our evolution has programmed us to be seduced by glossy advertising materials and confident, friendly salespeople. On the other hand, we have a tendency to be turned off by less glossy scientific figures, statistics and perhaps psychologists such as myself who speak about the science and real value behind a test, its validity! Ultimately then, both our clients and ourselves as psychologists have problems to overcome!!
Psychologists have to be able to explain in more “glossy” terms about the technical properties of a test and our clients, usually the HR and aligned professions, are invited to turn their ears our way for a little while, just long enough to get the notion that there is more to a psychometric test than meets the eye!
Technical Properties of Psychometric Tests
When we talk of the technical properties of a psychometric test, we are referring to things such as its reliability and validity as well as how it was constructed. If a test is constructed well, it will take time. Not months, often years. The test will also evolve over time such that more and validity data will be added to its manuals. This process is costly, hence good tests cost money.
If you come across cheap tests, that should start to ring alarm bells. It’s possible to write a few questions on a napkin in a restaurant and call it psychometric and even try to sell it. If it looks good and the questions look relevant perhaps it will sell and gain a huge following. But how reliable is that test?
In other words, can it provide consistent measurement of your candidate? If your bathroom scales provide different results each time you weight yourself you take them back and say these are not reliable. Likewise with a test, you need to ensure that it is consistently assessing the constructs that it purports to assess. We often come across new clients who are shocked when we tell them that good personality tests often contain around 200 questions. However, buyer beware! We know that the longer the test, the more reliable the results (as long as it is not so long that the candidate falls asleep!).
An unreliable test can not be a valid test, hence reliability is a precursor to validity. However, validity is arguably the most important aspect of a test. You choose to use tests because you want them to illustrate where a candidate stands in terms of their ability or personality or in order to predict how your candidate will perform or behave in a job. The test’s ability to meet this need is referred to as validity.
Some tests on the market are simply more valid that others. In fact, one test in the past year has proven to be more valid than all other tests it was compared with on the market! How come users stay with their current test then? Perhaps because of preference, habit, price, mass-following and so on. However, do ask yourself and your test supplier, how valid is your test – this is the single most important technical property in a psychometric test!
Sometimes tests which are more valid will be more expensive but this makes sense. If a test took a long time to develop, was developed well and by a reputable publisher and is based on well founded theories that have been researched internationally, then surely it is worth paying the extra as such a test will provide an excellent return on investment with its strong validity.
Training to use Psychometric Tests in Singapore
Properly developed psychometric tests require proper training to be used competently. If your test supplier requires that you undergo very limited or no training, this is a reflection of the test as well as their lack of understanding of psychometrics. You need to understand the concepts referred to above, as well as error in testing and how to make decisions based on test results, let alone how to feed back results properly to candidates and decision-makers. The type of questions (i.e., forced choice versus rating scales) will also dictate how you can use the results – you need to be trained to understand this! In some parts of the world (South Africa for example), only psychologists can use psychometric tests. Whilst this is a strict rule, it has its logical basis in how easy it is for untrained professionals to use tests wrongly.
Purchasing Psychometric Tests in Singapore
You may also wish to consider where you purchase your tests from, particularly in Singapore. In recent years we have seen an influx of profiteers in the industry who seek to make money but lack any depth of understanding in psychometrics or psychology at work. This will change in time as psychology in Singapore develops. For now however, be wary of this and we suggest that you only purchase psychometric tests from fully registered organisational psychologists who have a firm grounding in personality, psychometrics and psychology at work and who are answerable to professional competence and ethics boards. Many of those selling psychometric tests in Singapore are simply not answerable to anybody in terms of their conduct or competence. You can therefore not be certain that any advice they provide is relevant, up-to-date or will work in your organisation.
There are many more things to be aware of when choosing psychometric tests in Singapore. We cannot entertain them all here due to space constraints. You may wish to look out for training courses in Psychometric Assessment such as our our Psychometric Assessment at Work training which leads to the internationally recognised British Psychological Society Level A and B Certificates of Competence in Occupational Testing. Such courses will prepare you further for choosing the right test and therein avoid costly selection and development mistakes. Look for courses run by experts in psychometrics who are based in Singapore and hence have a strong understanding of test use aligned with local culture, laws and practice.
Note: some Singapore firms will ship in overseas trainers to run psychometric training. We suggest you avoid this training reseller model given that the facilitator is based overseas and is thus likely to lack knowledge of the Singapore business/legal and cultural environment for Psychometric Testing.
This article is Copyright PsyAsia International Pte Ltd.
It was originally written for Human Resources Magazine in Singapore
A shorter version of the article appears in the magazine’s November 2009 issue
Tags: aptitude test distributor, bps level a singapore, bps level b singapore, choosing psychometric tests, level a occupational testing singapore, level b occupational testing singapore, personality test distributor, personality test training singapore, psychometric assessment singapore, psychometric test singapore, psychometric test training singapore, singapore psychometric tests
Posted in BPS Level A & B Certificates, Competence in Psychometric Testing, Psychometric Test Training, Psychometric Tests, Reliability of Psychometric Tests, Validity of Psychometric Tests | Comments Off
Friday, November 20th, 2009
Welcome to our first session in Psychometric Testing!
In this first session we will explore the following:
1. What is a psychometric test?
2. A brief background of psychometric testing.
What is a psychometric test? How are they developed?
Let’s start out by telling you what a psychometric test is not!
You may have come across various different online tests. Not all of them are psychometric. Whether they are or not will depend on how they were designed, for what purpose and what they intend to measure. For example, you may have come across the Thematic Apperception Test or the Rorshach Inkblot Test. These tests are used mainly in clinical settings. The client is asked to look at scenes or pictures and to articulate what they see. Such tests are supposed to be able to assess the unconscious mind. However, interpretation is not as objective as we would like to see in occupational testing and assessment. Furthermore, what the client “sees” may be based on external factors such as culture, upbringing or season of the year.
Psychometric Tests are different! They are defined as quantitative (numerical) assessments of one or more psychological (in the head) attributes. So, psychometric tests are used to assess in a quantitative way things like numerical reasoning skills, verbal aptitude, extroversion, conscientiousness and so on. That alone does not make a test psychometric of course. In order to be psychometric the test must have been designed to be:
1. Administered in a standardised manner
2. Scored in a standardised manner
3. Interpreted in a standardised manner
4. Constructed according to psychometric principles
You’ll see later how important standardisation is throughout the use of psychometric tools. In fact standardisation is a critical element of all scientific HR processes.
As for being constructed according the psychometric principles, what we mean here is that the test must have gone through each aspect of a very scientific development process. It is not good enough to simply write down a few questions, produce a flashy report and start selling a new test! This is the process it must go through:
a. Development of a rationale behind the test that is supported by research. For example – if I decide to develop a new test of extroversion, the scientific research literature and models must inform the design of my questions and the aspects of extroversion that I attempt to assess.
b. Next I need to write some experimental questions. At this stage I’ll write more questions than I intend to have in the final version of my test because I am prepared to throw some out based on feedback.
c. Now I’ll find a sample of people (who represent the group I am designing the test for) and I’ll ask this sample to complete my experimental test. In other words, I pilot my test.
d. Next I head back to my office and assess the responses and how they relate to each other statistically. At this stage I am running item analysis to test that similar questions (items) are indeed related to each other in the way I would expect. More on this later. The point to grasp now is that this is an iterative process. Things won’t be perfect the first time around. Based on the statistics I’ll need to remove some questions, refine others and then go back to point C. I’ll do this again and again until I am happy with the statistics I get at point D!
e. Now that my test questions are performing well I need to enter a standardisation phase. Here, a larger group of people will complete my test and that will show me where people tend to score on the test. This group will become my benchmark or norm group later and add meaning to the scores of future test-takers.
f. The next question is “Does my new test actually assess what it is supposed to assess and/or does it predict something meaningful?”. As an example, you would expect my numerical reasoning test predict success of accountant trainees. This stage is called Validation, we are assessing the validity of the test or whether it is fit for purpose.
g. Now, we all know from science classes at school that all good experiments end with a write-up! That’s exactly what we do at the final stage of psychometric test development. We need to write up all of the above stages in a long document which is called the test’s technical manual. It is this manual that prospective clients with reputable training in psychometrics will consult before purchasing a psychometric test. So, if your test publisher tells you they don’t have such a document, it might be wise to stay away from them. On the other hand – do expect to pay for the manual. Some publishers will offer them free of charge but others will require a fee.
Based on the above, hopefully you can see that developing tests well takes time and effort. It is for this reason that good tests are usually not cheap! Not only that. The test is not static. People change, norms change and so validities may even change. The publisher cannot put the test on the shelf and forget as if it were a book they wrote years ago. This is why usually clients will pay a fee per test report or per candidate whenever they use the test. If you are attracted by free or very cheap internet-based tests you could be making a costly mistake. Particularly if you plan using the test in candidate selection for your business.
What is the history of psychometric testing?
Let’s give you a very brief background at this stage. Something that I find interesting being based in Asia and often working with Western-developed tests is the role Asia has in the development of psychometric tests for assessment at work! If it had not been for the Chinese Cultural Revolution, China may actually be far more advanced in this field than the West because China certainly was testing for individual differences before such testing was recorded in Europe or the USA. The Chinese government believed it important to test for entrants to the Civil Service more than 4000 years ago! However, at the time of the Cultural Revolution, this sort of assessment was outlawed as being too bourgeoisie.
This paved the way for the West to develop individual difference assessment and one of the first names to crop up in textbooks is French Psychologist Binet who around 1905 coined the term IQ. His application of testing was of course more related to education. The World Wars saw an increase in and marked use of psychological assessment for selection decisions. Here people were forcibly recruited into the army through conscription. The arduous task for decision-makers was where best to place these newcomers. The Army Alpha and Army Beta tests assisted in answering this question. Although successful for war-time placements, the same tests used in peacetime by commercial organisations gained a poor reputation as they worked less well (they were not designed for this application).
Nowadays tests are used extensively in organisations to assist in selection and development decisions as well as team-building, career guidance and performance appraisal. Reputable test publishers go out of their way to assess their tests and ensure they are free from bias and that they work to do the job they were designed to do. Tests are useful decision-making tools although they should never be used on their own. You’ll find out why later. One of the biggest problems we face in Asia is an influx of poorly designed tests along with non-psychologist distributors who know little about psychology and psychometrics. In this course you’ll learn more about this, how to spot good from bad and above all you’ll gain the confidence to make competent decisions about using the best psychometric test for your purpose.
THIS ARTICLE IS CLEARED FOR PUBLISHING ON PSYCHOLOGY1 GROUP SITES ONLY. IT REMAINS COPYRIGHT AND INTELLECTUAL PROPERTY OF PSYASIA INTERNATIONAL PTE. LTD. YOU ARE NOT AUTHORIZED TO PUBLISH IT ON ANY OTHER SITE. YOU ARE NOT PERMITTED TO COPY/PASTE THIS ARTICLE OR TO SAVE IT TO YOUR LOCAL DRIVE. YOU ARE ONLY PERMITTED TO READ IT ONLINE AT OUR WEBSITE. VIOLATION OF THESE TERMS WILL RESULT IN BANNING OF OFFENDING IPS AND LEGAL ACTION FOR THOSE WHO REPUBLISH THIS ARTICLE WHETHER IT BE WITH OR WITHOUT A REFERENCE TO THE ORIGINAL AUTHOR.
Tags: online psychometric course, online psychometric test training, psychometric test singapore, psychometric testing hong kong, psychometric training
Posted in BPS Level A & B Certificates, Competence in Psychometric Testing, Online Psychometric Training Mini-Course, Psychometric Tests, Psychometric Tests in HRM | Comments Off
Wednesday, November 18th, 2009
MEN are considered much more likely to cheat on internet selection tests than women, a Saville Consulting
survey reveals. The report, commissioned by work psychologists at Saville Consulting, questioned over a
thousand UK students. In this sample, 42% thought men were more likely to cheat than women, compared to just 9% who thought that females were more likely to cheat than males.
Internet selection tests measure a respondent’s abilities in certain areas to predict their likely level of performance at work. Tests typically measure verbal, numerical or logical skills. Many organisations use such tests as part of their recruitment process, but the vast majority of such tests can be completed by job applicants on their home computer, in an unsupervised manner.
Worryingly, 14% of students openly admitted to receiving help from others on such internet tests. Overall, respondents also revealed that they knew more men who had cheated on internet tests than women. When students who rated themselves as being of lower ability were compared to those who reported being of a higher level of ability, 58% more students of lower ability admitted to receiving help on internet tests.
Previous research commissioned by Saville Consulting indicated that personality and intelligence tests were actually seen by people as being more useful selection methods than job interviews. Eight out of ten students in the current survey confirmed that many people exaggerate on job application forms. Moreover, the same proportion of respondents agreed that if an organisation followed unsupervised tests with longer supervised tests prior to appointment, this was an effective method for preventing cheats from being selected.
These findings undermine the use of unsupervised tests without the re-testing of applicants
under supervised conditions. Organisations may not re-test applicants because they believe this
offers a more economical recruitment process. However, recruiting dishonest and less intelligent
workers will inevitably be counterproductive for an organisation’s effectiveness.
Professor Peter Saville, International Chairman of Saville Consulting, said:
“Following up applicants who complete online unsupervised tests with supervised tests of equal or greater rigour is arguably the most effective way to prevent candidate cheating. Other methods, such as randomising the order of questions or adapting questions to the ability of a candidate simply do not get around the problem of unsupervised cheating. If job applicants know that they are going to be retested in a properly supervised environment, this can do much to reduce the problem of cheating. The supervised test at the later stage also provides a safeguard that the successfully appointed applicant has the abilities to match the requirements of the role.”
Tags: cheating in aptitude tests, cheating in psychometric tests, online psychometric test, Validity of Psychometric Tests
Posted in Online Psychometric Test Systems, Psychometric Test Research, Psychometric Tests, Saville Consulting Wave, Validity of Psychometric Tests | Comments Off
Thursday, November 12th, 2009
There has been much research conducted examining the relationship between Conscientiousness as defined as one of the dimensions in the “Big Five” Model of personality and job performance. Much of the research has indicated that Conscientiousness is a positive predictor for job performance across a range of job positions. Simply put and what many advocate, is that it appears that people who have been assessed to be higher on Conscientiousness tend to perform better at work. Nevertheless, an article by Tett (1998) highlights some considerations regarding accepting this conclusion and cautions against accepting this at face value.
In this article, a number of examples suggest that Conscientiousness and job performance may not be irrevocably linked as previous research has suggested. It provides two examples of work situations where Conscientiousness may actually hinder the job performance. Firstly, the author highlights situations where being conscientious may result in lower productivity where either tasks require more time to complete or where fewer tasks can be completed. This is illustrated by the example of managers who at times are required to make decisions even when they do not have all the relevant information (Tett, 1998). In such a situation, being high on conscientious may affect the speed at which decisions can be made.
The second example refers to rules and procedures; conscientious individuals tend to adhere to rules and procedures. Tett (1998) highlights that in certain professions, strictly following rules and procedures may affect creativity and innovation. It is proposed that such adherence to rules and procedures can affect productivity as such individuals would be unlikely to develop novel ideas and solutions.
Although Conscientiousness appears to remain as a significant predictor of job performance, it is important to note that there are situations where other factors can play a part.
Tett, R. P. (1998). Is Conscientiousness ALWAYS positively related to job performance? The Industrial-Organizational Psychologist, 36(1)
Tags: personality questionnaire, psychometric assessment singapore, psychometric news, psychometric personality tests, psychometric research, psychometric test singapore
Posted in Personality Tests, Psychometric Test Research, Psychometric Tests | Comments Off
Wednesday, October 28th, 2009
For psychometric assessments to have utility and be effective when assessing people for various purposes, the assessment has to be reliable and valid for the situation.
All personality tests are not 100% accurate and measurement errors from a variety of sources can affect the results. The length (i.e. the number of items) of the assessment affects the reliabilty of the assessment and research has demonstrated that measurement errors are smaller in longer assessments than in shorter assessments. In addition, a larger number of items better represents the abstract characteristics that are being assessed. For example, when assessing personality, one cannot expect to obtain an accurate picture of an individual through a few questions, therefore more items are needed. It has to be noted that after a limit, increasing the number of items will not provide further increases to reliability as other factors such as fatigue will set in.
It is for this reason that good personality assessments will have a large number of items and therefore require some time for the candidates to complete the assessment (usually between 200-250 questions, taking around 30-40 minutes). Psychometric assessments that are shorter will tend to be less reliable and valid. With a large number of items, the reliability of the test will be better and in turn the validity of the assessment will be better too. Validity is all about predicting performance. So with high validity human resource professionals get a higher return on their investment.
Tags: choosing psychometric tests, hong kong psychometric tests, Personality Tests, psychometric assessment blog, psychometric personality tests, psychometric research, psychometric tests and error, singapore psychometric tests, standard error measurement
Posted in Competence in Psychometric Testing, Personality Tests, Psychometric Tests, Reliability of Psychometric Tests, Validity of Psychometric Tests | Comments Off
Wednesday, October 21st, 2009
PsyAsia International is pleased to announce a special offer from Saville Consulting. For clients interested in attending our Saville Consulting Wave® Training in Singapore or Hong Kong and who are able to send 2 or more delegates from their organisation, Saville Consulting Asia Pacific are offering a complimentary Saville Oasys Online Psychometric Assessment System.
Oasys is a cutting-edge management system for the administration, scoring, reporting and interpretation of the world’s most revolutionary psychometric assessments. These include the Saville Consulting Wave®, the Saville Consulting Wave Focus, the newly released and revolutionary Saville Personality Profile, a full range of modern and innovative aptitude tests*, a job profiler for use prior to selection and a 360 performance appraisal. Use of the Oasys system cuts down on the costs of psychometric testing in organisations as clients have full control over their candidates and pay a system fee per report rather than paying us a higher bureau service fee. Oasys usually sells for SG$6000 or HK$30,000 so this is a very special offer. It is limited and will expire after the following training course dates and the system will then sell at the regular fee.
To avail of the offer, please register at least 2 people from the same organisation for one of the following courses at http://www.psyasia.com/register
SINGAPORE COURSES
Wave Conversion: 25 November 2009
(for those who already hold a qualification in a substantive personality test or BPS Level B)
Wave Full Training: 25-26 November 2009
(for those without a qualification in a substantive personality test)*
HONG KONG COURSES
Wave Conversion: 1 December 2009
(for those who already hold a qualification in a substantive personality test or BPS Level B)
Wave Full Training: 1-2 December 2009
(for those without a qualification in a substantive personality test)*
IN HOUSE COURSES
PsyAsia International is also accredited to run in-house courses for the Saville Wave. The same offer will apply. By engaging PsyAsia to run an in-house course for you before December 2009, we will be able to supply one Saville Oasys System on a complimentary basis. Please contact us for further details on our in-house training for Saville Wave.
More details on the courses
More details on Saville Consulting Wave
*for those without training to BPS Level A or an equivalent, the aptitude tests will be disabled in the system until such time as the client undergoes such training. PsyAsia offers BPS Level A Certification Training in Singapore and Hong Kong.
Tags: online psychometric systems, online psychometric testing, psychometric test training hong kong, psychometric test training singapore, save consulting wave singapore, saville consulting oasys, Saville Consulting Wave, saville oasys, saville wave, saville wave test, saville wave training, saville wave training hong kong, saville wave training singapore, waves test
Posted in Online Psychometric Test Systems, Personality Tests, Psychometric Test Training, Psychometric Tests, Saville Consulting Wave | Comments Off
Friday, October 16th, 2009
PsyAsia International, Asia’s leading independent provider of psychometric assessment products and training today announced a free psychometric test training workshop in three prime Asian locations in December. The workshops will cover the basics in psychometric assessment, such as reliability, validity and error, along with pointers and discussion on how to choose the right psychometric test. Courses will be facilitated by PsyAsia’s organisational psychologists who have at least an MSc in Organisational Psychology along with the British Psychological Society Level A and B Certificates of Competence in Occupational Testing. PsyAsia’s psychometric training courses in Asia have been the subject of many years of positive reviews (many published at the website). This course will bring best practice and an introduction to competence to those who would otherwise not realise the issues involved in testing due to the costs of regular training courses.
PsyAsia will charge registrants a small fee to reflect the company’s costs in running the training session. The fee is just US$70 (SG$99/HK$550). This covers a full buffet lunch in a top hotel along with day-long refreshments, handouts and a certificate of attendance. However, PsyAsia will even refund this fee back to any participant who registers for one of the company’s psychometric accreditation courses within 6 months!
The dates are:
Hong Kong: 3 December 2009
Kuala Lumpur: 8 December 2009
Singapore: 9 December 2009
More information
Online Registration
Tags: choosing psychometric tests, psychometric assessment singapore, psychometric course hong kong, psychometric course singapore, psychometric news, psychometric test training hong kong, psychometric test training malaysia, psychometric test training singapore
Posted in Psychometric Tests | No Comments »
Friday, October 16th, 2009
Personality assessment can be divided into two categories, type and trait based personality assessments. Across both types of personality assessments, it is assumed that personality remains stable over time, involves a genetic basis and influences individuals to demonstrate similar behavior in most situations.
Trait based personality assessments assess various aspects of an individual’s personality which contributes to them behaving in particular ways. Across the population, different people’s aspects of their personality will tend to vary and this explains the wide variety of personality descriptions. Such assessments have tended to be developed to fulfill a certain need to explain personality in various contexts such as work. Although these assessments may be more difficult for people to understand and can be difficult to use in team building activities, they are more psychometrically sound and allow for more accurate comparisons between individuals. These assessments can be used in conjunction with other methods in activities that require differentiation between individuals such as in recruitment and selection.
Type based personality assessments assess an individual’s personality by categorizing them in particularly distinct theory-driven types. One way to understand this is that these types are collections of particular personality traits that influence them to behave in particular ways in certain situations. A number of personality theories have proposed classifications of certain personality types that explain a wide range of human behavior. These assessments are attractive as they are easy to understand and possesses utility in team building activities, but they are less psychometrically sound and have the tendency to “pigeon-hole” individuals into particular types. This results in less accurate comparisons between individuals. As such, they should not be used in activities that require accurate differentiation between individuals. Nevertheless, these assessments are particularly useful in team building activities.
Tags: personality assessment, personality questionnaire, Personality Tests, personality trait, personality type, psychometric personality tests
Posted in Personality Tests, Psychometric Tests | No Comments »
Monday, August 24th, 2009
The first thing to remember is that if you are using a purely ipsative personality test then you should not be comparing test results between candidates. Ipsative tests are self-referencing – they are comprised of force-choice items. They are useful in coaching, team-building and career guidance, but should not be used alone in recruitment and selection scenarios.
Some tests on the market, such as the Saville Consulting Wave or the Apollo Profile are joint normative-ipsative tests and these would be fine to be used to compare between candidates. A normative test is one which allows the candidate to respond based on the strength of their agreement or disagreement with a statement. The end results are then compared with a group of similar others who have previously taken the test (the norm group).
Purely normative tests such as the Identity Self-Perception Questionnaire would also be good to use for comparing candidates. Aptitude tests are by their nature normative tests and hence can be used to compare between candidates.
So, let’s assume that we have administered a normative personality assessment to two candidates and we are particularly interested in finding a candidate with a high tendency towards creative thinking. We have decided to use a personality assessment alongside other means of assessment including an abstract reasoning test to assess this. We ask Lee and Jane to complete both of these tests. These are their scores on the test scale of interest (presented in sten scores):
Lee
Creative thinking:8
Jane
Creative thinking:6
Now, keeping in mind that we would never use test results on their own to make a decision, let’s look at how most decision-makers would approach the above scenario based on test results alone for simplicity.
It obviously appears that Lee is somewhat better suited to the position than Jane.
However, in psychometric testing just as in any assessment procedure undertaken for Human Resources, there is always a chance of error. In fact, it’s more than chance! We know that error is always present.
When interviewing somebody the error is present, when running an assessment center the error is also present. Likewise, error is also present in the use of psychometric tests. Given a desire to be scientific, reputable test publishers will actually assess their tests for error.
One way of doing this is to ask a group of respondents to complete the test today and to invite them back a month later to complete the same test. Ignoring practice effects (which are controlled for), the expectation is that there should be a strong relationship between how a candidate scored at time one and how they score at time two. The idea is that test results should remain consistent over time. Psychometricians refer to this as test-retest reliability.
We hope for high test-retest reliability and we really should be choosing tests which have proven high levels. If we don’t we will have little confidence in test results and be very limited in terms of how we use them.
The assessment for error that shows us how much confidence we can have in test scores is referred to as the standard error of measurement (SEM). It uses an equation to ascertain how confident we can be that a candidate’s test result is a reflection of their true score as opposed to their true score PLUS error.
The equation is very simple, it is just: Standard Deviation multiplied by the square root of 1 minus the test-retest reliability of the assessment. If you don’t like statistics, sorry – they really are necessary to use tests competently!
If you choose a reputable test, often the publisher will quote the SEM in the test manual. If not, you can use the equation above to calculate it. You would use the standard deviation for your scale of interest taken from the manual alongside the test-retest from the manual (note…if your publisher fails to provide these figures you should probably not be using their tests!!).
The point is that the lower the SEM (or the higher the test-retest reliability), the better. Why?
Going back to Lee and Jane above. If our test has an SEM of 1.5 STENS, this would mean that we are 68% confident that Lee’s true score for the creative thinking scale is between 6.5 and 9.5 (we add and subtract the SEM from the observed score). It would also mean that we are 68% certain that Jane’s true score lies between 4.5 and 7.5 on the same test.
Now we can see that some doubt begins to arise as to whether the differences observed between the two candidates is as a result of a real score difference or an error difference (i.e., the true score for both candidates could be 7!). We don’t want to make a mistake and choose the wrong candidate, so let’s now look at how we can compare the differences.
We can take this further and calculate something called the standard error of difference. This tells us how confident we can be that there is a true difference between the scores of the two candidates. Because both candidates completed the same test, we use the following equation: SEdiff= the square root of (1.414 * SEM squared of the test in question).
Let’s say that our test has an SEM of 1.5 STENS. Using the SEdiff equation, we get a figure of 3.18 for the SEdiff. This represents our “critical figure”. It means that the difference between the candidate’s scores must be at least 3.18 before we can conclude there is a true score difference.
In our example, the difference between the candidate’s scores is only 2. Hence we cannot conclude there is a true score difference. The implication for selection is that we should not (everything else being equal) select one candidate over the other because, although we observe differences, the differences may not be true differences, they may be simply error differences.
Note that if we choose a more reliable test it will reduce the SEM. So for example, if we have an SEM of 1 STEN, our SEdiff for the above example would be 1.19. In this case, since the difference between the candidate’s scores is 2 STENS, we could conclude that there is a true difference. We would be at least 68% certain and almost 96% certain. We won’t go into degrees of certainty in this article, but the point is made!
In summary, do not compare candidate’s test results without a knowledge of the test’s reliability and standard deviation or in other words, do not ignore the SEM. Every assessment technique has an error variable. Competent users of psychometric tests will be aware of this and ensure they do not make the wrong selection decision or give incorrect development/careers advice on the basis on error rather than true score differences.
This article is (C) 2009 PsyAsia International. Some websites have been given permission to post this article. The article must always contain our copyright, publisher details and a live link to our website. Please do not violate these terms.
Tags: choosing psychometric tests, psychometric tests and error, Reliability of Psychometric Tests, standard error measurement, standard error of difference, using psychometric test results
Posted in Error in Psychometric Tests, Psychometric Test Training, Psychometric Tests, Reliability of Psychometric Tests | Comments Off
|
|
|
 |
|
 |
|
