 |
|
 |
| |
Archive for the ‘Validity of Psychometric Tests’ Category
Friday, January 15th, 2010
Types of Bias in Psychometric Test Translation
With the demand and need for psychological tests increasing in various different cultures and countries, there has been much greater awareness regarding some of the issues that are associated with the development or adaptation of tests to be used in contexts and situations that may be different from which the test was developed for. This article focuses on one of the key aspects of translating tests, the types of bias that can occur.
When utilizing the test in a new cultural group, it is not quite as simple as directly translating the test, administering it and then comparing the results for its validity. There are a number of issues that need to be considered such as whether the area assessed with the test applies to the new culture or whether is may be biased towards that group and whether what is assessed by the test also has similar behavioral indicators? These are just some of the potential areas where bias can be found in the translation of tests and affect the validity of the test being utilized in the new context.
Van der Vijer & Hambleton (1996) differentiates between three distinct types of bias that may affect the validity of tests that have been adapted for different cultural contexts and these are construct bias, method bias and item bias.
Construct bias occurs when the construct (e.g. personality) that is measured by the test displays significant differences between the original culture for which it was developed and the new culture where it is going to be utilized. These differences can occur in the way that the construct was formulated and developed as well as in the relevant behaviors that are associated with the construct. It is critical to examine whether the underlying theory of the test is subject to construct bias and this can be examined through the studies examining the construct and its associated behaviors in the context that it will be utilized in. If there are significant differences found in these studies, it may indicative that there is construct bias. Major revisions may be required to overcome this bias. If not, the validity of the test will be affected.
Method bias refers to factors or issues related to the administration of the test that may affect the validity of the test. Examples of areas that method bias can occur include social desirability, acquiescence response styles, the conditions in which the test was conducted and the motivation of the respondents. Across cultures, there potentially can be differences that can occur in these areas and these can affect the way that the respondents answer the items in the test. This potentially may lead to differences between found that can be erroneously attributed to cultural differences when in fact, these differences are the result of differences in the administration procedures. As a result, it is threat to the validity of tests that have been adapted for use in new cultures. Test developers also not only need to focus on the adaptation of the test itself but also need to be aware of issues regarding the implementation of the test in a new context.
Item bias is another source of bias that can occur in the translation of tests and these refer to biases that occur with the items in the test. This is usually the result of either poor translation choices for items or due to culturally inappropriate translations. For example, the phrase “kick the bucket” is essentially a phrase that referring to passing away in the Western context and is commonly known by most people in that culture; unfortunately, this phrase would have no meaning for people from cultures without any prior experience with that phrase. In this manner, a literal translation of that phrase would be a poor translation as it does not convey the correct meaning of the item. The items in the test need to be culturally equivalent, where the meaning of the items needs to be correctly translated so as to maintain the validity of the test in the new cultural context.
These are some of the biases that may occur during the translation of tests. Test developers will need to be aware of the sources of bias and take the appropriate measures to avoid these biases.
References:
Van der Vijer, F. and Hambleton, R. K. (1996). Translating tests: some practical guidelines. European Psychologist, 1, 89-99.
Psychometric Training in Singapore, Hong Kong, Malaysia, and China
If you are serious about using psychometric tests properly then we recommend joining PsyAsia International’s Psychometric Assessment at Work Course which leads to a certificate of competence in Occupational Testing Level A and Level B from the British Psychological Society. The Course is run publically in Singapore and Hong Kong or in-house anywhere.
More details about BPS Level A and B in Singapore and Hong Kong
Online Psychometric Training – Worldwide
Alternatively, you might be interested in introductory Online Psychometric Test Training presented live by a registered psychologist. PsyAsia is offering a special fee of just US$12 for anybody who registers for the February online psychometric training course!
More details about online psychometric test training
Tags: bps certificates of competence singapore, bps level a hong kong, bps level a singapore, bps level b hong kong, bps level b singapore, choosing psychometric tests, level a occupational testing singapore, level b occupational testing singapore, personality assessment, Personality Tests, Reliability of Psychometric Tests, using psychometric test results Posted in BPS Level A & B Certificates, Competence in Psychometric Testing, Error in Psychometric Tests, Personality Tests, Psychometric Test Training, Psychometric Tests, Reliability of Psychometric Tests, Validity of Psychometric Tests | No Comments »
Friday, November 20th, 2009
The Market for Psychometrics in Singapore
There are so many Psychometric Tests on the market in Singapore now, the task of choosing the right one is not easy. Choice is always a good thing, however as humans we often look for easy or stereotypical ways of making those choices and they are not always the best ones to make. For example, a client of ours was preparing for an upcoming team-building session. He approached us asking if we had a certain test that he could use in that session. Our answer was that we don’t supply that test for various very good reasons. The client’s response was “but so many people use it”. This is a typical response. Another potential client had been looking around in Singapore for Psychometric Personality Tests to use in his training sessions as an added benefit. He categorically advised us that he was not interested in validity and was looking for something simple and cheap! The reality here is that at best he is wasting his time and the time of those who will complete his tests. At worst and most likely, his trainees will be led to believe things about themselves which frankly may not be true (reliable or valid!).
Science, Psychology, Psychometrics and the Real World of Business
As busy professionals we often assume that if lots of other people are using a test it must be a good one. This is a huge mistake. Our evolution has programmed us to be seduced by glossy advertising materials and confident, friendly salespeople. On the other hand, we have a tendency to be turned off by less glossy scientific figures, statistics and perhaps psychologists such as myself who speak about the science and real value behind a test, its validity! Ultimately then, both our clients and ourselves as psychologists have problems to overcome!!
Psychologists have to be able to explain in more “glossy” terms about the technical properties of a test and our clients, usually the HR and aligned professions, are invited to turn their ears our way for a little while, just long enough to get the notion that there is more to a psychometric test than meets the eye!
Technical Properties of Psychometric Tests
When we talk of the technical properties of a psychometric test, we are referring to things such as its reliability and validity as well as how it was constructed. If a test is constructed well, it will take time. Not months, often years. The test will also evolve over time such that more and validity data will be added to its manuals. This process is costly, hence good tests cost money.
If you come across cheap tests, that should start to ring alarm bells. It’s possible to write a few questions on a napkin in a restaurant and call it psychometric and even try to sell it. If it looks good and the questions look relevant perhaps it will sell and gain a huge following. But how reliable is that test?
In other words, can it provide consistent measurement of your candidate? If your bathroom scales provide different results each time you weight yourself you take them back and say these are not reliable. Likewise with a test, you need to ensure that it is consistently assessing the constructs that it purports to assess. We often come across new clients who are shocked when we tell them that good personality tests often contain around 200 questions. However, buyer beware! We know that the longer the test, the more reliable the results (as long as it is not so long that the candidate falls asleep!).
An unreliable test can not be a valid test, hence reliability is a precursor to validity. However, validity is arguably the most important aspect of a test. You choose to use tests because you want them to illustrate where a candidate stands in terms of their ability or personality or in order to predict how your candidate will perform or behave in a job. The test’s ability to meet this need is referred to as validity.
Some tests on the market are simply more valid that others. In fact, one test in the past year has proven to be more valid than all other tests it was compared with on the market! How come users stay with their current test then? Perhaps because of preference, habit, price, mass-following and so on. However, do ask yourself and your test supplier, how valid is your test – this is the single most important technical property in a psychometric test!
Sometimes tests which are more valid will be more expensive but this makes sense. If a test took a long time to develop, was developed well and by a reputable publisher and is based on well founded theories that have been researched internationally, then surely it is worth paying the extra as such a test will provide an excellent return on investment with its strong validity.
Training to use Psychometric Tests in Singapore
Properly developed psychometric tests require proper training to be used competently. If your test supplier requires that you undergo very limited or no training, this is a reflection of the test as well as their lack of understanding of psychometrics. You need to understand the concepts referred to above, as well as error in testing and how to make decisions based on test results, let alone how to feed back results properly to candidates and decision-makers. The type of questions (i.e., forced choice versus rating scales) will also dictate how you can use the results – you need to be trained to understand this! In some parts of the world (South Africa for example), only psychologists can use psychometric tests. Whilst this is a strict rule, it has its logical basis in how easy it is for untrained professionals to use tests wrongly.
Purchasing Psychometric Tests in Singapore
You may also wish to consider where you purchase your tests from, particularly in Singapore. In recent years we have seen an influx of profiteers in the industry who seek to make money but lack any depth of understanding in psychometrics or psychology at work. This will change in time as psychology in Singapore develops. For now however, be wary of this and we suggest that you only purchase psychometric tests from fully registered organisational psychologists who have a firm grounding in personality, psychometrics and psychology at work and who are answerable to professional competence and ethics boards. Many of those selling psychometric tests in Singapore are simply not answerable to anybody in terms of their conduct or competence. You can therefore not be certain that any advice they provide is relevant, up-to-date or will work in your organisation.
There are many more things to be aware of when choosing psychometric tests in Singapore. We cannot entertain them all here due to space constraints. You may wish to look out for training courses in Psychometric Assessment such as our our Psychometric Assessment at Work training which leads to the internationally recognised British Psychological Society Level A and B Certificates of Competence in Occupational Testing. Such courses will prepare you further for choosing the right test and therein avoid costly selection and development mistakes. Look for courses run by experts in psychometrics who are based in Singapore and hence have a strong understanding of test use aligned with local culture, laws and practice.
Note: some Singapore firms will ship in overseas trainers to run psychometric training. We suggest you avoid this training reseller model given that the facilitator is based overseas and is thus likely to lack knowledge of the Singapore business/legal and cultural environment for Psychometric Testing.
This article is Copyright PsyAsia International Pte Ltd.
It was originally written for Human Resources Magazine in Singapore
A shorter version of the article appears in the magazine’s November 2009 issue
Tags: aptitude test distributor, bps level a singapore, bps level b singapore, choosing psychometric tests, level a occupational testing singapore, level b occupational testing singapore, personality test distributor, personality test training singapore, psychometric assessment singapore, psychometric test singapore, psychometric test training singapore, singapore psychometric tests Posted in BPS Level A & B Certificates, Competence in Psychometric Testing, Psychometric Test Training, Psychometric Tests, Reliability of Psychometric Tests, Validity of Psychometric Tests | Comments Off
Wednesday, November 18th, 2009
MEN are considered much more likely to cheat on internet selection tests than women, a Saville Consulting
survey reveals. The report, commissioned by work psychologists at Saville Consulting, questioned over a
thousand UK students. In this sample, 42% thought men were more likely to cheat than women, compared to just 9% who thought that females were more likely to cheat than males.
Internet selection tests measure a respondent’s abilities in certain areas to predict their likely level of performance at work. Tests typically measure verbal, numerical or logical skills. Many organisations use such tests as part of their recruitment process, but the vast majority of such tests can be completed by job applicants on their home computer, in an unsupervised manner.
Worryingly, 14% of students openly admitted to receiving help from others on such internet tests. Overall, respondents also revealed that they knew more men who had cheated on internet tests than women. When students who rated themselves as being of lower ability were compared to those who reported being of a higher level of ability, 58% more students of lower ability admitted to receiving help on internet tests.
Previous research commissioned by Saville Consulting indicated that personality and intelligence tests were actually seen by people as being more useful selection methods than job interviews. Eight out of ten students in the current survey confirmed that many people exaggerate on job application forms. Moreover, the same proportion of respondents agreed that if an organisation followed unsupervised tests with longer supervised tests prior to appointment, this was an effective method for preventing cheats from being selected.
These findings undermine the use of unsupervised tests without the re-testing of applicants
under supervised conditions. Organisations may not re-test applicants because they believe this
offers a more economical recruitment process. However, recruiting dishonest and less intelligent
workers will inevitably be counterproductive for an organisation’s effectiveness.
Professor Peter Saville, International Chairman of Saville Consulting, said:
“Following up applicants who complete online unsupervised tests with supervised tests of equal or greater rigour is arguably the most effective way to prevent candidate cheating. Other methods, such as randomising the order of questions or adapting questions to the ability of a candidate simply do not get around the problem of unsupervised cheating. If job applicants know that they are going to be retested in a properly supervised environment, this can do much to reduce the problem of cheating. The supervised test at the later stage also provides a safeguard that the successfully appointed applicant has the abilities to match the requirements of the role.”
Tags: cheating in aptitude tests, cheating in psychometric tests, online psychometric test, Validity of Psychometric Tests Posted in Online Psychometric Test Systems, Psychometric Test Research, Psychometric Tests, Saville Consulting Wave, Validity of Psychometric Tests | Comments Off
Wednesday, October 28th, 2009
For psychometric assessments to have utility and be effective when assessing people for various purposes, the assessment has to be reliable and valid for the situation.
All personality tests are not 100% accurate and measurement errors from a variety of sources can affect the results. The length (i.e. the number of items) of the assessment affects the reliabilty of the assessment and research has demonstrated that measurement errors are smaller in longer assessments than in shorter assessments. In addition, a larger number of items better represents the abstract characteristics that are being assessed. For example, when assessing personality, one cannot expect to obtain an accurate picture of an individual through a few questions, therefore more items are needed. It has to be noted that after a limit, increasing the number of items will not provide further increases to reliability as other factors such as fatigue will set in.
It is for this reason that good personality assessments will have a large number of items and therefore require some time for the candidates to complete the assessment (usually between 200-250 questions, taking around 30-40 minutes). Psychometric assessments that are shorter will tend to be less reliable and valid. With a large number of items, the reliability of the test will be better and in turn the validity of the assessment will be better too. Validity is all about predicting performance. So with high validity human resource professionals get a higher return on their investment.
Tags: choosing psychometric tests, hong kong psychometric tests, Personality Tests, psychometric assessment blog, psychometric personality tests, psychometric research, psychometric tests and error, singapore psychometric tests, standard error measurement Posted in Competence in Psychometric Testing, Personality Tests, Psychometric Tests, Reliability of Psychometric Tests, Validity of Psychometric Tests | Comments Off
Monday, August 3rd, 2009

Saville Consulting Wave® Professional Styles measures motives, talents, preferred culture and competency potential in one dynamic online questionnaire. It is designed for use in both selection and development, transforming the assessment process with the sophisticated use of internet technology and powerful models of effectiveness in the workplace. This comprehensive global questionnaire measures the 4 clusters, 12 sections, 36 dimensions and 108 facets of the Wave hierarchical model. Average completion time is 40 minutes.
In a 2008 co-validation study, Wave Professional Styles, outperformed the OPQ32i, NEO-PI-R, Hogan Personality Inventory 16PF5 and 15FQ+ in predicting both work competencies and overall global measures of performance. Additionally the Wave outperformed tests that should not really be used in recruitment and selection, such as the MBTI and DiSC.

For users, this higher validity enables:
- better predictions of performance
- a stronger return on investment
- accurate identification and management of talent
- highly targeted coaching
- the development of a performance culture throughout an organization
For details of the scientific validation study, please click here.
We are pleased to offer a free trial of the Saville Wave Focus Styles Questionnaire to the major HR decision-maker in Asian organisations. The trial is open until 15 August and to the first 100 applicants. Please send your full details to our Psychometrics Team by clicking here. Please provide the following:
Your full name:
Your corporate email address (free email accounts such as yahoo, gmail etc. not accepted):
Your job title:
Your company name:
Your location:
Kindly note that requests for trials from non-decision makers, those not based in Asia, those who provide inaccurate, fake date or non-corporate email addresses or those whose organisations have previously trialled the Wave will not be entertained. Thank you for your understanding.
For details on professional training to use the Saville Consulting Wave, please click on the links below:
Psychometric Assessment at Work Course – BPS Level A and B – Asia
Saville Consulting Wave Conversion Course – Asia
Tags: choosing psychometric tests, personality test distributor, psychometric assessment singapore, psychometric news, psychometric test singapore, Saville Consulting Wave, saville wave, singapore psychometric tests, wave personality test Posted in Psychometric Tests, Saville Consulting Wave, Validity of Psychometric Tests | Comments Off
|
|
|
 |
|
 |
|