| Psychometrika – Journal of Psychometrics Society Free Trial Access until July 31, 2010! |
| From now until July 31, 2010, please enjoy free access to the journal. Click here to begin your reading. |
Archive for the ‘Competence in Psychometric Testing’ Category
Psychometric Journal – Free Trial Access Until July 31
Monday, June 28th, 2010
Online Psychometric Testing Mini-Course: Session 2
Friday, June 25th, 2010In this session we will explore the following:
1. The different categories of psychometric tests
Different categories of psychometric tests
There are two major categories of psychometric tests:
Tests of MAXIMUM PERFORMANCE
and
Tests of TYPICAL BEHAVIOUR
Maximum Performance Tests
Tests of Maximum performance include tests that have right and wrong answers. This means aptitude tests, ability tests and attainment tests. It’s possible to break down the category of maximum performance tests further by distinguishing between Speed Tests and Power Tests. A speed test is more common in occupational testing. It is a test that has a time limit. The idea is that most candidates will not complete all questions by the end of the allotted time. Candidates will trade off speed with accuracy. It is quite conceivable however that candidates would do somewhat better if they were allowed more time. On the contrary, power tests are not timed. The idea here is that you either know the subject matter or you don’t and that even if you were given more time your score would be unlikely to improve. As an example, I know nothing about quantum physics. If you sit me in a room and ask me to sit a test on this topic I won’t do well – I’ll do badly and I’ll do badly irrespective of how long you give me to complete the test!
So, what is the difference between the main types of maximum performance tests?
Aptitude Tests
These assessments are future looking. They ask if the candidate has the aptitude to do something in the future, perhaps after a period of education or experience. If I want to know if my graduate respondent will make a good leader I may ask him to sit a number of aptitude tests. I am quite certain that the recent graduate will not be able to be a leader right now. But that is not what I am asking. I need to know if after a few years of on-the-job experience and leadership development training this person has the aptitude to be a successful leader. An example of an aptitude test would be Saville Consulting’s Swift Analysis Aptitude Test or their Customer Aptitudes Test.
Ability Tests
In the aptitude example above I noted that I know my recent graduate will not make a good leader now. Ability tests are about the “here and now”. They tell us what the candidate is able to do now. Ability tests appear at the top of the hierarchy in maximum performance assessment. An example of an ability test would be the General Ability Test (GAT) or the Graduate and Managerial Assessment (GMAT).
Attainment Tests
These are the tests that you are probably more familiar with. The tests we sit during our schooling and university years are attainment tests. The question is: following this period of learning, how much does the person now know? So, GCSE tests, A’Levels, University Degrees, Pilot Licence Tests, Piano Tests and so on are all examples of attainment tests.
However, just as you are beginning to think this isn’t too difficult to understand, let me add a thought for you! It is possible for a single test to actually be any of the above 3 categories depending on how you intend to use it! For example, if you have just completed a training course for apprentice mechanics and then give them a mechanical aptitude test in order to assess their learning, you are actually using it in a similar way to an attainment test. If you didn’t train these apprentices and you wish to see if they have the aptitude to be good mechanics and you have them sit this test, you are using it as an aptitude test. If you are about to select a mechanic who needs to perform right now without further training you could use this same test as an ability test!
Typical Behaviour Tests
These are not really tests in the purest sense because with typical behaviour (also called typical performance) there is no right or wrong. In view of this we try to speak about questionnaires and assessments for this category instead of tests! This category includes personality assessments and interest questionnaires.
Personality Assessments
These obviously assess personality. Some personality assessments assess a few types of personality and these are called PERSONALITY TYPE assessments (for example, the Myers Briggs Type Indicator – MBTI or the Eysenck Personality Questionnaire – EPQ). Other personality assessments assess traits. A type is a collection of traits. A trait is a relatively enduring characteristic of the person (such as extraversion or resilience) which can be used to predict behaviours. Trait tools are able to assess at a finer level and this is one reason why type tools should not be used in assessment for selection decisions despite being useful guides in development and team-building. Trait tools can be further split into 2 more categories – NARROWBAND and BROADBAND. Broadband Personality Questionnaires assess traits broadly – this means there are overall fewer traits to assess – perhaps 5 or 6 in total. An example of a broadband personality tool is the Minnesota Multiphasic Personality Inventory (MMPI). Narrowband tools assess more traits/contain more scales. They typically assess each scale with fewer questions than broadband instruments. Some narrowband assessments may have around 16 scales (e.g., the 16 Personality Factor Questionnaire – 16PF) whilst others may have around double that, for example the Identity Personality Questionnaire and the Apollo Profile assess 36 and 34 scales respectively. A benefit of narrowband tools is their ability to assess at a very specific level within the person. This enables a more valid prediction of workplace performance. However, broadband personality tools have their own advantage which is that because they contain more questions per scale/trait – reliability for each scale/trait is generally higher than with narrowband tools. We’ll discuss reliability and validity in detail later in the course.
Note: Some personality assessments were not developed specifically for workplace assessment but are nonetheless used in workplace selection and development (e.g., 16PF, MMPI) whilst other questionnaires were developed specifically to be used in workplace selection and development (e.g., Apollo, OPQ32, Hogan Personality Inventory)
Interest Questionnaires
As the name implies, these tools assess the respondent’s interest in various aspects of the working world in a structured and meaningful way and are therefore typically used in career counselling. They are also used in selection assessment to ascertain whether the candidate will comfortably fit the particular organisation they are applying to work in. For example, if I am interested in doing structured work in a well structured organisation I may not be very happy working for an organisation which values spontaneity and creativity. Some personality assessments also report on the respondent’s interest by way of “derived scales”. Derived scales come about by way of predicting statistically what the respondent would score in particular areas (such as interest) based on how they responded to the general personality assessment. An example of an Interest Questionnaire is the Strong Interest Inventory. An example of a personality questionnaire that uses derived interest scales is the Identity Self-perception Questionnaire.
Interested in learning more about psychometric testing for HRM? Keep reading – your next free session is not far away! To ensure you don’t miss a single instalment, we suggest you follow-us on twitter as each new post will be announced there. You may also like to join our face-to-face psychometric training courses in Singapore or Hong Kong – these range from simple introductory courses through to Certification Courses such as the BPS Level A and BPS Level B Certificates of Competence in Occupational Testing. Not in Singapore or Hong Kong? No problem – we also offer both recorded and live online training in psychometrics! For full details please see here or email us.
DO NOT COPY OR SAVE THIS ARTICLE TO YOUR COMPUTER.
THIS ARTICLE IS CLEARED FOR PUBLISHING ON PSYCHOLOGY1 GROUP SITES ONLY. IT REMAINS COPYRIGHT AND INTELLECTUAL PROPERTY OF PSYASIA INTERNATIONAL PTE. LTD. YOU ARE NOT AUTHORIZED TO PUBLISH IT ON ANY OTHER SITE. YOU ARE NOT PERMITTED TO COPY/PASTE THIS ARTICLE OR TO SAVE IT TO YOUR LOCAL DRIVE. YOU ARE ONLY PERMITTED TO READ IT ONLINE AT OUR WEBSITE. VIOLATION OF THESE TERMS WILL RESULT IN BANNING OF OFFENDING IPS AND LEGAL ACTION FOR THOSE WHO REPUBLISH THIS ARTICLE WHETHER IT BE WITH OR WITHOUT A REFERENCE TO THE ORIGINAL AUTHOR.
BPS Level A & B Certificates of Competence in Psychometric Testing Singapore – October Course Dates
Friday, June 18th, 201020 October 2010 – 27 October 2010 (weekend excluded) (5-day option also available)
Conrad Centennial Hotel, Singapore
**Please register early – previous courses have been fully booked up to 8 weeks before the start date**
Introduction
Train for BPS Level A and B Certificates of Competence in Occupational Testing in Singapore with the company that won an international quality award in 2010; the company which brought regular BPS Certification training to Singapore; and the company which uses a British Psychological Society Award-Winning Dr. of Psychology to facilitate the course!
This intensive professional course provides training in psychometric assessment in the workplace, leading to the fulfilment of requirements for the British Psychological Society Certificates of Competence in Occupational Testing – BPS Levels A & B. After successful completion of the course, delegates are qualified to order, administer, score, interpret and feedback on a complete range of ability, aptitude and personality questionnaires. They will also be able to apply for the British Psychological Society’s Certificates which means the holder will be recognised internationally as having trained to the highest levels in the practical application of psychometric tests.
PsyAsia International is renowned by Human Resource professionals and organisational psychologists throughout Asia for being the leader in psychometric training in the region. Whilst many overuse the term “leader”, PsyAsia demonstrates their position with tangibles. Recent innovations include moving of training to online mediums, blended learning for the BPS Level A and B and an online learning centre where delegates get immediate access to forums, chat with other delegates, virtual receptions with the facilitator and other course members, downloads of recorded training and more.
Dates:
20-22 October 2010: Level A
25-27 October 2010: Level B
Delegates may also opt to attend online training over 3 evenings instead of attending on 20 October, reducing total time away from the office to 5 days. Furthermore, delegates may choose to undertake Level A in October and place Level B on hold until Q1 of 2011 for no additional fee.
Key Learning Objectives for Level A
Delegates will be able to:
• Register with all UK and many international test publishers to purchase tests and know what to look for and avoid in Singapore. With many poorly developed tests and non-psychologist distributors in Singapore this is crucial!
• Understand the range and types of psychometric tests available.
• Know when to use tests, and importantly – when not to.
• Decide which tests are appropriate for a particular situation, be it selection, development, coaching or team-building.
• Conduct a job analysis and identify the assessment requirements.
• Have an appreciation of other assessment methods such as assessment centres or behavioural interviews.
• Administer, score and interpret aptitude tests.
• Make informed decisions using psychometric test results.
• Provide quality feedback to clients and candidates.
• Integrate tests into a wider recruitment or assessment process.
• Have a sound grasp of statistical concepts used in testing.
• Appreciate the importance of test reliability and validity and be able to assess it.
Key Learning Objectives for Level B
Delegates will be able to:
• Increase their awareness and understanding of individual differences between people.
• Integrate the use of personality assessments into selection and development activities.
• Decide which questionnaires are appropriate for particular situations.
• Administer, score and interpret personality data.
• Provide professional selection feedback to both candidates and recruiting customers.
• Coach individuals to develop using personality information.
• Use personality questionnaire information to support structured interviewing.
• Critically assess the value of different tools for specific purposes.
• Identify and reject inflated claims of validity – so-called ‘Barnum’ or ‘horoscope’ effects.
• Relate personality factors to organisations’ core competencies.
• Design and write selection reports for recruiting clients / managers.
• Design and write development reports to aid self-awareness of clients / managers.
• Maintain appropriate records and data for audit, research and feedback purposes.
• Demonstrate professional Best Practice and compliance with employment legislation.
See the course webpage for a full day-by-day syllabus.
No Exams!
At the current time it is up to individual course providers to decide how to assess their delegates; some may use exams whilst others use workbooks and other assessments. At the current time, there is no exam requirement for our Level A and B training in Singapore; we find that our delegates prefer this. Instead we assess via a workbook and practical exercises.
Facilitator
This course is facilitated by PsyAsia International’s resident director, Dr. Graham Tyler. Dr. Tyler is well known in Asia for his expertise and work in psychometrics. His PhD was by way of a double-scholarship award from the Australian Government and focussed on psychometric assessment, personality and performance at work in Asia. Dr. Tyler is a fully Registered Organisational Psychologist, a Chartered Scientist, a consultant/trainer to numerous Singapore Government departments and Fortune 500 companies and a university lecturer. He is a published author in Singapore-based publications (Straits Times, Human Capital Plus, Human Resources) and peer-reviewed journals such as the International Journal of Selection and Assessment, the BPS Selection and Development Review and The Occupational Psychologist. He is a reviewer for scientific journals and conferences and a supervisor for Intern Psychologists on behalf of Australian and local Universities. Dr. Tyler was the first to bring regular training leading to the British Psychological Society’s Certificates of Competence in Occupational Testing to Singapore in 2002 having introduced them to Dubai prior to that. His work with Psychometric Tests has received recognition from the British Psychological Society with an award for Scientific Contribution to Psychology. Plentiful delegate reviews of Dr. Tyler’s training can be seen at the course webpage.
Additional Benefits
The course facilitator and our psychologist consultants are based locally, this means that we can blend the BPS syllabus with knowledge and experience of local laws, culture and practices and that even when the course ends expert psychologists will be nearby to offer ongoing support. Given that we are not a training reseller, we don’t need to ship in overseas facilitators. As a result, we can be flexible in terms of our course offerings. Delegates can choose which parts of the course to sit and when – no need to attend all 6 days at once! You may attend Level A in October and then Level B in 2011. You can even start out by attending our 2-day Identity Accreditation Course or study our Online Test Administration Course and have those count towards your Level A/B! Finally, because we are local there is no need to send your follow-up work overseas for marking. You can essentially finish Level B as soon as you are ready – and don’t hang around for your results – we guarantee to mark your work within 10 days, although most is marked within 2-3 days of receipt – you can check our online performance review for recent dates!
Furthermore, because we are not a test publisher, our training is unbiased and we introduce many tests (both good and bad) to delegates. In addition to this, for those delegates who wish to study and be accredited in numerous tests, we offer 50% discount off our Saville Consulting Wave® one-day training which follows Level B (60% if you book before 30 July). Of course, we include Identity Questionnaire and Apollo Profile accreditation in the A/B fee as standard for all delegates. That’s not all! In addition to the SG$1200+ of materials we give you on the course (which include an online test system with starter credits), all delegates will receive a 30% discount on any Identity Personality Assessment products purchased within 1 month of their qualification and a 10% discount on all other non-discounted products which we offer. There’s even more, but we’ll save some surprises for the course!
More details
The Psychometric Assessment at Work Course in Singapore
http://www.psyasia.com/psychometric_test_training_course.php
Other Psychometric Training Courses in Singapore from PsyAsia International
http://www.psyasia.com/psychometric_training.php
Please follow the links at the web pages to register online. For any queries not addressed at the website, kindly call us on 6329 9629 or email us via http://www.psyasia.com/email
Online Psychometric Test Mini-Course: Lesson 1
Monday, June 14th, 2010Welcome to our first session in Psychometric Testing!
In this first session we will explore the following:
1. What is a psychometric test?
2. A brief background of psychometric testing.
What is a psychometric test? How are they developed?
Let’s start out by telling you what a psychometric test is not!
You may have come across various different online tests. Not all of them are psychometric. Whether they are or not will depend on how they were designed, for what purpose and what they intend to measure. For example, you may have come across the Thematic Apperception Test or the Rorshach Inkblot Test. These tests are used mainly in clinical settings. The client is asked to look at scenes or pictures and to articulate what they see. Such tests are supposed to be able to assess the unconscious mind. However, interpretation is not as objective as we would like to see in occupational testing and assessment. Furthermore, what the client “sees” may be based on external factors such as culture, upbringing or season of the year.
Psychometric Tests are different! They are defined as quantitative (numerical) assessments of one or more psychological (in the head) attributes. So, psychometric tests are used to assess in a quantitative way things like numerical reasoning skills, verbal aptitude, extroversion, conscientiousness and so on. That alone does not make a test psychometric of course. In order to be psychometric the test must have been designed to be:
1. Administered in a standardised manner
2. Scored in a standardised manner
3. Interpreted in a standardised manner
4. Constructed according to psychometric principles
You’ll see later how important standardisation is throughout the use of psychometric tools. In fact standardisation is a critical element of all scientific HR processes.
As for being constructed according the psychometric principles, what we mean here is that the test must have gone through each aspect of a very scientific development process. It is not good enough to simply write down a few questions, produce a flashy report and start selling a new test! This is the process it must go through:
a. Development of a rationale behind the test that is supported by research. For example – if I decide to develop a new test of extroversion, the scientific research literature and models must inform the design of my questions and the aspects of extroversion that I attempt to assess.
b. Next I need to write some experimental questions. At this stage I’ll write more questions than I intend to have in the final version of my test because I am prepared to throw some out based on feedback.
c. Now I’ll find a sample of people (who represent the group I am designing the test for) and I’ll ask this sample to complete my experimental test. In other words, I pilot my test.
d. Next I head back to my office and assess the responses and how they relate to each other statistically. At this stage I am running item analysis to test that similar questions (items) are indeed related to each other in the way I would expect. More on this later. The point to grasp now is that this is an iterative process. Things won’t be perfect the first time around. Based on the statistics I’ll need to remove some questions, refine others and then go back to point C. I’ll do this again and again until I am happy with the statistics I get at point D!
e. Now that my test questions are performing well I need to enter a standardisation phase. Here, a larger group of people will complete my test and that will show me where people tend to score on the test. This group will become my benchmark or norm group later and add meaning to the scores of future test-takers.
f. The next question is “Does my new test actually assess what it is supposed to assess and/or does it predict something meaningful?”. As an example, you would expect my numerical reasoning test to predict success of accountant trainees. This stage is called Validation, we are assessing the validity of the test or whether it is fit for purpose.
g. Now, we all know from science classes at school that all good experiments end with a write-up! That’s exactly what we do at the final stage of psychometric test development. We need to write up all of the above stages in a long document which is called the test’s technical manual. It is this manual that prospective clients with reputable training in psychometrics will consult before purchasing a psychometric test. So, if your test publisher tells you they don’t have such a document, it might be wise to stay away from them. On the other hand – do expect to pay for the manual. Some publishers will offer them free of charge but others will require a fee.
Based on the above, hopefully you can see that developing tests well takes time and effort. It is for this reason that good tests are usually not cheap! Not only that. The test is not static. People change, norms change and so validities may even change. The publisher cannot put the test on the shelf and forget as if it were a book they wrote years ago. This is why usually clients will pay a fee per test report or per candidate whenever they use the test. If you are attracted by free or very cheap internet-based tests you could be making a costly mistake. Particularly if you plan using the test in candidate selection for your business.
What is the history of psychometric testing?
Let’s give you a very brief background at this stage. Something that I find interesting being based in Asia and often working with Western-developed tests is the role Asia has in the development of psychometric tests for assessment at work! If it had not been for the Chinese Cultural Revolution, China may actually be far more advanced in this field than the West because China certainly was testing for individual differences before such testing was recorded in Europe or the USA. The Chinese government believed it important to test for entrants to the Civil Service more than 4000 years ago! However, at the time of the Cultural Revolution, this sort of assessment was outlawed as being too bourgeoisie.
This paved the way for the West to develop individual difference assessment and one of the first names to crop up in textbooks is French Psychologist Binet who around 1905 coined the term IQ. His application of testing was of course more related to education. The World Wars saw an increase in and marked use of psychological assessment for selection decisions. Here people were forcibly recruited into the army through conscription. The arduous task for decision-makers was where best to place these newcomers. The Army Alpha and Army Beta tests assisted in answering this question. Although successful for war-time placements, the same tests used in peacetime by commercial organisations gained a poor reputation as they worked less well (they were not designed for this application).
Nowadays tests are used extensively in organisations to assist in selection and development decisions as well as team-building, career guidance and performance appraisal. Reputable test publishers go out of their way to assess their tests and ensure they are free from bias and that they work to do the job they were designed to do. Tests are useful decision-making tools although they should never be used on their own. You’ll find out why later. One of the biggest problems we face in Asia is an influx of poorly designed tests along with non-psychologist distributors who know little about psychology and psychometrics. In this course you’ll learn more about this, how to spot good from bad and above all you’ll gain the confidence to make competent decisions about using the best psychometric test for your purpose.
Interested in learning more about psychometric testing for HRM? Keep reading – your next free session is not far away! To ensure you don’t miss a single instalment, we suggest you follow-us on twitter as each new post will be announced there. You may also like to join our face-to-face psychometric training courses in Singapore or Hong Kong – these range from simple introductory courses through to Certification Courses such as the BPS Level A and BPS Level B Certificates of Competence in Occupational Testing. Not in Singapore or Hong Kong? No problem – we also offer both recorded and live online training in psychometrics! For full details please see here or email us.
DO NOT COPY OR SAVE THIS ARTICLE TO YOUR COMPUTER.
THIS ARTICLE IS CLEARED FOR PUBLISHING ON PSYCHOLOGY1 GROUP SITES ONLY. IT REMAINS COPYRIGHT AND INTELLECTUAL PROPERTY OF PSYASIA INTERNATIONAL PTE. LTD. YOU ARE NOT AUTHORIZED TO PUBLISH IT ON ANY OTHER SITE. YOU ARE NOT PERMITTED TO COPY/PASTE THIS ARTICLE OR TO SAVE IT TO YOUR LOCAL DRIVE. YOU ARE ONLY PERMITTED TO READ IT ONLINE AT OUR WEBSITE. VIOLATION OF THESE TERMS WILL RESULT IN BANNING OF OFFENDING IPS AND LEGAL ACTION FOR THOSE WHO REPUBLISH THIS ARTICLE WHETHER IT BE WITH OR WITHOUT A REFERENCE TO THE ORIGINAL AUTHOR.
Online Psychometric Test Training – Free 10 Week Online Mini-Course Starts Soon
Wednesday, June 2nd, 2010PsyAsia International is pleased to announce a free, 10-week mini-course in psychometric testing. The course is mainly text based and introduce the basics of psychometrics in 10 weekly articles. You’ll learn about what psychometrics is, where it came from, how it is useful, discrimination, reliability, validity, error and more. PsyAsia is Asia’s leading independent provider of psychometric tests and psychometric training. The course however will be invaluable to readers anywhere. It will be interesting for those in HRM or consulting roles using or considering psychometric tests as well as for those about to undergo testing.
To ensure you don’t miss a single instalment, we suggest you follow-us on twitter as each new post will be announced there.
Interested in learning more about psychometric testing for HRM? Keep reading – your first free session is only a week away! You may also like to join our face-to-face psychometric training courses in Singapore or Hong Kong - these range from simple introductory courses through to Certification Courses such as the BPS Level A and BPS Level B Certificates of Competence in Occupational Testing. Not in Singapore or Hong Kong? No problem – we also offer both recorded and live online training in psychometrics! For full details please see here or email us.
Free Training in Psychometric Tests in Hong Kong & Singapore
Friday, March 26th, 2010
PsychometricAssessment.com / PsyAsia International offer Free Psychometric Testing Course in Hong Kong & Singapore
Introduction to Psychometric Testing Course: Hong Kong, 4 May 2010; Singapore, 11 May 2010
PsyAsia International is Asia’s independent Leader in Psychometric Test products and Training. We choose to distribute only the world’s best, most validated psychometric assessments and offer locally relevant, world-class training in psychometrics. The Introduction to Psychometrics Workshop expands on PsyAsia’s expertise in Psychometric Training in Asia by offering a course geared to those with very little experience or understanding in Psychometrics. Many first time clients don’t understand why they need to be careful in their choice or use of psychometrics and many do not understand why training is a necessity in competent test use.
This one-day course aims to provide experienced-based training in an accessible and economical way. The course is easy to understand and yet covers many of the important issues to be aware of when choosing and using psychometric tests. Given our passion for Asia and our passion for the competent use of psychometric tests in Asia, PsyAsia makes no profit on this course. We charge delegates a small fee that reflects the cost of the hotel venue (including buffet lunch and refreshments) where the training is held as well as materials that we provide to the delegates. What’s more, if you later decide to attend one of our accreditation courses in Psychometrics, we will issue you with a discount code that reduces your course fee by the amount you paid for this course!
|
|||
The history of psychometric testingTo view full course details and to register, please click here.
Free Psychometric Test Webinars by PsyAsia to showcase test and training products
Saturday, March 20th, 2010PsyAsia International is pleased to announce that until the end of March we will be offering free daily webinars to showcase our product range. Their will be no set agenda. The agenda will be set by attendees. Please note however that product knowledge may differ depending on which of our consultants is running the webinar. Come along and chat with our consultants, see the Saville Consulting Wave, Identity Personality Assessment and the Apollo Profile in action. Ask questions about training and consulting options and so forth!
For times and to register, please click here…
BPS Level A & B Psychometric Training in Singapore – New Dates – May and June
Friday, February 19th, 2010PsyAsia International has announced new dates for our Psychometric Assessment at Work Course in Singapore.
(Interested in attending this course in Hong Kong? Check the next post!)
PsyAsia has been running this course in Singapore and Hong Kong since 2002 and has collected many impressive delegate reviews. We were the first company to bring regular public BPS Level A and B training to Singapore and today we retain the only resident facilitators for this training!
The course is run by a British Psychological Society award-winning resident and registered business psychologist with a PhD earned through work with validation of Psychometric Tests in Asia. For more details of the course and facilitator and to register online, please click here.
The Singapore Psychometric Assessment at Work course will run over the following dates:
BPS Level A:
12-14 May 2010 or 9-11 June 2010
BPS Level B:
14-16 June 2010
Psychometric Test Administration only:
12 May or 9 June / or Live Online Training 30 March-01 April 5pm-6.20pm each day.
The dates are arranged as above to enable our delegates to choose whether to complete the whole course at once (9-11 June and 14-16 June) or to complete Level A first, then have a break before moving on to Level B (12-14 May and then 14-16 June). In addition, delegates may choose to only attend 5 days of the course and instead to attend our online Test Administration Training (see above), removing the need for attendance on 12 May or 9 June. Delegates will still need to be prepared to be assessed on Test Admin and Test Scoring once they attend! Please contact us before registering if you wish to choose this option.
Note: our March 2010 course in Singapore was fully booked (and in fact oversubscribed with a waiting list) 3 weeks before the course. Please ensure therefore that you register early. Early registration has additional benefits which include full access to this course’s area of our innovative online learning center where you can download training videos/courses, interact with other delegates, join in the forum and online chat and benefit from optional online live training with your facilitator even before the face-to-face training commences!
The day immediately following Level B is conversion training for the Saville Consulting Wave®. Level B delegates may register for this training at 50% off the regular fee by selecting that option during the booking process. In addition, anybody who registers for our BPS Level A/B Training before March 18 can join any other PsyAsia course at a 30% discount off the regular fee – please contact us for a special discount code if you wish to use this offer!
BPS Level B Certificate Training in Singapore – 1 place remaining for March
Wednesday, February 10th, 2010We would like to advise that there is now just one place remaining for the March 01-03 BPS Level B (psychometric assessment at work) training in Singapore. If you are the final delegate considering registering for the course, please do so by the end of this week! Thank you.
Links:
Identity Personality Test Research in Hong Kong
Wednesday, February 10th, 2010Identity Questionnaire Research Results – A synopsis
No reproduction without permission.
Introduction to the Study and Outline of the Phases
In September 2008, Quest Partnership Ltd, PsyAsia International, and the Hong Kong Institute of Vocational Education (HKIVE) embarked on a project to translate the Identity Self-Perception Questionnaire from English into traditional Chinese. The reason for translating the questionnaire was to produce an occupationally focused personality questionnaire that could be used in China and Hong Kong SAR. At the same time, Quest were also producing a new Careers Report for the Identity system. This enabled the volunteer students to gain useful feedback on their questionnaire. The project was headed up by Max Choi of Quest Partnership Ltd and Dr. Graham Tyler of PsyAsia International. Max Choi is an Occupational Psychologist with BPS chartered status and has substantial experience in designing and validating tests. Graham Tyler is a registered psychologist and has a PhD based on psychometric assessment and validating tools for predicting performance at work in Asia.
The research was split into several stages:
• Translations – involving the translation and back-translation of Identity into Simplified and Traditional Chinese by professional staff at HKIVE.
• Pilot Study – using the translated Identity questionnaire.
• Phase 1 Testing – a sample of participants at HKIVE completed the Chinese Identity questionnaire.
• Phase 2 Re-testing – participants were asked to complete the questionnaire for a second time one month later i.e. re-testing to determine the reliability of the questionnaire items.
• Data Cleansing – first to identify and remove ‘rogue’ answer sheets from students who did not complete the questionnaire seriously.
• Data Analysis & Results– analysis of the data and understanding the results.
• Producing Norms and Building this into the New Career Focus Report – norms were produced based on these Hong Kong students. This norm group was used for the new Career Focus Report which is now available for the Hong Kong education sector.
• Translation into Simplified Chinese – the project to translate the Identity Questionnaire into simplified Chinese and have it available online was completed in December 2009.
Translations
In September 2008, the questionnaire was translated into both Traditional and Simplified Chinese by 4 individuals at the HKIVE who hold the British Psychological Society’s Level A and B Certificates of Competence in Occupational Testing. This process was supervised by Dr. Graham Tyler, who has a good understanding of principles behind item construction. The translated questionnaire was sent to the test publishers (Quest Partnership Ltd) in the UK for evaluation and further refinement, working with Chinese natives now resident in the UK.
The translated questionnaire was then back-translated into English by lecturers in the English language department at the HKIVE. Independent back-translation provides the quality check of how effective the translation has been. The back-translation was checked against the original version of the questionnaire to ensure it retained its overall theme and meaning. A few items achieved poor back-translations and these were reviewed and improved and back-translated again to check that the translation had improved. The traditional Chinese translation took precedence on the basis that this would be evaluated first and then simplified Chinese would follow at a later date.
Pilot Sample
20 students at VTC completed the translated traditional Chinese questionnaire. They also completed a form which collected their feedback on items that they did not fully understand or where they felt the wording could be improved. This feedback was analysed and a few minor improvements were made for the next phase.
Phase 1 Testing
In October 2008, a large sample of 800+ Chinese students at HKIVE completed the Traditional Chinese Identity Questionnaire. Most of these administrations were conducted under standardised test administration conditions during classes. The final sample after data cleansing consisted of 421 students.
Phase 2 Testing
One month later many of the Chinese students from the Phase 1 testing were invited to complete the questionnaire again. The test-retest study is based on 206 students who completed the questionnaire again. Most of these administrations were conducted under standardised test administration conditions during classes.
Students were entered into a monetary prize draw as an incentive to take part in the research. Also, students received a Career Focus Report from their completed questionnaire.
Data Cleansing
Identifying ‘Rogue’ Responses
We placed stringent requirements on the data that could be used. It was evident that a proportion of the student responses were not usable. This may be as a consequence of asking the students to complete the questionnaire as part of class work. So although they were volunteers, the request during class time may have resulted in some slightly ‘reluctant’ volunteers. Also, others may have become bored after starting the questionnaire and may not have taken the whole questionnaire seriously, unlike real candidates applying for jobs. So a small minority will complete the questionnaire in a non-serious manner. Only a few rogue answer sheets can be visually identified (e.g. students who have put in the same response for the whole column or making neat zig-zag patterns on the answer sheet). So we needed to employ more sophisticated techniques to identify other ‘rogue’ respondents in order to remove these from our sample before conducting further analysis on the data.
Removing Answer Sheets with Too Many ‘3’ Responses
The instructions for completing the questionnaire clearly states that 3 should be used sparingly. But for this Chinese student sample, the mean number of ‘3’s chosen was 30.4, with a Standard Deviation (SD) of 32. For our UK sample however, the mean number of ‘3’s chosen was 9.85, with a SD of 15. It was decided that participants who responded with over 71 unsure ‘3’ responses would be removed from the sample i.e. this means that they are putting down ‘3’ to over a third of their questionnaire items – which is much too high. A caveat to this however is that given the “middle-way” philosophy in the East, it can generally be anticipated that central tendency responding will be higher in China than in the West.
Removing Answer Sheets with Random Responses
We employed two established methods to detect answer sheets which were being completed randomly i.e. the True Response Inconsistency (TRIN) and the Variable Response Inconsistency (VRIN) methods. Both methods are based on paired items which are highly associated in that knowing an individual’s response to one item will provide a very high level of prediction of their response to the other item. Therefore, when a person scores below a certain threshold with many paired items, we can be confident that their responses to the questionnaire have been random.
Data Analysis and Results
Test Re-Test Reliability
At Phase 2, students completed the Identity Questionnaire again about one month after the Phase 1 original completion of the questionnaire; we were able to conduct a Test-Retest analysis. This allows us to look into the stability or reliability of the questionnaire over time.
The final sample size for the test-retest was 206 after all the data cleansing procedures were conducted. Overall the vast majority of Identity scales were reliable. A small number of scales were below the benchmark of .70. However we need to be reminded that we are dealing with a translated questionnaire so we would expect some loss of reliability compared to the original questionnaire. So the original English Identity questionnaire sets the upper limits.
The original English Identity single scale test-retest coefficients ranged from .77 to .92 (based on a test-retest sample of 121). For the translated traditional Chinese questionnaire the test-retest coefficients ranged from .58 to .87. Seven of the 36 Identity questionnaire scales reported less than ideal test-retest coefficients:
• Consultative .57
• Psychological .61
• Empathy .57
• Adaptability .60
• Theoretical .62
• Rational .59
• Reflective .58
Interestingly, it might be argued that these scales are less meaningful to this student sample and different results are likely to be obtained in a business sample.
Internal Consistency Reliability
Another method to determine reliability is to look at internal consistency of each scale to see how well items within a scale correspond with one another. From this analysis we identified nine scales at a lower range of reliability coefficients than our ideal of 0.7:
• Social Presence .60
• Direct .61
• Empathy .58
• Adaptability .60
• Decisive .67
• Self Potency .53
• Self Protecting .62
• Social Desirability .63
• Reflective .43
Combining the two methods of establishing reliability it was useful to see if there were any scales that would have both low test-retest and low internal consistency reliability. The following 2 scales had lower reliabilities than ideal:
• Empathy
• Adaptability
We will be collecting more data so with more extensive use of the tool with participants who will be completing the questionnaire for non-research purposes we do expect the reliabilities to improve.
Study Results: Comparisons with UK Data
The results for this group of Hong Kong students were compared against the UK working population and also against a group of UK A Level applicants and Final Year Students for a Design & Technology course at a UK university.
The group of Hong Kong students compared to the other groups tended to be slightly lower on the following scales:
• Independent
• Critical
• Multi-Tasking
• Variety Seeking
• Determined
• Self Potency
• Positive
However, it is not possible to determine exactly why these differences are found as there are a range of variables as to how the groups differ from each other e.g. motivational aspects as the students were volunteers rather than real job applicants; age differences; cultural and educational experience differences; work experience differences.
Producing Norms & Developing the Career Focus Report
A set of Hong Kong student norms has been established (N= 421) and more data will be added to this at a later date when it becomes available.
At the same time as this research Quest Partnership also developed a new Career Focus Report for Identity and participating students were provided with a report. This new report has been developed with educational clients in mind but can be used by other clients supporting individuals with career guidance. Currently, the report can be normed against the UK working population and the Hong Kong students.
Translation into Simplified Chinese
The project then made traditional Chinese available as an online solution for clients with a view to collect on-going norms data and to work with any clients who can support with validation studies. In December 2009 the simplified Chinese version was also made available online.
If you are interested in training to use the Identity Questionnaire or if you would like to work with PsyAsia in distributing this assessment, please do get in touch with us.
