- Published: October 2, 2022
- Updated: October 2, 2022
- University / College: Cornell University
- Language: English
- Downloads: 39
As assessment practitioners we must accept responsibility for ensuring that the development and use of measures, and the interpretation and reporting of information are non-discriminatory, unbiased, and fair towards all South Africans. What are the major pitfalls (historical and current) associated with using psychological measures in the multicultural South African contextPsychological assessment is in South Africa faces many drawbacks due to its history of inequality and the cultural diversity of its population. This essay will aim to highlight all the major weaknesses related to psychological measures and assessment within South Africa, and make mention of some of the ways in which these weaknesses can be overcome. When looking at psychological assessment in the South African context, one has to consider the historical background of segregation and Apartheid in playing a major role in the development of all aspects of life. During the 1900’s psychological tests were imported from Western countries exclusively for the white population group (Van de Vijver & Rothmann, 2004). Apartheid was the political system set in place pre 1994, whereby racial division and discrimination was endorsed. Due to this form of segregation, psychological assessments that were used were prejudiced in favour of the minority white population. Job policies warranted all jobs to be divided along racial lines, with the white population receiving all high ranking jobs and the black population receiving only menial, labour oriented work. Taking in to consideration these circumstances, there was no need for psychological tests to be standardised across racial lines since race groups were not competing equally on the education or job fronts. During the Apartheid era, according to Owen (as cited in Foxcroft, 1997), the Institute for Psychological and Edumetric tests (IPER) and the National Institute for Personal Research (NIPR) of the Human Sciences Research Council (HSRC) were among the largest developers of psychometric tests in South Africa and were responsible for constructing separate tests for different race groups. It is also noteworthy that most tests were developed specifically for members of the white population rather than for the black population, even though members of the black racial group constituted majority of the South African population. It became the norm to use tests that were developed for the white population across racial, lingual and cultural lines (Foxcroft, 1997). Since measures were not created for a multicultural context, test results for the non white population were biased, inequivalent and invalid. Bias, equivalence, test fairness, validity and reliability are all important concepts which relate to some of the pitfalls of psychological measures used in South Africa. A basic definition and understanding of these terms will be given, and their relation with regards to South African psychological assessment will be covered throughout this paper. Bias refers to the presence of nuisance factors in cross cultural measurement resulting in unfair decisions being made regarding ” intervention, educational placement, or job selection…” (Foxcroft, 1997, p. 229). Test bias is statistical and objective (Visser & Viviers, 2010). Three types of bias have been identified, namely: Construct bias, Method bias and Item bias (Van de Vijver & Rothmann, 2004). The presence of any of these three types of bias prevents full scale equivalence. Construct bias arises when the actual construct being measured is not identical across groups (Van de Vijver & Rothmann, 2004). Family roles, perceptions, beliefs and values are all examples of constructs that may vary from one culture to the next. For example while members of the white population may value independence from family members in young adults, members of the Muslim community emphasise on close knit parent-child relations throughout adulthood. Culture therefore affects behaviour, resulting in different test results being obtained from test takers of different cultural backgrounds. Sample bias, administration bias and instrument bias are three types of what is known as method bias; which refers to a form of bias arising from the method or procedure that is followed during psychological assessment. When developing a test for a multicultural context, attention must be paid to the method that the test should be administered in, for example paper and pencil versus computer based tests; the format of the tests such as multiple choice questions versus open ended questions; and response modes of a test such as verbal versus written (Foxcroft, 2004). Computer literacy, reading and writing capabilities, education and language vary across cultures in South Africa and if not carefully considered and administered in a method that is fair to all, the results obtained will be biased. Item bias is a form of bias that arises from the item itself, with the most common item bias being ” poor item translation, ambiguities in the original item, low familiarity or appropriateness of the item content in certain cultures, and the influence of cultural specifics such as nuisance factors or connotations associated with the item wording” (Van de Vijver & Rothmann, 2004, p. 3). Test fairness and test bias are often used interchangeably, however test fairness refers to the significance of social values in tests, that which test developers cannot control (Visser & Viviers, 2010). It is a broad concept that has no single definition and is controlled by the test users. Historically, test fairness was not common with the apartheid system in place, catering only for the white population Presently there are factors that indicate test fairness among test administrators is still low (Setshedi, 2008). A further aspect that needs to be taken in to consideration with psychological assessment according to Van de Vijver and Rothmann (2004) is that of equivalence, which refers to how test results compare across different cultural groups. Construct inequivalence occurs when a construct being measured overlaps across groups or the instrument that is being used measured different constructs across groups (Visser & Viviers, 2010). Due to South Africa’s multicultural society, and inappropriate tests being used across cultural lines, inequivalence is a major factor that hampers test results. Validity refers to the extent to which a test measures what it claims to measure (Setshedi, 2008). If for example the theory used in a test is not substantiated, then the results obtained will be invalid (Foxcroft, 2004). In the South African context, not enough research has been done in to the use of tests that can be administered cross culturally, therefore tests that are being used do not necessarily measure what it is supposed to for all the population groups, resulting in invalid test scores. According to Setshedi (2008), reliability can be defined as ” the degree of accuracy and consistency with which the test measures in repeated administrations” (p. 22). Evidently, since tests used in South Africa were not created for cross cultural examination, results obtained may only be accurate for certain cultural groups, and inconsistent with others. Validity and reliability are crucial to all tests, however in a multicultural society, results tend to be low on these fronts. Tests during the apartheid era were also used to reinforce inequalities (Setshedi, 2008), and results were used mainly for the purpose of categorisation. Black students who obtained lower scores were said to have needed special or remedial classes without consideration of their demographic or scoio-cultural backgrounds. Furthermore, due to tests such as IQ measures, the black population due to their inferior educational provisions scored lower than average, and this was therefore used as justification for assigning them menial, labour oriented work (Setshedi, 2008). The black population group were placed at a further disadvantage during the apartheid era due to the fact that low results obtained from westernised measures were overlooked and was considered as being the norm. Since the black population were regarded as coming from a socio-culturally disadvantaged background, impaired functioning therefore often went unrecognised (Foxcroft, 1997). This can be seen as among some of the leading factors that resulted in the general negative view of psychological assessment in South Africa, specifically among the black population. Post 1994, with the advent of a democratic government taking rule over South Africa, a demand for fair tests that were standardized and unbiased across all racial, lingual and cultural lines was demanded (Foxcroft, 1997). Prejudice with regard to psychological tests was addressed with the enactment of the new employment equity act 55 of 1998, section 8 (Government Gazette, 1998), which stated ” psychological testing and other similar assessments are prohibited unless the test or assessment being used – (a) has been scientifically shown to be valid and reliable; (b) can be applied fairly to all employees; and (c) is not biased against any employee or group.” This resulted in test developers and administrators having to deal with both legal and ethical issues regarding psychological assessment. When comparing the new employment equity act to other countries, Vijver and Rothman (2004) note that while all countries forbid any form of discrimination in psychological testing, most countries assume psychological tests being used are fair unless otherwise proven; whereas South African law requires psychologists to proactively prove that tests are fair and unbiased before being applied, depicting the demand for fair testing.. Even though the need for fair and unbiased psychological tests post apartheid had been recognised in South Africa, not enough research had been developed during the 1990’s. According to Foxcroft (1997), one of the reasons for this could be that in the new democratic era, the HSRC focused on the critical national priorities, whereas test development was not among its core focus. Currently there is still not enough research or evidence in to how valid the psychological tests being used in South Africa are for its multicultural society (Visser & Viviers, 2010). Another reason for the inadequate amount of research in this area is that the development of tests that can be used across cultural and linguistic lines is a slow, complex and costly process (Foxcroft, 1997), while Shuttleworth & Jordan (1996) even consider it an ” unattainable goal” (p. 97). Many in South Africa question whether psychological testing could ever be fair across all cultural groups. Tests and measures that are currently being used were developed over fifteen years ago (Paterson & Uys, 2005), and are no longer suitable for current use in organisations due to the ever changing nature of the modern world. The use of outdated tests leads to erroneous and invalid results, which often has monetary repercussions in organisations. Furthermore, with the need to compete globally among international markets, test administrators often import international tests and applied them locally (Paterson & Uys, 2005). Due the multicultural context of South African society, these tests are often leads to inaccurate results, as they have been developed for a western culture, and use theories based on a western perspective. Due to this, there is an imperative need for test developers to create new tests based on valid, appropriate theories for South Africa’s multicultural context. Furthermore, psychological testing and psychological assessment are two terms that are often used interchangeably. These two constructs however hold different meanings. According to Cilliers and Rothman (2004) the term ‘ test’ can be defined as ” group and individually administered standardised measures of aptitudes, achievement, intelligence, personality, social, language, perception and motor skills” (p. 13); whereas the term psychological assessment according to Setshedi (2008) can be defined as ” the use of one or more standardised instruments to gather data that is combined with data from other sources to inform some type of intervention” (p. 16). In South Africa, test administrators need to remember that psychological tests are just one source of data for assessment and that results obtained from tests need to combined with other sources of information such as interviews, educational history and previous work experience. According to Foxcroft (2004), attention needs to be paid to cultural relevance from the initial planning and design phases of developing tests in order to avoid bias. The fact that the test is being developed for a multicultural context should also be mentioned in the purpose statement of the test. Furthermore, given South Africa’s poor history with regard to developing culturally appropriate tests, test developers need to pay careful consideration to basic matters such as what methods of administration will be culturally appropriate and what language to develop a test in. In this way, the test administrator will be able to ascertain whether or not there is a shared understanding of the constructs, and will be able to discover the common dimensions or differences that emerge across groups. A major pitfall in psychological assessment in a country such as South Africa is the vast range of quality of education and opportunities across cultural and racial groups(Foxcroft, 2004) , with certain groups having an unfair advantage over others with regard to acquiring certain jobs or gaining admittance in to certain educational programmes, with specifically those from rural areas being largely underprivileged. Abilities to read, write, calculate and higher order cognitive development are dependent on schooling and educational levels attained. Psychological tests are also believed to be used for ‘ gate keeping,’ (Setshedi, 2008) by refusing admission into elite schools to previously disadvantaged students who do not meet the required standards. According to Tylor (as cited in Foxcroft, 1997), a way to overcome this disadvantage is to develop tests that recognise learning potential. In this way, a persons potential is recognised as opposed to their limitations. This would be ideal in South Africa, since people who have been previously disadvantaged due to their race and limited access to education, can still be recognised according to their abilities to learn new skills and meet new demands rather than on their skill gaps and limitations. With limited tests developed specifically for South African use, psychologists are forced to adapt international tests and align them with the contextually standardised norms (Paterson & Uys, 2005). However, when adapting tests, a major factor which hampers results is that of language. Should tests that were originally developed in one language be translated in to the test takers first language, and if so will this change the meaning of certain phrases or terms resulting in a lower score for the test taker due to language barriers? When adapting international tests for local use, guidelines should be followed to ensure that translation is accurate. However, certain words or phrases in one language may have no accurate equivalence when being translated to another language, therefore altering the meaning of the word or phrase. Language has also been found to be problematic with regard to the level of difficulty of the test language if it is not to be translated (especially important if the test language is a second or third language to the test taker); and the language proficiency of the test taker (Paterson & Uys, 2005). It is important to remember that a test should reflect the test takers abilities, not their proficiency in the language of the test (Paterson & Uys, 2005). The administration of psychological tests also needs to be managed by properly trained and competent users, who are sensitive to cross cultural test use. It should also be noted that some test takers may speak an indigenous language at home, but are educated in English, and consequently are more comfortable with being tested in English rather than in their first language. It has therefore been suggested by Foxcroft (1997) that bilingual assessment could be the best method of psychological testing for such individuals. Bilingual assessment gives people who speak 2 or more languages a chance to be assessed in a way in which they are comfortable with to ensure language barriers do not detract from their test results. One way of overcoming language barriers in recent times has been through the use of non verbal techniques, such as projective tests (Paterson & Uys, 2005). However, problems with language still arise as the test takers response still has to be translated, in which process some valuable information may be ‘ lost in translation.’ Furthermore, the test takers response also has to be interpreted by the administrator, which in a multicultural country such as South Africa, has the risk of being interpreted incorrectly across cultures. It is also of the belief that psychological assessment is often used with incorrect intentions, such as for legal purposes to retain students from moving on to the next grade, since educators need proof to substantiate their belief that the student should be held back (Setshedi, 2008). Furthermore Setshedi (2008) concludes that psychological assessment is being used in many schools not for the use of intervention, but rather so that a sufficient amount of students would be identified for the need of remedial or special classes, so that these classes could be kept in existence. However, majority of the students found in these classes are black, creating the assumption that these assessments could be creating a self fulfilling prophecy with regard to certain educators or schools. Cultural loading is an aspect that further detracts validity of psychological assessment (Van de Vijver & Rothmann, 2004). Some cultural groups can be more familiar with certain topics on which questions are based, therefore giving them an unfair advantage. An example of this being if a question on an intelligence test for children asks from which animal do we get steak, children who come from vegetarian backgrounds (such as the hindu religion) may be placed at a disadvantage, whereas children who come from meat eating families will be more likely to have a familiarity with the subject. However, due to the vast amount of different cultural groups found in South Africa, creating tests that are completely free of culture and fair to all is an extremely difficult task. It has been found that response sets and social desirability are found in high numbers among populations that are historically less affluent and disadvantaged (Van de Vijver & Rothmann, 2004; Visser & Viviers, 2010). This can be linked to members of the black population, who due to their previously lower ranking status during the apartheid regime, may now find themselves trying to fit in to the social norms, thus adding bias to the results. This leads to an additional issue of norming (Paterson & Uys, 2005). There is a need for a standard level of achievement to which tests can be used to compare test takers performances. However, being such a diverse country with people from different educational backgrounds, a lot more research needs to be conducted in to how to develop standardised norms across all cultures and backgrounds. Race, age, gender, and socio-economic status are further aspects that need to be considered when developing norms. However, there is also a general consensus that norms should not be blindly accepted, and that a mixed approach should be adopted, so that quantitative statistical data influenced by western approaches are combined with qualitative, content driven interpretations that have a contextualised focus (Paterson & Uys, 2005). Another pitfall regarding psychological assessment in South Africa is that many tests being administered are not registered with the Health Professions Council of South Africa. This has led to unethical practices followed by litigation (Paterson & Uys, 2005). Tests that can be administered by non professionals should be avoided, and information gathered from test results should be handled sensitively and confidentially. However, according to Paterson and Uys (2005), sometimes a practitioner may need to use a specific type of test that is not incorporated in the HPCSA list. Registering a psychometric test is seen by most practitioners in South Africa is a very complex, expensive, rigid and bureaucratic process (Paterson & Uys, 2005); which may be part of the reason why unregistered tests are often administered. Also, most small organisations cannot afford to invest in assessment and therefore use unregistered tests that are easy to access and does not require trained professionals to administer (Paterson & Uys, 2005). According to Foxcroft (2004) there is a shortage of specialised test developers and measurement experts in South Africa who are sufficiently trained to be dealing with psychological measures and assessment. Not enough post graduate training had been provided by the HSRC with regard to test development, so when the previous generation of experts retired, there was a void in measurement expertise In South Africa. According to Paterson and Uys (2005), the HPCSA and other professional bodies like the Psychological Association of South Africa (PsySSA) needs to give more consideration to issues surrounding psychological assessment and offer more support with regard to providing training. The new democratic South Africa must not fall back in to the trap of unfair testing through affirmative action policies (Foxcroft, 1997), such as raising results of school children from certain cultural groups, as this ensures a continuation of biased results. As a concluding remark it should be noted that most psychological assessments conducted for organisations focus primarily on organisation effectiveness rather than on individual effectiveness, and employee dissatisfaction is seen as based on the individuals inability to cope rather than some facet of the job or work environment (Cilliers and Rothman, 2004). Psychological assessment needs to place more emphasis not only on how aspects of cultural difference affect results, but also on how to build on the strengths of culturally different individuals in a way that will maximise organisational as well as individual prosperity. While South Africa has come a long way since the enactment of the new employment equity act, there is still a long way to go with regard to developing psychological measures that are fair and unbiased across all cultural groups.