information products research distributors copyright bibliography links orders training contact

 

APPLICATIONS:


J C Raven Ltd. are responsible for publication of the Raven's Progressive Matrices and Vocabulary Scales. These tests, non-verbal and verbal, are widely used tools for the measurement of cognitive ability.
Originally created over 60 years ago, but with continual development and research, the tests have built a reputation for excellence. Raven's Progressive Matrices are used within educational, occupational, and clinical settings throughout the world. Possibly the most important uses are in research into the nature, development, and correlates of human abilities.
The tests measure the two main components of general intellectual ability. The Progressive Matrices measure "Eductive" ability. The term comes from the Latin root "educere", meaning "to draw out". It thus refers to the ability to make sense of complex situations, to draw meaning out of confusion, to perceive clearly and form orderly judgements. The Vocabulary Scales measure "Reproductive" ability - the ability to store and reproduce information reflecting a culture's common pool of knowledge. These two components are reasonably distinct, have different genetic and environmental origins, and predict different types of contributions to life in the community and workplace.

Raven's tests have various uses, either together, separately or as part of a custom made battery of tests. Used together, the two tests provide almost all the useful information that can be obtained from a full-length "intelligence" test. Because the test booklets are reusable, the Raven's Progressive Matrices and Vocabulary Scales are the most cost-effective measures of general cognitive
ability available. New computerised versions of the tests also enhance the effectiveness of the tests. They can be used to:

  • Identify more effective operatives, supervisors, professionals, business people, entrepreneurs, managers and administrators.
  • Predict children's and adult's success in training and educational programmes.
  • Identify neuropsychological damage
  • Carry out research involving ethnic and cultural groups.

top

Applications of the RPM and MHV

Although Raven's Progressive Matrices and Vocabulary Scales were developed primarily as theoretically-based tools for use in research, and have been so used in more than 2,500 published studies, they have also found wide application in educational, clinical, and occupational practice. General issues concerning validity are discussed here;

Accepted validity indices present a number of difficulties. Consider, for example, the nature of occupational competence. Conventional "validation studies" correlate some measure of job proficiency with scores on an ability test, such as the RPM, and usually find a positive correlation. However, there is more to job proficiency than mental ability. Studies employing "critical-incident methodology to identify the specific behaviours which characterise more and less effective employees (whether panel beaters, machine operatives, bus drivers, teachers, or doctors) tend to conclude that the two groups differ in many ways beside their levels of eductive ability. They differ in their ability to take initiative, to communicate, and to work with others. Studies also show that none of the members of the high-performing group engage in all the behaviours, or possess all the high-level competencies, which distinguish the two groups. On the contrary, any occupational group includes, and must include, a wide range of people who possess very different patterns of motivation and competence. Any group of outstanding panel beaters, for example, will include some people whose forte it is to understand the workings of an organisation and what needs to be done when things go wrong, some whose talent is to recognise activities which should be undertaken, and still others who are good at getting authorisation to attempt to translate new ideas into practice.

Three things follow from this discussion. Firstly, success in any particular occupational group is likely to be more dependent on motivational dispositions than on eductive ability itself. Secondly, eductive ability will be required for effective behaviour in any of these activities, however differently it manifests itself. Finally, different people within any one group will be successful in very different ways and because they possess very different clusters of self-motivated competencies.

What any one person can do, and the effects of his or her actions, are also very dependent on what others do. Thus, as Gardner has observed, intelligence and enterprise are cultural — "distributed" — rather than individual characteristics since generating new understanding and action is dependent on a number of people contributing in very different ways to a group effort.

In the light of these observations it is obvious, firstly, that the task of establishing the validity of a test is, as Messick has also emphasised, much more problematic than many people take it to be, and, secondly, that the ways in which a test can be used in selection and placement need to be more subtle than many people assume. Also, it seems that the best we can expect from traditional group-based correlational studies (as distinct from fine-grained ethnographic studies) are relatively low correlations between test scores and qualities which are called for, to some extent in many situations. No single ability can be expected to relate closely to each of the many different types of outstanding performance that are required.

The conclusions we have reached in our discussion of the occupational area apply with equal force in education and in the field of clinical psychology.

Thus, although the RPM and MHV do detect two very important concerns and abilities, and although their combined use yields a valuable index of g, the information they provide needs to be supplemented. Once it is established that an individual possesses one or another of the alternative talents needed within an occupational or educational group, a relatively low score on the RPM or MHV may be of little consequence. Conversely, even though the performance of many roles demands high levels of eductive and or reproductive ability, exceptionally high scores in these areas may be less important than such things as the willingness and ability to motivate people, to sift information for good ideas and translate them into practice, to work for the good of the organisation rather than for personal advancement etc. It may, then, be important to distinguish between threshold abilities, and those abilities required for effective performance of different aspects of a given job.

Psychologists have made little progress in developing ways of thinking about, and assessing talents outside the domain of the RPM and MHV. The adoption of these tests is to be recommended because (a) the results are clearly and directly interpretable; (b) their use would contribute to the data pool, and thereby generate greater understanding of the issues discussed here, and (c) used as indices of two relatively independent abilities within what is generally thought a fairly unitary cognitive domain, they will fuel demand for a framework for the discovery, development, and assessment of multiple talents.

top

Educational Applications

Although the most important application of the RPM and MHV in education is in the context of a multiple-talent framework, we will first review more familiar uses of the tests.

Perhaps the first thing which needs to be said is that the RPM and MHV between them provide, in an extremely quick and cost-effective way, most of the meaningful and useful information that can be obtained by using full-length Intelligence tests, As a result, many teachers and others use the tests for these purposes in their day-to-day work. They are also widely used by educational psychologists and diagnosticians after full-length Intelligence tests have been administered in order to obtain more accurate information than it is possible to derive from the subscales of these longer tests. They assist in the interpretation of subscale score discrepancies and in the identification of the causes of children's problems.

A more specific institutional use of the tests in educational selection has been as preliminary screening instruments to reduce the number of detailed assessments required to determine eligibility for Special Education programmes. The RPM has also been used as one of a number of alternative criteria for entry into Gifted Education programmes — alongside such things as academic achievement, musical performance, athletic prowess, artistic excellence, and leadership potential. Used in this way, the RPM brings into such programmes gifted young people who have undoubted abilities, but who may be more inclined than those with high academic grades to ask questions, make their own observations, and think for themselves — and less willing to learn whatever is put in front of them. Multiple talent programmes catering for such a range of abilities have the advantage that they are more likely than "accelerated academic" programmes to stimulate teachers to experiment with multiple-talent approaches, and thereby to develop ways of identifying fostering and recognising different types of giftedness. If the MHV were also used as part of the battery of selection instruments, it might promote wider recognition that those who are best at forging new insights are not necessarily good at expressing their ideas in words (and vice versa). This might encourage schools to be less stringent about verbal skills, and more willing to foster and recognise other abilities. As several researchers have shown, the use of educational performance as a principal criterion for the selection of senior administrators has led to the promotion into influential positions of a disproportionate number of people whose main talent is the ability to use words in such a way as to create a good impression on those in authority and thus secure their own advancement.

The RPM has found application in the diagnosis and handling of various forms of dyslexia. Some of those children experiencing great difficulty with schoolwork obtain high scores on either or both the RPM and MHV. Reactions to such pupils vary. Some seek to remediate the problem with schoolwork, others try to find ways in which the children can develop, and get credit for their other talents.

A wider implication of this last observation is that psychologists need to focus their efforts on breaking the stranglehold exercised by purely academic assessments on the talents which can be fostered and recorded in schools. While it is increasingly accepted that children should not be severely penalised for their inability to read, write, and spell, the view that traditional forms of "academic ability" are of overwhelming importance remains largely unchallenged. Yet there are good reasons to dispute such a view. What passes for "academic" knowledge in schools is generally outdated when it comes to be taught, unhelpful in the solution of problems, and forgotten by the time it is needed. Furthermore, "academic" grades at any terminal level have virtually no predictive validity to occupational performance. It seems, therefore, that in finding ways of dealing with "dyslexia", the problem is not so much to develop the diagnostic and remedial tools which teachers undoubtedly need as aids in teaching reading or even to find ways in which "clever" dyslexics can get the opportunity to display their academic prowess, but to evolve the concepts, tools, and societal arrangements which are required to implement multiple-talent educational programmes.

top

Clinical Applications

Clinical disorders are commonly identified and diagnosed because an individual functions in a manner incongruent with their perceived or previously established ability level. Whereas the conventional psychometric indicators of reliability and validity are high for RPM and MHV among healthy people, discordant results can, therefore be expected in clinical populations. In most cases there is no permanent loss of either eductive or reproductive ability, though both can be disrupted by some disorders. Low and inconsistent performance can be related to other factors, most notably motivation, and perception of the relevance of a given task

The clearest and most stable deficits occur in cases invoking cerebral dysfunction. The CPM in particular has been widely used to identify both mental retardation, whether from birth or trauma, and dementia in the elderly population. Among adults, the RPM can be used to monitor dysfunctional severity, and the processes of recovery or decline. The All-Open-Ended (or Definitions) version of MHV is a valuable instrument for the detection and analysis of verbal dysfunction (as in the aphasias).

Dysfunction is also characteristic of those suffering the toxic effects of alcohol and other drugs. The progress of an individuals functional level can be assessed by serial testing, and some investigators have item-analysed errors in order to detect patterns of abnormal response.

A great deal of early work with the SPM and MHV involved documenting the deficits of schizophrenic patients. Since disordered thinking is a clinical symptom in such patients, and verbal aberrations are commonplace, the combined instruments lend themselves well to such studies. The breakdown of complex thought processes, and the forging of loose associations, would be sufficient to impair SPM performance. Similarly, on the MHV, the choice of inappropriate synonyms based on similarity rhyming or other irrelevant associations, can point to difficulties with thinking and expression.

Less evident and often missed among seriously disabled groups, is a diminishing of function due to emotional and motivational factors. High levels of anxiety and/or depression can have a negative effect on performance, and it is important to recognise these symptoms in those suffering cerebral trauma, or in the acute phases of a schizophrenic episode. The effects of anxiety and depression can be similar in many ways to the disruptions of functional and organic psychoses. High anxiety can result in confusion, constricted thought patterns, blocks against the solution of apparently simple problems, and an inability to deal with perceptual rotations. The anxious subject will often rush to an uncritical first response failing to scan all the information available. A careful psychologist will be aware of the effects of depression and anxiety on performance, and guard against jumping to the conclusion that the observed dysfunctional performance has more serious causes.

Similarly, those suffering retarded depressive reactions can be handicapped in timed tests by slowness of thought and may settle for simple answers rather than struggle with complexity. Fluency with words is often reduced resulting in sparse performances on the MHV.

In addition to clinical indicators, and also among delinquent populations, questions about motivation must be considered when interpreting results.

A sound estimate of ability cannot be made when the tests are considered outside the context of everyday life. For many clinical patients, psychological testing is an unimportant and unattractive disturbance in their routine. When pain, confusion, depression, anxiety and survival itself are uppermost in a person's mind, motivation to sustain attention to a highly artificial task will be low.

It follows from this discussion that clinicians, in order to check the validity of results, must not only make careful use of scores and norms, but also of the tables showing the expected distributions of scores across subscales.

top

Occupational Applications

There are major problems associated with the establishment of test validity in occupational settings:

a) Problems associated with the criteria of success: The qualities apparently required to perform a job "well" depend on the criteria adopted when evaluating performance. Different qualities are, for example, required to procure rapid advancement in an organisation, to secure the survival of that organisation through the invention of new products, to secure its growth through financial and / or political manipulation, and to secure the survival of society. Those who are best able to secure the esteem of those above them are not necessarily best at releasing the energy and talents of their subordinates and, indeed, often advance themselves by applying their cognitive abilities to making their sections appear more "efficient" by getting rid of the personnel, eliminating the time, and destroying the networks of contacts which are required for institutional development. Also by eliminating those with alternative viewpoints who might challenge their views or compete for their position.

b) Problems deriving from the use of inadequate job analyses and job descriptions: The activities required for the effective performance of a job may differ from those identified in the job description and thus be overlooked when attempts are being made to validate selection procedures.

c) Problems created by the use of inappropriate selection procedures in the past: Those best able to perform a job may have been (intentionally or unintentionally) eliminated from those admitted to the workforce. If this has happened it will be impossible to demonstrate the importance of the required qualities.

d) Problems created by the non-attributable nature of outcomes: In most organisations it is extremely difficult to attribute observable effects to any one person or group of persons particularly when circumstances are continuously changing and the effects of actions may take many years to show up. This makes it very difficult to collect accurate information about whose work genuinely benefits the organisation and distinguish those who confer important benefits from those who are only able to create a good impression but then manage to move on before their mistakes are discovered.

e) External constraints: Organisational arrangements, and other people's expectations, may prevent people doing the things required for effective job performance.

f) Change over time: People do different things in the "same" job at different times of day and at different points in their life cycle. They may, for example, engage in routine activities for part of the day and in innovative ones at other times. They may develop technological innovations early in their careers and engage with the political processes which control the funding for such innovations later in their lives.

top

g) Effectiveness is a cultural quality: The effective performance of any job function often requires a wide range of people with very different abilities both within the job function and among those performing related roles within the organisation. "Enterprise", in particular, is a cultural rather than an individual characteristic.

Despite these problems, it has been established that both the RPM and MHV — unlike many other tests do relate to managerial performance: to staff and financial turnover, to profitability, and to the ability of the firm to survive financial and other crises. The two tests predict different ways of contributing to these processes. The ability to find new ways of thinking about and doing things — what Kirton calls "lnnovation" — is better predicted by the RPM, whereas Adaptation is better predicted by the MHV. For one group of managers (e.g. public servants), verbal fluency is important but for many others a more crucial talent may be the ability to make sense of new and confusing situations. The qualities required for any task depend on the prevailing circumstances. Reproductive behaviour often serves well in unchanging circumstances, but at times of flux (e.g. during wars or periods of rapid economic change) the reproduction of old habits can prove disastrous. This is why it is so important for the public service to recruit and promote a wide range of people who have different motives and abilities. Unfortunately, people who possess one set of concerns and abilities often find it very difficult to understand, let alone appreciate, those of others.

Although much of this discussion has revolved around management selection and the difficulties involved in validating tests in such settings, this is by no means the only area in which the tests have proved useful. The tests have been validated in many settings and with many occupational groups. More generally, meta-analyses and validity generalisation analyses show that tests of intellectual ability predict proficiency across eight very broad classes of work: managerial, clerical, sales, protective professions, service jobs, trades and crafts, vehicle operation, and simple industrial work.

Although, as we have seen, the predictive validity of the RPM and MHV to success within an occupation is (in absolute terms, though not in comparison with other tests) relatively low, Kendrick and Funder have shown how the use of a small number of independent tests with such predictive validity, or similar tests in a sequence of decisions, can cumulatively result in huge financial gains. Further, the use of such tests in association with Assessment Centre and Behavioural Event Interviewing approaches allows the staff concerned to concentrate on qualities which cannot be readily assessed with the formal tools currently available and thus greatly improves the validity of such procedures.

A still more positive conclusion concerning the use of the RPM and MHV in selection is obtained if, instead of seeking to validate tests within occupations, one adopts a broader perspective.

Jaques has argued that a large number of competencies which critical-incident studies have shown to be crucial to effective performance in a wide range of jobs are heavily dependent on "cognitive" abilities (defined to include their affective and conative components) of the kind assessed by the Matrices. These competencies include the ability to build up one's own understanding of the way in which one's organisation functions, viewing one's own part in it in appropriate ways, taking initiative to intervene in organisational processes, building up one's own understanding of the workings of external political and economic systems and intervening in them for the benefit of one's organisation and society, and thinking about the motives and talents of subordinates and how best to place them so as to harness their motives and develop their talents.

Jaque's conclusion is that high levels of cognitive ability are crucial to the effective performance of the activities we are discussing. This means that the RPM should be much better at predicting the ability to attain and retain jobs requiring high levels of work capacity than at predicting performance within any one occupational group. Evidence to support this contention comes from a number of sources, but most notably from the work of Hope who found that, in both the US and the UK, some two thirds of social mobility, both upward and downward, could be predicted from 11 year olds scores on tests of general intelligence.

If Jaques is right to argue that cognitive ability is related most closely to the level of job attained and retained it is not necessarily the case that the highest scoring personnel will make the best recruits to all jobs. There may well be an optimal range of scores. One study reporting such an effect was conducted by J C Raven and his colleagues among telephone engineers. The conclusion that the relationship between ability and performance is curvilinear may be reconciled with the Hunter's statement that the relationships within all groups are linear by recalling the criterion problem. In Raven's study, the finding was not that higher scoring employees performed worse: it was that those with higher scores left the employment.

A good illustration of the overlap between the qualities assessed by the tests and those actually required for effective performance comes from an unpublished study which found that success in computer programming could be predicted from the APM administered without a time limit. Both tasks demand high levels of persistence, the invention of ways of solving problems, attention to detail, and the willingness and the ability to check accuracy before moving on. It follows from this discussion that the effective development and deployment of human resources is dependent on developments in organisational psychology and appraisal systems. New ways of thinking about and documenting the interaction between these domains are required.

The authors would welcome any contact with others working in these areas.

top


home contact us