validity

validity The property of being genuine, a true reflection of attitudes, behaviour, or characteristics. A measure (such as a question, series of questions, or test) is considered valid if it is thought to measure the concept or property which it claims to measure. For example, it might be contested whether the answers to a question about job satisfaction are a valid indicator of alienation from modern society; the holding of paid employment on the part of a woman is a valid indicator of a feminist consciousness; or the divorce-rate in the United States is a valid indicator of the extent of social stress in that society. The validity of an attitude scale, or of the answers obtained by a question in an interview, is ultimately a matter for judgement, but techniques have been developed to supplement a researcher's own views, which may not be representative.

Rules of thumb have been developed which rule out certain types of question completely. For example, it is generally held to be pointless enquiring long after the event about the attitudes and reasons linked to a decision or choice made many years ago, on the grounds that views tend to be reconstructed with the benefit of hindsight. Arguments of validity rule out proxy interviews for anything except the most basic factual data, such as someone's occupation, if even that. Logical validation, checking for ‘face validity’ in theoretical or commonsense terms, continues to be the most important tool, which is strengthened by employing as wide a range of people as possible to make the checks. This can be extended into the use of panels of experts, judges or juries who are in fact ordinary people who have close familiarity with the topic in question, and can judge whether questions and classifications of replies cover all the situations that arise, and are appropriately worded. Another approach is to present the research instrument to groups of people who are known to have particular views or experience, and see whether it differentiates adequately between the groups. However, the ultimate test is whether the research tools, and the results obtained, are accepted by other scholars as having validity. It is rare for researchers to submit their work to the scrutiny of the research subjects themselves, though this sometimes happens in policy research. Population Censuses are unique in having post-enumeration surveys after each census to check data validity and general quality.

There are many different definitions of validity in the available literature. It is clear that different authors use the terminology in different ways. Part of the problem is that most of the relevant discussion takes place among psychologists, who also provide many of the examples and established procedures for testing for validity, but it is not clear that these always translate easily into sociological contexts.

One elementary and useful distinction sometimes made is that between criterion and construct validity. The former refers to the closeness of fit between a measure (let us say a concept) and the reality that it is supposed to reflect. For example, the Goldthorpe class scheme is intended as and claims to be a measure of ‘conditions and relations of employment’, in particular the material and other benefits of the ‘service relationship’ as against that of the ‘labour contract’. However, for practical reasons its operationization is effected via data about each individual's occupational title (teacher, nurse, or whatever) on the one hand, and employment status on the other (manager, employee, self-employed, and so on). One could therefore investigate the criterion validity of the Goldthorpe scheme by drawing a sample of individuals from within each of the social classes represented in the classification, and then collect independent data relating to the actual conditions and relations of employment of these respondents, for example evidence about their location on an established career ladder and incremental salary scale, enjoyment of enhanced pension rights, and of a certain autonomy in how they use their time at work. In other words, one would examine the degree to which the Goldthorpe classes measure those aspects of employment that they are said to measure, using independent criteria of the concept under investigation.

Construct validation, on the other hand, involves an assessment of whether or not a particular measure (a concept or whatever) relates to other variables in ways that would be predicted by the theory behind the concept. For example, in the case of Goldthorpe (or any other) social classes, one would expect—if this measurement of class was valid—that the social classes so identified would be readily differentiated in terms of (say) voting behaviour, levels of educational attainment, and inequalities in health (or literal ‘life-chances’). These are the sorts of things that, from our understanding of the theory of social class, we anticipate will be associated with the class location of individuals. For this reason construct validity is sometimes also referred to as ‘predictive validity’—rather misleadingly, perhaps, since one is here simply looking for correlations, rather than offering predictions about how weak or strong these correlations will be in actuality. Moreover, the term ‘construct validation’ is itself sometimes applied to the whole validation process, including those aspects earlier discussed under the label of criterion validity.

Arguably, all definitions and concepts of validity are to some extent circular, in the sense that one is attempting to confirm that a sociological construct (a classification, concept, or variable) actually measures what it claims to measure, by comparing that construct with something else (other indicators) that one hopes and assumes are independent of the original measurement. A useful discussion of the many complexities raised by this troublesome notion will be found in R. A. Zeller and and E. G. Carmines , Measurement in the Social Sciences (1980)
. For elaboration of the example provided above, and an illustration of how tests for validity are conducted in practice, see Geoffrey Evans and and Colin Mills , ‘Identifying Class Structure: A Latent Class Analysis of the Criterion-Related and Construct Validity of the Goldthorpe Class Scheme’, European Sociological Review (1998)
. See also RELIABILITY; VARIABLE.