MEASUREMENT

There are many standards that can be used to evaluate the status of a science, and one of the most important is how well variables are measured. The idea of measurement is relatively simple. It is associating numbers with aspects of objects, events, or other entities according to rules, and so measurement has existed for as long as there have been numbers, counting, and concepts of magnitude. In daily living, measurement is encountered in myriad ways. For example, measurement is used in considering time, temperature, distance, and weight. It happens that these concepts and quite a few others are basic to many sciences. The notion of measurement as expressing magnitudes is fundamental, and the observation that if something exists, it must exist in some quantity is probably too old to attribute to the proper authority. This notion of quantification is associated with a common dictionary definition of measurement: "The extent, capacity, or amount ascertained by measuring."

A concept such as distance may be considered to explore the meaning of measurement. To measure distance, one may turn to a simple example of a straight line drawn between two points on a sheet of paper. There is an origin or beginning point and an end point, and an infinite number of points between the beginning and the end. To measure in a standard way, a unit of distance has to be arbitrarily defined, such as an inch. Then the distance of any straight line can be observed in inches or fractions of inches. For convenience, arbitrary rules can be established for designating number of inches, such as feet, yards, and miles. If another standard is used—say, meters—the relationship between inches and meters is one in which no information is lost in going from one to the other. So, in summary, in the concept of measurement as considered thus far, there are several properties, two of which should be noted particularly: the use of arbitrary standardized units and the assumption of continuous possible points between any two given points. The case would be similar if time, temperature, or weight were used as an example.

There is another property mentioned above, a beginning point, and the notion of the beginning point has to be examined more carefully. In distance, if one measures 1 inch from a beginning point on a straight line, and then measures to a second point 2 inches, one may say that the distance from the beginning point of the second point is twice that of the first point. With temperature, however, there is a problem. If one measures temperature from the point of freezing using the Celsius scale, which sets 0 degrees at the freezing point of water under specified conditions, then one can observe temperatures of 10 degrees and of 20 degrees. It is now proper to say that the second measurement is twice as many degrees from the origin as the first measure, but one cannot say that it is twice the temperature. The reason for this is that the origin that has been chosen is not the origin that is required to make that kind of mathematical statement. For temperature, the origin is a value known as absolute zero, the absence of any heat, a value that is known only theoretically but has been approximated.

This problem is usually understood easily, but it can be made more simple to understand by illustrating how it operates in measuring distance. Suppose a surveyor is measuring distance along a road from A to B to C to D. A is a long distance from B. Arriving at B, the surveyor measures the distance from B to C and finds it is 10 miles, and then the distance from B to D is found to be 20 miles. The surveyor can say that the distance is twice as many miles from B to D as from B to C, but he cannot say that the distance from A to D is twice the distance from A to C, which is the error one would make if one used the Celsius temperature scale improperly. Measuring from the absolute origin for the purpose of carrying out mathematical operations has become known as ratio level measurement.

The idea of "levels of measurement" has been popularized following the formulation by the psychologist S. S. Stevens (1966). Stevens first identifies scales of measurement much as measurement is defined above, and then notes that the type of scale achieved depends upon the basic empirical operations performed. The operations performed are limited by the concrete procedures and by the "peculiarities of the thing being scaled." This leads to the types of scales—nominal, ordinal, interval, and ratio—which are characterized "by the kinds of transformations that leave the 'structure' of the scale undistorted." This "sets limits to the kinds of statistical manipulation that can legitimately be applied to the scaled data."

Nominal scales can be of a type like numbering individuals for identification, which creates a class for each individual. Or there can be classes for placement on the basis of equality within each class with regard to some characteristic of the object. Ordinal scales arise from the operation of rank ordering. Stevens expressed the opinion that most of the scales used by psychologists are ordinal, which means that there is a determination of whether objects are greater than or less than each other on characteristics of the object, and thus there is an ordering from smallest to largest. This is a crucial point that is examined below. Interval scales (equal-interval scales) are of the type discussed above, like temperature and these are subject to linear transformation with invariance. There are some limitations on the mathematical operations that can be carried out, but in general these limitations do not impede use of most statistical and other operations carried out in science. As noted, when the equal-interval scales have an absolute zero, they are called ratio scales. A lucid presentation of the issue of invariance of transformations and the limitations of use of mathematical operations (such as addition, subtraction, multiplication, and division) on interval scales is readily available in Nunnally (1978).

What is important to emphasize is that how the scales are constructed, as well as how the scalesare used, determines the level of measurement. With regard to ordinal scales, Nunnally makes a concise and precise statement that should be read carefully: "With ordinal scales, none of the fundamental operations of algebra may be applied. In the use of descriptive statistics, it makes no sense to add, subtract, divide, or multiply ranks. Since an ordinal scale is defined entirely in terms of inequalities, only the algebra of inequalities can be used to analyze measures made on such scales" (1978, p. 22). What this means is that if one carries out a set of operations that are described as making an ordinal scale, the moment one adds, subtracts, divides, or multiplies the ranks, one has treated the scale as a particular type of interval scale. Most commonly, the type of scale that is de facto created when ordinal data are subject to ordinary procedures like addition, subtraction, division, and/or multiplication is through the assumption that the difference between ranks are equal, leading to sets like 1, 2, 3, 4, 5, 6, and thus to the treatment for ties such as 1, 2, 4, 4, 4, 6. This is sometimes called a flat distribution with an interval of one unit between each pair of ordered cases. Effectively, this is the same kind of distribution in principle as the use of ordered categories, such as quartiles, deciles, or percentiles, but it is a more restrictively defined distribution of one case per category. To repeat, for emphasis: The use of addition, subtraction, division, and/or multiplication with ordinal data automatically requires assumptions of intervals, and one is no longer at the level of ordinal analysis. Thus, virtually all statistical procedures based on collected ordered or rank data actually assume a special form of interval data.

ISSUES ON LEVEL OF MEASUREMENT

A number of issues are associated with the notion of levels of measurement. For example, are all types of measurement included in the concepts of nominal, ordinal, interval, and ratio? What is the impact of using particular statistical procedures when data are not in the form of well-measured interval scales? What kind of measurement appears (epistemologically) appropriate for the social and behavioral sciences?

The last question should probably be examined first. For example, are measures made about attributes of persons nominal, ordinal, or interval? In general, we cannot think of meaningful variables unless they at least imply order, but is order all that one thinks of when one thinks about characteristics of persons? For example, if one thinks of heights, say of all males of a given age, such as 25, does measurement imply ordering them on an interval scale? We know that height is a measure of distance, so we assume the way one should measure this is by using a standard. For purposes of the example here, an interval scale is proposed, and the construction is as follows. The shortest, 25 year-old male (the category defined as 25 years and 0 days to 25 years and 365 days of age) and the tallest are identified. The two persons are placed back to back, front to front, and every other possible way, and the distance between the height of the shortest and the tallest is estimated on a stick. Many estimates are made on the stick, until the spots where agreement begins to show discretely are evident; and so, with whatever error occurs in the process, the locations of beginning and end are indicated on the stick. Now the distance between the beginning and the end is divided into equal intervals, and thus an interval scale has been created. On this scale it is possible to measure every other male who is 25 years old, and the measure can be stated in terms of the number of intervals taller than the shortest person. Note that all possible values can be anticipated, and this is a continuous distribution.

Now if a million persons were so measured, how would they be distributed? Here the answer is on the basis of naive experience, as follows. First, there would be very few people who would be nearly as short as the shortest or as tall as the tallest. Where would one expect to find most persons? In the middle of the distance, or at some place not too far from it. Where would the next greatest number of persons be found? Close to the biggest. With questions of this sort one ends up describing a well-distributed curve, possibly a normal curve or something near it.

It is proper now to make a small diversion before going on with answering the questions about the issues associated with level of measurement. In particular, it should be noted that there are many sources of error in the measurement that has just been described. First, of course, the age variable is specified with limited accuracy. At the limits, it may be difficult to determine exact age because of the way data are recorded. There are differences implied by the fact that where one is born makes a difference in time, and so on. This may seem facetious, but it illustrates how easily sources of error are bypassed without examination. Then it was noted that there were different estimates of the right location for the point of the shortest and the tallest person as marked on the stick. This is an error of observation and recording, and clearly the points selected are taken as mean values. Who are the persons doing the measuring? Does it make a difference if the person measuring is short or tall? These kinds of errors will exist for all persons measured. Further, it was not specified under what conditions the measurements taken or were to be taken. Are the persons barefoot? How are they asked to stand? Are they asked to relax to a normal position or to try to stretch upward? What time of day is used, since the amount of time after getting up from sleep may have an influence? Is the measurement before or after a meal? And so forth.

The point is that there are many sources of error in taking measures, even direct measures of this sort, and one must be alert to the consequences of these errors on what one does with the data collected. Errors of observation are common, and one aspect of this is the limit of the discriminations an observer can make. One type of error that is usually built into the measurement is rounding error, which is based on the estimated need for accuracy. So, for example, heights are rarely measured more accurately than to the half-inch or centimeter, depending on the standard used. There is still the error of classification up or down, by whatever rule is used for rounding, at the decision point between the intervals used for rounding. Rounding usually follows a consistent arbitrary rule, such as "half adjusting," which means keeping the digit value if the next value in the number is 0 to 4 (e.g., 24.456 = 24) or increasing the value of a digit by 1 if the next value is 5 to 9 (e.g., 24.789 = 25). Another common rounding rule is simply to drop numbers (e.g., 24.456 = 24 and 24.789 = 24). It is important to be aware of which rounding rule is being used and what impact it may have on conclusions drawn when the data collected are used.

The use of distribution-free statistics (often called nonparametric statistics) was popularized beginning in the mid-1950s, and quickly came to be erroneously associated with the notion that most of the measurement in the social and behavioral sciences is of an ordinal nature. Actually, the use of the distribution-free statistics was given impetus because some tests, such as the sign test, did not require use of all the information available to do a statistical test of significance of differences. Thus, instead of using a test of differences of means, one could quickly convert the data to plus and minus scores, using some arbitrary rule, and do a "quick-and-dirty" sign test. Then, if one found significant differences, the more refined test could be carried out at one's leisure. Some of the orientation was related to computing time available, which meant time at a mechanical calculator. Similarly, it was well known that if one used a Spearman rank correlation with larger samples, and if one were interested in measuring statistical significance, one would have to make the same assumptions as for the Pearson product moment correlation, but with less efficiency.

However, this early observation about distribution-free statistics suggests that measures can be thought of in another way. Namely, one can think of measures in terms of how much they are degraded (or imperfect) interval measures. This leads to two questions that are proper to consider. First, what kind of measure is implied as appropriate by the concept? And second, how much error is there in how the measure is constructed if one wants to use procedures that imply interval measurement, including addition, subtraction, multiplication, and division?

What kind of measure is implied by the concept? One way of answering this is to go through the following procedure. As an example, consider a personal attribute, such as aggressiveness. Is it possible to conceive of the existence of a least aggressive person and a most aggressive person? Obviously, whether or not such persons can be located, they can be conceived of. Then, is there any reason to think that persons cannot have any and all possible quantities of aggressiveness between the least and the most aggressive persons? Of course not. Thus, what has been described is a continuous distribution, and with the application of a standard unit, it is appropriately an interval scale. It is improper to think of this variable as intrinsically one that is ordinal because it is continuous. In fact, it is difficult to think of even plausible examples ofvariables that are intrinsically ordinal. As Kendall puts it, "the essence of ranking is that the objects shall be orderable, and the totality of values of a continuous variate cannot be ordered in this sense. They can be regarded as constituting a range of values, but between any two different values there is always another value, so that we cannot number them as would be required for ranking purposes" (1948, p. 105). While a few comments were published that attempted to clarify these issues of measurement in the 1960s (Borgatta 1968), most methodologists accepted the mystique of ordinal measurement uncritically.

Often measures of a concept tend to be simple questions with ordered response categories. These do not correspond to ordinal measures in the sense of ordering persons or objects into ranks, but the responses to such questions have been asserted to be ordinal level measurement because of the lack of information about the intervals. So, for example, suppose one is attempting to measure aggressiveness using a question such as "When you are in a group, how much of the time do you try to get your way about what kinds of activities the group should do next?" Answer categories are "never," "rarely," "sometimes," "often," "very often," and "always." Why don't these categories form an interval scale? The incorrect answer usually given is "because if one assumes an interval scale, one doesn't know where the answer categories intersect the interval scale." However, this does not create an ordinal scale. It creates an interval scale with unknown error with regard to the spacing of the intervals created by the categories.

Thus, attention is now focused on the second question: How much error is involved in creating interval scales? This question can be answered in several ways. A positive way of answering is by asking how much difference it makes to distort an interval scale. For example, if normally distributed variables (which are assumed as the basis for statistical inference) are transformed to flat distributions, such as percentiles, how much impact does this have on statistical operations that are carried out? The answer is "very little." This property of not affecting results of statistical operations has been called robustness. Suppose a more gross set of transformations is carried out, such as deciles. How much impact does this have on statistical operations? The answer is "not much." However, when the transformations are to even grosser categories, such as quintiles, quartiles, thirds, or halves, the answer is that because one is throwing away even more information by grouping into fewer categories, the impact is progressively greater. What has been suggested in this example has involved two aspects: the transformation of the shape of the distribution, and the loss of discrimination (or information) by use of progressively fewer categories.

CONSEQUENCES OF USING LESS THAN NORMALLY DISTRIBUTED VARIABLES

If one has normally distributed variables, the distribution can be divided into categories. The interval units usually of interest with normally distributed variables are technically identified as standard deviation units, but other units can be used. When normally distributed variables are reduced to a small number of (gross) categories, substantial loss of discrimination or information occurs. This can be illustrated by doing a systematic exercise, the results of which are reported in Table 1. The data that are used for the exercise are generated from theoretical distributions of random normal variables with a mean of 0 and a standard deviation of 1. In the exercise, one aspect is examining the relationship among normally distributed variables, but the major part of the exercise involves the data in ordered categorical form, much as it is encountered in "real" data. The exercise permits reviewing several aspects associated with knowledge about one's measurement.

As noted, the exercise is based on unit normal variables (mean = 0, standard deviation = 1) that are sampled and thus are subject to the errors of random sampling that one encounters with "real" data. The theoretical underlying model is one that is recommended in practice, specified as follows:

1. A criterion variable is to be predicted, that is, the relationships of some independent (predictor) variables are to be assessed with regard to the criterion variable.
2. For each sample in which the relationship of the independent variables is assessed, the theoretical underlying relationship of the independent variables is specified, and thus for the purposes of the exercise is known. For the exercise, four levels of underlying theoretical relationship are product

   Table 1

   median correlation of scores with criterion variable, range of 9 samples (n=150) of unit normal deviates, additive scores based on four items with theoretical correlations of .8, .6, .4, and .2 with the criterion variable in the population
   	xo	xor	xoa	xoar	xob	xobr	xo2	xoa2	xob2
   sumx.8	.94	.93–.96	.75	.69–.78	.66	.52–.73	.88	.56	.44
   suma.8	.92	.90–.94	.76	.74–.80	.62	.56–.67	.85	.58	.38
   sumb.8	.89	.87–.92	.68	.63–.74	.62	.58–.68	.78	.46	.38
   sumc.8	.85	.81–.86	.78	.69–.84	.51	.46–.59	.72	.61	.36
   sumdl.8	.72	.64–.74	.54	.48–.60	.72	.64–.77	.52	.29	.52
   sumdr.8	.71	.64–.76	.55	.49–.58	.26	.23–.30	.50	.30	.07
   sumx.6	.84	.78–.88	.65	.56–.75	.57	.49–.71	.71	.42	.32
   suma.6	.82	.77–.85	.66	.60–.71	.58	.48–.62	.67	.44	.34
   sumb.6	.79	.70–.81	.62	.55–.65	.51	.43–.60	.62	.38	.26
   sumc.6	.74	.70–.79	.65	.56–.70	.48	.40–.54	.55	.42	.23
   sumdl.6	.62	.56–.69	.53	.45–.58	.56	.45–.67	.38	.28	.31
   sumdr . 6	.62	.57–.67	.52	.45–.56	.32	.23–.36	.38	.27	.10
   sumx.4	.65	.59–.71	.53	.44–.57	.48	.31–.54	.42	.28	.23
   suma.4	.66	.51–.72	.52	.47–.60	.42	.27–.55	.44	.27	.18
   sumb.4	.61	.48–.67	.47	.37–.55	.41	.33–.46	.37	.22	.17
   sumc.4	.59	.49–.66	.48	.39–.58	.38	.27–.43	.35	.23	.14
   sumdl.4	.45	.32–.52	.39	.30–.47	.33	.15–.51	.20	.15	.11
   sumdr.4	.50	.45–.59	.43	.38–.49	.26	.22–.27	.25	.18	.07
   sumx.2	.39	.25–.51	.31	.20–.41	.25	.12–.45	.15	.10	.06
   suma.2	.31	.26–.42	.23	.16–.34	.27	.22–.31	.10	.05	.07
   sumb.2	.32	.20–.49	.29	.12–.29	.23	.10–.29	.10	.08	.05
   sumc.2	.28	.23–.42	.22	.15–.40	.17	.10–.24	.08	.05	.03
   sumdl.2	.29	.17–.35	.16	.12–.24	.22	.16–.35	.08	.03	.05
   sumdr.2	.24	.19–.37	.20	.04–.31	.17	.07–.30	.06	.04	.03

   
   moment correlation coefficients of .8, .6, .4, and .2, between the predictor variables and the criterion variable. This represents different levels of relationship corresponding to magnitudes commonly encountered with "real" social and behavioral data.
3. For each sample that is used in the exercise, totally independent distributions are drawn from the theoretical distributions. That is, each sample in the exercise is created independently of all other samples, and in each case the criterion and independent predictor variables are drawn independently within the definition of the theoretical distributions.
4. Corresponding to a common model of prediction, assume that the independent variables are of a type that can be used to create scores. Here we will follow a common rule of thumb and use four independent variables to create a simple additive score; that is, the values of the four independent variables are simply added together. There are many ways to create scores, but this procedure has many virtues, including simplicity, and it permits examining some of the consequences of using scores to measure a concept.
5. The independent variables are modified to correspond to grouped or categorical data. This is done before adding the variables together to create scores, as this would be the way variables encountered in research with "real" data are usually defined and used. The grouped or categorical independent variables are modified in the following ways:

A. Four groups or categories are created by using three dividing points in the theoretical distribution, ?1 standard deviation, the mean of 0, and +1 standard deviation. The values for four variables, now grouped or categorical data, are added to give the score used in the correlation with the criterion variable. Thus, looking at Table 1, the row SumA.8 involves samples in which four independent variables based on data having four categories as defined above, and having an underlying theoretical correlation coefficient for each independent variable to the criterion variable of .8, have been used to create a score. The variables defined by these cutting points correspond to a notion of variables with response categories, say, such as "agree," "probably agree," "probably disagree," and "disagree," and as it happens the responses theoretically are well distributed, roughly 16 percent, 34 percent, 34 percent, and 16 percent.

B. Three groups or categories are created by using two dividing points in the theoretical distribution, -1 and +1, corresponding to a notion of variables with response categories, say, such as "agree," "don't know or neutral," and "disagree," and these are again theoretically well distributed, but in this case the center category is large (about 68 percent ofresponses). The scores for these samples are identified as SumB.8 for the case of the underlying theoretical distribution of the independent variables having a correlation coefficient with the criterion variable of .8.

C. Two groups or categories are created by using one dividing point at the mean of the theoretical distribution, and this corresponds to variables with response categories such as "agree" and "disagree," or "yes" and "no." Division of a variable at the midpoint of the distribution is usually considered statistically to be the most efficient for dichotomous response categories.

DL. Two groups or categories may be created that are not symmetrical, unlike the case of C above. Here we create two sets of samples, one identified by Sum DL with the cutting point at -1 standard deviation, or the left side of the distribution as it is usually represented. Presumably, when variables are not well distributed, more information is lost in the data collection if the skew is due to the way the question is formulated.

DR. Two groups or categories are created with the skew on the right side of the distribution at +1 standard deviation. SumDR scores are created in parallel to the SumDL scores.

Finally, the rows of SumX are those where scores are computed and no conversion of the variables has been carried out, so the independent variables are drawn from the theoretically formulated unit normal distributions at the level of correlation coefficient between the independent variables and the criterion variable.

In summary, for each sample the predictor score is an additive score based on four variables, each related to the criterion variable in a theoretical distribution at a given level, with the examples in the exercise the levels chosen as product moment correlation coefficients of .8, .6, .4, and .2.

The rationale for the choice of four variables in each score is drawn arbitrarily from the theoretical distribution of reliability coefficients, which, for variables of equal reliability, is a curve of diminishing return with each increment of the number of variables. At four variables a score is usually considered to be at an efficient point, balancing such things as time and effort needed to collect data, independence of variable definitions, and the amount of improvement of reliability expected. Using more variables in the "real" world usually involves using less "good" items progressively from the point of view of content, as well as the diminishing efficiency even if they were equally as good as the first four.

With regard to the criterion variable, the dependent variable to be predicted, we have generated three as follows: First, the variable is drawn from a theoretical unit normal distribution without modification, and this involves correlation coefficients reported in column XO. Second, the dependent variable is drawn from the theoretical distribution dichotomized at the mean of 0, creating a symmetric dichotomous variable (two categories), and this involves correlation coefficients reported in column XOA. Third, the variable is drawn from the theoretical distribution dichotomized at -1 standard deviation, creating an asymmetrical dichotomous variable and this involves correlation coefficients reported in column XOB.

As noted above, we are dealing with generated data from theoretical distributions, so all that is done is subject to sampling error, which permits in a limited way for variation of results to be shown. This is useful in illustrating the variation that occurs simply by random processes in the study of variables. For the exercise there were 648 samples of 150 cases each. The choice of 150 cases was selected for several reasons, convenience in the data set generated being one, but also because 150 cases was, as a rule of thumb, a reasonable number of cases proposed for doing research at an earlier time in the discipline when a Guttman scale was anticipated, and too much "shrinkage" in the subsequent use of the Guttman scale was to be avoided. Shrinkage referred to the experience of finding out that the scale in a subsequent use did not have the same characteristics as in the first (generating) use. The samples are grouped in the six procedures, as noted above, of the full normal data and of five created sets of categories, each with nine repetitions (i.e., samples) carried out to permit examination of distribution of results for each theoretical level of correlation coefficient and for each of the three criterion variable definitions.

A sense of the stability of the relationships involved is provided by using many samples, and the values that are in columns XO, XOA, and XOB are the median product moment correlation coefficients between a Sum score and a criterion variable. In the table there are two other sets of columns. XOR, XOAR, and XOBR are the actual range of values for the product moment correlation coefficients between a Sum score and the criterion variable for nine independently drawn samples of 150 cases each. The additional (last three columns) at the right of the table are the squared values of XO, XOA, and XOB, and these values represent the amount of variance that is accounted for by the correlation coefficient.

The table is somewhat complex because it includes a great deal of information. Here we will only review a number of relatively obvious points, but with emphasis. First, note the relationship between SumX.8 and XO, which is .94. Recall that the theoretical correlation coefficient between each independent predictor variable and the criterion variable was defined as .8 in the exercise. Thus, we illustrate rather dramatically the importance of building scores rather than depending on a single predictor variable. The actual improvement in prediction is from a theoretical value of 64 percent of the variance being accounted for by a single predictor variable to the median value of 88 percent of the variance in the median sample of the exercise, an extremely impressive improvement! Examining the correlation coefficient between SumX.6, SumX.4, and SumX.2, it is seen that the advantage of using scores rather than single items is actually even more dramatic when the relationship between the criterion and the predictor variables is weaker.

Now, examine the relationships between all the Sum variables and XO. It is noted that having the SumA.8 as a well distributed categorical variable lowers the correlation to XO somewhat (.92) compared to the full normal SumX.8 (.94), and, indeed, with SumB.8 and SumC.8 the correlation coefficients are still lower. This illustrates that the more information that is available in the categorical data, the more variance can be predicted for a criterion variable. Note further, that for SumDL.8 and SumDR.8, which are based on dichotomous but skewed variables, the correlation coefficients are still lower. Note that the results for the weaker variables SumX.6 to SumDR.6, for SumX.4 to SumDR.4, and for SumX.2 to SumDR.2 are in general roughly parallel to the results noted, but some irregularity is noted when the relationship between the criterion and the predictor variables is weaker.

The results we have been examining are the median values of nine samples in each case. It is appropriate to look at column XOR and to note that there is quite a bit of variation in the correlation coefficients based on the nine samples of size 150 cases each.

One additional finding needs to be pointed out and emphasized. Compare the correlation coefficients of SumDT.8 and XOB, and of SumDR.8 and XOB. In the former case, the criterion variable and predictor variables are both asymmetric but matched (at the same cutting point), and the median correlation coefficient value is .72. In the latter case they are asymmetric but unmatched, and the median correlation coefficient is .26. This indicates the need to know how cutting points (difficulty levels) of variables can affect the values of correlation coefficients that are encountered.

The reader is urged to examine this table in detail, and it may be useful to keep the table as a guideline for interpreting results. How? If the researcher finds as a result a correlation coefficient of a given size, where does it fit on this table, taking into consideration the characteristics of the criterion variable and the scores based on the predictor variables? Doing this type of examination should help the researcher understand the meaning of a finding. However, this is but one approach to the issues of measurement. The articles in this encyclopedia that are vital for researchers to consider are those on Reliability, Validity, and Quasi-Experimental Research Design. Additionally, measurement issues are discussed in a variety of types of texts, ranging from specialized texts to books on research methods to general statistics books, such as the following: Agresti and Finaly 1997; Babbie 1995; De Vellis 1991; Knoke and Bohrnstedt 1994; Lewis-Beck 1994; Neuman, 1997; and Traub 1994.

OTHER MEASURES

The consideration of measurement thus far has concentrated on interval measurement, with some emphasis on how it can be degraded. There are many other issues that are appropriately considered, including the fact that the concepts advanced by Stevens do not include all types of measures. As the discussion proceeds to some of these, attention should also be given to the notion of nominal scales. Nominal scales can be constructed in many ways, and only a few will be noted here. By way of example, it is possible to create a classification of something being present for objects that is then given a label A. Sometimes a second category is not defined, and then the second category is the default, the thing not being present. Or two categories can be defined, such as male and female. Note that the latter example can also be defined as male and not male, or as female and not female. More complex classifications are illustrated by geographical regions, such as north, west, east, and south, which are arbitrary and follow the pattern of compass directions. Such classifications, to be more meaningful, are quickly refined to reflect more homogeneity in the categories, and sets of categories develop such as Northeast, Middle Atlantic, South, North Central, Southwest, Mountain, Northwest, and Pacific, presumably with the intention of being inclusive (exhaustive) of the total area. These are complex categories that differ with regard to many variables, and so they are not easily ordered.

However, each such set of categories for a nominal scale can be reduced to dichotomies, such as South versus "not South"; these variables are commonly called "dummy variables." This permits analysis of the dummy variable as though it represented a well-distributed variable. In this case, for example, one could think of the arbitrary underlying variable as being "southernness," or whatever underlies the conceptualization of the "South" as being different from the rest of the regions. Similarly, returning to the male versus female variable, the researcher has to consider interpretatively what the variable is supposed to represent. Is it distinctly supposed to be a measure of the two biological categories, or is it supposed to represent the social and cultural distinction that underlies them?

Many, if not most, of the variables that are of interest to social and behavioral science are drawn from the common language, and when these are used analytically, many problems or ambiguities become evident. For example, the use of counts is common in demography, and many of the measures that are familiar are accepted with ease. However, as common a concept as city size is not without problems. A city is a legal definition, and so what is a city in one case may be quite different from a city in another case. For example, some cities are only central locations surrounded by many satellite urban centers and suburbs that are also defined as cities, while other cities may be made up of a major central location and many other satellite urban centers and suburbs. To clarify such circumstances, the demographers may develop other concepts, like Standard Metropolitan Areas (SMAs), but this does not solve the problem completely; some SMAs may be isolated, and others may be contiguous. And when is a city a city? Is the definition one that begins with the smallest city with a population of 2,500 (not even a geographical characteristic), or 10,000 population, or 25,000 population? Is city size really a concept that is to be measured by population numbers or area, or by some concept of degree of urban centralization? Is New York City really one city or several? Or is New York City only part of one city that includes the urban complexes around it? The point that is critical is that definitions have to be fixed in arbitrary ways when concepts are drawn practically from the common language, and social concepts are not necessarily parsimoniously defined by some ideal rules of formulating scientific theory. Pragmatic considerations frequently intervene in how data are collected and what data become available for use. An additional point is appropriate here: that in demography and in other substantive areas, important measures include counts of discrete entities, and these types of measures do not easily fit the Stevens classification of levels of measurement. A discussion of several technical proposals for more exhaustive classifications of types of measures is considered by Duncan (1984).

CONSTRUCTING MEASURES

There are obviously many ways that measures can be constructed. Some have been formalized and diffused, such as Louis Guttman's cumulative scale analysis, so popular that it has come to be known universally as Guttman scaling, a methodological contribution that was associated with a sociologist and had an appeal for many. An early comprehensive coverage of Guttman scaling can be found in Riley and colleagues (1954). The essence of Guttman scaling is that if a series of dichotomous items is assumed to be drawn from the same universe of content, and if they differ in difficulty, then they can be ordered so that they define scale types. For example, if one examines height, questions could be the following: (1) Are you at least five feet tall? (2) Are you at least five and a half feet tall? (3) Are you at least six feet tall? (4) Are you at least six and a half feet tall? Responses to these would logically fall into the following types: a + to indicate yes and a - to indicate a no:

The types represent "perfect" types, that is, responses made without a logical error. The assumption is that within types, people are equivalent. In the actual application of the procedure, some problems are evident, possibly the most obvious being that there are errors because in applying the procedure in studies, content is not as well specified as being in a "universe" as is the example of height; thus there are errors, and therefore error types. The error types were considered of two kinds: unambiguous, such as - + + +, which in the example above would simply be illogical, a mistake, and could logically be classed as + + + + with "minimum error." The second kind is ambiguous, such as + + - +, which with one (minimum) error could be placed in either type + + - - or type + + + +.

Experience with Guttman scaling revealed a number of problems. First, few scales that appeared "good" could be constructed with more than four or five items because the amount of error with more items would be large. Second, the error would tend to be concentrated in the ambiguous error type. Third, scales constructed on a particular study, especially with common sample sizes of about one hundred cases, would not be as "good" in other studies. There was "shrinkage," or more error, particularly for the more extreme items. The issue of what to do with the placement of ambiguous items was suggested by an alternative analysis (Borgatta and Hays 1952): that the type + + - + was not best placed by minimum error, but should be included with the type + + + - between the two minimum error locations. The reason for this may be grasped intuitively by noting that when two items are close to each other in proportion of positive responses, they are effectively interchangeable, and they are involved in the creation of the ambiguous error type. The common error is for respondents who are at the threshold of decision as to whether to answer positively or negatively, and they are most likely to make errors that create the ambiguous error types.

These observations about Guttman scaling lead to some obvious conclusions. First, the scaling model is actually contrary to common experience, as people are not classed in ordered types in general but presumably are infinitely differentiated even within a type. Second, the model is not productive of highly discriminating classes. Third, and this is possibly the pragmatic reason for doubting the utility of Guttman scaling, if the most appropriate place for locating nonscale or error types of the common ambiguous type is not by minimum error but between the two minimum error types, this is effectively the same as adding the number of positive responses, essentially reducing the procedure to a simple additive score. The remaining virtue of Guttman scaling in the logical placement of unambiguous errors must be balanced against other limitations, such as the requirement that items must be dichotomous, when much more information can be gotten with more detailed categories of response, usually with trivial additional cost in data collection time.

In contrast with Guttman scaling, simple addition of items into sum scores, carried out with an understanding of what is required for good measurement, is probably the most defensible and useful tool. For example, if something is to be measured, and there appear to be a number of relatively independent questions that can be used to ascertain the content, then those questions should be used to develop reliable measures. Reliability is measured in many ways, but consistency is the meaning usually intended, particularly internal consistency of the component items, that is, high intercorrelation among the items in a measure. Items can ask for dichotomous answers, but people can make more refined discriminations than simple yeses and nos, so use of multiple (ordered) categories of response increases the efficiency of items.

The question of whether the language as used has sufficient consistency to make refined quantitative discriminations does not appear to have been studied extensively, so a small data collection was carried out to provide the following example. People were asked to evaluate a set of categories with the question "How often does this happen?" The instructions stated: "Put a vertical intersection where you think each category fits on the continuum, and then place the number under it. Categories 1 and 11 are fixed at the extremes for this example. If two categories have the same place, put the numbers one on top of the other. If the categories are out of order, put them in the order you think correct." A continuum was then provided with sixty-six spaces and the external positions of the first and the last indicated as the positions of (1) always and (11) never. The respondents were asked to locate the following remaining nine categories: (2) almost always (3) very often; (4) often; (5) somewhat often; (6) sometimes; (7) seldom; (8) very seldom; (9) hardly ever; and (10) almost never. It is not surprising that average responses on the continuum are well distributed, with percent locations respectively as 9, 15, 27, 36, 48, 65, 75, 86, and 93; the largest standard deviation for placement location is about 11 percent. Exercises with alternative quantitatively oriented questions and use of a series of six categories from "definitely agree" to "definitely disagree" provide similar evidence of consistency of meaning. In research, fewer than the eleven categories illustrated here usually used, making the task of discrimination easier and faster for respondents. The point of emphasis is that questions can be designed to efficiently provide more information than simple dichotomous answers and thus facilitate construction of reliable scores.

MEASUREMENT IN THE REAL WORLD

Many variations exist on how to collect information in order to build effective measurement instruments. Similarly, there are alternatives on how to build the measuring instruments. Often practical considerations must be taken into account, such as the amount of time available for interviews, restraints placed on what kinds of content can be requested, lack of privacy when collecting the information, and other circumstances.

With the progressive technology of computers and word processors, the reduced dependence of researchers on assistants, clerks, and secretaries has greatly facilitated research data handling and analysis. Some changes, like Computer Assisted Telephone Interviewing (CATI) may be seen as assisting data collection, but in general the data collection aspects of research are still those that require most careful attention and supervision. The design of research, however, still is often an ad hoc procedure with regard to the definition of variables. Variables are often created under the primitive assumption that all one needs to do is say, "The way I am going to measure XXX is by responses to the following question." This is a procedure of dubious worth, since building knowledge about the measurement characteristics of the variables to be used should be in advance of the research, and is essential to the interpretation of findings.

A comment that is commonly encountered is that attempting to be "scientific" and developing a strict design for research with well-developed measures forecloses the possibility of getting a broad picture of what is being observed. The argument is then advanced that attempting to observe and accumulate data in systematic research is not as revealing as observing more informally (qualitatively) and "getting a feel" for what is going on. Further, when systematic research is carried out, so goes the argument, only limited variables can be assessed instead of "getting the complete picture." This, of course, is the ultimate self-delusion and can be answered directly. If positive findings for the theory do not result from more rigorous, well-designed research, then the speculative generalizations of more casual observation are never going to be any more than that, and giving them worth may be equivalent to creating fictions to substitute for reality. The fact that attempted systematic empirical research has not produced useful findings does not mean that more intuitive or qualitative approaches are more appropriate. What it means is that the theory may not be appropriate, or the design of the research may be less than adequate.

Further, this does not mean that there is anything wrong with informal or qualitative research. What it does mean is that there is a priority order in the accumulation of knowledge that says that the informal and qualitative stages may be appropriate to produce theory, which is defined as speculation, about what is being observed, and this may then be tested in more rigorous research. This is the common order of things in the accumulation of scientific knowledge.

If a sociological theory has developed, then it must be stated with a clear specification of the variables involved. One cannot produce a volume on the concept of anomie, for example, and then use the word "anomie" to mean twenty different things. The concept on which one focuses must be stated with a clear specification of one meaning, and there are two elements that go into such a definition. The first is to indicate how the concept is to be measured. The second is more commonly neglected, and that is to specify how the concept is differentiated from other concepts, particularly those that are closely related to it in meaning.

The development of well-measured variables in sociology and the social sciences is essential to the advancement of knowledge. Knowledge about how good measurement can be carried out has advanced, particularly in the post–World War II period, but it has not diffused and become sufficiently commonplace in the social science disciplines.

It is difficult to comprehend how substituting no measurement or poor measurement for the best measurement that sociologists can devise can produce better or more accurate knowledge. Examples of the untenable position have possibly decreased over time, but they still occur. Note for example: "Focus on quantitative methods rewards reliable (i.e., repeatable) methods. Reliability is a valuable asset, but it is only one facet of the value of the study. In most studies, reliability is purchased at the price of lessened attention to theory, validity, relevance. etc." (Scheff 1991). Quite the contrary, concern with measurement and quantification is concern with theory, validity, and relevance!

Finally, it is worth emphasizing two rules of thumb for sociologists concerned with research, whether they are at the point of designing research or interpreting the findings of a research that has been reported. First, check on how the variables are specified and ask whether they are measured well. This requires that specific questions be answered: Are the variables reliable? How does one know they are reliable? Second, are the variables valid? That is, do they measure what they are supposed to measure? How does one know they do? If these questions are not answered satisfactorily, then one is dealing with research and knowledge of dubious value.

(see also: Levels of Analysis: Nonparametric Statistics; Reliability, Validity; Quasi-Experimental Research Design)

references

Agresti, Alan, and Barbara Finaly 1997 Statistical Methods for Social Sciences. Englewood Cliffs, N. J.: Prentice Hall.

Babbie, Earl 1995 The Practice of Social Research. Belmont, Calif.: Wadsworth.

Borgatta, Edgar F. 1968 "My Student, the Purist: A Lament." Sociological Quarterly 9:29–34.

——, and David G. Hays 1952 "Some Limitations on the Arbitrary Classifications of Non-Scale Response Patterns in a Guttman Scale." Public Opinion Quarterly 16:273–291.

De Vellis, Robert F. 1991 Scale Development. Newbury Park, Calif.: Sage.

Duncan, Otis Dudley 1984 Notes of Social Measurement. New York: Russell Sage Foundation.

Herzog, Thomas 1997 Research Methods and Data Analysis in the Social Sciences. Englewood Cliffs, N.J.: Prentice Hall.

Kendall, Maurice G. 1948 Rank Correlation Methods. London: Griffin.

Knoke, David, and George W. Bohrnstedt 1994. Statistics for Social Data Analysis. Itasca, Ill.: Peacock.

Lewis-Beck, Michael, ed. 1994 Basic Measurement. Beverly Hills, Calif.: Sage.

Neuman, Lawrence W. 1997 Social Research Methods: Qualitative and Quantitative Approaches. Boston: Allyn and Bacon.

Nunnally, Jum C. 1978 Psychometric Theory. New York: McGraw-Hill.

Riley, Matilda White, John W. Riley, Jr., and Jackson Toby 1954 Sociological Studies in Scale Analysis. New Brunswick, N.J.: Rutgers University Press.

Scheff, Thomas J. 1991 "Is There a Bias in ASR Article Selection." Footnotes 19(2, February):5.

Stevens, S. S., ed. 1966 Handbook of Experimental Psychology. New York: John Wiley.

Traub, Ross E. 1994 Reliability for the Social Sciences. Beverly Hills, Calif.: Sage.

Edgar F. Borgatta

Yoshinori Kamo