Validity

Validity may be defined as the extent to which one measures what one wants to measure. Measurement validation refers not to the properties of a particular instrument but rather to the conclusions made as a result of its use. In other words, although we speak of a valid instrument, what we really mean is an instrument from which we can make valid inferences.

In general, we are concerned with two types of validity: criterion related validity and construct validity. In the former, we examine the relationship between the instrument and the criterion of interest; for example, an instrument's ability to predict physicians' likelihood to prescribe a certain drug, using a set of the physicians' characteristics and attitudes. In construct validity, we relate the items in the instrument to the constructs that we are attempting to measure.

Criterion related validity is usually demonstrated by measures of association, such as chi-square statistics or correlation and regression coefficients. Construct validity is usually demonstrated with factor analysis.

Reliability

Reliability may be defined as the extent to which a measurement process or instrument consistently measures factors regardless of external conditions such as time or setting. For example, an automobile may be said to be reliable if it consistently starts and performs to expectations in varying weather conditions over a reasonable period of time. Similarly, a measurement instrument may be said to be reliable if it yields the same results in different conditions of administration, at different times, and in different places.

Reliable measurement is relatively free from the extraneous influences of administration context. The goal of a reliability study is to amass evidence demonstrating the reliability of a measurement instrument.

Our approach to a reliability study would typically involve test-retest reliability and internal consistency statistics. Test-retest reliability (usually indexed by a correlation coefficient) indicates the degree to which an instrument is responded to similarly by the same respondents at different times. Internal consistency (usually indexed by Cronbach's alpha) indicates the degree to which the items in a measurement instrument correlate with one another.