VALIDITY GENERALIZATION MODEL OF SELECTION

VALIDITY GENERALIZATION MODEL OF SELECTION

Primary Disciplinary Field(s): Industrial/Organizational Psychology; Psychometrics

1. Core Definition and Premise

The Validity Generalization (VG) Model of Selection is a comprehensive meta-analytic approach utilized primarily within Industrial/Organizational (I/O) Psychology to determine the degree to which the predictive validity of a specific employment test or predictor generalizes across different organizational settings, job types, and geographical locations. This model fundamentally challenges the older paradigm of situational specificity, which held that a selection test found valid in one company or job context required re-validation—often through costly and time-consuming local validation studies—before being applied elsewhere. Instead, the VG model posits that true differences in validity coefficients across various studies are often minimal once statistical and measurement artifacts are systematically removed.

At its operational core, the VG model involves conducting a meta-analysis of previously completed validation studies for a specific predictor (e.g., cognitive ability tests, personality inventories) against a defined performance criterion. The objective is not merely to calculate the average validity coefficient, but rather to investigate whether the observed variation, or standard deviation, in these coefficients across studies is primarily attributable to extraneous methodological factors rather than genuine differences in the underlying relationship between the predictor and performance. If the majority of the observed variance can be accounted for by these artifacts—such as variations among analyses in criterion gauges, sample sizes, and the spanning of test scores—the remaining, true variance is deemed negligible, leading to the conclusion that the validity is generalizable.

The practical implication of a successful validity generalization analysis is profound: if a predictor’s validity can be shown to generalize, employers are justified in using that predictor in new employment environments without necessarily conducting a new local validation study. This dramatically reduces the burden of proof for test utility and allows organizations to implement effective, scientifically supported selection instruments more quickly and economically. Based upon these proofs, an employer might be correct in concluding that the predictor will be valid in choosing workers in another employment environment, provided the job characteristics align broadly with those previously studied.

2. Historical Context and Development

Prior to the development and widespread adoption of the VG model in the late 1970s and 1980s, the dominant view in personnel psychology was one of strong situational specificity. This belief held that the effectiveness of a selection procedure was highly dependent on unique local factors—such as specific organizational culture, nuances of the job, or specific characteristics of the applicant pool—meaning validity coefficients obtained in one context were assumed to be unreliable when applied to another. This perspective necessitated repetitive, resource-intensive criterion-related validity studies for every new application, a process that was both expensive for businesses and often statistically weak due to small sample sizes inherent in single-organization studies, creating significant impediments to the systematic application of psychometric research findings.

The genesis of the VG approach is largely credited to the pioneering work of Frank L. Schmidt and John E. Hunter. They recognized that the discrepancies observed in validity coefficients across different studies were frequently more reflective of statistical errors and methodological flaws, which they termed artifacts, than genuine variations in the underlying validity. Their crucial insight was that by aggregating results across numerous studies, the true relationship between a predictor and performance could be estimated with high precision, and the “noise” introduced by differential sample sizes, unreliable criterion measures, and other statistical idiosyncrasies could be systematically corrected and removed.

This shift from situational specificity to generalizability represented a major paradigm change in I/O psychology. By demonstrating, often compellingly, that for broad categories of predictors—especially measures of general cognitive ability (GMA)—validity tends to be consistent across a vast array of jobs, Schmidt and Hunter provided the empirical foundation for a more standardized and scientifically rigorous approach to personnel selection. This advancement allowed researchers and practitioners to move beyond treating each validation study as an isolated event and instead view the entire body of literature as contributing to a unified, statistically corrected understanding of predictive relationships across contexts.

3. Methodology: Meta-Analytic Procedures

The methodological backbone of the Validity Generalization Model is meta-analysis, specifically tailored to psychometric data synthesis. The VG procedure begins by collecting all available criterion-related validity studies relevant to the predictor-criterion pairing of interest, such as the relationship between an integrity test score and counterproductive work behavior. For each study, the observed validity coefficient (the correlation, $r$) is recorded, along with crucial statistical and descriptive information necessary for subsequent artifact correction, including the sample size, estimates of reliability for both the predictor and criterion, and data concerning any restrictions in the range of test scores experienced by the sample.

The central statistical task involves calculating the mean observed validity ($bar{r}$) across all included studies and determining the total variance ($text{Var}(r)$) of those observed validities. This observed variance is then statistically decomposed into two primary components: the variance attributable to statistical and methodological artifacts ($text{Var}(A)$), and the remaining variance, which is presumed to be the true variance in validity ($text{Var}(rho)$), representing actual differences in predictor effectiveness across different situations. The fundamental hypothesis of the VG model is that $text{Var}(A)$ accounts for a very large proportion (typically aiming for 75% or more) of the total observed variance $text{Var}(r)$, leaving $text{Var}(rho)$—the true, unexplained variance—to be statistically insignificant or near zero.

The artifact correction process involves a series of sequential and complex statistical adjustments. First, corrections are applied to account for sampling error variance, which is the random fluctuation inherent in small-sample studies. Next, corrections are applied for criterion unreliability and predictor unreliability, adjusting the observed correlation upward to estimate the relationship as if both job performance and the test scores were measured perfectly. Finally, corrections for range restriction are implemented, compensating for the fact that studies on incumbents often underestimate validity because they exclude individuals who were either not hired or failed to persist in the job. Once these corrections are systematically performed, the resulting mean true validity ($bar{rho}$) and the residual true variance ($text{Var}(rho)$) provide the basis for the generalizability conclusion.

4. Key Artifacts and Corrections

One of the most powerful aspects of the VG methodology is its rigorous correction for sampling error variance. Many organizational validation studies, especially those conducted locally, operate with inadequate sample sizes, resulting in high variability in their validity coefficients purely due to chance. The VG model addresses this by using statistical formulas, often weighting studies by their sample size, to calculate the amount of variance expected solely from sampling error. This calculated error variance is then subtracted from the total observed variance, isolating the variance that is due to genuine population differences or other measurement issues.

Another crucial artifact is criterion unreliability. Job performance is a complex construct, and performance measures (such as supervisory ratings, objective output data, or behavioral metrics) rarely achieve perfect reliability. The attenuation caused by this measurement error systematically lowers the observed validity coefficient, making even excellent predictors appear mediocre. The VG method utilizes reliability estimates (e.g., test-retest reliability, inter-rater reliability) to statistically “disattenuate” the observed correlations. By correcting for the error variance in the criterion measure, the estimated correlation reflects the true underlying relationship between the construct being measured by the predictor and the construct of job performance.

A third necessary correction involves range restriction. This artifact arises because validity studies are typically conducted on groups that have already passed some form of selection hurdle, leading to a truncated distribution of predictor scores compared to the general applicant pool. Because correlation coefficients are sensitive to variance, restricting the range of scores artificially depresses the observed validity. The VG model employs specific psychometric equations to estimate what the validity coefficient would be in an unrestricted population, providing a more accurate and higher estimate of the predictor’s true utility when used for actual selection decisions involving a broad pool of candidates.

5. Practical Implications for Personnel Selection

The adoption of the Validity Generalization Model has profoundly impacted organizational decision-making by prioritizing empirically derived utility over localized, potentially flawed validation efforts. By establishing that the validity of certain predictors, particularly measures of general mental ability (GMA), is highly stable across diverse job families, the model drastically reduces the need for organizations to conduct costly, independent validation studies. This enables HR departments to adopt selection procedures with high predictive accuracy based on overwhelming synthesized evidence, leading to rapid implementation and significant cost savings associated with test development and validation.

Furthermore, VG directly enhances the predictive utility and quality of organizational hires. Since VG research overwhelmingly supports GMA as the single best predictor of job performance across nearly all jobs, organizations implementing selection batteries validated through this model benefit from substantially higher correlations between applicant scores and future job success. This improved forecasting capacity translates into better organizational performance, as successful predictors lead to a more competent and productive workforce, while simultaneously reducing indirect costs such as high training requirements and employee turnover.

From a legal and ethical standpoint, the VG model provides robust support for the use of selection instruments. When an organization utilizes a test whose validity has been established through a large-scale, methodologically sound VG study, it possesses powerful statistical evidence demonstrating that the test is a valid, job-related measure of necessary competencies. This strong evidentiary basis helps organizations comply with governmental regulations (such as those enforced by the EEOC in the United States) and provides a defensible position against challenges regarding the fairness or job-relatedness of their selection procedures, provided that the VG study covers the specific job and demographic context of the application.

6. Generalizability and Predictive Power

The conclusion of validity generalization is reached when, after correcting for major statistical artifacts, the amount of variance remaining in the validity coefficients ($text{Var}(rho)$) is statistically insignificant or accounts for only a small percentage of the original variance. If this condition is met, it is concluded that the validity of the predictor is generalized, meaning that the calculated mean true validity ($bar{rho}$) is a stable, accurate estimate of the true correlation between the predictor and the criterion across all settings included in the analysis. This finding permits the widespread, confident application of the predictor.

The strongest and most seminal evidence supporting the efficacy of the VG model relates to the predictive power of General Cognitive Ability. VG studies have demonstrated that GMA tests retain a substantial and consistent level of validity in predicting performance across virtually all job types and organizational contexts, provided the job demands some level of information processing. While the magnitude of the correlation is often moderated by job complexity (higher validity for complex professional roles compared to simple manual labor), the finding that GMA is reliably valid holds true universally, thereby strongly supporting the principle of generalizability in cognitive prediction.

Ultimately, the VG model confirms that the validity discovered in prior validation analyses can be generalized or carried to a new circumstance. This means that a new employer, reviewing the VG evidence for a specific predictor, can confidently conclude that the tool will be valid in selecting employees for their particular environment, eliminating the need for a costly, organization-specific validation study. This scientific confidence in predictive power maximizes organizational return on investment in selection and ensures that decisions are based on the aggregate weight of vast empirical data rather than isolated, potentially biased local observations.

7. Criticisms and Methodological Debates

A persistent criticism leveled against the Validity Generalization Model centers on the handling and interpretation of residual variance. While proponents argue that if statistical artifacts account for 75% or more of the observed variance, the remaining variability is minor and likely attributable to uncorrected, second-order artifacts, critics argue that this remaining variance might still represent genuine, meaningful situational differences or moderating variables that the meta-analytic framework has failed to identify. If the residual variance is indeed due to true moderators (e.g., specific organizational climate or team dynamics), then the conclusion of universal generalizability is overstated, potentially leading to inaccurate predictions in unique contexts.

Further methodological debate focuses on the broad categorization used in many large-scale VG studies, particularly the aggregation of validity coefficients across wide-ranging job families. Critics maintain that while VG holds well for extremely broad constructs like GMA, aggregating data across dissimilar jobs (e.g., technical engineering and basic customer service) may mask important variations in specific competency requirements. If a predictor, such as a specialized mechanical aptitude test, is generalized across all technical jobs, its utility might be inaccurately assessed for niche technical roles where specialized, situation-specific knowledge is paramount, leading to a generalized conclusion that is practically misleading.

Finally, concerns have been raised regarding the precise accuracy and reliability of the artifact correction formulas themselves. Specifically, the corrections for range restriction and criterion unreliability rely on assumptions about the underlying distributions and linearity of relationships that may not be perfectly met in practice. If the estimates used for criterion reliability are themselves unreliable or if the method chosen to correct for indirect range restriction is inappropriate for the specific context, the resulting estimate of the true validity ($bar{rho}$) and the calculation of residual variance ($text{Var}(rho)$) may be systematically skewed, potentially resulting in an unwarranted declaration of validity generalization.

Further Reading

• Schmidt, F. L., & Hunter, J. E. (1977). Development of a general solution to the problem of validity generalization.
• Personnel selection – Wikipedia.
• Society for Industrial and Organizational Psychology (SIOP). Principles for the Validation and Use of Personnel Selection Procedures.
• Hunter, J. E., & Schmidt, F. L. (1998). The problem of artifacts and corrections.