Job Analysis Ratings and Criterion-Related Validity: Are They Related and Can Validity Be Used as a Measure of Accuracy?

Simple item page
dc.contributor.ORCID0000-0002-1547-5469 (Weekley, JA)
dc.contributor.authorWeekley, Jeff A.
dc.contributor.authorLabrador, J. R.
dc.contributor.authorCampion, M. A.
dc.contributor.authorFrye, K.
dc.contributor.utdAuthorWeekley, Jeff A.
dc.date.accessioned2020-01-29T20:50:31Z
dc.date.available2020-01-29T20:50:31Z
dc.date.issued2019-04-29
dc.descriptionDue to copyright restrictions full text access from Treasures at UT Dallas is restricted to current UTD affiliates (use the provided Link to Article).
dc.description.abstractJob analysis data are largely judgements from subject matter experts (SMEs), judgements with unknown accuracy. To date, accuracy has been inferred largely based on inter-rater reliability or agreement between SMEs and without reference to an external criterion. The current research examined job analysis rating accuracy by comparing SME importance ratings of knowledge, skills, abilities, and other requirements (KSAOs) with the validity of measures of these same KSAOs in predicting job performance. We tested hypotheses about whether SME judgement accuracy is moderated by SME job tenure, industry experience, role, self-reported knowledge of the job, and data scrubbing. Four independent tests involving 48 separate validation studies were conducted. In three of the four samples, there was a large (r =.50 range) relationship between trait importance and trait validity, showing that job analysis ratings can be directly related to test validities and serve as a measure of job analysis accuracy. Moderator analyses showed that the best results may come from supervisors, rather than incumbents, and those who know the job extremely well (there were no differences due to SME job tenure, industry experience, or deletion of outliers). Showing a direct relationship between SME judgements and actual criterion-related validity provides a new lens for operationalizing accuracy in job analysis research. Practitioner Points: This study demonstrates that test validities can serve as a measure of accuracy, providing a new avenue for job analysis research. The most accurate job analysis ratings came from supervisors and those who reported knowing the job extremely well. ©2019 The British Psychological Society
dc.description.departmentNaveen Jindal School of Management
dc.identifier.bibliographicCitationWeekley, J. A., J. R. Labrador, M. A. Campion, and K. Frye. 2019. "Job analysis ratings and criterion-related validity: Are they related and can validity be used as a measure of accuracy?." Journal of Occupational and Organizational Psychology 92(4): 764-786, doi: 10.1111/joop.12272
dc.identifier.issn0963-1798
dc.identifier.issue4
dc.identifier.urihttp://dx.doi.org/10.1111/joop.12272
dc.identifier.urihttps://hdl.handle.net/10735.1/7209
dc.identifier.volume92
dc.language.isoen
dc.publisherJohn Wiley and Sons Ltd.
dc.rights©2019 The British Psychological Society
dc.source.journalJournal of Occupational and Organizational Psychology
dc.subjectJob analysis
dc.subjectCase studies
dc.subjectDecision making
dc.subjectPerformance standards
dc.titleJob Analysis Ratings and Criterion-Related Validity: Are They Related and Can Validity Be Used as a Measure of Accuracy?
dc.type.genrearticle

Files

Original bundle

Now showing 1 - 1 of 1
Thumbnail Image
Name:
JSOM-7170-260764.20-LINK.pdf
Size:
165.77 KB
Format:
Adobe Portable Document Format
Description:
Link to Article
Download

Collections

Weekley, Jeff