|
Title:
|
Analysis of pre-and-post-discussion Angoff ratings for evidence of social influence effects
|
|
Author:
|
Wessen, Catherine
|
|
Abstract:
|
The Angoff (1971) method is a widely used standard-setting approach in test development. It relies on Subject Matter Experts (SMEs) who examine the difficulty level of test items and then give probability estimates on item difficulty (Hurtz & Hertz, 1999). However, expert judgment is subject to social influences that may compromise the accuracy of the desired passing score. Archival data was utilized to investigate how social influence effects might have guided the judgments of SMEs (N = 260) in setting cutoff points for 38 licensure examinations. The results showed significant changes between SME’s’ pre-and-post-discussions standard deviations, indicating that SMEs’ changed their final ratings inward toward the mean of the group. There was no support for group polarization effects. There was support for the minority pressure influence. There was also support for the insufficient anchor and adjustment bias. These results indicated that social influence factors are present in Angoff workshops. |
|
Description:
|
Thesis (M.A., Psychology (Industrial/Organizational Psychology)) -- California State University, Sacramento, 2010. |
|
URI:
|
http://hdl.handle.net/10211.9/618
|
|
Date:
|
2010-08-31 |
