부산대학교 도서관

Measurement errors attributable to cultural issues are complex and challenging for educational assessments. We need assessment tests sensitive to the cultural heterogeneity of populations, and psychometric methods appropriate to address fairness and equity concerns. Built on the research of culturally responsive assessment, this dissertation explores the conceptual, analytical, and methodological aspects of culturally responsive and equitable testing. To that end, three coordinated studies were conducted to gather validity evidence based on cultural characteristics of contextualized items, students, and multigroup fairness from empirical and simulated datasets. The first study contributed to the understanding of culturally valid and equitable contextualized items, by implementing a coding rubric of item contexts followed by an illustrative analysis of science assessment items. The rubric of fifteen attributes was developed to represent context domains of cultural equity, context representation, and knowledge construction. The mixed methods, including item response theory, clustering profiles, and discourse analysis were conducted to explore context codes and student responses, along with qualitative analysis of item contexts. The results found various associations of contextual characteristics such as cultural bias, linguistic bias, and others with item difficulty and differential item functioning (DIF) parameters. The second study implemented a Bayesian hierarchical explanatory item response model on a physics contextualized assessment. It examined the statistical relationships between the sociocognitive and sociocultural aspects of item contexts and student performance. Context features were systematically and iteratively coded, modified, and analyzed based on their feature importance. Results showed the statistical associations of context codes, including sociolinguistic familiarity, length of context, cognitive demand, and sociocultural bias on students' success probability given their latent proficiency. Conditional context effects accounting for students' racial backgrounds were also discussed. The third study considered context features as item attributes in general and explored the methodological issues of DIF in cognitive diagnostic computerized adaptive testing (CD-CAT) through a simulation study. Results revealed the performance of uniform and nonuniform DIF detection using different item selection algorithms, DIF-contaminated banks, and other various simulated conditions of the CD-CAT. Classification accuracy, exposure rate, and overlap rate were also explored. In summary, this dissertation demonstrated the significance and relevance of fair, culturally valid, responsive, individualized, and adaptive items and algorithms in educational measurement and testing. Practical and methodological implications were discussed for test and item development, validity, science knowledge, and fairness. [The dissertation citations contained here are published with the permission of ProQuest LLC. Further reproduction is prohibited without permission. Copies of dissertations may be obtained by Telephone (800) 1-800-521-0600. Web page: http://www.proquest.com/en-US/products/dissertations/individuals.shtml.]