12 References

Accessible Teaching, Learning, and Assessment Systems. (2021). 2020–2021 DLM administration during COVID-19: Participation, performance, and educational experience (Technical Report No. 21-02). University of Kansas.

American Educational Research Association, American Psychological Association, & National Council on Measurement in Education. (2014). Standards for educational and psychological testing. American Educational Research Association.

Centers for Disease Control and Prevention. (2020a). Considerations for school closure. https://www.cdc.gov/coronavirus/2019-ncov/downloads/considerations-for-school-closure.pdf

Centers for Disease Control and Prevention. (2020b). Interim guidance for administrators of US K–12 schools and childcare programs. https://www.cdc.gov/coronavirus/2019-ncov/community/schools-childcare/guidance-for-schools.html

Cicchetti, D. V., & Feinstein, A. R. (1990). High agreement butt low kappa: II. Resolving the paradoxes. Journal of Clinical Epidemiology, 43, 551–558. https://doi.org/10.1016/0895-4356(90)90159-M

Clark, A. K., Thompson, J. W., Kobrin, J., Kavitsky, E., & Karvonen, M. (2021). The Impact of COVID-19: Validity Considerations and Scoring and Reporting in Flexible Scenarios. University of Kansas, Center for Accessible Teaching, Learning, and Assessment Systems.

Clark, A., Karvonen, M., & Wells-Moreaux, S. (2016). Summary of results from the 2014 and 2015 field test administrations of the Dynamic Learning Maps Alternate Assessment System (Technical Report No. 15-04). University of Kansas, Center for Educational Testing and Evaluation.

DLM Consortium. (2021). Test Administration Manual 2020–2021. University of Kansas, Accessible Teaching, Learning, and Assessment Systems.

Dynamic Learning Maps Consortium. (2016a). 2014–2015 Technical Manual—Integrated Model. University of Kansas, Center for Educational Testing and Evaluation.

Dynamic Learning Maps Consortium. (2016b). 2014–2015 Technical Manual—Year-End Model. University of Kansas, Center for Educational Testing and Evaluation.

Dynamic Learning Maps Consortium. (2017a). 2015–2016 Technical Manual Update—Integrated Model. University of Kansas, Center for Educational Testing and Evaluation.

Dynamic Learning Maps Consortium. (2017b). 2016–2017 Technical Manual Update—Integrated Model. University of Kansas, Accessible Teaching, Learning, and Assessment Systems.

Dynamic Learning Maps Consortium. (2018). 2017–2018 Technical Manual Update—Integrated Model. University of Kansas, Accessible Teaching, Learning, and Assessment Systems.

Dynamic Learning Maps Consortium. (2019). 2018–2019 Technical Manual Update—Integrated Model. University of Kansas, Accessible Teaching, Learning, and Assessment Systems.

Dynamic Learning Maps Consortium. (2020). 2019–2020 Technical Manual Update—Instructionally Embedded Model. University of Kansas, Accessible Teaching, Learning, and Assessment Systems.

Dynamic Learning Maps Consortium. (2021a). 2020–2021 Technical Manual Update—Science. University of Kansas, Accessible Teaching, Learning, and Assessment Systems.

Dynamic Learning Maps Consortium. (2021b). Accessibility Manual 2020–2021. University of Kansas, Accessible Teaching, Learning, and Assessment Systems.

Dynamic Learning Maps Consortium. (2021c). Educator Portal User Guide. University of Kansas, Accessible Teaching, Learning, and Assessment Systems.

Elementary and Secondary Education Act of 1965, Pub. L. No. 89–10, 79 Stat. 27 (1965). https://www.govinfo.gov/content/pkg/STATUTE-79/pdf/STATUTE-79-Pg27.pdf

Every Student Succeeds Act, 20 U.S.C § 6301 (2015). https://www.congress.gov/114/plaws/publ95/PLAW-114publ95.pdf

Feinstein, A. R., & Cicchetti, D. V. (1990). High agreement butt low kappa: I. The problems of two paradoxes. Journal of Clinical Epidemiology, 43, 543–549. https://doi.org/10.1016/0895-4356(90)90158-L

O’Leary, S., Lund, M., Ytre-Hauge, T. J., Holm, S. R., Naess, K., Dalland, L. N., & McPhail, S. M. (2014). Pitfalls in the use of kappa when interpreting agreement between multiple raters in reliability studies. Physiotherapy, 100, 27–35. https://doi.org/10.1016/j.physio.2013.08.002

Pontius, R. G., Jr., & Millones, M. (2011). Death to kappa: Birth of quantity disagreement and allocation disagreement for accuracy assessment. International Journal of Remote Sensing, 32, 4407–4429. https://doi.org/10.1080/01431161.2011.552923

Recommended Waiver Authority Under Section 3511(d)(4) of Division A of the Coronavirus Aid, Relief, and Economic Security Act (“CARES ACT”). (2020). https://www2.ed.gov/documents/coronavirus/cares-waiver-report.pdf

Rosenblum, I. (2021). United States Department of Education, Office of Elementary and Secondary Education. https://www2.ed.gov/policy/elsec/guid/stateletters/dcl-assessments-and-acct-022221.pdf

Templin, J., & Bradshaw, L. (2013). Measuring the reliability of diagnostic classification model examinee estimates. Journal of Classification, 30(2), 251–275. https://doi.org/10.1007/s00357-013-9129-4