A discussion event presenting the findings of the TIMSS 2023 reports authored by a UCL IOE Team

24

3/14/2025

In England, vocational has come to have a variety of meanings. My interest in vocational education and training began with my first role at City & Guilds and I have remained fascinated by the emotive nature of the word vocational ever since. This webinar will explore the extent to which those meanings have had a negative impact on the reputation of the qualifications or outcomes to which they relate. It will also explore the way a range of ‘vocational’ qualifications are assessed, how this relates to the purpose of the qualification and the intended destination of the holder, and how this has affected the perceived status of a given qualification.

20

1/23/2025

The presentation will report on a multi-disciplinary project seeking to reconceptualise educational assessment at the approach to the second quarter of the 21st century. Three main lessons have been learned from this review. The first is that context always matters. Secondly, there is a lack of fit between constructs for assessment deemed relevant to the 21st century and traditional assessments. Thirdly, challenge to the traditional caution to the traditional caution and risk-nervousness of the assessment world.

18

1/23/2025

Speakers: Damian Murchan, Stuart Shaw & Evgenia Likhovtseva This study investigates why and how different jurisdictions conceptualise and operationalise external moderation of School Based Assessment (SBA). A range of concerns about high-stakes examinations at upper secondary level have prompted some systems to incorporate SBA into their system of qualifications. While potentially addressing issues of validity and student stress, SBA raises reliability concerns that can also compromise trust in the qualifications. External moderation is frequently used to allay such concerns.The aim of the study is to identify illustrations of moderation in selected secondary school exit examinations and understand the local contexts that have contributed to the development of such systems. A two-phase sequential survey design was used to explore the variables of interest across 13 jurisdictions.

18

11/22/2024

WEDS 30th October 2024 17.30-19.00 - a Zoom link is sent to participants on the day of the event. All EAG@IOE talks are recorded and can be accessed after the event. Over our field's 100-year-plus history, standardization has been a central assumption in test theory and practice. The concept's justification turns on leveling the playing field by presenting all examinees with putatively equivalent experiences. Until relatively recently, our field has accepted that justification almost without question. In this talk, I present a case for standardization's antithesis, personalization. Interestingly, personalized assessment has important precedents within the measurement community. As intriguing are some of the divergent ways in which personalization might be realized in practice. Those ways, however, suggest a host of serious issues that will need to be addressed if personalized assessment is to avoid perpetuating past inequities and work toward the benefit of all.

15

11/12/2024

Comparative judgement is an increasingly popular method in educational assessment. While much of the literature focuses on different applications of comparative judgement, less attention has been given to developing a clear, robust theoretical framework for the method. Drawing on interviews with 13 comparative judgement practitioners, this webinar discusses how the assessment construct is understood in comparative judgement, and the implications this might hold for assessment practice.

52

10/14/2024

Dr Alexander Scharaschkin (AQA) Educational assessment can take many forms, but always involves a process of making inferences from learners’ performances on (explicit or implicit) tasks. Traditional quantitative methods of test theory use quantitative scores, derived from learners’ responses to test items, to underpin claims about their proficiency in a domain. Yet in many cases (for example GCSE or A level examinations in England) the “raw data” that forms the input for these methods is inherently qualitative: written responses to questions, which are appraised against fuzzy scoring criteria expressed in natural language. The recent development of large language models, such as BERT and GPT4, has opened new possibilities for processing such linguistically expressed information mathematically. In this talk Alex will explore the scope for using large language models to extend psychometric approaches to educational assessment.

10

9/30/2024

Several factors are affecting the field of educational testing that could have enormous impact on its evolution. Paramount among these factors are a decline in public trust of the testing industry, an acknowledgment of the role tests have had in systemic racism, innovations in educational assessment technology, the use of tests as tools to help students learn, and the development of culturally sustaining assessment systems. In this talk, these and other factors will be discussed to suggest several ways the educational testing industry may change in the near future. Many of these possible futures are positive, and will involve more inclusive and flexible assessment systems to promote fairness and validity in educational assessment and leverage the power of assessments to help people achieve their academic, occupational, and other personal goals.

6

9/30/2024