Evidence-Centred Design

Evidence-centred design (ECD) is a framework for building assessment systems — and, by extension, any AI inference system — by working backward from the intended conclusion. Most assessment development starts with tasks and asks what can be learned from them. ECD inverts this: it starts with the claim to be made about a person, specifies the evidence that would warrant that claim, and only then designs the tasks and situations that would elicit that evidence. The result is an assessment whose structure is transparent, whose inferential chains are explicit, and whose validity argument is built into the design rather than retrofitted afterward.

Robert Mislevy, Linda Steinberg and Russell Almond's 2003 paper "On the Structure of Educational Assessments" formalised ECD into five interconnected models. The Student Model defines the latent variables — the attributes, competencies, or knowledge states the system aims to infer. The Evidence Model specifies observable features of performance that serve as indicators of student model variables, and the statistical rules (often Bayesian networks) for updating belief in those variables based on observed evidence. The Task Model defines the tasks, situations, or prompts that elicit the relevant performance. The Presentation Model governs how tasks are delivered and interactions recorded. The Assembly Model governs how tasks are selected and sequenced across an assessment.

This architecture makes the inferential structure of an assessment explicit and auditable. Each arrow from evidence feature to student model variable is a claim: this observable behaviour is a reliable indicator of this latent attribute. The task model defines the conditions under which evidence is produced. The evidence model is explicit about how observations update beliefs. The result connects directly to validation theory: the ECD framework is a natural expression of Kane's interpretive argument, making each inference link visible and testable. Assessment systems built on ECD are inherently more defensible because their inferential commitments are stated, not assumed.

Mislevy's later work (Sociocognitive Foundations of Educational Measurement, 2018) extends ECD to connect with broader theories of cognition and social practice, recognising that what counts as valid performance is always embedded in social and cultural contexts that must be theorised rather than assumed. This is particularly relevant for AI-derived inferences from conversational or workplace data, where the social context of performance is both more variable and more consequential than in controlled testing environments.