A TIRF Perspective on using artificial intelligence to strengthen language assessment while protecting validity, fairness, and public trust

Editor’s note: TIRF Perspectives is a new publications activity from TIRF. It profiles recent research and developments in English Language Teaching (ELT) in a practical and accessible format. TIRF Perspectives is aimed at parents, teachers, school leaders, and all student advocates who want to review recent educational trends regarding AI in a non-academic way. This piece is authored by Dr. Nick Saville (Cambridge University Press & Assessment).

Where We Are

TIRF Perspectives aims to make recent developments in English language education accessible and useful to teachers, school leaders, parents, and others who support learners. So, rather than beginning with the technology, I want to start with the educational purposes it should serve.

I have worked in language education and assessment long enough to see successive technologies arrive with confident promises of transformation. Some have changed practice in important ways, but others have faded once the initial excitement encountered more demanding questions about purpose, evidence, and impact. AI is different in scale, speed, and reach, but it does not alter the profession’s central responsibility: Before asking what the technology can do, we must ask what language assessment is for, whose interests it serves, and how AI might improve its quality and usefulness.

By AI-powered assessment, I mean the use of artificial intelligence at one or more points in the assessment cycle: 1) designing tests and tasks, 2) administering them, 3) scoring and standard setting, 4) reporting results, and 5) reviewing performance. I’m referring to established technologies such as natural language processing, machine learning and speech technology, (Old Fashioned AI), as well as newer generative systems based on large language models (GenAI). Used carefully, AI tools can help experts draft materials, identify patterns in learner performance, support adaptive delivery, provide feedback more quickly, and strengthen quality assurance. Yet the evidence base on these uses of AI is uneven. A 2026 scoping review of 212 studies found that research has focused mainly on the automated scoring of speaking and writing, with much less attention to validation, receptive skills, sign languages, and practical guidance for stakeholders (O’Grady et al., 2026).

Current Issues

For me, the central professional question is one of construct validity – that is the extent to which a test assesses what it is intended to. A construct is the technical termused to refer to the ability or skill that an assessment is intended to measure.

Assessment design should begin with a clear definition of the construct we want to measure rather than with the capabilities of an AI tool. For example, if the aim is to assess independent writing ability, we must first specify what counts as good writing and what evidence a task should elicit. Only then should we decide whether AI can support task design, scoring, or feedback without changing what is being measured.

While AI offers many potential benefits for English language assessment, we should not underestimate the risks. AI systems can reproduce bias, obscure how decisions have been reached, weaken privacy protections, create new inequities, or produce outcomes that look more defensible than the evidence warrants. These concerns become more serious in high-stakes assessment, where results may affect access to education, employment, or migration.

Cambridge University Press & Assessment’s six principles for ethical AI in language learning and assessment provide a useful starting point in considering AI’s risks: Systems should 1) meet the standards expected of human examiners, 2) demonstrate fairness, 3) protect privacy and consent, 4) remain transparent and explainable, 5) retain humans in the loop, and 6) address sustainability.

The AI-Forward Declaration, from the Associate of Language Testers in Europe (ALTE), takes a similar position, arguing that AI should extend human capacities, support communicative and plurilingual approaches, and be governed through interdisciplinary collaboration rather than treated as a substitute for professional judgement.

What Responsible AI for Assessment Practice Looks Like Now

1. Start with the intended interpretation and use. Specify the language ability to be assessed, the decisions the evidence will support, and the consequences of error before selecting an AI application.
2. Validate the whole system, not only the model. Compare machine-supported outcomes with expert judgements, investigate performance across relevant languages and learner groups, and monitor unintended changes after deployment.
3. Keep accountability visible. Assign named human responsibility for consequential decisions and provide routes for review, challenge, and correction.
4. Be transparent with users. Explain when AI is used, what data it processes, how outputs contribute to decisions, and what safeguards are in place.
5. Pilot before scaling. Begin with low-risk uses—such as drafting items for expert review or generating formative feedback—before considering high-stakes applications.
6. Review proportionality and sustainability. Ask whether AI is necessary, whether a simpler method would suffice and whether the educational benefit justifies environmental, financial, and social costs.

There are already promising examples of AI for assessment best practice. For example, within Cambridge English’s test development teams, AI supports expert-led task creation, and for more than a decade, the Cambridge Institute for Automated Language Teaching and Assessment has offered an interdisciplinary research model by bringing together assessment, learning, computational linguistics, machine learning and speech processing.

Useful Resources

Within the wider language assessment profession, EALTA’s (European Association for Language Testing and Assessment) AI in Language Assessment Special Interest Group supports professional learning, transparency, and dialogue with regulators. ALTE’s Principles of Good Practice, quality-management system, and revised 2026 Manual offer an established framework for applying validity, fairness, and quality assurance across the testing process, including digital assessment and AI. (For further information about these resources, see the “Best-Practice Resources” and “References” section below.) 

Where We’re Going

The most promising future is not simply one in which speed and efficiency are the main goals. It is one in which assessment becomes more responsive to learners, more closely connected with learning, and better able to represent language use in its complexity. AI could help us connect evidence gathered over time, provide richer diagnostic feedback, and create adaptive pathways among assessment, teaching, and progression. It may also help us represent constructs that conventional tests have found difficult to capture, including interaction, mediation, and plurilingual repertoires (as defined in the CEFR Companion Volume). None of this will happen through technology alone. Assessment organisations will need stronger AI literacy, shared evaluation protocols, representative data, interdisciplinary governance and sustained engagement with test-takers, teachers, and score users.

The AI-Forward Declaration captures this direction in four practical maxims: 1) align AI with human values; 2) use AI to extend communicative approaches; 3) enable all aspects of language learning and assessment; and 4) make environmental, social, educational, and economic sustainability a central design concern. These maxims offer a useful way of testing proposed innovations. Speed, cost, and scalability matter, but they are not enough. We also need to ask whether an AI-enabled process produces better evidence, widens access, protects agency, and strengthens confidence in assessment outcomes.

Over to You

I suggest beginning any discussion of AI-supported assessment with a number of practical questions. These are questions for colleagues, stakeholders, and, importantly, the people whose language abilities are being assessed:

1. What claim about language ability are we trying to make, and what evidence would justify it?
2. What does AI add that could not be achieved as effectively through a simpler approach?
3. Which groups might benefit, and which might be disadvantaged or excluded?
4. How will we compare AI-supported outcomes with expert human judgement and test for bias?
5. Who remains accountable, and how can a learner question or appeal an outcome?
6. What will we disclose about the role of AI, data use, and limitations?
7. How will we monitor educational impact, model drift, and sustainability after implementation?

Our role is not to protect established practice, nor to pursue every new technology that comes along. It is to shape assessment systems around clear educational purposes, providing credible evidence of learning and taking responsibility for the consequences. In considering the use of AI tools in your own context for assessment purposes:

• Why are you considering using AI in your context?
  • What is it for and what are the intended benefits? In what ways will it improve your existing practices?
  • Are these improvements proportional to the cost and sustainability of using AI?
  • How will you investigate the impact of the changes you make?
• Do you have the knowledge, skills, and resources to implement the intended improvements, and if not, how do you intend to improve your AI literacy and acquire the necessary resources?

Join the conversation on this TIRF Perspectives piece and others on Substack – click here.

Best-Practice Resources

• Cambridge Papers in English Language Education: Six guiding principles for ethical AI in language learning and assessment
• AI-Forward Declaration 2026: Human-centred, communicative, and sustainable principles for language learning, teaching, and assessment
• ALTE Resources: Principles of Good Practice, quality-management guidance, and the 2026 Manual for Language Test Development and Examining
• EALTA AI in Language Assessment Special Interest Group: Professional exchange, resources, and discussion of transparent, responsible practice
• O’Grady, Gokturk, and Rossi (2026): A scoping review of empirical AI research in language assessment and its outstanding gaps

Author’s Bio

Professor Nick Saville has 45 years’ experience in language education and assessment. Formerly Director of Research and Validation, he is now Director of Thought Leadership at Cambridge University Press & Assessment, as well as a TIRF Trustee. His interests include plurilingualism, Learning Oriented Assessment, impact by design, and AI. He is ALTE Secretary-General and visiting professor at CRELLA (University of Bedfordshire).

References

Allen, H., Galaczi, E., Halley, K., Khabbazbashi, N., Lopes, S., & Nakatsuhara, F. (2025). Exploring the impact of generative AI on language education: Insights from teachers. Cambridge University Press & Assessment.

Association of Language Testers in Europe. (2020). Principles of good practice. https://www.alte.org/Materials

Association of Language Testers in Europe. (2026). Manual for language test development and examining. https://www.alte.org/Materials

Cinganotto, L., Montanucci, G., & Saville, N. (Eds.). (2026). Artificial intelligence for language learning, teaching, and assessment. Edilingua.

Galaczi, E., & Luckin, R. (2024). Generative AI and language education: Opportunities, challenges and the need for critical perspectives. Cambridge University Press & Assessment.

Galaczi, E., & Pastorino-Campos, C. (2025). Ethical AI for language assessment: Principles, considerations, and emerging tensions. Annual Review of Applied Linguistics, 45, 294–314. https://doi.org/10.1017/S0267190525100081

Macinska, S., & Vinkler, A. (2025). The role of AI in language learning: Research evidence and strategies for success. Cambridge University Press & Assessment.

O’Grady, S., Gokturk, N., & Rossi, O. (2026). Artificial intelligence in language assessment research: A scoping review. Research Synthesis in Applied Linguistics. Advance online publication. https://doi.org/10.1080/29984475.2026.2633389

Organisation for Economic Co-operation and Development. (2024). Recommendation of the Council on artificial intelligence (OECD/LEGAL/0449).

Pastorino-Campos, C., & Saville, N. (2025). Ethical AI for language learning and assessment. Cambridge University Press & Assessment.

UNESCO. (2022). Recommendation on the ethics of artificial intelligence. https://unesdoc.unesco.org/ark:/48223/pf0000381137

Xu, J., Schmidt, E., Galaczi, E., & Somers, A. (2024). Automarking in language assessment: Key considerations for best practice. Cambridge University Press & Assessment.

Zeronis, R., Mullooly, A., & Somers, A. (2026). Using generative AI responsibly in assessment content creation: Cambridge principles and practice. Cambridge University Press & Assessment.