How a Law School Cut Bar Exam Question Time 75%
A law school's legal SMEs spent an hour on each exam question. A pair-prompting AI now drafts compliant questions in 15 minutes — turning SMEs from writers into reviewers.
75%
Cut from exam question review time
Mel Moeller
Chief AI Officer
GenFutures Lab
Connect ↓
The Challenge
BPP University needed to rapidly expand its bank of multiple-choice questions for the UK Solicitors Qualifying Examination (SQE). Crafting each legally compliant question took up to an hour per question because wrong-answer 'distractors' must be plausible and precise. With growing student demand for more practice questions, the law school lacked the capacity to scale production through manual effort alone, creating a significant bottleneck in exam-content creation and straining faculty time and budget.
What They Built
GenFutures Lab deployed a rapid 'create, test, learn' sprint methodology, pairing an AI engineer with BPP law school subject-matter experts (SMEs) in a pair-prompting approach. Using OpenAI's large language models hosted on Microsoft Azure, the team fine-tuned models on a structured legal knowledge base drawn from the SQE question handbook. Iterative feedback loops between the AI engineer and legal SMEs refined prompts and model outputs until question accuracy met exam-board standards. Once validated, the solution was scaled to continuous production, with SMEs shifting from drafting questions to reviewing and approving AI-generated ones — dramatically accelerating throughput while maintaining legal compliance.
GenFutures Lab used a rapid create-test-learn sprint methodology, pairing an AI engineer directly with BPP law school subject-matter experts in a pair-prompting approach. The team began by fine-tuning OpenAI large language models — hosted on Microsoft Azure — on a structured legal knowledge base drawn from the SQE question handbook. This domain grounding was essential: wrong-answer distractors in legal MCQs must be plausible and precisely incorrect, which generic LLM output cannot reliably achieve. Iterative feedback loops between the AI engineer and legal SMEs refined both prompt design and model output across structured sprint cycles, continuing until question accuracy met exam-board standards. Once validated, the workflow was scaled to continuous production. SMEs shifted from drafting questions from scratch — a process taking up to an hour each — to reviewing and approving AI-generated questions, which take 15 minutes. The result was a 75% reduction in review time and approximately £10,000 in monthly savings, with the capacity to scale question volume without proportionally increasing faculty time.
AI Role
OpenAI large language models, hosted on Microsoft Azure and fine-tuned on a structured legal knowledge base drawn from the SQE question handbook, generate multiple-choice exam questions including plausible wrong-answer distractors. The models operate through iteratively refined prompts co-developed by an AI engineer and legal subject-matter experts, producing exam-ready drafts that SMEs review and approve rather than write from scratch.
Infrastructure
OpenAI LLMs (fine-tuned on legal knowledge base) • Microsoft Azure (hosting) • SQE question handbook (training corpus)
Integration Points
SQE question handbook → LLM fine-tuning pipeline • OpenAI LLMs → Azure-hosted generation endpoint • SME review workflow → iterative prompt refinement loop
Impact
75% Reduction in Review Time
Time to produce each exam question dropped from 1 hour to 15 minutes, freeing SME capacity for higher-value review and editorial work rather than question drafting from scratch.
4x SME Throughput & 500 Questions/Month at Scale
SMEs went from reviewing 10 questions per day to 40. Monthly output reached 500 high-quality MCQs (1,000 produced within the first two months), meeting student demand that was previously unachievable given resource constraints.
£10,000 Monthly Cost Savings Within 4 Months
The efficiency gains translated to £10,000 per month in cost savings just four months into the engagement, delivering a clear, quantifiable ROI and demonstrating the financial case for AI-assisted content workflows in professional education.
Implementation Complexity
The solution required fine-tuning LLMs on a domain-specific legal knowledge base and developing iterative prompt engineering workflows through a structured pair-prompting methodology with legal SMEs. While hosted on a standard platform (Azure/OpenAI), the domain specificity and quality validation requirements make this more than a simple off-the-shelf deployment.
Best Fit For
Large enterprises and educational institutions in early-to-mid AI adoption stages that need a value-first, structured approach to identifying and implementing AI use cases. Particularly strong fit for organizations in media, entertainment, education, and sports with complex content workflows and a desire to move from AI pilots to measurable outcomes.
