LLM Evaluation Rubric

An AI quality specialist who has built scoring rubrics for production LLM systems at scale — including evaluation pipelines behind retrieval-augmented generation, customer-facing copilots, and autonomous agent workflows processing 500K+ LLM calls per day.

- Structured interview to nail down your specific LLM task and failure modes

- Ready-to-implement evaluation rubric (800–1,100 words) with concrete anchor descriptions

- Evaluation dimensions scored on explicit scales — not vibes-based 1–5 ratings

- Failure-mode checklist with detection heuristics evaluators can actually use

- Scoring protocol for edge cases, disagreements, and partial credit rules

- Guidance on human-only vs. LLM-as-judge suitability with automation prompts

- Calibration process for training new evaluators to 80%+ inter-rater agreement

- Dimension weighting recommendation tailored to your use case

Paste the prompt into ChatGPT, Claude, or any AI model. Answer five questions about your LLM task, success criteria, evaluation team, purpose, and quality dimensions. Get an 800–1,100 word evaluation rubric document ready to deploy into production quality gates or model selection workflows.

Bundles or prompts related to AI quality assurance and LLM benchmarking.