AI QA Analyst

A QA systems thinker who sits between model behavior and user trust, dissecting AI outputs with forensic precision to catch reproducible failure patterns, coverage gaps, and regression signals before they erode trust in production.

- The INSPECT AI QA methodology — 7-pillar framework from behavioral requirements to continuous drift detection

- Test case design for non-deterministic systems: equivalence partitioning, edge case generation, golden dataset curation

- Multi-dimensional rubric design decomposing quality into factual accuracy, instruction adherence, tone, safety, format

- LLM-as-judge calibration with bias auditing, position sensitivity testing, and human rater baseline comparison

- Failure mode taxonomy and root cause triage: hallucination clustering, regression identification, guardrail gap analysis

- Production monitoring setup with output sampling workflows, drift detection signals, and quality SLA definition

- Statistical analysis of evaluation results: inter-rater reliability, A/B prompt significance testing, confidence intervals

- Actionable quality reporting: specific failure concentrations tied to input categories, severity scoring, reproducible examples

Drop into Claude, ChatGPT, Cursor, or any AI tool. Bring your real QA problem — a hallucination cluster you need to isolate, a prompt change that regressed accuracy, a production quality signal you can't interpret. It thinks like a QA engineer who has built evaluation pipelines and caught subtle degradations before users complained.

Bundles or prompts related to AI evaluation, product quality assurance, and model testing frameworks.