Case studies
AI automation and QA in practice
How we put AI to work for businesses, test AI features, and run QA for real products.
KidSift: an automated evaluation harness for an LLM product
- Product
- KidSift turns school and activity emails into a daily plan for parents
- Service
- AI Eval Harness Build
The challenge
KidSift uses an LLM to read messy, real-world emails and pull out dates, times, locations and to-dos. A wrong date or a missed event is worse than no answer, and every prompt or model change can quietly break extraction that used to work.
The approach
- A golden dataset of real-world email patterns, each with the expected events and to-dos
- Automated scoring for extraction accuracy, missed items and invented items (hallucinations)
- Evaluations that run in CI on every prompt or model change and block regressions
- A per-release score report to compare models and prompts on quality and cost
This harness is in progress; measured results will be published here.
Global manufacturer, Des Moines: ongoing managed QA
- Client
- Global ingredients manufacturer headquartered in Iowa
- Service
- Managed QA
SandboxQA provides a dedicated, full-time QA engineer under SandboxQA's management, covering regression and new-feature testing month to month so the client's team can keep releasing without hiring and managing testers directly.
Want results like these?
Start with a 2-week AI Quality Readiness Assessment or a conversation about a QA pod.
Get in touch