Case studies

AI automation and QA in practice

How we put AI to work for businesses, test AI features, and run QA for real products.

KidSift: an automated evaluation harness for an LLM product

Product
KidSift turns school and activity emails into a daily plan for parents

The challenge

KidSift uses an LLM to read messy, real-world emails and pull out dates, times, locations and to-dos. A wrong date or a missed event is worse than no answer, and every prompt or model change can quietly break extraction that used to work.

The approach

  • A golden dataset of real-world email patterns, each with the expected events and to-dos
  • Automated scoring for extraction accuracy, missed items and invented items (hallucinations)
  • Evaluations that run in CI on every prompt or model change and block regressions
  • A per-release score report to compare models and prompts on quality and cost

This harness is in progress; measured results will be published here.

Global manufacturer, Des Moines: ongoing managed QA

Client
Global ingredients manufacturer headquartered in Iowa
Service
Managed QA

SandboxQA provides a dedicated, full-time QA engineer under SandboxQA's management, covering regression and new-feature testing month to month so the client's team can keep releasing without hiring and managing testers directly.

Want results like these?

Start with a 2-week AI Quality Readiness Assessment or a conversation about a QA pod.

Get in touch