Healthcare AI training data & evaluations
Data for the cases healthcare models get wrong.
We build training datasets, preference data, and held-out evaluations around specific model failures, then ship them with the checks, provenance, and limitations you need to trust them. Starting with clinical nutrition.
3 passed1 flagged1 pending review
Scenario
Adult with stage 4 chronic kidney disease, a plant-forward preference, and no record of dialysis status or recent labs.
- protein_range_renal
- potassium_limit
- plant_forward_preference
- Clinical constraints referenced3 of 3 in the planPassed
- Missing context recogniseddialysis_status, current_lab_valuesFlagged
- Structured output parsedschema validPassed
- Potassium limit respectedno high-potassium staples in 28 daysPassed
- Clinician reviewrequired before usePending
Failures we build data around
- Silent constraint violations
- Right direction, wrong dose
- Unflagged missing context
- Allergens served after being listed as forbidden
- Fluent plans for impossible patients
01The problem
Healthcare data isn't a volume problem.
A model can produce thousands of plausible answers and still fail when clinical constraints interact. Useful training data has to capture those failures, not merely look medically convincing.
- 1
Hard clinical constraints
A sodium limit, a protein range, a fluid restriction. Each is simple on its own. A model that treats them as suggestions produces plans that read well and cannot be followed.
- 2
Conflicting requirements
Two legitimate constraints can pull in opposite directions. The useful behaviour is to notice the tension, state it, and defer where a clinician has to decide. Most training data never shows that.
- 3
Rare and difficult cases
Generated corpora over-represent easy scenarios because they are easy to generate. The cases that break models have three interacting conditions and a missing lab value.
- 4
Evidence, not volume
A dataset that cannot show what was checked, how, and with what coverage is a file, not evidence. Buyers need to know which failures the data was built to expose.
02What we build
Four ways to work with us.
Each comes with its own documentation. Formats, review coverage, and licence terms are agreed per engagement.
Specialized training data
Instruction-response and schema-bound records designed around clinical reasoning and constraint handling, for supervised fine-tuning.
- SFT JSONL
- Structured JSON
Preference data
Candidate comparisons labelled on explicit criteria, so post-training rewards constraint adherence instead of fluency.
- Preference pairs
- Rubric annotations
Clinical evaluations
Held-out tasks and deterministic checks that surface unsafe doses, contradictions, and missing-context failures against your model's own baseline.
- Evaluation tasks
- Gate reports
Custom development
Engagements scoped from the failures you are seeing in production or evaluation, through to a delivered, documented dataset.
- Discovery
- Specification
- Delivery
03Sample package
See exactly what a delivery looks like.
Twenty real records from our generated corpus, with the schema, rubric, provenance notes, and quality report that ship with every dataset. Download it and audit it yourself.
- Plans scanned250,000
- Passed all 45 gate rules67,269
- Passed 10 package checks439
- Selected for the sample20
Log scale. 0.18% of scanned plans met every criterion. That gap is what unfiltered generation looks like, and why we ship the filter with the data.
- Datameethealth-sample-v1.jsonl20 records, one JSON object per line. Source fields unmodified, plus a provenance and checks block.
- Schemaschema.jsonJSON Schema (draft 2020-12) for every record.
- Rubricrubric.jsonSix weighted axes and a clinically-unsafe flag. Definition only; not applied to these records.
- Quality reportQUALITY_REPORT.mdSelection funnel, every check and what it excluded, composition, and known limitations.
- Machine-readablequality-summary.jsonThe same report as JSON, including per-record check results.
- ProvenanceREADME.mdHow records were generated and selected, and the terms of use.
Package 1.0.0 · 345 KB of JSONL · automated checks only, no clinician review.
04Technical foundation
We measure our own data before anyone else does.
Every corpus runs through the Assessment Gate: a deterministic, versioned contract of clinical and structural rules. No LLM judges this step, so results are reproducible to the record.
Read the certification note- 1.53M
- AI-generated diet plans audited across four model configurations
- 9.59M
- Synthetic patient profiles run through the gate
- 45
- Deterministic rules in the diet-plan contract
| Corpus | Records | Zero findings | Blocked |
|---|---|---|---|
| Gemma 4 plans | 1,012,105 | 47.9% | 22.6% |
| Llama 4 plan pairs | 257,655 | 24.0% | 23.0% |
| Claude Opus plans | 904 | 20.5% | 14.7% |
Blocked means at least one critical finding, such as an unsafe protein dose for kidney disease or an incoherent patient record. Passing means internally consistent, not clinically validated.
05Where this comes from
We started by building healthcare AI, not a dataset catalog.
Meethealth began as an AI-native platform for dietitians: clinical workflows, structured meal planning, and generation pipelines that had to respect real dietary constraints for real practitioners.
Making that software useful forced us into the hard parts. Nothing off the shelf told us whether a generated plan was usable, so we built the generation infrastructure, the verification layer, and the evaluation rubrics ourselves.
Meethealth Labs turns that work into infrastructure other teams can build on.
What the product taught us to do
- 1Model nutritional requirements as explicit, checkable constraints
- 2Handle several simultaneous constraints without silently dropping one
- 3Support culturally relevant diets across languages and food availability
- 4Catch outputs that read well but cannot be followed
- 5Keep structured outputs consistent across millions of generations
- 6Evaluate with deterministic rules and rubrics, not impressions
06How it works
Define. Generate. Verify. Review. Deliver.
Every step produces a written artifact you can inspect. Steps marked as in use already run on our research corpus today.
- 1Per engagement
Define the capability
Pin down the model behaviour you want to change and the failures you are seeing.
- Target behaviour
- Failure inventory
- 2In use
Write the specification
Constraints, specialty coverage, output schema, difficulty mix, and acceptance criteria, in writing.
- Schema
- Contract
- Acceptance criteria
- 3In use
Generate candidates
Synthetic scenarios and responses from controlled, versioned generation runs.
- Versioned records
- Prompts
- 4In use
Verify and review
Deterministic gate rules on every record, rubric scoring, and qualified human review where required.
- Gate verdicts
- Rubric scores
- Review plan
- 5Per engagement
Package and deliver
Accepted records with schema, provenance, quality report, and documented limitations.
- Dataset
- Quality report
- Provenance
07Dataset categories
Where we have depth today.
Each category shows its current status. Access starts with a short scoping conversation.
Clinical Nutrition Instruction Data
Structured nutritional reasoning and generation tasks in instruction-response form.
Research previewComplex Clinical Constraint Scenarios
Cases with multiple interacting requirements, incomplete information, or conflicting recommendations.
Custom on requestNutrition Preference Comparisons
Comparative response examples for investigating which answers satisfy domain-specific requirements more consistently.
Research previewMultilingual Dietary Reasoning
Language, food availability, cultural preferences, and local dietary practices as first-class task parameters.
Custom on requestNutrition Safety and Consistency Evaluations
Held-out tasks for detecting unsupported recommendations, contradictions, numerical errors, and missing-context failures.
Evaluation development
Tell us what your model gets wrong.
Bring the failure modes, the model, and the deployment context. We'll tell you what is feasible, what expertise it needs, and how we'd measure it.