Physical AI evaluation
Deepen AI's trained reviewers watch real scenes with your model's decisions alongside, judge whether each call was right, and correct the ones that weren't.
Rate in your plan before you pay A plan within 2 working days No sales call
What an evaluated decision looks like
Each decision comes back right or wrong, with the evidence, and the right call when the model missed it.
- Scene
- Unprotected left turn, with an oncoming van 40 metres away
- Your model decided
Proceed.- Reviewer judged
- Wrong callYield. The van reaches the junction in 2.1 seconds, and the turn needs 3.
- Reason
- Speed from telemetry, timing from the front and left cameras.
- Scene
- A pedestrian waits at a crossing, looking at a phone
- Your model decided
- Slow down and prepare to stop.
- Reviewer judged
- Right callCorrect and smooth: braking stayed inside your comfort limit.
- Reason
- Your rubric scores both safety and comfort.
- Scene
- A worker steps into the aisle 3 metres ahead of a warehouse robot
- Your model decided
- Stop.
- Reviewer judged
- Right callStopped 1.8 metres from the worker, inside your safety margin.
- Reason
- Distance measured from the LiDAR at the stopping frame.
- Scene
- A plastic bag blows across a highway lane
- Your model decided
- Brake hard.
- Reviewer judged
- Sent to youBraking wasn't needed but wasn't unsafe. Your rubric doesn't cover harmless objects. How should it be scored?
- Reason
- Your answer is added to the rubric.
How we start on your work
No sales call. A plan in your inbox, then a short paid pilot on your real scenes.
What we need from you
- Scenes to evaluate: multi-camera video, telemetry and your model's outputs
- Your rubric: what a right call looks like
- 20 to 30 scenes you've already judged
- 5 minutesTell us the task
What your model decides, how many scenes each round, and when you need results.
- Within 2 working daysGet a plan and a price
Reviewers, hours, the rate, start date and the total price.
- 2 weeksRun the pilot
We train reviewers on your rubric, evaluate real scenes and send two weekly reports. Then you decide.
Who evaluates, and how judgements stay consistent
Trained reviewers, working in English. Each one learns your rubric and judges a set of known scenes before live work.
Every reviewer's accuracy is measured
Items you've already answered are hidden in the real work, so we know how accurate each person is.
Nobody guesses on unclear cases
The team lead decides, or asks you. Your answer joins the guidelines.
You see the results every week
A weekly report shows work done, QA results and what's improving.
Standard evaluation covers vehicle and robot decisions from video and telemetry. Large programs, and evaluation in Deepen AI's own tools, are scoped by the enterprise team.
How much does physical AI evaluation cost?
Physical AI evaluation is quoted in your plan, because the rate depends on your scenes, your tools and the expertise the work needs. You pay for the hours reviewers work, with a team lead, QA checks and a weekly report included.
Set before you pay. Your plan shows the rate, team size, hours and total price.
Starts with a two-week pilot on your real scenes, paid upfront, with two weekly reports.
Your data stays protected. We work under NDA with the access you set. Deepen AI holds SOC 2 Type II, ISO 27001, ISO 9001 and TISAX.
Questions
What do reviewers see?
The scene from every camera you provide, the telemetry, and your model's decision at each moment. They can step through it frame by frame.
How do they decide whether a call was right?
Against your rubric, with evidence: speeds, distances and timings from your telemetry and sensors. Every judgement cites what it was based on.
Can corrected decisions be used for training?
Yes. When a reviewer corrects a call, the right decision and its reason can go back into your training data.
Can we compare model versions?
Yes. Keep the scenes and rubric fixed, and each version is judged the same way, so you see which calls a change fixed and which it broke.
How do you keep judgements consistent?
Reviewers judge known scenes before live work, a share of scenes is judged by more than one person, and agreement is in your weekly report.
How is our data protected?
We work under NDA, with the access you set. Deepen AI holds SOC 2 Type II, ISO 27001, ISO 9001 and TISAX, the assessment automotive companies ask for.