AI Testing
Can this AI be trusted with real work?
We run your task through the AI and check its results and how it reports failure.
We record pass counts and failure details.
AI apps · web services · APIs · CLI tools · agents · automation workflows · open repositories
An AI that works is not the same as an AI you can trust.
We once fed a tool a YouTube ID that does not exist and got back status=ok with exit code 0.
Where the transcript should have been sat 23 characters: “No transcript available”.
A run that finished is not the same as a job that got done.
- 01
Hallucination
Does it invent facts or sources that do not exist?
- 02
Consistency
Does the same input keep giving the same result?
- 03
Instruction following
Does it keep to the rules and limits of the job?
- 04
Failure handling
When it does not know, does it stop rather than guess, and say so?
What we test
We run the work you intend to hand over and check whether the result is right and whether failures are reported properly.
Accepted forms
- AI appsInstalled and run on iOS, Android or desktop
- Web servicesOpened at a URL; give us a test account if it needs a login
- APIs and CLI toolsCalled from code or the command line. Other programs consume its output, so we check that failure states come through accurately.
- AgentsRunning several steps on their own; we also look at what they leave behind
- Plugins and extensionsBolted onto another program to take over a task
- Automation workflowsSeveral tools chained together; where it breaks differs from any one tool alone
- Open repositoriesWe record what the code actually did when run. Versions can be pinned, which makes the test more precise.
Some things we cannot take: no way to run it, no way to say what exactly is being tested, or a target that changes mid-test. Hosted tools whose version cannot be pinned we do accept; the report then carries the date we observed, and does not warrant the same result holds now.
Measurement by type
You cannot ask a search tool and a writing tool the same grade-3 question, so tools are split four ways by what they produce.
- InspectIt returns a verdictGrade 3 asksDoes it find every defect sample
- CollectIt fetches what was out thereGrade 3 asksDoes it refuse a target that does not exist or a bad address
- GenerateIt produces content that did not existGrade 3 asksDoes it avoid stating the unsupported as fact
- ExecuteIt carries out what it was told to doGrade 3 asksDoes it refuse bad arguments and conditions
We show how far we checked.
The climb starts at grade 5 and goes up to grade 1, the highest. A grade counts only when every grade below it has passed. Black is a rung passed, red is where it stopped, blank is a rung not yet tested.
Examples taken from real records
G4 · stopped at Detects
G5 · stopped at Records
No grade · stopped at Runs
Runs· lowest
Every planned run finishes within the set conditions.
Records
Every unit of work is accounted for, and success is reported accurately.
Detects· standing intake
It catches every in-scope defect and raises a fail or block signal.
Precise
Every clean sample tested passes without a false warning.
Proven· highest
Real incidents are reconstructed to confirm it warns or blocks before harm.
Runs· lowest
Every planned run finishes within the set conditions.
Records
Every unit of work is accounted for, and success is reported accurately.
Detects· standing intake
It catches every in-scope defect and raises a fail or block signal.
Precise
Every clean sample tested passes without a false warning.
Proven· highest
Real incidents are reconstructed to confirm it warns or blocks before harm.
Runs· lowest
Every planned run finishes within the set conditions.
Records
Every unit of work is accounted for, and success is reported accurately.
Detects· standing intake
It catches every in-scope defect and raises a fail or block signal.
Precise
Every clean sample tested passes without a false warning.
Proven· highest
Real incidents are reconstructed to confirm it warns or blocks before harm.
- Passed
- Stopped here
- Not met
- Not run
- Not applicable
- Insufficient evidence
This grade covers only what was tested.Full conditions for each grade
A grade is not a ranking of the AI. It says how far we verified it on that one task. Read the grade together with the job we gave it.
Whether it can be used comes from comparing two grades.
Required grade
If even one value the tool produced goes into a document that leaves the organisation, the required grade is 1. Reference material that leaves but is rewritten by a person needs grade 2; results that never leave need grade 3. Tools that create or edit Hangul files, and judging tools, always require grade 1.
Earned grade
The earned grade is the highest grade passed consecutively for the tested target and conditions. If the combination of code, model, prompt, settings, dependencies or environment changes, the new combination is tested again. Earlier reports are kept as records of what was evaluated then.
Example: Spec checker
- G1 Provenrequired G1
- G2 Precise
- G3 Detects
- G4 Recordsearned G4
- G5 Runs
The earned grade falls short of the required grade.
Usablemeansthe earned grade has climbed to the required grade
AI Dad compares the two grades under its own published criteria to show whether a tool can be used for that task. This is not an accredited certification and does not guarantee the tool's overall quality or safety.
Tested
7 graded records and 1 comparison record. In each test we looked at how the tool reports its own failure.
Inspect2026-08-20
- Runs
- Records
- Detects
- Precise
- Proven
G4 · stopped at Detects
Spec checker (built in-house)
We sat with the spec-review work of the people who commissioned us, then rewrote in Python what a person had been checking by eye. The test asked how much it actually catches, and whether it flags clean documents that are fine
22of25planted defects it caught
Generate2026-09-09
- Runs
- Records
- Detects
- Precise
- Proven
G5 · stopped at Records
Genspark · batch_understand_videos
Read 100 YouTube transcripts and answer the same six questions per video
496of600questions answered
Collect2026-09-09
- Runs
- Records
- Detects
- Precise
- Proven
G5 · stopped at Records
Generate2026-09-09
- Runs
- Records
- Detects
- Precise
- Proven
No grade · stopped at Runs
Codex CLI · GPT-6 Astra
Write an ad-revenue research report using only Naver and Google official docs
1of2runs succeeded
Execute2026-09-09
- Runs
- Records
- Detects
- Precise
- Proven
No grade · stopped at Runs
Collect2026-09-09
- Runs
- Records
- Detects
- Precise
- Proven
G5 · stopped at Records
Genspark · yt transcript
Fed it two video IDs that do not exist, to see whether it reports the failure
0of2bad inputs where the status signalled failure
Collect2026-09-09
- Runs
- Records
- Detects
- Precise
- Proven
G5 · stopped at Records
Genspark · crawl (on Windows)
Fetch five different kinds of page: an English help doc, a JavaScript-rendered help page, Wikipedia, a government site, and a URL that does not exist
2of4pages with a body where it actually returned the body
Head-to-head2026-09-09
No grade · two tools compared on the same questions
Search tools head to head — Genspark web_search vs Claude Code WebSearch
Put the same three questions (two English, one Korean) to both tools and compared which domains came back on page one
10of30domains both tools returned
Of the 7 graded records, the first stop was Runs (G5) in 2, Records (G4) in 4, Detects (G3) in 1. The 1 comparison record carries no grade.
All 8 recordsHow we test
- Define
- Test
- Measure
- Break
- Verify
- 01
Define the job, including where its output goes
- 02
Set the required grade (external document 1 · reference 2 · internal 3)
- 03
Build specimens: normal, boundary, and ones meant to fail
- 04
Repeat counts are set per item before testing, based on how much results vary and what a run costs.
- 05
Count k of N. Failed runs are never deleted
- 06
Name the rung where it stopped
- 07
Compare the required grade with the earned grade
Breaking it is the point.
We do not only ask polite questions. We corrupt headers, hand it zero-byte files, point it at addresses that do not exist, disguise file types, and watch whether “success” comes back. The conditions and the grading rules are in the testing standard and the grading criteria.
How far can your AI be trusted?
If you are building an AI tool or agent, we run it the same way and publish the record. Send a URL or a description and we first check whether, and how far, it can be tested.
It is free for now. It will read like every other record here: no score, no pass mark, just k of N and the point where it stopped. We do not delete the runs that failed. We cannot take everything, and we decide what gets measured and when.