AI Dad

How far can you trust
the AI tool you work with?

If a number an AI produced is going into a document you send a client, you need to know how far that tool can be trusted before it leaves. We put the tool to work ourselves and test whether it reaches the level its destination requires.

Five rungs, climbed from the bottom: does it run, does it record honestly, does it catch, is it precise, is it proven. Set the grade it earned against the grade its destination demands, and you have your answer.

The more AI you use, the more there is to check

At first you can read every AI-written document yourself. Once several people are producing dozens a day, checking by hand stops keeping up. And an internal summary does not need the same scrutiny as a quote going out to a client.

That calls for a line: which tool's output is good enough for which job. Knowing what went unchecked, what it missed, and whether it cries wolf on clean documents is what lets you decide where a human still has to look.

How to read a record

We mark each task on five rungs. They say how far we got on that task — black for a rung it passed, red for the point where it stopped, blank for a rung we have not tested. Leave the blank ones out of your judgement.

  1. 1

    Runs

    It runs. Install it, run it, something comes out.

  2. 2

    Records

    It leaves a trace you can check later.

  3. 3

    Detects

    It says when something is wrong instead of passing silently.

  4. 4

    Precise

    Same input, same answer. It does not drift.

  5. 5

    Proven

    Verified on the real thing — output seen with our own eyes.

Two grades decide whether you can use it

Every tool carries two grades side by side: the grade its destination demands, and the grade it has actually earned in testing.

The required grade comes from where the output goes. If a number the tool produced lands in a document that leaves the building, that is grade 1. Reference material is grade 2, internal use grade 3. Nobody gets to lower it for themselves.

The earned grade moves only on test results. Rungs are climbed in order, never skipped. And when the code changes, the earned grade is void — measurement starts over.

Usable = earned grade ≥ required grade

Grade 1 is the highest. A tool that earned grade 2 can feed reference material, but not a number in a document going to a client. It is a calculation, not a judgement call made case by case.

Every report uses the same form, the one built for grade 1. Tests we did not run stay on the page marked “not run” rather than being trimmed away, because a trimmed report reads as a complete one.

Right now a person runs every test

We built this grading system to vet the tools we bring in ourselves, and we still run each test by hand.

What we are building is an agent that runs this verification for you, because hand-checking stops scaling. That agent is still a goal, though. There is no product to buy yet.

Records measured on this scale

7 records are public so far. Each one holds only for the task and the conditions we ran it under, so compare those against your own work.

Most recent

Genspark · batch_understand_videos

Grade 3

Usable, but it does not always answer all six questions — roughly one in six goes missing.

Read 100 YouTube transcripts and answer the same six questions per video

  1. Runs
  2. Records
  3. Detects
  4. Precise
  5. Proven

496 / 600questions answered

104 questions never appeared in the output

What failed 104 of 600 questions never appeared in the output. It fails silently. If you match answers by position, they shift into the wrong question — we made exactly that mistake once and caught it.

Not measured We did not re-run the same videos to check whether answers repeat (precision).

2026-09-09 · gsk CLI, as of 2026-09-09

Genspark · web_search

Grade 2

Works in both Korean and English. All 25 returned results, and the same keyword gives the same answer on a repeat run.

Pull the first page of search results for 25 keywords

  1. Runs
  2. Records
  3. Detects
  4. Precise
  5. Proven

25 / 25keywords answered

What failed It does not return the ten results asked for — between six and ten depending on the keyword (stable within a keyword). It never says why the rest are missing.

How we measured How we measured precision: three keywords (two English, one Korean), three runs each, back to back on the same day, comparing the ranked list. All nine comparisons matched exactly, order included. Every run returned nine results.

Not measured We only repeated within a single day. How far results drift across days is still unknown. We also did not open the ranked pages to confirm they are what they claim (the Proven rung).

2026-09-09 · gsk CLI, as of 2026-09-09

Codex CLI · GPT-6 Astra

Grade 2

Good output — every claim linked to an official doc, unknowns marked as uncertain. But the first run hung.

Write an ad-revenue research report using only Naver and Google official docs

  1. Runs
  2. Records
  3. Detects
  4. Precise
  5. Proven

1 / 2runs succeeded

run 1 hung waiting on stdin

What failed Run 1 hung waiting on stdin — no error, it simply never finished. Closing stdin (`< /dev/null`) made run 2 exit cleanly.

Not measured We did not ask the same question twice to see if the conclusion holds.

2026-09-09 · codex exec, -m gpt-6-astra

TwelveLabs · Gemini MCP servers

No grade · stopped at Runs

Did not connect. Both servers were closed when the session started.

Connect for video understanding and cross-checking

  1. Runs
  2. Records
  3. Detects
  4. Precise
  5. Proven

What failed Both returned CONNECTION_CLOSED. This is a connectivity failure, not a capability one — read it as "unusable in our setup that day", not "a bad tool".

Not measured With no connection, nothing about its actual behaviour could be measured.

2026-09-09 · auto-connected at session start

Genspark · yt transcript

Grade 2 · stopped at Detects

It answers "success" for videos that do not exist. The body does say there is no transcript, but the status field and the exit code both report success.

Fed it two video IDs that do not exist, to see whether it reports the failure

  1. Runs
  2. Records
  3. Detects
  4. Precise
  5. Proven

0 / 2bad inputs where the status signalled failure

both runs reported success

What failed Both runs returned status=ok, message=success, exit code 0. Where the transcript should be, there is a 23-character line: "No transcript available". Anything that trusts the status and passes that string along will treat it as a transcript — which is exactly what happened to one video in our 100-video run, caught only by reading the output.

How we measured One nonexistent ID (zzzzzzzzzzz) and one malformed ID (not-a-real-id-xxxxx), checking the exit code, status, message, and body separately.

Not measured This test says nothing about whether transcripts of real videos are accurate (the Precise and Proven rungs).

2026-09-09 · gsk CLI, as of 2026-09-09

Genspark · crawl (on Windows)

Grade 2 · stopped at Detects

It reports a missing page as a genuine failure. But on a JavaScript-rendered page it returns only the navigation chrome and still calls it a success. And on Windows, a URL containing & produces the wrong exit code.

Fetch five different kinds of page: an English help doc, a JavaScript-rendered help page, Wikipedia, a government site, and a URL that does not exist

  1. Runs
  2. Records
  3. Detects
  4. Precise
  5. Proven

2 / 4pages with a body where it actually returned the body

2 URLs returned no body — one gave only menus, one failed after 21 seconds

What failed Two things. (1) The Naver help page returned 586 characters, all of it menus and footer — the article body was absent, and status still said ok. The caller believes it has the page. The government site failed after waiting 21 seconds. (2) On Windows, gsk.cmd is a batch file, so cmd.exe re-parses the arguments. Everything after & in a URL is handed on as a separate command, producing "'osType' is not recognized as an internal or external command" and an exit code of 1 — while the response itself is fine. Reproduced three times; encoding & as %26 removes it. Anything checking exit codes will count a healthy request as a failure. Since characters from the URL reach the shell, it is better not to pass untrusted URLs straight through.

How we measured One fetch per URL, recording exit code, status, body length, and elapsed time separately. For the & problem: the same URL three times, then the same URL with & encoded as %26 and a version without & at all, to isolate the cause.

Not measured We did not repeat on another day (precision), nor compare the fetched text against the live page word by word (the Proven rung).

2026-09-09 · gsk CLI, as of 2026-09-09, on Windows

Search tools head to head — Genspark web_search vs Claude Code WebSearch

Head to head

Same question, and two thirds of the answer differs. Neither tool alone can tell you what page one looks like.

Put the same three questions (two English, one Korean) to both tools and compared which domains came back on page one

10 / 30domains both tools returned

Where they differ The differences run in a clear direction. Genspark gave eight of twenty-eight slots to Reddit and Quora; WebSearch gave one of twenty-five. The Korean question splits them further — Genspark returned five Naver blog slots plus AdPost's own help page, while WebSearch returned no Naver blogs at all and spent four of nine slots on Google Ad Manager documentation about native ads, which has nothing to do with the question.

How we measured One run per question per tool, comparing only the set of domains on page one. Rank and titles were ignored — the question was simply which sites appeared. Repeat appearances of a domain counted once.

Not measured We did not judge which set is more useful. This record only says the two disagree. Whether either tool repeats itself on another day is a separate question.

2026-09-09 · gsk CLI vs Claude Code's built-in WebSearch

We can measure yours too

If you are building an AI tool or agent, we can run it the same way and publish the record. It is free for now. It will read like every other record here — no score, no pass mark, just k of N and the point where it stopped. We do not delete the runs that failed.

If that works for you, send the tool and the job you want it tested on through contact. One person runs this, so we cannot take everything, and we decide what gets measured and when.