Python / FastAPI / AI evaluation

The answer sounds good. Can it pass the test?

An answer cites a document. The document is old, unrelated or hostile. Check the actual claim, then make a slow dependency reveal whether the workflow respects its limits.

Try the experiment Learn how it works

The idea in plain language

An AI-facing workflow needs a decision about whether an answer is supported, not just a string to display. A repeatable evaluation also makes abstention, invalid output and time limits visible.

Actual Python runs in the open-source Pyodide runtime. Candidate answers and documents are deterministic fixtures; no live model is called. The exact policy validator is intentionally narrow and does not establish general factual accuracy or comprehensive prompt-injection resistance.

Your independent testing environment

Make the failure visible.

Start with evidence checks. Compare the five fixture decisions. Then inspect the hostile-document case and slow-task cancellation. The download runs the same engine behind a local FastAPI endpoint.

Scenario

Evaluate supported, unsupported, superseded, hostile and malformed candidate answers.

Ready. Choose an example and run both implementations.

Explain the result

The safeguarded evidence run matches all five expected fixture decisions. The slow run limits active tasks to two and cancels all five before returning. Active-at-finish is zero. These labels are regression cases, not an accuracy estimate.

Open the exact source behind this edition. The viewer checks each file against its SHA-256 manifest and explains the decision it implements.

Read this extracted package’s setup

Choose your next step

Build the safeguards

Eight explained steps, a source example and a checkpoint at each decision.

Read the tutorial

Compare the alternatives

Understand where each pattern helps and which tradeoffs remain.

Compare approaches

Run and extend it locally

Download the independent environment, run the native checks and try a specific extension.

Open the how-to guide

Before you try it

Do I need to know how to code?

Start with Run and read the comparison. The tutorial introduces the code after you have seen the behavior. The how-to guide separates a no-install route from the native developer exercise.

What happens to the experiment when I reset?

Reset discards this lab’s temporary state and cancels or terminates its pending work. These examples use invented records. No account, production data or cloud deployment is needed.

What does the browser experiment establish?