Build the safeguards
Eight explained steps, a source example and a checkpoint at each decision.
Read the tutorialPython / FastAPI / AI evaluation
Run it. Change it. Verify it.
Start in your browser, then take the complete Python Evidence Evaluation Lab into an independent local environment.
Try it without installing a toolchain
- Open the playground and keep its default scenario.
- Run the comparison and read the results before opening the code.
- Change the scenario, run again, and identify which decision changed.
- Open View the code, choose a source file and read the explanation above it.
- Reset the environment, including a run that is still pending.
Run your downloaded copy
Download and extract the ZIP into a new folder. With Python 3 installed, open a terminal in that folder and run the command below. It serves only this package on loopback; opening index.html directly is insufficient for workers and WebAssembly.
python preview.pyOpen http://127.0.0.1:4210/ in your browser. Stop the server with Ctrl+C. The prebuilt preview needs no dependency installation and makes no automatic requests to another site. The source code and notices are included alongside it.
Run the native exercise
The commands below start in the extracted package root. Node-based examples require Node 24 or later; the Go adapter requires Go 1.27 or later; the Python adapter requires Python 3.11 or later. Install only the toolchain used by this lab. Exact dependency versions and the Node lockfile are in the download.
python -m venv .venv
# Activate .venv for your operating system, then:
python -m pip install -r requirements.txt
cd src/python
python -m unittest -v
python api.pyThe safeguarded evidence run matches all five expected fixture decisions. The slow run limits active tasks to two and cancels all five before returning. Active-at-finish is zero. These labels are regression cases, not an accuracy estimate.
Read GETTING_STARTED.md for the native request examples and shutdown steps. Native servers bind to 127.0.0.1. The optional monitoring containers expose only loopback ports. They are a separate exercise from the browser preview.
Make one meaningful change
Add a new policy version and a fixture whose citation is valid but whose numerical claim is wrong. Make the evaluation fail before changing the validator. For live-model work, create a separate adapter and record provider usage rather than inventing token cost.
Add the expected result first, reproduce the failure, repair it, then rerun the native and browser checks. Use npm run rebuild for the Node/browser examples. The Go package includes rebuild_wasm.py; the Python package includes refresh_source.py.
Record what you learned
Capture the scenario, edition, input records, observed outcome and the test that checks it. Explain one alternative you rejected and why. Avoid converting a small local result into a throughput, security or accuracy claim about a production system.
Read the primary documentation
Ascertain describes Python/FastAPI engineering for its healthcare AI platform; this evaluation curriculum is a proposed extension of that work.
Choose your next step
Compare the alternatives
Understand where each pattern helps and which tradeoffs remain.
Compare approachesRun and extend it locally
Download the independent environment, run the native checks and try a specific extension.
Open the how-to guideLab discussion / Python Evidence Evaluation Lab
What would you try next?
Ask about the code, compare an approach, or describe an edge case. Reader comments are reviewed before publication.
Code walkthrough
Follow the implementation in code.
These short excerpts connect the lab’s behavior to its implementation. Each example links to the full tutorial step for its context, tradeoffs and checkpoint. Names and dependencies belong to that source; the excerpts are not standalone programs.
Keep retrieval separate from generation
A simple token-overlap retriever ranks the invented documents. The candidate answers are recorded fixtures, so the same inputs can be tested repeatedly without a provider. Retrieval relevance, source authority and answer support are different questions; none is guaranteed by the others.
docs = retrieve(question, DOCUMENTS)Reject an invalid output shape
The validator requires an exact set of fields, a nonempty list of citation strings and a true integer period. Python booleans are a special case because bool subclasses int; the example uses an exact type check. A valid schema is a useful boundary, but it cannot establish a true claim.
type(answer["return_days"]) is intCheck authority, version and the actual claim
The active policy allows 14 days; the superseded policy allowed 30. A citation must identify an active relevant record, and the structured claim must match it. The text check is deliberately exact. General natural-language support would require a broader design and separately evaluated failure modes.
allowed = {d["id"]: d for d in docs if d.get("active") and "days" in d}