By Chason ArthurRevision 157.024Published Updated

Python / FastAPI / AI evaluation

Run it. Change it. Verify it.

Start in your browser, then take the complete Python Evidence Evaluation Lab into an independent local environment.

Try it without installing a toolchain

  1. Open the playground and keep its default scenario.
  2. Run the comparison and read the results before opening the code.
  3. Change the scenario, run again, and identify which decision changed.
  4. Open View the code, choose a source file and read the explanation above it.
  5. Reset the environment, including a run that is still pending.
Open the playground

Run your downloaded copy

Download and extract the ZIP into a new folder. With Python 3 installed, open a terminal in that folder and run the command below. It serves only this package on loopback; opening index.html directly is insufficient for workers and WebAssembly.

python preview.py

Open http://127.0.0.1:4210/ in your browser. Stop the server with Ctrl+C. The prebuilt preview needs no dependency installation and makes no automatic requests to another site. The source code and notices are included alongside it.

Run the native exercise

The commands below start in the extracted package root. Node-based examples require Node 24 or later; the Go adapter requires Go 1.27 or later; the Python adapter requires Python 3.11 or later. Install only the toolchain used by this lab. Exact dependency versions and the Node lockfile are in the download.

python -m venv .venv
# Activate .venv for your operating system, then:
python -m pip install -r requirements.txt
cd src/python
python -m unittest -v
python api.py

The safeguarded evidence run matches all five expected fixture decisions. The slow run limits active tasks to two and cancels all five before returning. Active-at-finish is zero. These labels are regression cases, not an accuracy estimate.

Read GETTING_STARTED.md for the native request examples and shutdown steps. Native servers bind to 127.0.0.1. The optional monitoring containers expose only loopback ports. They are a separate exercise from the browser preview.

Make one meaningful change

Add a new policy version and a fixture whose citation is valid but whose numerical claim is wrong. Make the evaluation fail before changing the validator. For live-model work, create a separate adapter and record provider usage rather than inventing token cost.

Add the expected result first, reproduce the failure, repair it, then rerun the native and browser checks. Use npm run rebuild for the Node/browser examples. The Go package includes rebuild_wasm.py; the Python package includes refresh_source.py.

Record what you learned

Capture the scenario, edition, input records, observed outcome and the test that checks it. Explain one alternative you rejected and why. Avoid converting a small local result into a throughput, security or accuracy claim about a production system.

Read the primary documentation

Ascertain describes Python/FastAPI engineering for its healthcare AI platform; this evaluation curriculum is a proposed extension of that work.

Related role from the job review

Choose your next step

Build the safeguards

Eight explained steps, a source example and a checkpoint at each decision.

Read the tutorial

Compare the alternatives

Understand where each pattern helps and which tradeoffs remain.

Compare approaches

Run and extend it locally

Download the independent environment, run the native checks and try a specific extension.

Open the how-to guide