Workflow engineering / Lab
Research arrives. One useful article leaves.
Follow an item from source check to publication, then send it twice. The interesting part is what happens on the second delivery.
The handoff is the design
A recurring researcher and a website publisher need a shared contract. Give each research item a stable identity, retain its public source, check whether the finding is suitable for publication, and record the resulting article. Email can carry an item into that handoff; a direct API can carry a structured article out.
Our live workflow uses a scheduled desktop task and a narrowly scoped authenticated API on Zyel's existing AWS host. This page is a separate learning fixture. It uses no credentials, reads no conversations and publishes nothing to a server.
- 01 · ResearchA source and a stable item identity.
- 02 · ReviewFacts checked; private material held.
- 03 · PublishOne article, with a recorded receipt.
- 04 · VerifySaved version and public result agree.
Try the awkward delivery
The browser fixture starts with a fictional item. A ready item can publish. An unchecked source or private note is held. Send the ready item again to see an idempotent replay, then try changing its content under the original request identity.
Published in this simulation: 0
Ready. No network request will be sent.
Keep identity separate from an attempt
A research identity describes the finding. A request identity describes a particular publication payload. An article identity describes its permanent public address. A retry uses the same request identity and should return its original receipt. A correction uses the same article, a new request identity and the expected current revision.
That distinction is why a transport timeout should not immediately create another article. First recover the original result. In the live implementation, an exact replay returns the saved receipt, a conflicting payload is rejected, and stale revisions cannot overwrite a newer article.
Five examples, one complete fixture
These source excerpts come from publisher.py in the independent package. They use Python's standard library and an in-memory SQLite database. The browser simulation illustrates equivalent decisions; it does not execute this Python code.
01 · Give content a repeatable fingerprint
Stable serialization lets the sender recognize the exact same content. The fixture hashes sorted JSON; production clients must use the canonicalization contract agreed with their API.
def fingerprint(article):
encoded = json.dumps(article, sort_keys=True, separators=(",", ":"))
return hashlib.sha256(encoded.encode("utf-8")).hexdigest()Source excerpt: complete publisher.py and tests in the source package
02 · Hold unchecked and private research
The learning fixture requires a source check and a public visibility decision before an item can enter the article table. A real source check means reading and assessing the original material.
def validate(article):
if article.get("visibility") != "public":
raise ValueError("Private research stays out of public articles")
if not article.get("source_checked"):
raise ValueError("Check the original source before publishing")
if not article.get("item_id") or not article.get("title"):
raise ValueError("A stable research identity and title are required")Source excerpt: complete publisher.py and tests in the source package
03 · Put research identity in a unique constraint
The synthetic database makes each research item a primary key and stores publication receipts separately. That preserves one article identity across repeated deliveries.
def database():
db = sqlite3.connect(":memory:")
db.execute("CREATE TABLE articles (item_id TEXT PRIMARY KEY, digest TEXT, title TEXT, revision INTEGER)")
db.execute("CREATE TABLE receipts (request_key TEXT PRIMARY KEY, digest TEXT, item_id TEXT, revision INTEGER)")
return dbSource excerpt: complete publisher.py and tests in the source package
04 · Commit the article and receipt together
The complete publish function checks a prior receipt, rejects changed content under the same request key and requires the expected article revision. The database transaction covers both writes.
def publish(db, article, request_key, expected_revision=0):
digest = fingerprint(article)
with db:
receipt = db.execute("SELECT digest, item_id, revision FROM receipts WHERE request_key = ?", (request_key,)).fetchone()
if receipt:
if receipt[0] != digest:
raise ValueError("A request key cannot describe different content")
return {"item_id": receipt[1], "revision": receipt[2], "replayed": True}
validate(article)
previous = db.execute("SELECT revision FROM articles WHERE item_id = ?", (article["item_id"],)).fetchone()
current_revision = previous[0] if previous else 0
if current_revision != expected_revision:
raise ValueError("Read the current revision before changing an article")
revision = current_revision + 1
db.execute("INSERT INTO articles VALUES (?, ?, ?, ?) ON CONFLICT(item_id) DO UPDATE SET digest=excluded.digest, title=excluded.title, revision=excluded.revision", (article["item_id"], digest, article["title"], revision))
db.execute("INSERT INTO receipts VALUES (?, ?, ?, ?)", (request_key, digest, article["item_id"], revision))
return {"item_id": article["item_id"], "revision": revision, "replayed": False}Source excerpt: complete publisher.py and tests in the source package
05 · Read the result back
The verifier compares the stored article revision with the returned receipt. In production, verify the API result and the public page before reporting that publication is complete.
def verify(db, receipt):
row = db.execute("SELECT title, revision FROM articles WHERE item_id = ?", (receipt["item_id"],)).fetchone()
if row is None or row[1] != receipt["revision"]:
raise ValueError("Saved article does not match the publication receipt")
return {"title": row[0], "revision": row[1], "verified": True}Source excerpt: complete publisher.py and tests in the source package
Run and interpret the source package
Download and extract the source. Open a terminal in the extracted directory and run python -m unittest -v. Four tests exercise replay, conflicting content, unchecked or private research and stale revision protection. Each test uses a fresh in-memory database that disappears when it closes.
The fixture's passing result establishes those local behaviors. It does not prove that an arbitrary publishing API is authenticated, sources are true or a public website rendered successfully. Those are separate production checks. A useful implementation retains its draft until the public result can be verified.
What scheduling can and cannot promise
A desktop schedule can use the workstation's authorized tools while the computer and app are available. It has no end date, but it still has operating dependencies. An independent cloud worker would need its own durable source handoff, service credentials, inference configuration, monitoring and budget. A schedule by itself is not a guarantee of uninterrupted operation.
For a production queue, retain unresolved drafts, retry safely after temporary failures and report a concrete blocker when an operator's decision is needed. Keep private comments and article state outside the web release so that a code deployment cannot erase them.
Read next
Why repeated automation needs stable identity · Zyel's editorial approach · Codex to Scrum sync · Official scheduled-task documentation