AI Lab · Generative Systems
Collaboratively designed and built a production generative workflow for an early-stage wealth-tech team.
The system started as a draft-generation workflow, but production use exposed a harder product problem: controlling how context, editorial feedback and persistent rules behave as the system scales.
My focus was the AI workflow architecture — context design, quality loops, deterministic controls, human-feedback memory, and the reliability layer governing provenance, conflicts, freshness and regressions.
31+
Live articles published
8–12
Long-form articles / week · pipeline capacity
≤3
Writer–Critic iterations
2
Publishing variants
A prototype could turn a topic into a credible draft. But production use exposed a different class of problems: instructions accumulated, retrieved context became noisy, human edits could contradict previous rules, and exact publishing requirements were still being delegated to probabilistic models.
Naive
Mature system
The product problem shifted from “Can the model write?” to “Can the system remain predictable as context and feedback accumulate?”
The system deliberately separates different kinds of context instead of feeding one giant prompt. Each context type has a different scope and lifetime. Keeping them separate reduces instruction drift and prevents a one-off editorial correction from silently becoming permanent system behavior.
Long-lived facts and positioning.
Voice and editorial patterns.
Relevant source material selected for the task.
Reusable rules learned from prior human feedback.
Instructions specific to the current article or edit.
More context is not always better context.
I didn't maximize agent count. I separated responsibilities so different failure modes could be controlled independently — each stage with a clear responsibility, input contract and fallback behavior.
Topic selection + duplication avoidance
Draft generation + critique
SEO metadata + deterministic validation + exact publishing rules
Human review + targeted local edits
Feedback reusability + persistent-rule governance
The implementation uses specialized pipeline stages, but the design goal was responsibility isolation rather than agent count.
Platform outputs
Platform outputs — Website + newsletter variants share editorial state, with targeted edit propagation and duplicate-processing protection.
Each design decision below was a direct response to a failure mode observed in use — not a theoretical concern.
Failure mode
Prompt / instruction drift
Design response
Separate stable context, retrieved evidence and persistent rules.
Failure mode
Context overload
Design response
Retrieve bounded relevant chunks rather than injecting the entire corpus.
Failure mode
Local feedback polluting global behavior
Design response
Gatekeeper decides whether feedback is local or reusable.
Failure mode
Conflicting persistent rules
Design response
Conflict governance classifies proposed rules as safe, duplicate, conflict or needs review before memory mutation.
Failure mode
Exact requirements handled probabilistically
Design response
URLs, metadata limits, formatting and publishing invariants enforced in code.
Failure mode
Invalid structured output
Design response
Runtime schema validation before downstream use.
Failure mode
Ambiguous reviewer instructions
Design response
Escalate to Needs Clarification instead of guessing.
Failure mode
Stale context
Design response
Track source lifecycle and freshness; inactive or superseded sources are excluded or deprioritized.
Failure mode
Model / prompt regressions
Design response
Offline regression suite tests critical reliability dimensions before changes are trusted.
More context is not always better context.
Move certainty into code.
Ambiguity is a workflow state, not permission to guess.
Don't let a local correction silently become global behavior.
Every draft passes through a Writer–Critic loop before it reaches human review.
Generates the draft.
Evaluates quality and returns a score.
Feedback sent back to Writer for a targeted rewrite.
Loop bounded to prevent unproductive cycling.
Score ≥ 7 continues to validation and human review.
The critic reduces single-pass quality risk. It is not an independent truth-verification system — human review remains the final authority.
Models handle probabilistic judgment. Code enforces deterministic constraints. Humans resolve ambiguity.
Model judgment
●Topic generation
●Drafting
●Critique
●Semantic edits
●Feedback reusability
●Semantic rule-conflict assessment
Code / system control
●Canonical URLs
●Slug construction
●Metadata limits
●Structured schema validation
●Status transitions
●Duplicate-processing protection
●Deterministic publishing rules
●Audit records
Human authority
●Ambiguous editorial direction
●Conflicting persistent policies
●Final publication decision
●Unresolved quality judgment
Human feedback should improve the workflow without silently rewriting global behavior. Ambiguity therefore becomes a workflow state, and persistent memory changes require an explicit governance decision.
Flow A — Ambiguity
Primary lifecycle
Ambiguity branch
Ambiguity is a workflow state, not permission to guess.
Flow B — Memory
A reviewer leaves feedback on a draft.
The Editor rewrites the affected paragraph locally.
The Gatekeeper classifies the feedback.
Proposed rules checked against existing persistent memory.
Local
Keep change article-specific.
Reusable + Safe
Promote to persistent editorial memory.
Conflict / Unclear
Escalate for human review.
Persistent memory is a product surface, not a dumping ground.
As the source corpus and persistent editorial memory grew, the next failure modes were no longer about generation quality. They were about deciding which context to trust, whether rules conflicted, whether knowledge was stale, and whether system changes introduced silent regressions.
01
Retrieved context now carries metadata about where it came from, its lifecycle state, authority and freshness.
02
Reusable feedback is checked against existing persistent rules before memory is mutated. Conflicts are surfaced rather than automatically resolved.
03
Active, authoritative and relevant context should beat content simply because it is newer.
04
A fixed offline evaluation suite checks critical behavior when prompts, models or system logic change.
Retrieval priority
Newer is not automatically better.
The system already validated output structure. The next step was validating behavior across changes.
10 / 10
Unit tests passing
13 / 13
Offline regression evals passing
Automated CI
Runs on pull requests
These are reliability signals, not proof of semantic correctness for every generated article.
Generate
Topic → Writer → Critic → SEO → Notion
Learn
Human micro-edits + reusable feedback extraction
Control
Deterministic publishing invariants + lifecycle hardening
Validate
Structured schemas + modular prompts + safer model outputs
Govern
Source provenance + rule conflict checks + freshness-aware retrieval + regression safety
Strategic progression
Each architecture change followed a new failure mode becoming visible in use.
31+
Live articles published
8–12
Long-form articles / week · pipeline capacity
2
Publishing variants
Human approval
Retained before publication
The outcome wasn't simply faster generation. It was a repeatable editorial operating model in which creative judgment remained probabilistic while exact requirements, memory changes and ambiguous decisions were increasingly governed.
Public contribution trail
Architecture and reliability improvements were contributed through a reviewed GitHub pull request and merged into the shared repository.
Retrieved material is bounded and governed, but the system does not yet provide a complete trust or sanitization layer for arbitrary untrusted documents.
The critic checks quality and consistency; it does not independently verify every factual claim against authoritative external sources.
Rule-conflict detection reduces unsafe memory mutation, but no semantic classifier can guarantee detection of every linguistic contradiction.
Offline regression evaluation catches known failure patterns, but continuous production monitoring of model-version drift is not yet implemented.
The next maturity step is moving from context governance toward continuous trust and evaluation.
01
02
03
04
05
The system started as a generation problem. In practice, the harder product challenge became governing how context, memory and human authority interact over time. That shift — from output generation to system governance — is the part of building AI products I find most consequential.