Intake
Capture the source, requested action, deadline and evidence required to close the task.
Case 02 / AI product operations / 2026
Operational QA and Human Review
A release-control model for AI-assisted work where a plausible output is not enough: the system must know the current state, block unsafe actions and prove what was actually released.
- Role
- Product operator · QA owner
- Scope
- Intake to verified release
- Controls
- State, approval, deterministic QA, evidence
- Status
- Applied operating model
AI can finish the wrong task convincingly.
The dangerous failures are often operational, not linguistic: acting on stale information, submitting twice, using the wrong file, treating a draft as approved or declaring completion without checking the exact artifact.
The system was designed around one rule: confidence in the output cannot substitute for evidence about the state and the action.
Six stages from request to release.
AI assists with synthesis, drafting and comparison. Ownership, approval and release stay explicit.
Artifact AControlled operating flow
Resolve state
Check the authoritative record, prior actions, dependencies and whether the request is still current.
AI-assisted execution
Use models for research, comparison, drafting, transformation and repetitive review.
Human approval
Inspect context, claims, risk, visual quality and the irreversible action before execution.
Evidence and release
Verify the exact final artifact and record what happened in the authoritative state.
Learn
Turn defects and near-misses into deterministic checks, policy and reusable review rules.
One authoritative state prevents accidental action.
Notes, chat messages and old drafts are not allowed to redefine what has already happened. A single ledger records prepared, approved, submitted, rejected and closed states.
Artifact BState model
| State | Meaning | Allowed next action |
|---|---|---|
| Researched | Opportunity assessed; no recruiter-facing material is approved. | Prepare or reject. |
| Materials ready | Exact assets exist but have not been submitted. | Review live form and exact files. |
| Approved | Human owner has approved the specific external action. | Execute once. |
| Submitted | Action completed with confirmation evidence. | Follow up; never resubmit automatically. |
| Closed | Rejected, withdrawn, expired or otherwise complete. | Archive and learn. |
The approval boundary is attached to the action.
“Human in the loop” is too vague. The system names the irreversible action, the evidence the person must inspect and the exact moment automation must stop.
Prepare without executing
AI may research, draft, compare and populate a review package. It may not infer permission to publish, submit or message.
Review the live surface
The human checks the current form, current recipient, uploaded filename and any new question that was not present in the preparation data.
Approve one exact action
Approval applies to the specified artifact and route. It does not remain open-ended for future variants.
Review the exact artifact, not the source that produced it.
A correct source file can produce a broken PDF, clipped page or wrong hyperlink. A green build can still produce a visually weak page. Release QA must inspect the delivered object.
Artifact CExact-artifact gate
| Layer | Check | Evidence |
|---|---|---|
| Deterministic | File count, dimensions, page count, required links, filename and expected checksums. | Machine-readable test output. |
| Visual | Hierarchy, clipping, collisions, mobile layout and recruiter-facing readability. | Exact screenshots or rendered pages. |
| Semantic | Claims, freshness, action state and whether the output answers the real request. | Human review record. |
| External | Production route, HTTPS, final URL, uploaded file and confirmation state. | Live-surface evidence. |
The system produces reviewable traces.
Evidence is not a decorative appendix. It is how the team proves which version was checked, which action was approved and whether the result matches the decision.
Authoritative ledger
Current state, role, route, freshness check, duplicate protection and next permitted action.
Branch and PR gate
Risky changes are isolated, tested and reviewed before they can affect production.
Visual QA artifacts
Desktop, tablet, mobile and exact-file previews expose failures that tests cannot describe.
Release record
Commit, deployment, exact URL, artifact identity and confirmation evidence remain traceable.
The release gate caught four concrete defects before production.
This operating model was applied to the portfolio redesign itself. Automated checks found structural failures; exact screenshots and human review found problems that a green build could not.
Artifact DPre-release defect log
| Defect | Evidence | Release decision |
|---|---|---|
| Mobile overflow | The case study measured 996 px wide inside a 390 px viewport. | Block release; repair shrinking and mobile table behavior. |
| Broken title wrap | “Production” split inside the word on the phone screenshot. | Adjust mobile typography and re-run exact screenshots. |
| Weak media state | The showreel rendered as a black frame and weakened the proof-led direction. | Remove it from the final release candidate rather than decorate around it. |
| Invalid contact route | The public CTA linked to an email address the owner did not control. | Replace it with the verified Gmail route and show the neutral label “Email”. |
The release remains gated until CI, responsive QA, exact screenshots and final human approval all agree on the same candidate.
Control is valuable only when the claims stay honest.
This case shows an applied operating model across portfolio delivery and career-operations workflows. It does not claim enterprise-scale deployment, fully autonomous execution or externally measured productivity gains.
- DemonstratedApproval gates, state management, branch/PR workflow, deterministic checks and exact-artifact review.
- Not claimedLarge-team adoption, audited reliability percentages or commercial SaaS performance.
- Next validationMeasure defect escape rate, duplicate prevention, review time and recovery performance across a larger live workflow set.
Automation is useful when the system knows when not to act.
The product is not the prompt or the agent. It is the combination of state, evidence, boundaries and accountable decisions that makes the output safe enough to use.
The next maturity step is not “more autonomous”. It is better observability: faster detection of stale state, clearer escalation and evidence that the control actually prevents defects.