Human-in-the-loop review for every AI output

Certified reviewers validate text, audio, image, code, and translation before it reaches your users. One API call. 24-hour turnaround.

< 24hr turnaround 5 content types HMAC-signed records
Python Node.js curl JSON webhooks
Approved by human Example

Confident tone, but wrong policy cited. Corrected before delivery.

AI output: You can return this item within 60 days. Our policy allows full refunds on all electronics. Reviewed: Return window is 30 days for electronics. Restocking fee applies after 14 days.
rev_cs_012 · Customer support specialist
42 tokens · SLA met
sha256:7b3e…a8f2 · HMAC verified · DPO export
Every review record is timestamped and exportable; webhooks are HMAC-signed.
<24hAvg turnaround
5Content modalities
HMACSigned webhooks
$0.015Per token PAYG

One API call between your AI and your users

No new dashboard to log into. No infrastructure to manage. Just send your AI output to our API, and we route it to a certified reviewer. The verified result comes back to your webhook.

Works with any language or framework
Python & Node.js SDKs included
Idempotent, HMAC-signed, batch-ready
Sandbox mode — test before you go live
submit_task.sh
# Submit any AI output for human review
curl -X POST https://api.verifiedworkflows.com/v1/tasks \
  -H "Authorization: Bearer vw_live_xxx" \
  -H "Content-Type: application/json" \
  -d '{
    "content": "Your AI output here",
    "content_type": "text",
    "callback_url": "https://your-app.com/webhook"
  }'
review.py
from verifiedworkflows import VerifiedWorkflows

client = VerifiedWorkflows(api_key="vw_live_xxx")

task = client.tasks.create(
    content="Your AI output here",
    content_type="text",
    callback_url="https://your-app.com/webhook"
)

# Result delivered to your webhook within 24h
print(task["id"])  # tsk_live_f8a29
review.ts
import { VerifiedWorkflowsClient } from "@verifiedworkflows/sdk";

const client = new VerifiedWorkflowsClient({
  api_key: "vw_live_xxx",
});

const task = await client.tasks.create({
  content: "Your AI output here",
  content_type: "text",
  callback_url: "https://your-app.com/webhook",
});

// Result delivered to your webhook within 24h
console.log(task.task_id); // tsk_live_f8a29
response.json
{
  "id": "tsk_live_f8a29",
  "status": "pending",
  "estimated_turnaround": "24h"
}

Works with your stack

Native connectors for the tools you already use — or drop in the raw API. Your model, your UI, your rules.

n8n

Drag a Verified Workflows node into any workflow. Submit AI output for review, wait for the signed result, and route it downstream — no code.

npm install n8n-nodes-verifiedworkflows
  • Submit Review node (action)
  • Review Completed trigger (webhook)
  • Self-hosted or n8n Cloud
View setup guide
workflow.json
{
  "nodes": [
    {
      "name": "AI Generate",
      "type": "@n8n/n8n-nodes-langchain.openAi",
      "parameters": { "prompt": "Summarize patient labs" }
    },
    {
      "name": "Human Review",
      "type": "verifiedWorkflows.submitReview",
      "parameters": {
        "content": "={{ $json.message.content }}",
        "domain": "medical"
      }
    },
    {
      "name": "Deliver",
      "type": "verifiedWorkflows.waitForResult",
      "parameters": { "task_id": "={{ $json.task_id }}" }
    }
  ]
}
OpenAI
Message a model
Verified Workflows
Submit for Review
HUMAN
Wait for Result
Poll / Webhook
Slack
Send message
VERIFIED

Zapier

Create a Zap that intercepts AI output, submits it for human review, and delivers the verified result — all no-code.

  • Trigger: Review Completed (webhook)
  • Action: Submit Task for Review
  • 7000+ app integrations
View setup guide
zap-config.txt
# Step 1 — Trigger
TRIGGER: New AI Output
  Source: OpenAI / your app
  Output: {{completion}}

# Step 2 — Submit for human review
ACTION: Submit Task (Verified Workflows)
  content:      {{output}}
  content_type: text
  domain:       medical

# Step 3 — Wait for review, then deliver
ACTION: Review Completed (Verified Workflows)
  → verified body: {{result.body}}
  → corrections:    {{result.corrections}}
  → signature:      {{result.hmac}}

ACTION: Send Result (Slack / Gmail / webhook)
1
New AI Output
Trigger · OpenAI
2
Submit for Review
Action · Verified Workflows
HUMAN
3
Send Verified Result
Action · Slack
VERIFIED

Make

Build a visual scenario with Verified Workflows modules. Submit tasks, poll for results, and route verified output to any destination.

  • Create Task module (action)
  • Get Result module (poll/search)
  • Instant trigger (webhook)
View setup guide
scenario.txt
# Make Scenario

Module 1:  HTTP → POST to your AI API
             ↓ response.body
Module 2:  VW → Create Task
             content:      {{1.body}}
             content_type: text
             domain:       general
             ↓ task_id
Module 3:  VW → Get Result (repeat until completed)
             task_id: {{2.task_id}}
             ↓ result.body (verified + signed)
Module 4:  HTTP → Respond to user
             body: {{3.body}}
             hmac: {{3.hmac_signature}}
HTTP
Make a request
Verified Workflows
Create Task
HUMAN
Slack
Send message
VERIFIED

Pipedream

Code-first workflows with managed auth. Submit tasks, handle webhooks, and transform results — all in JavaScript.

npm i @verifiedworkflows/sdk
  • Action component: Submit Review
  • Source: Review Completed webhook
  • Managed API key auth
View setup guide
review.mjs
import { VerifiedWorkflows } from '@verifiedworkflows/sdk';

export default {
  async run({ $, steps }) {
    const vw = new VerifiedWorkflows(
      process.env.VW_API_KEY
    );

    const task = await vw.tasks.create({
      content:      steps.ai_output.$return_value,
      content_type: 'text',
      domain:       'medical',
      callback_url: `https://${steps.trigger.url}/hook`
    });

    return { task_id: task.id, status: 'pending' };
  }
};
1 source ai_generate
export default { async run({ steps }) {
  const output = await openai.chat({ model: "gpt-4" });
  return { content: output };
}}
2 action verifiedworkflows.submit_review HUMAN
const vw = new VerifiedWorkflows(process.env.VW_API_KEY);
const task = await vw.tasks.create({
  content: steps.ai_generate.$return_value,
  content_type: "text",
  domain: "medical"
});
3 action return_result VERIFIED
return {
  body: task.body,
  corrections: task.corrections,
  signature: task.hmac_signature
};

LangChain & Agents

Register Verified Workflows as a tool any agent can call. Also works with CrewAI, LlamaIndex, AutoGen, and LangGraph.

pip install verifiedworkflows-langchain
  • @tool decorator for any agent
  • LangGraph human-in-the-loop
  • CrewAI, LlamaIndex, AutoGen
View setup guide
agent.py
from verifiedworkflows import VerifiedWorkflows
from langchain_core.tools import tool

vw = VerifiedWorkflows(api_key="vw_live_xxx")

@tool
def submit_for_review(content: str, domain: str = "general") -> str:
    """Submit AI output for certified human review.
    Call this before sending AI content to users."""
    result = vw.review(content, domain=domain)
    return result.body  # verified, corrected

# Register with any agent framework
agent = create_agent(model, tools=[submit_for_review])
Agent
LangChain / CrewAI
submit_for_review
@tool
HUMAN
Verified Result
Agent continues
VERIFIED

Dify & Flowise

Import the Verified Workflows OpenAPI spec and instantly add human review as a tool your AI agents can call. No SDK, no code.

  • One-click OpenAPI import
  • Works with Dify agents & workflows
  • Flowise custom tool node
View setup guide
openapi.yaml
# 1. Go to Dify → Tools → Import OpenAPI
# 2. Paste this URL:
#    https://api.verifiedworkflows.com/openapi.yaml

openapi: 3.1.0
info:
  title: Verified Workflows
servers:
  - url: https://api.verifiedworkflows.com/v1
paths:
  /tasks:
    post:
      summary: Submit for human review
      operationId: submitReview
      security:
        - bearerAuth: []

# 3. Add "Submit Review" tool to your agent
# 4. Configure API key auth → done
🤖
LLM
Dify Agent
Generate AI output
Tool
Submit Review
OpenAPI · Verified Workflows
HUMAN
End
Verified Output
Agent continues
VERIFIED

MCP Server

Native Model Context Protocol support. Verified Workflows appears as a tool in Claude Desktop, Cursor, Windsurf, and any MCP-compatible client.

npx verifiedworkflows-mcp-server
  • Claude Desktop & Cursor
  • submit_review, get_result tools
  • Zero-code setup
View setup guide
claude_desktop_config.json
{
  "mcpServers": {
    "verifiedworkflows": {
      "command": "npx",
      "args": ["-y", "verifiedworkflows-mcp-server"],
      "env": {
        "VW_API_KEY": "vw_live_xxx"
      }
    }
  }
}

# Also works with Cursor, Windsurf, Continue,
# and any MCP-compatible client.
MCP Client
Claude / Cursor / Windsurf
tools/call
VW MCP Server
submit_review · get_result
HUMAN
result
Verified Result
Back to client
VERIFIED

Custom API

Drop the raw REST API into any stack. Python and Node.js SDKs included, or call it directly with curl. HMAC-signed webhooks, idempotency keys, batch operations.

  • REST API + Python/Node SDKs
  • HMAC-signed webhook payloads
  • Idempotency keys & batch ops
API reference
submit_task.sh
# Submit any AI output for human review
curl -X POST https://api.verifiedworkflows.com/v1/tasks \
  -H "Authorization: Bearer vw_live_xxx" \
  -H "Content-Type: application/json" \
  -d '{
    "content": "Your AI output here",
    "content_type": "text",
    "domain": "medical",
    "callback_url": "https://your-app.com/webhook"
  }'

# Response (24h SLA)
{
  "id": "tsk_live_f8a29",
  "status": "pending",
  "estimated_turnaround": "24h"
}
POST /v1/tasks
Your App
Submit AI output
API call
VW Verified Workflows
Human Review
Certified reviewer
HUMAN
HMAC webhook
POST your-webhook
Verified Result
Signed payload
VERIFIED

Watch certified reviewers catch what AI gets wrong

Real input/output pairs from production tasks. Every modality — text, audio, OCR, translation, code, and medical — validated by certified humans.

INPUTLLM output via API
{
  "content": "The capital of France is Berlin. The Eiffel Tower was built in 1889.",
  "content_type": "text",
  "review_type": "fact_check",
  "callback_url": "https://api.example.com/webhook"
}
OUTPUTVerified result
{
  "status": "corrected",
  "changes": [
    {
      "original": "The capital of France is Berlin",
      "corrected": "The capital of France is Paris",
      "reason": "Factual error: Paris is the capital"
    }
  ],
  "quality_score": 0.92,
  "reviewer_certified": true
}
AUDIOTTS review · welcome_message.mp3
Waveform 0:00 — 0:16.4
0:04.2
0:12.8
LOW Pronunciation mismatch: "schedule" — should match en-US "sked-jool" 0:04.2
MEDIUM Pacing: pause too short between sentences, sounds rushed at 1× speed 0:12.8
SOURCEinvoice_00234.png
INVOICE #
DATE
VENDOR
TOTAL
$1,234.00
$1,284.00
REVIEWEDField validation
Invoice # INV-00234
Date 2025-01-15
Vendor Acme Corp
Total $1,234.00 → $1,284.00 !
OCR misread 8 as 3. Reviewer verified against source.
AI TRANSLATION EN → ES
El paciente es estable y será dado de alta mañana.

No se detectaron complicaciones durante la observación nocturna.
REVIEWED ES verified
El paciente está estable y será dado de alta mañana.

No se detectaron complicaciones durante la observación nocturna.
Correction: "ser" → "estar" for medical status — "estar" reflects temporary conditions, critical for clinical context.
ORIGINAL query.py
CRITICAL
def get_user(id):
query = f"SELECT * FROM users WHERE id = {id}"
return db.execute(query)
FIXED query.py
SAFE
def get_user(id):
query = "SELECT * FROM users WHERE id = %s", (id,)
return db.execute(query)
PATIENT REPORT task #med-0247

Clinical summary

Patient reports chest pain radiating to left arm, shortness of breath for 2 hours. BP 140/90, HR 98. Suspected acute coronary syndrome (ACS). High-sensitivity troponin ordered.
Annotation 1
Use standard abbreviation on first mention — "acute coronary syndrome (ACS)" ensures clarity for all clinical readers.
Annotation 2
Specify assay type — "High-sensitivity troponin" is more specific than "Troponin ordered" for clinical accuracy.
Run your own content through a reviewer. No API key required — see the verdict in seconds.
Try it yourself →

The last human check before your AI reaches a real user.

Every output — signed, audited, exportable

Every reviewer is credentialed. Every decision is traceable.

Skill-gated routing to domain reviewers, HMAC-signed webhook delivery, and exportable review records on every task.

SOC 2 program Type II in progress
Enterprise HIPAA BAA on request
GDPR tools Export & delete APIs
End-to-end encryption TLS 1.2+ in transit, AES-256 at rest
Certification-gated routing Medical, legal, code, and audio tasks auto-route to reviewers with matching skills.
HMAC-signed webhooks Cryptographic signature on every callback. Verify payload integrity.
Full audit trail Reviewer ID, timestamp, verdict, and rationale. Exportable JSON for compliance workflows.
Consensus voting High-stakes tasks route to multiple reviewers. Disagreements escalate to senior arbitrators.
Audit trail — task #vw-88421
14:02:11receivedPOST /v1/tasks
14:02:11routed→ rev_cs_012 · customer support specialist
14:18:44reviewedverdict: corrected · consensus 1/1
14:18:44signedsha256:7b3e…a8f2
14:18:45deliveredPOST your-webhook · 200 OK

The platform underneath every review

From task routing to training data export — the infrastructure for human-validated AI.

01

Every output audited before it ships

Certified reviewers validate every AI output against your guidelines. Approve, correct, or flag issues before anything reaches users.

Skill gating routes each task only to reviewers with active certifications — medical, legal, audio, translation, code, and more.

  • Human-in-the-loop QA with configurable rubrics
  • Certification-gated reviewer routing
  • Consensus voting for high-stakes corrections
Reviewer routing Live
MD Board-certified
JD × Not medical
AE Assigned
Task #vw-88421
Medical transcription review required: ["emergency_medicine"]
Assigned: rev_med_047 ETA: <24h
POST /v1/tasks
POST your-webhook
HMAC-SHA256 Idempotent Signed payload
02

One API call. Signed webhooks back.

Submit tasks via a REST API, then receive HMAC-signed webhooks when reviews complete. One call in, one signed payload out.

Every correction can be exported as structured DPO pairs to feed your fine-tuning pipeline.

  • Idempotent, batch-ready REST API
  • HMAC-signed webhook payloads
  • JSON / DPO training data export
03

Corrections become training data

Every reviewer correction is automatically structured as a DPO (Direct Preference Optimization) pair — a rejected AI output and the human-corrected chosen version.

Export these pairs to fine-tune your models. Each correction makes your AI incrementally better, instead of being lost in a review log.

  • Automatic DPO pair generation from every correction
  • Export via API or dashboard in JSON format
  • Build a growing dataset with every review cycle
Rejected (AI output)
The capital of France is Berlin. The Eiffel Tower was built in 1889 and stands 324 meters tall.
vs
Chosen (human corrected)
The capital of France is Paris. The Eiffel Tower was built in 1889 and stands 324 meters tall.
1 DPO pair Exportable JSON Improves with every review
REST API Webhooks HMAC Signed Idempotent Batch Operations DPO Export GDPR Export Skill Gating

Evals catch bugs. Humans catch everything else.

Automated evals are necessary but insufficient. They verify structure. Humans verify meaning, safety, and real-world impact.

Automated evals
Format and structure validation
Token-level similarity scores
Factual keyword presence
Syntax and grammar rules
Tone and cultural appropriateness
Domain-specific clinical accuracy
Legal enforceability
Real-world safety implications
Subtle hallucination detection
Certified human reviewers
Everything evals catch
Context and intent understanding
Cultural and tonal nuance
Domain expertise (medical, legal)
Edge case and ambiguity judgment
Safety and harm assessment
Corrected output for training data
Full audit trail with rationale

Evals are a gate.
Humans are the quality layer.
Use both.

Context
Evals can't tell if a medical dosage makes sense in context. Humans can.
Nuance
Evals can't catch tone, cultural fit, or subtle misrepresentation. Humans do.
Feedback
Every human correction exports as a DPO pair. Your model improves with each review.

Wherever AI reaches your users

Wherever an AI output reaches a user, a certified human signs off first. The same pipeline adapts to your industry, your reviewers, and your review criteria.

Healthcare

Clinical notes, medical transcription, and diagnosis summaries validated by medical-domain reviewers before they reach the EHR.

Medical-skill reviewers
Exportable audit trail
24-hour turnaround

Developer Tools

AI-generated code, refactors, and patches reviewed for correctness, security, and style before they ship to your repo.

Senior engineer reviewers
Security-focused checklists
DPO pair export
Not sure if your workflow fits? Talk to our team →

$0.015 per token — or as low as $0.01 on Enterprise.

Pay-as-you-go at $0.015/token. Subscriptions scale down to $0.01/token for high volume.

$0.015
base rate
$0.01 on Enterprise — per token
Text

LLM outputs, prompts, text QA

Audio / Image
1.5–2×

Transcription, TTS, OCR

Decision approval

Chatbot responses, CS decisions

Express SLA

1–4 hour turnaround, priority routing

Estimate your review cost

~25 words ≈ 6 tokens
Base tokens25
Content type
Expertise
Review depth0.5×
Review type0.5×
Effective tokens6
Estimated cost
$0.06
24-hour turnaround included
Start building free

Pre-paid credit bundles and volume discounts on the pricing page. · 50% off your first $10 — auto-applied on signup.

Common questions

Product

Text, audio, images, documents, OCR, translations, code, and structured data — any AI output across any modality.

Certified domain experts. Every reviewer passes a domain-specific exam (medical, legal, audio, translation, and more), and tasks are skill-gated so they only route to reviewers holding an active certification.

Standard tasks complete within 24 hours. Express SLA delivers in 1-4 hours with priority routing.

Yes. Define custom rubrics, accept/reject thresholds, and reviewer instructions per task type. Scorecards are fully configurable.

Your AI generates output, you submit it via our REST API, certified reviewers validate it against your criteria, and you receive the reviewed result via webhook — typically in under 24 hours. Consensus voting, skill gating, and custom scorecards ensure quality. Build your pipeline →

Yes. Human reviewers catch factual errors, fabricated citations, invented statistics, and reasoning failures that automated metrics miss. Every claim is checked against sources before the output reaches your users. Read our hallucination analysis →

Integration

REST API for submissions, webhooks for results. HMAC-signed payloads, idempotency keys, and batch operations. Live in under 5 minutes with any stack.

Yes. Export as DPO preference pairs or structured human-corrected data, ready for fine-tuning.

Pricing & Security

$0.015/token PAYG, scaling to $0.01/token on Enterprise. Multipliers apply for modality, depth, and SLA. Pre-paid subscriptions for volume.

Data is encrypted in transit (TLS 1.2+) and at rest. Webhook callbacks are HMAC-signed. GDPR export and account-deletion APIs are available. SOC 2 Type II and HIPAA BAA are offered on Enterprise — contact us. Review records include reviewer ID, timestamps, and rationale; we do not use your data for model training without consent.

Yes — no contracts and no lock-in. Pay-as-you-go or cancel your subscription whenever. Credits stay valid while your account is active.

What builders say about Verified Workflows

"We caught a critical hallucination 20 minutes before it shipped to 50k users. The reviewer flagged a citation that didn't exist. Worth every dollar."
SR
Sarah Reeves
Head of AI, Fintech startup
"Setup was 12 minutes. We replaced three QA engineers' worth of human review with one API call. Same accuracy, 40% cheaper."
MK
Marcus Klein
CTO, legal-tech platform
"The medical reviewers actually understand HL7 and FHIR. We stopped getting back 'looks fine' on actual clinical text."
AP
Dr. Aisha Patel
VP Engineering, healthtech

Ship AI your users can trust

Put certified human reviewers between your model and your users. Live in under 5 minutes.

Live in 5 minutes No commitment Cancel anytime