When a government chatbot
recommended breaking the law
NYC's official MyCity Business chatbot — built to help small employers navigate city rules — told businesses they could fire workers for being pregnant, pocket employee tips, and pay below minimum wage. Every answer was illegal under NYC labor law. The city had to issue corrections.
The city's own chatbot told employers to violate city law
In October 2023, New York City launched the MyCity Business chatbot — an AI assistant promoted as a way for small business owners to get fast, authoritative answers about city regulations, permits, and labor law.
In March 2024, an investigation by The Markup found the bot was systematically giving employers advice that violated NYC's own labor protections. Asked whether an employer could fire a worker for being pregnant, the bot said yes. Asked whether bosses could keep worker tips, it said yes. Asked whether they could ignore the city's minimum wage, it said yes.
Every one of those answers was illegal under NYC labor law — the city has some of the strongest worker protections in the country. The same government that enforces those laws was telling employers how to break them.
After publication, NYC officials acknowledged the errors, said they were working on fixes, and the bot's responses were quietly corrected. But for the months it had been live, the official city answer to "can I fire someone for being pregnant?" was yes.
What the chatbot actually said
What it cost
"The city's chatbot told business owners they could break the city's own laws — and it was the official source small businesses were told to trust."
— Reporting by The Markup, March 2024
Three review criteria that would have caught this
Each criterion below maps to a real review task you can configure in the sample builder. A certified reviewer checks every government chatbot response against these before it reaches a citizen or business.
Verify advice against current law
Every statement about what an employer or citizen may legally do must cite and match the current statute. If the AI says "you can fire a pregnant worker," the reviewer checks the NYC Human Rights Law — and flags the answer.
Flag illegal recommendations as CRITICAL severity
Any answer that, if followed, would cause the user to break the law is treated as the highest-severity failure. It blocks publication entirely — no soft warnings, no "low confidence."
Jurisdiction-specific compliance check
Labor law varies by city and state. A reviewer confirms the answer against the correct jurisdiction — NYC rules for NYC businesses, not generic federal defaults that miss local protections.
Paste any chatbot response. See what gets flagged.
This is a simplified version of what our reviewers see. Paste an employment-advice response (yours or a competitor's) and run the check. The criteria above are applied automatically.
Don't ship illegal advice
If your AI gives legal, safety, or compliance answers, every response is a liability. Put certified reviewers between your model and your users. 50% off your first $10 — live in under 5 minutes.