Tianrun Qiu: Writing

Guardrails for an AI that publishes on its own

2026-10-07T12:00:00Z

I co-built Fluentide, a Chinese listening app with 2,000+ learners. It turns breaking news into Chinese episodes that sound native but stay at each learner's level.

Our AI has the autonomy to write and publish every episode. Before a script is voiced, it has to pass checks on level, repetition and new words, and AI reviewers flag any Chinese a native speaker wouldn't say. I built many of those guardrails.

This summer, an episode for Mandarin learners still went live in Cantonese, which they can't understand, and none of our checks caught it.

The episode in Cantonese

I noticed it myself. Some of our episodes came out in Mandarin and some in Cantonese, and I traced it to our text-to-speech service. It takes the language as a separate setting from the voice, and one part of our code never sent it. The service guessed. The script itself was fine, and every check we had only looked at the script.

I locked the language setting and added a test that fails if it's ever left out again.

Easy words, Chinese nobody says

We asked for words at the learner's level. The AI kept to easy words but sometimes wrote phrases no native speaker would say, like 走海 ("to walk the sea"), or 有米饭吃 ("have cooked rice to eat") where anyone would say 有饭吃 ("have food"). Our level check only counted vocabulary. It passed both, and they made it into a beginner episode before we caught them.

They were easy to miss because of how our scripts are written. Each Chinese line sits next to its English translation, and the English tells you what the Chinese means, so you don't notice when it's wrong. A beginner would memorize a phrase like that as real Chinese. I built an AI reviewer that deletes the English and flags any line a native speaker wouldn't say.

A goose leg on repeat

New words should come back a few times so they stick. Our level check had a floor on reuse but no ceiling. A draft for our lowest level took that literally and repeated whole sentences: "It's a goose leg. It's a goose leg. It's not a goose leg…" It scored 99.8% known words and passed.

I added a ceiling on repetition and set it by measuring all our published scripts. It sits high enough that normal scripts pass, and only real padding gets flagged.

The exception the AI leaned on

We let the AI leave a word out of the new-word count if the episode explained that word. It explained so many that one beginner episode hit about 15 new words a minute, against our usual 9.

So we added a cap that counts every new word, explained or not. Over the cap, the AI is told to cut the words it uses only once first.

City guides with no facts

We also wanted almost every word to be one the learner already knows. The first batch of beginner city guides hit up to 99% by leaving out names and places. Half of them had no checkable facts at all.

Now a check counts facts like years, distances and prices, with the bar set at our own news episodes. Numbers are some of the first words beginners learn, so the facts came back to our city guides without making them harder.

Testing the checks on real HSK papers

A check can also be too strict and block good work. That's harder to spot, because a blocked script never reaches anyone to read. So I started testing our checks on work we knew was good. When our AI began writing practice exams for HSK, China's official Chinese proficiency test, every check ran on the official sample papers first. If a real exam question failed one of our checks, we fixed the check.

Our plan said the right answer is almost always a paraphrase of what you hear, so we were going to reject questions where it's said word for word. On the real HSK 4 paper, 9 of the 32 answers are said word for word. The rule would have failed 28% of the real exam. On the official papers, not one of 256 wrong answers is said word for word, so the check now rejects any question where a wrong answer is said word for word.

Whatever AI I work on next, the first thing I'll look for is its goose leg.

Shipping the same day a runner asks: how I build with Claude Code

2026-10-04T12:00:00Z

My Claude Code sessions message each other. They test my changes before I look, and they re-film my demo videos when the app changes. My median message to them is under 20 words.

I spent the summer as an AI-native builder on two products: Fluentide, an AI-powered Chinese listening app with 2,000+ learners, and TAOSport, an AI running coach with 250+ active runners. Twenty words are enough because Claude runs a loop that starts and ends with the people using the app. Here's that loop.

Hear

Requests from people reach me in two ways. After a team meeting I paste the raw transcript: four agents read the code in parallel, and Claude comes back with a plan for every request in it. Bugs come from runners, as screenshots in our group chat, and a screenshot plus a few words is enough for Claude to reproduce the problem on its own.

On Fluentide, though, more of the work started from our data than from anyone's message. That loop starts in a different place, so it gets its own post, the next in this series. This one follows the requests people bring.

Decide

A plan is only as good as the research behind it, and Claude's research comes back confident and well cited. So I make it argue against itself: it studies how others solve the problem, tries to prove each finding wrong with our own code, history and usage, and only then advises.

That step often changes the plan. When Claude studied Telegram's code and 20 other chat apps for our coach's chat, it named our biggest problem, citing about ten past bug fixes. Made to prove itself wrong against our own code, it recounted: one. Six of its recommendations, including the one it ranked first, were overturned or scaled back before the plan was written.

Most decisions are smaller than that, and they shouldn't wait for me. Agents like to end a task with a list of open questions; mine settle the small calls with evidence and write the call and its reason into the commit. Only four kinds come back to me: logins, money, taste calls I keep for myself, and anything public or irreversible.

Build

Claude Code sessions can message each other. It's called cross-session messaging, and many people don't know it exists. So once a plan is settled, I give each step its own session and let them coordinate. On one plan they agreed who owned which files, confirmed "226 tests pass on the merged file", and passed findings across. I didn't relay a single message. The most useful bug report came from the session next door: it caught our AI grader failing correct translations, and told the session testing the grader.

When an app changes, its demo video usually goes stale. Mine don't, because Claude is also my video editor: it drives the real app on camera and times every cut from the measured narration, never by hand. One command re-films the demo and re-cuts the edit in about two minutes. One demo survived four rewrites of the app it was filming without me touching a single timing; its captions still light up word by word, and a click still lands exactly on the spoken word "back".

Check

A change isn't done when the code is written, but for most changes I don't click through anything myself. Claude runs the change end to end in Ego Lite, a browser built for agents, logged in like a real user, and fixes what it finds before reporting. Then it hands me a storyboard: one captioned screenshot per step, loading, empty and error states included. Scrolling through it is my review.

Changes to our AI coach need one more check, since a screen can look right while the coach's answer is wrong. Claude replays the exact words the runner complained about in the real app, saves the answer with screenshots and a full trace, then reads it and signs a verdict. The fix counts only when that runner's own question gets a good answer.

Back to the user

That runner is also where the loop ends. It closes in the chat where it started: every change gets posted back to the group, and many requests shipped the same day a runner asked.

That is why my messages can stay short. Claude does most of the building and checking. My time goes to the people in that chat, and to the calls only I can make.

I built an on-device AI that judges every screen in a second

2026-09-25T12:00:00Z

Almost no screen-time tool uses AI. The idea is obvious: judge every screen instead of blocking a site. But that is a model call every few seconds on everything you read. In the cloud it costs money and privacy; on the device it was too slow.

I built the cloud version anyway. SeeNot, for Android, was, as far as I know, the first to judge the screen itself, and it became my thesis at SUSTech. It worked, and it never took off: people liked the idea, few downloaded it, and I never liked where the screenshots went.

On September 15, TypeSafe released Jev, a model that doesn't write text. You give it a text and a list of questions with fixed answers (yes or no, or one option from a list), and it answers all of them at once, in well under a second; because nothing is generated, ten questions take about as long as one. That is exactly what my app had been asking a VLM to do with a page-long prompt. And Kev, an open-source version of the same idea, does it on an Apple silicon Mac in about a second.

What was new was where it could run. The on-device model I had tried for SeeNot took 20 to 40 seconds per judgment; this takes about one, with nothing sent anywhere. So I built the second try, Qualm, a Mac menu bar app.

YouTube is a lecture and a Shorts feed; the lecture stays, and the Shorts get a pop-up that says why. Each new screen gets a short list of questions (what kind of page is this, what is it for, is it private, does it break each of my rules), and plain code decides. I no longer write a prompt; I choose the questions and write the code that acts on the answers.

Most of what the first version taught me is a list of what the tool must never do. It never hard-blocks. It runs local by default, and you can switch to TypeSafe's hosted Jev, but it never falls back to it quietly, because the tool reads everything you read. And rules are generalizable sentences, so "short videos made for endless swiping" catches a site nobody listed.

A week after the Jev launch, I sat down with it, ran trials the first day, had a working app that evening, and have used it daily since. I built it with Claude Code, and for agents too: every setting is also a command, so an agent can add a rule, test it on your own recent screens, and undo it. The night before shipping I asked for an audit as a fresh user and went to sleep; by morning 77 subagents had filed 204 problems, two of them serious.

You can try it at qualm.r-q.name, and the code is on GitHub.