AAIF Pune Connect #3 · Agentic AI Day · 10 Oct 2026
Building a domain‑specific harness
Varun Srinivas · Coditas
The mascot measures a half-made harness on a tailor's table
Hi, I’m Varun.

Coditas

CTO and Chief AI Officer. We help companies use AI to do real work.

Build

A toy car that Claude drives. JyotishBench. Lots of small apps.

Community

I speak at meetups, and host some of them here at Coditas. Like this one.

AI is the best thing to happen to builders.
We have a new thing to manage.
Network. Compute. Storage. We have managed these for years.
Now there is a fourth: intelligence.
Four meters on a wall. The fourth, labelled intelligence, is new.
You pay for it per use. It can slow down, break, or be wrong.
The pattern
We never manage these by hand.
ThingWhat manages itWhat it does for us
NetworkThe network stackretries, limits, routing
ComputeKubernetesstarts and scales servers
StorageThe databaseindexes, caching, backups
IntelligenceThe harnesscontext, tools, checks, limits
The harness does this job for AI.
AI is a strange thing to manage.

Not repeatable

Same question. Different answer.

Wrong but sure

A wrong answer sounds just like a right one.

Costs money

Every step costs money and time.

Doesn’t know you

It knows the internet. Not your business.

This talk is about the last one.
Quick recap
agent = model + harness
for (step = 0; step < 10; step++) { msg = await callLLM(messages) // think if (!msg.tool_calls) return msg // done for (tc of msg.tool_calls) // act messages.push(run(tc)) }
The mascot pushes a cart around a loop: observe, act, check, decide
The loop is about 20 lines. The real work is around it.
Same AI model. Same settings.
6.5%→31%
A research team changed only the harness. Success on a big coding task went up almost five times.
HERMES paper, arXiv 2610.07832, 6 Oct 2026
Two carts with the same engine. One is rickety, one is well built.
Same engine. Better car.
Same paper
A good harness helps a small model.
+12.4
points better than other harnesses, on average
4.5
points. A small, free model came this close to a top model
−26%
lower cost
You may not need a bigger model. You need a better harness.
Code checks itself. Most work doesn’t.
In code, tests fail when something is wrong. The AI knows right away.
In most work, a wrong answer looks fine. You find out much later.
Left: a machine stamps a tick on code. Right: the mascot holds a sheet next to an empty stand.
That is why coding AI got good first.
General vs built for your field
General · Claude Code, CodexBuilt for your field
What it seesfiles, the webyour data, already worked out
What it can doread, edit, run commandsthe actions your experts take
How it knows it’s donethe tests passchecks your experts wrote
Rulesasks you for permissionyour rules, written in code
How it improvesa notes fileevery fix becomes a test
How to build one
Five things to tune. Same loop.
The mascot pulls the middle of five levers: context, tools, checks, rules, memory
A harness I built
Let me show you one I built.
Jyotish is Indian astrology. You give a birth date, time and place.
It works out where the planets were at that moment. Then it reads meaning from that chart.
Why I picked it: strict rules, exact maths, and nothing to tell you when you’re wrong.
The mascot studies a square birth chart on a desk beside an astrolabe
This is not about whether astrology works. It’s about whether AI can follow the rules.
Why this is hard
Easy to get wrong. Hard to notice.
Jyotish splits your life into periods, each linked to a planet.
Get one small rule wrong, and a date moves by ten years.
The answer still reads perfectly.
The mascot points confidently at a 2013 calendar while 2023 is circled
No error. No warning. Just a wrong date.
Let’s follow one question · a made-up example
Rahul, 32, Pune
“When will I become a millionaire?”
Let’s see what each of the five does with it.
1 · Context · a real run of the harness
Code works out his chart. AI reads it.

Code works out

  • Born 2 Oct 1994, 9:30 pm, Pune: Taurus rising
  • Mercury rules his savings house. Jupiter rules his gains house.
  • Today he is in the Moon’s life period, until 2031

The AI gets

  • One list of those facts
  • “Use only these facts”
  • Saved, so his next question costs about a tenth
The AI never works out a planet’s position. Code does.
2 · Tools · 7 calls in the real run
He asked “when”. So the AI calls tools.
house_reporthis money houses, and who rules them
house_reportthe same, read again from the Moon
find_windowswhen a money period meets a trigger
graha_reporthow strong his money planets are
The mascot stands on the just-right rung of a ladder
The AI never guesses a date. Every date comes from a tool.
3 · Checks · for Rahul’s question
Before Rahul sees it, code checks it.
Did it look at both money houses? Code checks the steps.
Did every date come from a tool? Code checks.
Any jargon, like “11th house lord”? Code checks.
Does it promise he’ll be rich? An AI reviewer blocks it.
Three sieves stacked: exact checks, judge, human
Code first. Then AI. Then a person. In that order.
4 · Rules
Written down once. Checked by someone else.
The ruleWhere it’s writtenWho checks it
Use only the chart facts. Never work out a date.The expert AI’s instructionsCode. No date tool called? The expert is sent back.
No promises. No diagnosis. Nothing about death.The expert AI’s instructionsAI reviewer. If broken, the answer is rewritten.
No jargon for the person asking.The plain-words AI’s instructionsCode, with a word list. Always rewritten.
Say nothing the expert didn’t say.Built in: the plain-words AI never sees the chartAI reviewer. If broken, the answer is rewritten.
The AI that follows a rule never checks its own work.
Rules in action · Rahul’s real run, shortened
Three steps. Each with limits.
Expert AI · the reader
“Jan 2033 to Jan 2034. Jupiter, his 11th lord, runs as sub-period while Saturn moves through his 2nd house. This is the convergence.”
Plain-words AI · the voice
“The year that stands out most is late January 2033 to early January 2034.”
Never sees his chart
AI reviewer · can only say no
✓ Passed. No flags.
Would reject: “You will be a millionaire by 2033.” That’s a promise.
A writer can make things up. A checker can only say no.
What Rahul got · real run, shortened
Not one date. A sequence.
Jun 2027 – Nov 2028A real step up in base income. A raise, not a windfall.
Jun 2029 – Feb 2031The best time to save. Put money away.
Jan 2033 – Jan 2034The year that stands out most. Many signals line up.
2041 – 2043Later, but possibly the largest of all.
“Money comes through sustained work, not luck.” And no promise of a million.
5 · Memory · for Rahul’s question
An expert spots a gap. It never comes back.
An astrologer reviews it: “You only looked at Jupiter. Start with who rules his money houses.”
That fix becomes a rule for every money question.
And a test, so it is never missed again.
A loop: draft, fix, new rule. The mascot pins a corrected note to a board.
You are not training the model. You are training the harness.
What it costs · Rahul’s question
Pay for thinking. Not for writing.
47¢
for Rahul’s whole answer, all three steps
90%
of that went to the reader, the step that does the thinking
1¢
for the voice. The reviewer: about 3¢
Each step uses its own model.
19 AI models · 56 real questions
Which AI reads a chart best?
ModelOverallRelationshipsHealthCareerMoney
GPT-6 Astra86.693.269.992.290.9
Grok 4.785.691.370.492.588.2
GPT-5.6 Terra85.192.667.991.188.6
Claude Opus 5.583.093.064.589.584.9
Kimi K381.492.060.888.584.3
MiniMax M371.075.453.483.571.6
QR code to jyotishbench.fyi
jyotishbench.fyiScan for the full results
Every AI is weakest on health. The best gets 70 out of 100.
Build strong where it’s yours.
Build strong: your data, your rules, your checks. No new AI model will know these.
Keep it light: prompt tricks and workarounds. The next AI model makes them useless.
A strong stone base labelled your data, rules and checks, with light scaffolding on top labelled prompt tricks
The rules and tests will outlast every AI model I use.
Start on Monday
Six steps to your first harness
1
Pick one task an expert does every week.Start small.
2
Watch them. Write down what they check.This list is your plan.
3
Move all the maths into code.AI reads. Code calculates.
4
Turn their actions into tools.Anything the AI might guess.
5
Write their rules as code.Not in the prompt.
6
Turn every fix into a test.Then repeat.
Build or reuse
Build the loop once to learn. Then use a ready one.

Learn

  • Build it by hand, once
  • Small. One file per part.
  • github.com/varun1505/nomadcoder-harness-engg

Use

  • Claude Agent SDK
  • LangChain create_agent
  • Pi
Spend your time on the five things that make it yours.
Take home
Three things to take home.

AI is a new thing to manage.

The harness manages it. Same model, better harness: 6.5% to 31%.

Your field doesn’t check itself.

Write the checks your experts already do in their heads.

AI reads. Code calculates.

If the AI might guess it, put it in code.

Try it yourself · free · no sign-up
Plug the tools into your own agent.
In Claude Code:
claude mcp add --transport http \ jyotish https://mcp.jyotisha.fyi/mcp
Cursor, or any agent that speaks MCP: use mcp.jyotisha.fyi/mcp
QR code to mcp.jyotisha.fyi
mcp.jyotisha.fyiScan for setup
Then just ask:
“I was born on 14 March 1992 at 6:45 am in Nagpur. When will I get married?”
“I was born on 21 June 1988 at 11:20 pm in Mumbai. Is next year good for a job change?”
“I was born on 5 January 2000 at 4:10 pm in Bengaluru. Am I in sade sati right now?”
“I was born on 2 October 1994 at 9:30 pm in Pune. When will I become a millionaire?”
The tools bring the rules with them. Try different models. See who reads best.
Thank you
Own the harness.
Everyone gets the same AI models. Your harness is what makes yours better.
jyotishbench.fyi
QR code to Varun's LinkedIn
Scan to connect on LinkedInVarun Srinivas · Coditas
← → navigate · G all slides · F fullscreen
All slides · click to jump · Esc to close