Lesson 81 — Intermediate

Harness Engineering

Same model, same task — alag harness se alag result. Harness = model ke around ka poora system: loop, prompts, tools, context policy, state, limits, permissions, environment. Is lesson mein woh system design hota hai.

Harness kya hai

Ek line ka definition: jo model ke haath mein nahi, woh harness mein. Model sirf decide karta hai; kya allow hai, kab rukna hai, kya verify hua — sab harness.

Harness problem vs model problem

Dono alag hain, diagnose alag — pehle yeh split karo:

• Model problems: reasoning gap, missing knowledge, format na samajhna — training se aaye, harness se theek nahi hote.
• Harness problems: instructions unclear, stopping condition nahi, ghalat tools expose, context mein junk, feedback loop missing.
• Test: same model, better harness se output badla → harness problem tha. Yehi bina cost ke experiment hai.

Agent loop design

def run(task, max_steps=20, max_minutes=15, max_tokens=200_000):
    state = boot(task)                 # NOTES.md + plan load
    while (state.steps < max_steps and within_time(max_minutes)
           and budget_ok(max_tokens)):
        out = model(state.assemble(), tools=tools_for(state.plan.active))
        for call in out.tool_calls:
            if risky(call) and not approved(call):
                return ask_human(call)            # permission gate
            state.apply(slim(run_tool(call)))     # slim results wapas
        if out.task_done(state):
            break
    return finalize(state)                       # durable artifacts likho

• Calls alternate: model → tools → model — har turn par context assemble hota hai (L6.2).
• Errors short + structured wapas — poora stack trace nahi (L6.6).
• Stop conditions: task done / max steps / time limit / token budget — teeno harness ka hissa.
• Har step ke baad write-back (L6.4) — crash par wapas recover.

Harness kya enforce karta hai

• Fixed workflow steps: validation/verification step kabhi skip nahi hota.
• Permission checks: write ops, external calls — allowlist se guzarna padega.
• Human approval: send/delete/pay jaise risky actions — model khud nahi kar sakta.
• Safe defaults: pehla version read-only; write permissions step-by-step.
• Output validation: schema fail = wapas retry, galat format aage nahi jaata.

Kai windows tak kaam

Lamba task ek window mein nahi fit hota — structure aisa banao ki har run chhota rahe par kaam aage badhe:

• Initial run: environment setup, plan, task list banaya — usme hi khatam.
• Later runs: notes padho, ek task karo, changes record — wapas notes.
• Durable artifacts = truth: progress files, task lists, git commits, tests — chat history kabhi nahi.
• Self-run feedback: tests, linters, type checks, e2e — agent khud chalaye bina "done" bole.
• Legible environment: AGENTS.md, saaf project structure, agent-readable docs.
• Sub-agents: focused subtask ke liye fresh isolated context, wapas condensed result.

Exercise

Common mistakes

• Har fail par model blame: Harness audit pehle — instructions, tools, limits, feedback. Woh fix karna sasta hai aur model upgrade se pehle ka standard step hai.
• Bina limits ka loop: Steps/time/tokens ka cap nahi toh ek ghalat plan ghanton chalega — bill bhi, junk bhi. Cap harness code mein hona chahiye.
• Chat history ko truth banana: Crash hua, history gayab, task beech mein. Durable artifacts (files, commits, tests) hi resume aur debugging dete hain.

model   = batsman
harness = {loop, prompts, tools, limits, permissions, env}
# Match hara? Pehle drill (harness) dekho, phir batsman badlo.