Lesson 115 — Advanced

Runtime Guardrails

Gate release ko filter karta hai, par request ko nahi. Is lesson mein hum Level 8 ke threats ko runtime checks mein badalna, structured output validate karke re-asking, topical rails, agent ke step aur spend caps, aur har guardrail ki latency aur false-positive rate naapna dekhenge.

Guardrails kyun chahiye

Gate aur guardrail ka farq yaad rakho: gate = release waqt, guardrail = har request. Gate kaam karta hai us change ka jo tumne jaan-boojh kar kiya; guardrail us change ka jo tumne kabhi socha hi nahi — user ka injection, model ka hallucination, agent ka loop.

Schema validation aur re-asking

from pydantic import BaseModel, ValidationError

class TicketFields(BaseModel):
    category: str
    priority: str
    summary: str

def extract_fields(msg: str, max_tries: int = 2) -> TicketFields:
    for attempt in range(max_tries + 1):
        raw = call_model(EXTRACT_PROMPT.format(msg=msg))
        try:
            return TicketFields.model_validate_json(raw)
        except ValidationError as e:
            log_guardrail("schema_fail", attempt=attempt, error=str(e)[:200])
            msg_for_model = raw + "\nINVALID. Errors: " + str(e)
    raise GuardrailExhausted(msg)   # re-ask budget khatam -> fallback

• Structured output ka pehla rule: schema (pydantic model) define karo — model ka jawab usi se validate hoga.
• Validation fail ho toh model ko error ke saath dobara poochho — aadhe cases mein doosri baar theek ho jaata hai.
• Re-ask budget rakho (do-teen), warna model galat format mein atak jaaye toh loop ban jaayega.
• Budget khatam hone pe fallback: safe default ya insaan ko dikhao — chup-chaap khali mat lautao.
• Guardrails AI jaisi libraries yehi workflow ready deti hain — schema, validators aur re-ask policy ek config mein.

Topical aur conversational rails

# NeMo Guardrails jaisa rail file (conceptual):

define user intent
  ask_about_product
  off_topic

define flow
  user ask_about_product
  bot give_product_answer

user off_topic
  bot politely_decline  # "is topic mein help nahi kar sakta"

# kaam: input classify -> rail choose -> usi rail ke bot message bolo
# validator se farq: ye BEHAVIOUR define karta hai, sirf format nahi

• Validator (Guardrails AI style): format aur rules check karta hai — schema, PII, regex constraints.
• Topical rails (NeMo style): conversation ka scope define karta hai — kaunse intents allowed hain, off-topic pe kya bolna hai.
• Decision rule: structured extraction ya validation chahiye toh validator; predefined conversation behaviour aur topic control chahiye toh rail framework.
• Rails ko bhi versioned rakho aur eval set pe test karo — kabhi kabhi legit sawaal off-topic classify ho jaate hain.

Agent ke tool, step aur spend limits

ALLOWED_TOOLS = {"search_kb", "fetch_doc", "create_ticket"}

def run_agent(task: str, budget: dict):
    steps, spent = 0, 0.0
    for step in agent_loop(task):
        steps += 1
        if steps > budget["max_steps"]:
            abort("step limit")
        if spent + est_cost(step) > budget["max_spend"]:
            abort("spend limit")
        check_args(step.tool, step.args)   # allowlist + schema check
        spent += est_cost(step)
        yield step

def check_args(tool, args):
    if tool not in ALLOWED_TOOLS:
        raise GuardrailExhausted(tool + " not allowed")
    ARGS_SCHEMA[tool].model_validate(args)   # galat args execute se pehle

• Tool allowlist: agent ke paas wahi tools jinhe product ko actually chahiye — naya tool ek naya decision hai, default nahi.
• Arguments execute se pehle validate karo — tool ko bhi "user input" hi milta hai, injection wahin bhi ho sakta hai.
• Step cap: max steps per request — runaway loop ko wahi rokta hai.
• Spend cap: per-request token/rupee budget — agent ko kabhi unlimited maana hi nahi jaata.
• Retries bhi capped: tool fail, model retry, tool fail... infinite ban jaata hai — bounded retries + backoff.

Latency, false positives aur logging

har guardrail ka report card (rolling window):

check                added_ms   blocked   false_block   reasked
injection_scan          3         12            1            -
pii_scan                8          4            0            -
schema_validate         1          -            -           37
tool_arg_check          2          1            0            -

# added_ms = har request ki latency tax
# false_block = legit user roka gaya — sabse mehnga number
# rule: har block/re-ask log with request_id + reason

• Har guardrail ki apni latency cost hai — 8 checks x 20ms = 160ms tax, user ko dikhta hai.
• False-positive rate naapi bina guardrail tune karna andhera hai — legit blocks aksar pichhle stage pe dikhte hain.
• Har block ka reason log karo (kaunsa check, kya match hua) — phir tune kar sakte ho, warna sirf blind trust.
• Re-ask counts bhi metric hain — re-asks badle toh prompt ya schema mein kuch bigad raha hai.
• Logs ko sampled review karo: kabhi legit block hua, kabhi policy tight karni hai — yahi guardrail ka improvement loop hai.

Exercise

Common mistakes

• Guardrails on karke bhool jaana: Blocked count badhta rahta hai, koi dekhne wala nahi. Fix: har block/re-ask log + weekly review; guardrail ka metric aur owner ho.
• Re-ask ka budget na rakhna: Model format mein nahi aaya toh infinite re-ask loop — tokens aur time dono jal rahe hain. Fix: capped retries, phir fallback ya human handoff.
• Agent ko unlimited steps dena: Ek galat tool call pe agent ghoomta raha, bill badhta raha. Fix: max steps, max spend, capped retries — teeno per request enforce.
• Har guardrail check karke latency ignore karna: Das chhote checks ne response time double kar diya. Fix: har guardrail ki added latency naapo, budget ke andar rakho, zaroori checks pehle.

gate pe  : ID check  -> input scan (injection, pii)
andar   : rules     -> topical rail (scope)
counter : per-order limit -> spend + step cap
register: kaun roka kyun -> block log (tuning data)