Lesson 162 — Advanced

Resume ↔ JD Screening Agent

HR teams ke paas hundreds resumes aate hain — yeh agent triage karta hai: JD se requirements nikaalta hai, har resume se match karta hai, aur structured score deta hai. Final decision insaan leta hai; agent sirf rank aur reasons deta hai.

Project kya hai

Yeh project ka dikkat scoring subjective hai — isliye asli kaam score ko defend karne layak banana hai: har score ke saath evidence (resume ki line) aur ek fixed rubric.

Structured output — schema hi contract hai

from pydantic import BaseModel
from typing import Literal

class SkillMatch(BaseModel):
    skill: str
    verdict: Literal["matched", "partial", "missing"]
    evidence: str          # resume se hi line, model ke fabrications nahi

class Screening(BaseModel):
    overall_score: int      # 0-100
    must_have_gaps: list[str]
    matches: list[SkillMatch]
    recommendation: Literal["interview", "reject", "needs_review"]

result = client.responses.parse(
    model=MODEL,
    messages=[{"role": "user", "content": prompt}],
    text_format=Screening,        # model ya SDK se schema enforce
)

• Schema ke bina har run mein alag JSON shape — downstream systems toot jaate hain.
• Enum fields (Literal) se free-text masla khatam: "maybe" type values nahi aayengi.
• Scores kabhi 0–100 band se bahar nahi aane chahiye — validate + clamp karo, model par blind trust nahi.

Matching logic — kya score hota hai

Scoring ko deterministic + LLM ka mix rakho: jo ho sakta hai code se (years, degree keyword), woh code se; jo nuance hai (skill depth, transfer learning), woh LLM se.

• Requirements extraction: JD → must-haves[] + nice-to-haves[] + seniority (structured output se hi).
• Skill normalization: "JS", "JavaScript", "ES6" ek hi bucket mein — alias map se.
• Weighted rubric: must-have missing = cap on total score; nice-to-have sirf bonus.
• Seniority check: title vs years vs responsibility language — teenon se ek picture.

def rubric_score(reqs, matches):
    must = sum(1 for r in reqs.must_have if any(
        m.skill == r and m.verdict == "matched" for m in matches))
    gap = [r for r in reqs.must_have if r not in {m.skill for m in matches
           if m.verdict == "matched"}]
    if gap:
        return {"overall_score": min(59, must * 10), "must_have_gaps": gap}
    bonus = sum(1 for m in matches if m.verdict == "matched") * 3
    return {"overall_score": min(100, 60 + bonus), "must_have_gaps": []}

Fairness guardrails

Hiring data mein puraane biases hote hain — model unhe seekh kar repeat kar sakta hai. Guardrails project ka feature hain, disclaimer nahi:

• Blind fields: name, photo, age, gender, college ko scoring prompt se nikaal do — skills aur evidence hi jayein.
• Same rubric for everyone: har resume par wahi weights, koi per-candidate special instructions nahi.
• Audit slice: random10% ko human double-review karo — agreement rate tumhara bias alarm hai.
• Reject reasons mandatory: bina reason ke reject kabhi export mat hone do.

Evaluation aur pitfalls

• Golden set:20–30 resumes jinke human shortlist labels pehle se hon — rank agreement (precision@5) nikaalo.
• Fabrication test: resume mein na-dikhne wali skill kabhi matched to aayi? Yeh raddi check har change par chalao.
• Rubric drift: purane runs re-score mat karo bina version tag ke — rubric v2 ke scores v1 se compare nahi hote.

Exercise

Common mistakes

• Agent ko auto-reject dena: Agent rank aur reasons de, decision insaan. Auto-reject se bias + legal dono ka jokhim hai — aur demo mein bhi yahi point strong lagta hai.
• Rubric ke bina score: Har run par alag basis par number nikaloge toh scores meaningless hain. Rubric likho, version karo, code mein lock karo.
• Bias ko data se chhupana: Purane shortlist data se model puraane patterns repeat karega. Blind fields + audit slice nahi kiya toh system discrimination ka pipeline ban jaayega.

requirements = extract(jd)                 # checklist bani
for r in resumes:
    score = rubric_match(requirements, r)   # wahi rule har par
    result = {"resume": r.id, "score": score, "next": "human review"}