Lesson 57 — Intermediate

Reranking

Top-10 chunks mein se 4 sabse best chahiye — toh 10 bhejne se behtar hai pehle 50 candidates nikaal kar unme se 4 chunna. Is lesson mein hum two-stage retrieve-wide-then-rerank pattern, cross-encoder ka kaam, latency ka hisaab, aur kab rerank worth it nahi hai dekhenge.

Reranking kya hai

Funda: pehla retriever fast hai par andaza lagata hai (alag-alag embeddings). Reranker dheere hai par query aur document ko ek saath padhta hai — isliye rishta zyada sahi pakadta hai. Recall wide rakho, precision reranker ko de do.

Cross-encoder aur reranking models

import os, numpy as np
from sentence_transformers import CrossEncoder   # example library

reranker = CrossEncoder(os.environ["RERANK_MODEL"])

def rerank(query, cands, top=4):
    pairs = [(query, c["text"]) for c in cands]
    scores = reranker.predict(pairs)          # har candidate pe model chalta hai
    order = np.argsort(-scores)[:top]
    return [cands[i] for i in order]

# usage: cands = hybrid_search(query, k=50); top = rerank(query, cands, 4)

• Bi-encoder (dense retrieval): query aur doc alag-alag encode — doc vectors precomputed, search tez.
• Cross-encoder (rerank): query+doc saath mein padhta hai — accurate, par har candidate pe compute, precompute nahi hota.
• Isliye cross-encoder ko kabhi first-stage retrieval pe nahi lagate — 50 candidates pe hi use hota hai.
• Rerank ke baad scores alag scale pe hain — pehle ke score se mix mat karo, order reuse karo.

Relevance gained, latency added

import time

def timed(fn, *a, **kw):
    t0 = time.perf_counter()
    out = fn(*a, **kw)
    return out, time.perf_counter() - t0

top, t_search = timed(hybrid_search, query, k=50)
best, t_rr    = timed(rerank, query, top, 4)
print("search", round(t_search, 3), "s | rerank", round(t_rr, 3), "s")
# har stage ka apna budget rakho: retrieval vs rerank alag-alag dikhega

• Rerank ki latency candidates ke saath linear badhti hai — budget fixed karo, scale mat karo.
• Cost bhi wahi hai: hosted reranking per-pair billing karti hai; local model ho toh GPU time.
• Asar dekhne ke liye apne eval set pe answer quality compare karo — rerank ke pehle vs baad.
• Real-time product (voice, chat live) mein latency budget tight — yahan rerank ka trade-off sabse sensitive hai.

Jab reranking kuch khaas nahi karta

• Chhota/homogeneous corpus: 50 docs hain jinke beech fark hi nahi — rerank kya rank karega.
• First-stage hi weak: agar top-50 mein hi sahi chunk nahi, toh reranker kisi ko bacha nahi sakta — pehle recall sudharo (chunking, hybrid, query rewrite).
• Exact-ID queries: sparse hi pakad lega, rerank ki zarurat nahi.
• Latency budget zero: offline batch pipelines mein bhi latency matter nahi karta — tab rerank free improvement hai; live mein nahi.

Exercise

Common mistakes

• Rerank se weak retrieval theek hoga: Top-50 mein hi jawab nahi toh reranker bebas hai. Fix: pehle recall (chunking, hybrid, rewrite) sudharo, rerank uske baad.
• Bahut zyada candidates rerank karna: Har pair pe compute — 500 candidates pe latency explode. Fix: fixed budget 20-100, usi pe tune karo.
• Rerank score ko confidence samajhna: Wo relative order deta hai, calibrated probability nahi. Fix: abstain ke liye retrieval score/threshold use karo, rerank score ko uncha mat samjho.

50 candidates (retrieval, tez, wide recall)
   -> chef taste (cross-encoder, dheere, precise)
   -> 4 final plates (context) -> table (LLM)

budget: chef 500 plates taste nahi kar sakta - fixed rakho