Lesson 49 — Intermediate

Chunking Strategies

Embedding model ek poore 50-page document ko agar seedha vector bana de toh retrieval ka target itna bada ho jaata hai ki chhota sawaal bhi usme doob jaata hai. Is lesson mein hum documents ko kyun todte hain, kaunsi chunking strategy kab lagti hai, aur size-overlap-boundaries ke knobs kaise tune karte hain dekhenge.

Documents ko split kyun karna padta hai

# Poora doc ek chunk = retrieval target bahut bada
doc = open("policy.txt").read()
print(len(doc), "chars in ONE retrieval unit")
# -> chhote sawaal ke liye poora doc embed hota hai,
#    relevance dilute, context bhi zyada kharch

Fixed aur recursive chunking

def fixed_chunks(text: str, size: int = 800, overlap: int = 120) -> list:
    step = size - overlap
    return [text[i:i + size] for i in range(0, len(text), step)]

# problem: beech mein sentence/paragraph aadha kat jaata hai

SEPARATORS = ["\n\n", "\n", ". ", " "]

def recursive_chunks(text, size=800, seps=None):
    seps = SEPARATORS if seps is None else seps
    if len(text) <= size:
        return [text]
    if not seps:
        return [text[:size], text[size:]]   # hard cut last resort
    head, rest = seps[0], seps[1:]
    if head not in text:
        return recursive_chunks(text, size, rest)
    out = []
    for part in text.split(head):
        out.extend(recursive_chunks(part, size, rest))
    return out

• Fixed: sabse aasaan — har size characters pe kaat do. Tez aur predictable, par boundary kabhi sentence ke beech gir sakti hai.
• Recursive: pehle paragraph ( ), phir newline, phir sentence, phir word — bade piece ko baar baar chhote separators se todta hai. Boundaries kaafi had tak saaf rehti hain.
• Dono mein hi overlap do (size ka ~10-20%) warna boundary wala sawaal dono taraf se kat jaayega.
• Split ke baad har chunk ko trim karo aur khali chunks hatao — naapne mein kharabi bachti hai.

Structural aur semantic chunking

import re

def heading_chunks(md_text: str, size: int = 1200) -> list:
    # headings ke baad todo (markdown/DOCX heading style se)
    parts = re.split(r"(?m)^#{1,3} ", md_text)
    chunks = [p.strip() for p in parts if p.strip()]
    return [c if len(c) <= size else c[:size] for c in chunks]

# semantic: adjacent sentences ka embedding similarity dekho,
# jahan meaning gap aaye wahan boundary maaro
def semantic_chunks(sents, embs, threshold=0.3):
    chunks, buf = [], [sents[0]]
    for i in range(1, len(sents)):
        if cosine(embs[i], embs[i - 1]) < threshold:
            chunks.append(" ".join(buf)); buf = []
        buf.append(sents[i])
    chunks.append(" ".join(buf))
    return chunks

• Structural: headings, sections, slide breaks, table boundaries — document ka apna structure hi boundary hai. Manuals aur policies ke liye best.
• Semantic: do sentences ka embedding similarity gir jahan wahin naya topic shuru hota hai. Sabse accurate par embedding calls zyada — ingestion cost badhti hai.
• Decision rule: structure milta hai toh structural; structure nahi (plain transcripts, chats) aur budget hai toh semantic; warna recursive.
• Har strategy ke saath parent metadata carry karo — chunk chhota ho ya bada, doc_id/heading rahe.

Size, overlap aur boundaries

SIZE, OVERLAP = 500, 100        # rule of thumb: overlap ~10-20% of size

for s in [200, 500, 1200]:
    ch = fixed_chunks(raw, size=s, overlap=s // 5)
    print(s, "->", len(ch), "chunks")

• Chhota chunk: precise match, kam tokens — par sawaal ko apna context chahiye ho toh jawab adhura reh sakta hai.
• Bada chunk: context rich — par relevance dilute hoti hai aur prompt mein zyada tokens kharch.
• Overlap: boundary ke dono taraf thoda content doobara, taaki beech ka sawaal ek hi chunk mein aa jaaye.
• Boundaries: sentence, paragraph ya heading pe rakho — kabhi bhi shabd ke beech nahi.

Precise chunks vs context-rich chunks

# parent-child: chhote chunk se search, bade parent ko LLM ko bhejo
child  = {"id": "p12-c3", "text": "...", "parent_id": "p12"}
parent = {"id": "p12", "text": "poora section (bada)"}

hits = search(child_index, k=3)                        # precise match
context = [parent[h["parent_id"]] for h in hits]       # rich answer

• Sawaal aksar ek line ka hota hai, jawab ko kabhi kabhi paragraph chahiye — isi tension ko parent-child (small-to-big) se solve karte hain.
• Table rows ya list items ko uska heading/footer sath rakho — warna row akela padhne mein meaningless hoga.
• Ek hi doc se bahut se closely related chunks aayein toh dedupe ya merge karo — top-k ka space waste nahi hona chahiye.

Exercise

Common mistakes

• Har document ka size same rakhna: Fixed length se heading, sentence aur paragraph aadhe kat jaate hain. Fix: recursive ya structural splitter use karo, fixed sirf tab jab size contract ho.
• Chunk size sirf guess se decide karna: "Log 500 bolte hain" — tumhare corpus ki language aur sawaal alag hain. Fix: chhota eval set banao, size/overlap ke grid pe recall compare karo.
• Overlap hata dena: Boundary wale sawaal (do sections ka rishta) dono taraf se kat jaate hain. Fix: 10-20% overlap rakho — thoda storage zyada, retrieval kaafi behtar.
• Metadata na rakhna: Chunk ka doc_id, heading aur page nahi hai toh citation aur filters mumkin hi nahi. Fix: har chunk ek record hai — text + metadata, kabhi nanga text nahi.

kamra (document)
  -> dabbe (chunks)     : size decide karo
  -> label (metadata)   : doc_id, heading, page
  -> tape (overlap)     : boundary wala saman bacha
  -> shelf (vector DB)  : dabbe search hote hain