Docling for RAG

A RAG pipeline has six stages; Docling owns the first three — the ones that decide answer quality before any embedding exists:

• Parse — layout, reading order, tables, formulas, pictures → one structured DoclingDocument.
• Serialize / chunk — structure-aware chunks with headings, captions and provenance (or a Markdown/HTML/JSON export for post-splitting).
• Enrich text — contextualize() prepends heading context so each chunk stands alone.
• Embed → store → retrieve → generate — your embedding model, vector store and framework; Docling-agnostic.

There are two chunking philosophies: export to Markdown and post-split (e.g. LangChain’s MarkdownHeaderTextSplitter), or chunk natively on the DoclingDocument. Native chunking preserves tables, headings and provenance that Markdown post-splitting silently drops — prefer it unless you have a reason not to.

For a programmatic pipeline, convert in Python and keep the document object — chunkers operate on it, not on files:

from docling.document_converter import DocumentConverter

converter = DocumentConverter()
result = converter.convert("report.pdf")
document = result.document

Exports are shorthands over serializers (MarkdownDocSerializer etc.). For RAG the choice matters mostly for tables:

Serializers are customizable: subclass BaseTableSerializer and friends for full control (see the advanced chunking & serialization example).

All native chunkers implement BaseChunker (chunk() → chunk stream, contextualize() → enriched text), so LlamaIndex-style integrations accept any built-in, custom or third-party chunker through the same interface.

• Repeat headers (repeat_table_header=True, default): each chunk of a split table starts with the header row, so every chunk is self-describing for the embedding model.
• Overflow escape (omit_header_on_overflow=True): for wide tables, rows that fit without the header but overflow with it skip the header — token efficiency without breaking row integrity.
• Merged cells: Markdown flattens spans; if merged headers matter, chunk from HTML/JSON serialization or a custom table serializer instead.
• Line-based alternative: LineBasedTokenChunker with a repeated prefix keeps CSV-like rows intact and supports the same overflow logic via omit_prefix_on_overflow.

Worked demonstrations: hybrid chunking (incl. header repetition on a wide CSV), line-based chunking, table extraction.

Every chunk carries dl_meta — keep it in your vector payload; it is what turns “an answer” into “an answer with a source”:

Provenance includes page numbers and bounding boxes per item — enough to cite “page 3” or draw the source region (see visual grounding).

from transformers import AutoTokenizer
from docling_core.transforms.chunker.tokenizer.huggingface import HuggingFaceTokenizer
from docling.chunking import HybridChunker

EMBED_MODEL_ID = "sentence-transformers/all-MiniLM-L6-v2"
tokenizer = HuggingFaceTokenizer(
    tokenizer=AutoTokenizer.from_pretrained(EMBED_MODEL_ID),
    max_tokens=512,
)
chunker = HybridChunker(tokenizer=tokenizer, merge_peers=True)

At corpus scale, the Data Prep Kit chunk+tokenize pipeline applies the same principle in batch.

LangChain — langchain-docling

from transformers import AutoTokenizer
from docling_core.transforms.chunker.tokenizer.huggingface import HuggingFaceTokenizer
from docling.chunking import HybridChunker
from langchain_docling import DoclingLoader
from langchain_docling.loader import ExportType

tokenizer = HuggingFaceTokenizer(
    tokenizer=AutoTokenizer.from_pretrained("sentence-transformers/all-MiniLM-L6-v2")
)
loader = DoclingLoader(
    file_path=["https://arxiv.org/pdf/2408.09869"],
    export_type=ExportType.DOC_CHUNKS,  # native chunks (default)
    chunker=HybridChunker(tokenizer=tokenizer),
)
docs = loader.load()  # one LangChain Document per chunk, dl_meta in metadata

Two modes: DOC_CHUNKS (default — one LangChain Document per native chunk, metadata preserved) and MARKDOWN (one Document per file; split downstream, e.g. with MarkdownHeaderTextSplitter on #/##/###). Full flow in the official LangChain RAG example (Milvus + Mixtral) and the LangChain guide.

LlamaIndex — reader + node parser

from llama_index.readers.docling import DoclingReader

reader = DoclingReader(export_type="json")  # lossless; "markdown" for lossy
documents = reader.load_data("report.pdf")

DoclingReader populates LlamaIndex Documents (lossless JSON or lossy Markdown); the Docling Node Parser turns Docling-format Documents into Nodes using its knowledge of the format. Any BaseChunker implementation plugs into the same interface. See the official LlamaIndex RAG example.

Haystack and the rest

Haystack ships Docling as a converter component (example, integration docs). Beyond the big three: txtai, Kotaemon, DocETL, Vectara, Semantica, Hector, haiku.rag, Bee, CrewAI, Langflow, Open WebUI, spaCy, NVIDIA, Granite cookbook RAG and Data Prep Kit all integrate Docling — browse the official integrations index.

The store never affects Docling — chunks + embeddings upsert anywhere. Official worked examples exist for:

• Tokenizer == embedding model. Always. Re-check on every model swap.
• Embed contextualize() output, store chunk.text alongside for display.
• Keep DocMeta (headings, page_no, bbox, origin) in the payload for citations.
• Repeat table headers; use HTML/JSON source for merged-cell tables.
• Trim conversion cost: --no-ocr for digital PDFs, skip enrichment you don’t rank on, batch-convert and pre-download models (docling-tools models download --all).
• Scale-out: Data Prep Kit chunk+tokenize for corpora; docling-serve /v1/convert/source with --to chunks behind a queue for services.

• Answers lack context / “orphan” chunks — you embedded chunk.text; switch to contextualize() and confirm headings appear.
• Table answers are wrong — enable repeat_table_header; check merged cells (Markdown flattens spans → use HTML/JSON).
• Transformers sequence-length warning — documented false alarm, safe to ignore (official FAQ).
• Chunks too big/small for the embedding model — tokenizer mismatch; set the chunker tokenizer to the embedding model’s and tune max_tokens.
• No citations possible — metadata dropped at upsert; persist dl_meta (page_no, headings, origin).
• Ingestion too slow — see conversion is slow: disable unneeded OCR/enrichment, use GPU, batch.