Docling's VLM pipeline and Granite-Docling, explained

Standard vs VLM pipeline#

Run it#

pip install "docling[vlm]"
docling convert report.pdf --pipeline vlm                       # default preset: granite_docling
docling convert report.pdf --pipeline vlm --vlm-model smoldocling
docling convert report.pdf --pipeline vlm --to html_split_page --show-layout

from docling.datamodel.base_models import InputFormat
from docling.datamodel.pipeline_options import VlmConvertOptions, VlmPipelineOptions
from docling.document_converter import DocumentConverter, PdfFormatOption
from docling.pipeline.vlm_pipeline import VlmPipeline

opts = VlmPipelineOptions()
opts.vlm_options = VlmConvertOptions.from_preset("granite_docling")

converter = DocumentConverter(format_options={
    InputFormat.PDF: PdfFormatOption(pipeline_cls=VlmPipeline, pipeline_options=opts)
})
doc = converter.convert("report.pdf").document
print(doc.export_to_markdown())

VlmConvertOptions.list_presets() returns every preset with its model and description; from_preset() is exactly what the CLI calls for --vlm-model.

Granite-Docling-258M in brief#

• Made by: IBM Research · licence: Apache 2.0 · released: 17 September 2025.
• Architecture: Idefics3 design with a siglip2-base-patch16-512 vision encoder and a Granite 165M language model.
• Output: DocTags — compact markup for layout, text, tables, code, formulas and charts that maps onto a DoclingDocument.
• Languages: English; Japanese, Arabic and Chinese are marked experimental.
• Prompts: “Convert this page to docling.” for full pages; short instructions such as “Convert chart to table.” for single elements.

All 22 presets#

Read from VlmConvertOptions.list_presets() in Docling 2.143.1. “Remote code” presets only run with --trust-remote-code — review the model's code first.

“MLX only” and similar notes come from each preset's list of supported engines. Licences differ per model — check the model card before commercial use.

Choosing between standard and VLM#

1. Start with the standard pipeline. It is the default for a reason: fast, exact text on born-digital PDFs, provenance for every element.
2. Check the result visually. --to html_split_page --show-layout puts each page next to what Docling extracted.
3. Try Granite-Docling on the pages that fail. Combine with --page-range to test a few pages before committing GPU time.
4. Compare on your own documents. Published scores are averages over other people's documents; ten of yours tell you more.

Picture descriptions and chart extraction#

You do not need the full VLM pipeline to use vision models. The standard pipeline can call a small VLM only for pictures:

docling convert report.pdf --enrich-picture-description          # captions for figures
docling convert report.pdf --enrich-chart-extraction             # bar/pie/line charts → tables
docling convert report.pdf --enrich-chart-extraction --chart-extraction-preset granite_vision_v4

Chart extraction presets in 2.143.1: granite_vision_v3_3, granite_vision_v4 (default) and granite_vision_v4_mlx. The default resolves to IBM Granite-Vision-4.1-4B (ibm-granite/granite-vision-4.1-4b) and granite_vision_v3_3 to a 2B chart-to-CSV model — plan for a GPU or Apple Silicon. We checked these preset names against 2.143.1 but did not run chart extraction on our CPU test machine.

Frequently asked questions

Sources

• ibm-granite/granite-docling-258M model card
• SmolDocling paper (arXiv 2503.11576) — DocTags
• Docling source: VlmConvertOptions presets (docling/datamodel)
• IBM announcement: Granite-Docling