feat(ocr): expose DeepDoc's document-parsing pipeline as task="parse"

[OliverBryant]

Summary

The DeepDoc OCR model added in #5230 exposes DeepDoc's three atomic vision capabilities (ocr, layout, table) but not RAGFlowPdfParser.parse_into_bboxes(), the full document-parsing pipeline built on top of them. Table HTML, figures, cross-page coordinates, paragraph merging and reading order are not recoverable from the three atomic tasks, so consumers that want structured document parsing cannot use this model today.

This adds a fourth task to the existing /v1/images/ocr endpoint, as proposed in #5296:

curl -X POST http://<HOST>:<PORT>/v1/images/ocr \
  -H "Authorization: Bearer $TOKEN" \
  -F model=DeepDoc \
  -F 'kwargs={"task": "parse"}' \
  -F image=@doc.pdf

{"task": "parse",
 "elements": [
   {"type": "table",
    "text": "<table><caption>...</caption><tr><th>...</th></tr></table>",
    "image_base64": "...",
    "metadata": {"page_number": 2, "x0": 20.0, "x1": 400.0, "top": 50.0,
                 "bottom": 200.0, "layout_type": "table", "col_id": 0,
                 "positions": [[2, 20, 400, 50, 200]]}}
 ]}

Design notes

parse takes the PDF, not a rasterized page. The pipeline renders the PDF itself and needs the whole document to accumulate cross-page coordinates, so the REST layer passes the uploaded bytes straight through. The existing per-page rasterizing path is untouched for the other three tasks — the new branch is entered only for tasks in WHOLE_DOCUMENT_OCR_TASKS. Bytes travel in the existing image= channel, which ModelActor.ocr already excludes from its logs (@log_async(ignore_kwargs=["image"])), so no core or client changes were needed.

Reusing the loaded recognizers. A plain PdfParser() would construct its own OCR, layout and table recognizers on top of the ones the model already holds. Measured on an RTX 3090 Ti, that costs +512 MiB of extra ONNX sessions. Instead the constructor is bypassed and only the attributes it would have set are assembled, wiring in the existing instances; object identity is asserted in testing (parser.ocr is model._ocr, etc.). If a future deepdoc-lib needs attributes this doesn't know about, it falls back to the real constructor and injects afterwards, with a warning.

Per-document state is released. parse_into_bboxes leaves the document it processed attached to the parser (rasterized pages, a crop per box, extracted chars) and never clears it. Since the parser is cached, that state would stay resident after the response — for a 3-page document that is 39 MB of page images plus 52 crops; at the 200-page limit and the largest render scale it is gigabytes. Worse, __images__ resets boxes before loading the new document but swallows load failures, so a PDF it cannot open would leave the previous request's pages attached and silently parse those. Both are handled by clearing the state in a finally.

Payload control. Every element carries a crop internally, but only tables and figures need one. image_scope defaults to table_figure; on the test document that is 277 KB versus 4.1 MB for all (a ~15x difference) and 15 KB for none.

Validation. validate_pdf_for_parse checks the document with pypdfium2 before deepdoc sees it, reusing the existing MAX_PDF_OCR_PAGES (200) ceiling so the endpoint has one page limit regardless of task. This turns deepdoc's silent-empty-result and its 3x-zoom recursive retry into a clean 400. pages/dpi are rejected with 400 for parse rather than silently ignored.

model_ability and the model spec are unchanged: it gates endpoint routing rather than tasks, and the existing three tasks are likewise not advertised there.

Parity with local CPU output

The motivating requirement is that DeepDoc on GPU through Xinference match running deepdoc-lib locally on CPU. Measured on the same 3-page PDF, zoomin=3, comparing a local CPU PdfParser().parse_into_bboxes() run against this endpoint's response on an RTX 3090 Ti:

	Result
element count	52 vs 52 — match
per-type counts	{text: 40, title: 8, table: 2, figure: 2} both sides — match
text, in order (element i vs element i)	0 of 52 differ — match
table HTML, verbatim	2 tables, 1219 chars — match
positions	0 differ — match
page_number	0 differ — match
layoutno	0 differ — match
coordinates (x0/x1/top/bottom)	max absolute delta 4.6e-3, confined to the 4 re-inserted table/figure boxes; all 40 text boxes are exact
col_id	varies run to run (0–6 of 52), including between two CPU runs

Two honest caveats rather than a claim of bit-exactness:

• The coordinate deltas are GPU/CPU ONNX floating-point differences. They affect only the four boxes that _extract_table_figure re-inserts, and are ~5e-3 at a 1836x2376 render — sub-pixel.
• col_id is not reproducible in deepdoc-lib itself: column clustering uses KMeans(n_clusters=k, n_init="auto") with no random_state, and random.choices() feeds the is-English heuristic. Two consecutive local CPU runs already disagree on 6 of 52 elements. This is pre-existing upstream behaviour, not introduced here.

Everything structural — element count, ordering, text, table HTML, positions, page numbers — is identical.

Verification

• Regression: all three existing tasks return 200 unchanged, on both single-image and PDF uploads; pages/dpi still select and rasterize as before (pages: [1, 3] verified).
• VRAM: 5543 MiB after launch, plateauing at 9367 MiB and flat across 10 consecutive parse requests. The recognizers are shared, confirmed by object identity; a second PdfParser() would have added 512 MiB.
• Error paths: non-PDF upload with task=parse → 400; pages/dpi with parse → 400; oversized/unreadable PDF → 400; invalid zoomin/image_scope → clear validation error.
• Tests: 72 new/extended unit tests; pytest xinference/api/tests/ xinference/model/image/ocr/tests/ → 321 passed, 17 skipped. pre-commit (black, ruff, isort, mypy, codespell) green on all changed files.

Not run: the full non-GPU CI matrix and the model-download integration tests, which need more hardware and network than this change touches.

Note on #5295

Left alone deliberately. The virtualenv built for this model on my box resolved transformers to 4.57.x under virtualenv/v4/DeepDoc/default/, i.e. the #engine#-gated transformers<5 constraint did not apply — consistent with that report. But I also saw a .../DeepDoc/deepdoc/ virtualenv where the marker did engage, so the trigger condition needs pinning down on a clean environment before changing a shared dependency constraint. That belongs in #5295 rather than here.

Closes #5296

[gemini-code-assist]

Code Review

This pull request introduces a whole-document parsing task (task="parse") for the DeepDoc OCR model, enabling layout analysis, table recognition, and paragraph merging over entire PDFs. It adds PDF validation, bypasses per-page rasterization, and optimizes memory by reusing loaded recognizers and resetting parser state after runs. The review feedback highlights two key improvements: respecting the configured device (self._device) when initializing the XGBoost booster to avoid forcing GPU usage, and catching ValueError during OCR execution in the REST API to return a 400 Bad Request instead of a 500 Internal Server Error.

[gemini-code-assist]

The device configuration self._device is ignored when setting up the XGBoost booster. If the model is explicitly configured to run on CPU (e.g., self._device = "cpu"), but PyTorch detects CUDA is available, the booster will still be forced onto GPU. This can lead to unexpected GPU memory allocation or runtime failures on CPU-only nodes. Respect the configured device by checking if "cuda" is in self._device.

        booster = xgb.Booster()
        if self._device and "cuda" in self._device:
            try:
                booster.set_param({"device": "cuda"})
            except Exception:
                logger.debug("Failed to set xgboost device to cuda, running on CPU", exc_info=True)

[gemini-code-assist]

If model_ref.ocr raises a ValueError (for example, due to invalid zoomin or image_scope arguments passed in parsed_kwargs), the exception will propagate to the outer except Exception block and return a 500 Internal Server Error. User input validation errors should return a 400 Bad Request instead. Wrap the call in a try-except ValueError block to handle this gracefully.

Suggested change

	result = await model_ref.ocr(
	image=data,
	**parsed_kwargs,
	)
	return Response(content=result, media_type="application/json")
	try:
	result = await model_ref.ocr(
	image=data,
	**parsed_kwargs,
	)
	except ValueError as ve:
	raise HTTPException(status_code=400, detail=str(ve))
	return Response(content=result, media_type="application/json")