How to use compact_table with MarkdownTableSerializer?

[ofcat]

Hi @dosu! i am now wondering how to adjust my code so that i can get compact markdown tables (without excessive spaces in rows)? This is the PR i am talking about, fix(markdown): add an option to compact table serialization
#495: docling-project/docling-core#495

Here is some of my code:
def _get_docling_chunker(max_tokens: int) -> HybridChunker:

# remove this if you want table in triplet notation
class MDTableSerializerProvider(ChunkingSerializerProvider):
    def get_serializer(self, doc):
        return ChunkingDocSerializer(
            doc=doc,
            table_serializer=MarkdownTableSerializer(
                # compact_table=True
            ),  # configuring a different table serializer
        )

if max_tokens not in _CHUNKER_CACHE:
    tokenizer = OpenAITokenizer(
        tokenizer=tiktoken.encoding_for_model(EMBED_MODEL_ID), max_tokens=MAX_TOKENS
    )
    _CHUNKER_CACHE[max_tokens] = HybridChunker(
        tokenizer=tokenizer,
        serializer_provider=MDTableSerializerProvider(),  # remove for triplet tables
        merge_peers=True,  
    )
return _CHUNKER_CACHE[max_tokens]
.
.
.

try:
    t0 = time.perf_counter()
    doc = converter.convert(
        source
    )  # .document  ## converter.convert(source, max_num_pages=100, max_file_size=20971520) limit pages or file size ## file size limited to 50mb by /file endpoint
    t1 = time.perf_counter()

    for stage_name, profiling_item in doc.timings.items():
        print(f"{stage_name}: {profiling_item.total():.2f}s")

except Exception as e:
    # brainstorm what to do in case of failed conversion, for now we just log and skip the file, but maybe we want to return an error to the frontend?
    print(f"Failed to convert {file_name}: {e}")
    raise ValueError(f"Failed to convert {file_name}: {e}")

for chunk in chunker.chunk(dl_doc=doc.document):
    chunk_text = chunker.contextualize(
        chunk
    )  ## add exception handling here in case of failure in contextualization, for now we just skip the chunk
    if not chunk_text.strip():
        continue

    page_num = _extract_page_range(chunk)
    chunks.append(
        {
            "file_name": file_name,
            "text": chunk_text.strip(),
            "page": page_num,  # rename to page_num later when you are more in the mood :)
            "token_count": chunker.tokenizer.count_tokens(
                chunk_text
            ),  # havent found a way to extract token count from chunk directly (not sure this is possible)
        }
    )

[dosubot[bot]]

The parameter is actually compact_tables (plural), not compact_table. You need to pass it via MarkdownParams to the ChunkingDocSerializer. Here's how to update your code:

from docling_core.transforms.chunker.hierarchical_chunker import ChunkingDocSerializer
from docling_core.transforms.serializer.markdown import MarkdownTableSerializer, MarkdownParams

class MDTableSerializerProvider(ChunkingSerializerProvider):
    def get_serializer(self, doc):
        return ChunkingDocSerializer(
            doc=doc,
            table_serializer=MarkdownTableSerializer(),
            params=MarkdownParams(compact_tables=True),  # Enable compact table format
        )

The key change is adding params=MarkdownParams(compact_tables=True) to your ChunkingDocSerializer. This parameter removes padding and alignment spaces from tables, producing minimal whitespace which is better for downstream AI processing and reduces token count.

To reply, just mention @dosu.

---

^{How did I do? Good | Irrelevant | Incorrect | Verbose | Hallucination | Report 🐛 | Other}