BadgeprintingEvent registration · badge automation

Designing Scalable Badge Templates with Python ReportLab

Symptom Statement

A ReportLab badge job runs clean against a 200-record test export and then falls over the moment it meets a real conference manifest of 20,000+ attendees: RSS climbs linearly until the worker takes a MemoryError or an OOM kill somewhere past the first few thousand records, long names bleed into the QR zone or clip silently at the die-cut edge, and the print spooler rejects the finished file with a PDFDocException or malformed-stream error. This page is the runbook for that exact failure surface — turning a validated attendee record into a print-ready, byte-deterministic PDF at scale without leaking memory, drifting coordinates, or emitting a stream the RIP refuses. It is the ReportLab implementation detail behind the badge layout architecture boundary, one step downstream of the attendee field mapping rules that produce the canonical record and one step upstream of the PDF routing workflows that dispatch the file to a printer. It assumes the record already passed its data contract; the failures here are rendering-layer failures, not schema failures.

Root Cause Analysis

Three concrete, independently reproducible faults account for nearly every broken large-batch ReportLab run. They compound under load, but each has a distinct trigger and a distinct fix.

  1. Unbounded canvas memory growth. A single long-lived pdfgen.canvas.Canvas retains every page state, font cache, and image buffer in RAM for the life of the document. canvas.showPage() finalizes a page logically but does not flush the underlying C-level buffers — only canvas.save() does. Rendering all 20,000 badges into one canvas therefore grows RSS linearly until the process is killed.
  2. Coordinate drift and text bleed. ReportLab’s drawString places text at an absolute (x, y) point with no bounding box, no wrapping, and no overflow guard. An unvalidated long name or a wide organization string overruns into the QR matrix or past the bleed margin, and a string that exceeds the printable width is drawn off-card rather than clipped or truncated.
  3. Spooler-rejecting PDF streams. Dynamic payloads carry control characters, unescaped markup, or invalid barcode checksums that corrupt the content stream, and a leaked graphics-state stack (an unbalanced saveState/restoreState) produces a structurally invalid PDF that the raster image processor rejects at spool time.

Symptom-to-Resolution Matrix

Unbounded Canvas Memory Growth

Symptoms

  • RSS climbs roughly linearly with record count; the job dies with MemoryError or an OOM kill several thousand records in.
  • Memory is never released between records even though each badge is independent.
  • A tracemalloc peak far above the working set of a single page.

Root cause. One monolithic canvas accumulates all page and font state until save(). showPage() does not reclaim it, so the whole run is resident at once.

Fix

  1. Cap each PDF at an atomic chunk of records (≤500) and open a fresh canvas.Canvas per chunk so state is bounded to one file.
  2. Stream records with an iterator instead of materializing the full manifest as a list.
  3. Drop the canvas reference and force gc.collect() between chunks; watch tracemalloc.get_traced_memory()[1] and reset the peak after each flush.
  4. Never hold decoded QR Drawing objects across chunks — build them inside the per-badge draw call so they are collectable.

Coordinate Drift and Text Bleed

Symptoms

  • Long names overrun into the QR zone or clip at the die-cut edge.
  • Titles render off-card or overlap the attendee identifier.
  • The same template looks fine for short names and broken for long ones.

Root cause. drawString honors absolute coordinates with no bounding box; length validation and clamping have to be applied before the draw call, not by ReportLab.

Fix

  1. Normalize and clamp every display string in a deterministic pre-render pass — unicodedata.normalize("NFKC", s) then slice to a max glyph count — so len() agrees with what will be drawn.
  2. Map all fields to a fixed coordinate grid derived from the card dimensions before the canvas is instantiated; never compute anchors from the string.
  3. Clamp each y anchor into the printable band with a max(min_y, min(y, max_y)) guard so a mis-sized field can never escape the card.
  4. For strings that must fit a known width, measure with canvas.stringWidth(s, font, size) and truncate to fit rather than trusting the source data.

Spooler-Rejecting PDF Streams

Symptoms

  • The print queue rejects the file with PDFDocException or a malformed-stream error.
  • Some badges render and the file still fails validation downstream.
  • QR codes scan inconsistently or the barcode checksum is rejected.

Root cause. Untrusted markup and control characters leak into the content stream, and an unbalanced graphics-state stack corrupts the document structure.

Fix

  1. Strip HTML and non-printable characters at ingestion (re.sub(r"<[^>]+>", "", s) plus an isprintable() filter) before any string reaches the canvas.
  2. Validate barcode and QR payloads against their checksum/length rules and quarantine anything that fails rather than drawing it — malformed records go to the dead-letter queue for review, never onto a badge.
  3. Balance every saveState() with a restoreState(), or avoid manual state pushes entirely and draw each badge with a stateless function.
  4. Pin a known-good ReportLab release; a minor upgrade can change default stream compression and break a legacy RIP (see rollback below). Register any non-standard font with reportlab.pdfbase.pdfmetrics.registerFont so glyphs embed instead of silently falling back.

Minimal Working Implementation

The pattern below is self-contained and runnable: a strict sanitizer, a stateless per-badge draw function with coordinate clamping, and a chunked writer that caps memory and routes malformed records to quarantine. It consumes the same render-safe record produced upstream and chooses ReportLab’s explicit pdfgen canvas over platypus, whose flowable pagination conflicts with rigid die-cut tolerances. Barcode and QR specifics — encoding mode, error-correction level, and module size — belong to QR code generation and barcode threshold tuning; this code only places the finished matrix at a fixed anchor.

Chunked, memory-capped ReportLab badge writer with quarantine routing Attendee records are streamed lazily from an iterator into a sanitize_payload gate that NFKC-normalizes each field, strips HTML and control characters, clamps field lengths, and verifies the QR checksum length. Records that fail — bad markup or a short checksum — peel off on a dashed branch into a quarantine.json dead-letter sink. Surviving records accumulate in a fixed-size buffer. A decision node checks whether the buffer has reached chunk_size; while it is below the limit the pipeline keeps buffering, and once it reaches the limit the buffer flushes through a fresh per-chunk canvas.Canvas into an atomic badges_chunk_NNN.pdf written with fsync. A dashed memory-cap loop returns from the flushed chunk back to the buffer, where buffer.clear(), gc.collect(), and tracemalloc.reset_peak() bound resident memory before the next chunk begins. record passes the gate fails validation → quarantine memory-cap loop 1 · NFKC normalize 2 · strip HTML + ctrl 3 · clamp field length 4 · QR checksum len attendee records streamed iterator sanitize _payload checksum fail · bad markup quarantine.json dead-letter sink buffer fixed size · chunk_size len(buffer) ≥ chunk_size else: keep buffering flush fresh canvas.Canvas badges_chunk_NNN.pdf atomic write · fsync memory-cap loop · buffer.clear() · gc.collect() · tracemalloc.reset_peak()
PYTHON
import gc
import io
import os
import re
import json
import unicodedata
import tracemalloc
from pathlib import Path
from dataclasses import dataclass
from typing import Iterator, Generator

from reportlab.lib.units import mm
from reportlab.lib.pagesizes import A4
from reportlab.pdfgen import canvas
from reportlab.graphics.shapes import Drawing
from reportlab.graphics import renderPDF
from reportlab.graphics.barcode.qr import QrCodeWidget

# --- Fixed grid: CR80 card (85.6 x 53.98 mm), points = 1/72 inch ---
BADGE_WIDTH = 85.6 * mm
BADGE_HEIGHT = 53.98 * mm
MARGIN = 5 * mm
FONT_NAME, FONT_TITLE = "Helvetica-Bold", "Helvetica"
FONT_SIZE_NAME, FONT_SIZE_TITLE = 14, 10


@dataclass(frozen=True)
class BadgeSchema:
    max_name_len: int = 32
    max_title_len: int = 40
    qr_checksum_len: int = 8


def sanitize_payload(raw: dict, schema: BadgeSchema = BadgeSchema()) -> dict:
    """Deterministic pre-render pass: strip markup, NFKC-normalize, clamp, verify QR."""
    def clean(value: str, limit: int) -> str:
        text = re.sub(r"<[^>]+>", "", str(value)).strip()
        text = "".join(ch for ch in text if ch.isprintable())
        return unicodedata.normalize("NFKC", text)[:limit]

    qr = str(raw.get("qr_payload", ""))
    if len(qr) < schema.qr_checksum_len:
        raise ValueError(f"QR payload too short: {len(qr)} chars")

    return {
        "name": clean(raw.get("name", ""), schema.max_name_len),
        "title": clean(raw.get("title", ""), schema.max_title_len),
        "qr": qr,
        "tier": clean(raw.get("tier", "GA"), 8).upper(),
    }


def _clamp(val: float, low: float, high: float) -> float:
    return max(low, min(val, high))


def render_badge(c: canvas.Canvas, data: dict, x: float, y: float) -> None:
    """Stateless draw: absolute coordinates, every anchor clamped into the card."""
    c.setFillColorRGB(0.95, 0.95, 0.97)
    c.rect(x, y, BADGE_WIDTH, BADGE_HEIGHT, fill=1, stroke=0)

    c.setFillColorRGB(0, 0, 0)
    name_y = _clamp(y + BADGE_HEIGHT - MARGIN - FONT_SIZE_NAME,
                    y + MARGIN, y + BADGE_HEIGHT - MARGIN)
    c.setFont(FONT_NAME, FONT_SIZE_NAME)
    c.drawString(x + MARGIN, name_y, data["name"])

    title_y = _clamp(name_y - FONT_SIZE_TITLE - 6,
                     y + MARGIN, y + BADGE_HEIGHT - MARGIN)
    c.setFont(FONT_TITLE, FONT_SIZE_TITLE)
    c.drawString(x + MARGIN, title_y, data["title"])

    # QR matrix at a fixed anchor, built inside the call so it stays collectable.
    qr_size = 25 * mm
    widget = QrCodeWidget(data["qr"])
    bounds = widget.getBounds()
    scale = qr_size / (bounds[2] - bounds[0])
    drawing = Drawing(qr_size, qr_size, transform=[scale, 0, 0, scale, 0, 0])
    drawing.add(widget)
    renderPDF.draw(drawing, c,
                   x + BADGE_WIDTH - qr_size - MARGIN, y + MARGIN)
    c.showPage()


def _write_chunk(records: list[dict], out_dir: Path, idx: int) -> Path:
    """One fresh canvas per chunk; fsync so the file is atomically visible."""
    out_path = out_dir / f"badges_chunk_{idx:03d}.pdf"
    c = canvas.Canvas(str(out_path), pagesize=A4)
    for rec in records:
        render_badge(c, rec, x=MARGIN, y=A4[1] - BADGE_HEIGHT - MARGIN)
    c.save()
    fd = os.open(str(out_path), os.O_RDONLY)
    os.fsync(fd)
    os.close(fd)
    return out_path


def generate_badges(
    records: Iterator[dict],
    out_dir: Path,
    chunk_size: int = 250,
    memory_limit_mb: int = 128,
) -> Generator[Path, None, None]:
    """Chunked, memory-capped writer with deterministic quarantine routing."""
    out_dir.mkdir(parents=True, exist_ok=True)
    tracemalloc.start()
    quarantine, buffer, idx = [], [], 0

    for raw in records:
        try:
            buffer.append(sanitize_payload(raw))
        except Exception as exc:                       # noqa: BLE001
            quarantine.append({"raw": raw, "error": str(exc)})
            continue

        if len(buffer) >= chunk_size:
            yield _write_chunk(buffer, out_dir, idx)
            buffer.clear()
            idx += 1
            _, peak = tracemalloc.get_traced_memory()
            if peak > memory_limit_mb * 1024 * 1024:
                gc.collect()
                tracemalloc.reset_peak()

    if buffer:
        yield _write_chunk(buffer, out_dir, idx)
    if quarantine:
        (out_dir / "quarantine.json").write_text(json.dumps(quarantine, indent=2))
    tracemalloc.stop()


# --- Verification step ---
if __name__ == "__main__":
    sample = ({"name": f"Attendee {n}", "title": "Engineer",
               "qr_payload": f"CHK{n:07d}"} for n in range(1000))
    out = Path("/tmp/badge_out")
    chunks = list(generate_badges(sample, out, chunk_size=250))
    assert len(chunks) == 4, chunks          # 1000 records / 250 per chunk
    assert all(p.stat().st_size > 1024 for p in chunks)  # no truncated writes
    print(f"OK: {len(chunks)} chunks, peak stayed bounded per chunk")

Memory and Performance Constraints

These are the resources the fix actually touches during a registration surge; tune them together, since dropping chunk_size trades throughput for a lower memory peak.

Component Constraint Mitigation
Canvas lifetime One canvas holds all page/font state until save() Fresh Canvas per chunk; drop the reference and gc.collect() between chunks
chunk_size Larger chunks raise the memory peak; smaller chunks add file overhead Default 250; drop to 100 under memory pressure, raise for throughput
Record stream Materializing the full manifest as a list spikes RSS Consume an iterator/generator; never list() the source export
Font subsetting Full embedding inflates every PDF and CPU cost Register and subset fonts once; embed subsets only, not full faces
Worker concurrency (GIL) ReportLab draw calls are CPU-bound and hold the GIL Scale process-per-core, not threads; one interpreter per worker
Quarantine growth A bad export can flood quarantine.json Alert when quarantine rate exceeds ~0.5% of records

Incident Triage and Rollback

When a batch run stalls, bleeds coordinates in print, or the spooler starts rejecting files, work these steps in order. Target MTTR is under fifteen minutes.

  1. Classify the failure. Check the worker’s memory peak: grep tracemalloc /var/log/badge-render.log | tail. A climbing peak points to memory growth; a clean peak with print complaints points to coordinate drift or a bad stream.
  2. Contain memory growth. Lower the chunk size and restart the affected run: BADGE_CHUNK_SIZE=100 systemctl restart badge-render. Confirm the per-chunk peak stays under the limit.
  3. Inspect quarantine. jq '.[].error' /var/badge_out/quarantine.json | sort | uniq -c shows whether failures are short QR payloads, injected markup, or oversized fields — that names which fix to apply.
  4. Roll back on a bad stream. If the spooler rejection followed a dependency bump, pin the last known-good release — pip install 'reportlab==4.5.1' — and redeploy; newer minors can change default stream compression and break a legacy RIP.
  5. Fail over to the legacy path. For immediate continuity, set LEGACY_RENDER_MODE=1 to bypass the chunked writer and route through the single-file pipeline while you fix forward.
  6. Validate recovery. Regenerate one chunk and confirm it is well-formed before resuming: python -c "from pypdf import PdfReader; print(len(PdfReader('/var/badge_out/badges_chunk_000.pdf').pages), 'pages OK')". Delete any partial .pdf under 1 KB; the pipeline regenerates it on retry, and quarantined records are only replayed after manual review.