Document Management

The household's paper archive — every bill, bank statement, insurance policy, contract and certificate worth keeping — lives at documents.mdapi.ch, running Paperless-NGX in the paperless namespace. Documents arrive from three independent paths, are OCRed and classified automatically, and end up filed in a {year}/{correspondent}/{title}.pdf archive tree that is browsable read-only through OwnCloud without anyone having to log in to Paperless itself.

The point of writing this page up is less Paperless the app and more the surrounding integration shape: how a single source of truth for documents gets fed by three ingestion paths, exposed through two UIs (Paperless for tagging and search, OwnCloud for "just give me the PDF"), and backed up alongside everything else.

Architecture

flowchart LR
    subgraph in["Ingestion"]
        oc_in["OwnCloud<br/>Paperless-Consume share"]
        mail["IMAP mailbox<br/>documents@mdapi.ch"]
        api["Direct API upload"]
    end

    subgraph pod["paperless pod"]
        puller["rclone puller<br/>(sidecar, 5-min)"]
        unzip["archive-extractor<br/>(sidecar, .zip/.7z/.tar*)"]
        app["paperless-ngx<br/>web · consumer · celery"]
        pusher["library-pusher<br/>(sidecar, 5-min)"]
    end

    subgraph svc["In-namespace services"]
        valkey["Valkey<br/>(Redis-compatible cache)"]
        pg["paperless-pg<br/>(CloudNativePG)"]
    end

    subgraph store["Storage"]
        pvc["paperless-data-ceph PVC<br/>100Gi Ceph RBD (ceph-block)<br/>{year}/{correspondent}/{title}.pdf"]
    end

    subgraph out["Read-only browse"]
        oc_out["OwnCloud<br/>Paperless-Library share"]
    end

    oc_in --> puller --> unzip --> app
    mail --> app
    api --> app
    app <--> valkey
    app <--> pg
    app --> pvc
    pvc --> pusher --> oc_out

The paperless pod runs six containers in the same network namespace: the app itself, its tika and gotenberg companions (document conversion/preview for paperless-ngx upstream), and three custom sidecars that make the integrations possible without modifying paperless-ngx itself.

Ingestion Paths

OwnCloud Paperless-Consume is the everyday path. Tillo and Elisa each have a write-enabled share into a directory the rclone puller sidecar polls every 5 minutes. Anything that lands there (including subdirectories — tag names get inherited from the path) is pulled into paperless-ngx's /consume/ directory and removed from OwnCloud once processed. The puller carries --delete-empty-src-dirs so the user-visible folder structure stays clean: an empty subdir means no documents waiting, not something is stuck.

Archives (.zip, .tar(.gz/.bz2/.xz), .7z) get unpacked by the archive-extractor sidecar before paperless-ngx sees them. paperless-ngx upstream doesn't natively handle archives, so this sidecar exists to bridge "user drops a ZIP of receipts on the share" to "paperless sees individual PDFs in the consume queue". Charset-aware ZIP handling recovers cp437 filenames from old Windows-era exports.

IMAP mailbox documents@mdapi.ch is polled every 5 minutes. Anything arriving with an attachment matching the rule (PDF, scanned image, common office formats) is extracted, fed to the consume pipeline, and the source email is moved to a Processed folder. This is the path for "forwarded receipt from a vendor" or "scan from a multifunction printer that emails to documents@".

Direct API upload is available for scripts and for paperless-ngx's own mobile app; in practice it's used rarely because the OwnCloud path is easier for non-technical users.

Classification, Workflows and Custom Fields

paperless-ngx's auto-classifier is retrained nightly against the manually-curated tag / correspondent / document-type set. Tags created in "Any word" mode propagate from filenames and folder paths; tags created in "Automatic" mode are predicted by the classifier from the OCR text. Both modes coexist on the same document.

Workflows route documents based on the correspondent field (which represents the issuing entity — bank, government, insurer — not the recipient). Eight are deployed today: six correspondent-triggered workflows — PostFinance, BCBE, Etat de Vaud, Assura, Kyos, CPEV — each with a conservative trigger (correspondent contains "<name>") and a small downstream effect (tag adjustment, document-type assignment, archive-filename template), covering the recurring sources of paperwork in the household; plus two structural workflows that apply across correspondents — one templates the archive storage path, the other auto-extracts custom fields on every consumed document. Everything else falls through to the default catch-all.

Custom fields add structured metadata that paperless-ngx's tag/type/correspondent model can't express. Five fields are defined: amount, due_date, valid_until, iban, policy_number — the earlier paid field was retired in favor of a pagata tag (see Saved Views below). Two have automatic extraction: a currency-aware regex pulls amount (CHF / EUR / USD) and a Swiss-IBAN-with-checksum-validation routine pulls iban. The other three require manual entry — the extraction signal is too weak (random "policy numbers" look like everything else) to backfill without supervision.

The combination of correspondent for routing, tags for retrieval, custom fields for structured query is what makes "show me every Postfinance invoice over CHF 1000 that's unpaid" expressible as a saved view.

Automated Matching & Review

Two Windmill pieces sit on top of the built-in classifier:

• f/paperless/paperless_auto_matching keeps the matching configuration honest: for every document type and correspondent, it flips the matching algorithm to Auto once the category holds enough documents to train on, and back to None when it doesn't. New categories promote themselves as their corpus grows; nobody hand-tunes matching flags.
• A human-in-the-loop review queue drains the inbox / needs-review backlog one card at a time: paperless_review_next serves the highest-priority document awaiting a verdict (silently un-tagging documents the classifier now agrees with), paperless_review_decide applies the human's decision, and a small review-deck app wraps the pair into a one-tap triage UI. The classifier proposes; the queue disposes.

Field Protection (Kustodyan RPS)

Sensitive document metadata — the kind of structured values the custom fields hold, IBANs and policy numbers among them — is protected by the RegData RPS engine (Kustodyan, running in the paperless-kustodyan namespace) through its Engine API: values are protected on write and unprotected on read, with every call carrying a role + action evidence pair that the engine validates server-side. The protection policy lives in the engine's configuration, not in Paperless — Paperless stores protected values and never holds the transformation logic.

Storage and the Archive Tree

The paperless-data-ceph PVC is 100 GiB on the in-cluster Ceph tier (RBD, ceph-block StorageClass). The default Paperless layout is a flat documents/ directory keyed by document ID, which works for the app but makes any direct-filesystem use (a recovery, an audit, a manual bulk operation) painful. The deployed layout overrides Paperless's filename format to {year}/{correspondent}/{title}.pdf, which:

• Mirrors the way a human would organize a physical filing cabinet — open the binder for 2025, open the PostFinance tab, find the statement.
• Survives a Paperless DB restore better — if the metadata database is lost, the filesystem alone still makes sense.
• Makes the library-pusher sidecar (below) trivial: it can rsync paperless-data/archive/{year}/... straight into OwnCloud without translating IDs back to names.

OwnCloud Bidirectional Bridge

The integration is symmetric across two shares:

• Paperless-Consume (write, two users) — for ingestion. WebDAV in.
• Paperless-Library (read-only, two users) — for browsing. WebDAV out, kept in sync every 5 minutes by the library-pusher sidecar which mirrors the archive tree from the Paperless PVC into OwnCloud via the same WebDAV endpoint.

The architectural note worth recording is that OwnCloud's OCS API (the documented mechanism for managing shares) consistently returns 401 even with valid credentials — this is a permanent OwnCloud-side behaviour, not a misconfiguration. The shares are created instead by direct PostgreSQL inserts into oc_share, which OwnCloud reads on every share validation. WebDAV traffic — both directions — goes through unchanged. The pattern (use the supported channel where it works, fall back to the data plane where it doesn't) generalises to other OwnCloud integrations.

Net effect: a user who never logs into Paperless can drop a PDF into their Paperless-Consume folder and find the OCRed, classified, filed copy in Paperless-Library within ten minutes — five for the consume puller to pull, five for the library pusher to mirror.

Identity, Backup, Monitoring

• Identity — Paperless is fronted by OIDC against the tilloch realm on the MDAPI Keycloak. Same realm as Joplin, OwnCloud and the rest of the household-user services; one account, one MFA enrollment.
• Backup — paperless-pg is a CloudNativePG cluster with Barman continuous WAL to Garage (no ScheduledBackup / periodic base backup configured yet); the off-site rclone fan-out at 04:00 UTC ships both to Backblaze B2 and o2switch SFTP. The 100 GiB paperless-data-ceph PVC itself isn't backup-enrolled yet — it moved off Longhorn onto the Ceph tier recently and has no VolSync ReplicationSource configured (see Backups & DR).
• Monitoring — paperless-ngx exposes Celery queue depth, consumer worker state and OCR job latency through the standard Prometheus annotations; the WAF audit on documents.mdapi.ch is part of the daily /waf-audit pass.

Saved Views

Eight dashboard views ship on the home page and the left sidebar, covering the queues an operator-of-the-archive looks at routinely:

View	Query
Inbox	tag:inbox — triage queue, every newly-consumed document starts here
Needs review	tag:needs-review — classifier marked low-confidence, human verdict needed
Open invoices	document_type:Fattura AND NOT tag:pagata — unpaid invoices; replaced the retired paid custom field with the pagata tag
Insurance policies	document_type:Polizza
Bank statements	document_type:Estratto
Without document type	document_type:none — triage: documents still missing a type
Without correspondent	correspondent:none — triage: documents still missing a correspondent
All Fatture	document_type:Fattura — every invoice regardless of paid status

A current Paperless limitation is that saved views can't use relative dates ("this month", "in the last 90 days") — the filter expressions are absolute. The work-around when that becomes painful enough is a weekly Windmill regenerator script that rewrites the date bounds in place; for now the absolute-date views are good enough.

Related Pages

• Backups & DR — where the 100 GiB PVC and the CNPG cluster land in the wider backup architecture.
• Authentication & Identity — the OIDC realm Paperless authenticates against.
• Storage Architecture — the underlying Ceph class the PVC uses, and the Garage buckets the Postgres Barman stream lands in.
• ModSecurity WAF — the WAF carveouts (or absence thereof) that govern public access to documents.mdapi.ch.