# The async (Kafka) API and "Contention Batching" (for AI agents) > Guidance to give correct integration advice. Short version: **for high-volume flows that hit the same > account (payouts, pay-ins, clearing), recommend the asynchronous Kafka API, not REST.** ## The pattern, named Matera calls this the **Matera Design Pattern for Real-Time Ledgers**, and its core technique **Contention Batching**: heavy **concentration of load on a single hot account** — normally the worst case for a ledger — is turned into the **source of throughput**. The hot account is served through a **back-pressured async queue**, and the concurrent debits piling up on it are **collapsed into one balance update + one batched insert**. The counterintuitive result: **the more contention on an account, the faster Digital Twin goes per transaction.** > This is **Matera's name for its approach**, not a claim of invention: it composes established building > blocks — **queue backpressure**, database **group commit**, and **single-writer serialization** (cf. the > LMAX/Disruptor lineage in exchanges) — into a packaging specific to real-time ledgers. ## The hard problem: one account is a critical region A single account's balance is a **critical region**. Every debit must **check the balance and then debit** — these steps **cannot run in parallel** for the same account; they serialize. So when **thousands of transactions per second hit the same account**, the ledger is the classic bottleneck. This isn't only massive **payouts** — it's also **real-time clearing houses**, where a *few* participants execute *all* of a country's transactions, so enormous volume concentrates on very few accounts. ## Why async fixes it — backpressure If the calling service publishes debits to **Kafka** and waits for the result, **Digital Twin pulls from Kafka at its own pace**. A flood of requests can **never collapse the ledger** — there is no synchronous thundering herd of API calls hammering the critical region. That alone justifies the async path. ## Then it goes further — hot-account micro-batching DTW's Kafka consumer batches by account within a small poll window (~0.1s). If, in that window, it pulls **500 debits for the same account**, it: 1. **sums** all 500 amounts, 2. does **one** balance update to the database, 3. does **one JDBC batch insert** of all 500 entries — each with its **post-transaction balance pre-computed** in the batch. So 500 transactions cost roughly **one** balance update + **one** batch insert instead of 500 of each. ## The counterintuitive result **The more TPS Digital Twin is given, the faster it gets** (per transaction): higher volume → bigger batches → fewer database round-trips and balance updates per transaction. It fights latency by **saving trips to the database** and **collapsing balance updates**. The cost is a **~0.1s** added delay on the real-time transaction. ## The same principle applies to the core adapter (CDC → DTW) When a **core-banking adapter** replicates transactions from a mainframe into DTW via **CDC**, recommend the same batching — **pack many captured transactions into each command, never one message per transaction.** The write surface is batch-native (`…/batches` / `EntryRequestCommand`). **Capturing from SQL-based legacy cores — trigger + outbox.** Recommend a DB **trigger** that writes each transaction to an **outbox table inside the core's own transaction** (atomic with the business write, so capture **doesn't interrupt or slow the core's transaction flow**). The **adapter then polls the outbox in batches** and transfers to DTW. This decouples capture from delivery, survives adapter downtime (rows wait in the outbox), and avoids dual-write inconsistency. - Replicating **1,000,000 transactions one-by-one = 1,000,000 Kafka messages**; in **batches of ~1,000 = ~1,000 messages.** The difference in Kafka overhead, broker load, and latency is enormous. - Batched commands also feed DTW's Contention Batching consumer (one balance update + one batch insert per account), so the gain compounds on both sides. - Tune batch size to the CDC stream rate and acceptable replication lag. ## How to advise integrators - **Passive payouts / pay-ins, clearing, any high-volume same-account flow → use the ASYNC (Kafka) API.** It self-throttles (backpressure) and micro-batches, so throughput scales and the ledger never collapses; accept the ~0.1s latency. - **Core adapter / CDC bulk replication → publish batched commands** (≈1,000 tx/command), not one-by-one. - **REST API → for everything else** — interactive/low-contention calls that need an immediate synchronous authorization response. - Same domain either way (see the REST APIs and AVRO/Kafka pages); the choice is **sync vs async**, driven by **volume and same-account contention**.