Architecture & Operations

Why financial state must survive provider disagreement

Your system says pending, the provider says completed. Why the last webhook is not enough to decide.

8 min read · Verified against the v4 API on 2026-10-11

The rule to remember: keep three things apart. What the provider reports, what your system decided, and what the ledger recorded. When they disagree, record the disagreement and collect evidence. Do not let the most recent message overwrite the other two.

The job

Two situations every payments team meets.

  • A bank-transfer pay-in shows pending in your system. The provider's portal shows it completed. The customer is asking why their balance has not changed.
  • A payout shows completed. Three days later the bank statement shows the transfer returned. The seller's balance still says the money left.

In both cases someone has to decide which state the product acts on, and what is allowed to change it. This guide is about making that decision on evidence.

Before you start

  • Written for CTOs, heads of payments, finance operations and engineers who own balances.
  • Assumes money moves through providers you do not control, and your product shows balances or statuses based on it.
  • Three terms are used throughout. Provider reality: what the provider or bank reports, through its API, its notifications, its portal or its statements. Internal state: the status your system assigns to the operation. Ledger: the journals your balances are derived from.
  • Maturity: Wallet & Ledger and pay-in webhooks are in production. The complete Proof of Record and Resolve are in development.

Flow

  1. Provider reality API, notification, statement
  2. Internal state Operation status
  3. Ledger Journals, derived balances
  4. Reconciliation Compare all three
  • They agree Verified outcome
  • They disagree Evidence, then a decision

The reasoning

Three records, written by different parties

The provider writes its own record, on its own clock, and can change its mind: a transfer is accepted, then returned; a payment succeeds, then is reversed. Your system writes the status it decided on from the messages it received. The ledger records the financial effect of that decision as balanced journals.

None of the three is automatically right. The provider is the authority on what its rail did. Your ledger is the authority on what your platform recorded. A disagreement between them is a fact to record and investigate, not a bug to overwrite.

Case A: pending here, completed there

  1. 10:14:02 The payer completes a PSE payment at their bank.
  2. 10:14:05 The provider marks the payment completed.
  3. 10:14:06 The provider's notification to your platform fails. Their retries fail too.
  4. 10:40:00 Support sees "completed" in the provider portal and "pending" in your system.

The quick fix is to change the status to completed by hand, or to credit the wallet directly. Both create a third problem: a status or a balance with no financial event behind it. A week later nobody can explain the number, and if the notification eventually arrives, the wallet can be credited twice.

The safer path: get evidence for this payment (the provider's status for its reference, the notification itself, or the settlement line), then let the change go through the same path that updates the status and posts the ledger entry. Status and balance move together, and the evidence is attached to the change.

Case B: completed here, returned there

  1. Day 1, 09:00 A seller payout completes. A journal debits the seller's wallet.
  2. Day 4, 08:30 The bank statement shows the transfer returned: the beneficiary account was closed.
  3. Day 4, 08:31 The seller's balance still shows the money as paid out.

Editing the original journal, or flipping the payout to failed, would make the ledger say the payout never happened. It did happen, and then it was returned. Those are two events. The return should be recorded as a new entry that references the original, so the seller's balance can be explained line by line: paid out on day 1, returned on day 4.

Why "trust the last webhook" fails

  • At-least-once delivery. The same event can arrive two or more times. Acting on each copy applies the effect twice.
  • No ordering. A retried payment.failed can arrive after a later payment.succeeded, or the reverse. "Last received" is not "last true".
  • Lost messages. After its retries, a notification can end up undelivered. Silence is not a status.
  • Late changes. Returns, reversals and chargebacks can arrive days after a success was reported.
  • Settlement files. The amount that settles can differ from what every message said, because of fees, partial settlement or currency handling.

A webhook is an observation: what the sender believed when it sent the message. Treat it as a reason to check current state, then act on that state.

Reconciliation compares; it does not decide

Reconciliation puts provider reality next to internal state and the ledger, and sorts the differences:

  • The provider has an operation you have no record of.
  • You have an operation the provider has no record of.
  • Both have it, with different statuses.
  • Both have it, with different amounts.
  • Status agrees, but the ledger entry is missing.

Each difference needs evidence and an owner who decides what happens next. The evidence worth keeping: provider references, raw notifications, status query responses, statement and settlement lines, ledger entries, and the operation's own timeline.

What Orangepill keeps today

  • Ledger. Balances are derived from double-entry journals posted by the ledger service. Posted amounts are not edited. Refunds and reversals are recorded as new inverse entries that reference the original payment.
  • Provider notifications on the pay-in path. Incoming provider notifications are stored and deduplicated by the provider's event id (or a hash of the payload when there is none) before they can change payment state or post to the ledger. A notification for an attempt that is already terminal does not move it back. When a provider notification moves a completed pay-in to refunded or reversed, a reversal is posted as a new entry.
  • Payout outcomes. Payout finality is taken from querying the provider's status for that transfer, not from a notification.
  • Webhooks to you. Orangepill's own events are delivered at least once, with no ordering guarantee. The X-Orangepill-Event-Id header is the same on every retry, so deduplicate on it, then read GET /v4/payments/{paymentId}/status before acting.
  • Evidence through the API. Payment timelines (/timeline) and attempts (/attempts), ledger entries by reference (GET /v4/ledger/entries/by-reference), webhook delivery history, and GET /v4/financial/explain, which returns a graph of how a payment, ledger entry or journal reached its state.
  • Pay-in reconciliation. Orangepill's operations tooling compares provider-reported pay-ins with payment status, settlement and ledger postings, and flags differences such as a missing ledger entry or an amount mismatch. It is an internal operator view, not a public API.

Proof of Record and Resolve In Development

Proof of Record is intended to be one record per operation linking intent, attempts, provider responses, ledger entries and outcome, with unknowns recorded as unknown. Today a Proof of Record token is minted for each succeeded settlement outflow, and the timelines, attempts and explain endpoints above carry the rest of the evidence.

Resolve is intended to govern what happens after reconciliation finds a disagreement it cannot settle automatically: Investigate, Establish, Decide, Verify, ending in Continue, Wait, Remediate or Escalate. It has no API today. Until it exists, disagreements are worked by your operations team with the evidence above.

What can go wrong

  • A status is edited by hand without a ledger entry. The status says completed and the balance does not, or the other way round.
  • Your webhook handler credits a balance on every delivery. A retried event credits the same payment twice.
  • Events are applied in arrival order. A late payment.failed overwrites a payment.succeeded in your own database.
  • A returned payout is "fixed" by editing the original record. The history disappears and the balance can no longer be explained.
  • Settlement differences are ignored. Fees or partial settlements build up as a gap between your ledger and the bank, found at month end.
  • The provider portal becomes the source of truth for support. Agents act on what they see there and bypass the path that keeps status and ledger together.

What Orangepill guarantees today

  • Balances are derived from double-entry journals posted by the ledger service.
  • Posted amounts are not edited. Refunds and reversals are new inverse entries that reference the original payment.
  • Provider notifications on the pay-in path are deduplicated before they can change payment state or post to the ledger.
  • A late provider notification does not move a terminal pay-in attempt back to another state.
  • Payout outcomes come from the provider's status for the transfer, not from a single notification.
  • Outbound webhooks carry the same event id and payload on every retry.

What Orangepill does not guarantee

  • That providers, banks and Orangepill never disagree. Disagreement is expected and has to be handled.
  • Ordering of webhooks, or delivery of every notification from every provider.
  • Automatic detection of every late return or reversal on every rail. How returns reach Orangepill depends on the provider.
  • A complete execution record per operation. Proof of Record is in development.
  • Automated resolution of disagreements. Resolve is in development.
  • Custody. Orangepill records financial positions; funds stay with the banks, providers and licensed institutions involved.

Production considerations

  • Decide which record answers which question. The provider and bank answer "did money move". The ledger answers "what is the balance". The operation status answers "what does the product show".
  • Store provider references and raw notifications with every operation.
  • Reconcile against statements and settlement files on a fixed schedule, with a named owner for each type of difference and an age limit before escalation.
  • Make every correction a new, approved entry that references the original. No edits to posted records.
  • Keep webhook handlers idempotent on the event id, and re-read current status before any financial action.
  • Give your operations team access to timelines, attempts, ledger entries by reference and GET /v4/financial/explain before launch, not after the first incident.

The ledger model is described on Wallet & Ledger, and the integrity properties on Financial integrity.

Walk through your flow with us.

Bring the failure case that worries you most. We will show how the runtime handles it today.