Fintech / Rewards14-week deliverySolutions Architect

Savyour High-Throughput Fintech SaaS Ledger

Enterprise Cashback & Affiliate Ecosystem

How I architected high-throughput webhook ingestion, double-entry financial ledgers, and hierarchical Redis caching for a consumer rewards platform serving 100+ brand partners and millions of monthly transactions.

100+ brand partners99.4% cache hit ratio<50ms P95 wallet query
System Architecture Blueprint
S
SavyourIdempotent Webhook Ingestion & Cache
100+ partners
Partner WebhookHMAC-SHA256 PayloadNginx GatewayRate Limiter (2500/s)Ingestion RouterHeader GateBloom FilterDeduplication KeyRewards CalcBase & Promo RatesSQS Ingest QueueDecoupled BufferRedis Cache EvictionWallet NamespacePostgreSQL DBRepeatable Read ACIDDouble-Entry LedgerBalance MutexIdempotence SyncDeduplication SyncWallet Balance SyncDisbursement API
01Ingress
Partner WebhookHMAC-SHA256 Payload
Nginx GatewayRate Limiter
Ingestion RouterHeader Gate
02Deduplication
Bloom FilterDeduplication Key
Rewards CalcBase & Promo Rates
03Buffering
SQS Ingest QueueDecoupled Buffer
Redis Cache EvictionWallet Namespace
04Ledger
PostgreSQL DBRepeatable Read ACID
Double-Entry LedgerBalance Mutex
05Settlement
Idempotence SyncDeduplication Sync
Wallet Balance SyncDisbursement API
[SYSTEM DETAILS]: Hover any architecture node to inspect structural details, runtime constraints, and scaling tradeoffs.

The problem

Affiliate webhooks are noisy, duplicated, and financially dangerous.

Savyour operates at the intersection of hundreds of affiliate networks and consumer wallets. When a major shopping festival launched, merchant partners hammered the backend with hundreds of thousands of status updates, purchase confirmations, and order cancellations.

A single duplicate webhook processing bug would credit real money to a user's wallet twice. Without immutable double-entry bookkeeping, auditing where money entered and exited the system required manual database forensic queries.

The challenge was to engineer a high-concurrency ingestion barrier that validates signatures, guarantees idempotency, maintains absolute ledger consistency, and scales reads seamlessly.

Operating constraints

The financial rules that governed every database transaction.

When an application handles real money, architectural shortcuts result in regulatory penalties and balance corruption. These constraints defined our technical boundaries.

Latency SLA

<50ms P95 Balance Lookups

Consumer mobile app users expected real-time wallet balance displays, preventing expensive dynamic balance aggregations on every user screen load.

Ledger Consistency

Double-Entry Balance Accounting

Zero financial discrepancy tolerance across millions of historical user reward line items and merchant commission credits.

Throughput Resilience

Burst Ingestion Capacity

Mega-sale campaigns (11.11 / Black Friday) generated 5,000+ simultaneous webhook hits from partner networks that could not be dropped or delayed.

Architecture decisions

Designing for zero financial drift and high-throughput ingestion.

Every architectural boundary prioritized mathematical auditability, cryptographic payload verification, and system resilience under volatile third-party traffic surges.

Idempotent Webhook Processing via HMAC & Redis Keys

Problem

100+ affiliate networks and merchant partners dispatch webhook notifications with unpredictable retry behaviors, out-of-order deliveries, and duplicate payloads that threatened wallet double-crediting.

Decision

Engineered an ingestion barrier verifying HMAC-SHA256 partner signatures and enforcing deterministic idempotency keys via Redis atomic operations (SETNX with 72-hour TTL) before entering async task queues.

Tradeoff accepted

Added ~15ms of ingestion verification latency, but mathematically eliminated duplicate transaction credits across millions of webhook events.

Immutable Double-Entry Ledger with Serializable PostgreSQL

Problem

Concurrent transaction events (crediting cashback while users initiated bank withdrawals) risked balance drift, race condition exploits, and non-reconcilable financial discrepancies.

Decision

Implemented a strict double-entry bookkeeping ledger where user balances are computed from immutable credit/debit transaction lines under PostgreSQL Serializable isolation levels.

Tradeoff accepted

Requires rigorous transaction isolation and advisory locks on hot accounts, but guarantees 100% financial audit integrity.

Two-Tier Hierarchical Redis Invalidation Architecture

Problem

Shopping festival surges caused millions of concurrent database queries against brand commission tables, partnership tiers, and user balance caches, threatening database saturation.

Decision

Designed a hierarchical Redis caching architecture with namespace-level invalidation triggered via admin partner updates and probabilistic early expiration (XFetch) to prevent cache stampedes.

Tradeoff accepted

Cache eviction patterns require exact key hygiene across microservices, but achieved a sustained 99.4% cache hit ratio during peak campaigns.

Decoupled Asynchronous Payout Outbox Engine

Problem

Third-party banking APIs and 1-Link payout gateways frequently timed out or rate-limited requests during high-volume withdrawal windows, blocking client threads.

Decision

Implemented the Transactional Outbox pattern, queueing payout jobs in Celery/RabbitMQ with exponential backoff, jittered retries, and automated operator dead-letter escalation.

Tradeoff accepted

Payouts execute asynchronously with a 1-2 minute verification window rather than synchronous execution, but unhandled gateway failures dropped to zero.

System architecture

How the financial pipeline flows.

Financial Ingestion & Ledger Security Boundary
100+ Partner WebhooksAffiliate networks & merchants
HMAC & Idempotency GateSHA-256 validation & Redis SETNX
Async Task Queue (Celery)Decoupled event calculation & commission splits
Two-Tier Redis CacheCached brand tiers & balance reads
Payout Outbox EngineBank & 1-Link gateway dispatch queue
Double-Entry Ledger StorePostgreSQL Serializable ACID records
Redis Sentinel

Idempotency & cache cluster

PostgreSQL RDS

ACID double-entry ledger

RabbitMQ / Celery

Async transaction queues

Bank & 1-Link APIs

Disbursement endpoints

Double-entry consistency guaranteed. Balance queries never read arbitrary mutable columns; they verify the sum of signed credit/debit entries across auditable ledger accounts.

Failure modes & defenses

How financial edge cases and ingestion failures are neutralized.

Financial ledgers cannot rely on optimistic assumptions. These are the worst-case failure scenarios pressure-tested during implementation and the concrete defenses engineered into the backend.

Failure Mode 01

Duplicate Affiliate Conversion Webhooks

Financial Risk

Network retries from merchant affiliate networks crediting the consumer's wallet multiple times for a single order.

Architectural Defense

HMAC request signature verification coupled with atomic Redis idempotent key caches (7-day TTL). Duplicate requests resolve to cached 200 OK without reaching the database.

Failure Mode 02

Database Lock Contention on Balances

Financial Risk

Multiple cashback events trying to mutate the same user's balance row concurrently, leading to transaction rollbacks.

Architectural Defense

Immutable append-only ledger architecture. New transactions write new credit/debit journal rows with zero row locks on user profiles, with balances projected asynchronously.

Failure Mode 03

Redis Eviction & Cache Stampede

Financial Risk

Sudden cache invalidation on popular brand commission rates triggering hundreds of simultaneous PostgreSQL queries.

Architectural Defense

Hierarchical cache warming with probabilistic early expiration (XFetch) and distributed mutex locking on cache misses to ensure only one worker re-queries PostgreSQL.

Failure Mode 04

Order Return Fraud & Chargebacks

Financial Risk

Consumers collecting cashback and immediately cancelling the purchase on the merchant's store.

Architectural Defense

Two-phase settlement state machine (Pending -> Verified -> Cleared). Cashback remains locked in escrow until the merchant's return dispute window expires.

Build timeline

Fourteen weeks to enterprise financial stability.

Week 1-2

Financial architecture and risk discovery. Audited affiliate webhook endpoints, mapped double-credit vulnerabilities, and designed the immutable ledger schema.

Week 3-4

Built the idempotent ingestion gateway: HMAC validation, Redis atomic deduplication locks, and high-throughput SQS ingestion buffers.

Week 5-6

Engineered double-entry ledger backend: PostgreSQL Serializable transaction boundaries, debit/credit audit logging, and automated balancing triggers.

Week 7-8

Designed hierarchical Redis caching layers: brand commission caches, user balance projections, and cache stampede protection.

Week 9-10

Built payout outbox engine: asynchronous 1-Link bank gateway adapters, retry state machines, and dead-letter reconciliation playbooks.

Week 11-12

Load testing with 5,000+ concurrent simulated webhook bursts, chaos engineering on database failover, and zero-downtime cutover.

Week 13-14

Telemetry dashboards, automated discrepancy alarms, security penetration testing, and engineering team handoff.

Outcomes

Hard metrics from real consumer scale.

Eliminated duplicate payout exploits while maintaining responsive balance reads during peak nationwide retail promotions.

100+

Integrated brand partners

Supported across Daraz, Foodpanda, local retailers, and international affiliate networks without ingestion failures.

99.4%

Cache hit ratio

Sustained cache hit rate on high-velocity brand commission queries during national shopping campaigns.

<50ms

P95 wallet query latency

Cached balance retrieval under high concurrent mobile consumer traffic.

100%

ACID ledger reconciliation

Zero financial balance discrepancies across millions of double-entry ledger line items.

Work together

Building high-concurrency transactional systems?

Let's build an idempotent, verifiable data pipeline that keeps your financial records accurate and your customer experience instant.