Failure Scenarios

Failure	Detection	Retry	Recovery	Compensation	Final State
Database outage	Health check fail	Circuit breaker	Failover to replica	Queue requests	Degraded
Kafka outage	Producer error	Buffer locally	Broker recovery	Replay buffer	Caught up
Redis outage	Cache miss spike	Bypass cache	ElastiCache failover	Direct DB reads	Normal
External API outage	Timeout	3 retries	Provider recovery	Manual fallback	Degraded
Service outage	K8s restart	Auto-restart	New pod	Route around	Recovered
Network timeout	Client timeout	Exponential backoff	Network recovery	Idempotent retry	Success
Duplicate messages	Idempotency check	N/A	Return cached	N/A	No-op
Consumer crash	Consumer lag alert	Auto-restart	Rebalance partitions	Replay from offset	Caught up
Partial payment failure	Saga timeout	Compensate	Reverse completed steps	Refund if needed	FAILED
Ledger failure	TX rollback	Retry 3x	Manual intervention	Payment FAILED	FAILED
Fraud service failure	Timeout	Default REVIEW	Service recovery	Manual review queue	REVIEW
Reconciliation failure	Job failure alert	Restart job	Fix data, re-run	Manual matching	RESOLVED