The Challenge
The Challenge
The ecosystem relied on custom-built APIs to process notifications from external systems. This led to:
- High Costs: Each integration required unique code, testing, and monitoring.
- Slow Onboarding: New notifications took significant time to integrate.
- Fragmented Processing: No standardized way to validate, route, or audit notifications.
Goal: Replace point-to-point integrations with a scalable, event-driven architecture.
The Solution: Kafka-Centric Architecture
The Solution: Kafka-Centric Architecture
-
Kafka REST Proxy:
- Provided a single entry point for external systems to publish notifications via HTTPs.
- Enabled authentication/authorization via API gateway.
-
Topic-Based Routing:
- Notifications were ingested into dedicated Kafka topics.
- Internal consumers (micro-services) subscribed to topics.
-
Phased Migration:
- Piloted with non-critical notifications, then scaled to high-volume scenarios.
- Unified Processing Pipeline
- Reduced integration time and maintenance costs
-
The solution won Tech Superstars and Legends Award (September 2024)