Kafka tuned for fintech transactions
Load tested and tuned Kafka on Kubernetes for high-throughput, low-latency transaction processing.
- Client
- US fintech clients
Situation
Fintech clients processed transactions through event streaming on Kafka, on GKE and on self-managed Kubernetes clusters. The clusters had to stay available and handle high-throughput, low-latency transaction processing.
What I did
- Built and maintained highly available Kafka clusters with ksql on Kubernetes.
- Load tested the event streaming architecture and tuned Kafka and the Kubernetes workloads around it.
- Built multi-environment Codefresh pipelines into Databricks.
Result
Tuning raised the capacity of the clusters for high-throughput, low-latency transaction processing.
Tools
- Kafka
- ksql
- Kubernetes
- GKE
- Codefresh
- Databricks