Shahab | Senior Data Engineer

Hey there 👋

I'm Shahab Akmal

Senior Data Engineer & Systems Architect

7+ Years Experience Kraken & Shopify Alumni Waterloo M.Eng

Specialized in scaling high-throughput Spark & Flink engines, distributed stream processing, relational modeling, and cloud-native Kubernetes data infrastructure. Reduced compute costs by 90% and co-architected autonomous LLM self-healing pipelines.

90%

Compute Cost Reduction

Spark optimization & partition pruning for Kraken compliance reporting

Billions/Day

Real-Time Merchant Stream

Flink & Kafka event pipelines with 60% faster search response at Shopify

Self-Healing

Autonomous ETL Patches

ReAct LLM agents + pgvector querying stack traces in Docker sandboxes

GitHub Profile

Featured Projects </>

Real-Time Event Processing & Analytics LakehouseFeatured

Replicates PostgreSQL transactional data in real-time via Debezium CDC and Kafka Connect into ClickHouse OLAP warehouse with sub-2-second sync delay.

CDCKafkaClickHousedbtRust

Event-Driven Message Routing EngineOpen Source

High-performance streaming pipeline built with Apache Kafka and Python for message routing and telemetry validation.

KafkaPythonStreaming

Kubernetes-Native Spark & Airflow PlatformArchitecture

Local K8s cluster setup orchestrating distributed Apache Spark jobs using Airflow and monitored with Prometheus & Grafana.

SparkAirflowKubernetes