Main tasks & responsibilities
- Build and scale robust data pipelines using Python, SQL, Airflow, and DBT
- Develop and maintain real-time data processing with Kafka and Spark Structured Streaming
- Ingest and transform data from diverse sources and formats (JSON, Avro, Parquet, APIs, etc.)
- Influence and contribute to the evolution of our data platform and architecture
- Deploy and manage services using Kubernetes and IaC tools
- Mentor engineers and promote a collaborative, learning-focused team culture
- Collaborate with stakeholders to deliver actionable, high-impact data solutions
Requirements
- Strong Python and SQL skills
- Hands-on experience with the modern data stack
- Experience with Kubernetes for service orchestration
- Experience with distributed data processing using Spark or Trino
- Familiarity with Kafka (especially for data consumption use cases)
- Proven ownership of data initiatives from idea to production
- Comfortable working autonomously and making architectural calls
- Experience with cloud platforms (AWS, GCP, or Azure)