A Senior Data Engineer responsible for architecting scalable data lakehouse solutions, developing APIs, building data pipelines with Spark and Flink, and establishing data contracts to support Sword Health's AI-driven healthcare platform.
Key Responsibilities
Architect a scalable lakehouse by migrating workloads to Apache Iceberg
Develop a Jobs API for data processing workflows
Build Spark and Flink pipelines for data processing
Establish and enforce data contracts to ensure data quality and consistency
Collaborate with cross-functional teams to support data platform needs
Utilize Python and Kafka to develop and maintain data solutions
Requirements
Architect a scalable lakehouse by migrating workloads to Apache Iceberg.
Develop a Jobs API.
Build Spark Flink pipelines.
Establish data contracts.
Proficiency in Python.
Proficiency in Kafka.
Candidates must possess a valid EU visa and be based in Portugal.