The Senior Data Engineer will architect and develop scalable data infrastructure, including migrating workloads to Apache Iceberg, building Spark Flink pipelines, and establishing data contracts, to enable a data-driven organization at Sword Health.
Key Responsibilities
Architect a scalable lakehouse by migrating workloads to Apache Iceberg
Develop a Jobs API for data processing workflows
Build Spark Flink pipelines for data processing
Establish data contracts to ensure data quality and consistency
Requirements
Proficiency in Python programming language.
Proficiency in Kafka messaging system.
Experience in architecting a scalable lakehouse architecture by migrating workloads to Apache Iceberg.
Experience developing a Jobs API.
Experience building Spark and Flink pipelines.
Ability to establish data contracts.
Candidates must possess a valid EU visa and be based in Portugal.