Job Description
The role involves leading the scaling and reliability of Gridmatic's data infrastructure to support advanced energy price forecasting models, handling large-scale data ingestion, improving data pipeline robustness, and enabling the use of petabyte-scale weather data for climate and energy applications.
Key Responsibilities
- Lead the scaling of data infrastructure to handle petabyte-scale weather and other external data sources.
- Improve the reliability of data pipelines by addressing issues like downtime, late-arriving data, and changing schemas.
- Enhance data ingestion processes to support increased spatial granularity and larger data volumes for ML models.
Requirements
- Experience in scaling and improving the reliability of data infrastructure, including data pipelines and platforms.
- Ability to handle real-world data ingestion challenges such as downtime, late-arriving data, and changing schemas.
- Experience with scaling data processing capabilities by a factor of 10-100x, specifically incorporating petabyte-scale weather data into ML models.
- Knowledge of increasing spatial granularity in data for price forecasting models.
- Proficiency in designing and maintaining data pipelines that ensure high reliability and robustness.
- Strong understanding of data engineering principles and best practices for large-scale data systems.
- Experience working with external data sources and managing data quality issues.
- Ability to work collaboratively in a hybrid work environment, with in-person presence in Cupertino 3 days a week.
Benefits & Perks
Hybrid work policy requiring 3 days per week in Cupertino office
Opportunity to work on complex real-world problems related to energy and climate
Involvement in large-scale data challenges and ML applications
Collaborative team environment with expertise in ML, energy, and optimization
Ready to Apply?
Join Gridmatic and make an impact in renewable energy
Stay Updated on Sustainability Jobs
Get the latest renewable energy jobs and career tips delivered to your inbox.
Job Alerts
Get notified about new sustainability jobs
More jobs at Gridmatic
Lead Data Infrastructure Engineer
Gridmatic
Cupertino
Full Time
Dec 24
ML Performance Engineer
Gridmatic
Cupertino
Full Time
Dec 28
ML Performance Engineer
Gridmatic
Cupertino
Full Time
Jan 4