Apply Now
Application opens on company website
Job Description
A Data Engineer at Anew is responsible for designing, building, and maintaining scalable data infrastructure, including ETL pipelines, data storage solutions, and cloud-based data platforms, to support data-driven decision making and sustainability initiatives.
Key Responsibilities
- Design and implement ETL/ELT data pipelines from various sources into data repositories
- Create and maintain scalable data pipelines for large volumes of structured and unstructured data
- Automate data workflows through scripting and coding
- Monitor, troubleshoot, and optimize data pipelines for performance and reliability
- Design, maintain, and optimize data architecture, storage solutions, and data models
- Research and implement new data technologies and tools to improve infrastructure
- Manage and deploy data solutions on Azure cloud platform
- Ensure data quality, integrity, and security through validation, governance, and access controls
- Provide data access tools such as dashboards, analytics tools, and APIs
- Document technical designs, workflows, and processes to ensure transparency
Requirements
- Design and implement robust Extract, Transform, Load (ETL) or Extract, Load, Transform (ELT) pipelines to move data from various sources such as databases, APIs, streaming sources, and external data providers into data warehouses, data lakes, or other data repositories.
- Create efficient and scalable data pipelines capable of handling large volumes of both structured and unstructured data.
- Write code and scripts to automate repetitive tasks in data collection, processing, and delivery.
- Continuously monitor pipeline performance, identify and resolve data-related issues, errors, and performance bottlenecks.
- Design and maintain data architecture by selecting appropriate technologies for data storage, such as relational databases, NoSQL databases, data lakes, data warehouses, and cloud storage services, and designing efficient data models and schemas.
- Ensure data is stored in a way that allows for high-performance queries and efficient analytical and operational use cases.
- Research and implement new data technologies, tools, and frameworks to improve data infrastructure and processes.
- Proficiency in Azure cloud platform to deploy and manage data solutions in the cloud.
- Implement data quality and integrity measures by applying validation rules, cleansing procedures, and monitoring systems to detect and rectify anomalies.
- Implement data governance frameworks by defining standards and policies for data usage to ensure consistency, reliability, and compliance with regulations such as GDPR and HIPAA.
- Manage data security and access by implementing security controls and access management policies to protect sensitive information from unauthorized access or theft.
- Set up data access tools such as dashboards, analytics tools, and API endpoints to make processed data accessible to end-users and applications.
- Create comprehensive documentation for data pipelines, architecture, and processes to facilitate system transparency.
- Support data-driven decision making by providing clean, reliable, and accessible data.
- Proficiency in programming languages such as Python, Java, and SQL.
- Strong knowledge of relational database systems such as PostgreSQL, MySQL, and SQL Server, and NoSQL databases such as MongoDB and Cassandra.
- Experience with data warehousing platforms such as Snowflake.
- Proficiency with ETL tools such as Snowflake and Databricks.
- Experience with cloud platforms, specifically the Azure Suite.
- Experience with big data technologies including Hadoop, Spark, Kafka, and Hive.
- Ability to design efficient data models and schemas.
- Experience with scripting and automation of data processes.
Benefits & Perks
Compensation/salary range not specified
Work schedule not specified
Work environment perks not specified
Opportunities to work on meaningful climate impact projects
Collaborative and innovative work environment
Support for professional development in data engineering and cloud technologies
Inclusive workplace committed to equal employment opportunities
Ready to Apply?
Join Anew Climate and make an impact in renewable energy
Stay Updated on Sustainability Jobs
Get the latest renewable energy jobs and career tips delivered to your inbox.
Job Alerts
Get notified about new sustainability jobs
More jobs at Anew Climate
Ideas IP Intern
Anew Climate
Houston
Full Time
Dec 29
Environmental Products - Origination Analyst
Anew Climate
Not specified
Full Time
Dec 19
IT Software Engineer
Anew Climate
Budapest
Full Time
Oct 24