Apply Now
Application opens on company website
Job Description
The Data Engineer is responsible for designing, building, and maintaining scalable data infrastructure, including ETL/ELT pipelines, data storage solutions, and automation processes, to support data-driven decision making and analytics within the organization.
Key Responsibilities
- Design and implement ETL/ELT data pipelines from various sources into data repositories
- Create and maintain scalable data pipelines for large volumes of data
- Automate data workflows through scripting and coding
- Monitor, troubleshoot, and optimize data pipelines for performance and reliability
- Design, select, and maintain data architecture, storage solutions, and data models
- Evaluate and implement new data technologies and tools
- Manage cloud-based data solutions using Azure
- Ensure data quality, integrity, and security through validation, governance, and access controls
- Provide data access tools such as dashboards and APIs for end-users
- Document technical designs, workflows, and processes to ensure system transparency
Requirements
- Design and implement robust Extract, Transform, Load (ETL) or Extract, Load, Transform (ELT) pipelines to move data from various sources such as databases, APIs, streaming sources, and external data providers into data warehouses, data lakes, or other data repositories.
- Create efficient and scalable data pipelines capable of handling large volumes of structured and unstructured data.
- Write code and scripts to automate repetitive tasks in data collection, processing, and delivery.
- Continuously monitor pipeline performance, identify and resolve data-related issues, errors, and performance bottlenecks.
- Select appropriate technologies for data storage, including relational databases, NoSQL databases, data lakes, data warehouses, and cloud storage services, and design efficient data models and schemas.
- Ensure data is stored in a way that allows for high-performance queries and efficient analytical and operational use cases.
- Research and implement new data technologies, tools, and frameworks to improve data infrastructure and processes.
- Proficiency in deploying and managing data solutions in the cloud using Azure platform.
- Implement data quality and integrity measures, including validation rules, cleansing procedures, and monitoring systems to detect and rectify data anomalies.
- Define standards and policies for data usage, ensuring data governance, compliance with regulations such as GDPR and HIPAA.
- Implement security controls and access management policies to protect sensitive information from unauthorized access or theft.
- Set up dashboards, analytics tools, and API endpoints to make processed data accessible to end-users and applications.
- Create comprehensive documentation for data pipelines, architecture, and processes to facilitate system transparency.
- Provide data-driven support to enable organizations to make informed business decisions by delivering clean, reliable, and accessible data.
- Strong knowledge of programming languages such as Python, Java, and SQL.
- Experience with relational databases (e.g., PostgreSQL, MySQL, SQL Server) and NoSQL databases (e.g., MongoDB, Cassandra).
- Experience with data warehousing platforms such as Snowflake.
- Proficiency with ETL tools such as Snowflake and Databricks.
- Experience with cloud platforms, specifically Azure Suite.
- Knowledge of Big Data technologies including Hadoop, Spark, Kafka, and Hive.
- Ability to design efficient data models and schemas.
- Skills in scripting and automation for data processes.
Benefits & Perks
Compensation/salary range (not specified)
Work schedule (not specified)
Work environment perks (not specified)
Equal employment opportunity and non-discrimination policies
Ready to Apply?
Join Anew Climate and make an impact in renewable energy
Stay Updated on Sustainability Jobs
Get the latest renewable energy jobs and career tips delivered to your inbox.
Job Alerts
Get notified about new sustainability jobs
More jobs at Anew Climate
Ideas IP Intern
Anew Climate
Houston
Full Time
Dec 29
Environmental Products - Origination Analyst
Anew Climate
Not specified
Full Time
Dec 19
IT Software Engineer
Anew Climate
Budapest
Full Time
Oct 24
More jobs in Houston, Texas
Energy Sales Director
5
Houston
Full Time
Dec 29
Senior Software Engineer
5
Houston
Full Time
Dec 29
Energy Advisor- Commercial Sales
5
Remote
Full Time
Oct 31