A Senior Data Engineer responsible for designing, developing, and maintaining scalable data platforms and pipelines using cloud technologies like AWS, to support analytics, machine learning, and data governance initiatives within a growing energy solutions company.
Key Responsibilities
Design, develop, and maintain scalable data ingestion pipelines and cloud-native big data environments on AWS.
Lead data governance, data profiling, and ensure data quality and proper metadata documentation.
Collaborate with Data Scientists, BI developers, and Product Managers to design data models, schemas, and processing logic.
Develop and optimize ETL processes for data validation, transformation, and feature modeling using Spark, Python, SQL, and AWS technologies.
Implement best practices for code development, testing, performance optimization, capacity planning, and incident response.
Define SLAs for data availability and correctness, automate monitoring, and respond to data delivery alerts.
Requirements
A bachelor’s degree in computer science or information technology
Minimum of 8 years of relevant experience in data engineering or related fields
High proficiency in programming languages commonly used in ETL development, such as PLSQL, SQL, and Python
Ability to write efficient SQL queries, SQL stored procedures, develop scripts for data transformations, and utilize programming frameworks and libraries to create enhanced ETL mappings and workflows
Expertise in utilizing AWS services, including but not limited to Amazon S3, AWS Glue, AWS Data Catalog, Amazon Redshift, Redshift Spectrum, and Amazon Athena, to build scalable, reliable, and performant data pipelines and analytics solutions
Ability to build, automate, and manage near-real-time scalable data ingestion pipelines for master data management, deep-learning, and predictive analytics
Experience in building and maintaining cloud-native big data environments on AWS that are highly secure, scalable, flexible, and highly performant using appropriate SQL, NoSQL, and NewSQL technologies
Proficiency in working with relational databases such as Postgres, Oracle, MySQL, or SQL Server, including knowledge of database design, optimization techniques, and advanced querying capabilities
Experience in performance tuning and optimizing database operations
Experience leading data governance and data profiling efforts to ensure data quality and proper metadata documentation for data lineage
Ability to lead by example, demonstrating best practices for code development and optimization, unit testing, CI/CD, performance testing, capacity planning, documentation, monitoring, alerting, and incident response to ensure data availability, data quality, and usability
Ability to define SLAs for data availability and correctness, automate data availability and quality monitoring, and respond to alerts when data delivery SLAs are not being met
Excellent communication skills to clearly communicate progress across organizations and levels from individual contributor to executive
Ability to identify and clarify critical issues that need action and drive appropriate decisions and actions
Benefits & Perks
Salary range of $140,000 - $165,000 annually, with a target of $156,750 based on experience and qualifications
Hybrid work opportunity with flexible in-office attendance requirements
Generous retirement package
Medical, dental, and vision insurance
Pre-tax contribution plans
Employee Stock Ownership Plan (ESOP)
Ready to Apply?
Join Energy Solutions and make an impact in renewable energy