Job Description
The role involves managing and optimizing the company's data platform, including administering Databricks and AWS infrastructure, developing data pipelines, and collaborating with cross-functional teams to enable data-driven decision making and AI initiatives.
Key Responsibilities
- Administer, monitor, and troubleshoot Databricks workspaces and AWS infrastructure.
- Develop platform administration tools and capabilities for managing Databricks resources.
- Collaborate with data teams to optimize workflows and data pipelines.
- Oversee data engineering development ecosystem, including CI/CD pipelines and monitoring frameworks.
- Implement and manage IaaC automation for Databricks and AWS resources using Terraform and CloudFormation.
- Ensure security, high availability, disaster recovery, and compliance of data services.
- Engage with stakeholders to understand data needs and design scalable solutions.
- Lead rapid prototyping and proof-of-concept evaluations for performance, security, and scalability.
- Mentor junior team members and provide technical guidance and training.
Requirements
- Bachelor's or Master's degree in Computer Sciences, Software Engineering, Electrical Engineering, Computer Engineering, or related discipline
- 10 years experience in a Software Engineering, Platform Engineering, Data Engineering, DevOps DataOps or similar technical role, including at least 2 years experience in data infrastructure or platform focused role
- Experience building, delivering, and administering large-scale production-grade data platforms and services to data engineering, business analyst, and data-science teams
- Solid knowledge of Databricks features and administration, including Unity Catalog, cluster management, security, troubleshooting and root-cause analysis, and performance optimization
- Experience and familiarity with AWS services and Cloud Infrastructure provisioning, management, monitoring, and security, e.g., S3, IAM, RDS, Lambda, API-Gateway, VPC, EC2, ECS, EKS
- Strong knowledge of SQL and Python, and hands-on data-engineering experience in designing and developing data pipelines and ETL routines from a variety of sources such as SaaS corporate systems, APIs, RDBMS
- Skilled with change-data capture, incremental and batch loading techniques
- Experience troubleshooting underlying issues and translating them into technical solutions
- Excellent problem-solving, communication, and stakeholder management skills
- Experience delivering and managing Databricks and AWS Infrastructure for hundreds of users
- Experience as a technical lead
- Knowledge of DevOps tools and practices, including GitHub, GitLab, CI/CD systems, Terraform, monitoring and logging tools such as Datadog and CloudWatch
- Experience implementing robust data governance and security measures to ensure data integrity and compliance
Benefits & Perks
Annual Base Salary: $134,470 - $180,800 USD
RSU grant with no vesting cliff and ongoing refresh opportunities
Performance-based bonus variable pay
Equity for eligible roles
Flexible working model (remote, hybrid, or in-office options)
Comprehensive health and parental leave plans
Professional development stipend
Ready to Apply?
Join Samsara and make an impact in renewable energy
Stay Updated on Sustainability Jobs
Get the latest renewable energy jobs and career tips delivered to your inbox.
Job Alerts
Get notified about new sustainability jobs