Data Engineer Web Scraping Data Ingestion

AlertMedia

Mexico City, Mexico

Full Time

Posted May 5, 2026

Not Specified

~18 people viewed this recently

Apply Now

Application opens on company website

Job Description

The Data Engineer will design, build, and maintain large-scale social media web scraping and data ingestion pipelines to support real-time threat detection and risk intelligence, collaborating with cross-functional teams to ensure data quality, reliability, and scalability.

Key Responsibilities

Design, build, and maintain scalable social media web scraping and data ingestion pipelines.
Optimize existing data pipelines for performance, reliability, and cost efficiency.
Handle challenges such as anti-bot protections, rate limiting, schema changes, and data quality issues.
Build and maintain ingestion pipelines that feed downstream systems like Elasticsearch and other data stores.
Collaborate with cross-functional teams to ensure data meets business and customer needs.

Requirements

Demonstrated experience in data engineering or backend engineering roles for 3-4 years.
Hands-on experience with social media web scraping in production, including large-scale or high-reliability systems.
Strong proficiency in Python for data ingestion, scraping, and pipeline development.
Experience with cloud infrastructure, specifically AWS, or other cloud platforms.
Solid understanding of databases and data storage systems such as Postgres, Redis, or similar.
Experience building and operating ETL pipelines, including data validation and transformation.
Experience with stream processing and/or event-driven architectures such as Kafka, Kinesis, SQS, etc.
Ability to build and maintain scalable ETL and data processing pipelines, both batch and streaming.
Ability to handle challenges such as anti-bot protections, rate limiting, schema changes, and data quality issues.
Experience with Elasticsearch or similar search data indexing systems (nice to have).
Excellent organizational skills and ability to manage multiple pipelines and priorities.
Excellent written and verbal communication skills, with the ability to collaborate cross-functionally.
English proficiency in written and oral communication.
Bachelor’s degree in Computer Science, Engineering, or a related technical field (preferred).