The role involves designing, building, and maintaining large-scale data pipelines for social media web scraping and data ingestion to support real-time threat detection and risk intelligence at AlertMedia.
Key Responsibilities
Design, build, and maintain scalable social media web scraping and data ingestion pipelines.
Optimize existing data pipelines for performance, reliability, and cost efficiency.
Handle challenges such as anti-bot protections, rate limiting, schema changes, and data quality issues.
Build and maintain ingestion pipelines that feed downstream systems like Elasticsearch and other data stores.
Collaborate with cross-functional teams to ensure data meets business and customer needs.
Requirements
Bachelor's degree in Computer Science, Engineering, or a related technical field (preferred).
4 to 6 years of demonstrated experience in data engineering or backend engineering roles.
Expert-level, hands-on experience with social media web scraping in production, including large-scale or high-reliability systems.
Strong proficiency in Python for data ingestion, scraping, and pipeline development.
Experience building and operating ETL pipelines, including data validation and transformation.
Experience with stream processing and/or event-driven architectures such as Kafka, Kinesis, SQS, etc. (strong plus).
Experience working with Elasticsearch or similar search data indexing systems.
Solid understanding of databases and data storage systems such as Postgres, Redis, or similar.
Experience with AWS or other cloud platforms.
Ability and desire to work in a fast-paced, technically challenging environment.
Excellent written and verbal communication skills, with the ability to collaborate cross-functionally.
English proficiency in written and oral communication.
Physical ability to manage multiple pipelines and priorities effectively.
Benefits & Perks
Competitive Base Salary
Annual Company Bonus
100% Employer-Paid Medical, Dental, and Vision Coverage
Parental Leave Program
Christmas Bonus
Aguinaldo
Generous Vacation and Holiday Policy
Monthly Work-from-Home Stipend
All Necessary Equipment Provided (Laptop, monitor, accessories)
Recognition Rewards
Language Immersion Program
Continuous Learning and Growth Opportunities (mentorship, tech talks, career development sessions)
Award-Winning Culture
Ready to Apply?
Join AlertMedia and make an impact in renewable energy