What You'll Do
- Own the Airflow codebase end-to-end (MWAA, ~100 DAGs currently): build reusable templates, scale patterns, enforce standards, improve testing infrastructure
- Serve as delivery teams’ go-to on pipeline architecture and troubleshooting
- Own data onboarding when we connect to a new source including connection setup, schema discovery, initial pipeline design
- Operate and extend large-scale Spark pipelines on AWS Glue, including multi-TB joins and compaction jobs, and support migration of those workloads to Databricks as we move platforms
- Drive data model improvements around our commercial pharma data (patient claims, KOL and HCP data, CRM activity) with a focus on structure, lineage, and how it flows through the platform
- Contribute to and maintain our internal Python package used across the data team
Requirements
- 5+ years building data models and pipelines
- 2+ years production Airflow experience
- 2+ years Spark at scale: hands-on experience with multi-TB datasets; AWS Glue experience preferred
- Strong python and SQL
- AWS fluency in S3, ECS/Fargate, Glue, IAM, Secrets Manager
- Clear communicator who can teach: onboarding analytics team to the codebase and developing team Airflow capability is a core part of this job, not a side responsibility
Nice to Have
- Experience working with privacy-sensitive or governed data
- Databricks experience – we’re actively migrating Glue workloads there
- Proficiency with Claude Code
- Exposure to data science workflows and ML pipeline tooling
- Background in life sciences or healthcare
- Experience with Open Meta data or similar data catalog and data dictionary tooling
Perks & Benefits
- Medical, dental, vision, and basic life insurance
- Flexible PTO and company paid holidays
- Retirement programs
- 1% charitable giving program
Compensation
- Base pay: $115,000 - $175,000
- The salary range listed here has been provided to comply with local regulations and represents a potential base salary range for this role. Please note that actual salaries may vary within the range above or below, depending on experience and location. We look at compensation for each individual and base our offer on your unique qualifications, experience, and expected contributions. This position may also be eligible for other types of compensation in addition to base salary, such as variable bonus and/or stock bonus.
🇧🇷 Essa vaga exige inglês. Você está pronto?
A DevSpeak Academy prepara desenvolvedores brasileiros para conquistar vagas internacionais. Domine o inglês técnico com professores que entendem o mundo dev.
Conheça a DevSpeak AcademyCandidaturas encerradasVer outras vagas
