Accountabilities
- Design and build the control plane for the curated data lifecycle, including dependency-aware orchestration, backfills, restatements, retries, partial-failure handling, and recovery mechanisms.
- Own architectural decisions across datasets, determining when a problem is best addressed through a data model, service, or job.
- Design and maintain clear contracts between data ingestion and curation layers so datasets can be understood and managed end to end.
- Build reliable alerting, monitoring, and data-quality signals that identify genuine issues while minimising operational noise.
- Work across Go, Kotlin, Rust, Python, and SQL, selecting technologies based on the requirements of each problem.
- Develop software that orchestrates and manages thousands of interdependent data models and supports large-scale data operations.
- Design systems capable of propagating schema changes and data corrections without unnecessarily disrupting downstream consumers.
- Translate ambiguous product requirements into technical designs, execution sequences, and actionable work for the broader engineering team.
- Personally implement the most technically challenging components while enabling other engineers to own and deliver complementary work.
- Debug complex production issues independently and lead root-cause analysis through to durable fixes.
- Contribute to technical architecture, engineering practices, and system design decisions across the curated data domain.
- Collaborate effectively with a distributed engineering organisation through clear written communication and a strong async-working approach.
- Use AI-assisted development tools thoughtfully to increase engineering productivity while validating outputs and maintaining high standards for correctness and quality.
Requirements
- Significant experience as a backend software engineer with deep expertise in data systems, or as a data engineer who has developed strong software engineering capabilities.
- Proven track record of building and operating production-grade services, not solely data pipelines.
- Experience designing, building, or materially extending orchestration and scheduling systems, with a strong understanding of their failure modes and scaling challenges.
- Demonstrated experience with schema evolution and data correctness in systems where downstream consumers depend on reliable and stable data.
- Production experience with stateful stream-processing technologies such as Flink, Kafka Streams, Spark Structured Streaming, RisingWave, Materialize, or Feldera.
- Strong SQL and data-modelling capabilities, particularly when working with large datasets.
- Solid computer science fundamentals and a strong understanding of distributed systems.
- Experience reasoning about query execution and the underlying behaviour of data-processing engines is highly valuable.
- Strong debugging and root-cause-analysis skills, with a track record of driving issues through to durable production fixes.
- Strong architectural and systems-thinking abilities, with the confidence to make decisions across complex technical domains.
- Ability to take ambiguous requirements, develop a clear design and implementation sequence, and turn large technical problems into manageable workstreams.
- Strong written communication skills and the ability to collaborate effectively within distributed engineering teams.
- Practical experience using AI development tools and an understanding of their limitations, failure modes, and appropriate use in production engineering workflows.
- Experience with transformation frameworks such as dbt or SQLMesh is a plus, particularly experience extending them after encountering their limitations.
- Familiarity with data-lake formats such as Parquet, Iceberg, or Delta Lake is advantageous.
- Experience working in an organisation where data itself is a core product is a plus.
Benefits
- Competitive salary and equity package, positioned within the top 25% of companies in the sector.
- Employee equity plan with employee-friendly terms, including a heavily discounted strike price and a 10-year exercise window.
- 5 weeks of paid time off plus local public holidays, with flexibility to swap holidays where appropriate.
- Fully remote-first working environment within a distributed international team.
- Flexible working hours, allowing you to structure your working day around your productivity and priorities.
- Strong focus on asynchronous collaboration and a deliberate approach to reducing unnecessary meetings.
- Private medical, dental, and vision insurance.
- 16 weeks of fully paid parental leave for primary caregivers and 6 weeks for secondary caregivers.
- Two-week phased return to work at full pay following parental leave.
- Quarterly company or team offsites at international locations.
- Annual travel allowance for connecting and co-working with colleagues.
- Home-office setup allowance, with local coworking space costs covered if preferred.
- Opportunity to work alongside experienced engineers and specialists in a highly collaborative environment.
- Company merchandise and team perks.
🇧🇷 Essa vaga exige inglês. Você está pronto?
A DevSpeak Academy prepara desenvolvedores brasileiros para conquistar vagas internacionais. Domine o inglês técnico com professores que entendem o mundo dev.
Conheça a DevSpeak Academy