Software Engineer - AI Tooling Infrastructure

Accountabilities:

• Build, deploy, operate, and scale production backend services supporting AI-powered developer tools, including MCP servers, agent skills, and AI assistants.
• Own service architecture and infrastructure decisions, improving system reliability, maintainability, scalability, and deployment processes.
• Define and monitor service reliability objectives (SLOs), track error budgets, and prioritize improvements in collaboration with Site Reliability Engineering (SRE) teams.
• Participate in the team's on-call rotation, investigate production incidents, coordinate resolutions, and develop incident reports, runbooks, and preventive solutions.
• Manage operational support requests and identify recurring issues that can be addressed through long-term engineering improvements.
• Design and maintain authentication and authorization systems for AI agents and machine clients, including OAuth applications, scoped tokens, permissions, and secure access management.
• Establish and maintain comprehensive observability through metrics, distributed tracing, logging, dashboards, and alerting systems.
• Manage AI agent traffic, implementing rate limits, capacity controls, cost safeguards, and mechanisms to handle dependency failures and unexpected traffic spikes.
• Collaborate with Platform, SRE, Authentication, Billing, and Edge Functions teams to improve shared infrastructure, release processes, security, and operational practices.
• Contribute to the development of AI tooling infrastructure, ensuring that reliability, security, and performance remain central to engineering decisions.

Requirements:

• At least 5 years of experience building, deploying, and operating production backend services, including participation in on-call rotations.
• Strong programming skills in TypeScript or Go, with practical working knowledge of the other language.
• Proven experience designing and maintaining APIs, including versioning, error handling, rate limiting, and backward compatibility.
• Strong understanding of OAuth, OpenID Connect (OIDC), JWT, and API authorization, with the ability to troubleshoot authentication and permission issues in production environments.
• Good knowledge of PostgreSQL and relational database systems.
• Experience with CI/CD pipelines, Docker, containerized deployments, and Infrastructure as Code (IaC).
• Ability to troubleshoot distributed systems using metrics, logs, and traces to identify performance bottlenecks and service failures.
• Strong written and verbal communication skills, with the ability to collaborate across teams and work effectively in globally distributed, asynchronous environments.
• Adaptability, ownership, and the ability to navigate ambiguity while delivering solutions in fast-paced engineering environments.
• Deep expertise in at least one of the following areas:
  • APIs and Authentication: Experience designing permission models, token scopes, and OAuth flows for third-party integrations or machine clients, including safely migrating authentication systems without disrupting existing users.
  • Production Operations: Hands-on experience managing production services, participating in on-call rotations, developing alerts and runbooks, implementing SLOs, and improving system reliability.
  • Gateways and Distributed Systems: Experience building or operating proxies, gateways, or distributed services, troubleshooting retries, partial failures, upstream outages, and making appropriate fail-open or fail-closed decisions under degraded conditions.

Nice to have:

• Experience with AI agent tooling, Model Context Protocol (MCP), or infrastructure serving AI agents and LLM clients.
• Experience extracting services from larger codebases while maintaining compatibility with existing integrations.
• Familiarity with observability platforms such as Grafana, OpenTelemetry, or Logflare, along with practical experience implementing SLOs.
• Contributions to open-source projects, particularly within the Node.js, Go, TypeScript, or API tooling ecosystems.