Lead AI/ML Engineer @ ASAPP — Vaga New York

What you'll do

Build real-time conversational AI systems, including voice interfaces powered by speech-to-text, text-to-speech, and streaming inference pipelines
Design and optimize low-latency inference workflows for multimodal applications involving text, speech, and real-time interactions
Integrate and apply foundation models from major providers (OpenAI, AWS Bedrock, Anthropic, etc.) for prototyping and production use cases
Adapt, evaluate, and optimize LLMs for domain-specific enterprise applications
Build and maintain infrastructure for experimentation, deployment, and monitoring of AI models in production
Improve model performance and inference workflows with attention to latency, cost, and reliability
Provide technical leadership within the team, mentoring engineers and promoting best practices in ML engineering
Partner with product and cross-functional stakeholders to translate requirements into scalable ML solutions
Contribute to the evolution of internal standards for experimentation, evaluation, and deployment

6+ years of experience in Machine Learning or AI systems, with hands-on experience in LLMs, speech, or conversational AI systems
Experience building on integrating speech-to-text and text-to-speech systems
Strong experience integrating voice models into production applications
Proficiency on Python and ML frameworks like PyTorch or TensorFlow
Proven experience leading complex, cross-functional AI initiatives
Deep understanding of latency-sensitive system design and distributed architectures
Strong proficiency in Python and ML frameworks such as PyTorch or TensorFlow
Understanding of RAG pipelines, prompt engineering, and vector search
Experience deploying and scaling AI systems using AWS (required), Docker, Kubernetes, and CI/CD practices
Strong communication skills with the ability to align engineering, product, and executive stakeholders
Comfortable operating in fast-paced environments and driving clarity in ambiguous problem spaces

Experience with speech model fine-tuning and acoustic/language model optimization
Experience with production applications of S2S models
Hands-on experience with real-time or streaming audio systems (WebRTC, gRPC streaming, or similar architectures)
Experience optimizing TTS prosody, pronunciation control, and voice customization
Background in MLOps, experimentation platforms, or evaluation frameworks for speech and conversational systems
Contributions to open-source AI or speech tooling
Graduate degree (MS or PhD) in Computer Science, Machine Learning, Speech Processing, or related field