Machine Learning Engineer, Ops
Description
About Cantina: Cantina Labs is a social AI company, developing a suite of advanced real-time models that push the boundaries of expression, personality, and realism. We bring characters to life, transforming how people tell stories, connect, and create. We build and power ecosystems. Cantina, our flagship social AI platform, is just the beginning. If you're excited about the potential AI has to shape human creativity and social interactions, join us in building the future! About the Role: We are looking for an ML Engineer to own the production inference stack for our text-to-speech (TTS) and speech recognition (ASR) models, end to end. Our research team delivers trained model checkpoints; you will productionize them as low-latency, real-time streaming services that run cost-efficiently on GPUs. The role covers the inference engines, the Kubernetes infrastructure they run on, and the performance benchmarking and accuracy validation that let us ship changes safely. You will bridge research and production, and own latency, throughput, cost and accuracy as we scale. What You’ll Do: Build and optimize streaming inference engines for TTS and ASR models, including multi-stage model pipelines. Optimize latency (time to first chunk, p50/p99) and throughput using continuous batching, CUDA Graphs, torch.compile and mixed precision. Maintain numerical parity with the reference implementation through automated accuracy regression tests, per stage and end to end. Deploy and opera --- Source: jobicy - Cantina Labs
