Member of Technical Staff, Machine Learning Infrastructure at Abridge in SF Office

Job description

About Abridge Founded in 2018, Abridge is the first AI-native clinician intelligence platform organized around the patient, built for clinicians, and designed to help health systems coordinate the clinical, financial, and evidence-based decisions for every moment of care. Powered by purpose-built AI for healthcare, Abridge connects patients, providers, payers, and life sciences companies. This year, the platform will support more than 100 million patient-clinician conversations across more than 300 of the largest and most complex health systems in the U.S. With deep workflow integration, the Abridge clinician intelligence platform has been validated to support encounters in 28+ languages across specialties and care settings. Abridge is setting the industry standard for the responsible deployment of Al across health systems with continual and transparent Al evaluation. Clinical decision support references and cites gold-standard sources of evidence, and features like Linked Evidence map AI-generated documentation to source data, helping clinicians quickly trust and verify the output. Abridge was awarded Best in KLAS for Ambient AI in 2025 and 2026, in addition to other accolades, including TIME Best Inventions of 2024 and 2025, CNBC Disruptor 2025, and Fast Company’s Most Innovative Companies 2025. We are a growing team of practicing clinicians, AI scientists, PhDs, creatives, technologists, and engineers working together to transform healthcare. We have offices located in San Francisco, New York, and Pittsburgh. The Role As an ML Infrastructure Engineer at Abridge, you’ll play a pivotal role in building and optimizing the core inference infrastructure that powers our machine learning models. Your work will be instrumental in enhancing the scalability, efficiency, and performance of our AI-driven solutions. You will work with our Infrastructure and Research teams to build, deploy, optimize and orchestrate across our AI models. What You'll Do Design, deploy and maintain scalable Kubernetes clusters for AI model inference and training Develop, optimize, and maintain ML model serving infrastructure, ensuring high-performance and low-latency. Collaborate with ML and product teams to scale backend infrastructure for AI-driven products, focusing on model deployment, throughput optimization, and compute efficiency. Optimize compute-heavy workflows and enhance GPU utilization for ML workloads. Build a robust model API orchestration system Collaborate with leadership to define and implement strategies for scaling infrastructure as the company grows, ensuring long-term efficiency and performance. What You’ll Bring 5+ years of experience in building and deploying machine learning models in production environments. Deep understanding of container orchestration and distributed systems architecture Expertise in Kubernetes administration, including custom resource definitions, operators, and cluster management Experience developing APIs and managing distributed systems for both batch and real-time workloads Excellent communication skills, with the ability to interface between research and product engineering Ideally, You Have Expertise with model serving frameworks such as NVIDIA Triton Server, VLLM, TRT-LLM and so on. Expertise with ML toolchains such as PyTorch, Tensorflow or distributed training and inference libraries. Familiarity with GPU cluster management and CUDA optimization Knowledge of infrastructure as code (Terraform, Ansible) and GitOps practices Experience with container registries, image optimization, and multi-stage builds for ML workloads Experience orchestrating across ASR models or LLM models for building various GenAI applications Why Work at Abridge? At Abridge, we’re transforming healthcare delivery experiences with generative AI, enabling clinicians and patients to connect in deeper, more meaningful ways. Our mission is clear: to power deeper understanding in healthcare. We’re driving real, lasting change, with million