Machine Learning Engineering Manager - LLM Serving (Remote - US)

This position is posted by Jobgether on behalf of a partner company. We are currently looking for a Machine Learning Engineering Manager - LLM Serving & Infrastructure in the United States.

This role offers the opportunity to lead and shape the development of high-scale, low-latency LLM serving infrastructure for advanced personalization and recommendation systems. You will manage a talented engineering team while collaborating closely with machine learning researchers, data scientists, and product teams to deliver robust, scalable, and cost-efficient ML systems. The position focuses on deploying and maintaining LLMs across production environments, optimizing performance and reliability, and driving adoption of innovative AI-powered features. This is a highly visible role where your technical leadership and strategic vision will directly impact millions of users, improving their experience through intelligent recommendations and interactions.

Accountabilities:

• Lead a high-performing engineering team to design, build, and deploy large-scale LLM serving infrastructure.
• Drive the implementation of a unified serving layer for multiple LLM models, supporting batch, offline evaluation, and real-time inference.
• Oversee the development and management of the Model Registry, including deployment, versioning, and monitoring of LLMs across production environments.
• Collaborate with internal product and personalization teams to integrate LLM-powered features seamlessly into recommendation systems.
• Establish standardized technical interfaces, protocols, and best practices for efficient model deployment, scaling, and operational monitoring.
• Optimize serving infrastructure for latency, cost, and reliability, leveraging techniques like quantization, pruning, and efficient batching.
• Develop observability and monitoring frameworks, including dashboards, alerting, and SLA tracking for inference traffic and system health.

• 5+ years of software or machine learning engineering experience, including 2+ years managing engineering teams.
• Hands-on experience in ML engineering, building and scaling production-quality ML systems and datasets.
• Strong technical expertise in LLM infrastructure, recommendation or personalization systems, and high-volume, high-velocity ML platforms.
• Proven ability to lead complex, multi-partner projects with federated contribution models.
• Experience designing loosely coupled, scalable systems with clear separation of concerns and robust APIs.
• Familiarity with CI/CD pipelines, experiment tracking, and results visualization tools (e.g., MLFlow).
• Excellent leadership, communication, and collaboration skills, with the ability to influence peers and stakeholders.
• Pragmatic, results-oriented mindset, balancing speed and production rigor.