ML Infrastructure Engineer
About the Role
This is a hands-on infrastructure engineering role at an early-stage enterprise AI company building a context layer that makes AI agents reliable, accurate, and secure for mission-critical business operations. You'll own the systems that keep those agents running fast and reliably in production — from design through deployment — working closely with ML and infrastructure teams to scale inference at increasing concurrency.
What You'll Do
Own inference and model-serving infrastructure end to end, from architecture design through production deployment.
Build and scale systems that enable AI agents to run reliably and efficiently under high concurrency in production environments.
Collaborate with ML and infrastructure teams to ensure seamless integration and drive performance optimization.
Identify infrastructure bottlenecks and lead the engineering effort to resolve them.
What We're Looking For
5+ years of experience building and operating machine learning inference systems, model-serving platforms, or ML infrastructure in production environments.
Hands-on experience designing and scaling inference-serving infrastructure using tools such as TensorFlow Serving, TorchServe, Triton, KServe, or equivalent custom systems.
Demonstrated ability to optimize production ML systems for latency, throughput, and reliability at scale.
Strong proficiency with containerization and orchestration technologies — Docker and Kubernetes — for deploying ML workloads.
Experience building or maintaining distributed systems that handle concurrent requests and manage resource allocation under load.
Solid command of monitoring, observability, and debugging tooling for production systems (e.g., Prometheus, Grafana, ELK, distributed tracing).
Experience deploying and managing ML systems on cloud platforms such as AWS, GCP, or Azure.
Proficiency in at least one systems or backend language: Python, Go, Rust, C++, or Java.
Experience with knowledge graphs, semantic search, or graph databases (e.g., Neo4j, Amazon Neptune) is a plus.
Familiarity with real-time or low-latency inference systems, agentic AI pipelines, or enterprise data infrastructure is a plus.
Location
On-site in San Mateo, California, United States. Visa sponsorship is not available for this role.
