Senior Machine Learning Engineer, Inference

Inworld — realtime AI research lab and inference provider · ~50 employees · $125M+ raised

Remote — US, Canada, UK, Germany, Switzerland or SerbiaFull-time · Remote$250K – $300K USD base (adjusted by location)

About the role

Inworld is a research lab and inference provider focused on realtime AI for consumer-facing applications. It builds first-party speech models, serves LLMs and runs the inference behind modular APIs designed for high-volume, realtime workloads. Hundreds of millions of users interact with Inworld-powered apps every day, and it serves over 10 trillion LLM tokens a month. Founded in 2021 and based in Mountain View, it has raised more than $125M from investors including Lightspeed Venture Partners, Kleiner Perkins, Founders Fund and Microsoft's M12.

This role is on the inference side: serving models in production at sub-second latency and scale. Inworld says reliably working agentic systems and sub-second multimodal inference barely existed a year ago and nobody has a decade of experience in them, so they aren't screening for a resume template. They want strong people from varied backgrounds who learn fast, thrive in ambiguity and can show what they've built, broken and understood.

Salary is adjusted by location, with offers closer to the top of the range for the SF Bay Area and New York. This role is for senior candidates; staff-level candidates go to a separate Inworld role.

What you will own

Optimize inference with modern serving frameworks and techniques such as vLLM or TRT-LLM
Accelerate models with quantization, distillation, caching strategies, continuous batching, paged attention and speculative decoding
Write high-performance systems code (C++, CUDA, Rust or highly optimized Python) and profile it to get the most out of NVIDIA GPUs
Scale distributed inference with Kubernetes, Ray, custom load balancing and multi-GPU, multi-node serving that handles thousands of concurrent connections
Own the full cycle: take a model from the research team, containerize it, optimize its serving and keep it running reliably in production

Requirements

Must haveSenior-level experience (Google L5 or Meta E5 equivalent) as an ML inference or systems engineer, with 5+ years in the field

Must haveExperience in ML inference optimization (vLLM or TRT-LLM)

Must haveLocated in the United States, Canada, the United Kingdom, Germany, Switzerland or Serbia

Must haveExperience building backend or ML systems and serving machine learning models in production

Must haveProficiency in systems programming and GPU optimization (C++, CUDA, Rust or optimized Python)

Must haveExperience with model acceleration techniques (quantization, distillation, speculative decoding, caching, continuous batching or paged attention)

Must haveExperience in distributed systems and scaling (Kubernetes, Ray, custom load balancing, multi-GPU or multi-node inference, high concurrency)

Nice to haveA history of internal career progression, with multiple promotions within one company

Nice to havePublic technical work: systems programming projects, open-source contributions to inference engines or technical write-ups

Nice to haveA PhD in Computer Science, Physics or Math, or equivalent practical systems experience

Nice to haveAble to commute to the office in Mountain View, CA

Not a fitMore than two tenures of a year or less

Not a fitTen or more years at a single company

Not a fitBackgrounds centred on recommender systems, CPU services, agents, training or fine-tuning, or edge/AV TensorRT work

Not a fitContractors or consultants

We interview honestly against this list — if you meet most of the “Must have” rows, apply.

Compensation

Remote — US, Canada, UK, Germany, Switzerland or Serbia

Location

Full-time · Remote

Employment

$250K – $300K USD base (adjusted by location)

Compensation

Competitive equity

Equity

Role details

LocationRemote — US, Canada, UK, Germany, Switzerland or Serbia

EmploymentFull-time · Remote

Compensation$250K – $300K USD base (adjusted by location)

EquityCompetitive equity

DepartmentEngineering

VisaOpen to visa transfers (e.g. OPT, H-1B transfers); relocation assistance for candidates willing to move to the SF Bay Area

Vikki

Your recruiter for this role

Not sure you tick every box? Reach out before you apply — happy to tell you what actually matters here.

Message Vikki

Similar roles

Junior Software Engineer, Shopping

Remote — United States · $115K – $150K USD base

Staff Full-Stack Engineer, Core Application

Hybrid — San Francisco Bay Area (Oakland office) · $240K – $275K USD base

Senior Software Engineer

Remote — United States · $197K – $207K USD base (fixed pay bands, no negotiation)