Member of Technical Staff - Distributed Systems

Sail builds the world's most efficient software for inference (processing LLM tokens) and agent hosting (cloud VMs). Together, our technologies allow our customers to deploy AI agents at extreme scale for their most ambitious work.

Build the systems that make AI inference fast, reliable, and cost-efficient at global scale. You’ll design the control plane that schedules and autoscales our enormous queue of tokens over a diverse fleet of machines, spread all over the world.

What you’ll do

• Design and implement high-performance schedulers (admission control, queuing, priority, fairness, preemption, bin packing).

• Build global routing and traffic management (latency-aware dispatch, predictive autoscaling, failover strategies, cache-aware routing).

• LLM-specific routing optimizations, e.g. KV caching that lets us trade memory for compute, across the cascading layers of GPU RAM, CPU RAM, and NVMe flash.

• Build deep observability: we want to trace every millisecond of our systems, and catch failures early enough that we can make things right before customers even notice.

What we’re looking for

• Strong distributed systems fundamentals (concurrency, networking, databases, queues, performance engineering).

• Eagerness to work with agents coupled with a healthy dose of skepticism. We start by focusing our attention on the highest-level design decisions first, and try to debate and justify each of decisions. Then, we use our finite review time on the critical pieces of the system that we must build good mental models of to do good work.

• Bonus: experience with ML inference stacks (vLLM/SGLang), GPUs/accelerators, high-RPS systems (e.g. trading, messaging)

Interview process

1. Meet the CTO, who will ask about your experience, and share as much technical detail about Sail as you want to hear. This is the first step because we respect your time.

2. Share an online whiteboard with a team member and work through a technical problem. We spend a lot of time at whiteboards, building intuition about complex systems together. It's a great way for us to see how you communicate technically, and a even better way for you to see what working at Sail is like.

3. Come in to Sail's SF office for an interview day. Meet the whole team, and work on a bunch of problems that closely simulates the work we do daily. We'll also ask you to give us a 20-30min 'chalk talk' about an interesting problem you've worked on before.

4. Offer. Once the team decides we want to work with you, we make a strong offer quickly and will be quite persistent over email/text/calls :)

Life at Sail

We work out of a beautiful, sunny office in downtown San Francisco. All meals are on us (and actually great; SF is a food paradise and it would be a shame to eat only bowl slop). Everyone gets a Studio Display at their desk. We are serious about investing in anything that saves us time or energy. There are six different ways to make coffee or tea in the office. A friendly (hypoallergenic) black cat named Coco visits occasionally.