Software Engineer, Inference Systems
Company websiteriver.ai (River AI Inc, opens in new tab)
Base salary $200K – $420K
Employer-published annual base salary, USD. Advertised base range in Palo Alto, CA
You apply on the employer's own site. Six Figure Work does not handle applications.
- LocationPalo Alto, CA
- WorkplaceWorkplace not stated
- Status at last checkSource accepting applications when checked
- Last checked (1 h ago) · observed since
About this job
At River AI, our mission is to create personal AI owned and shaped by each individual. To achieve this, we are rewriting the entire stack from scratch: personal hardware for local inference, bespoke training infrastructure, next-generation UIs, and frontier deep learning research.
Who we are
We are scientists, engineers, and builders from the industry's top tech companies and AI labs. We bring a proven track record of scaling consumer systems for hundreds of millions of users and architecting the pre-training infrastructure behind today's frontier models.
About the Role
We are looking for exceptional inference systems engineers to build the engines that serve large models through the River API. Your goal is to deliver fast, reliable inference while making efficient use of GPU compute and memory.
You will take ownership of the serving runtime, from request scheduling and continuous batching to KV-cache management, distributed model execution, and checkpoint loading. Your work will support both customer-facing inference and the sampling workloads that power reinforcement learning.
Working closely with GPU kernel engineers, researchers, and infrastructure engineers, you will bring new models into production and improve their performance across realistic workloads. You will measure success through latency, throughput, reliability, and cost, with careful attention to numerical correctness and model behavior.
What You’ll Do
- Optimize inference for dense and mixture-of-experts models, including fine-tuned models and adapters.
- Improve batching, caching, and admission control to balance throughput, latency, memory use, and fairness.
- Accelerate multi-GPU execution, communication, and model loading while preserving model-version consistency.
- Improve RL sampling throughput while keeping samples and log probabilities tied to the correct model version.
- Build reliable streaming, cancellation, and recovery under failures and overload.
- Profile bottlenecks and validate improvements through reproducible performance and correctness tests.
Skills & Qualifications
Minimum Qualifications:
- Bachelor’s degree in Computer Science, Computer Engineering, or equivalent practical experience.
- Experience building inference engines or performance-sensitive distributed services.
- Strong understanding of transformer inference, GPU memory, concurrency, and networking.
- Proficiency in Python and C++ or Rust.
- Strong debugging and profiling skills across models, runtimes, and services.
- A collaborative mindset and strong ownership of engineering outcomes.
Preferred Qualifications: (We encourage you to apply even if you don't meet all of these)
- Experience extending SGLang, vLLM, TensorRT-LLM, or similar frameworks.
- Work on advanced serving techniques, such as speculative decoding or disaggregated prefill and decode.
- Familiarity with expert parallelism, GPU collectives, and quantized inference.
- Experience with multi-adapter serving, dynamic checkpoint loading, or RL sampling.
- Familiarity with CUDA graphs, custom kernels, and NVIDIA profiling tools.
- Experience operating model-serving systems under production traffic.
Logistics & Benefits
- Location: Palo Alto, California.
- Compensation: $200,000–$420,000 USD annual base pay, depending on experience and skills, plus equity.
- Benefits: Comprehensive health, dental, and vision insurance; unlimited PTO; and relocation assistance as needed.
- Visa Sponsorship: We sponsor visas and support the process for the right candidate.
Source: River AI Inc on Greenhouse · View original posting (opens in new tab)
How this pay compares
Base floor $200,000, above 70% of the 4,901 Software Engineering roles in California on this site. The employer's range is $220,000 wide, 110% of the floor. Observed open for 1 day so far (since Nov 19, 2026).
Pay in context
Among 4,901 Software Engineering roles in California listed on this site, the median base floor is $176,360 and the middle half runs from $149,100 to $202,000. This role's floor ($200,000) is above 70% of them; 193 others list the same floor.
Base floors of 4,901 Software Engineering roles in California
Table
| Floor band | Share | Postings |
|---|---|---|
| $100K to $125K | 7.9% | 386 |
| $125K to $150K | 17.5% | 859 |
| $150K to $175K | 22.7% | 1,111 |
| $175K to $200K | 22.7% | 1,112 |
| $200K to $225K | 12.7% | 624 |
| $225K to $250K | 6.6% | 324 |
| $250K to $275K | 4.6% | 227 |
| $275K and up | 5.3% | 258 |
How this is calculated
- Statistic
- Base floor distribution of the cohort
- Formula
- count of cohort postings in each $25K band of published base floor / postings in the cohort
- Denominator
- 4,901 open postings with this job family in California
- Sample size
- 4901
- Hidden when
- the cohort has fewer than 30 postings
- Calculation version
- jobpage-analysis-v1
- Data as of
- 2026-11-20
- Employer set
- Open jobs on this site that publish a base salary of $100,000 or more
Only jobs on this site count, and each one publishes a base salary of $100,000 or more, so this compares six-figure jobs with each other, not with the whole market. Computed Nov 20, 2026.
State against nationwide
Median base floor in California vs nationwide
Table
| Value | postings | |
|---|---|---|
| Software Engineering, California | $176,360 | 4,901 |
| Software Engineering, all states | $160K | 13,273 |
| This job's base floor (marked) | $200K |
How this is calculated
- Statistic
- Median floor, state vs nationwide
- Formula
- median published base floor of each cohort
- Denominator
- open postings of this job family
- Sample size
- 4901 in California; 13273 nationwide
- Hidden when
- either cohort has fewer than 30 postings
- Calculation version
- jobpage-analysis-v1
- Data as of
- 2026-11-20
- Employer set
- Open jobs on this site that publish a base salary of $100,000 or more
How River AI Inc pays
1 of River AI Inc's 21 live roles on this site list a higher base floor and 1 list a lower one. Across them the median floor is $200,000; the middle half runs from $200,000 to $200,000.
Base floors of River AI Inc's 21 live roles
Table
| Floor band | Share | Roles |
|---|---|---|
| $100K to $125K | 0.0% | 0 |
| $125K to $150K | 0.0% | 0 |
| $150K to $175K | 4.8% | 1 |
| $175K to $200K | 0.0% | 0 |
| $200K to $225K | 90.5% | 19 |
| $225K to $250K | 0.0% | 0 |
| $250K to $275K | 0.0% | 0 |
| $275K and up | 4.8% | 1 |
How this is calculated
- Statistic
- Employer base floor distribution
- Formula
- count of the employer's open roles in each $25K band of published base floor / its open roles
- Denominator
- 21 open public roles at this employer
- Sample size
- 21
- Hidden when
- this employer has fewer than 5 roles
- Calculation version
- jobpage-analysis-v1
- Data as of
- current
- Employer set
- Open jobs on this site that publish a base salary of $100,000 or more
Similar pay elsewhere
Other employers' Software Engineering roles in California with the nearest published base floor to $200,000, one per employer.
- Software Engineer, Product Descript · base floor $200,000
- Engineering Manager, Platform Doctronic · base floor $200,000
- Senior Software Engineer, AI Tooling Airwallex · base floor $200,000
- Senior ML Infra Engineer General Legal · base floor $200,000
- Member of Technical Staff - Inference Sail · base floor $200,000
This listingSoftware Engineer, Inference Systems$200K – $420K
Product Engineer, Personal AI$200K – $420K
Software Engineer, GPU Kernels$200K – $420K
People Lead$200K – $380K
Kernel Engineer (Custom Silicon), Hardware$200K – $420K
Software Engineer, River API$200K – $420K
Performance Engineer, Hardware$200K – $420K
Chart as a table
| Role | Published base |
|---|---|
| Software Engineer, Inference Systems (this listing) | $200K - $420K |
| Product Engineer, Personal AI | $200K - $420K |
| Software Engineer, GPU Kernels | $200K - $420K |
| People Lead | $200K - $380K |
| Kernel Engineer (Custom Silicon), Hardware | $200K - $420K |
| Software Engineer, River API | $200K - $420K |
| Performance Engineer, Hardware | $200K - $420K |
Observed changes
No changes to the pay or listing since we first observed this role on Nov 19, 2026 (1 day ago).
- First observed on the employer's board, already publishing a qualifying base salary
- Observed since
- Employer-stated posting date
- 2026-09-12
Where this pay comes from
Location as the employer wrote it: Palo Alto, CA
- Compensation: $200,000–$420,000 USD annual base pay, depending on experience and skills, pl
- Method
- Employer text, read with fixed rules and re-verified
- Checked
- Rule set
- us-base-v2
The pay passage appears in the employer's own job text. Fixed, deterministic rules located it, and the passage below was checked against the stored copy of the posting.
What we verified
- The pay figures come from the employer's own listing, quoted exactly above.
- The pay is an annual base salary in US dollars with a minimum of at least $100,000.
- The role was publicly listed on the employer's job board when last checked.
We do not verify what the employer will actually offer, whether the employer is still recruiting for the role, or anything beyond what the employer published. Full methodology.
