Inference Performance Engineer, Agent Driven Inference Optimization
Company websitenvidia.com (NVIDIA, opens in new tab)
Base salary $124,000 – $241,500
Employer-published annual base salary, USD. Advertised base range in Santa Clara, CA +1 more
You apply on the employer's own site. Six Figure Work does not handle applications.
- LocationSanta Clara, CA +1 more
- WorkplaceWorkplace not stated
- Status at last checkSource accepting applications when checked
- Last checked (1 h ago) · observed since
All places named
- Santa Clara, CA
- California
About this job
NVIDIA is recruiting a Senior Inference Performance Engineer to push NVIDIA's performance limits on large-scale AI inference benchmarks. This position provides an outstanding opportunity to employ your optimization knowledge in an autonomous optimization framework. AI agents use this framework to repeatedly run benchmark, profile, and tune processes, amplifying the impact of every technique you design. If you enjoy extracting maximum performance from GPUs and scaling your skills beyond your individual efforts, this role is a great fit!
What you'll be doing:
Distill your performance instincts into reusable skills, workflows, and evidence-backed methodologies that AI agents can complete autonomously. Review agent-generated experiments, validate findings, and curate best-known configurations.
Performance improvement of AI inference workloads that methodically increase throughput-per-GPU and user interactivity by exploring configuration options, parallelism techniques, batching, KV cache handling, quantization, and speculative decoding settings.
Measure and optimize both aggregated and disaggregated serving architectures across TensorRT-LLM, SGLang, vLLM, and Dynamo on NVIDIA's latest GPU platforms.
Profile workloads using Nsight Systems, kernel traces, and internal analysis tools. Use roofline and speed-of-light analysis to find credible headroom and drive fixes from hypothesis to measured wins.
Land improvements upstream: serving framework patches, optimized kernels, and deployment recipes that advance the public Pareto frontier while maintaining strict model correctness.
Collaborate with TensorRT-LLM, SGLang, vLLM, kernel, benchmarking, and GPU architecture teams to convert profiling insights into delivered performance improvements.
What we need to see:
BS, MS, or PhD in Computer Science, Computer Engineering, Electrical Engineering, Applied Math, or a related field, or equivalent experience.
3+ years of relevant engineering experience.
Must have: Extensive knowledge of the efficiency and optimization involved in AI model execution, covering continuous batching, throughput-latency tradeoffs, KV cache and memory limitations, parallel processing techniques, MoE serving, quantization, and meeting serving SLAs.
Must have: Hands-on experience benchmarking and profiling GPU workloads using tools such as Nsight Systems, Nsight Compute, CUPTI, or PyTorch profiler, and interpreting kernel-level performance data.
Strong Python engineering skills and the ability to navigate and modify large C++/CUDA serving codebases.
Rigorous experimental methodology with controlled single-variable comparisons, reproducible benchmarks, and evidence-backed optimization decisions.
Strong written and verbal communication skills to explain performance tradeoffs clearly to both humans and documentation for autonomous systems.
Ways to stand out from the crowd:
Direct contributions to TensorRT-LLM, vLLM, SGLang, FlashInfer, Dynamo, or comparable inference frameworks.
Experience with disaggregated serving, wide expert-parallel MoE inference, KV cache transfer, or NCCL/NIXL/NVSHMEM communication at multi-node scale.
CUDA kernel authorship or optimization experience on Hopper/Blackwell architectures, focusing on Tensor Cores, TMA, and warp specialization.
Proven results on public inference benchmarks such as MLPerf Inference or SemiAnalysis InferenceX.
Experience building or operating agentic AI workflows to automate engineering tasks.
Widely considered to be one of the technology world’s most desirable employers, NVIDIA offers highly competitive salaries and a comprehensive benefits package. As you plan your future, see what we can offer to you and your family www.nvidiabenefits.com/
#LI-Hybrid
Your base salary will be determined based on your location, experience, and other objective job-related factors. The expected base salary ranges for this position are:124,000 USD – 195,500 USD for Level 2; and 152,000 USD – 241,500 USD for Level 3.
You will also be eligible for equity and benefits.
Applications for this job will be accepted at least until November 13, 2026.This posting is for an existing vacancy.
NVIDIA uses AI tools in its recruiting processes.
NVIDIA is committed to fostering an inclusive work environment and proud to be an equal opportunity employer. As we highly value diversity in our current and future employees, we do not discriminate (including in our hiring and promotion practices) on the basis of race, religion, color, national origin, gender, gender expression, sexual orientation, age, marital status, veteran status, disability status or any other characteristic protected by law.Source: NVIDIA on Workday · View original posting (opens in new tab)
How this pay compares
Base floor $124,000. The employer's range is $117,500 wide, 95% of the floor. Observed open for 11 days so far (since Nov 10, 2026).
How NVIDIA pays
1,090 of NVIDIA's 1,178 live roles on this site list a higher base floor and 73 list a lower one. Across them the median floor is $184,000; the middle half runs from $152,000 to $200,000.
Base floors of NVIDIA's 1,178 live roles
Table
| Floor band | Share | Roles |
|---|---|---|
| $100K to $125K | 7.5% | 88 |
| $125K to $150K | 13.6% | 160 |
| $150K to $175K | 26.9% | 317 |
| $175K to $200K | 26.1% | 308 |
| $200K to $225K | 13.8% | 162 |
| $225K to $250K | 3.8% | 45 |
| $250K to $275K | 6.0% | 71 |
| $275K and up | 2.3% | 27 |
How this is calculated
- Statistic
- Employer base floor distribution
- Formula
- count of the employer's open roles in each $25K band of published base floor / its open roles
- Denominator
- 1,178 open public roles at this employer
- Sample size
- 1178
- Hidden when
- this employer has fewer than 5 roles
- Calculation version
- jobpage-analysis-v1
- Data as of
- current
- Employer set
- Open jobs on this site that publish a base salary of $100,000 or more
This listingInference Performance Engineer, Agent Driven Inference Optimization$124,000 – $241,500
System Software Engineer, Performance - CUDA Driver$124,000 – $195,500
AI Compute Engineer - NVIS$124,000 – $195,500
Deep Learning Framework Communications Engineer - New College Grad 2026$124,000 – $241,500
Senior Revenue and Pricing Business Analyst$124K – $230K
Power Architect - New College Grad 2027$124,000 – $241,500
Solutions Architect, Executive Briefing Center$124,000 – $241,500
Chart as a table
| Role | Published base |
|---|---|
| Inference Performance Engineer, Agent Driven Inference Optimization (this listing) | $124,000 - $241,500 |
| System Software Engineer, Performance - CUDA Driver | $124,000 - $195,500 |
| AI Compute Engineer - NVIS | $124,000 - $195,500 |
| Deep Learning Framework Communications Engineer - New College Grad 2026 | $124,000 - $241,500 |
| Senior Revenue and Pricing Business Analyst | $124K - $230K |
| Power Architect - New College Grad 2027 | $124,000 - $241,500 |
| Solutions Architect, Executive Briefing Center | $124,000 - $241,500 |
Observed changes
No changes to the pay or listing since we first observed this role on Nov 10, 2026 (11 days ago).
- First observed on the employer's board, already publishing a qualifying base salary
- Observed since
- Employer-stated posting date
- 2026-11-09
Where this pay comes from
Location as the employer wrote it: US, CA, Santa Clara • US, CA, Remote (Levels 2-3 combined)
124,000 USD – 195,500 USD for Level 2; and 152,000 USD – 241,500 USD for Level 3.
- Method
- Employer text, read with fixed rules and re-verified
- Checked
- Rule set
- us-base-v2
The pay passage appears in the employer's own job text. Fixed, deterministic rules located it, and the passage below was checked against the stored copy of the posting.
What we verified
- The pay figures come from the employer's own listing, quoted exactly above.
- The pay is an annual base salary in US dollars with a minimum of at least $100,000.
- The role was publicly listed on the employer's job board when last checked.
We do not verify what the employer will actually offer, whether the employer is still recruiting for the role, or anything beyond what the employer published. Full methodology.
