Six Figure Work

Sr Principal Machine Learning Engineer (Advanced Threat Protection)

Palo Alto Networks

Company websitepaloaltonetworks.com (Palo Alto Networks, opens in new tab)

Base salary $190,400 – $308,000

Employer-published annual base salary, USD. Advertised base range in Santa Clara, CA

Apply (opens in new tab)

You apply on the employer's own site. Six Figure Work does not handle applications.

  • LocationSanta Clara, CA
  • WorkplaceWorkplace not stated
  • Status at last checkSource accepting applications when checked
  • Last checked (10 h ago) · observed since Last checked 10 h ago. We could not reach the employer's site, so this listing may have closed. Check the employer's page before you apply.

About this job

Our Mission

At Palo Alto Networks®, we’re united by a shared mission—to protect our digital way of life. We thrive at the intersection of innovation and impact, solving real-world problems with cutting-edge technology and bold thinking. Here, everyone has a voice, and every idea counts. If you’re ready to do the most meaningful work of your career alongside people who are just as passionate as you are, you’re in the right place.

Who We Are

In order to be the cybersecurity partner of choice, we must trailblaze the path and shape the future of our industry. This is something our employees work at each day and is defined by our values: Disruption, Collaboration, Execution, Integrity, and Inclusion. We weave AI into the fabric of everything we do and use it to augment the impact every individual can have. If you are passionate about solving real-world problems and ideating beside the best and the brightest, we invite you to join us!

We believe collaboration thrives in person. That’s why most of our teams work from the office full time, with flexibility when it’s needed. This model supports real-time problem-solving, stronger relationships, and the kind of precision that drives great outcomes.

Job Summary

Your Career

As a Senior Principal Machine Learning Engineer, you will serve as the chief technical authority driving architectural strategy, design excellence, and AI innovation across cloud security systems. In this role, you will define the multi-year engineering roadmap, set the technical direction for foundation model adoption, and lead high-stakes architectural design across complex, distributed environments.

You will act as an organizational force multiplier—partnering across product, security research, infrastructure, and engineering teams to transform cutting-edge AI breakthroughs and high-throughput telemetry into resilient, production-grade Security-as-a-Service (SECaaS) offerings.


Your Impact


Strategic Leadership & Team Direction

  • Set Technical Direction: Serve as the primary technical compass for the organization. Evaluate frontline AI research, filter transient hype from production-viable technology, and steer engineering investments toward high-leverage architectural paths.
  • Cultivate Engineering Culture: Champion engineering rigor by instituting RFC standards, leading architectural review boards (ARBs), and mentoring Staff and Principal engineers to scale technical excellence across teams.
  • Cross-Functional Partnership: Bridge the gap between Product Management, Security Researchers, Data Science, and Infrastructure teams. Translate ambiguous business objectives and complex threat landscapes into concrete technical requirements, milestone architectures, and executable designs.

Large-Scale System Design & Architecture

  • End-to-End System Design: Lead the macro- and micro-level architecture of highly distributed, fault-tolerant cloud systems that process real-time security data streams at petabyte scale.
  • Agentic Ecosystems: Architect robust, production-grade agentic frameworks featuring dynamic multi-step planning, stateful memory persistence, secure tool execution, and deterministic guardrails.
  • Data & Telemetry Backbone: Drive the architectural blueprint for ingestion, preprocessing, and feature engineering pipelines using Apache Spark, SQL, and BigQuery across Google Cloud platforms (Dataproc, GKE, and GCP SDKs).

Frontier Modeling, Diagnosis & Remediation

  • Frontier Model Craftsmanship: Maintain deep, hands-on mastery of the evolving model landscape. Guide architectural choices across Large Language Models (LLMs), Small Language Models (SLMs), and multimodal systems to optimize for cost, latency, privacy, and domain-specific efficacy.
  • Root-Cause Diagnosis & Quality Recovery: Establish systematic frameworks to diagnose underperforming models in production (e.g., drift, context dilution, alignment collapse, hallucination spikes). Pinpoint failures across training data distribution, loss convergence, or prompt/context limits, and lead targeted remediation via fine-tuning (SFT, DPO, RL, LoRA/PEFT).
  • GPU & Accelerator Debugging: Troubleshoot low-level compute bottlenecks on distributed GPU clusters, resolving distributed training/inference issues, CUDA out-of-memory (OOM) states, NCCL communication overhead, and kernel performance stalls.

Qualifications

Your Experience

  • Experience & Scale: 10+ years of progressive software engineering and systems design experience, with a documented history of technical leadership delivering high-availability, enterprise-scale platforms.
  • Architectural Mastery: Proven ability to author complex system designs from scratch, establish data contracts, navigate intricate architectural trade-offs, and steer large-scale legacy-to-modern system migrations.
  • Cross-Functional Influence: Demonstrated track record of aligning cross-functional teams (Product, Data, Applied Research, SecOps, Infrastructure) around unified technical visions and resolving competing design priorities.
  • Frontier AI/ML Depth: Deep hands-on knowledge of modern AI architectures (Transformers, attention mechanisms, tokenization strategies) and practical operational experience deploying and serving both LLMs and SLMs.
  • Model Triage & Fine-Tuning: Expert-level capability in analyzing why models degrade and executing targeted recovery pipelines (data curation, synthetic data generation, supervised fine-tuning, preference alignment, and parameter-efficient tuning).
  • Hardware & Systems Debugging: Practical proficiency profiling and debugging distributed workloads on GPUs, diagnosing memory leaks, concurrency locks, and cluster communication bottlenecks.
  • Languages & Data Engineering:
    • Advanced proficiency in Python for AI/ML pipelines and scientific computing.
    • Production-level fluency in Go or Java for low-latency, concurrent backend microservices.
    • Strong foundation in data engineering with Apache Spark, distributed processing, SQL, BigQuery, and GCP orchestration (Dataproc, GKE).
  • Education: BS, MS, or Ph.D. in Computer Science, Data Science, Artificial Intelligence, or a related technical discipline.

Preferred Qualifications (Strong Bonus)

  • Cybersecurity Domain Expertise: Deep background in threat modeling, endpoint detection and response (EDR), SIEM architectures, or cloud security posture management.
  • Network Topologies: Working knowledge of L3–L7 network protocols, packet telemetry, and deep packet inspection systems.
  • Industry Thought Leadership: Contributions to open-source foundation model tooling, published research, or active participation in the broader AI systems community.

#LI-TD1

Compensation Disclosure

The compensation offered for this position will depend on qualifications, experience, and work location. For candidates who receive an offer at the posted level, the starting base salary (for non-sales roles) or base salary + commission target (for sales/com-missioned roles) is expected to be the annual range listed below. The offered compensation may also include restricted stock units and a bonus. A description of our employee benefits may be found here.

$190,400.00 - $308,000.00/yr

Our Commitment

We’re trailblazers that dream big, take risks, and challenge cybersecurity’s status quo. It’s simple: we can’t accomplish our mission without diverse teams innovating, together.

We are committed to providing reasonable accommodations for all qualified individuals with a disability. If you require assistance or accommodation due to a disability or special need, please contact us at  [email protected].

Palo Alto Networks is an equal opportunity employer. We celebrate diversity in our workplace, and all qualified applicants will receive consideration for employment without regard to age, ancestry, color, family or medical care leave, gender identity or expression, genetic information, marital status, medical condition, national origin, physical or mental disability, political affiliation, protected veteran status, race, religion, sex (including pregnancy), sexual orientation, or other legally protected characteristics.

All your information will be kept confidential according to EEO guidelines.

Is role eligible for Immigration Sponsorship?: Yes

Source: Palo Alto Networks on Workday · View original posting (opens in new tab)

How this pay compares

Base floor $190,400, above 56% of the 1,520 Data & ML roles in California on this site. The employer's range is $117,600 wide, 62% of the floor. Observed open for 2 days so far (since Nov 18, 2026).

Pay in context

Among 1,520 Data & ML roles in California listed on this site, the median base floor is $182,000 and the middle half runs from $153,000 to $218,175. This role's floor ($190,400) is above 56% of them.

Base floors of 1,520 Data & ML roles in California

Base floors of 1,520 Data & ML roles in California$100K to $125K: 6.9% (105); $125K to $150K: 12.0% (182); $150K to $175K: 21.2% (322); $175K to $200K: 22.8% (347); $200K to $225K: 16.6% (253); $225K to $250K: 6.4% (97); $250K to $275K: 5.7% (86); $275K and up: 8.4% (128)0%8%16%24%7%10012%12521%15023%17517%2006%2256%2508%275This job $190K
Each bar counts postings on this site by published base floor, in $K. The marker is this job's floor.
Table
Base floors of 1,520 Data & ML roles in California
Floor bandSharePostings
$100K to $125K6.9%105
$125K to $150K12.0%182
$150K to $175K21.2%322
$175K to $200K22.8%347
$200K to $225K16.6%253
$225K to $250K6.4%97
$250K to $275K5.7%86
$275K and up8.4%128
How this is calculated
Statistic
Base floor distribution of the cohort
Formula
count of cohort postings in each $25K band of published base floor / postings in the cohort
Denominator
1,520 open postings with this job family in California
Sample size
1520
Hidden when
the cohort has fewer than 30 postings
Calculation version
jobpage-analysis-v1
Data as of
2026-11-20
Employer set
Open jobs on this site that publish a base salary of $100,000 or more

Only jobs on this site count, and each one publishes a base salary of $100,000 or more, so this compares six-figure jobs with each other, not with the whole market. Computed Nov 20, 2026.

State against nationwide

Median base floor in California vs nationwide

Median published base floor of the same job family, in this state and across all states, next to this job's floor.
Table
Median base floor in California vs nationwide
Valuepostings
Data & ML, California$182K1,520
Data & ML, all states$165K4,414
This job's base floor (marked)$190,400
How this is calculated
Statistic
Median floor, state vs nationwide
Formula
median published base floor of each cohort
Denominator
open postings of this job family
Sample size
1520 in California; 4414 nationwide
Hidden when
either cohort has fewer than 30 postings
Calculation version
jobpage-analysis-v1
Data as of
2026-11-20
Employer set
Open jobs on this site that publish a base salary of $100,000 or more

How Palo Alto Networks pays

15 of Palo Alto Networks's 169 live roles on this site list a higher base floor and 152 list a lower one. Across them the median floor is $156,400; the middle half runs from $147,000 to $176,800.

Base floors of Palo Alto Networks's 169 live roles

Base floors of Palo Alto Networks's 169 live roles$100K to $125K: 7.1% (12); $125K to $150K: 26.0% (44); $150K to $175K: 37.3% (63); $175K to $200K: 21.9% (37); $200K to $225K: 1.2% (2); $225K to $250K: 5.3% (9); $250K to $275K: 1.2% (2)0%10%20%30%40%7%10026%12537%15022%1751%2005%2251%250This job $190K
Each bar counts this employer's live roles by published base floor, in $K. The marker is this job's floor.
Table
Base floors of Palo Alto Networks's 169 live roles
Floor bandShareRoles
$100K to $125K7.1%12
$125K to $150K26.0%44
$150K to $175K37.3%63
$175K to $200K21.9%37
$200K to $225K1.2%2
$225K to $250K5.3%9
$250K to $275K1.2%2
How this is calculated
Statistic
Employer base floor distribution
Formula
count of the employer's open roles in each $25K band of published base floor / its open roles
Denominator
169 open public roles at this employer
Sample size
169
Hidden when
this employer has fewer than 5 roles
Calculation version
jobpage-analysis-v1
Data as of
current
Employer set
Open jobs on this site that publish a base salary of $100,000 or more

Similar pay elsewhere

Other employers' Data & ML roles in California with the nearest published base floor to $190,400, one per employer.

This range next to 6 other live Palo Alto Networks roles with the nearest advertised base floor
Chart as a table
This range next to 6 other live Palo Alto Networks roles with the nearest advertised base floor
RolePublished base
Sr Principal Machine Learning Engineer (Advanced Threat Protection) (this listing)$190,400 - $308,000
Principal AI Scientist - Enterprise AI$190,400 - $308,000
Director, Product Management (Cortex Exposure Management)$191,800 - $310,200
Senior Manager of Product Management (Operations & Reliability - AI Firewalls)$185,900 - $300,675
Sr Principal / Architect Engineer, Software - Backend, Alerting (Chronosphere)$183,600 - $297,000
Senior Principal Software Engineer (Cortex Platform)$183,600 - $297,000
Sr Principal / Architect Engineer Software — SaaS Security Posture Management (SSPM)$183,600 - $297,000

Same $100K to $400K scale as every card. All Palo Alto Networks roles · How to read the bar

Observed changes

No changes to the pay or listing since we first observed this role on Nov 18, 2026 (2 days ago).

  1. First observed on the employer's board, already publishing a qualifying base salary
Observed since
Employer-stated posting date
2026-11-17
Where this pay comes from

Location as the employer wrote it: Santa Clara, United States of America

$190,400.00 - $308,000.00/yr

Method
Employer text, read with fixed rules and re-verified
Checked
Rule set
us-base-v2

The pay passage appears in the employer's own job text. Fixed, deterministic rules located it, and the passage below was checked against the stored copy of the posting.

What we verified

  • The pay figures come from the employer's own listing, quoted exactly above.
  • The pay is an annual base salary in US dollars with a minimum of at least $100,000.
  • The role was publicly listed on the employer's job board when last checked.

We do not verify what the employer will actually offer, whether the employer is still recruiting for the role, or anything beyond what the employer published. Full methodology.

Get Six Figure Work by email

Pick what you want. Nothing is selected for you unless it matches the page you are on, and nothing is sent until you confirm from a link we email you.

What would you like to receive?

A short weekly note on what employer-published pay data shows: who pays what, where, and what is changing. Once a week.

Practical notes for job seekers: interviewing, applying for six-figure roles, and making an upward move. About twice a month.

New jobs that match a search you choose. Every job listed has an employer-published base salary of $100,000 or more. At most one email a day (or weekly, your choice). Nothing is sent on days with no new matches. Alerts cover every new job, because this page has no filters.

We email you only after you confirm from a link we send to this address. We do not sell or share your address. Every email has a one-click unsubscribe link. How we handle your email.