Six Figure Work

Model Behavior Engineer

Overtone

Company websiteoverto.ne (Overtone, opens in new tab)

Base salary $160K – $190K

Employer-published annual base salary, USD. Advertised base range in New York City, NY

Apply (opens in new tab)

You apply on the employer's own site. Six Figure Work does not handle applications.

  • LocationNew York City, NY
  • WorkplaceOn-site
  • Status at last checkSource accepting applications when checked
  • Last checked (1 h ago) · observed since

About this job

Your Role

Your job is to work with our relationship scientist and eval engineer to build and execute an eval strategy that keeps our LLM content high quality at scale.

  • Our relationship scientist defines what good looks like across our content surfaces and drafts golden examples. They also label for LLM judges that need subject matter expertise or calls on tone and voice.

  • Our eval engineer builds the infrastructure to power our content monitoring and iteration: human labeling tools, automated judge calibration tools, Braintrust experiments, and production monitoring dashboards.

  • You are the hands-on operator of this system: the eval engineer builds the tools, the relationship scientist defines goldens, and you use both every day to find and fix problems in our content.

The team is early, so you'll have real influence over how this workflow gets built.

What You’ll Do

  • Prompt iteration and engineering. Fix straightforward errors yourself: when our judges flag a problem in production content, diagnose the failure, change the prompt, and confirm the fix with evals before it ships. When a fix depends on subject matter expertise, partner with our relationship scientist. You’ll draft prompts for our content surfaces and our judge rubrics.

  • Eval + content engineering strategy. Design our infrastructure for evals and content iteration, including labeling, judge calibration and production quality monitoring and keep evolving it as the product grows.

  • Define what we measure. Work with our relationship scientist to decide which content features to measure, from proactive baseline quality checks to reactive checks driven by errors we find. Set up automated error analysis so new failure patterns surface on their own, not only when someone spots them.

  • Labeling strategy and tools. Own how we label and calibrate our judges, including the tools and workflows that scale our relationship scientist's judgment as efficiently as possible. Make sure they label only where their expertise is needed. Everywhere else, label production outputs and match assessments yourself or through outsourced labelers, so we can check that our judges agree with human judgment.

  • Synthetic testing. Design synthetic tests that tell us whether a new prompt is working before it reaches human testing, and review the results.

  • Production review and prioritization. Read real production outputs across our core surfaces and spot quality issues. Prioritize what to fix first across everything our judges, automated error analysis and manual reviews surface.

Who You Are

We're open to a very product-minded engineer, or a highly technical product person:

  • 3+ years in product management or engineering, including shipping LLM-powered features to real users

  • Hands-on experience with evals: writing rubrics, building or running LLM-as-judge setups, and aligning judges against human labels

  • Hands-on prompt engineering experience: you can diagnose why an output went wrong, rewrite the prompt, and show with evals that the fix worked

  • Comfortable writing code when the work calls for it, like scripts for error analysis or changes to our eval pipeline

  • A detail-oriented, editorial eye for tone, structure and voice in AI-written content

  • Eagerness to dive in and read lots of AI outputs by hand

Great to have:

  • Experience with Braintrust or similar eval tooling

  • Experience running human labeling programs

  • Experience with conversational or voice AI products

  • Interest in relationships, psychology or matchmaking

Compensation & Benefits

Salary of $160k to $190k. The offer within that range depends on experience and skills. This position will receive meaningful stock options with considerable upside potential.

We provide standard benefits such as:

  • Medical, dental and vision insurance

  • 401(k)

  • Commuter benefits, daily lunch and other perks

  • Paid time off, holidays and sick leave

Our Core Qualities

We look for these qualities in everyone we hire at Overtone.

Curiosity: You seek to understand before you decide. You ask thoughtful questions, test assumptions, and stay open to changing your mind as new information emerges.

Craft: You care deeply about quality and coherence. You simplify complexity, exercise good judgment, and take pride in work that feels intentional, thoughtful, and well-made.

Collaboration: You engage with others honestly and generatively. You challenge ideas with respect, listen deeply, and help synthesize perspectives into clearer, better outcomes.

Care: You take responsibility for people and outcomes. You act with integrity, follow through on commitments, and protect trust—with users, teammates, and the work itself.

About Overtone

Overtone is a modern matchmaking company, from the creator of Hinge. We’re building a voice- and audio-forward service, enabled by AI, grounded in relationship science. There’s no profiles or swiping. We get to know people in their own voice, then make thoughtful introductions, one at a time, to people we think are worth meeting. The service is in beta-testing and not yet open to the public. We’re building a small team of people passionate about relationships and human connection, to work together in person in our NYC office (5 days a week in Flatiron).

Equal Opportunity

Overtone is an equal opportunity employer. We consider all qualified applicants without regard to race, color, religion, creed, sex, pregnancy, sexual orientation, gender identity or expression, national origin, ancestry, age, disability, genetic information, marital or partnership status, caregiver status, veteran or military status, or any other status protected by law. We will consider qualified applicants with arrest and conviction records in a manner consistent with the New York City Fair Chance Act and other applicable law. If you need an accommodation at any point in the application or interview process, email [email protected] and we'll work with you.

Source: Overtone on Ashby · View original posting (opens in new tab)

How this pay compares

Base floor $160,000. The employer's range is $30,000 wide, 19% of the floor. First observed today.

This range next to 1 other live Overtone role with the nearest advertised base floor
Chart as a table
This range next to 1 other live Overtone role with the nearest advertised base floor
RolePublished base
Model Behavior Engineer (this listing)$160K - $190K
Head of Marketing$250K - $300K

Same $100K to $400K scale as every card. All Overtone roles · How to read the bar

Observed changes

No changes to the pay or listing since we first observed this role on Nov 20, 2026 (today).

  1. First observed on the employer's board, already publishing a qualifying base salary
Observed since
Employer-stated posting date
2026-11-20
Where this pay comes from

Location as the employer wrote it: New York City

USD 160,000 – 190,000 · 1 YEAR · Salary

Method
Structured pay field published by the employer
Checked
Rule set
us-base-v2

The employer's job board publishes this pay as a dedicated data field. We read the field directly; no interpretation was needed.

What we verified

  • The pay figures come from the employer's own listing, quoted exactly above.
  • The pay is an annual base salary in US dollars with a minimum of at least $100,000.
  • The role was publicly listed on the employer's job board when last checked.

We do not verify what the employer will actually offer, whether the employer is still recruiting for the role, or anything beyond what the employer published. Full methodology.

Get Six Figure Work by email

Pick what you want. Nothing is selected for you unless it matches the page you are on, and nothing is sent until you confirm from a link we email you.

What would you like to receive?

A short weekly note on what employer-published pay data shows: who pays what, where, and what is changing. Once a week.

Practical notes for job seekers: interviewing, applying for six-figure roles, and making an upward move. About twice a month.

New jobs that match a search you choose. Every job listed has an employer-published base salary of $100,000 or more. At most one email a day (or weekly, your choice). Nothing is sent on days with no new matches. Alerts cover every new job, because this page has no filters.

We email you only after you confirm from a link we send to this address. We do not sell or share your address. Every email has a one-click unsubscribe link. How we handle your email.