The setting nobody will show you.

By Jim Stroud

Sooner or later, somebody in your graduating class will say a version of this out loud: I did not get that job because of my race.

Students of every background say it. They cannot all be right, and none of them can check.

Here is what is actually going on, minus the group chat version.

The math problem underneath

To correct for bias, a screening model has to know your race.

A tool cannot adjust for a group it refuses to identify. The moment it identifies you, it is making a decision that touches a protected characteristic, and federal law gets interested. Vendors call this fairness-aware design. Critics call it a quota with a software license.

Both describe the same equation.

Why the law cannot referee cleanly

Two rules sit in the same statute and point in opposite directions.

Employers can be liable for a practice that produces lopsided results even without meaning to. That is disparate impact. Employers can also be liable for treating people differently because of race or sex, which is disparate treatment. Fix one and you can trip the other.

The Supreme Court hit this in Ricci v. DeStefano in 2009. New Haven threw out firefighter promotion exam results because too few Black candidates passed. The Court held that discarding the results was itself race-based action. Whether a model that reweights candidate rankings falls under the same reasoning is still an open question. Georgetown Law professor Jason Bent wrote an entire article on it in 2019 called “Is Algorithmic Affirmative Action Legal?” and did not close the case.

Two things shifted in 2025

The courthouse door opened wider. On June 5, 2025, the Supreme Court decided Ames v. Ohio Department of Youth Services 9-0. Some appeals courts had required plaintiffs from majority groups to clear an extra evidentiary hurdle. Justice Jackson wrote that the statute draws no such distinction. Justice Thomas, joined by Justice Gorsuch, added that diversity programs have often produced discrimination against people seen as the majority.

Federal enforcement also reversed. President Trump signed Executive Order 14281 in April 2025 directing agencies to eliminate disparate-impact liability wherever possible. By September 2025 the Equal Employment Opportunity Commission was closing pending disparate-impact charges.

An employer that tuned its model in 2023 to survive one kind of audit now owns a design choice a rejected applicant can point at.

Both arguments, fairly

Defenders say the raw model was never neutral. It learned from decades of human hiring decisions, so leaving it alone preserves whatever those decisions contained. On that view, checking outcomes by group is auditing, and refusing to look is also a choice.

Opponents say the fix trades one wrong for another. The law protects individuals, not ratios, and a person rejected to improve a demographic number has a real injury no matter how the aggregate looks. They add that “correcting historical bias” lets an employer set the correction wherever it likes, with nobody outside able to check the number.

Both sides agree on one thing. No applicant can see which setting was used.

What the evidence shows so far

The lawsuits on file run mostly the other way. Mobley v. Workday alleges AI screening tools disadvantaged Black, older, and disabled applicants, and a federal court let the claims move forward. Researchers at Stanford’s Institute for Human-Centered AI tested resume screening and found 26 percent of Black applicants and 15 percent of Asian applicants applied to roles where the system recommended their group below the federal four-fifths threshold.

No published American case has produced a verified audit showing a company boosting candidates by demographic. That does not prove it never happens. It means the evidence lives inside vendor contracts, and only discovery pulls it out.

Ranked before anyone saw your face

Dating apps order profiles before either person exists to the other. Boost one group in the feed and both sides experience the result as chemistry.

Hiring works that way now, except the outcome is rent money.

What you can actually do about it

• Do not diagnose your own rejection. You cannot tell a demographic adjustment from a keyword miss from a hiring freeze from a job that was never real. Every hour spent theorizing is an hour not spent on the next application.
• Ask what the tool is. In New York City and Illinois, employers owe you notice when automated screening is used. Ask a recruiter who scores the application and whether a person reviews the result. The answer, or the dodge, tells you something.
• Get in front of humans. Referrals, alumni contacts, campus recruiters, and career fairs move you past the ranking layer entirely. That is the whole play, and it works regardless of how this fight resolves.

Nobody has settled this

Courts have not decided whether tuning a model for fairness is compliance or discrimination. The agency that would normally referee has stepped back. California and Illinois still treat unequal outcomes as actionable, so employers are reading two rulebooks at once.

Expect this argument to follow you through your whole career. Expect nobody to show you the setting.

ABOUT THE AUTHOR  

Jim Stroud is a Career Intelligence Analyst, labor market strategist, and Head of Market Strategy & Industry Engagement at ProvenBase. With more than two decades of experience in recruiting, sourcing, and labor market analysis, he helps organizations and job seekers make sense of a rapidly evolving employment landscape. 

He is also the publisher of Career Intelligence Weekly (which tracks the hidden job market), Job Search 3.0 (an online job-hunting course) and host of The Jim Stroud Podcast (commentary on the world of work). He is also an international conference speaker, job search workshop facilitator for college students and author of multiple books on career strategy and recruiting.