Verdict
✓ CERTIFIED
All checks passed
Mean κ (AI vs recruiter)
0.773
Substantial agreement
Counterfactual drift
0.000
Zero proxy bias detected
Min 4/5 ratio
1.00
All groups above 0.80
AI Resume Screening Audit

Audit your AI screener for
name-proxy and prestige bias
before it sees one résumé

Three independent checks — disparate impact, counterfactual drift, and recruiter–AI agreement — certify or block your screener. Every metric is reproducible and audit-ready.

3
Independent bias checks
450+
Counterfactual pairs per req
0.80
EEOC four-fifths threshold
0.60
κ threshold for certification
How it works

Three layers. One verdict.

HR Fidelity runs every screener through the same gauntlet regulators use, then issues a machine-readable certificate.

⚖️

Four-fifths disparate impact

EEOC § 60-3.4 / NYC Local Law 144: any group selected at less than 80% of the top group's rate triggers a violation. Computed across name-based race signal and gender before any hire.

🔀

Counterfactual drift

Every résumé gets a matched twin — identical content, one proxy signal swapped. If mean score drift exceeds 5%, the screener is blocked. No bias hides in identical pairs.

📏

Recruiter–AI calibration (κ)

Cohen's κ measures whether the AI and human recruiters agree on A/B pairs. κ ≥ 0.60 — "substantial" agreement — is required. Gold pairs act as attention checks.

🔒

Synthetic-only data

All résumés are 100% synthetic. Names from Bertrand-Mullainathan, SSA, and Census public datasets. No real candidates, no employer hiring records, no privacy risk.

📋

Rubric-only scoring

Screeners score against a fixed job-derived rubric — never trained on past hires. This is the Amazon rule: historical data encodes historical bias.

⚡

Live configuration knobs

Toggle prestige bonuses, name signals, and skill weights in real time. Watch the verdict flip from BLOCKED to CERTIFIED as you remove bias vectors.

Why this matters

The Amazon problem repeats itself

In 2018, Amazon discovered its ML hiring tool had learned to penalize résumés containing the word "women's" and downgraded graduates of all-women's colleges. The system was trained on ten years of hiring decisions — data that reflected a historically male engineering workforce.

It was shut down before deployment. But the pattern didn't stop at Amazon. Any screener trained on past hires inherits past bias. HR Fidelity audits before the screener ever sees a live application.

Amazon's system learned to penalize résumés that mentioned women's organizations and downgraded graduates of women's colleges — a direct consequence of training on a decade of hiring decisions made by a male-dominated workforce.

— Based on Amazon's internal audit findings, reported 2018
What this audit covers
  • ✓ Name-proxy bias — Bertrand-Mullainathan race-signal first names
  • ✓ Institution prestige-tier bias — elite vs regional school weighting
  • ✓ Gender-signal bias from name-based proxies
  • ✓ EEOC four-fifths disparate impact across demographic groups
What it doesn't claim
  • — Zip code, address, or neighbourhood correlates
  • — Employment gap or career-path patterns
  • — Résumé phrasing that encodes demographic signal
  • — Bias learned from real historical training data
How recruiters use this

Three steps to a compliance-ready audit

1
Run the default audit

Leave all knobs at their defaults and click Run audit. You'll see CERTIFIED — the screener is scoring on qualifications only, selection rates are even across groups, and it agrees with human recruiters. This is your baseline: a fair screener.

2
Turn on name-based signals

Toggle Name-based signals on and run again. The verdict flips to BLOCKED. The screener is now using applicant names as a proxy for race and gender — and the four-fifths ratios show it. Black and Hispanic applicants are advancing at less than 80% the rate of white applicants. That's an EEOC violation.

3
Tune your screener's configuration

Try raising the Prestige bonus slider. Even without name signals, favouring elite-school graduates introduces proxy correlation — and the counterfactual drift check catches it. The knobs represent real configuration choices your AI vendor offers. The audit tells you which combinations are legally safe before a single real applicant is touched.

AI Audit Pipeline — data flow from resume pool through screener, three parallel checks, to CERTIFIED or BLOCKED verdict
Full evaluation pipeline — screener output feeds all three audit checks simultaneously
The output is your compliance artifact. The CERTIFIED/BLOCKED verdict, the four-fifths ratios, and the drift measurements are exactly what NYC Local Law 144 requires you to document before deploying an automated hiring tool. Run this before the screener touches a live application — not after.
Live demo

Try it yourself

Start with defaults → Run audit → then toggle name-based signals and watch the verdict flip.

🔒 Synthetic data only — résumés generated from public name datasets. No real candidates. No employer records.
Screener configuration
0.00
No prestige weighting — all institutions scored equally
0.60
Balanced — skills and experience weighted equally
0.70
Standard bar — top 30% of qualified candidates advance
Turn on to simulate a screener that learned to use applicant names as race and gender proxies — then watch the audit catch it.
Configure your screener and click Run audit to see results.

Your screener. Audit-ready.
Before a single hire.

HR Fidelity gives you the audit trail that legal, compliance, and regulators expect — built on open, reproducible methodology.

🔒 All data is synthetic · No employer records used · Open-source corpus
Regulatory Frameworks Referenced
EEOC Uniform Guidelines — Four-Fifths Rule NYC Local Law 144 (2023) California FEHA / CRD AI Employment Guidance EU AI Act — High-Risk Employment Systems