Before you buy an AI résumé screener, ask the vendor to run two fictional résumés through the settings it recommends for one of your open roles. Give both applicants equivalent experience and change how one describes a task that matters to the job. Then ask the vendor to show you what the tool recognized, what the recruiter would see, and whether both applicants would reach the shortlist.

Two résumés can reveal a missed qualification or a screening rule worth investigating. Establishing how reliably the tool evaluates applicants, or how its use affects different groups, takes broader evidence (Society for Industrial and Organizational Psychology [SIOP], 2023).

Why Is Résumé Screening an Inclusion Issue?

A warehouse employee might describe part of their job as inventory reconciliation. Someone doing the same work at a small shop might write that they checked the shelves against the stock record and investigated differences. If your hiring team accepts both descriptions as evidence of the required skill, either applicant should have a fair opportunity to advance.

That’s why this purchasing decision matters to Diversity.com readers. A recruiter can ask about unfamiliar wording only if the application reaches them. A vendor demo gives the hiring team a chance to follow that application through the screening process before the software influences a real decision.

How Do You Set Up the Two-Résumé Demo?

Start with one vacancy and one qualification the hiring manager can explain. Imagine you’re hiring an inventory coordinator who needs to compare physical stock with inventory records and investigate discrepancies. Before the demo, your team agrees that either of the following descriptions would satisfy that criterion at the initial screen.

Create two fictional résumés with the same job title, employment dates, work history, layout, and file type. Label them A and B in your notes. Change only this bullet.

Résumé A

Reconciled weekly inventory counts for 300 items against system records and investigated discrepancies before submitting corrections.

Résumé B

Each week, checked the quantities on hand for 300 items against the stock system and looked into differences before submitting corrections.

Both describe the same frequency, scale, responsibility, and follow-up. The difference you want to observe is whether the wording affects how the tool recognizes that experience. Using fictional records also keeps real applicants’ personal information out of the sales demonstration.

What Should the Vendor Show You Live?

Ask the vendor to process both files during the meeting. These four requests let you follow each résumé from upload to the recruiter’s decision.

1. “Show us the settings before you upload either file.” Confirm which vacancy and qualifications are active. Does the feature extract skills, assign a score, rank applicants, or apply a cutoff? Identify which output determines what happens next in the setup your team would use.

2. “Show us where each applicant received credit for this qualification.” Ask to see the original bullet, the text the system extracted, and any skill or qualification it recorded. If the product shows recruiters only a result, ask what information they can access to check it.

3. “Open the screen our recruiter would normally use.” Can the recruiter find both applicants and read the wording behind each result? If the tool ranks applications, ask where a lower-ranked résumé appears. If it filters applications, ask who can see the ones outside the shortlist.

4. “Show us how we would correct a missed qualification.” Have the vendor use the permissions your recruiters would have. Can someone return an applicant to review, record the reason, and check whether the same rule affected other applications? If the vendor changes a setting, save the original result and rerun both files so you can compare what happened.

What Would a Result Actually Tell You?

Suppose the recruiter can read both bullets in the uploaded résumés, but only Résumé A receives credit for inventory reconciliation. Ask the vendor to show the text the screening feature actually evaluated for Résumé B. If the wording is missing there, investigate how the system read the file. If the wording is present, ask how the tool evaluated it against the job requirement and whether the result kept B off the shortlist.

The consequence for the applicant determines your next question. If both applicants advance despite a difference in scores, record the difference and ask how often the vendor has observed it in broader testing. If B falls below a cutoff and disappears from the normal review queue, ask the vendor to investigate before your team relies on that cutoff. If both receive equal credit, you can document exactly what the demo showed: the tool recognized these two descriptions under the settings you observed.

What Evidence Should You Request After the Demo?

Evidence for the job you’re filling. Ask what the tool was designed to measure, how the vendor evaluated its accuracy and consistency, and which jobs and applicant populations its testing covered. SIOP recommends evaluating AI-based selection assessments for their intended use. Ask the vendor to explain how its evidence applies to your inventory coordinator role and to the particular decision the tool would influence, such as who receives an interview (SIOP, 2023).

Evidence about who could be screened out. Ask what the vendor knows about qualified applicants the tool misses, whether it has examined differences in outcomes across groups, and where its testing has limitations. Accessibility deserves its own question: “If someone encounters a barrier submitting a résumé, where can they request help, and how does their application get back into review?” The U.S. Department of Justice advises employers to examine whether hiring technology can screen out people with disabilities who can do the work (U.S. Department of Justice, Civil Rights Division, 2022).

What Should You Know Before You Buy?

Return to the applicant who wrote “checked the quantities on hand.” Could your recruiter see that experience, recognize its relevance, and advance the application? If only a salesperson with special permissions could find the résumé, ask who on your hiring team would have that access and when they would know to use it. That answer matters when a qualified applicant is waiting for an interview decision.

Use the vendor worksheet in Diversity.com’s Before You Screen: A Practical Review Guide for AI and Automated Hiring Tools to record the settings you saw, what happened to each résumé, and who will follow up on unanswered questions before the purchasing decision.

Get Started with Diversity.com

At Diversity.com, we connect employers with qualified candidates through inclusive hiring, and help job seekers find employers committed to it.

For Employers & HR Professionals:

✔ Post a job and reach candidates across healthcare, higher education, government, nonprofits, and private industry
✔ Browse our resources for guides, checklists, and reports on inclusive hiring
✔ Join our community in LinkedIn for hiring and compliance updates

For Job Seekers:

✔ Find Inclusive Jobs at organizations that chose to post here
✔ Create a free account to apply and get matched to new openings
✔ Browse our resources for career guides and tools

Have questions? Reach out to our team. We're here to support you.

Frequently Asked Questions

Can Two Résumés Show Whether an AI Screener Is Biased?

Two résumés can flag a specific inconsistency. Assessing fairness across an applicant population requires broader data, appropriate comparisons, and attention to how the employer uses the results. Equal treatment of this pair alone cannot establish fairness or legal compliance (SIOP, 2023).

What If the Vendor’s Tool Doesn’t Give Applicants a Score?

Follow the output the recruiter uses. A tool might extract skills into searchable fields or flag whether a requirement appears to be met. In the inventory example, ask the recruiter to search for applicants with inventory reconciliation experience. Do both résumés appear, and can the recruiter see the wording behind each match?

What If We Already Use a Résumé Screener?

Run the exercise in a test environment using your current settings. If you’re reviewing a live application, compare the applicant’s original résumé with the job requirements before acting on a screening recommendation. Diversity.com’s guide to human review before rejecting a candidate walks through that decision.

Related Articles

• When Employers Drop Degree Requirements, Who Actually Gets Hired?

• Amazon and Target’s DEI Policies in 2026: What Job Seekers Can Verify

• Neuroinclusive Onboarding: A Manager’s Plan for the First Week

• The Job You Interviewed For Is on Hold. What Should You Ask the Recruiter?

Sources & References

• Society for Industrial and Organizational Psychology. (2023, January). Considerations and recommendations for the validation and use of AI-based assessments for employee selection. Full report (PDF)

• U.S. Department of Justice, Civil Rights Division. (2022, May 12). Algorithms, artificial intelligence, and disability discrimination in hiring. ADA.gov guidance