Has AI made AI-powered employment assessments irrelevant? | From Dorms to Desks Podcast | ep44

On today’s episode of the From Dorms to Desks Podcast, our cohosts discuss how online pre-employment assessments have become a foundational element in modern hiring, with over half of organizations incorporating them into their processes. Employers are increasingly asking candidates to complete tests – everything from coding challenges to personality questionnaires – to verify skills and qualities that might not be evident solely from a résumé. These tools are seen as a way to provide a more objective, data-driven method to identify qualified talent, sometimes allowing companies to give opportunities to candidates without traditional credentials who can nonetheless demonstrate their abilities.

These assessments come in various forms. Technical skills assessments, including work samples and job simulations, aim to confirm hands-on abilities claimed on a résumé, asking candidates to perform job-like tasks. Cognitive ability tests, measuring general mental skills like reasoning and problem-solving, are widely used because research shows they are a strong predictor of job performance. Personality and psychometric tests delve into behavioral traits and work styles, helping gauge cultural fit and identifying characteristics like conscientiousness or teamwork preference. Beyond these, there are also situational judgment tests, job knowledge tests, and innovative gamified assessments designed to evaluate candidates in engaging ways. When designed correctly and validated, these tools promise benefits such as improving the quality of hire, increasing efficiency in screening, and potentially boosting diversity by reducing bias. Many HR professionals now view assessment scores as as important or more important than traditional criteria like education or experience.

However, the widespread use of online assessments has coincided with increased attempts to game them. The rise of generative AI, particularly tools like ChatGPT, has drastically changed the landscape of cheating. These AI programs can quickly solve coding problems, generate written answers, and provide information that candidates might use to gain an unfair advantage, raising significant concerns that the assessments are no longer measuring the candidates’ own abilities. Real-world anecdotes, such as hiring managers receiving identical, obviously AI-generated incorrect answers from multiple candidates or AI providing flawless solutions in live coding experiments, underscore the reality and scale of AI-assisted cheating.

There are several reasons why candidates are turning to AI for help. The intense competitive pressure in the job market can lead some to feel desperate for any edge. A significant factor is often a lack of confidence or impostor syndrome, pushing candidates to use AI as a crutch even if they might be capable on their own. Frustration with hiring processes that candidates perceive as unfair, irrelevant, or overly difficult also plays a role, leading some to rationalize using AI as merely navigating a flawed system. Crucially, the sheer ease and accessibility of powerful AI tools have dramatically lowered the barrier to attempting to cheat.

This widespread AI use presents a serious threat to the integrity of hiring. It can erode trust in assessment results, potentially leading to hiring individuals who cannot perform the required tasks despite high test scores. Such bad hires are costly in terms of time and resources. It might also push companies toward more burdensome evaluation methods, like a return to extensive in-person or highly proctored exams, which are inconvenient for everyone involved. Furthermore, there’s a security risk if candidates paste proprietary assessment content into public AI models.

Fortunately, employers are actively developing and implementing strategies to combat AI-assisted cheating. A primary approach is smarter test design. This involves creating unique or adaptive questions that are difficult for standard AI to answer correctly, using dynamic or open-ended tasks requiring human reasoning or explanation, incorporating elements that are challenging for text-based AI like game-based puzzles or visual interpretation, and designing questions where the process or rationale is more important than a single right answer. Assessments like personality tests, which lack clear “right” answers, are also less susceptible to straightforward AI gaming.

Another crucial strategy involves enhanced proctoring and AI-detection technologies. This includes using automated proctoring software that monitors candidates via webcam and locks down browsers, looking for suspicious behavior. Employers are also deploying plagiarism and AI output detection tools to scan submitted code or text for signs of machine generation or matches to known AI outputs. Human monitoring through live proctoring or auditing by comparing assessment results with interview performance helps verify consistency. Explicitly stating policies against AI use and requiring candidate attestation can also act as a deterrent.

Beyond test design and proctoring, employers are exploring alternative evaluation methods that are inherently harder for AI to game. This means supplementing or reducing reliance on easily cheatable tests by emphasizing structured interviews, live problem-solving exercises, or take-home project-based assessments where candidates must present and defend their work. Looking at holistic signals like portfolios, contributions to open-source projects, detailed reference checks, or even trial periods can offer a more complete picture of a candidate’s actual abilities. Some forward-thinking companies are even considering ways to allow AI use in specific contexts to assess how candidates responsibly leverage tools, though this requires careful design to distinguish between assistance and outright cheating.

Ultimately, maintaining a fair and effective hiring process benefits everyone. While AI presents significant new challenges, employers can continue to utilize online assessments by combining thoughtful design, robust security measures, and diverse evaluation methods. The goal remains to identify and hire candidates based on their true abilities and fit, not their capacity to exploit technology. In this evolving landscape, the essential human elements – judgment, integrity, and adaptability – remain key to successful hiring decisions.