Promise and Perils of Using AI for Hiring: Guard Against Data Bias

By AI Trends Staff  

While AI in hiring is now widely used for writing job descriptions, screening candidates, and automating interviews, it poses a risk of wide discrimination if not implemented carefully. 

That was the message from Keith Sonderling, Commissioner with the US Equal Opportunity Commision, speaking at the AI World Government event held live and virtually in Alexandria, Va., last week. Sonderling is responsible for enforcing federal laws that prohibit discrimination against job applicants because of race, color, religion, sex, national origin, age or disability.   

“The thought that AI would become mainstream in HR departments was closer to science fiction two year ago, but the pandemic has accelerated the rate at which AI is being used by employers,” he said. “Virtual recruiting is now here to stay.”  

It’s a busy time for HR professionals. “The great resignation is leading to the great rehiring, and AI will play a role in that like we have not seen before,” Sonderling said.  

AI has been employed for years in hiring—“It did not happen overnight.”—for tasks including chatting with applications, predicting whether a candidate would take the job, projecting what type of employee they would be and mapping out upskilling and reskilling opportunities. “In short, AI is now making all the decisions once made by HR personnel,” which he did not characterize as good or bad.   

“Carefully designed and properly used, AI has the potential to make the workplace more fair,” Sonderling said. “But carelessly implemented, AI could discriminate on a scale we have never seen before by an HR professional.”  

Training Datasets for AI Models Used for Hiring Need to Reflect Diversity  

This is because AI models rely on training data. If the company’s current workforce is used as the basis for training, “It will replicate the status quo. If it’s one gender or one race primarily, it will replicate that,” he said. Conversely, AI can help mitigate risks of hiring bias by race, ethnic background, or disability status. “I want to see AI improve on workplace discrimination,” he said.  

Amazon began building a hiring application in 2014, and found over time that it discriminated against women in its recommendations, because the AI model was trained on a dataset of the company’s own hiring record for the previous 10 years, which was primarily of males. Amazon developers tried to correct it but ultimately scrapped the system in 2017.   

Facebook has recently agreed to pay $14.25 million to settle civil claims by the US government that the social media company discriminated against American workers and violated federal recruitment rules, according to an account from Reuters. The case centered on Facebook’s use of what it called its PERM program for labor certification. The government found that Facebook refused to hire American workers for jobs that had been reserved for temporary visa holders under the PERM program.   

“Excluding people from the hiring pool is a violation,” Sonderling said.  If the AI program “withholds the existence of the job opportunity to that class, so they cannot exercise their rights, or if it downgrades a protected class, it is within our domain,” he said.   

Employment assessments, which became more common after World War II, have provided  high value to HR managers and with help from AI they have the potential to minimize bias in hiring. “At the same time, they are vulnerable to claims of discrimination, so employers need to be careful and cannot take a hands-off approach,” Sonderling said. “Inaccurate data will amplify bias in decision-making. Employers must be vigilant against discriminatory outcomes.”  

He recommended researching solutions from vendors who vet data for risks of bias on the basis of race, sex, and other factors.   

One example is from HireVue of South Jordan, Utah, which has built a hiring platform predicated on the US Equal Opportunity Commission’s Uniform Guidelines, designed specifically to mitigate unfair hiring practices, according to an account from allWork.   

A post on AI ethical principles on its website states in part, “Because HireVue uses AI technology in our products, we actively work to prevent the introduction or propagation of bias against any group or individual. We will continue to carefully review the datasets we use in our work and ensure that they are as accurate and diverse as possible. We also continue to advance our abilities to monitor, detect, and mitigate bias. We strive to build teams from diverse backgrounds with diverse knowledge, experiences, and perspectives to best represent the people our systems serve.”  

Also, “Our data scientists and IO psychologists build HireVue Assessment algorithms in a way that removes data from consideration by the algorithm that contributes to adverse impact without significantly impacting the assessment’s predictive accuracy. The result is a highly valid, bias-mitigated assessment that helps to enhance human decision making while actively promoting diversity and equal opportunity regardless of gender, ethnicity, age, or disability status.”  

The issue of bias in datasets used to train AI models is not confined to hiring. Dr. Ed Ikeguchi, CEO of AiCure, an AI analytics company working in the life sciences industry, stated in a recent account in HealthcareITNews, “AI is only as strong as the data it’s fed, and lately that data backbone’s credibility is being increasingly called into question. Today’s AI developers lack access to large, diverse data sets on which to train and validate new tools.”  

He added, “They often need to leverage open-source datasets, but many of these were trained using computer programmer volunteers, which is a predominantly white population. Because algorithms are often trained on single-origin data samples with limited diversity, when applied in real-world scenarios to a broader population of different races, genders, ages, and more, tech that appeared highly accurate in research may prove unreliable.” 

Also, “There needs to be an element of governance and peer review for all algorithms, as even the most solid and tested algorithm is bound to have unexpected results arise. An algorithm is never done learning—it must be constantly developed and fed more data to improve.” 

And, “As an industry, we need to become more skeptical of AI’s conclusions and encourage transparency in the industry. Companies should readily answer basic questions, such as ‘How was the algorithm trained? On what basis did it draw this conclusion?” 

Read the source articles and information at AI World Government, from Reuters and from HealthcareITNews.