Gender Bias in AI and Machine Learning Systems
HTML-код
- Опубликовано: 9 июн 2021
- Every year, Amazon gets more than 200 thousand resumes for the various jobs they are hiring for. Google, gets 10 times that, with over 2 million resumes each year. Imagine being the HR managers responsible for vetting all those. That seems like an absolutely daunting task, but, in the modern age we live in, this seems like a task that could be given over to something that could process those resumes nearly instantaneously: an artificial intelligence system. In fact, that’s exactly what companies like Amazon and Google have tried in the past, though the results were not what they expected.
Welcome to Data Demystified. I’m Jeff Galak and in this episode we’re going to talk about gender bias in artificial intelligence. To be sure, there are many examples of bias in machine learning and AI systems and I plan to make videos about those too, but, for now, I want to focus on one big example in the world of resume vetting. After all, one of the goals of things like gender and racial equity is to ensure that everyone, regardless of their gender or race, has a fair shake at the most desirable jobs out there. But when companies let AI algorithms have a say in those decisions, bias has a sneaky way of creeping in. In this episode, I’m going to try and provide you with the intuition to understand how this type of bias could emerge, even when a big goal of these systems is to take potentially biased humans out of the equation all together.
Learn more about who I am and why I'm doing this here: • Data Demystified - Who...
Follow me at:
LinkedIn: / jeff-galak-768a193a
Patreon: / datademystified
Equipment Used for Filming:
Nikon D7100: amzn.to/320N1FZ
Softlight: amzn.to/2ZaXz3o
Yeti Microphone: amzn.to/2ZTXznB
iPad for Teleprompter: amzn.to/2ZSUkNh
Camtasia for Video Editing: amzn.to/2ZRPeAV
An example:
If you type, *"Who has scored the most goals in international football?"* It'll automatically say Cristiano Ronaldo with 118. But, when in fact it's Christine Sinclair with 190 goals.
i highly appreciate the fact you answered all the coments in a detalied and civil manner, this is very uncommom on the internet.
(Sorry for any spelling mistakes, english is not my native language)
Thnx for making such videos
In a system that provides a supply of 80/20 male to female students in the field from ages 14 to 22, wouldn't a deviation from that supply proportion be the only indication of bias? If there were some bias from birth to hiring, wouldn't the bias be at the education level that created the imbalance and not the hiring procedures that are hiring in line with the proportions of its supply?
Hi Scott. First, you are right that the supply of women interested in, say CS, jobs is smaller than the supply of men. So, at minimum, the benchmark isn't 50/50 employment (for now), but, at the very least, parody with the supply. However, there are two lingering problems: 1) there is a very good argument to made to over-represent women in hiring in order to create roll models for girls that might not otherwise see themselves in these roles. When people (of all genders/races/etc) see people like themselves in various roles, they are more likely to then want to pursue those paths. 2) The issues with these systems is that they will treat two otherwise identical applications differently, based on gender. If the systems see two highly qualified applicants, they will favor men. This is bias, no matter what the baserates happen to be. Thanks for the comment.
@@DataDemystified the over representation would be biased in itself. Possibly there are people that argue that men should be provided preferential treatment in fields like nursing, teaching, social work, etc. , but i have not heard this much compared to something like engineering male bias.
I'm not sure how you would come to the conclusion that these applications will come to a bias if the results are either in line with supply or possibly overrepresenting females as you allude to in the case of Google. I feel like you fall into the trap of filling a void, just because there is a void there.
@@scottw9267 Thanks for the comment and I think I need to clarify my previous reply to you a bit. When we think about bias in things like hiring systems, the goal isn't just to get to 50/50 representation, but it's rather to consider the historical context of what we're dealing with. With jobs in CS, historically, women have been left out BECAUSE of intentional biases against them. As in, men ran the field (they still largely do) and intentionally chose to prefer hiring other men over women. So, that is bias that is driven the gatekeepers in the field. AI/ML hiring systems like the ones I discuss in this video continue that pattern...albeit without explic intent (the programs don't say "exclude women") but rather just by continuing the same behaviors that the humans before them engaged in. In cases like nursing, teaching, social work, there is NO evidence (that I know of) of intentional bias against men. The higher rates of women in those fields is entirely a function of preferences. As in, it's not that there is this glut of men who are dieing to become nurses and teachers but women are actively keeping them out. Rather, there just aren't as many men interested in those fields. So the outcome appears biased against men (there are fewer of them), but the process isn't biased. That said, if it were the case that men were unable to get nursing jobs because of their gender, I would 100% agree that this is a problem, just like it is a problem that women can't get CS jobs.
On to over-representation. The comment I made was suggesting that to correct a historical problem, it's enough to just get to 50/50 hiring of men/women. If you do that, it'll take a very long time to reach something like 50/50 EMPLOYMENT of men/women because the starting point is already so skewed towards men. So, there is a case to over-hire women to help balance the gender representation rates in the field. This is entirely predicated on enough supply, and, I am arguing, there IS enough supply of qualified women. Just see how CMU computer science managed to get to 50/50 CS undergraduates by actively deciding to change the paradigm about what counts as "qualified." And note that they did this without sacrificing quality one bit (details here: cacm.acm.org/magazines/2019/2/234346-how-computer-science-at-cmu-is-attracting-and-retaining-women/fulltext
In our opinion what should be an unbiased system? What do you mean by gender equity in tech hiring process? For me it seems like the predictors chosen in the statistical model are such that they identify best possible ability of a candidate. For example do they know python? Women being less on stem fields may be the reason lesser women know python. What is your solution to solve this? Thank you
Thank you for your comment. If the models really only selected on things like programming qualifications, then I would see no issue. The problem is that they DON'T do that (or at least, didn't). They select on variables that have nothing to do with actual qualifications and simply reflect the (intentional or unintentional) biases of those who made previous job hiring decisions. And you are right that women are underrepresented in STEM fields. But there are ways to fix that too. A personal example, at my institution, Carnegie Mellon University's Computer Science dept. It is one of the best departments in the world and was, like everywhere else, incredibly male dominated in terms of undergraduate students. And the story was that there just aren't enough girls in high school interested in STEM to get "high quality" undergraduates. Well, that was entirely based on beliefs that were just inconsistent with reality, but consistent with pre-existing biases that folks here at CMU had. So, CMU, to their incredibly credit, decided to fix this. They changed how they considered admissions and focused on a holistic take on students, rather than on things like "do you already know how to code before even coming to college". In doing so they increased female representation in the undergraduate CS program to ~50%...and, critically, the quality of students graduating did not drop one bit. As in, it's not that the school took underqualified female applicants...rather they redefined what it even means to be qualified. And without ever watering down the curriculum, that redefinition resulted in equitable gender admissions AND kept standards high. It can be done (see details here: cacm.acm.org/magazines/2019/2/234346-how-computer-science-at-cmu-is-attracting-and-retaining-women/fulltext
With algorithms, it's tricky, but there are new approaches to this. You can penalize AI systems while training them if, for instance, gender bias is detected after the fact (this is more technical than I care to get into here, but if you're curious see here: hbr.org/2019/11/4-ways-to-address-gender-bias-in-ai). You could do all that is possible to ensure equitable representation in training data (e.g. facial recognition training datasets use overwhelmingly white faces and so they are bad (lots of false positives) in identifying faces from people of color). There are ways to fix this....they aren't easy, but it can be done.
@@DataDemystified Thank you for your detailed reply. I understand your point and I also believe this is a interesting and challenging problem to solve as ultimately the goal is to hire without uncontrollable human characteristics such as race, gender etc. Women are also underrepresented at stem subjects here at my institution at Waterloo and they are improving on this. The top two links you have given are unavailable and if you could provide updated links, I would love to read them. Thank you
@@ahnafkhan5a Sorry about that. Not sure why the links were broken. Try this (and I updated above):
cacm.acm.org/magazines/2019/2/234346-how-computer-science-at-cmu-is-attracting-and-retaining-women/fulltext
hbr.org/2019/11/4-ways-to-address-gender-bias-in-ai
@@DataDemystifiedGreat explanation. It seems to me like bias has to be addressed at the source level. In this case, high schools have to take into consideration why more boys into computer science. For instance, It could simply be because more boys play virtual games compared to girls. And, that could be one reason for their interest in computers and coding. So, the bias creeps in when Universities consider only students who know how to code before school. Thanks for explaining how CMU has addressed this. Taking such measures just cuts off bias at the University level preventing it from creeping into the workforce. It is very interesting to see how biases happen when we work our way backward to the source. At the algorithmic level, It is indeed very challenging when we don't have a biased data source. Thanks for sharing the methods on how this could be addressed. I just found your channel and I am enjoying it a lot.
@@kavinveerapandian568 Thanks for the comment! I agree that bias can be traced to MUCH earlier than hiring decisions, and don't mean to suggest we should only focus on bias in work outcomes...far from it. If we can address, for example, gendering if of educational preferences (e.g. girls like reading and boys like math) and instead just let kids gravitate towards what they enjoy, that would take a big step in the right direction.
I fully understand the concept of training bias and the importance of being aware of it and avoiding it. However, one should not assume that, since the human resume reviewers, and then the AI, tended to pick men at a greater rate than women, that this represents a bias problem. Whatever those characteristics (other than gender specifically) were, they may have, indeed, selected the ideal candidates in the vast majority of cases. Those humans, and AIs, may have picked the most successful candidate again and again. So, if you reject the result, just because women are selected at a lower a random population might suggest, you could be settling for a sub-optimal process instead. In fact, if you're not careful, you could continually tweak your process until you get to the point that you are semi-consciously introducing bias FOR women by rejecting any solution that doesn't select them at a proportionate rate.
I wish you had your sources
Based AI
Hahaha a biased robot. I just watched the whole video. I don't think you're wrong about the AI being biased because at the end of the day, when looking for possible candidates for a job, one needs to be biased. What you didn't explain however, was how that was inherently bad. If the code of the system was trained to compare men and women, sure that would be bad. But the AI doesn't have that programmed, so you can't say it has gender bias. You even explained in the infographic that it's only bias was "good" and "bad" after examining characteristics. Maybe the video should have been called "AI reflects candidate preferences in male dominated industries"
he DID explain that though in the video. if the data is biased then the AI will be biased as well.
What..... gender wasn't biased cos they didn't think of gender in the algorithm. It wasn't in the big black box. Plus you assume hr managers and the humans who vet the resumes are sexist.
Thank you for your comment. I'd appreciate it if you would watch the entire video before complaining about the content. Happy to have a discussion then.
@@DataDemystified Aight but u did say that just because it's related to like chess its sexist, lets say it was a science degree and it happened that more men had it would that be sexist? And it goes both ways it could favour women who have those qualifications isnt that sexist by your logic? And you said it doesn't have anything to do with women. What did you mean by "If they look like this(women) then they are denied" that's ofc inherently sexist but you said it wasn't inherently sexist. Plus you said it was from a bad training set, and the bad ofc came from a sexist human. If they are not sexist then how did it become bad?
@@DataDemystified I watched the whole video
@@pepsiman4418First, thank you for watching the entire video. I appreciate you taking the time to do that. So let's say you have an HR manager that loves chess and thinks that chess is the best predictor of success. They also have absolutely nothing against women and would gladly hire a qualified women with absolutely zero bias of any kind. But, because they like chess, they decide to recruit exclusively from chess clubs at universities. They aren't saying "I don't want women", rather they are just saying "I want chess players," But, as it happens the overwhelming majority of chess players are men. So, when this HR manager, without any intent to harm women, exclusively hires from chess clubs, you wind up with an overwhelmingly male workforce. And that has nothing to do with women being unqualified...if the HR manager looked more broadly, they would find plenty of qualified women...but they choose not to. They choose to just look at chess players. So now we train an algorithm on this HR managers choices, which favor chess players, who are mostly male. The algorithm isn't told to exclude women, but what winds up happening is that the algorithm "learns" to prefer chess players. Then you feed the algorithm resumes from a whole host of sources (not just chess clubs), but the algorithm is already trained to prefer chess players, regardless of where the new resumes come from. So, it flags chess players to be interviewed. Those chess players, as we already said, are mostly male, thus the algorithm has perpetuated the (unintended) bias of the original HR manager. Now, I'm obviously exaggerating in that HR managers aren't just going to recruit from chess clubs, but what you DO find is that the default place to look for employees is wherever you came from yourself. And since most businesses have been male dominated for ages, the people making hiring decisions, disproportionately, are male. That means they tend to look in places where they feel comfortable looking (chess clubs or otherwise) and thus the outcome is biased, even if they don't intend the bias. I hope that helps explain things.
Here's an article that discusses exactly how this played out when Amazon tried to implement AI to screen resumes: becominghuman.ai/amazons-sexist-ai-recruiting-tool-how-did-it-go-so-wrong-e3d14816d98e?gi=24b4e1830b4c
@@DataDemystified I mean if the qualifications they want are male dominated so? Why would they purposely make their workforce worse just for "equality". It wouldn't even be equality cos by purposely choosing stuff that women have and not just the best qualifications that's sexist against men. Yeah the outcome seems biased and there would be less women, but there would be more people who could you know... actually do the job? And why would they choose something they are comfortable in instead of the qualifications necessary. If they chose the spaces that aren't exactly qualifications but where they are comfortable that's human error not sexism. Maybe they want more people like them to be friends with. I think that's fine cos they aren't choosing specifically because it's male dominated. They jus want more friends :(.