What Type of Bias Happens With AI?

Many look to artificial intelligence (AI) as a way to reduce the amount of human bias that goes into decision-making. However, since AI is designed and implemented by humans, the potential for bias still exists.

What are the main sources of biases in machine learning?

Three sources of biases in AI or machine learning (ML) have been identified, and while sometimes called by slightly different names, they are essentially: algorithmic, data, and human or user. When biases occur within the design and implementation of an algorithm, it is referred to as algorithmic bias. This means that something in the way the algorithm was coded or in the way it is analyzing data is resulting in bias. Because AI and ML rely heavily on data to analyze, learn from, and predict human behaviors when making decisions, it is essential that the data used is free of bias. Data bias occurs when the data being used to train AI is biased or unrepresentative of the entire population. The third source of AI/ML bias is human bias, also referred to as user bias. This kind of bias happens when user generated data teaches AI to be biased based on the way that people interact with it. Whether it is done deliberately or accidentally, users alter the output of the AI based on the information they are putting into it. If the information going in is heavily biased, the resulting output will be as well.

Because of the risk of bias and how easily it can creep into all different levels of AI development and implementation, it is important for organizations to examine the AI they are using and consider the AI bias questions they should be asking to help eliminate bias and discrimination as much as possible. Some of these questions can include:

• Who is designing the algorithms, and do they take responsibility for how their work is used?
• Who is leading the effort to identify AI bias?
• Is the data set comprehensive and representative of multiple sources of data?
• How can we test for bias, and how much resources should be allocated to assess potential bias?

What is selection bias in AI?

One common type of data bias is selection bias. Selection bias occurs when the data used to train AI is either unrepresentative of a population as a whole or isn’t selected with appropriate randomization. This is especially prevalent in situations where developers are focused on solving a specific problem without taking into consideration how the solution will be used on a broader scale. 

What is an example of selection bias in AI?

There are various examples of AI bias in healthcare that have occurred as a result of selection bias. A healthcare algorithm used by many US hospitals only used data from healthcare cost history to determine patients’ level of care. Because black people historically spent less on healthcare treatments than white people, black patients were inaccurately assigned the same level of risk as healthier white patients. A meta-analysis of AI neuroimaging models that are used for psychiatric diagnosis revealed some alarming AI bias statistics. Researchers found that 83% of the 555 AI models had a high risk of bias, in large part due to the inadequate sample sizes used by 72% and the insufficient handling of data complexity in 99% of the AI models. AI bias in healthcare can be especially dangerous due to misdiagnosis or insufficient care, so it is imperative that organizations work to mitigate AI bias.