LAS VEGAS–Does AI + FIs = the CFPB?
It potentially could, and the problem issue is bias in AI models, according to a discussion at the Money 20/20 conference here.
During a session with just that name—"Does AI + FIs = the CFPB? That was the name of a session at the Money,” three people with an expertise in AI models, data and regulation sought to offer some answers to that questions, along with advice on what financial institutions can do.
Sharing their insights were:
- Matt Wallaert, founder at BeSci (Behavioral Science) in La Jolla, Calif. Wallaert is an applied behavioral scientist
- Kathleen Yeh, director of product compliance at Galileo Financial Technologies in Sandy, Utah
- Yinglian Xie, CEO and co-founder at DataVisor in Mountain View, Calif.
“People expect AI to make no mistakes, said Wallaert. “But there is really a different definition: 'Better than humans.' Yes, it is biased, because it’s trained on human data. But what we need to ask ourselves is, it is better than humans? It should be enhancing what we do.
Wallaert: So, where is the bias?
Xie: It’s a very good question. Bias exists, we have to acknowledge that. When we switch to AI and machine learning, it’s a different methodology that’s data driven. So, you have to ask, is my data representative? If it is biased, whatever model you build is biased.
Several years ago researchers built model designed to see if it could tell the difference between a husky and a wolf. It worked, but what was discovered was the AI wasn’t differentiating between the animals, but instead whether the background had snow or not. The same thing can happen in certain populations that are under- or overrepresented in data. In those cases the model can be biased.
What Needs to be Determined
Wallaert: How do we determine if we have good coverage with the data?
Xie: We have to understand the distributions of the data and the elements we want to capture. When we apply AI, the number-one thing to check is the data comprehensiveness, whether the customer data we are analyzing is comprehensive and captures data in an unbiased way. We can do data and distribution analysis to discover bias.
Wallaert: Do those representations generally follow the representations we see in society, such as white males being overly represented?
Xie: It is very difficult in that case to say. Whatever data we collect, it’s inherently biased. Even with a large population, it is still a challenging task. We don’t have a model to overcome that , but recognizing it and being aware of the problem is the starting point.
Above, from left, Matt Wallaert, Kathleen Yeh, and Yinglian Xie at Money 20/20.
The Regulatory Risks
Wallaert: As we think about regulatory bias, awareness is key. How do you feel about regulation, where are risks?
Yeh: In the regulatory landscape at the federal level there isn’t a specific federal regulation that speaks to bias as we know it with AI tech. We have existing regulation and guidance from the CFPB, which is actively monitoring this. The guidance is ‘Continue to innovate, but do so responsibly. Be aware of the existing regulations and how you are utilizing and leveraging data.
The biggest risk right now is in decisioning and what you are looking for from an outcome perspective, that you’re not targeting, inadvertently, under-represented populations.
Wallaert: What should folks be doing from a compliance perspective? An AI ethics panel? What are some concrete steps?
Yeh: First, understand the data. We’re not going to eliminate bias, but there are ways to reduce it. Understand the algorithms and the decisioning model. Understand your risks; every entity’s tolerance is different. Understand what your controls are.
Wallaert: We know the AI we use in mortgage lending is more biased than zero, but less biased than humans making the same decision. So, what about human-based risks.
Yeh: I go back to understanding the data and the controls. From a mortgage-decisioning standpoint, are we decisioning loans in a way from a population standpoint where we are targeting a specific subset that should have some protections?
AI vs. Humans
Wallaert: Should there be A/B tests, humans vs. AI? What is best way to detect bias?
Xie: The key element is to understand the particular goal we want to achieve, there are specific elements the human knowledge can bring to mitigate the bias.
If we just take data into a model without any treatment, there could be data elements that should not be brought. For example, the time stamp. If we do not treat it specifically, you could build a model that (concludes) activities from a certain time to a certain time could be fraud. Fraud often happens in a batch, so you will see fraud spikes. You could mistakenly say, ‘These time segments are more likely to be fraudulent,’ and the model becomes biased. That is not a good way to detect fraud, your model is fragile. Attackers can easily figure that out and change the time of their attacks.
OK, But What Is It?
Wallaert: What is explainability and feature generation, and what it means to understand why the model is doing what it is doing?
Xie: Explainability plays a big role in understanding what the model is generating. It’s understanding how the model is identifying the wolf and the husky. Unfortunately, some of the very powerful machine learning models today, those behind ChatGPT, their explainability is not great—yet. There are other algorithms that provide better explainability.
You can augment these two approaches together to somewhat reveal from the purely feature engineering model, what are the top data points contributing to the model.
What About Regulation?
Wallaert: Should we be regulating individual model usage? Or things like explainability, which we can’t do with humans?
Yeh: Explainability will probably be the most integral part of looking at the decisioning capabilities. We’re looking to see what the end-result is from question of inadvertently or purposely targeting populations we shouldn’t target. Are we violating any consumer, civil liberties? Infringing on existing consumer compliance?
Wallaert: Beyond decisioning, where are other places you see AI being used and how do you view from compliance perspective?
Yeh: AI is being leveraged in a lot of different spaces. You are seeing that with banks leveraging the use of chatbots and simpler decisioning. How do I find my account balance? We’re also seeing the shift to more complex generative AI.
Where to Begin
Wallaert: Let’s say I am head of customer service and say, ‘I want to use Generative AI. How do I proceed?’
Yeh: What is it we’re trying to accomplish? Underwriting? Increased utilization of improved customer satisfaction? How are we using it so we can understand the risks? What controls can we put in place to reduce the levels of risk?
Also, listen to your customers. Do we have issues with how we are leveraging the data? Is the issue broader than we think? We do have the tools in our current arsenal.
Wallaert: What three things can be done to mitigate risks moving forward?
Yeh: Do not wait for (new) laws. Continue to innovate. I think we should take the CFPB guidance. They have been very vocal about saying, ‘Leverage the technology but do so in a responsible manner.’ Don’t go against existing regulations. Make sure you have the proper controls in place. And work with your regulators. If you are regulated, bring your regulators into the picture.