Blind trust in AI tools will entrench bias, add to misinformation and inequity
We live in a dangerous age of misinformation and disinformation – one that is often fuelled by dubious data and analysis, social media, lazy communication and increasingly, AI.
I’m feeling increasingly nervous about the number of smart people I’m seeing cheering on AI without considering how it works, how it is evolving and the limitations and implications of even seemingly innocuous apps.
AI can be time saving and exciting. But it exacerbates the bias that is everywhere around us, and must be used with caution when used to make sense of information or data about people or topics where nuance is key.
I’m not an academic, but Ruhi Khan from the London School of Economics is and I highly recommend reading her article about ChatGPT and gender bias and how it can harm women’s careers. Ms Kham asked ChatGPT to write a performance review for ‘John’ based on information she entered. She then repeated the exercise, replacing the name ‘John’ with ‘Jane’.
ChatGPT categorised John as ‘exceeds expectations’. It described John as a strong leader, a strategic thinker with strong financial management skills and recommended him for future leadership roles.
Meanwhile – despite identical information inputs – ChatGPT stated that Jane ‘meets expectations’. Jane was described as showing a willingness to learn and improve. It was stated that she could enhance her strategic thinking skills and approach to client engagement.
In a world where huge gender pay gaps are endemic, does this sound familiar? We need to be concerned.
AI amplifies bias
AI helps us solve problems we already know how to solve much faster. But it also amplifies the negative effects of things we are yet to get right, as we have seen here.
Some of you may argue that enormous effort is going in to addressing bias in AI. That may be so. But AI has been built and trained overwhelmingly by men, and a certain type of man at that who may have a vested interest in ensuring people like Jane know their station.
In Sydney, where I live, I see diversity. But we all know our world is often not inclusive, nor equitable. Even as a privileged woman with white skin I experience the impacts of the patriarchy every day.
I believe that AI can be weaponised, at an industrial scale, to maintain the dominant culture. It can be used to shape and control narratives and symbols so that others are marginalised, ignored, or even erased. In plain terms it can be used to hold people – like Jane – down.
So please be very, very careful about deferring to a machine to weigh up information for you when you are doing something important that will impact on others, such as their lives or careers. Complicated problems can become worse, and faster, if convenient AI-driven content replaces critical thinking and conscious choice.