Speaker 1: Hello, Hello, Welcome to Smart Talks with IBM, a podcast
Speaker 1: from Pushkin Industries, iHeartRadio and IBM. I'm Malcolm Babbo. This season,
Speaker 1: we're talking to new creators, the developers, data scientists, CTOs,
Speaker 1: and other visionaries who are creatively applying technology and business
Speaker 1: to drive change. Channeling their knowledge and expertise, they're developing
Speaker 1: more creative and effective solutions no matter the industry. Our
Speaker 1: guest today is Padre Bonidius, trust in AI practice leader
Speaker 1: within IBM Consulting. Advocating for artificial intelligence built and deployed
Speaker 1: responsibly is no longer just a compliance issue, but a
Speaker 1: business imperative. Part of Phader's job is to help companies
Speaker 1: identify potential risks and pitfalls way before any code is written.
Speaker 1: In today's show, you'll hear how Phader's team at IBM
Speaker 1: is approaching this challenge holistically and creatively. Phedre spoke with
Speaker 1: doctor Laurie Santos, host of the Pushkin podcast The Happiness Lab.
Speaker 1: Laurie is a professor of psychology at Yale University and
Speaker 1: an expert on human cognition and the cognitive biases that
Speaker 1: impede better choices. Now, let's get to the interview Phedro.
Speaker 1: I'm so excited that we get a chance to chat today.
Speaker 1: You know, just to start off, I'm wondering how did
Speaker 1: you get started in this role at IBM, Like, what's
Speaker 1: the story to how you got where you are today?
Speaker 1: Oh goodness. My background is actually from the world of
Speaker 1: video games for entertainment, So AI has always been very
Speaker 1: interesting to me, especially when you intersect AI and play.
Speaker 1: But several years ago I began to get very frustrated
Speaker 1: by what I was reading in the news with respect
Speaker 1: to malintent through the use of AI. And the more
Speaker 1: that I learned and the more that I studied about
Speaker 1: this space of AI and ethics, the more I recognize
Speaker 1: that even organizations that have the very very best of
Speaker 1: intentions could inadvertently cause potential harm. And so that's super cool.
Speaker 1: I love that your interest in more responsible AI came
Speaker 1: from the gaming world. You have to talk a little
Speaker 1: bit about your history with gaming and that how that
Speaker 1: informed your interest in trustworthy AI. Well, it wasn't as
Speaker 1: much necessarily the ethical components of AI when I was
Speaker 1: working in games. It was more things like, look at
Speaker 1: what non player characters can do? You know, I mean
Speaker 1: if you've got an AI acting as a character within
Speaker 1: the game, and how is it that you can use
Speaker 1: AI in order to make a game a more interesting experience. Actually,
Speaker 1: I ended up joining IBM to be our first global
Speaker 1: lead for something called serious games, which is when you
Speaker 1: use video games to do something other than just entertaining.
Speaker 1: And so the idea of integrating real data and real
Speaker 1: processes within sophisticated games powered by AI to solve complex problems.
Speaker 1: It wasn't until, as I mentioned, like later, when we
Speaker 1: started to hear all of us more and more news
Speaker 1: about just problems what could happen with respect to rendering
Speaker 1: or putting out models that are inaccurate or unfair. I
Speaker 1: know one of your inspirations for hearing other interviews that
Speaker 1: you've done is sci Fi. I'm also a sci Fi nerd,
Speaker 1: and I know sci Fi has talked a lot about,
Speaker 1: you know, the trustworthiness issues that come up when we're
Speaker 1: dealing with AI and so on, and so talk a
Speaker 1: little bit about how you bring that to your work
Speaker 1: in developing AI. That's a little bit more ethical. A
Speaker 1: lovely question. So my parents were major technofiles. They both
Speaker 1: were immigrants to the United States, came here to study
Speaker 1: engineering and they met in college. Growing up, my sister
Speaker 1: and I we had Star Trek playing every night. My
Speaker 1: parents were both big fans of Gene Roddenberry's vision of
Speaker 1: how technology could really be used to help better humankind,
Speaker 1: and that was the ethos that, of course, we grew
Speaker 1: up in. The wonderful thing about science fiction isn't that
Speaker 1: it predicts cars, for example, but that it predicts traffic jams,
Speaker 1: you know. And I think there's just so much we
Speaker 1: can learn from science fiction, or in fact, like I said,
Speaker 1: play as a mechanism to be able to teach science
Speaker 1: fiction predicting traffic jams. I love it. But when we
Speaker 1: think about AI and science fiction, we need to be careful.
Speaker 1: We need to remember that AI is not something that's
Speaker 1: going to enter our lives at some point in a
Speaker 1: distant future. AI is something that's all around us today.
Speaker 1: If you have a virtual assistant in your house, that's AI,
Speaker 1: your phone app that predicts traffic AI. When a streaming
Speaker 1: service recommends a movie, you've guessed it AI. Phaeder says.
Speaker 1: AI maybe behind the scenes determining the interest rate on
Speaker 1: your loan, or even whether or not you're the right
Speaker 1: candidate for that job you applied for. AI is both
Speaker 1: ubiquitous and invisible, which is why it is so crucial
Speaker 1: the companies learn how to build trustworthy AI. How do
Speaker 1: we do that? When thinking about what does it take
Speaker 1: to earn trust in something like an AI, there are
Speaker 1: fundamentally human centric questions to be asked, right like, what
Speaker 1: is the intent of this particular AI model? How accurate
Speaker 1: is that model? How fair is it? Is it explainable
Speaker 1: if it makes a decision that could directly affect my livelihood?
Speaker 1: Can I inquire what data did you use about me
Speaker 1: to make decision? Is it protecting my data? Is it robust?
Speaker 1: Is it protected against people who could trick it to
Speaker 1: disadvantage me over others? I mean, there's so many questions
Speaker 1: to be asked. Earning trust in something like AI is
Speaker 1: fundamentally not a technological challenge, but a socio technological challenge.
Speaker 1: It can't just be solved with a tool alone. What
Speaker 1: are the kinds of risks that companies have to think through?
Speaker 1: Is they're developing these technologies to make sure they're as
Speaker 1: trustworthy as possible, Well, you know, they may be putting
Speaker 1: a lot of money into investing in AI. That gets
Speaker 1: stuck in proof of concept, planned likes get stuck in pilot.
Speaker 1: We've done some research where we have found about eighty
Speaker 1: percent of investments in AI get stuck, and sometimes it's
Speaker 1: because the investment isn't tied directly to a business strategy,
Speaker 1: or more often than not, people simply don't trust the
Speaker 1: results of the AI model. As a company who is
Speaker 1: of course thinking about this so deeply what a businesses
Speaker 1: need to consider when they're trying to figure out how
Speaker 1: to solve this big puzzle of AI ethics. It has
Speaker 1: to be approached holistically, so you've got to be thinking about,
Speaker 1: for example, what culture is required within your organization in
Speaker 1: order to really be able to responsibly create AI, what
Speaker 1: processes are in place to make sure that you're being
Speaker 1: compliant and that your practitioners know what to do, and
Speaker 1: then of course AI engineering frameworks and tooling that can
Speaker 1: assist you on this journey. There is so much fundamentally
Speaker 1: to do. We found that actually those that were leading
Speaker 1: responsible AI trust were the AI initiatives within their organization
Speaker 1: has switched in the last three years. It used to
Speaker 1: be technical leaders, for example, chief data officer or someone
Speaker 1: who is a PhD in machine learning and now it's
Speaker 1: switched to be eighty percent of those leaders are now
Speaker 1: non technical business leaders maybe you know, chief compliance officer,
Speaker 1: chief diversely inclusivity officers, chief legal officer. So we're seeing
Speaker 1: a shift, and I believe firmly it's a recognition from
Speaker 1: organizations that are seeing that in order to really pull
Speaker 1: this off well, there has to be an investment than
Speaker 1: a focus in culture, in people and getting people to
Speaker 1: understand why they should care about this space. And so
Speaker 1: I see two challenges with doing that right. One is,
Speaker 1: you know a lot of these technology companies are really
Speaker 1: built to be tech companies, not necessarily you know, social
Speaker 1: tech companies or having this sort of training in ethics
Speaker 1: and beyond. Another issue seems to be that you're really
Speaker 1: proposing a switch that's truly holistic, right, that's like rethinking
Speaker 1: the way the company thinks about its bottom line. And
Speaker 1: so as you think about working through these kinds of
Speaker 1: challenges at IBM, how have you tackled this, like, how
Speaker 1: have you brought new talent in? How have you thought
Speaker 1: really carefully about this big holistic switch that needs to
Speaker 1: come to make AI more trustworthy. Data is an artifact
Speaker 1: of the human experience and if you start with that
Speaker 1: as your definition and then think about well, data is
Speaker 1: curated by data sideists. All data is biased, and so
Speaker 1: if you're not recognizing bias with eyes fully open, then
Speaker 1: ultimately you're calcifying systemic bias intosystems like AI. So some
Speaker 1: of the things that we've done and IBM again recognizing
Speaker 1: this important need for culture is big, big, big focus
Speaker 1: on diversity, not only looking at teams of data scientists
Speaker 1: and saying how many women are on this team, how
Speaker 1: many minorities are on this team, but also insisting on
Speaker 1: recognizing that we need to bring in people with different
Speaker 1: world views too, for example, what's your definition and of fairness?
Speaker 1: Is your definition equality? Or is an equity? Also bringing
Speaker 1: people with a wider variety of skill sets and roles,
Speaker 1: including our social scientists, anthropologist, sociologist, psychologist like yourself, right,
Speaker 1: behavioral scientists, designers. I mean we have one of the
Speaker 1: leading AI design practices in the world. I mean the effort,
Speaker 1: the investments we've been making in design thinking as a
Speaker 1: mechanism to create frameworks for systemic empathy well before any
Speaker 1: code is written, so people can think through how would
Speaker 1: you design in order to mitigate for any potential harm
Speaker 1: given not only the values of your organization, but what
Speaker 1: are the rights of individuals asking oneself? These kinds of
Speaker 1: questions reinforces than the idea that ethics doesn't come at
Speaker 1: the end, like it's some kind of quality assurance, like
Speaker 1: check I passed the audit, I'm good to go, you know.
Speaker 1: But instead, really, you know, as soon as you're thinking
Speaker 1: about using an AI for a particular use case, thinking about,
Speaker 1: you know, what is the intent of this model, what's
Speaker 1: the relationship we ultimately want to have with AI? And again,
Speaker 1: these are non technology questions. This is where social scientists.
Speaker 1: Having a social scientist on your team helping think through
Speaker 1: these kinds of questions is critical. Let's pause here for
Speaker 1: a second, because this is a really profound idea. Building
Speaker 1: responsible AI does not mean that you create a system
Speaker 1: then check in at the end and say is this okay?
Speaker 1: Is this ethical? If you don't ask those questions until
Speaker 1: the end of the process, you've already failed. You have
Speaker 1: to think about ethics from the jump from the makeup
Speaker 1: of the team to the data you're using to train
Speaker 1: the model, to the most basic question of all, is
Speaker 1: this even the right use case for artificial intelligence. The
Speaker 1: big lesson from IBM is this responsible AI is something
Speaker 1: you build at every step of the process. So this
Speaker 1: season of smart Talks is all focused on creativity and business.
Speaker 1: My guess is that thinking about trustworthy AI involves a
Speaker 1: lot of creativity. But talk to me about some of
Speaker 1: the spots where you see this work as being most creative.
Speaker 1: Oh goodness, I would say incorporating design design thinking in
Speaker 1: particular as well as straight up design in order to
Speaker 1: craft AI responsibly. You've used this word design thinking, and
Speaker 1: so I'm wondering exactly what you mean here. How do
Speaker 1: you define this idea of design thinking. Design thinking is
Speaker 1: a practice that we established here at IBM many years ago.
Speaker 1: In essence, what it is, it's a way of working
Speaker 1: with groups of people to co create a vision for something,
Speaker 1: for a product or service or an outcome. And typically
Speaker 1: it starts with things like, for example, empathy maps, like
Speaker 1: if you're thinking about an end user, thinking through what
Speaker 1: is this person thinking, seeing, hearing, feeling, like what are
Speaker 1: they experiencing in order to ultimately craft an experience for
Speaker 1: them that is targeted specifically for them. So we use
Speaker 1: it in a really wide variety of different ways with
Speaker 1: respect to trustworthy AI, even rendering an AI model explainable
Speaker 1: to a subject. And I'll give you an example. So
Speaker 1: we've got this wonderful program with an IBM call our
Speaker 1: Academy of Technology, and we take on initiatives that steer
Speaker 1: the company in innovative new directions. So we had an
Speaker 1: initiative where it was titled what the Titanic taught Us
Speaker 1: about Explainable AI? And the project was imagining if there
Speaker 1: was an AI model that could predict the likelihood of
Speaker 1: a passenger getting a life raft on the Titanic. And
Speaker 1: we broke up into two workstreams. One was the workstream
Speaker 1: full of the data scientists who were using all the
Speaker 1: different explainers to come up with the predictions and they
Speaker 1: would crank out the numbers. And the other team here's
Speaker 1: where the social scientists lived and the designers were right
Speaker 1: where we were thinking through how do we empower people? Well,
Speaker 1: how do we explain this algorithm and this predictor and
Speaker 1: the accuracy behind this prediction in such a way as
Speaker 1: to ultimately empower an end users. They could decide I'm
Speaker 1: not getting on that boat, or I want to get
Speaker 1: a second opinion please, or I want to contest the
Speaker 1: outputs of this model because I upgraded to first class
Speaker 1: just yesterday, see what I'm saying, And that takes a
Speaker 1: lot of creativity how you design and experience for someone
Speaker 1: in order to ultimately empower them. So design design, design
Speaker 1: is critically, critically important. And why I mentioned you know,
Speaker 1: we've got to open up the aperture with respect to
Speaker 1: who we invite to the table and these kinds of conversations.
Speaker 1: Taking the time to really understand other people's perspectives is
Speaker 1: so important when you're doing anything creative, and it is
Speaker 1: fundamental to the way the new creators work. The core
Speaker 1: question you should always be asking is where will the
Speaker 1: user be meeting this product? As Peder said, what will
Speaker 1: they be thinking, seeing, hearing, feeling. If you can answer
Speaker 1: those questions the way IBM does in its design thinking practice,
Speaker 1: you will be in great shape to create almost anything. Really,
Speaker 1: let's hear how it works in practice. And so we've
Speaker 1: been mostly talking kind of at the metal level about
Speaker 1: you know, how to think about AI ethics generally, but
Speaker 1: of course the way this probably occurs in the trenches
Speaker 1: as a client approaches IBM and they want to help
Speaker 1: with a specific problem in AI and so I'm wondering
Speaker 1: from a client based perspective, where do you start having
Speaker 1: some of these tough conversations. It has varied, To tell
Speaker 1: you the truth, we had one client that approached us
Speaker 1: to expand the use of an AI model to infer
Speaker 1: skill sets of their employees, but not just to infer
Speaker 1: their technical skills but also their soft foundational skills, meaning,
Speaker 1: let me use an AI to determine what kind of
Speaker 1: communicator you might be A Laurie right. Others might come
Speaker 1: to us with, Okay, we recognize we need help setting
Speaker 1: an AI efics board. Is this something you can assist
Speaker 1: us with? Or we have these values, we need to
Speaker 1: establish AI ethics principles and processes to help us ensure
Speaker 1: that we're compliant given regulations coming down the pike. Or
Speaker 1: you've had clients come to us saying, please train our
Speaker 1: people how to assess for unexpected patterns in an AI model,
Speaker 1: but then also how to holistically mitigate to prevent any
Speaker 1: potential harm. And those have been phenomenal engagements. They're huge
Speaker 1: learning moments. And so it seems like the additional value
Speaker 1: that IBM is bringing through this process isn't necessarily just
Speaker 1: providing an AI algorithm or consulting on same AI algorithm.
Speaker 1: It seems like the real value added is explaining how
Speaker 1: this design thinking works. You're almost like this therapist or
Speaker 1: like a really good bartender who talks to people, who
Speaker 1: talks about companies through some of their problems to try
Speaker 1: to figure out where they're going astray before they start
Speaker 1: implementing these things. Can I put chief bartender off and on.
Speaker 1: I like the metaphor, I'll tell you some of our
Speaker 1: most valuable people on the team for that engagement. We
Speaker 1: had an industrial organisational psychologist, we had an anthropologist. That's
Speaker 1: why I'm saying it's important that we bring in the
Speaker 1: social scientists because you're exactly right. It's more than just
Speaker 1: scrutinizing the algorithm in its state. You have to be
Speaker 1: thinking about how is it being used holistically? And So
Speaker 1: if I was a business that was trying to think
Speaker 1: about how a company like IBM could come in and
Speaker 1: help out with more trustworthy AI, what would this process
Speaker 1: really look like. Well, what we're finding more often than
Speaker 1: not is that they'll be smaller teams within broader organizations
Speaker 1: that either have the responsibility of compliance and see the
Speaker 1: writing on the wall, or they've been the ones investing
Speaker 1: in AI and are trying to figure out how to
Speaker 1: get the rest of the organization on board with respect
Speaker 1: to things like setting up an ethics board or establishing
Speaker 1: principles or things like that. So some things that we've
Speaker 1: done to help companies do this is we kick off
Speaker 1: engagements with what we called our AI for leaders workshops.
Speaker 1: On the one hand, it's teaching why you should care,
Speaker 1: but on the other hand, it's meant to get people
Speaker 1: so excited across the organization that they want to raise
Speaker 1: their hand and say, I want to represent this part, like,
Speaker 1: for example, I want to be part of the ethics
Speaker 1: board as it is being stood up. The heart parts,
Speaker 1: not the tech. The hard part is human behavior. And
Speaker 1: I know I'm preaching to the choir given your background,
Speaker 1: it's so nice as a psychologist to hear this. I'm
Speaker 1: like snapping my fingers, like preach exactly. The hard part
Speaker 1: is human behavior. So it's been like drinking from a
Speaker 1: fire hose. I mean in terms of the kinds of
Speaker 1: things that we've all been learning, and there's still so
Speaker 1: much to learn. It really bugs me that those who
Speaker 1: are lucky enough to be able to take classes in
Speaker 1: things like data ethics or AI ethics, self categorize as coders,
Speaker 1: machine learning scientists, or data scientists. If we're living in
Speaker 1: a world where AI is fundamentally being used to make
Speaker 1: decisions that could directly affect our livelihoods, we need to
Speaker 1: know more. We need to have more literacy, and also
Speaker 1: make sure that there is a consistent message of accessibility
Speaker 1: such that we are saying you don't just have to
Speaker 1: be interested in coding, like you're interested in social justice
Speaker 1: or psychology or anthropologies. There's a seat at the table
Speaker 1: for you here because we desperately need you. We desperately
Speaker 1: need that kind of skill set. Just getting people to
Speaker 1: think about how do you design something given an empathy
Speaker 1: lens to protect people? I mean that, I think is
Speaker 1: such a crucial skill to learn. You know, one thing
Speaker 1: I love about your approaches that when you're talking to clients,
Speaker 1: you're almost doing what I'm doing as a professor, where
Speaker 1: you're kind of instructing students, getting them to think in
Speaker 1: different ways. But I know from my field that I
Speaker 1: wind up learning as much from students as I think
Speaker 1: sometimes they learned from me. And so I'm wondering what
Speaker 1: you've learned in the process of helping so many businesses
Speaker 1: approach AI a little bit more ethically, Like have there
Speaker 1: been insights that you've gotten through your interaction with clients
Speaker 1: and the challenges they've been facing. I'm learning with every
Speaker 1: single interaction. For example, in my mind, given the experiences
Speaker 1: that IBM has had with respect to setting up our principles,
Speaker 1: our pillars ARII ethics board, there's a process to follow,
Speaker 1: right if you're thinking about it like a book, these
Speaker 1: are the chapters in order to optimize the approach, let's say.
Speaker 1: But sometimes we work with clients that say I'm going
Speaker 1: to install this tool and I want to jump to
Speaker 1: chapter seven, and it's like, oh, okay, you know, how
Speaker 1: do we help navigate clients that want to skip over
Speaker 1: steps that we think are important. Another one is again
Speaker 1: the social scientists and bringing them in to really push
Speaker 1: hard on what is the right context where this data
Speaker 1: tell me the origin story? Again like really pushing us
Speaker 1: to think hard and with their perspective, you don't know,
Speaker 1: just constant, constant learning, which is why one of the
Speaker 1: things we did at IBM is we've established something called
Speaker 1: our Center of Excellence, where we said, you know what
Speaker 1: ibm ors. We don't care what your background is, We
Speaker 1: don't care who you are. If you're interested in this space,
Speaker 1: you can become a member. The Center of Excellence is
Speaker 1: a way in which we have not only projects people
Speaker 1: can join in order to get real life experience, but
Speaker 1: then also share back. Here's what we learned. We did
Speaker 1: this with this particular Yet here was our epiphany, because
Speaker 1: if we're not sharing back and we're not constantly educating,
Speaker 1: then we're missing the opportunity to establish the right culture.
Speaker 1: Establishing the right culture to share what we're learning is
Speaker 1: so important, and so I wanted to end. But going
Speaker 1: back to where we started, you with your technofile family
Speaker 1: watching the Star Trek, I think if we were to
Speaker 1: fast forward a couple of decades, we probably couldn't have
Speaker 1: imagined that we'd be in the place with AI generally
Speaker 1: where we are now, and especially as we think through
Speaker 1: more trustworthy AI. And so you know, with such change
Speaker 1: happening right now, with the fact that it's a fire
Speaker 1: hose that's gonna just get even more powerful over time,
Speaker 1: what do you think is next in this world of
Speaker 1: thinking through more trustworthy AI. I would say next is
Speaker 1: far more education, far more understanding, and we're starting to
Speaker 1: see that shift. Far more CEO is saying, yeah, ethics
Speaker 1: has to be corrid or a business. There's but there's
Speaker 1: a shift. Barely half of the CEOs in twenty eighteen
Speaker 1: we're saying that AI ethics was key or important to
Speaker 1: their business, and now you're saying the great majority so education, education, education,
Speaker 1: And again I would underscore making it far more accessible
Speaker 1: to far more people, which means it's not just our
Speaker 1: classes in higher ed institutions, it's our conferences, it's anytime
Speaker 1: we write white papers, anytime we publish articles, anytime we
Speaker 1: do podcasts like this. Right, the way we talk about
Speaker 1: this space has to be far more accessible and open
Speaker 1: and inviting to people with different roles, different skill sets,
Speaker 1: different worldviews, because else again we're just codifying our own bias. Well, Phaedre,
Speaker 1: I want to express my gratitude today for making AI
Speaker 1: a little bit more accessible to everyone. This has been
Speaker 1: such a delightful conversation. Thank you so much for joining
Speaker 1: me for it. The pleasure was mine. Loie, thank you
Speaker 1: for being the consummate host. I want to close by
Speaker 1: going back to that moment when Lourie suggested that Phaedra
Speaker 1: was actually IBM's Chief Bartender Officer, not just because that's
Speaker 1: the best C suite title ever, but because it gets
Speaker 1: at what I think is the biggest, most important idea.
Speaker 1: In today's episode, Pedro boiled it down into a single
Speaker 1: line when she said, the hard part is not the tech,
Speaker 1: The hard part is human behavior. Why is building AI
Speaker 1: so complicated? Because people are complicated. IBM believes that building
Speaker 1: trust into AI from the start can lead to better outcomes,
Speaker 1: and that to build trustworthy AI, you don't just need
Speaker 1: to think like a computer scientist. You need to think
Speaker 1: like a psychologist, like an anthropologist. You need to understand people.
Speaker 1: Smart Talks of IBM is produced by Molly Sosha, Alexandra Garatin,
Speaker 1: Royston Preserve and Edith Russolo with Jacob Goldstein. We're edited
Speaker 1: by Jan Guerra. Our engineers are Jason Umbrel, Sarah Bruger
Speaker 1: and Ben Tolliday. Theme song by Gramoscope. Special thanks to
Speaker 1: Carlie Migliore, Andy Kelly, Kathy Callaghan and the eight Bar
Speaker 1: and IBM teams, as well as the Pushkin Marketing team.
Speaker 1: Smart Talks with IBM is a production of Pushkin Industries
Speaker 1: and iHeartMedia. To find more Pushkin podcasts, Listen on the
Speaker 1: iHeartRadio app, Apple Podcasts, or wherever you listen to podcasts.
Speaker 1: I'm Malcolm Gladwell. This is a paid advertisement from IBM.