Keynote Session by CommBank with Andrew McMullan
Andrew McMullan goes over how CommBank became a customer obsessed and AI driven company with the help of H2O.
Talking Points:
Speakers:
Read the Full Transcript
Andrew McMullan:
Absolute privilege to be with you this morning. There's always a little bit of nervousness when you're following the chief executive officer of the organization. Tough act to follow. As Sri said, my name is Andrew McMullan. I'm the Chief Data and Analytics officer at the Commonwealth Bank of Australia. What that means is I've got the privilege every day of working with the data professionals across the group. Our job is to make sure that we're building the right data platforms and capabilities to be able to look after that data. To use it to make better decisions on behalf of the businesses that we support. As well as improve the experiences that our customers get as a result of using that data on their behalf.
What I want to do for the next 20 minutes or so is talk a little bit more about our aspiration to build a customer obsessed, AI driven organization. I'm going to share a little bit more detail on some of the examples of things that we're already doing as an organization to use machine learning and AI to provide better experiences for the customers and the communities that we support.
Customer Obsessed AI Driven Organization
One of the first things that I wanted to start on was when you're thinking about building a customer obsessed or AI driven organization, I'm sure there's many aspects of things that need to be true or need to come together to allow organizations to do that. I think that I'm probably, for many of you who go to events or conferences like this, you'll get lots of leaders who stand up and they say things like, we are a customer obsessed, customer driven organization. I think for a lot of organizations that is true. But I think there are, for me, four things that are unbelievably critical for an organization to be customer obsessed and to be able to use their data to drive the experience for customers.
Culture and Purpose
The first and probably most important is the culture and purpose of an organization. Rather than have me tell you that CBA is the best organization in the world and coming to work every day to make a difference for our customers. I thought I would share an example of Harvard Business School. If you go to Harvard Business School to do an MBA, the use case that they teach is how the Commonwealth Bank of Australia built a customer engagement engine to serve and look after our customers. If you read the use case, it's not as much about the science or the technology, it was all about the culture and the leadership of the organization. As Matt mentioned it, when we started on the journey five or six years ago, we had. Well, firstly, we had the privilege of Matt's leadership who led the customer relationship banking program and the transformation that the organization went on.
We had all of the leaders of the organization and all the component parts come together. Every day we were committed to working, to building something that understood and provided better experiences for our customers. What that meant was the marketing team, the analysts, the scientists, the technologists, the digital team, the branch team, the call centers, every part of the organization came together with a common purpose to use in our data to create better experiences for our customers. That's fundamentally why I think we've had a lot of the success we've had.
Leadership
Then as part of that, the leadership of the organization. The leadership awareness and understanding of what we need to do and what we need to invest in to be able to be the best in the world at this. As a simple example of this, I mean the expression the team and I use sometimes is nothing ever happens by accident.
Our partnership with H2O didn't happen by accident or the investments that we've made in becoming a customer obsessed, AI driven organization didn't happen by accident. About two years ago, myself with Matt and some of the executive leadership team, we were on board. The management and board challenged us, the data community, to go away and come back, and tell the board and the management team what we needed to do to be an AI driven organization. When you've got the top of the organization from the executive leadership team on the board, demanding and driving the organization to be better at building and using data to serve our customers. Then you've got the capability that is beneath that to rise to the challenge and take that opportunity to push the organization and get the investment and talent needed to be able to do that. Then you stand a real chance.
I think it's a privilege at CBA to have both of those things to be true. Matt was very kind to mention that the capability that we have at CBA and the data team is some of the best on the planet. I would agree with that. I have a privilege of working with in my career the most capable and talented group of data professionals that I've ever had the privilege of working with and supported by the leadership. Understanding and awareness and support means that we're able to do some of the great things that we've been able to work on over the last few years.
You Can't Do It On Your Own
The final part of this, you can't do it all on your own. You need to make sure. We talked a little bit about the evolution of technology and how fast sciences are evolving over the last 5, 6, 7,10 years. I was kindly introduced as a doctor.
I have a PhD from over 20 years ago in machine learning, as it turns out. Dan German who's going to be on the stage in a little bit. The team often ridicules me and says something like, "it took me four years, a little bit more, to do my PhD using H2O." Now they reckon they could replicate that in under 15 minutes. It just makes that like a real good feeling of investment of four years to do something that the team can now do in less than 15 minutes.
What Makes H2O the Best?
The point is that the evolution of computers and science means that you just need to be staying up to date with what's possible. When we started testing H2O, Sri and I caught up at the weekend and we were joking about this. Which was about four years ago when we were piloting H2O. When we were using it for some use cases. We were obviously undertaking some due diligence to make sure we were partnering with the best product on the planet.
We tested H2O against multiple other providers who did similar things. When we were looking at the results on the use cases, the team and I were a little bit skeptical because the results seemed a little bit too good to be true. We were inspecting what was going on, trying to work out how the performance of the models using H2O was so much better. Not only than what we were doing previously, but against everything else that was apparently the top of its field globally. But H2O was surpassing everything that we were seeing. Then it became clear to me that the reason, having spent time with the company and as we inspected what they were doing, a lot of it was because of the grandmasters. Actually as Sri talked about earlier, I think like a little bit of context in Grandmasters, if you think about chess, I think there's over 2000 grandmasters in chess on the planet today.
So that's the best people who play chess in the world. In a world of data science, Kaggle Grandmasters, there are less than 300. I think it's about 260, and the number varies. But between 260-300 Kaggle Grandmasters on the planet today. H2O has over 30 of them. Of the top 30, I think there's probably about 15 or 20 that work with H2O. When we inspected what the platform was doing and what the technology was doing, it was because they had the best people in the world building the product. That meant the models that were in it were better than anything else we were testing it against. For us to be able to build the global best customer obsessed, AI driven company, we had to partner with someone like H2O. Who would be able to help us improve every decision that we make for our customers.
How Do You Democratize?
We make hundreds of millions of decisions every day. Simple as some of the things that Matt described, which was for every transaction we make a decision on that. Is this transaction safe for our customers? Is someone trying to fraud or scams? Every one of those is a decision point and we want to make those decisions better for our customers. The other things that Sri and Matt touched upon, which I'll double click on a little bit is, "how do you democratize," if that word makes sense to everyone? How do you create an organization where in every part of the organization people are able to use the most advanced machine learning techniques and capability to make better decisions for their business unit or for their customers and communities that they serve? The way that we've tried to do that, actually the H2O product, I should add, when we tested it and we tried to understand its capability against our data scientists, what we ended up equating was when we partner with H2O and when we have a part of the organization who used one of the licenses. An H2O license of using driverless machine learning capability is the equivalent to 10 data scientists.
So if anyone and anyone is here, you just take that away as one of the key outputs of this talk. Because as we were thinking about how we became the global leaders on being an AI-driven organization, we realized that we weren't going to be able to do it by just hiring all of them. There aren't enough data scientists in Australia, frankly, to be able to do all of the things that we want to do. To the level that we want to be able to operate at. We had to find a partner like H2O where that capability means you can do so much more and have so much more predictive power in the capability for the decisions that you make. Using something like H2O as opposed to having hundreds or even thousands of people who are hand coding models and trying to test those models.
Executive AI Challenge
Alright. What I wanted to do now was. I wanted to share a little bit more of the executive story that Sri touched on and Matt mentioned. We are part of a concept which we call data week. Every year we have a week where as an organization we create a lot of energy and understanding around what the organization is doing in the data space. We have demonstrations of some of the technologies or the strategy. As a part of that, this year we did an executive AI challenge. What we wanted to demonstrate here was that through the simplification of how H2O have built the product, executives in the organization would be able to build AI models and deploy them for a specific use case. Actually, in this particular example, we got the grip executive of marketing and corporate affairs to face off against the grip executive of the chief operating office and then against the machine to see who would be able to build the most predictive model to help customers in a campaign that we had. Which was to support one of our shopping partners.
We had a little bit of a cashback offer. We pitched the three of them against each other. I'm going to show you a little video and then we'll talk a bit about the results.
Andrew McMullan:
So over to Mon. Mon, you've seen how it's done.
Monique Macleod:
I have.
Andrew McMullan:
You've seen the pressure.
Monique Macleod:
I have, it's building.
Andrew McMullan:
You've had time to think about that. But it's time now to make your choice. Let's start with the accuracy. What are we thinking?
Monique Macleod:
I think we're going to increase the accuracy.
Andrew McMullan:
The competitive streak is coming out. Let's make this thing as accurate as we possibly can. Remember, no matter what happens, it's going to be very accurate. We're dealing with the fine edges here. So we're increasing the number of ensembles that's going on with the various models that get produced. So over to you for time.
Monique Macleod:
Time. I'm going to decrease.
Andrew McMullan:
There we go. I like it. I like your style. How far are you going to decrease?
Monique Macleod:
I'm going to go down to three.
Andrew McMullan:
You're going to go all the way down to one?
Monique Macleod:
No.
Andrew McMullan:
Not going to go? I can't persuade you?
Monique Macleod:
No.
Andrew McMullen:
Determined. Very good. Three it is. And so interpretability, what are we thinking?
Monique Macleod:
I'm going to drop that down as well.
Andrew McMullan:
Ah, this could be the difference, ladies and gentlemen. She said, turning it down, the interpretability. Tell me, what was it about CV targeting code in the feature engineering space that particularly attracted you, Mon? It's one of your favorites, right? One of your favorite feature engineering?
Monique Macleod:
Four is my lucky number.
Andrew McMullan:
Yeah. Very good. What we've done here is look at a much more sophisticated feature engineering search space. Something. Using some of the most complex deep learning techniques in the world. Which Mon herself has now been able to implement. It's very impressive. Okay, so if you're done, if you're happy. Let's launch this thing
Monique Macleod:
As much as I'll ever be. Let's lock and load.
Andrew McMullen:
And it's live. So we're off ladies and gentlemen, looking back at Sinead. 3% complete. Not too bad. See, despite your attempts to, let's say, test this thing to the limits. We're still well on our way here. We'll get a scale model in probably about 20 minutes or so. Mon just started. She won't be far behind you. What we're going to do now. Generate those models. Push them through the NBC process. Find the customers who we think are going to score really highly now for this NBC to give them a great experience. Target them for that brilliant chair offer and we're going to see what happens. See how our customers resonate with it. Track it through the week. At the end of it, one of you will be crowned the CBA Grandmaster of Grandmasters. Look forward to seeing how it goes. Thank you very much for your participation. Let's see how it goes.
Monique Macleod:
Wonderful. Thank you.
Results of the Executive AI Challenge
Andrew McMullan:
And now for the results and as you think about the power of this anyway. Maybe it feels a little bit gimmicky where you've got executives, but the ability to be able to build the most advanced models using no code drag and drop. Just by having someone who's beside you explaining the decisions that you're making is an incredible opportunity for organizations across the world. Each of those models, before our partnership, would probably have taken us three months to build end to end. Mon and Sinead. Were building a model that we would deploy live into the production system for our customers that day. It's a pretty incredible transformation. Let's have a look at the results. We had a control group and an existing campaign that was live. Now have a look at how Sinead's model did. Using the H2O capability, she managed to get an uplift of 350%. That was 350% better for customers engaging with and taking out the offer that was presented to them.
How did Mon do? 500% better and the reason that this is so inspirational for us as an organization. If you think about Sinead, who was a good sport, but her career was much more like she's doing operations and she's had other experiences. Mon had a head start, right? She's a marketing professional. A career in marketing. The combination of someone in the business who knows what the data would be that they need to interrogate and use to get something like this live to market. That business experience with science is an incredible opportunity in every part of the organization. When you can connect those two things together. That for us is what we mean when we talk about the democratization of AI and machine learning. We want to create the ability for every part of the organization to use the capability we're building with H2O to make every single decision that they make. Either on behalf of their business or on behalf of our customers and communities make every single decision better.
For those of you who are either with H2O or a little bit more on the tech side, how did both the executives do against the very best that the model and the platform could do? It only just beat Mon? So it was pretty close to having our executive using her experience and the capability to get a marketing model to market that performed incredibly well.
What Are We Doing With Machine Learning and AI?
Okay, I'm going to finish by taking you through some examples of things that we're already doing with machine learning and AI capability to create better experiences and services for our customers. There's a few examples which I'll flip up. First, we talked about fraud and scams. Again, under Matt's incredible leadership, it isn't a week that goes past that Matt and the executive leadership team aren't demanding that we partner with the fraud team. That we do more, that they expect more for us to be able to do a better job of making sure that we're looking after our customers and we're protecting them.
I'm sure everyone in the room is familiar over the last 12. It feels like in the last 12 or 24 months the number of messages that you might personally get from people pretending to be. For me, the one I get most seems to be to pay for my tools. I'm sure many of you're getting to see them and a lot of our customers, maybe not as savvy as some of you in the room. They're trying to engage with those.
Bill Sense
We want to do more and more every single day so that every decision we make on every transaction is better for our customers. We've also created a feature in our digital channels, which we call Bill Sense. This is really important for our customers cause what it does is every single day it's going over billions and billions of bills and payments to help our customers understand what bills are coming up both today, in the next month, in the next three months, and the next 12 months.
What that does is it helps our customers just plan for the year ahead a little bit more. There isn't a period where there's shock. Particularly, if you think about what's happening in energy prices, that's allowing our customers to be much better prepared for the things that are going to come out of their account this month, next month, and the next six months. So that they can make sure that they're having enough money to pay all of the bills that are coming.
Benefit Finder
We talked about well, sorry. Sri and Matt mentioned Benefit Finder. It's one of the favorite examples we've got where there are over 320 different benefits, rebates, and business refunds that are available to customers and businesses across Australia. We've delivered more than $500 million worth of benefits back into the accounts of our customers. But I wanted to share one story with you on Benefit Finder. The team and I heard from our branch colleagues about an elderly customer who went in and she was sitting down with a member of the branch team and explaining that as a result of one of the natural disasters, she'd. Her fridge freezer wasn't working.
As part of that conversation, she was working out. She was like, "I think I need your help. I can't survive without a fridge freezer. I don't know how anyone can kind of live in today's society without a fridge freezer." The member of staff understanding Benefit Finder was able to help the customer by saying, "did you know that in New South Wales, if you buy a more energy efficient fridge freezer, you would get a 50% discount if you're in a particular category?" Also in the conversation using Benefit Finder, they were able to identify someone claimed money that the customer could get. Through that conversation experience, the member of the branch team actually helped the customer be able to get a new fridge freezer at no cost to them.
The story goes, I think a few days later, the customer was going past the branch and she'd been out to buy some shopping of stuff that she was going to put in the fridge freezer and she popped into the branch to show everybody how excited she was that they had been able to help her with that particular purchase and experience at that time. Those are the moments that you can create for millions and millions of customers across Australia when you're a customer obsessed, AI driven organization.
Natural Disaster Support
The other example that I'll share is the natural disaster support. This I mean, everyone's familiar with the natural disasters that we get in Australia. It was a few years ago when the bush fires were raging across the country and the team were working every single day to try and understand who was impacted by that.
Some days it would take a few days before we understood the impact in Western Australia, New South Wales, wherever the bush fires were. The team was working really hard to try and understand what regions were impacted, what customers were impacted, what those customers had with us, how we could help them. I remember I got a call on a weekend where I remember the team was like, "Look, we really want to be able to do more to help customers in natural disaster situations. That means we don't have to wait until it happens and then we have to try and find them." What we did was we looked across all of Australia to identify any weather system data that we can get access to and we built a weather model. I was just wondering if we could turn that on so that if there's ever a natural disaster we can actually get to our customers before it hits them.
To which I remember I called Angus Sullivan, who runs the retail bank to say "Angus the team have just called, said they don't on the weekend while they were at home to help our customers. They've created a weather data model that will help us get to our customers even before the natural disasters with personalized messages to let them know exactly how we can help. And they wanted to know if we should switch it on." And he obviously laughed and said, "I don't think there's another organization in the world today. They probably A can do it or B has the care for its people to just want to be able to build that in their spare time so that we can help our customers when there's a moment that really matters for them." Anyone who's been in a flawed or a bushfire situation or a natural disaster event, you will know that at that moment in time. Having your bank there right in front of you, telling you exactly what you can get and how we can help you is a really powerful thing.