Keynote Session by Commbank Team with Dan Jermyn
Dan Jermyn goes over the successful partnership with CommBank and H2O
Talking Points:
Speakers:
Dan Jermyn, Chief Decision Scientist, Commonwealth Bank
Read the Full Transcript
Dan Jermyn:
So let's talk about personalization and what it means from a CommBank perspective in terms of the way that we serve our customers. Now, you heard Matt talking about the retail customer base that we have, also the business, and the institutional. I'm aware that there are people who are joining us today, maybe not aware of the context of CBA being huge within the Australian market. And really continually adding ways to serve our customers across many different propositions. With our venture incubator X15. With the way that we think about our shopping ecosystem. We have thought about things like utility to providers and providing a lot of information to our customers that allows us to be able to service them in many ways that previously wouldn't have been thought about as a banking context.
We want to be helpful, but we're aware as well that that brings complexity. One of the key things that we wanted to do early on with the partnership with Sri and the folks from H2O was to think about recommendation engines. What do you think about all those things that we can provide to customers? How do you make it simpler? How do you make it easier to lower the burden to the customer about what it is that's the next best thing from all of the propositions that we have. The first place that we started was with shopping. I think the folks might have some slides on another deck that might be able to show in the future which has a little bit more articulation of the way that we are running things to date.
CommBank App
So while they're working on that, let me articulate in words. If you've ever used the CommBank app, you will find a place that's called the Rewards Hub. We have a few different shopping partners that we mentioned. Actually, the exec challenge that was referred to previously was a good example of this. You can see the folks bringing up the slides in real time here. I'll articulate verbally. CommBank rewards, one of our propositions whereby customers get cash back for spending with partners. We curate a series of offers that we think are relevant for them, and you can get cash back with your spend. Little Birdie, some of you may be aware of, that's a shopping hub that looks out across the entirety of all the available deals. More than a thousand sites.
It may be that somebody has a flash 50% offer on an electrical retailer or there may be a particular sale that's going on for a limited period of time. That's a different proposition for our customers. That's kind of. Without knowing specifically which offer from within our program they want, they may be something else out there that's available for them. Each of those is provided sequentially within the shopping experience that we have. You will get a list of the top curated CommBank reward offers and then a list of the top curated Little Birdie offers. We thought that's great, but why do we think about the customer as an individual? It may be that listing them sequentially like that is not the best way to treat the customer in terms of what's most relevant.
Recommendation Engine
That's where the recommendation engine came in. And I have to say we've talked quite a bit about the talent that we have access to with H2O and some fabulous brains. The data scientists within our team and some of them are in the audience today, have been sensational as well. Actually, one of the really great aspects of this partnership has been the common ground that the Kaggle Grand Masters and our own incredible talent to be able to understand each other, provide context of the bank, and be able to leverage some of the most advanced techniques possible. Really key for us that our own internal talent is right up there at that level too and so we did a trial. We created a recommendation engine, which uses three different models. One to think about the individual vendors each. So that's two.Then a merchant to merchant similarity model and combine those in a weighted way to see if we think about things simply from the customer point of view, what can we do? What does that show?
We went out to 300,000 customers and what we saw in the results was remarkable. For the Little Birdie shopping example, we saw a 44% uplift in the number of clicks through to the third party advertiser ecosystem. The third party sales that were being made available to customers. 44% uplift in engagement for our customers with those offers that were out there across a variety of retailers. CommBank Rewards, I think we score a 25% uplift in engagement with the top offers. At the very top through the rescoring that we had done. Each of those individually is great, but what this means is that by treating the customers as an individual and inverting the process to think about simply all of our partner ecosystems. What's best for the customer? You get a much better customer outcome, but you also get a much better merchant or business outcome.
This is very similar to what we did with the customer engagement engine. Which Matt and Andrew referred to earlier. When you think about things from the customer perspective, you can do incredible things that are just good for everybody. They talked also about the billions of data points that the customer engagement engine uses. I think it's worth saying that within the first year of scaling the recommendation engine work that we've done, there will be over a trillion data points being managed. That's just through one example really.
CommBank AI
I think the slides are coming back up now. You can see a huge amount of propositions here. Incredible for customers, but incredible complexity. CommBank AI is helping us to think about those things in a much more structured way. Here, you can see the AI models personalizing and ranking across all partners. For the pilot, we're just looking at two. The ecosystem itself has many different propositions for customers and we continue to grow. It's so important that we make it easier for our customers to do business with us and a really exciting way in which AI is helping us to bring this to life.
Abuse and Transaction Description
I wanted to move on to a slightly different example now of how we're helping our customers. This is the topic of abuse and transaction description. Here, I have to give a huge shout out to the customer advocacy team at CommBank. Who a couple of years ago identified a really insidious problem whereby you use payment. If you make payments to anybody, usually. Certainly it's true CommBank and I think most banks around the world these days, you can label the transaction. You can say pay Dan $10 for coffee. What that team found was unfortunately, a very small proportion of customers were using it to send abuse, and in some cases really terrible abuse. So we wanted to use data to try and help those customers. fIn the first instance, you think, "well, let's remove swearing."
Let's look for swear words and say, "we can't do that anymore which is a good thing to do." Unfortunately, the most insidious and the most terrible things that we were seeing through the way that some of our customers were using the system were not swear words at all. Things like I'm watching you or please don't leave me. Some of the worst things imaginable in the scripts that we saw on a very small level. But for those people, it was just an awful experience. What AI allowed us to do is to target those in a much more efficient way. By using network analysis or various other aspects of trying to understand human behavior rather than the text itself, we were able to find a much higher level of accuracy where those terrible edge cases will happen and therefore deal with them. Our great people in the front lines and some of our community liaison people are able to work with the most sensitive cases in a very considered and thoughtful way.
H2O Wave
This is actually, we think, the first such example anywhere in the world. Some of the banks, I think, are catching up with some of the swear filters. We wanted to make sure that this was available to everybody. This is where H2O comes in really, as well as the expertise. It's the ability to reach into other markets to be able to proliferate throughout the AI community. You heard Sri talk about the number of customers particularly in banking in other geographies. This is global AI leadership from Australia. What you're looking at here is a simplified version of an interface that the team from H2O, help us with on H2O Wave. Which makes it simpler to demonstrate what the possibilities are with this tooling. To put in front of the right people to make decisions about how to help our customers or how to help customers anywhere.
What Have We Done?
I wanted to give a little bit of a shout out to Australia. Actually, you can tell probably from the accent, it's not my country of birth, although I did become a citizen the other day. I'm very proud to say I've been here for five years and it's a country that punches massively above its weight. Australia gave the world the black box flight recorder, wifi. Some incredible inventions are happening in Australia. It's been a leader far outside of what you would expect of a country of its population. When the team was talking to me about, "well, what are you going to present to Dan? What's the story? It's about a year now since we made the investment." One of the challenges I had was there are just so many stories. So many things that we have done.
It was really hard to think about focus. We talked about fraud. I think Matt mentioned 25% uplift on cards not present fraud through one of the models that we detected against the incumbent. Home buying, for example. If you think about the way traditionally banks have thought about their credit books and the way that they manage lending. It's a process that generally takes a long time and involves thinking about learning over a long scale. Now in the current environment where rates are changing quickly. Where there may be other economic factors that mean that the model decays quickly with H2O, we're able to trial and experiment models. At the moment, we're doing it almost every week to update and become more accurate. Much better about the way that we manage our products and manage our interface with our customers through those two.
Perhaps my favorite is this idea of AI at scale safely and responsibly. One of the things that was absolutely paramount and we'll talk about it a little bit later. I think on the panel is how we manage AI ethics. And we talk about it in terms of responsibility. How do you make sure that you're safe? You're not introducing unintended bias. How do you make sure all of your models are explainable? Through the way that we've integrated H2O into our tech stack, we're producing scaled accessible tool length to precisely that purpose that sits within our well-established governance framework so we can safely proliferate solutions for our customers. This is from the customer engagement engine. Andrew mentioned it. I think somebody talked about the Harvard business school case study. Actually, I was lucky to be a guest in a couple of lectures on that within the last six months.
We were very proud about that solution. Some of our very best people are creating something that, again we think, in Australia is the best example of its type in the world. I like doing live demos and I like watching. I was very jealous to some extent of Sri when he was flipping around slides and shows. It's because he's got an answer for everything and there's something going on in every space. It creates a more, let's say, haphazard view for a presentation of this style. Which we curate slightly more closely from a CBA perspective. But the best I could think of is to show something that's incredibly up to date. And so in the last. This is literally within the last couple of days. Each of these things which I think mostly have been talked about have been worked on by some of the 170 people that we've already upskilled with the H2O platform. Specifically, of which, I think 70% had no formal data science training at all.
They wanted to think about things within the context of the cost of living, which is obviously very topical here in Australia, as I'm sure it is everywhere around the globe. So I thought about those conversations that we have about customers for whom savings and accounts are a good option. 102% uplift within the last couple of days for a model push live that goes to customers for whom that may be the most relevant product to think about. Benefits Finder and all of those phenomenal stats that we were talking about there. 125% app against what we had already done, which we are very proud about. Don't forget, within the last couple of days, the team, not data scientists. The team who manage some of the communications that we have with customers, have been able to uplift the product response for Benefits Finder. The number of customers finding that engaging by 125%.
Then Bill Sense finally. That idea about helping our customers to get in front of bill shock. What's happening as rates change, this becomes incredibly important. Particularly for up home buying customers. 168% uplift in the number of customers responding favorably to the next best conversations about this. This wasn't from our brilliant AI team, brilliant as they are. This is the power of democracy of AI throughout the business. These were people who care within the context of the economy right now. Australia, right now, "what's the best way we can help?" Within days introducing phenomenal AI capability that's producing an incredible outcome, not just for our customers, but for Australia as a whole. That's why we're so proud about being partnered with H2O, who can share a very similar purpose ethos. We're very proud about being able to produce incredibly complex and difficult technical AI capabilities, but above all else we're proud of our people and the way that we continue to strive to serve all of our customers and Australia as a nation. Thank you very much.