H2O WORLD: INDIA 2023
Fireside Chat with Andy Markus, AT&T & Sri Ambati, H2O.ai
Watch now
Video Synopsis:
Speakers:
Read the Full Transcript
Sri Ambati:
It's a joy to have you here, Andy and I have probably expressed how highly your deep integration of data science and AI and math and start at large into business. I think you could probably talk about your journey and how you came to that. I would say one of the top data analytics leaders in the world. How did you figure that out, that connecting the business questions and bringing value business probably start off your journey at the time and as well.
Andy Markus:
It's a great question, and I just wanted to say to everybody, like, this is my first trip to India, and I'm so impressed. We're truly doubling down on great data engineering and great data science resources here in India for AT&T and I've been amazed at how our team has grown and the capabilities that we're seeing, and it's fantastic.
The question tree really is around value for business. That's at the forefront of everything we do. We always say we're not doing technology for technology's sake. We're doing it to drive value to our customers and to our business. That's the top of mind thing that we always check that box. Now, we always talk about let's not recreate a commodity. Let's don't chase technology for technology's sake. Let's do things that create true value for our company and our business. I'll wrap it into our journey a little bit at AT&T with the chief data officer. When I was brought in, I've been with the company for about two and a half, three years now.
My boss basically said, you have three big rocks you have to nail. Number one, create value for the company. Number two, data AI and AT&T one of the world's largest telecoms, it's such an incredible asset, but it really wasn't the way we were operating at AT&T. Make it a great asset for the company. Number three, was to modernize our environment. I saw with Commonwealth Bank, you guys have been around for a long time. AT&T been around for a long time, 148 years. Sometimes I feel like I encounter a system that's 148 years old. Not true, but I have seen things from the 1980s. But we are modernizing that. We started in earnest two, two and a half years ago, and we are almost to the end of that journey, kind of to a really great point.
But with the modernization it's really looking at our data and AI tech stack and making sure that we're cloud first, we're cloud native. That we can do things more effectively and more efficiently, again driving value to the business than we could in the old environments. That's what we're doing it for. But by the end of this journey, we will be at least 35% more efficient than we were in the environment that I've moved into. That will really drive tens of millions of dollars on an annual basis to the company, given the scale of AT&T, that's number one. Number two, it's really about making data and having trust in it. Creating common architecture that all of AT&T can leverage and should leverage to do the data AI world.
In your chat example, it said 300,000 employees, we've downsized a bit. We're about 175,000 employees, but still, that's a massive group of people. How do you get 175,000 people to march to the same beat of the drummer? It's very hard. But by will and also by offering really first class technologies like H2O to our customers across the business we've seen an incredible adoption of common technology, and we're doing that again, to drive business value. If we can communicate across the business in the same language, how much more effective can we be? That's allowed us to democratize and truly federate data and AI across the company. Because you think about it like, we're data scientists, we're data engineers. We want our subject matter experts in business to be with us in our journey, and they are now.
One last thing real quick. You asked about value, let's tie that in. Last year, our CEO asked us to do a state of the AI evaluation across AT&T. In that evaluation, what we found is that we were driving about two and half billion dollars of value with AI on an annualized basis across AT&T, and that's really impressive. I think the other great thing about it, it ties into the other two things, is that we found about 50% of that value came from our team, and this is a great thing. About 50% of that value came from other parts of business. That shows the value, that we are democratizing this out that we have the competency across the business, and I think that's only the beginning. Things are becoming exponential in terms of what we can achieve.
But we are rolling out generative AI as fast as we can. I think, next time we do that evaluation, which will be here in another month, I think we'll top 300 billion dollars of value.
Making Data a First Class Asset
Sri Ambati:
Every time I meet, Andy and his team work, and I get a new artifact of how much money was saved. How much every week there's a new use case or dozens of use cases, not just efficiency, but it's revenue as well. What does that mean to have many, many customers who would love to know how to make data as a first class asset? What does that mean from your point of view?
Andy Markus:
I think it really means that AT&T sits on such massive data. I think we see 500 and I think 90 petabytes of data per day, and it's growing like 60% of our volume is video. We as consumers, we're just consuming more and more video traffic and that curve is this. We consume a ton of data today. How do we take that 590 petabytes today and make it useful for the business? That's what the CDO team is really great at, and that's how I think we lead the charge for the north star for AT&T in helping the business understand how to do that. Making sense of our data really maximizes the reuse of our data, if we do things twice, we're just wasting money. Let's find the right way to do it. Doesn't matter where we do it in the business, find the right way to do it. All revolve around that, all the docs that and I think it's really then using that data to add value to our customers. To make the experience at AT&T better, to provide better coverage, to provide better customer service, to provide better just interaction. That's what we're here for, so when we say make AT&T data a first class asset, it's all about making it consumable for our citizen data scientists and our professional data scientists to drive value for our customers.
Overcoming Conflicts between Technology and Data Business
Sri Ambati:
If we roll back even a few years ago, what conflicts that emerged between technology and data business. How did you resolve that and that led to this?
Andy Markus:
That's a really good question. I think this will resonate with the room. Anybody that's in technology that wants to own their own technology. Our technologists, sometimes we want to, Hey, let's try this, or let's do this. Sometimes though, it's not a democracy. Sometimes we have to say, we're going to adopt truly the best tools. We're not going to be in an environment where we take our sacrifice there, but we also say that we can test and try things, but we need to be common in terms of architecture.
That's maybe the biggest challenge because as technologies, we all kind of want to be individualist. But in terms of making sure data is a first class asset, we do have to speak in that common language across the company. We have to be able to exchange data and AI in a very efficient way. It did take a certain adoption of architecture. I think the way we were able to deliver that is making sure we had tools that were better than the tools that were in house. Everybody's happy to go to a new party if that's a better party. That's what we serve up with a better party.
Evolution of Data Science and AI Centers of Excellence
Sri Ambati:
That's amazing. Let's talk a bit on the 50% of the value not coming from the data science teams. We've seen this the evolution of data science and AI centers of excellence. We will first centralize, then they're decentralized. Could you double click on how AT&T achieved that?
Andy Markus:
I mean, through my career, I've worked in both worlds. A centralized world and a decentralized world. For a smaller company, a centralized world can work. Where you have fewer customers, fewer clients, fewer use cases. At the scale of AT&T, it's not sustainable. I mean, no matter what a person wants to do, it's not sustainable, it can't drive the value of the business. If you truly democratize it, and you democratize it in a way that everybody is using those best in class tools. It's really about how we maximize the value of our data for the company. To me, at AT&T it was never a choice to think about centralizing this because it's truly not sustainable to execute the use case.
It's like cloud computing, we're multithreading data and AI. How do we multi-thread that so that the experts across the business can do this to deliver their job. I mean, that's one of the core principles that we focused on is not just for the data science community, the professional data scientists, the code warriors. We have common practices of how we create AI with code all the way through the AI lifecycle of getting data, finding data, creating your algorithm, deploying your algorithm, monitoring it, and governing it. All code driven, but the vision that we have and have had is to create a low code, no code version of that so that not just the professional data scientists, but our citizen scientists can use that for their business.
So smart people in HR, those smart people in non-technical areas, marketing, finance. Again, they're technical, it's technical in different ways. They can use this legally, but they can use this to really drive value for the business. We did that through decentralization, through creating the tools, creating the techniques, creating the governance to allow those smart people to do data science, create AI, not just use AI, but to create AI in a way that they can deliver value to their company. They're part of the company, and not go through our team. We don't want to be the bottleneck, and in that position that we have, professional data scientist, and again, speaking to the room, someone that's skilled both in the academics, the theory, the practice of true data science. You guys are unicorns.
There aren't many of you out there, and there aren't many being created. We have to democratize that. We have to bring new talent. How do we do that, we do that by democratizing this by making the citizen data science capable of doing the responsible way. In our vision we think we can grow the people at AT&T that are creating AI by five x in three years, and that was pre Jitter AI. I'd love to talk about that too. We're doing some incredible things there.
Direction of Generative AI at AT&T
Sri Ambati:
That's a great segue. Make AI, not just consumers. I know you're working very closely with several of the leaders in the space as well. What ideas and what directions are you thinking of generative AI at AT&T, and I think we'll hear more of that later as well.
Andy Markus:
I'm sure everybody that you talk with, I mean, I loved your version of H2O, it was wonderful. What I would say about this is that when GPT three came out in November, it was an amazing step function change in language analytics. Both the foundational model and the super app write up ChatGPT around it. My view of this is we think about things that have changed business. You can kind of go back and you say, what iPhones are a really big event. And yes, the iPhone created business opportunities and it created the consumer experience, and it created for AT&T a great business opportunity to sell smartphones.
But I think you have to go back, honestly, to the eighties and the nineties to find something as disruptive in business, how we operate as a business, as what Generally AI will be soon. What I mean by that, and I lived it, so back when I had hair you know, I lived this. The advent of bringing the personal computer into the workplace and not living off of a monolithic mainframe, of bringing in the killer app of a word processor or a spreadsheet. This changed the way people worked across businesses across the globe. Instead of having armies of accountants You were quickly able to get things done faster and easier instead of having somebody to help with delivering your communication or creating your communications, you're able to do it yourself.
The internet is part of this too. But General AI is going to change everything we do. As a leadership team, I'm asking our team to watch and read these disruptive periods of technology. We have a great one that we just watched called Triumph of the Nerds. Not revenge for the Nerds, but Triumph of the Nerds. If you haven't seen it, watch it. But there's so much we can learn of how Generative AI is going to change the way we work today. What I mean by that, and it goes to that, basically bringing the citizen data scientists forward. Is this going to have more impact for non-technical areas, groups, we think about these non-technical areas as technical areas?
Yeah, we'll be able to code with it, but think about the legal profession. Think about the finance profession, think about the HR profession. This is going to bring AI into their every day, every minute workspace. It's going to allow us to be so much more productive and efficient in ways we haven't thought of before. I'll tell you at AT&T, it became really important for us to create our own environment. We talk about OpenAI. It's only one of the Generative AI services. We have an orchestration layer on top of it, but we leverage the best in class Generative AI for the given use case. In some cases it's OpenAI. In some cases it's, maybe NVIDIA, in some cases it may be other tools. Love to plug in H2O to them.
It's really delivering value for the company that it's protecting our IP. That's the most important thing too, is that everybody saw what happened with Samsung two weeks ago. Two weeks ago, Samsung was using OpenAI in a way that their IP wasn't protected, and their IP became public domain. In our installation of this. Our IP is protected, it's safe and secure, but we're also making it smart with AT&T knowledge. We're doing it in a way, like you said, the prompt engineering skills we've become so fast, so far in how to prompt it, how to vet it, and hallucination rates are really minimal. It's a game changer for AT&T.
AT&T Contribution to India
Sri Ambati:
Protecting your data, protecting your models is producing, IT, and the brand. We think that our vision for democratizing GPTs is to make sure our customers own their code and own their models and and improve them. Make them more transparent for themselves and for their communities. As long as our vision is executed, there'll be a democratization space. I'm super excited, and I think this is the tip multimodal right there on the corner. Speech based large language models. AT&T is the largest data for speech. Being safe to use it to improve your customer in other places, super clean partnership. What brings you to India? Obviously, you're so privileged to have you here at H2O World, and what is your vision for AT&T historically thought of as an American telegraph company? What is exciting for work as well?
Andy Markus:
It's another good question. I mentioned that data scientists or unicorns, India has a lot of unicorns in data science. We want to have a real need for more professional data scientists. I think this is an incredible hotbed for data science talent, and that's a key reason that we are here. We love the community, the technology, the way that we are kind of thinking about our Indian based employees as being connected to our US based employees and it gives us more capacity I mean, we're so excited about what we're doing.
We're hiring like crazy, we're doubling our staff in India. Prince Paul Raj here in the front, Prince will talk in a second, but Prince is US based. Prince is one of our rock stars in the US and he'll talk about what we're doing with fraud and the sky net of AI models we have over our customers and our company that protect our customers and company from fraud. He's been great in that, in so many other different areas. Prince was the MongoDB developer of the year last year. We wanted Prince to come and really attract that rockstar superstar talent in India that we have in the states. Allow us to have access to this incredible talent pool that exists in India. We've been so pleased with the reception we've had here in Bengaluru and Hyderabad and everywhere else we visited, so it's super exciting.
Using AI for Good
Sri Ambati:
This is great. I think it's converging into a small village where we can take the best minds to solve the hardest problems that face humanity. I think that the last time we were hosted by AT&T in Dallas for H2O World, I posed the question to you on how you are using AI for good, and you gave a fantastic answer.
Andy Markus:
You showed one, that's a really great use case. AT&T has a really big commitment for the climate and for being good stewards of the planet and in so many different ways we're using AI to reduce our energy consumption. We're using AI to plan better to help others plan better for natural disasters. I think AI is not only foundational for how we run the company, but it's foundational for how we interact with our customers. Our customers are not just consumers. Our customers are businesses as well. We help them be good stewards as well, be more efficient as well using AI.
Advice for Data Scientist Getting Started
Sri Ambati:
Someone's just getting into becoming head of data science and addition sciences. What advice would you give them as you think about trying to transform a large organization?
Andy Markus:
It's another good question. I mean, I think you really have to understand the business, You have to understand what the business needs. A classic thing is this, and you can ask ChatGPT this because it will give you a great answer. What is the difference between business strategy and data strategy? So often strategy people, strategy teams trip over that, and what they'll say is Hey Andy, or Hey CEO, I want you to basically do this for the business. The real marriage Is the marriage of business strategy and data strategy. The data strategy and inclusiveness of AI is having the capabilities and the foresight to help the business, but the business sets the vision and the objectives and to me, that's the marriage. It is to make sure we're properly aligned with our business partners, that we are true partners because our business partners are so smart they know their subject area.
We just bring our knowledge of data and AI to the table, and then we share. So that partnership, I think, is what creates true value. That's the thing that I've always used in my career, is to really try to establish that partnership, that it's not always used and we do it with our team. We have one of the strongest words that we have, and when we say we, it's not just we as our team, but we as our partners, we as our company, we are in this together. We're not trying to take the credit, we're at the table to help them solve problems, and solving those problems will both benefit from that. I think partnership and having that app is super important in building that partnership. You need to make AI and data the true fabric of a company's ecosystem.
What an incredible culture that data culture and AI first culture that you've grown and built in such a large organization as well over the years. It's often lost on folks that AT&T where the Bell Labs is where Shannon came up with information theory Language C was invented.
Andy Markus:
Statistics were John, right now all this book, yes.
Sri Ambati:
One of the interesting things was that a good approximate answer to a good question is a fair bit more valuable than a natural answer to the wrong question.
Andy Markus:
It's a good quote.
What Have You Done While in India?
Sri Ambati:
We use that in our fundraising side. I was going to ask what fun stuff have you done over the first trip to India? I'm sure it is like a multi-dimensional force of signal. Anything fun you want to share with us?
Andy Markus:
I've been on a tour. Mark, Austin and Prince are here with me, they've been on the tour with me, but we flew into Mumbai. We spent some time in, we went to Delhi, we went to Agra, and then we went to Hyderabad and now we are here in Bengaluru. We've done a very quick tour of India and just the warmth of the people, the warmth of the hospitality has been fantastic everywhere we've gone. Seeing the special things that each city brings is very interesting. Of course, we've gotten to see the Taj Mahal, but other great attractions as well. But I think the people, right, the people have been the most fascinating and it's been the most welcoming thing that we had. It's been awesome.
Sri Ambati:
India is rich in humanity, a very rich nation in that scene where the people make the country really incredible. I'm glad and hope to see you more here and of course, see more back in Atlanta and Plano. But just a dream come true to see two of my favorite companies here together, and hopefully you'll spend some time with Commonwealth Bank as well and other customers. Thank you so much for being with us and sharing wisdom as always. This is a joy hosting you on my side. I hope to do more of these and in my view, one of the top chief data analytics officers or head of data analytics in AI, one of the strongest business leaders in the world. Thank you for coming to India and sharing.
Andy Markus:
Thanks, Sri, and thanks for having me. Thank you.