Building Data and ML Products at Chipotle with H2O Driverless AI by Jeremy Elster
Jeremy Elster goes over AI/ML use cases and projects within the company Chipotle.
Talking Points:
Speakers:
Jeremy Elster, Data Science Manager, Chipotle
Read the Full Transcript
Jeremy Elster:
Hello everybody. I'm here to talk to you about Chipotle Mexican Grill and our work on ML products and AI with the help of H2O. So I'll talk to you a little bit today about Chipotle, as well as some of our use cases, and our the organization that I'm a part of, Enterprise Analytics. A little bit about myself. My name is Jeremy Elster. I'm a data science manager at Chipotle. My team is responsible for driving demand forecasting models as well as working on analytics POCs across the industry. We use Driverless AI to power those key models and experiments. I'm fortunate to have experience, 10 plus years of experience in the analytics space, working on projects that I love: Soccer analytics startup, nonprofit activism, e-commerce company in Brazil allowing for travel, and burritos. When I was a senior in high school I was accepted to Berkeley. My friend and I planned a road trip, more or less just to try to see all of the taquerias we could find in the South Bay and Mission district of San Francisco. We then ranked all the burritos based on a selection of algorithms that we had tried to come up with, reviewed them, and put them in a blog that nobody read. I think, except for our mothers.
The Only Difficult Pronunciation at Chipotle is Chipotle
So I'm not the only person to fall in love with the mission style burrito. This is our founder Steve Ells at the first Chipotle restaurant in 1993. He was working as a chef in San Francisco before realizing his dream to open up a Chipotle near his hometown in Denver. And we think that he was successful for a number of reasons, but I'll talk about two. Number one, he became a leader in fast casual dining as well as Chipotle's commitment to serving food with integrity. And what that means, what we like to say at Chipotle, is that the only ingredient difficult to pronounce at Chipotle, is Chipotle. I don't know if that's still true after we released garlic guajillo steak, but I'll have to get back to you on that. Chipotle has over 3000 restaurants. Sometime this year we crossed that mark in the United States, Canada, the United Kingdom, France, and Germany, and we're the largest restaurant company of our size to own and operate all of our restaurants. Another exciting milestone that we recently hit was our 30 million rewards members enrolled in our loyalty program.
Cultivate a Better World Values
And at Chipotle, everything comes back to our main purpose: which is to cultivate a better world and our values guide us in everything that we do. The first value that we like to talk about is the line is the moment of truth. As a restaurant company, it's imperative that we deliver to our customers, the guest and the food that they expect when they want it, how they want it, with as much guac as you asked for. And so what that means is that every morning our food is prepped: real food, real ingredients, real people every day in the restaurant. And we think that that shows when you take a bite out of your chicken bowl, when our guest wins so do we.
The next piece is teach and taste Chipotle. So our mission as part of enterprise analytics, as part of our Data and Analytics Team and our restaurant support center, is how do we ensure that we have the people, process, and technology for our restaurants to succeed? Again, making sure that everything comes back to when you sit in line at Chipotle or you're ordering on the app. You get what you want when you want it. And so that's really our mission, is to help support. Whether you're in marketing, culinary app development, and also on the data science side, driving our AI and ML ideas.
Next, authenticity lives here. At Chipotle, we invest in our employees so that they can bring their full selves to the restaurant, to their work every day. That includes things like debt-free degrees, things like expansion of our employee assistance program for mental health and more. And this is part of Chipotle building a world-class and award-winning culture. So we were. We got a score from the Human Rights Campaign corporate equality index of a hundred percent as of 2021. And we take real pride in the type of culture that we build at Chipotle.
And then finally, the movement is real. So what that means for us is every year now, or every other year now, we put. We post a sustainability report and so we've set ambitious goals across environmental sustainability as well as our food and sourcing practices and also our commitment to our employees and people. And then we also hold ourselves accountable for that, including to the executive level of tying executive compensation to hitting those sustainability targets. Some examples that we're investing in include grants to young farmers and community resources, reducing the amount of waste that's going to landfills, and other sustainability and recycling efforts.
How Does Chipotle Data Architecture and Infrastructure Support Our Restaurants?
So how does Chipotle data architecture and infrastructure work to help support our restaurants? So our IT department, our data and analytics team has been hard at work, modernizing our data platform to go into Azure data platform with Snowflake as our data warehouse. And so some of these logos that you'll see are driven from our demand forecasting models. And I'll talk to you a little bit about some of the data sources. So as a restaurant company, NCR is where our point of sales data for sales and transactions, items, etc come in every day. Kronos, we use as our labor scheduler. DoorDash, if you've ordered Chipotle on the app, is a first party partner if you're using our app or web as well as a marketplace delivery partner. Planalytics is a weather vendor that we use in order to understand weather related driven demand. That is an input into our forecasting as well as PredictHQ, another vendor working on events data.
That data goes into our enterprise data warehouse via two different ways. First, Snowflake. So some of those partners I mentioned are able to now deliver data directly into our Snowflake environment through a Snowflake marketplace share. And our other vendors were using data factory copy jobs to put that data into generating ADLS Gen2 storage and then using Fivetran as our tool to pipe that into Snowflake. For our compute, we're using Azure Databricks. That's where we're connecting not only to our Snowflake environment as well as the other data we have across our enterprise and as well as pulling in other APIs, etc. And then of course, we use H2O. For the most part we're running with H2O through our H2O Driverless AI through the Puddle and Python API Client. And then finally, for visualization reporting, we're using a mix of Power BI. I have Snowflake dashboards up there, and then we're playing around with our first wave apps as well. And at the bottom here, I just have a few examples of how the data science and enterprise analytics team at Chipotle is managing our data pipelines as well as our production code.
Enterprise Analytics at Chipotle
So enterprise analytics at Chipotle. Here are a few of our partners that we work with regularly to deliver analytics and value our finance team, supply chain operations, guest experience, as well as marketing. So here's a few examples of the work that we've done. So as I mentioned, we run demand planning and forecasting at the restaurant level daily. We have, I'm sure, many of you working on forecasting needs. There's plenty of different options, models, needs, timeframes, etc. We have, we need, for example, for our supply chain folks, or sorry, for our labor scheduling folks, we need three weeks in advance. And then for intraday, our executive leadership is interested in how we are performing against the plan today in 10 minute increments. Also for demand planning, we're able to look at supply chain product purchasing and more.
When it comes to personalization and customer journey we worked with our H2O partners to help set up and build the models and the data sets necessary in order to deliver on customer lifetime in churn. This is really helping our marketing department understand the health of our loyalty program, as well as more understanding of the segmentation required to deliver more value to those customers. And finally, with H2O Driverless AI. Understanding the features and the combinations of features that are coming out of those models to really understand what's driving churn or a repurchase on the guest and customer experience side, we're getting comments daily. Perhaps you've sent in a handful yourself to the Chipotle team or interacted with our chatbot Pepper saying, "Hey, something wasn't right" or hopefully for the few of you who leave positive comments, for the businesses that you interact with, not just at Chipotle but elsewhere, something that makes that makes our team smile.
So we look and we do parsing on those comments to look not only for sentiment, but also for food safety/quality issues. As well as a handful of other factors. And we're always looking to improve our digital ordering experience. That includes operations and improvements in our allocation system for when you're able to order your food. And then finally, on the operations and process improvement, we're working with our teams to help optimize our labor schedule. To have the right number of people in the restaurants every day, prepping that food and then also making sure that they are able to prep and deliver the right amount of food such that when you go into a Chipotle and you ask for a chicken bowl, that chicken is available. It's well seasoned and it's fresh.
Demand Forecasting
So now talk a little bit more about the demand forecast use case and how we used H2O. So this one, this product, or project is about accurate restaurant level forecasts in the retail industry are needed to power all aspects of Chipotle's business. That means labor's adequately scheduled. Supply chain meets the demands exactly. Customers have that item when they want, when they want it. And also we're helping restaurants reduce food waste among many other benefits to the business. And so when we designed this project. So this project came out when I started, so talking about modern data architecture improvement. When I started at Chipotle, the task was to help improve or help assist the finance team in building out their annual plan. And before we had some fancy architecture that meant querying our on-prem SQL server query for years of historical data transactions, internal promotions, US and international holidays, etc.
That whole pipeline would take about 24 hours to run, right between just if you know, querying on-prem SQL server. You know what I mean? Hopefully you get to new and more powerful technology soon than I feel for you. And so not only that, not only are we able to do that now, in under, in hours or minutes and not a day for each iteration but we're able to really deliver the type of value that we need in order to use and incorporate these values, these data products into our business. And so, one of the great things about AI and I have two use cases. So this one is the one where we had more of, let's say, working with the finance department that's been playing around with their model for years.
A lot more input, a lot more structure on what the types of features that they had they wanted and were interested in using as opposed to what are the types of things that we're just going to let driverless AI run with. So some of those things that we played around with in the expert settings, for those of you on the data science side that are popping up the hood of your the expert settings on Driverless. So we played around with the lag based transformations. Again, with some of our forecasting models, we needed seven days. Some of them 21 days, gaps included. For our use case for forecasting we had a lot of input into how we wanted to build out our time groupings, right? So we're doing restaurant calendar date channels as our required groupings, but also what are the other features that could be added as time groups, but not necessarily needed to be in all of our features.
Things like is our restaurant based on our region or based on our market? What about our restaurants that are in mall locations versus end caps versus other retail locations? Another thing that we use with Driverless AI is including a number of custom recipes. Some of which were available open source in the GitHub repository. What happens when we add the profit library or profit models into our models? And what was the improvement that we were able to see with those? And then finally testing an array of scores to try to figure out how to best optimize for our use case across our restaurants. The results from this of using Driverless AI over this lengthy process for getting this model into production, into use, and trying to demonstrate that value to a team. Resistance to. With resistance to wanting to use an. Use the enterprise analytics version of things. We were able to cut variants on the restaurant level in half which was a huge performance improvement over a previous benchmark. And I think I was able to drive a lot of interest in not only the ability of enterprise analytics and the Data and Analytics Team at Chipotle but also the interest in H2O as well.
And so now we'll talk about a second model. So this one was used by our marketing department to try to understand, "Okay, we've just launched garlic guajillo steak." Who are our customers most interested in buying that product? If you have vegetarian parents like myself or you're a vegetarian yourself, why are you getting push notifications for a new steak option? You say, "Hey, I've been ordering a Chipotle for 10 times a year for five years and I've never ordered a steak option once." I didn't order the chicken. I'm just eating sofritas. Why am I getting push notifications? So that was the example that we were trying to solve. That the marketing folks as well as myself had put together a basic data set for the features that we had thought might be useful and had run that model to send off our segmentations for our marketing friends.
So then the idea was let's throw this into Driverless and see what actually happens. And as was described earlier in terms of the speed to market, within a few minutes, we were able to substitute that same existing data set into our Driverless AI instance and we got better accuracy in the same day without tuning. So about six and a half percent just throw it in. No effort needed. And not only that, we found all these new features that we hadn't even thought about. The final model included six times the number of features from our initial dataset, including novel combinations that we hadn't thought of across customers, across channels, across a number of different factors that we hadn't thought to put together. And so what's very important for us as a smaller data science and enterprise team is being able to have a team impact multiplier like H2O. Being able to drive this outsize impact relative to the number of headcount we've had we have.
AI Projects Chipotle is Investing in
And then finally, I'll talk to you a little bit about some of the projects that AI is or that Chipotle is investing in. In terms of the next generation of technology in restaurants. As well as some of the AI and ML capabilities that we're interested in with various levels of real world experimentation. So first I want to highlight Cultivate Next. So Cultivate next is Chipotle's 50 million dollar venture fund that intends to make early stage investments in strategically aligned companies that further its mission to cultivate a better world and help accelerate Chipotle's aggressive growth plans. Those are the words of our CTO Kurt Garner and by aggressive growth plans I had mentioned we're at 3000 restaurants. We have a goal of getting to 7,000 or more. And so what are the types of investments that are going to help? So on our left, the first example is Hyphen.
This is a company that has a fully automated digital make line or fully automated make line. Not only for digital orders that would be able to reliably and accurately fulfill your order every time. Helping reduce the problem at Chipotle where sometimes your bowl isn't made exactly to your standard. Especially as we're adding near unlimited customizations for your order. Or as our CEO likes to say, isn't it cool that we have rocket scientists working on Chipotle products? Next up, for those of you who are selling in California like myself and want to go eat tortilla chips made by a robot named Chippy. Chippy's actually live in one of our restaurants in Fountain Valley, California. And the idea for Chippy is how would we make an automated robot to be able to portion, fry as well as season our tortilla chips perfectly every time?
And really what's driving a lot of this automation at Chipotle is how can we take away some of the tasks that our crew with, through their feedback, really don't like to do so that we can also free up their time to focus more on providing that guest experience for those of us who go to Chipotle in person. Our third example is our partnership with a company called PresiTaste. And this is a kitchen management cook to needs tool. Using cameras, an array of sensors, they can look at our pan fill levels at the, on the frontline and the digital make line to say, "Okay, based on the amount of chicken you have currently in your pan, based on our expected demand forecasting for the next 10, 15, 20 minutes, this is how many. Not only whether or not sending a push notification to our grill operators when to cook, but also how much to cook. Do you anticipate needing one line of chicken, two lines of chicken, or more.
As well as for all of the other products, not only on the grill, although those do take longer to fulfill. And what the idea for this is, how do we ensure that there's fresh, high quality food for folks from open to close so that everybody can order and get exactly what they're looking for. And then finally, we have our investments in H2O Driverless AI. Again, as a small team working on a vast array of retail and other enterprise needs, H2O driverless AI is able to deliver the, for some use cases the full end-to-end experience necessary for some of those teams getting off the ground or is inputs into or is inputs. Like I mentioned, for propensity, a model that's able to understand more about our customers to help drive further insights, to help drive further growth for our business and for our company. So that's it for me. Thank you for listening to me talk to you about Chipotle and some of our AI/ML use cases.