AI-generated transcript

 

00:06

Fixed with the introduction and that saves me a lot of time to introduce myself and tell who I am I do I am a data scientist for more than ten twenty years building disorder systems at my day job at Commonwealth Bank of Australia We do I do manage a number of teams and these teams do a lot of work that's very important to our customers And that includes many things including something that we open sourcing today together with HDL that AI Just very very exciting is happening literally right now Which is our model for detection of abusing transaction descriptions? 

 

00:44

And this is allowing any institution any financial institution in the world to actually use our model to kickstart the journey to Preventing abusing transactions using the combank model. I'm not here to talk about them. 

 

00:57

I'm not talking about CBA at all This work is literally The work I've done on my spare time at home So let me walk you through this and I'm not quite sure how I'm gonna change the slides I'm guessing someone else to do that for me If you can go to the next slide, please Thank you, so what I'm gonna walk you through today is Literally how to build a young email bot that will respond to email I'm kind of on a journey to automate myself and there's a bunch of tasks. 

 

01:33

I'm trying to do that one of them is emails So I'm gonna walk you through Luis bot and how then how to create your own LLM with your emails Then going through how do you actually enable this on Gmail and so on and how you actually achieve profit at the end? 

 

01:50

Right, so let's go to the next one So Luis bot my email bot is waiting for you. So I Expect everyone in the audience send me an email to this email address. It will be replied automatically And if it doesn't in a few minutes, I'll make sure it happens after the conversation. 

 

02:11

The reason is that I have two systems working. One that actually decides whether a message is worth replying. So it means that if it didn't reply automatically, it thinks it's like a marketing message or something that is not worth replying. 

 

02:25

If it does reply automatically, it went through. Otherwise, in the end, I'll do make sure that I send these two systems. One, they always replies, but it's not an automate. And the other one, they're always replies automatically. 

 

02:37

So send your message during the talk, and maybe we'll share the replies later. If you go to the next slide. So this is an open source project that I made on my GitHub. So you can go there, you can clone it, and then do everything that we're going to run through here and do your own email bot. 

 

02:58

And I can see some messages coming by. It was amazing. Next slide. So let's go and try to create our own LLM. Let's do it. The next one, please. So the first thing you need to do is actually get your emails. 

 

03:13

You need an email corpus to train your own LLM on your emails. So the easiest thing to do this is to use Google Takeout. So you obviously go to Takeout and, oh, now we go. My video is going to start very soon. 

 

03:30

There I am. Thank you. Now you can see me. And I'm also even automating myself out of my eyesight. So I'm using NVIDIA to make sure that I'm looking at the camera. So you use Google Takeout, get all your emails. 

 

03:49

You can actually do some selections there, but ideally you get everything. And if you go to the next slide, then you have a prop inside the LLM me. You have a file there that's called convert inbox to CSV. 

 

04:09

So you run that, which means that you need to take your inbox file. You need to tell what is your email address, what CSV you're going to do. And one of the latest addition I did was actually to make sure that you only take emails off to a certain data. 

 

04:25

Notice my previous bots would take information that were 10, 20 years old. And that was an ideal. So you can select that. You then have a CSV file, which looks like this, which the message fool is your prompt. 

 

04:44

And your reply is actually the autocomplete from LLM. And that's why you're fine tuning. And it's an example of my wife saying, oh, have you paid for this thing? And then to say, oh, I updated the details yesterday. 

 

04:55

It's like pretty much this kind of chain of information. On the next slide, please. you Next slide. Now having the CSV file, you can use LNM Studio and I'm pretty sure I've seen Sanjeev's presentation before talk about LNM Studio. 

 

05:16

Amazing tool by H .Dot .ai. Very simple, add your data set there, and then create experiments. I use a Facebook 2 .7 billion parameters for this. I tried the 125 million parameters, didn't really work that well, but the 2 .7 worked really well. 

 

05:38

We don't need bigger models on that for our email. It's really overkill for that simple task of emails. Go for the next one. After you train your model, what do you want to do? You want to test it. There's a file there, LNM reply which also guides the replies of the bot. 

 

05:57

But if you run it, you then you have a Gradle app that you literally can work. through the responses. So you just pretend to be an email, receive an email. Here in this case, it's not, it is how do you reply? 

 

06:12

So if you wanna see how our reply goes, you have to add the re in the subject and add the context, which is a message. And then you can see how this would reply. He is like, I'm pretending Tom Cruise asked me to participate in one of his movies. 

 

06:27

And I can see the reply there and the chain of like some other made up messages that the bot made is very, very, very interesting. So yes, I'll do that. If you go to the next slide, I tested with the Louis bot and kind of shun like said to Tom Cruise, I don't have time for Tom Cruise, which is kind of hilarious and then sort of like, go into the next, next slide. 

 

06:53

So now let's enable the bot and API. So go one more slide please. What you need to do after cloning, after training your model, after testing, you go to the Google API, you need to enable the API. You need to then enable, do the things that follow here, right? 

 

07:15

Or off, then you go to the next slide and, and you have, sorry, take so off of cat on my screen. Then you need to get your credentials. So here you created credentials to authentication credentials. And you see on the right here that you have that file, you download that JSON file called credentials. 

 

07:40

So you add the credentials to your folder. Then when you execute the LNM bot, you ask you to click a link, which then you're gonna click on that link and then you give permission to the app to read your email messages and communicate with your email messages like send messages as well. 

 

08:01

Now the one thing that you can do is going to happen is every week that gets refreshed. I need to figure out ways to work this better, but currently I think every week you need to do a refresh of that authentication. 

 

08:16

Not the credentials themselves, but I click on the link and make sure that it's authenticated. On the next slide, please. So the next step now is to configure this file, config .py. Very simple. You have to get your model name there. 

 

08:34

You can see the model has, there's Facebook opt 2 .7b. That will not work. That's just a placeholder there. What I do, I add my own models on HuggingFace. So I push my models to HuggingFace in the private environment. 

 

08:49

And that means that every single computer that I work and I can actually pull the models from there. And you just, add your name, add information you have here, the templates you want in your response. 

 

09:00

And some of you might have seen some of my responses already. And that's pretty much it, right? So you run, oh, sorry, just one detail. I have this running normally on my Mac and you can actually add to the Chrome tab just as you see it below there. 

 

09:18

Or you can add a parameter that you check every certain number of minutes and currently it's checking every minute. But this is the way you wanna decide. Like if you wanna run in a Mac, you probably run on Chrome. 

 

09:32

So it doesn't consume any power, it just doesn't consume much memory and so on. Next slide. So yeah, what can go wrong, right? And I'm pretty sure a lot of you saw some of my responses and thought, yeah, a lot can go wrong. 

 

09:49

But let's have a look at some examples. Next slide, please. So here's some examples, a friend of mine sent me a message to test the system. How are you? Are you liking your job? Are you liking living in Sydney? 

 

10:02

It's a pretty good response, right? Enjoy my work, staying in Sydney. I might move back to Brazil. I'm Brazilian in my background. And it is mentioned that I'm considered moving to Hong Kong. To be honest, I'm not. 

 

10:16

I'm not moving to Hong Kong. I'm not moving to Brazil. But it's a really good response. I actually lived in Hong Kong for a little bit and that has that context somehow. Next slide. This is an early one that I wrote, literally my first version of the system. 

 

10:33

I co -organized the ACM, recommend the system conferences this year. And this first email is so spot on that amazes me. We were deciding options about the program of the conference, how it was going to be. 

 

10:47

And this answer is so right, it's kind of scary. The second one is interesting because I'm as organizing an industry chair with just just in from Netflix. This was not a, it was a good response for the program, not necessarily that good. 

 

11:06

If you notice there, there's like barely anything to do in this program. But the good thing that it identifies at 12 30 if you go for drinks and I just love that response. Go ahead, next slide, please. 

 

11:19

This is an interesting one because I've received most of my messages are no messages that need to reply to. And this was the government in New South Wales where I leave, send a message about new active kids and creative kids, which I do have kids. 

 

11:33

And hence that's really pertinent message. But the bot fell that I needed to respond to, can you send, please send the voucher to me by email, which is not necessarily what you do, but it's a perfect response. 

 

11:45

And nevertheless, now the interesting thing about this is that because my message are mostly like this and everyone's message, you probably don't need to respond. One addition I had to the LLM recently, and that's the reason why some of you I'm not probably getting the responses, is I try to identify what message we're responding. 

 

12:04

So on the CSV file, you see things like do not respond, which means that the bot notice that the completion of the LLM is do not respond, is simply do not respond, which then makes my life is not too much read in the draft emails on the inbox. 

 

12:21

Next line. Yeah, this is another example that I shouldn't respond, but I just thought it was interesting to add here that Google send me articles of people, and then I, and you might have seen in my response that sometimes I'm still a PhD student in Macquarie or some university that make up, and it makes up a paper. 

 

12:41

So there's a little bit of hallucination here, but nevertheless some responses are pretty good. Next one, and I think, yeah, now I wonder how did my bot go? How does Luis bot go? I'll just have a very quick check here and some answers. 

 

13:01

Yeah, if you didn't, I can see some that were just testing and hence the bot refused to respond. There's some here that, one of my favorite topics is the pizza. This one did not decide to respond, maybe it's a secret, and what's my stance on pineapple? 

 

13:19

I'm gonna, their response wasn't very good. I will hit reply very soon. Let me see if I find one that the automated responded. Oh, so Jessica asked me my favorite color, and there's a lot of weird answers here, but somehow it picked up there, you from H2O, and it's talking about water. 

 

13:44

Anyhow, it's not perfect, but you probably see some good things and some bad things there. I hope you're able to clone my project, work out and your own LLM, and I'm not quite sure, I don't think we have time for answers, questions, but thank you so much for having me, and I hope you have fun with Luis bot.