// transcript — 389 segments
0:02 An open source project just dropped that when you combine it with claude code
0:05 literally becomes self-improving AI. This is not engagement farming or hype
0:09 bait. This is a real repo that was just released by Andre Karpathy who is widely
0:13 renowned as one of the foremost voices in AI and machine learning research. And
0:17 basically what he did was while training his model he thought why don't I just
0:21 have my models train my models instead. He built an elegant pipeline which he's
0:25 calling auto research and essentially completely and fully automates the
0:29 process of experimentation. He says it right here. The idea is to give an AI
0:34 agent a small but real LLM training setup and just let it experiment
0:38 autonomously overnight. It'll modify the code, train for 5 minutes, check if the
0:42 results improved, keep or discard, and then just repeat. You wake up in the
0:45 morning to a log of experiments and hopefully a better model. Now, I'm not
0:50 in machine learning training. I don't help make models more intelligent. What
0:54 I do is I take models that other people have made and then I use them for the
0:58 purposes of making money. And so immediately when this dropped, I started
1:01 thinking about ways that I could apply this principle of auto research into my
1:05 own life to improve obviously my own economic outcomes. And there are so many
1:08 it's not even funny. So what I'm going to do is I'm going to run through some
1:11 real practical examples in a moment, things that I'm actually doing in my own
1:14 business that you can implement inside of Cloud Code. Then I'm going to show
1:17 you how to actually do it. So go through the step by step of setting up the repo
1:21 and building some experimentation done totally autonomously for you. And at the
1:25 end you will have a fully automated self-improving pipeline just like Andre
1:29 Krypathy here has done for his own machine learning training. So here's one
1:32 of many examples. I do a lot of cold email in my own business and then for
1:36 clients. And cold email in case you didn't know is where you package up a
1:40 really nice sexy sounding offer and then you send it to people you've never met
1:42 with the hopes that they take you up on it and then maybe convert. So jump on a
1:46 call with you, fill out a form, whatever. Now the key metric in cold
1:49 email is usually reply rate. specifically positive reply rate, but
1:52 reply rate's easier for our purposes, so that's what I'm going to go with. And as
1:55 you can see, most cold email software tracks this for you out of the box. So
2:00 2.4% of people replied to this campaign. 2.5% of people replied to this campaign
2:05 and so on. Well, turns out that's all you need in order to build an automatic
2:08 experimentation pipeline. You need some metric that you want to improve and then
2:13 you need some way factor, some thing you can modify to improve it. And so what I
2:17 have is my metric is reply, right? And then what I have is the thing that I can
2:20 adjust is my cold email copy. So what this looks like in practice for me is a
2:25 folder called email optimizer. There are a bunch of additional code files here,
2:30 configs, places where I'm storing the results of my data and so on and so
2:33 forth that aren't super important. What is important is this file right over
2:37 here called orchestrator.py. And this contains all of the prompt that
2:41 I'm feeding into my orchestrator agent who essentially is responsible for
2:45 spinning up new cold email campaigns and then testing them against each other
2:48 until I get better and better results. And it was as simple for me as literally
2:51 copying the repo and making some slight adjustments. What I do is I tell it that
2:55 it's inspired by Carpathy's auto research pattern. The core idea is an AI
2:59 agent that runs experiments autonomously in a tight loop using an objective
3:03 metric as a feedback signal. I have the architecture over here. I run the loop
3:08 every 4 hours and at the end of every 4 hours I actually have better copy that's
3:13 self-evolving over time based on the results from my previous test. And you
3:16 can see some of the examples right over here. Anything with C is what we call a
3:20 challenger. Anything with B is what's called baseline. And so the model starts
3:25 with a baseline type of copy, which we could see right over here. And then it
3:30 makes slight modifications based off of what it knows to perform really well in
3:34 cold email copy before testing it out. It runs the two side by side and then
3:38 automatically harvests based off of the results, aka the number of replies and
3:41 so on and so forth for both campaigns. Now, that part isn't the important bit.
3:44 I mean, we've been optimizing cold emails for for many years at this point.
3:49 The important bit is it then creates new copy based off of the learnings from
3:54 previous experiments. As the models get better and better and better, they log
3:58 all of their learnings to a resource.m MD that significantly improves future
4:04 models abilities to make changes. And so here's a big list of things that this is
4:09 essentially figured out. Move reply rate up. And so in that way, we get to push
4:13 towards that direction over time. Now, this has only been running for a few
4:16 days now. Imagine this running for a year. And instead of optimizing on a
4:20 basis of once every 4 hours, imagine if this optimized on a basis of once every
4:24 5 minutes. Well, that's what my next leg of testing is going to do. We're going
4:26 to be significantly improving the volume, pumping all this stuff out at
4:31 10x the level, and then optimizing and iterating our results again fully
4:34 autonomously. So, that's just one example. I'm going to run you guys
4:37 through a bunch of other use cases that you can apply auto research to, whether
4:40 you're in machine learning engineering or whether you're just trying to improve
4:44 the profitability of, let's say, a paperclick ad campaign. But first, let's
4:47 make sure we all know how to actually use this thing. So the way that auto
4:52 research works to make a long story short is we start with an experiment and
4:56 just like in science everything begins with some sort of hypothesis. Okay. So
5:02 my hypothesis might be hey if I make a slight adjustment to the copy of this
5:06 campaign so that it's a little bit punchier. I think it's going to go well.
5:10 You insert that using this little test.md. It's your goal metric and some
5:15 highle instructions. From there, the auto research agent will go through
5:21 employ the experiment usually using API calls. In my case, in the example we
5:26 just saw um to instantly in Karpathy's specific example we saw, he's doing it
5:30 through adjusting what are called hyperparameters and then after that we
5:34 measure the results. Now, in order for us to make sure that this works, you
5:37 know, the hypothesis isn't enough. We need some sort of metric that we're
5:41 tracking. Now, in my case, the metric was obviously pretty simple. It was
5:44 reply rate. In Karpathy's case, it was pretty simple. It was something called
5:47 validation loss. As long as you have that, you can then just pick the winner
5:52 and then make a slight change before looping back. And depending on how tight
5:55 this feedback loop is, you could theoretically do this in a minute or
5:59 two. I mean, if he had more infrastructure when he's training his
6:02 models, he could probably do in 5 minutes what he does in one. And then in
6:05 that way, you know, progress really, really quickly over to some, you know,
6:09 desired goal. In my case, if I had more cold email infrastructure, I could do
6:12 the same thing. So, at this point, scale is more or less all you need. This
6:15 allows you to run hundreds of tests with literally zero human involvement. I
6:19 mean, I'm not even in the loop anymore. And to be clear, like, if I was in the
6:23 loop, would I be making better decisions than the AI model? Like, probably. I'd
6:27 be a much more efficient optimizer, but that doesn't really matter because the
6:31 reality is I take a lot more time to optimize than a model does. I also eat,
6:36 sleep, have to go to the washroom, and do a variety of other things with my
6:40 day. AI agents don't. you could very quickly and easily set this up on again
6:44 an hourly loop and have this run 24 times a day whereas realistically if you
6:48 were to try and do it all yourself you could only do it a couple times and so
6:51 in that way whatever metric that you're tracking goes up over time right in my
6:56 case reply rates significantly go up you know test one I might be at a 1.5% test
7:01 12 I might be at a 2.7% before you know it I reach literally like the optimal
7:06 quality possible for my set of cold emails and then their audiences and you
7:09 can apply this has mentioned to a bunch of other strategies. So what are those
7:13 strategies? The requirement that you need is anything that has an objective
7:20 metric you can track and an API or application programming interface that
7:26 you can send a request to to get. Okay, so some brief examples of this cold
7:32 email copy obviously fantastic. Why? Well, because in our case we have the
7:37 instantly API. The Instantly API allows us to query metrics and so I can give
7:41 the agent the ability to call a quick tool, call up the Instantly API, see how
7:45 the performance was relative to um, you know, the the challenger in the base
7:49 campaign. At the same time, you know, I have a very clear metric, which in my
7:54 case is reply rate. Okay, how about landing pages? Let's say you're doing
7:58 some form of CRO, which is conversion rate optimization, and you want to test
8:02 to see how you can make your landing page as efficient as humanly possible.
8:06 Well, you can now completely automate it with auto research. What you do is you
8:10 pick the metric that you want, which in our case would literally just be
8:15 conversion rate. Okay? And then if your website is hosted locally or it's hosted
8:18 using some API or something like that, let's say a website builder like Wix or
8:24 or or or WordPress or Web Flow, what you can do is you could give it access to
8:28 the API and then you could say, "Hey, change this according to this resource
8:32 of best practices that other agents have done. Make your change. test that for I
8:35 don't know a day depending on how much volume you have and at the end
8:39 consolidate the winner and then get rid of the loser. You can do the exact same
8:43 thing for ad creatives. Okay, what's the main thing that you want for ad
8:46 creatives? Obviously, it's going to be some form of conversion rate as well.
8:49 Whatever specific type of conversion rate is, that's up to you. Okay, but all
8:53 you need to do is query some sort of API. Now, you know, a lot of these ad
8:58 platforms like Facebook and Google basically already do this for you.
9:02 minded grant you granted I don't think they do it anywhere near as effectively
9:05 as you can with modern models like Opus 4.6 or GPT 5.4 for what you can do is
9:10 you could give it the API to call a specific ad resource and then you could
9:14 also just give it the metric to optimize for which is CVR and then it'll crush.
9:19 How about some form of customer satisfaction for chatbot scripts? Maybe
9:23 use some sort of customer satisfaction score and then you know now you just
9:28 adjust the main template, okay, that all customer service agents, whether human
9:33 or AI are are going off of. That's super simple and easy to do. How about product
9:37 descriptions for some sort of ecom if you have like, I don't know, Amazon FBA
9:41 or something like that? you know, maybe they don't necessarily have APIs, but
9:45 maybe now you set up um what's called Chrome DevTools MCP, give it a very
9:50 tightly scoped list of steps it has to do to update the actual body of the
9:55 landing page, and then based off of metrics like uh I don't know, how many
9:58 freaking dollars you've sold in the last little while, you can very quickly
10:02 optimize and make your product landing page better and better and better. You
10:04 know, in my case, I make a lot of YouTube content these days. I could do
10:07 this automatically with YouTube titles and the YouTube data analytics v3 API.
10:11 You know, you could optimize subject lines for your newsletters in the same
10:14 way. You could optimize pricing pages the same way. You could optimize
10:17 literally whatever you want. And so hopefully it's clear that at least for,
10:21 you know, uh most sales and marketing purposes, and we're not even going into
10:24 the back end here, Auto Research allows you to build a consolidated set of
10:29 knowledge on what works, what doesn't, and then have that running in the
10:32 background for you 24/7 with no human involvement. But how the heck does this
10:36 actually work? Well, three simple steps. The first is we're going to clone the
10:39 repo. I'll show you how to do that in a moment. We're then going to write some
10:42 sort of test. Okay? And you can call it tests that I'm you can call it whatever
10:45 you want. But this only needs to include a goal, a metric, and a test method. And
10:50 then you just give the agent the ability to run this on autopilot. In my case,
10:54 I'm using a service called GitHub actions, which allows me to store this
10:57 in the cloud and then run this on regular intervals like 4 hours. You can
11:02 use GitHub actions, you could use modal, you could use a billion other providers,
11:04 and I'll show you how to do all of that right now. So the first thing you need
11:07 to do is you just need to get the auto research repo. Now I have a link. it's
11:11 the uh top one or top two in the description. So, click on that. You'll
11:14 head over to this page. This will include all the information including
11:18 the Python training scripts, the project description, a bunch of other scripts,
11:23 and then what he's calling program.md where you provide the model everything
11:26 that it needs in order to manage this whole research process. So, you see in
11:30 his case, he says this is an experiment where you're going to do your own
11:33 research, work with the user to agree on this, create this, read that, verify
11:37 this exists, and so on and so forth. And I want you to know this stuff is not
11:39 super important. He's actually explicitly said that his prompt is
11:43 probably pretty crappy and that it'd be very easy to make a better one. So with
11:47 all that in mind, now we need to go over to our agent. Next, head over to an
11:51 integrated development environment or some sort of tool that allows you to run
11:54 clawed code. In my case, I'm using what's called anti-gravity. You guys
11:57 could use Visual Studio Code. You guys could use like a 100 different apps to
12:01 be honest. Um, by the way, if this seems like magic to you, you don't know what
12:03 any of the buttons on the page are. I literally run through all of it in an
12:07 extensive 4-hour cloud code course that even teaches you like what the different
12:10 icons are and so on and so forth. Really holds your hand through it. So just head
12:13 to the top uh right hand corner of the video for that. Anyway, assuming you
12:16 have all this stuff open, we're going to want to create a new folder. So I'm
12:21 going to go here to open folder and then I'm going to go new. I'm going to say
12:25 Carpathy auto research demo. And then I'll click create. Then I'm going to
12:29 open this folder. Okay. Okay. And now in order to open up cla code, I'm just
12:32 going to double click anywhere in here, click on my little claude code button,
12:35 and then what I want to do is I basically want to clone this. So I'll
12:41 say, hey, clone this in the current working directory. What this is going to do is it's going
12:47 to make an HTTP request over to GitHub, and then clone this service, store that
12:51 down below in a folder called auto research. And the reason why we're doing
12:54 this is because we just want all of the context of this whole repo before we go
12:58 ahead and actually define, you know, what it is that we're going to do on
13:01 this. And so what I want to do just for demonstration sake is I'm just going to
13:04 reproduce my cold email example. After that, I'm going to give myself a little
13:07 bit of space. And now, because I have access to a voice dictation tool called
13:11 Whisper Flow down over here, I'm just going to hold my Fn key and then tell it
13:16 what I want. Hey, I want you to use the context in the auto research folder to
13:20 help me build a very similar idea, except instead of testing for validation
13:24 loss and iterating on a machine learning model. I want you to do all of this, but
13:29 for cold email. The metric I'm interested in optimizing for is my reply
13:33 rate. The platform I'm going to be doing all this stuff on is instantly, and I'll
13:36 give you the API credentials and everything that you need in a moment.
13:40 And finally, the thing that you're going to change between one experiment and the
13:44 other is going to be the copy of the cold emails. Finally, I want you to take
13:48 all this and then put this on the cloud using GitHub actions. So, it runs once
13:53 every hour and it has everything it needs to work on autopilot. Once I've
13:57 pasted that in, I'll press enter. Now, it's going to go through all of the auto
14:01 research documentation. You know, it has a few things here that's probably not
14:04 super important, like this image, which shows uh I don't know, the progress on
14:08 Carpathy's side. You can see here his baseline was validation BPB. It's some
14:14 form of basically accuracy how good the model is. Started up here and then after
14:18 just a few runs it got all the way down over here by adjusting various
14:21 parameters. This is more or less everything that's going to occur except
14:25 with our cold email. So it'll be like um you know invert this graph reply rate
14:29 will start here and then the idea is the reply rate will go up over time. Anyway,
14:33 I'm going to let it run for however long it needs to before it does everything
14:36 that it has to. And then at the end of it, we're going to have a fully
14:39 functional autoresearch campaign. It's now asking me some questions. How should
14:43 the system generate new email copy variants? So, I'll say Claude, do you
14:46 already have campaigns running and instantly or will this create everything
14:50 from scratch? So, I'm going to say from scratch. Then click submit answers. And
14:53 it's now going through and building the email optimizer. For simplicity, I'm
14:58 naming it similar to the other one just so you guys could see what's going on.
15:02 Now it's actually building an instantly client which will contain all of the API
15:06 calls that it needs to make to instantly to get the information. We also have the
15:10 orchestrator. Now orchestrator just for anybody that doesn't uh isn't inherently
15:15 familiar with the language is basically almost always going to be like your top
15:19 top level agent. And the idea is it's the orchestrator which orchestrates okay
15:24 kind of like a conductor in a symphony or something the function of a bunch of
15:29 lower level agents or tools. And so in this case, what this orchestrator is
15:32 doing, I'm going to try and draw a little blue cloud code logo. Didn't do a
15:36 very good job there. But basically what's occurring is this is
15:39 orchestrating any sub aents that we'll need for maybe the purposes of writing
15:43 copy. Um it'll orchestrate the calling of like the instantly API. It'll
15:49 orchestrate the I don't know storing of documents and uh I don't know JSON
15:53 results and obviously you could build in a database. You could do whatever the
15:56 heck you want there. And so this is what is essentially going to be us just
15:59 speaking to the orchestrator saying, "Hey man, here's what you are. You're an
16:03 email optimizer orchestrator and you have access to all this stuff. Next, the
16:07 utility scripts are just little one-off API calls like tools that allow it to do
16:11 things like purge old leads, deploy and batch, test my parsers, and so on.
16:16 [gasps] the config files here like baseline resource and in this case I fed
16:20 it some additional documentation from a big course I did inside of Maker School
16:24 that teaches people how to write good quality cold emails. This is just things
16:28 that I have the ability to change. So I can change my baseline test. That's the
16:33 first test the cold email optimizer will ever test against. I could change what's
16:36 in resources, although obviously that's going to be added to. And then I also
16:40 have little tokens over here so it can access the um APIs. And then finally,
16:44 the GitHub actions workflow. All right. And I'm scrolling through here. It's
16:47 doing the vast majority of the work, which is pretty nice. And in in this
16:51 case, it's actually creating some sub agents to do it for me. And if I open
16:54 this up, you can see we actually have a bunch of data. So this is av.example.
16:59 So um I'm going to ask it to basically set this up as a demo, meaning you guys
17:02 can just pump in whatever the heck you want. And then also I'm going to add a
17:06 Slack web hook. The reason why I'm doing that is because I basically just want it
17:09 to be able to tell me how it's doing whenever it makes the changes. Now that
17:12 it's doing some testing, we can basically go ahead. So just for
17:16 demonstration purposes, I'll say great create a baseline and a challenger and
17:23 show me dry run. This is a demo. You can see what it's written over here as well.
17:26 The way it works is it runs every hour via GitHub actions cron. That's just a
17:30 scheduling tool that triggers once per hour. There's three steps. It's going to
17:34 harvest by collecting results from the previous experiment. It'll generate by
17:38 creating a new challenger and then it'll deploy by creating the campaigns,
17:42 drawing the leads from a pre-existing pool or database that I've given it and
17:46 then finally activating everything. So you see the leads are over here. We have
17:51 uh the usage then obviously we have like the big fat long script as well. I
17:55 didn't have to write any of it. And this all follows very similar logic to what
17:58 Carpathy was doing. It's just instead of doing this for like machine learning
18:01 purposes, we're obviously doing this for financial purposes, better reply rates.
18:04 Now, because so much of this stuff occurs completely autonomously, I would
18:08 recommend you always have a way to visualize or at least keep track of
18:12 things as they go. And so what I've done is I've set up a little Slack ping via
18:16 web hook that notifies me every time a new challenger or a baseline variant
18:21 test is created. And so what happened is the other day we actually tested three
18:24 different ones. You can see some of these tests were pretty small and pretty
18:27 minor. We just made adjustments to the subject line and so on and so forth. But
18:31 it stores things like the baseline and the challenger. And then whenever a
18:34 harvest occurs, it tells us more or less which one won. Okay. And we now have the
18:37 baseline which is a subject of quick question. Gives me a bunch of uh
18:41 baseline copy here. So I actually wrote this initial first email. And then it's
18:45 generating the challenger. The hypothesis is the baseline is too long.
18:49 It buries the offer and it also lacks a specific CTA time. So it's going to try
18:53 rewriting it to sub 75 words leading with relevance frontloading the risk
18:56 reversal and ending with a concrete time ask. You'd see it's quite the
19:00 significant change here. Hey, first name. My drive PPC leads for a 2 mill a
19:03 year dental marketing firm in Calgary. I've sold over 10 million business
19:05 agencies like yours through cold outdown alone. Got a backlog of people wanting
19:09 PPC right now variety of verticals. I'd send you booked appointments and only
19:11 charge if we hit a number you and I agree on beforehand. Zero risk on your
19:15 end. Is this worth a quick call? It even then gives a specific time. So I mean it
19:18 remains to be seen whether the challenger is going to be better than
19:21 the the baseline, of course, but that's just part of the game. And you can see
19:24 this has now actually been deployed. We have uh that same copy over here. And
19:29 then if we go over to our baseline, we also have the baseline copy over here,
19:33 which is that old cold email. And basically what's going to occur now is
19:35 they're just going to test against each other until we figure out which one is
19:39 better. Now on net, because this is an AI model we're working with, in my
19:43 experience, most challengers are not up to the task of the baseline. Usually the
19:47 baseline is better because I wrote it, but eventually the challengers do become
19:52 better and you start seeing significant improvements in the reply rate relative
19:56 to the original. um which then you know makes them higher and higher and higher
19:59 and higher over time. Then the challenger becomes the new baseline and
20:03 then you just repeat. And so basically what this is to be honest is like the
20:06 automation of like scientific experiments. Um you know this is
20:10 something where right now there's just so much logistical overhead and friction
20:13 involved in like running any sort of experiment whether you're a marketer, a
20:17 salesperson, somebody doing some backend functionary business or whatever. And
20:20 this just eliminates all that friction. I no longer have to like copy and paste
20:23 the leads. I no longer have to like do anything manually. Um, it's all done via
20:27 simple API calls. And now that we can put this thing on a loop, even though
20:30 every time I run the orchestrator, it's technically like a different agent. It
20:33 has all the context from all of the previous runs, which allows it to grow
20:37 more intelligent over time. I anticipate that eventually after something like 500
20:41 to maybe a,000 runs. You'll probably have to consolidate some of the previous
20:45 learnings so that that document doesn't get super long. But whatever you're
20:49 using this for, whether landing pages, PPC, uh newsletter copy, whatever the
20:54 heck, you know, SEO pages, hopefully you guys understand that as things get
20:57 better, the new challengers are just going to be millions upon millions of
21:02 times more profitable and e efficacious than um what your initial baseline was.
21:05 Now, I should note there's some other things that you have to do in order to
21:08 set this up completely. Like for instance, I actually had to go grab my
21:11 API keys. The way you do this on Instantly is pretty straightforward.
21:13 Settings, then you go integrations, then you go API keys down here. Then you
21:17 create an API key, say whatever the heck you want, select scopes all, and then
21:20 actually copy it over. Um, you obviously need to do that as well with whatever AI
21:23 model you're using to do the orchestration. In my case, I was using
21:27 Claude Opus 4.6, so I just went over to anthropic, got their API key. Um, and
21:30 then, you know, if you have any other other services you want to use like
21:33 GitHub, for instance, or whatever, you also have to push that up. But agents
21:37 will handle all that stuff for you. Just ask them, hey, you know, where do I go
21:40 to get my API key? Okay, can you sign me in? And so on and so forth, and you'll
21:43 be good to go. The final thing I want to talk about are use cases that I would
21:47 consider not ideal for some form of auto optimization. Um, in general, things
21:51 that work really well are things that have fast feedback loops. Okay, so why
21:57 did Carpathy's um AI agent or like nano GPT loop work so well? Because it was
22:03 literally a fiveinut loop. You know, if you have a five-minute loop, technically
22:07 speaking, that means that in 60 minutes you could run 12 experiments. And so
22:11 obviously 12 experiments is a lot of data. And assuming that you know you're
22:14 running it on your own servers or whatever, you can just have that thing
22:17 churn. But basically that means that your your iteration loop will be much
22:21 faster because you'll be able to kind of draw it like this as opposed to like
22:25 this. You know, it's going to take a lot longer to figure out what works and what
22:28 doesn't. If you're all the way down here, another good thing to keep in mind
22:32 is you need a clear metric. So in my case, reply rate was a fantastic metric.
22:35 Why? That's objective. How many people actually replied to my email campaigns?
22:38 Click-through rate. It's very very objective because you know obviously
22:42 this is people clicking through an email all the stuff is automatically tracked
22:45 but if you had something that was way fuzzier in addition to way slower you
22:49 know the probability of you actually making this work uh is is much lower
22:53 because how do you subjectively measure like warmth you know you can't it's like
22:58 happiness it's like you can't what you have to do is you have to find proxies
23:01 for all these things which are usually like scales and metrics and analytics
23:05 and so on and so forth and then the third thing is you need some sort of API
23:10 I access to change the inputs. Um, and if you don't have the API access, you
23:12 could build some sort of Chrome dev tools or CLI based flow, but like you
23:16 need to have that because if you don't, how the heck is the agent supposed to
23:18 make any changes? What are they going to do? Just give you a list of changes to
23:21 manually go in. You can do that, but that sort of defeats the whole purpose.
23:25 Anyway, so what I'm going to do with this is I'm going to provide everything
23:28 that you guys need, including the email optimizer repo, uh, the Carpathy, um,
23:33 auto research, GitHub repo, and everything else down below. Feel free to
23:36 take a look at it, give it a click, explore it, and use it for your own use
23:40 case. I'd be really interested to hear what you guys end up using this on. Um,
23:45 this is what all major labs that are working on machine learning models
23:47 around the world are currently doing, by the way, in case it wasn't clear.
23:51 They're constantly running many, many, many experiments behind the scenes
23:54 overnight to like make their models better and so on and so forth. So, the
23:56 fact that we're able to democratize that and now do that for ourselves, for our
24:00 own businesses, and for our own uh own models is now like incredible. But I'd
24:03 be really curious to hear what sort of use cases you guys have with this. And
24:06 you know, if it makes sense, I could compile a list of these use cases and
24:09 then I could make another follow-up video that just goes through every
24:12 single one and then even gives like real examples of them. Um, because that'd be
24:15 really dope. Aside from that, if you guys do me a big solid, something like
24:19 73% of you are not subscribed to the channel, which really hurts cuz I try
24:22 and make high-quality content for both subscribers and non-subscribers. But
24:25 YouTube pushes my content way more heavily when that ratio improves. So, if
24:29 I give you any value today whatsoever, please do click the subscribe button. I
24:32 hate asking for it, but it just makes a difference on YouTube, so I'll do what
24:35 works. If you guys want more on cloud code and so on and so forth, definitely
24:38 check that out. And yeah, I mean, I will catch all y'all in the next video.
24:40 Thanks so much for watching, as per