Deploying AI Models with Hugging Face – Hands-On Course

[00:00] This tutorial is a comprehensive end [00:02] to-end guide to the hugging face [00:04] ecosystem showing how modern AI moves [00:07] from research ideas to real deployed [00:10] systems. Hugging face is a leading [00:13] open-source AI platform and community. [00:16] It connects models, data sets, [00:17] libraries, demos, and deployment into [00:20] one coherent practical workflow. [00:23] >> In the past few years, artificial [00:24] intelligence has moved incredibly fast. [00:27] We now have models that can write, [00:29] translate, see, hear, and even reason [00:32] across multiple modalities. But one [00:34] question remains for most developers and [00:37] learners. Where do all these models [00:39] live? And how do we actually use them? [00:41] That's where hugging face comes in. [00:43] Hugging face is more than a library. [00:45] It's the open platform that connects AI [00:48] research, real models, data sets, and [00:50] applications into one ecosystem. In this [00:52] course, we'll take a practical [00:54] end-to-end journey through the hugging [00:56] base ecosystem. We'll start with a [00:58] transformance library where you will [00:59] learn how to use state-of-the-art [01:01] language vision and multimodal models [01:03] and how to use them for tasks like text [01:05] generation, classification, translation, [01:08] and question answering. Then we'll move [01:10] into diffuses library. The library [01:12] behind modern image video and generative [01:14] AI systems like Sable Diffusion where [01:16] you will learn how these diffusion [01:18] models actually work and how to use them [01:20] to generate and control high quality [01:22] outputs. Then we'll dive into Gradial [01:25] the tool that lets you convert or turn [01:27] any machine learning model into an [01:29] interactive web application in minutes [01:31] without any front-end experience. We'll [01:33] also deal with video and audio models [01:36] for tasks like speech recognition, audio [01:39] generation, and many more. By the end of [01:41] the tutorial, you won't just understand [01:43] hugging phrase, but you will know how to [01:46] navigate it, build it, use it, and how [01:48] to deploy real AI applications into [01:51] hugging phrase ecosystem. Whether you [01:52] are a beginner, developer, or [01:54] researcher, this course will provide you [01:56] a solid and future proof foundation into [01:58] opensource AI. Let's get started [02:00] together and let's dive into one of the [02:03] most brilliant tasks in the world of [02:06] natural language processing which is [02:08] text generation. We will download and [02:11] load the model GBT2 from the hugging [02:13] face platform and use it to do this task [02:16] particularly text trans text generation. [02:19] So the first thing let's put a title [02:21] here a nice title as text generation [02:26] using GBT2 [02:29] model from hugging [02:32] base library and let's put here the icon [02:35] hugging [02:40] as you can see. Then let's dive into the [02:43] first thing which is importing the [02:44] libraries. [02:46] How to grab or fetch a model from inside [02:50] the hugging face platform. So first here [02:53] we have the main page or the homepage of [02:56] hugging face. You can see here we have [02:58] models, data sets, spaces. These are the [03:01] three main components inside hugging [03:03] face. Models which are machine learning [03:06] deep learning models. Dead sets which [03:08] are the data sets the data that you want [03:10] to use to train a model and to test the [03:13] model. spaces which are a visual visual [03:18] gradial interface that visualizes the [03:21] model and how it works how it operates. [03:23] So let's go to models. [03:26] Here you can see we have the uh models [03:28] number. We have 2,285,99 [03:33] models, a large number. Let's search [03:35] into here GBT2. You can see we have [03:38] OpenAI- community/GBT2. [03:41] Let's go and click on it. Here we are [03:44] directed to the main page, the main [03:46] model card. They call it model card. As [03:48] you can see, model card. This is the [03:50] model card of GPT2. And you can see here [03:53] how many likes did they get? They have [03:56] 3,000 likes. Follow. You can follow that [03:58] if you like. I can give this a like. You [04:01] can see now I liked it. Here you can see [04:04] we can copy the name of the model. [04:06] OpenI-/GBT2. [04:08] And if you go down, how many downloads [04:10] did they have? Downloads last month [04:13] 8,868469. [04:17] Okay. And then you can go here, you can [04:18] see the number of parameters, the model [04:20] size. Uh and here feel free to read [04:23] about this. This is the the explanation [04:25] and the description some meta [04:28] information about this model. So you can [04:29] see here how to use it. Actually there's [04:31] there there is some sales that explain [04:34] how to use this model. You can see [04:35] generator equals pipeline pass text- [04:38] generation. This is the name of the [04:40] task. The task want to conduct is text [04:42] generation model equals dity tool. Then [04:45] you can see they have they have passed [04:47] some info here some prompt. Hello I am a [04:49] language model. And then and this is the [04:51] answer of the language model. Okay, [04:55] let's go ahead and use this model by [04:57] going here and then click on [04:59] transformers. [05:00] Here we have a model that shows two uh [05:03] blocks or two sections. The first [05:05] section uses a highle helper which is a [05:09] great option and the second option is [05:11] load model directory. This way the way [05:14] that we that we will use in in this [05:16] lesson and uh that we used we we saw [05:20] that from transformers import auto [05:21] tokenizer comma auto model for com and [05:24] this way uses a highlevel wrapper or [05:27] helper called pipeline let's use the [05:29] both options let's copy this go ahead [05:32] and paste it over here from transformers [05:34] library import pipeline this is a highle [05:37] helper then we use that function and [05:40] pass here two arguments just two [05:42] parameters is the task you want to do [05:45] text dash generation and the model that [05:47] you want to leverage openai-d2 [05:51] let's run that you can see now it is [05:53] trying to download this model the uh [05:56] from the using the pipeline function [06:00] now it's connecting Okay. [06:13] Here you can see it's now loading the [06:14] model. Uh the task as we said text [06:17] generation. So this model tries to [06:19] generate some text. The model that we [06:21] that we need to leverage or use is GBT2. [06:26] Now it takes some time to to download [06:28] and load the model into the environment [06:31] collab. [06:34] And you can see here now it's executing. [06:36] It's taking 27 seconds so far. [06:44] By the way, you can go to hugging face [06:46] and then models and then go to tasks. [06:48] Then choose the task text generation [06:51] from here. NLB natural language [06:53] processing. go to text generation here. [06:56] We have plenty and a plethora of models [06:58] of text generation models that you that [07:01] you can use. Okay. [07:04] So, so for example, we have this model [07:06] from OpenAI. It's called circuit [07:08] sparity. Nvidia also has a model called [07:10] Neometron orchestrator orchestrator [07:14] none none biggy. You can see llama also [07:18] you can use llama model. So these are [07:19] all the text generation models. You can [07:21] see here we have text generation. How [07:23] many models do we have? Well, we have [07:24] here 303,000 [07:27] models all dedicated to the task of text [07:30] generation. [07:32] Here we can see that the model is [07:34] loading currently. You can see it's [07:36] trying to load some libraries some for [07:38] example models tenses. This is a sub [07:41] library generation config.json [07:44] and we have obvious then device set to [07:46] use CPU. So this is the process to load [07:50] a model using the haba using the [07:52] pipeline. Now as you can see the model [07:54] now is successfully loaded. Let's use [07:57] that model by going to here and calling [08:00] pipe. You can see pipe and then pass [08:02] into here some text. Consider this as a [08:05] prompt. So you can define a a variable [08:07] called prompt equals [08:10] prompt equals what is machine learning. [08:13] Then pass this prompt directly to the [08:16] pipe. That easy. [08:19] It's easy as that. So you can see now [08:21] the model is trying to pass this prompt [08:23] to the pipe. You can see here the [08:25] benefit of using the pipeline here is [08:27] that you don't need to tokenize the [08:30] sentence. So you don't need to pass the [08:32] sentence as we did to pass it to the [08:35] tokenizer. Then you get token ids. Then [08:38] you pass to the GBT model. No. Here you [08:40] can see we have a list. Inside this list [08:43] we have a dictionary. Then we have the [08:45] key called generated text. Now to fetch [08:48] that key specifically, we can go ahead [08:52] and store this inside a variable called [08:54] output like that. Okay. Then we can step [08:59] through that output, step into it and [09:02] fetch the value of the text. Okay. [09:06] So we can go output [09:09] and then add index zero to get the the [09:12] dictionary. Then we can go the key [09:15] called generated underscore text. So [09:18] wait until this finishes loading. So you [09:23] pass the sentence to the pipe. Then we [09:26] store that that value inside the [09:28] variable. Then we can use that that [09:29] variable freely. Now you can see run [09:31] that. Nice. Now we have this text. What [09:34] is machine learning? It's not very hard [09:36] to understand. This is the answer of [09:38] GBT. You can see this is my prompt. [09:40] Okay. Then this is the answer. It's not [09:42] very hard to understand. It's one of the [09:45] reasons that why we've been be able to [09:47] develop the neuronet for some time and [09:49] blah blah blah. Now, if you want a nicer [09:52] and neater format of this, we can pass [09:54] this to the print function. Okay? So, [09:57] pass this because why? Here you can see [09:59] we have back slashn [10:02] and it's it's it's not interpreted well. [10:04] So, yeah, here we have back slashn. So [10:07] print function will interpret that into [10:10] an actual line into a new line print. [10:12] You can see we have a a nicer output as [10:16] if you ask GBT in its interface and it's [10:19] it answers like that. What is machine [10:21] learning? It's not very hard to [10:22] understand. It's one of them of the [10:24] reasons why we've been able to develop [10:26] the neuron for some time and you can [10:28] feel free to read that. So give whatever [10:30] prompt you would like and the Jeep will [10:32] answer GP2. The second approach is that [10:35] we can go ahead go up use this model. [10:39] You can see the library here is [10:40] transformers. Then we have another [10:42] option here which is this from [10:44] transformers import auto tokenizer, [10:46] automodel for com. Let's go ahead and [10:49] use that approach. [10:53] Run run run run it again. You can see uh [10:56] this approach is uh is a lot faster. Now [10:58] we have a model this model. Okay, we can [11:00] go ahead and this is the archae of the [11:03] model GP2. But keep in mind you have to [11:05] tokenize your text before passing to [11:07] this model as we saw before. So that [11:10] that's it that is for text generation. [11:13] You can go ahead as as we said models [11:15] and then choose the tasks from here [11:18] choose the task as text generation. Now [11:21] let's move on to another task in [11:23] transformers library. [11:26] Now as an alternative way rather than [11:29] copying those codes from hugging face [11:31] platform we can write them straight away [11:33] in here. In this section we will use [11:37] GBT2 model from hugging face platform. [11:41] So the first thing the very first thing [11:43] we need to do is to import the [11:45] transformers library. So let's go from [11:48] transformers [11:50] import [11:53] auto [11:55] tokenizer. [12:00] So this is auto tokenizer class from [12:02] transformers library. Then we will call [12:05] the method from pre-trained and then [12:07] pass in GBT2. [12:10] So as you can see now it is trying to [12:12] import the class autoto tokenizer from [12:15] the library transformers. [12:18] Then the next step is to instantiate a [12:21] new object called tokenizer equals auto [12:25] auto tokenizer [12:28] and then dot from pre-trained [12:32] pass into here GBT2. [12:35] So as you can see and then run that. So [12:38] now we are uh fetching the the model GP2 [12:44] using the method from pre-trained that [12:46] is coming from the class auto tokenizer. [12:48] As you can see here this class this is a [12:50] generic class tokenizer class as you can [12:53] see that will be instantiated. Then we [12:56] call the method from pre-trained and [12:58] then use one of the language models that [13:01] is provided in hugging face for example [13:03] in this example or in this case we use [13:06] GBT2. [13:08] Then we have a tokenizer ready. So let's [13:10] try this tokenizer by going sentence or [13:13] sentence equals unsure. Okay. Then we [13:17] need to pass this sentence to the [13:18] tokenizer object by going input ids [13:23] equals tokenizer [13:25] and then pass into here the sentence. [13:28] Then I want you to notice here if you [13:30] pass this sentence as you can see tense [13:34] then we didn't specify the parameter [13:37] return tenses as pt it will return list. [13:40] So let's take a look at this input [13:42] underscore ids we pass the sentence into [13:44] here as you can see now we have uh this [13:46] is a dictionary then the input ids are [13:49] in the form of list. So if we go ahead [13:51] and access this key input [13:55] ids then we will get here a list. If you [13:58] want to retain these ids inside of a [14:01] tensor inside pytorch tensor let's [14:04] specify return tensors equals pytorch [14:08] pt. So pt is short for pytorch and then [14:11] let's run that again. And as you can see [14:14] successfully now we have pytorch tensa. [14:17] Now let's take a look at unsure word. [14:20] Why unshaw word is splitted into two [14:23] numbers or into two ids? Well, because [14:25] unshaw is a word that is divided using [14:30] the sub level tokenization [14:33] level. So we have un as a separate token [14:36] which is this ID 13,271. [14:40] Then we have sure uh we have shore a [14:42] separate word a separate token as the ID [14:46] 495. Okay. So some words you have to [14:49] keep in mind that some words will be [14:52] splitted will be segmented into more [14:54] than one segment. So not necessarily one [14:58] word is one token. Maybe one word [15:00] generates more than one token gen more [15:03] than a single token. Okay. So as you can [15:06] see we have no ideas how to get back to [15:08] the word unsure. Well to get back to the [15:11] unsure word we have to use the method [15:13] decode by going tokenizer. Decode as you [15:16] can see. Then we pass into here the [15:18] input underscore ids. Then let's access [15:21] at index zero. It token it decodes that [15:24] into the word unsure. So we got back [15:26] into the word unsure as you can see by [15:30] passing the ids to the method decode. [15:33] Okay. Then it gets back to the unsure [15:35] word as you can see. The next step is [15:37] let's try another word. Let's try for [15:39] example the word unbelievable by going [15:42] for example sentence equals unbelievable [15:48] like that. Then we pass this sentence to [15:51] the tokenizer by going input ids equals [15:54] tokenizer and then pass into here the [15:57] sentence sentence then we specify the [16:00] parameter return tensors equals pt for [16:04] returning the tenses in the form of [16:06] pytorch tensor. So please so here we are [16:09] saying hey please so canera return the [16:12] ids stored into a pytorch tensor then [16:16] dot input ids so we get right up the [16:20] pytorch tensor then we have the input [16:22] underscore [16:24] ids as you can see we have these ids so [16:26] here so as you can see here we have the [16:29] word unbelievable and it is divided into [16:32] three tokens let's figure out what are [16:35] the actual parts what are these actual [16:38] parts by doing a for loop. So for token [16:42] id for every single token id in these [16:44] ids in input ids as you can see add [16:48] index zero to get this segment here this [16:52] list the inner list then we go ahead and [16:55] print tokenizer dot decode and then pass [16:59] into here the token underscore id. So if [17:02] now if you run this 3 2 1 we have the [17:06] actual segments the actual parts that [17:09] these ids correspond to. So 403 [17:13] correspond to un then we have 667 [17:16] correspond to corresponds to bill. Then [17:19] we have eable. So as you can see [17:21] unbelievable. [17:23] Then let's figure out another word. [17:25] Let's take another example. For example [17:27] the the the word homoscadasticity [17:30] by going word. for example equals homo [17:34] skod skadicity skasticity [17:39] like this. Then let's pass it to the [17:41] tokenizer by going tokenizer dot decode [17:45] and then pass word. Then we let's set [17:48] the parameter return underscore return [17:52] tenses equals [17:54] pt [17:57] short for pytorch.input input underscore [18:00] ids and let's let's store this value [18:03] into a variable called for example my [18:06] ids [18:08] then let's print that my ids as you can [18:12] see here we have input ids let's run [18:14] that [18:16] so now this tokenizer will work on the [18:18] word homoscadacity and tokenize it [18:21] segment it and return the ids let's see [18:24] what this error is so ids can't extract [18:27] string to vic So yeah I think here we [18:30] have to return tenses pt we pass yeah we [18:33] need to specify the function as [18:35] tokenizer. [18:37] Okay so now we have these ids as you can [18:39] see. So I used decode and in fact I have [18:43] to pass the word straight away to the [18:45] tokenizer. So as you can see here we [18:47] have these ids 26452 418 771 and to the [18:53] rest. So the word homoscadaci is being [18:55] divided up into one, two, three, four, [18:59] five into five different tokens into [19:02] five different words. So I want you to [19:05] imagine or have an idea that a single [19:08] word doesn't necessarily represent one [19:11] single token. It could be splitted up [19:14] splitted up into several tokens. It [19:17] could generates multiple tokens and [19:20] generates multiple uh multiple segments. [19:22] Okay. So let's take also another example [19:26] like for example let's decode it. So [19:28] tokenizer do decode and let's get back [19:31] to the real world homoscadacasti. So now [19:34] I have ids. How do I know what these ids [19:38] correspond to? What is the word behind [19:41] these ids? By passing my ids to the [19:45] function decode. And we have to specify [19:47] here at index zero or for example dot [19:49] squeeze dot squeeze. Okay. Then we can [19:52] pass it here. We have here homos [19:54] dacasti. So if you use here add index [19:56] zero, it will also work [20:00] in a nice way. And if you specify here [20:02] dot squeeze, so you squeeze it, you get [20:05] rid of this outer list brackets, then it [20:10] also works. Homoscadacity. [20:13] Okay. And feel free to use this [20:14] tokenizer. So this tokenizer is based on [20:17] what? Based on GPT2. If you call this [20:20] tokenizer on its own to get some [20:22] information about it, you can see that [20:24] hm name or path is GBT2. So this [20:27] tokenizer is based on the tokenizer that [20:30] open AAI used for GBT2 model. The [20:33] vocabulary size here so the number of [20:36] vocabulary words is 50,257 [20:40] tokens. So keep in mind these are not [20:43] words, these are tokens. So maybe there [20:47] are words that was splitted into [20:50] multiple tokens. So these are the tokens [20:52] 50,257. [20:54] We have model max length. So the model [20:56] maximum length is 1,024. This is in [20:59] terms of GBT2. But here you can take a [21:02] look at uh several information about [21:04] this tokenizer. So this tokenizer now is [21:06] using the GBT2 model tokenizer. Feel [21:09] free to change it to any word. Now let's [21:12] try a a long word. For example, uh the [21:16] word numino ultramoscopic silicico [21:19] volcano coniosis. If you go to Google [21:21] and write in here numo [21:24] numino [21:25] u ultramicroscopic silico volcan volcano [21:29] conosis there. So this is the longest [21:32] word in the English language. Let's copy [21:34] it and [21:37] go word variable equals this word and [21:39] then let's tokenize it. I wonder how [21:42] many tokens this word results into. [21:45] Let's figure that out together by [21:47] passing this word to the tokenizer. So, [21:49] let's go. My ids equals tokenizer dot [21:54] tokenizer. Then pass into the word. Pass [21:56] here the word this variable. Pass it to [21:59] the tokenizer. Then let's specify here [22:01] return tenses. Should I return tenses? [22:05] Yeah. And in the form of PyTorch dot [22:07] input ids. Then print out my ids. Show [22:11] me your ids. Wow. You can see here we [22:14] have multiple ids. So one, two. Oh, [22:17] let's get the length. We have the length [22:20] method. We have uh at index zero, [22:24] we have 15 different tokens. So this [22:27] word, this long word, this is the [22:28] longest word in English. Actually, it [22:31] results into 15 different segments, 15 [22:36] different tokens. Okay? And if you want [22:38] if you want to if you want to see them [22:39] they are as this my ids then these are [22:43] my ids 79 256 68 261. So these are ids. [22:50] So the first thing we have tokenization. [22:52] Tokenization is the process of [22:54] tokenizing or segmenting or breaking up [22:57] the sentences the words into tokens. [23:00] Then we map these tokens into ids. Then [23:03] after that we pass these ids to an [23:05] embedding layer. The embedding layer [23:08] maps these ids into vector embedding [23:11] representations. Then we have vector [23:14] representation objects. We can uh [23:16] preserve the positional encoding. Then [23:18] pass these embeddings to the transformer [23:21] encoder. Then we have a GBT model. Okay? [23:24] Or or to a decoder then we have a GBT [23:27] model. So the first thing we tokenize [23:28] the text into tokens into segments. We [23:32] map these tokens into ids. We convert [23:34] these ids into vectors. Then we preserve [23:36] the position coding. Then we pass it to [23:38] an encoder or decoder depending on the [23:40] architecture you are using or [23:42] leveraging. [23:45] So now uh that's enough for the uh [23:47] tokenization process. We saw multiple [23:50] words also. We can do a for loop on this [23:52] by going for token ID. So for every [23:55] single ID in my ids dots squeeze. [24:01] Okay, let's go ahead and print tokenizer [24:05] dot decode tokenizer dot decode. So [24:09] decode it and then pass here token [24:13] id. Uh these are the segments you can [24:16] see numo uh p is a p is a separate token [24:20] h n is a separate token on numino ultra [24:25] microscopic silicico volcano chinosis. [24:28] So GBT model sees the the this word this [24:31] long word sees it as these are as these [24:34] segments. Okay. So it processes the word [24:38] numo ultramicoscopic silicico volcanois [24:41] as these segments. You as human you see [24:43] it like that uh in in in a string [24:46] fashion numo ultra microscopic silicico [24:49] volcanois. But for the GBTs they see it [24:52] as these segments separately. [24:55] Now let's move on to the word for [24:58] example anti-isestablishment turnism by [25:01] going sentence equals and then let's [25:04] search on Google anti- disestablishment [25:10] terism. Let's copy this word. This is I [25:12] think the second longest word in English [25:15] language. Let's paste into here then [25:17] pass it to the tokenizer. Let's see. [25:19] Let's discover how many tokens that will [25:23] be resulted when we pass this word pass [25:26] it to the tokenizer object. So by going [25:29] my ids or for example let's name it uh [25:32] token ids. Let's verify the name of the [25:35] variables. Then pass here tokenizer. [25:38] Pass here the sentence. [25:40] Then comma return tenses equals pt dot [25:44] input ids by going dotinput the [25:47] attribute input [25:50] ids. Then let's [25:53] print this this variable token ids and [25:57] see how many ids that are being [26:00] generated. [26:02] Well, as you can see, we have these ids. [26:04] And if you get the length length and [26:06] then pass token ids at index zero, we [26:10] have five different tokens. So, 1 2 3 4 [26:14] 5. So, this word is being splitted up [26:16] into five different tokens. [26:19] Also, this uh there is a last word [26:22] called flexo fluxification. [26:26] So, let's go word equals [26:28] search here. Flock uh flocks [26:31] hillification [26:33] which is also considered uh one of the [26:35] most longest words in English and then [26:38] pass this to the tokenizer by going my [26:41] ids equals tokenizer tokenizer [26:45] dot and then pass word return tenses [26:50] specify it as pt for pytorch token.input [26:53] input underscore ids then let's spread [26:56] out my ids and these are my ids as you [27:00] can see and how many ids we have let's [27:02] go my ids add index zero dot or pass [27:06] here the length to get the length [27:10] I have nine different tokens nine [27:13] different segments that this word is [27:15] being spitted into as you can see these [27:17] are the ids and we can loop through it [27:20] as you as we saw before for each token [27:23] id in my ids at index zero. Then go [27:27] ahead and [27:29] print print print [27:32] the tokenizer decode and then pass into [27:35] here my token id. So we have f l is a [27:40] single token o so flocken [27:44] si helibilification as you can see which [27:47] is a word that indicates nothing. [27:52] Then let's move on to a [27:55] another another section which is how to [27:58] generate. Now we we saw how to tokenize [28:00] the words. We saw the ids. Now what if I [28:03] wanted to generate to use GP2 and [28:07] actually generate using GP2. Well, we [28:10] have to we have to import another class. [28:12] So let's go from transformers import [28:16] automod [28:18] for causal lm. So we have a class we [28:22] have a nice class called autom model for [28:25] causal language model. So this is a real [28:28] causal language model. Let's import it. [28:32] Then let's instantiate a new object. [28:35] Let's call it GBT2 equals. So my nice [28:38] GPT2 model equals automodel [28:41] for causal lm. I like that class name. [28:45] Then let's pass into here GPT2. [28:49] Run that. As you can see uh we have to [28:52] call the method from pre-trained. So we [28:54] have to go from underscore pre-trained [28:57] and then pass into here GBT2. So do not [28:59] forget you have to call the method from [29:02] pre-trained always when you use a class [29:04] from hugging face. Now as you can see we [29:06] have a warning. Uh don't worry about [29:08] this warning. It just warns you that [29:10] detected not written. [29:13] No worries. [29:14] As you can see now we have successfully [29:17] downloaded the model GPT2. Okay, let's [29:20] go ahead and see the archetype. So what [29:22] is the architecture looks like of GT2? [29:25] If we type here the name just the name [29:27] GT2 and run that you can see we have the [29:30] real architecture here. So this is the [29:31] architecture. First we have embedding [29:33] layer then we have another embedding [29:35] layer. I think this is for uh converting [29:38] ids into vectors and this is for adding [29:40] the positional encoding. We have drop [29:42] out then we pass it to the HBT2 block [29:45] and we duplicate this block 12 times. [29:48] Inside this block we have layer [29:50] normalization layer to normalize the [29:52] inputs. We have GBT to attention. We [29:54] have curve layer convey [29:57] convolution. So this so this convolution [29:59] is the attention. They implement the [30:01] attention via a convolution layer. Okay. [30:03] So feel free to explore this [30:04] architecture on your own. But now let's [30:06] dive in and use this architecture use [30:09] this really clever architecture to [30:11] generate. [30:14] So the first thing we need to uh to [30:17] build a sentence by going sentence [30:20] sentence equals and let's use a sentence [30:24] that is that has a tendency for chity to [30:28] complete it. So if you say that I like [30:31] machine learning then we ask jubility to [30:35] complete then jity will complete on this [30:37] onto this sentence. Okay then let's go [30:41] ahead. So I like to machine learning too [30:43] for example uh to to do something or to [30:46] be able to do something. Let's let's [30:47] figure out what GBT will will generate [30:50] as a next token prediction. So let's go [30:52] ahead and use GBT2. As you can see we [30:54] have GBT2. Then pass into here this [30:57] sentence. That's all we have to do is [30:59] pass this sentence to the model GBT2. [31:02] Then let's run that. As you can see here [31:06] we have to [31:09] so the size uh string has no size. Yeah [31:13] we have to tokenize this string. Yeah we [31:15] have forgotten a important step uh [31:18] before we pass this sentence to the GBT. [31:20] We have to tokenize it. So we have to [31:22] get the ids as we did before. So here we [31:24] have token ids equals tokenizer then [31:28] pass my sentence. Then this tokenizer [31:31] will work on the sentence and then [31:33] return underscore tenses equals pt [31:36] pytorrch and then get the input [31:38] underscore ids. Then we have a token [31:41] underscore ids. So these ids now has the [31:45] capability to be passed to the to a gb [31:47] model. So don't pass the string straight [31:50] away to the GBT model. You have to [31:52] tokenize it. Why we explained this [31:54] tokenizer before? because we need to [31:56] tokenize any string before passing this [31:59] string or this prompt to the GBT model. [32:01] So this is a prompt we tokenize it we [32:04] process it. So this is a data processing [32:07] uh processing technique. Then after we [32:11] proc it we can we could then pass these [32:14] to ids to a GP model by going GPT 2 and [32:18] then pass token ids. [32:21] Hooray. Now we have here the output of [32:24] GT2. Here you can see we have logits. [32:27] Let's access the logits dot logits. [32:33] Now how many how many vectors do we have [32:36] the same number as the word here as the [32:38] W. So we have I 1 2 3 4 5. Here we have [32:45] 1 2 3 4 5. So the same number of words [32:53] you have here, the resultant logists [32:56] will have the same length as the number [32:58] of words you have. So we have five words [33:00] and also we have here five corresponding [33:02] vectors. Okay, I care mostly in the last [33:07] vector. So let's get at index 0, minus [33:11] one. So because I my my vector of [33:14] interest is the last vector. So let's [33:16] tap into that and get it. And now we [33:19] have it. Now let's store that in inside [33:21] a variable called outputs. Outputs. [33:24] Okay. [33:26] After that let's show it our outputs. [33:29] Then let's call the function argmax. The [33:31] arg max function. It will give me the [33:35] the index at which the highest [33:38] probability exists. [33:40] Okay. So let's pass this [33:44] to the tokenizer. Decode to decode it. [33:48] So this is a a token ID. We pass it as [33:51] we learned before. We pass it to what? [33:54] To the decode function and then it [33:56] decodes it. Uh B. Now I like machine [34:00] learning to let's put here B [34:04] able [34:07] to [34:10] predict [34:12] the [34:15] then we put this to predict the [34:19] future. As you can see we have the word [34:21] future. [34:23] Okay. [34:26] Then we have a dot. So as you so as you [34:29] saw we predicted the next token using a [34:32] gypy model here we have logits okay then [34:36] we got the arg max so give me the [34:39] position so returns this function [34:41] basically returns the position at which [34:44] the highest probability exists so model [34:47] generates probabilities then we decode [34:49] it and then we generate okay let's take [34:53] another example for example the sentence [34:57] Sentence I learn machine learning to [35:03] enhance [35:06] then let's ask GBT to to generate the [35:10] following words the following tokens by [35:13] passing this sentence to the tokenizer [35:16] first to do data prep-processing. So we [35:18] have token ids equals tokenizer my [35:21] little tokenizer and then pass in here [35:24] the sentence comma return says we have [35:28] to specify it as pt short for pytorch [35:31] input [35:33] ids. Then let me scroll down here to see [35:36] the output. Then let's pass these token [35:38] ids to the GPT model by going outputs [35:42] equals GPT2. Then pass in token ids. [35:48] Then get the logits tab into the logits [35:51] and index 0, minus one to get the last [35:54] token to get the logit of the last [35:56] token, the last word. For example, [35:58] enhance. And then we have tokenizer [36:01] decode. I then pass into here the [36:04] outputs. But please give me or return [36:08] the arc max the position at which the [36:10] highest probability exists. [36:13] Run that we have here to enhance our so [36:17] you can see here GBT2 now is now [36:19] predicting the next token as our so [36:22] let's put in here our to enhance our [36:26] understanding. So GP now is predicting [36:29] the word understanding as the next [36:32] token. [36:35] understanding of so you can see of the [36:41] world [36:44] around [36:46] let's uh inject that word to around us [36:51] then we have a full stop so you can see [36:54] this is the way that GBT is running the [36:57] way GBT works the way GBT operates it [37:01] predicts the next token [37:03] So it chooses the highest probability of [37:06] the token that it thinks or it believes [37:08] that this word should be the next word. [37:11] Okay. So this is another example. Now [37:14] what are these sampling strategies? [37:16] Well, we have many sampling strategies. [37:19] The ways you can extract the token to be [37:22] selected as the next token to be chosen [37:25] as the next token. First we have a [37:27] function inside torch called topk. So [37:30] let's use that function by going let's [37:33] first use our sentence for example [37:37] sentence equals I learn machine learning [37:42] to enhance the same the same sentence [37:45] and then let's tokenize it using the [37:47] tokenizer by going token ids equals [37:51] tokenizer [37:53] and then pass into here the sentence [37:56] return tensors equals [38:00] PT pytor dot input ids [38:05] input ids then let's pass these ids to [38:09] the gbt2 model by going outputs equals [38:13] gbt2 pass into here the token ids so [38:16] remember you have to preprocess your [38:18] sentence by passing it to the tokenizer [38:20] then let's get the logits at index 0 [38:24] minus one and then at the last tokenizer [38:26] decode and then pass Here [38:28] outputs.orgmax. [38:34] Here we have the outputs.orgmax. [38:37] Let's write. Let's read the error. [38:40] Retrained our unexpected keyword [38:42] argument. Yeah, we have a typo in the [38:45] return underscore tenses parameter. [38:49] So we have our Now let's apply the [38:51] torch.topk top k by passing the logets [38:57] into the top k function. So you can see [39:00] let's get here the output [39:03] final underscore logets equals [39:09] torchtop k and then pass into here the [39:13] logets the outputs and then comma 10. So [39:17] we need to choose the first top 10 [39:20] lojets by passing the outputs to the [39:23] torch to topk and remember we have to [39:25] import the library torch by going into [39:27] here import torch. [39:30] Run that again and now we are able to [39:32] use that function. Now let's first [39:35] visualize those final logits. [39:39] As you can see here we have a tensa a [39:41] nice tensa with 10 logs. Okay. So we [39:44] have min - 100 - 100 comma 4 and so on. [39:48] What if we limit this to one? Well, we [39:51] have then just one logit. Okay, so this [39:55] is the way of top K. Top K is a strategy [39:58] of sampling or is a sampling strategy [40:00] that selects the most top K highest [40:04] probabilities from the token pool and [40:08] then we limit it to a specified K [40:11] number. For example, seven. Then we have [40:13] here seven tokens or seven logits. We [40:16] just have seven logits. If we make it [40:19] here for example 30 then we have 30 [40:22] logits. [40:23] If we limit it to three then have just [40:25] three logits. Okay. As you can see this [40:28] is the way of operating with topk. Now [40:31] let's decode them by going for index. So [40:34] for each index now let's visualize what [40:37] are these tokens? What are these chosen [40:39] tokens by doing a for loop. So for index [40:43] in top 10 which is this final logits [40:48] dot indices. So let's get the indices. [40:52] Do the following. Print tokenizer. So as [40:55] we saw before tokenizer dot use the [40:58] function decode and then pass in the [41:00] index. So all we're doing here is we are [41:03] looping through the final logits dot [41:05] indices. Then we are storing that index [41:09] step by step into the index variable. [41:11] Then pass that variable to the decode [41:13] function. Run that. Nice. Now we have [41:16] our the my. So these are the top most [41:21] tokens that the model thinks that this [41:23] is our token is the most high is the [41:27] highest probability. The second highest [41:29] probability is the the third highest [41:31] probability is my. So to enhance this [41:34] could be to enhance our or this could be [41:37] to enhance the the world for example or [41:40] this could be to enhance my [41:42] understanding or my skills. So let's put [41:44] here my then it will change as you go we [41:47] have work to enhance my work or to [41:50] enhance my own things to enhance my own [41:53] life or to enhance my skills as as I [41:56] told you to enhance skills. So I [41:58] predicted that I didn't know that GBT [42:00] will output the word skills to enhance [42:04] my skills as you can see and in comma. [42:07] So let's choose for example and [42:11] my for example life. So I learn machine [42:16] learning to enhance my skills and my [42:18] life. So this is the way that topk [42:21] sampling strategy works. It limits the [42:25] outputed logits or the outputed tokens [42:28] to a specified number called K. Feel [42:31] free feel free to play around with the K [42:36] value. It's very fun to play around with [42:39] the K value. Let's change it to seven to [42:42] suggest seven candidates. Then we have [42:45] here seven candidates. Let me get rid of [42:47] this to trigger GBT to complete the [42:51] sentence. [42:52] So as you can see I learn machine [42:55] learning to enhance our the my human to [43:00] enhance a and for example let's choose [43:02] human to enhance human [43:05] life for example human intelligence or [43:08] human performance or human cognition or [43:10] behavior learning cognitive [43:12] understanding. So now we have seven [43:15] candidates here. Let's make the list [43:17] bigger and let's for example go go crazy [43:20] 20. [43:22] Now we have here 20 candidates inside [43:24] the list. GBT now will choose one of [43:28] these candidates. Okay. So and then if [43:31] you choose for example performance then [43:33] it will also suggest another 20 logest [43:36] or another 20 tokens. For example let's [43:38] here choose the word intelligence. [43:42] Intelligence intelligence. [43:47] Then we have these are the candidates [43:49] and in by is and you can see based on [43:53] what the previous word is or was the [43:56] token see changes or the token Z had [43:58] changed they are varied depend what what [44:01] is the previous token was okay let's [44:04] take another example now we have saw the [44:06] top K the top K is a function remember [44:09] inside torch library. Let's search into [44:11] here torch top K. [44:14] Remember always your documentation is a [44:16] reference. Please feel free to read here [44:18] in your own time. But but you can see [44:20] here top K returns the K largest [44:23] elements of a given input tensor along a [44:26] given dimension. So from this definition [44:28] we have a full summary of what this [44:31] function does. Returns the k largest [44:33] elements of the given input tensor along [44:36] a given dimension. We give it the input [44:38] tensor of loget. then it returns or gets [44:41] back the k largest element of a given [44:44] tensor at a given dimension and then [44:47] what you pass here is the input tensor [44:49] just you provide it with input tensor [44:51] and keep in mind you have to provide the [44:54] number of k the k in top k let's go [44:57] ahead and see what are the remaining [45:00] sampling strategies are let's take a [45:03] look at what are the different sampling [45:05] strategies are so we have several many [45:08] method [45:09] methods or methodologies to sample using [45:12] different methods from the token pool. [45:15] The first method is greedy [45:19] decode. This function receives from me [45:22] logits and then let's put a nice [45:25] documentation here. It returns or return [45:29] token index with maximum probability. [45:34] So all it does this function it returns [45:36] the token index with maximum [45:39] probability. So the index or the [45:41] position at which the maximum [45:43] probability exists and lives. So let's [45:46] return torch dot arg max and then pass [45:50] into here the logets comma dimension [45:53] equals minus one. So keep in mind minus [45:55] one. We have to provide here -1 as the [45:58] last dimension. Then let's use this nice [46:01] little old function by going greedy [46:04] decode and passing here my outputs. So [46:07] the logits [46:11] let's read the error torch has no [46:13] attribute arg yeah I have a typo a small [46:16] typo in the function arg max classic [46:18] error arg max [46:22] as you can see here we have the tensa at [46:24] index 13 so now it is indicating that at [46:27] index 13 the maximum probability exists [46:31] the maximum probability lives so I [46:33] wonder what is this [46:36] token by going tokenizer.de decode. [46:39] Let's decode it. Let's visualize it. [46:42] It's a full stop. So now GBT is [46:44] suggesting that the highest probability [46:48] token should be fully stop. Okay. If we [46:51] change this sentence here [46:56] for example to enhance human [46:58] intelligence to enhance and then run [47:01] that again. [47:03] These are the candidate tokens. Let's do [47:06] a greedy D code. Now we have our as you [47:08] can see this is the greedy code. It [47:10] simply returns the token index that has [47:13] a maximum probability value. Then we [47:16] choose that token. Let's visualize [47:19] another sampling strategy by going def [47:23] top k sampling. So this is the strategy [47:27] top k sampling. So we are we are [47:29] limiting the top k the top the tokens to [47:33] the top k. Okay, top k sampling and then [47:38] let's pass into here logits comma and [47:42] then and then you have to define the k. [47:43] So how many tokens you want to keep? How [47:46] many tokens you want to preserve? I want [47:49] to preserve 50 tokens. The rest will be [47:52] set to zero. The rest will be discarded. [47:54] Then we have here a nice documentation [47:58] keeps this method keeps [48:02] only top k logets [48:07] normalize [48:09] normalize [48:12] them into probabilities using the [48:14] softmax function. then sample [48:19] one token one token from the filtered [48:24] distribution. [48:26] Okay, so this is a nice little [48:29] explanation of what this sampling [48:31] strategy does. This sampling strategy [48:34] which is topk sampling keeps only top k [48:37] logets normalizes them into probability [48:40] using softmax function. Remember logets [48:44] are unnormalized scores. Then we pass [48:47] them to softmax. The softmax takes care [48:50] of them and normalizes them. What is the [48:53] normalization process? It makes the sum [48:57] of all those values add up to one. And [49:00] all those values will be limited between [49:03] zero and one. So currently if you see [49:07] here these are the the logits. You can [49:09] see that they are negative. They are [49:11] unnormalized scores. But once we pass [49:14] them to the softmax function. So let's [49:17] pass this value to the softmax function [49:19] by going torch dots soft max. And [49:23] remember we have to we have to to [49:25] specify dimension equals zero. [49:28] Here we have normalized scores. So soft [49:31] max now makes the sum of all these [49:34] values add up to one. This is the first [49:37] point. The second point is that every [49:39] single value is limited between zero and [49:43] one. So we don't have negative values [49:45] and also we do not have uh values that [49:48] are larger than one. What is the proof? [49:52] Well, if you try to take the sum of [49:53] these values I think sum like that well [49:57] it is one. So the sum or the the total [50:01] summation of all these values is one [50:04] because they are normalized. Unlike when [50:07] we just have the final logits which are [50:10] unnormalized scores they are ugly. You [50:14] can see dot values these values when you [50:17] when we take the sum of it sum [50:20] it is a unnormalized value but once we [50:24] pass them to a torch softmax [50:28] this this is a magical function softmax [50:31] the most important function in the world [50:33] of machine learning. Then we specify [50:35] dimension equals to zero. [50:39] So here we have to pass into here dot [50:42] values. Okay. Then here like that dot [50:47] values. Okay. We have a sum of one. [50:50] Okay. As you can see. So let's delve [50:53] into the top case sampling by going [50:55] values [50:57] comma indices equals. So we so we we are [51:00] unpacking unpacking the returned values [51:03] into a values variable and indices [51:06] variable and let's go torch do topk [51:09] let's use the top k function from [51:11] pytorch library and pass in my logs my [51:15] magical logs k so the k will be defined [51:18] by the parameter k equals 50. The second [51:21] step is that props we want to normalize [51:25] them into probabilities. Hence we we [51:27] wrote here normalize them into [51:30] probability. So we have to pass these [51:33] values into torch or f do soft max and [51:38] then pass into here values my literal [51:40] values comma dimension equals minus1 and [51:43] remember we have to import at the top [51:46] there [51:48] the f function from the pytorch by going [51:53] import torch.n nn as do functional. So [51:56] import torch dot n.functional [52:01] as f [52:03] like that to be able to use this f in [52:07] here f.s softmax and then pass in my [52:09] values and then specify dimension as [52:11] minus one. Then lastly sampled equals [52:15] torch dot multinnomial. Let's use the [52:18] function multi-nomial [52:20] and then pass into here props my [52:22] probabilities comma one at the end [52:25] return indices [52:28] and then indexing on the sampled like [52:31] that. So you can see sampled we are [52:33] passing this as an index to the indices [52:35] variable and then we have here [52:37] multinnomial function. Okay run that. So [52:41] now we have top k sampling top k [52:44] sampling this function expects from me [52:46] logits. Let's pass my outputs here. [52:51] You can see it chooses the tensor at the [52:55] index 674. [52:58] If we pass this to a tokenizer decode to [53:01] visualize what is what is this token? [53:04] What is this willing token? Well, it is [53:07] hour. So, top case sambling now chooses [53:09] the token hour. If we change slightly [53:13] this function function to enhance for [53:16] example hour, [53:18] it will give me another candidates or [53:20] results into another candidate words. [53:23] Let's run that again to see the this is [53:26] a dynamic function dynamically changing. [53:29] Run that if to change our customer. So [53:33] if you want to see this word into here, [53:36] this is the chosen word customer [53:43] understanding ability work lives. [53:47] If you write into here customer for [53:49] example to enhance our customer then it [53:52] will give me another candidates you can [53:55] see. And then if you choose this [53:56] tokenizer decode run this again [53:59] experience. So one of the words here is [54:02] experience and I saw it here experience. [54:05] Okay, this is the way of operating using [54:09] top K sampling. It just returns the top [54:12] K logets and then samples from it. The [54:16] third step is that we have the top [54:18] probability sampling top B sampling [54:22] and then it takes from me logits [54:25] comma P equals 0.9. And let's write a [54:29] little comment into here to separate the [54:31] code and make it neat by going top p [54:35] which is called also nucleus nuculus [54:39] sampling. [54:41] So uh you can name it nucleus sampling [54:43] or top p sampling. This is a nice method [54:46] to sample from the token pool. It is [54:49] called top p sampling and it returns the [54:52] smallest set of possible tokens whose [54:55] cumulative probabilities add up to a [54:57] specified threshold called P. So let's [55:00] go and write a little documentation or [55:02] documentary here by going star or sort [55:08] sort tokens [55:10] by probability [55:13] keep smallest. So keep smallest [55:18] number whose cumulative cumulative [55:24] probability values probability exceeds [55:29] the threshold P [55:31] then sample then sample one token. So [55:36] this is the the the explanation of what [55:38] this top B sampling does. All it does it [55:42] sorts tokens by probability values. So [55:45] it first the model when it generates [55:47] logits it it it says okay I need to sort [55:51] these tokens based on their probability [55:53] values. Once the probability values are [55:56] sorted from highest to lowest keep [56:00] smallest number whose cumulative [56:03] probability exceeds the threshold P. I [56:05] need to write a threshold exceeds a [56:08] threshold value P. This P is the [56:10] hyperparameter that you need to specify. [56:13] Here I'm specifying it as 0.9 90%. Then [56:18] sample one token. [56:20] So the first thing is we need to sorted [56:23] tokens or logits. Let's call it logits. [56:26] It's better to call it logits comma [56:29] sorted underscore indices equals torch [56:34] dots sort. Let's use the function saw [56:37] and then pass into here logets [56:40] that is passed to the function logets. [56:42] pass it to the function sort sort it and [56:45] then descending. Do you want to sort it [56:47] descending or ascending? Well, in my [56:49] case, yes, I want to sort it in a [56:51] descending order, in a descending [56:53] manner. Then let's go ahead and sort it [56:56] and score probabilities. Let's normalize [56:59] these logets by passing them to the [57:01] softmax function. So all we need to do [57:04] is pass the logits to a softmax [57:07] function. Hence the softmax will [57:10] normalize these logets and make the [57:12] addition or when when you sum these [57:14] values add up to one by going sorted [57:17] underscore logits pass it to the [57:20] function to to the function softmax [57:22] comma dimension equals minus1. [57:25] So we have to specify here minus one. [57:26] Then we have to calculate the to [57:29] calculate the commitive probabilities. [57:30] So cumulative commulative [57:36] props equals sorted [57:40] probabilities dot sum. So I like the [57:43] name of the of this function sum [57:45] meaning cumulative sum. So we have to [57:48] come sum cumulative sum and then pass [57:50] dimension equals minus one. So come is [57:54] short for cumulative sum summation. [57:57] Okay, this is the commutive [57:59] probabilities. [58:01] Then we need to mask tokens outside [58:07] nucleus [58:09] nucleus [58:10] by going mask. So a little masking here [58:13] operation equals cumulative cumulative [58:17] probabilities that is larger than the p. [58:19] So please go ahead and if this [58:23] cumulative properties exceeds P is [58:25] larger than P then mask. So then we have [58:29] here the variable sorted [58:32] logets and then pass the mask equals [58:35] float and then pass into minus infinity. [58:38] So I I want to explain this by going to [58:41] the polar club here because I have a [58:43] little explanation. If you can see we [58:46] can go to polo club transformer [58:50] explainer [58:52] of polo club. You can see when we go to [58:55] the top B probability here we set the [58:58] value as minus infinity. If we choose [59:01] top B [59:04] let the tool work. [59:06] Now it tries to generate as you can see [59:08] data visualization empowers users to [59:10] visualize. And now the tool is trying to [59:13] generate the next token. [59:19] Now if you go to top B top P and then [59:21] probabilities you can see here if you [59:24] choose the method top P and go to [59:26] probabilities it sets the rest of the [59:30] let's make the sampling here for example [59:34] like that. So the rest are here zero 0%. [59:38] As you can see, if you choose top K, [59:41] then we choose the top K five. So the K [59:43] now is set to five. Whereas if you set [59:46] the sampling strategy as top, it sets [59:49] the rest as zero and then it just [59:52] chooses the smallest set of possible [59:54] tokens whose cumulative probabilities [59:56] add up to a specified threshold. For [59:59] example, 0.5. Okay, so the sum here you [60:02] can see the sum is 0.5 [60:05] around that. And if you increase that [60:07] the the sum takes more logets takes more [60:11] tokens until it satisfies the the [60:16] cumulative probability that is you are [60:17] specifying here. So the sum is 0.81 [60:20] close to this value 0.8. Okay. This is [60:23] the idea behind this. The rest we are [60:25] setting them to what? To 0%. Here we are [60:28] setting them to minus infinity. [60:32] At the end let's sample. sample from [60:35] filtered logits by going filtered [60:40] filtered underscore probabilities equals [60:44] f.oftmax. [60:46] So this function softmax and then pass [60:49] sorted logets [60:52] and specify the parameter and dimension [60:55] equals minus1. Then sampled [60:59] equals torch dot multi-nomial [61:03] function and then pass into here the [61:05] filtered probabilities. We can copy that [61:10] comma one and here remember to add a [61:13] equal sign. So sampled equals torch dot [61:16] multinnomial and then pass filtered [61:18] probabilities and then specify the [61:20] dimension as one. At the end return [61:24] return token index in original [61:29] vocabulary vocabulary [61:32] by going return [61:35] and then we have to specify the sampling [61:38] here as sorted indices sorted indices [61:42] and then pass the index here as sampled. [61:46] So this is this is the variable that we [61:48] have to specify inside the square [61:50] brackets the variable sampled [61:53] and this and that is top P sampling. [61:56] That's it. Top P sampling. Now we have a [61:58] function that uses the top piece [62:01] sampling strategy and samples from the [62:04] token pool by keeping the smallest set [62:07] of tokens or of logits whose cumulative [62:11] probability value exceeds a threshold p [62:15] value. [62:17] After we wrote this function top p [62:19] sampling, let's try it together. [62:22] So let's call this function top piece [62:24] sampling [62:27] and then pass into here my outputs which [62:29] are the normal logits. You can see my [62:32] logits. Pass it to the top P sampling [62:35] function and then specify the P as a a [62:38] threshold value equals 0.9. [62:42] Let's run that. As you can see it [62:44] returns a tensor with the value 1,998. [62:50] So I wonder what is the corresponding [62:52] token value or the corresponding word [62:55] value by going token or tokenizer decode [62:58] and then just pass in here the token ID. [63:01] It will simply return the corresponding [63:04] token or word that corresponds to this [63:07] ID value [63:09] experience. So now the top sibling shows [63:12] us the experiences word. If we if we go [63:15] ahead and for example to enhance our [63:18] customer to enhance something let's try [63:21] this together you can see the the [63:24] sentences I learn machine learning to [63:27] enhance then these are the candidates [63:29] performance to enhance learning to [63:31] enhance user to enhance data social [63:34] let's see what is the chosen word using [63:36] the top piece sampling strategy [63:39] to enhance human so the top piece [63:41] sampling thinks that hm The most word I [63:45] believe should be the next token is the [63:47] word human which exists here human. [63:51] Okay. Then if you write here human [63:56] is just another set of tokens to enhance [64:00] human. You can see creativity to enhance [64:02] human health judgment knowledge [64:05] interaction skills. Let's see here what [64:07] is the chosen token or word judgment. So [64:11] as we saw it chooses the word judgment [64:13] but the top K let's compare this to the [64:16] top K earlier here it chooses to enhance [64:19] performance. So the top K sampling [64:21] thinks the most high probable high [64:25] probable value should be performance but [64:27] the top P sampling suggests that [64:30] judgment. So you can see the difference [64:32] here. Top P enhance uh chooses empathy. [64:36] Well uh it's random. So uh when you run [64:40] it multiple times maybe it will change [64:42] its sensitivity to another uh word. Okay [64:47] the top case sampling is [64:49] also random. Okay. So you can see here [64:51] when you run again it changes its [64:54] decision. Now let's go ahead and go to [64:57] the another uh sampling strategy which [65:00] is called temperature sampling. So this [65:02] is one of my favorite strategies. It's [65:06] called temperature temperature [65:09] dampling. [65:12] The temperature is a hyperparameter that [65:14] is specify it and it changes and it [65:17] redistributes [65:19] redistributes the probability [65:21] distribution. What it does is that the [65:23] higher the temperature value is, the [65:26] more creative and the more random the [65:28] model outputs are. And the lower the [65:31] temperature values are, the lower you [65:33] set the temperature to, then the model [65:36] will be deterministic. The model is [65:38] deterministic. And it it isn't it isn't [65:41] creative. It just it just does what it [65:45] thinks is more deterministic. So def [65:48] it's called let's call it temperature [65:50] temperature underscore [65:53] temperature [65:55] sampling and then pass in my here my lo [65:58] jets set the temperature value as a [66:01] default value equals to 1.0. So feel [66:04] free to play around with the temperature [66:06] value. It's fun to change and adjust [66:10] this value to different values the [66:13] temperature. Then let's write a nice [66:15] little documentation here by going [66:18] scale logets. So remember temperature [66:22] always scales logets. It returns a [66:25] scaled version of the logets. Scale [66:28] logets by temperature by temperature [66:32] before sampling. So before it samples it [66:34] scales those logets. [66:39] Then lower temperature. This is a note [66:41] lower temperature. Keep in mind that [66:43] lower temperature values [66:46] leads to sharper [66:48] more deterministic. So sharper more [66:52] deterministic distribution distribution. [66:54] So if you search here in Google sharp [66:57] probability distribution to see its [67:01] image [67:03] go to images [67:05] this is a sharp distribution. So it's a [67:08] sharp probability distribution. But if [67:10] you if you search here flattened [67:12] flattened probability distribution this [67:15] is a flattened one. You can see here you [67:17] can see this these are the different [67:19] probabilities there. We have here sharp [67:21] probability distribution. We have a [67:22] flattened probability distribution. [67:24] Okay. And this is also the image of a [67:26] flattened probability distribution. [67:28] Okay. [67:34] The steps here are simple. We have to we [67:37] have to create a variable called scaled [67:39] equals scaled equals logets and then [67:42] divide the logets by the temperature [67:45] value temperature and then we have [67:47] probabilities props equals f do softmax [67:51] and then pass into here the scaled [67:54] variable scale variable specify the [67:57] dimension as minus one okay negative [68:02] one. [68:03] So these are the uh settings or the the [68:07] steps. First we divide the logets by a [68:10] temperature value. This value you [68:12] specify it as 1 0.5 0.3. Then we take [68:17] those scaled logets and pass it to the [68:20] softmax function to normalize it and [68:23] then store this returned value into a [68:25] variable called props. At the end we [68:28] have to go ahead and return torch do [68:31] multi-nomial [68:33] and then pass probabilities props comma [68:36] one. [68:37] And that's it. We have a temperature [68:39] sampling strategy. Let's try it together [68:42] by going ahead and call this function [68:46] sampling [68:49] or temperature temperature [68:52] underscore sampling and then pass in my [68:54] outputs. [68:56] as you can see and then let's specify [68:58] the temperature value. Temperature [69:01] equals to let's try first 1.0. So let's [69:04] try to set the temperature as one. So [69:07] remember when you divide anything by one [69:10] it will be the same. So if you take the [69:12] logist and divide it by one then it's [69:15] the same. Okay let's go ahead and run [69:18] that. Nice. Now we have a tensor of [69:22] 4,430. [69:24] I wonder what is this corresponding [69:26] token value or word value. Let's go [69:29] tokenizer do decode. Pass it to the [69:32] decode function. [69:35] Run that. It's performance. Okay. So, [69:38] let's go ahead and see the candidates [69:41] here. These are the candidates. We have [69:43] performance. Now, if we change the [69:46] temperature value a little bit, for [69:48] example, let's set the temperature to a [69:51] lower value. For example, let's set it [69:53] to 0.1. [69:55] Now remember, lower temperature values [69:58] leads to more sharper outputs, leads to [70:02] more deterministic outputs, less [70:04] creativity. Run that intelligence. So [70:08] you can see now the model chooses the [70:10] word intelligence as the next token. [70:13] Why? Let's read the sentence again. I [70:15] learn machine learning to enhance human [70:17] intelligence. So it it is less creative [70:20] and it chooses the word intelligence. [70:23] But if we increase the creativity rate [70:25] of the model by going ahead and set the [70:27] temperature value to for example 0.5 [70:31] cognition. So you can see the model now [70:33] is thinking in in complex manifolds it [70:35] chooses the word cognition. You can see [70:38] go ahead here this is the word that it [70:41] chooses cognition. Okay. And if you [70:45] increase the temperature further and [70:47] further 0 [70:50] 9 for example physical. So the the model [70:53] now is random and choosing uh creative [70:56] words choosing a very hot words. You can [71:00] see here I learn machine learning to [71:03] enhance human physical let me write [71:06] physical physical [71:10] for example abilities or performance [71:12] health functions capabilities let's see [71:14] what it chooses to enhance human [71:17] physical function okay and the [71:20] temperature I think the uh you can [71:22] increase further 1 for example three it [71:25] chooses and okay if increase further to [71:28] 1.5 it chooses Enjoy. So the model now [71:32] chooses the word enjoy as the next token [71:35] that it thinks the most probable word. I [71:38] learn machine learning to enhance human [71:39] physical enjoy. Enjoy. [71:43] Okay. Like that. Then we have several [71:46] several candidates here. It it it will [71:48] choose one of them [71:51] vulnerabilities at the temperature 1.5. [71:54] So these are the different sabling [71:55] strategies. Okay. And then at the end we [71:57] have the last uh subling strategy which [72:00] is called random sampling. So this [72:03] strategy is isn't common. You will not [72:05] you will see it rare in the world of [72:07] machine learning. You will not use it as [72:10] common as the the previous ones but it's [72:12] called random sampling. Random sampling. [72:18] And then let's write the function [72:20] define. It's called let's call it [72:22] random. Let's name it random sampling. [72:24] Then pass into here my logs. [72:29] Then specify a little documentation [72:31] here. [72:33] So your your future self or someone else [72:36] comes here, he doesn't confused. He [72:39] doesn't get confused what this function [72:40] does. He is he has a little explanation [72:43] what this function does. So if you come [72:45] back later after 6 months for example or [72:47] or after one year and you come back to [72:49] these functions, well I don't remember [72:51] what this function does. No, you don't [72:53] get to this situation. You have a [72:55] explanation that tells you what this [72:58] function does. This is the benefit of [73:00] writing a documentation. Sample directly [73:03] directly [73:06] from softmax distribution distribution [73:11] without filtering. So it samples [73:13] directly with softmax or from softmax [73:16] distribution without any filtering [73:19] strategy. props equals torch or f let's [73:22] use f dots softmax and then pass glogets [73:26] comma dimension equals minus1 then [73:30] simply return return torch dot [73:33] multinnomial [73:35] and then pass my probabilities one this [73:38] and that's it now you have a random [73:40] sampling strategy let's run that let's [73:43] call it together and see what happens [73:46] random sampling and then pass in my [73:49] outputs Here as you can see it returns a [73:52] you can see when I run that run it again [73:56] it changes a decision so every time it [73:59] chooses another uh token it's random so [74:03] tokenizer tokenizer do decode let's [74:06] decode it let's see what is the word uh [74:10] meant meant of I meant let's change the [74:15] sentence [74:19] I learn machine learning to enhance so [74:23] to to trigger GBT to to complete the [74:27] sentence in a correct way to to enhance [74:31] every day to enhance my to enhance my [74:34] doctors to enhance abilities our so it's [74:37] a random sampling strategy it's a random [74:39] sampler sample directly from softmax [74:42] distribution without filtering what it [74:44] does we pass the logist this to softmax [74:46] function. It normalizes these logets. [74:49] Then we pass these probabilities, these [74:51] probabilities to a multinnomial function [74:53] and return that. And that's it. We have [74:56] now a uh four methods, four [74:59] methodologies to sample from the token [75:02] pool. [75:04] Let's try to create another another [75:07] sentence. For example, sentence [75:11] equals the sentence today I decided to [75:17] go to the and let's go ahead inputs [75:21] equals tokenizer. So let's tokenize the [75:24] sentence. Remember you have to [75:26] pre-process the sentence before passing [75:28] it to the chity model by passing this [75:30] sentence to the tokenizer. [75:33] Comma return tenses or return tenses [75:37] equals PT short for PyTorch. Then we [75:40] have output equals GBT2 and then pass [75:44] inputs here. So these are the input ids. [75:46] These are ids. We pass them to the GBT2 [75:49] model and and remember you have to put [75:53] two asterisks here to to unpack it. then [75:57] logs equals output dot loss and then [76:02] pass 0 comma minus one to index to the [76:05] last word. [76:06] So at the end let's print here print f [76:10] string a little fing and try to print [76:14] the first method the first methodology [76:16] which is greedy decode [76:19] as then inject the function tokenizer do [76:23] decode. So we need to decode to decode [76:26] to the corresponding token by passing [76:29] the list greedy decode call this nice [76:33] function and pass into here my logits [76:36] then the same idea print fing let's name [76:39] it here top k sampling then print the [76:43] function or inject the function [76:45] tokenizer decode tokenizer [76:48] make sure we read that right dot [76:51] tokenizer [76:52] decode and then pass here a the function [76:55] top k top_k [76:58] sampling and pass my logest into here [77:02] specify the k feel free to change it to [77:05] your own k I will set it by default to [77:07] 10 [77:09] then let's go ahead and print the third [77:11] method which is top p sampling and the [77:14] top p sampling remember it is called [77:16] also the nucleus sampling let's go [77:18] tokenizer decode and then pass into here [77:22] the function [77:23] top_p [77:25] sampling and then pass to here logets [77:28] comma p specify the p as you would like [77:31] but in this case I would keep it simple [77:33] and pass 0.9 the fourth methodology is [77:36] remember the temperature then we have to [77:39] pass into here the function tokenizer do [77:42] decode and then pass into here the [77:45] temperature temperature [77:48] sampling and then pass into here loget [77:52] and then specify the temperature with [77:54] any value between 0 and one. I will keep [77:56] it here 0.7. [78:00] Then at the last we have the random [78:01] sampling methodology which is a simple [78:04] tokenizer decode a simple uh sampling [78:07] strategy random sampling and then pass [78:10] in my logistics. And with that being [78:12] said we can run that again. So let's go [78:14] ahead and 3 2 1 run it. And I'm excited [78:19] to see that we have no typos. We have no [78:22] errors as you can see greedy code local. [78:25] So the greedy decode chooses the the [78:27] word local. Top case sampling chooses [78:29] doctor. Top piece sampling chooses [78:31] lesson. Temperature chose nearest. And [78:35] you can see random chose military. Let's [78:38] change the uh sentence here. I today I [78:40] decided to go to the local doctor [78:43] lesson. Let's choose for example local. [78:46] Then how this would change the outputs [78:48] here. Run that. Here we can see the [78:50] greedy chooses library. So I today I [78:53] decided to go to the local library. Top [78:55] case choose library. Topy uh library. [78:58] Okay library. [79:01] And you can see they they are all now [79:02] unified. They chooses the word and and [79:05] here we have uh the gre said read and [79:08] it's reasonable because I go to the [79:10] library then I would do the what I would [79:13] do there is reading. But the top K here [79:16] says find. So let let me go with find [79:18] and find you can see find out. Yeah [79:21] because you find find out in English we [79:24] always put the word out after a find. So [79:27] it is reasonable then find out what. So [79:31] the greedy says what the top case [79:33] sampling says the top piece says what. [79:35] So let me you can see that the greedy [79:38] now is equal to the top p sampling [79:39] strategy. Find out what [79:43] what was what they what was let me say [79:46] what was [79:49] going. Okay. So you can see the top P [79:51] here says in what was in for example in [79:54] the [79:56] in there or in my for example in my [80:00] wallet. You can see now we have the word [80:03] wallet as the top case sampling. The [80:05] temperature says wallet al uh also the [80:07] temperature says wallet. Let me go ahead [80:10] and you can see now they are all saying [80:12] that we need to put a dot here. Then if [80:14] you run it again they they would say I. [80:17] Okay. And as another example quickly uh [80:20] sentence equals I am [80:25] really [80:26] happy because [80:30] you can see I ate they let me say [80:32] because I [80:34] have for example then you can see the [80:38] temperature I have gone it uses gone [80:42] back [80:44] and if you change the temperature here [80:46] you can feel free to play around with [80:47] that make it 0.1 it will say temperature [80:50] two but if you make it here for example [80:53] one really really creative it will say [80:56] temperature in I will go I have gone [80:59] back in time you can see I am really [81:02] happy because I have gone back in time [81:05] okay and feel free to change and adjust [81:07] the sentence and also change the K the P [81:11] the temperature and play around with [81:12] that and always remember you have this [81:15] nice tool to visualize analyze and learn [81:17] about top P, top K sampling and also the [81:20] temperature here. If you if you want to [81:22] play around with the temperature, you [81:23] can see let me write here the the next [81:26] sentence. I like machine learning [81:30] because [81:33] then try to generate now it is uh it [81:36] will be fast. You can see the tool is [81:39] already loaded. Then we have here [81:40] temperature. Now if we make the [81:42] temperature 0.1 you can see the [81:44] distribution here is really sharp. It's [81:47] really just pointing to one token. The [81:50] rest are zero. But once you increase the [81:53] temperature, [81:55] it it distributes that value. So it [81:57] doesn't make the the distribution goes [81:59] towards one token, but rather it makes [82:02] the distribution more flattened flatter. [82:04] So if you increase it, it gives some [82:07] proportions to the next and to the and [82:10] it chooses another tokens other tokens. [82:12] So it gives some some uh possibility to [82:14] choose other tokens. Okay, rather than [82:16] just uh centering all the mind towards [82:20] one token, we give some opportunity to [82:23] other tokens to show up. Okay, as as you [82:26] can see the sampling here methodology is [82:28] K. Let me increase the K. So you can see [82:30] the K here is five. So just you want to [82:33] keep five tokens. The rest are [82:35] discarded. But let me make it for [82:38] example 17. Now you can see or for [82:41] example [82:43] six. You can see 1 2 3 4 5 6. If you [82:46] increase that again 10. Now we have 10 [82:49] tokens. The rest are discarded. We just [82:51] skip 10. And also if we go to the top P, [82:54] now you don't have here K, but rather [82:56] you have a P, which is the cumulative [82:58] sum, the cumulative probability. And it [83:01] chooses the smallest set the smallest [83:03] possible set of tokens whose cumulative [83:05] probabilities exceeds a threshold P. For [83:08] example, let me make the the threshold [83:10] 0.2. Then these are all makes up to uh [83:14] 0.2. And if we go ahead and increase [83:18] that, you can see it chooses different [83:21] tokens. Uh let me decrease the [83:23] temperature a little bit 0.5. You can [83:26] see here when we make the top P 0.3, it [83:29] chooses 100% I. The rest are discarded. [83:33] If we increase that again, it will [83:36] choose we here because we and if you go [83:39] for example like that increase it, it [83:42] will redistribute the probability [83:44] distribution. So these are different [83:46] sampling strategies that are employed to [83:49] manipulate the distribution or the [83:51] probability distribution of the token [83:54] pools and changes the decision of models [83:57] like GBT and other any other model that [84:00] is considered a generative model. So [84:02] keep in mind you can feel free to play [84:04] around with these values in our collab. [84:06] So now we have finished about GPT and [84:09] generation and let's dive into another [84:12] topic within the transformers library. [84:16] Now I have a little code that visualizes [84:19] the probability confidence rates [84:22] associated with the corresponding [84:24] tokens. So to write this for loop first [84:27] we need to construct a sentence sentence [84:31] and then set this sentence variable to [84:33] any sentence you would like. For example [84:35] I would say I learn machine learning to [84:39] enhance. [84:41] Then I will ask GB to complete that. Now [84:44] let's go ahead and pass this sentence to [84:46] the tokenizer by going token ids equals [84:49] tokenizer [84:51] tokenizer dot and then pass into into [84:54] here sentence [84:56] return tenses equals pt pytorchinput [85:02] ids. Now if you visualize this token [85:06] ids you will see that we have here ids [85:09] just a normal tensor a simple tensor [85:11] with these ids 40 200 and it all now [85:16] let's pass these ids to the gt by going [85:18] output equals gpd2 and then pass in my [85:21] token ids the gb here will return if we [85:26] get this output as output you will see [85:30] this whole output which which which [85:33] consists of logits plus also dynamic [85:36] layer, dynamic layer and so on. We need [85:39] just to fetch the logs just to retrieve [85:41] the logist. So let's go ahead and tab [85:44] into dot logist. So we need just to [85:47] fetch the logits. [85:50] If you print that you can see now we [85:52] just have the logits the raw [85:54] unnormalized scores that the model [85:56] outputs. So these are raw values raw. [85:59] Okay, we can see here row unnormalized [86:02] scores or values. [86:06] Now let's go ahead and go to outputs. [86:09] Override that again equals torch dots [86:12] softmax and then pass the outputs here. [86:16] But keep in mind we have to index and [86:17] get the last out item. So if you do that [86:20] again like that outputs [86:23] and visualize it 0, minus one. Well, [86:26] zero will get rid of this outer bracket. [86:29] So here if you goes at index zero, [86:32] it gives me here these brackets. Then if [86:36] you go comma minus one, it will fetch [86:39] the last token, the last word. So as we [86:41] said, we have 1 2 3 4 5 6. We have six [86:45] words. I learn machine learning to [86:48] enhance. So we have six words here. We [86:50] have six vectors. 1 2 3 4 5 6. Inside [86:57] every single vector, how many values do [86:59] we have? We have 50,257 [87:03] logits or values. Why? Because when GPT [87:06] takes this sentence, it rolls out on it [87:09] and then predicts 50,000 tokens or [87:12] 50,000 probabilities or logits for every [87:14] single word. Okay? And then we we need [87:17] to access the last token the last word [87:20] which is if you go ahead outputs we have [87:22] to put s here you can see we have the [87:24] last token the last item. Now we need to [87:28] call the function as you can see torch [87:30] dos softmax on this. Now note with me [87:34] what would happen to these values and [87:36] then we have to pass here dimension [87:37] equals minus one 3 2 1. Note that these [87:42] are negative values. What would happen [87:43] to it now? They are converted into what? [87:47] Into positive values. Some values are [87:49] really small. You can see they are they [87:51] are multiplied by the ula number raised [87:53] to the power of negative6. And some [87:55] values are really high. The most high [87:58] value you can get it by going to argmax [88:01] argmax. Okay. And then call the argmax [88:04] function here. [88:06] You can see the most high value is at [88:10] index 674. But I don't want to get this. [88:12] I need to store all of these values [88:15] inside the outputs variable and override [88:18] on it. Okay, after that let's go ahead [88:21] and get the top 10 equals torch do top [88:24] k. This function is a really helpful [88:25] function inside torch. It's a really [88:28] helpful function. Let's pass into here [88:30] outputs, specify the k as 10 since I [88:33] named the variable top 10. If you [88:36] visualize top 10, we have a nice top 10 [88:38] values here. So that's really cool. [88:41] Let's go ahead now and do a for loop for [88:43] index comma value in zip and then pass [88:45] into here top 10 dot indices [88:49] comma top 10 values. [88:52] Let's go ahead and print into here [88:54] tokenizer tokenizer decode and then pass [88:58] into here the index dash dash put a [89:01] curly brackets here injected values and [89:04] then get there just one number after the [89:09] uh the dot. Okay, then put the percent [89:12] sign. Now let's go ahead and run that 3 [89:15] 2 1 [89:18] values isn't defined. So we have to yeah [89:21] we need to call this variable value just [89:23] one value. So here note we have values [89:26] but here value is referring to this [89:28] variable. So we unpack this zip object [89:31] into index. So the indices the indices [89:34] value gets returned to index and the [89:36] values value gets returned to value. [89:38] Then we loop loop loop and then print [89:40] that print those outputs. And then [89:43] hooray now we have the tokens associated [89:47] with their confidence rates. So juby [89:50] thinks that the most probable word is [89:53] our. So if you read the sentence again I [89:55] learn machine learning to enhance our [89:57] our is is inserted is being inserted [90:00] 11.5%. If you choose now to enhance for [90:03] example our [90:06] understanding is 4.3% ability is 3.9%. [90:11] Let's go to enhance our understanding. [90:13] Uh feel free to choose any word from [90:15] this list. You can you can actually [90:17] choose any word understanding. Then we [90:19] have you can see here of 91% soy thinks [90:23] that 91.8% [90:26] the this word of is should be the next [90:28] token. [90:31] Then if we go the how you can see how [90:34] human we can choose now uh of them for [90:36] example the world human brain. So we can [90:40] say to to enhance our understanding of [90:42] the brain for example [90:45] then we can see we have here comma and [90:47] comma in let's choose for example in [90:53] the order in ways in general and so on [90:55] and so forth. So here we have the tokens [90:57] associated with their confidence rates. [91:00] GBT is assigning probabilities or [91:02] probability distribution to these [91:04] tokens. Okay. And this is how you can [91:06] visualize and print what is the [91:09] confidence rate for for every single [91:11] token. So not only choosing the next [91:13] token using top K and top B, but also [91:15] visualizing the confidence rates. And [91:18] this is a really cool feature you can [91:20] have in the Python library uh of the zip [91:24] function and also how to uh print the [91:27] index passing it to the decode function [91:30] and get the value. [91:32] Let's dive into the second task of NLB, [91:35] natural language processing, which is [91:38] the task of sentiment analysis. So, [91:41] let's name this as a title, sentiment [91:44] analysis. [91:46] Now, we need a dips set. We need a dips [91:48] set to be able to classify the the text [91:52] as positive or negative or even neutral. [91:55] Let's go to hugging face and then go to [91:58] their website and search into here. Go [92:01] to dets and let's go to the filter by [92:04] name and search for IMDb [92:08] by Stanford NLB. [92:11] Then straight away go to use this data [92:14] set and then you can see we have a [92:17] library called data sets. Click on it. [92:20] Then let's copy this code. You can see [92:22] here this code is simple from dets [92:24] import load data set. Then we've used [92:28] this function and pass into here the [92:29] name the username of the account [92:31] Stanford NLB/ IMDb. Here you can see we [92:35] have a nice visual of the samples. You [92:38] can see here we have some text and its [92:40] label negative. So ne is short for [92:42] negative and then we have positive. You [92:45] can see we we can scroll down and see [92:47] all the samples. Then we have many [92:49] pages. Page one, page two. Feel free to [92:52] explore this data set in your own time. [92:54] Keep in mind you have to become one with [92:57] the data in order to be able to model [93:00] it. Okay. Then we have here if you [93:02] scroll down you can see we have if you [93:05] go to this IMDb again [93:08] we have here also a way how to uh use [93:11] the data sets using code. So let's go to [93:13] our collab and paste this code. Run it [93:16] away. [93:21] Now we have here readme.md which is a [93:24] file readme.m markdown. You can see [93:27] plain text we have this data. The data [93:29] now is being loaded and downloaded. [93:33] So all we've done is we've copied it [93:35] from from hugging face and straight away [93:38] pasted into the into the Google collab. [93:40] Now after this code finishes loading, [93:43] let's go ahead and get the type of this [93:46] data s data set which is DS. [93:50] You can see now it's loading. [93:53] Now it's loaded successfully. Now let's [93:55] go ahead and type this data set. Now [93:57] since my variable name here is DS, I'm [94:00] going to use DS. But you can feel free [94:01] to name it whatever you want. So here [94:04] the type of this data set is data sets. [94:07] Dictionary dot data set dict. So the [94:10] type of this data set is what is data [94:12] set dictionary. Let's go ahead and go [94:15] here data set and then let's access this [94:18] data set as this. [94:21] You can see we have training data set. [94:23] We have testing. Then we have [94:24] unsupervised. So we have the train data [94:26] set. Test data set and unsupervised. The [94:29] train data set is a data set object or a [94:32] data set class. Then we have features [94:34] text. Every single text is associated [94:36] with a label positive, negative, [94:38] neutral. And the number of rows here is [94:41] 25,000. And for the test data set, here [94:44] we have features. We have text column, [94:46] label column or text text key and the [94:49] label key. Then we have num rows as [94:52] 25,000. [94:54] Okay, [94:55] let's go ahead and now explore this set [94:58] in your own uh in in our own way. The [95:01] first thing is we need to go DS and then [95:04] choose the key train. [95:08] You can see now we are indexing and [95:10] fetching the key train from this DS [95:12] variable. You can see we have a del [95:14] class inside we have features and we [95:16] have num rows. Let's go ahead and try to [95:18] pass this into our import pandas as pd [95:26] and then let's go ahead and pass this ds [95:30] train into a pandas column. So data set [95:33] and then choose train column or train [95:37] key dot2 pandas. [95:41] So now we are converting the data set [95:43] into a pandas data frame. You can see we [95:45] have a nice little data frame text. I [95:49] rented I am curious yellow from my [95:51] video. This is the text. Then label is [95:53] zero negative. So the label is zero or [95:55] one. Zero for negative one for positive. [95:59] Now let's go ahead and try to [96:03] index on this data frame by going my [96:06] data data set [96:09] frame data frame equals DS2 [96:14] pandas and then let's fetch the key [96:17] train by this way. Then we have we can [96:20] go my data set data frame and then [96:23] access the column text. You can see we [96:25] have the column text. Okay. And if you [96:28] want to know how many samples do we [96:30] have, we can go ahead and fetch the [96:32] length of my data set the frame and then [96:36] choose the key the the the column text. [96:39] We can get the length as 25,000. [96:42] Now we need a model to classify the [96:45] sentiment. So from transformers [96:49] import [96:51] the pipeline function pipeline. [96:55] Then let's go ahead and import that. The [96:57] second step is we need a model. So let's [97:00] go pipe equals or you can name it also [97:02] classifier. Let's use classifier equals [97:04] pipeline and then pass into here the [97:07] task you want to conduct. In this case [97:09] is sentiment sentiment analysis. Also [97:12] you can write into here text [97:14] classification like that text [97:16] classification. It will be the same but [97:18] in this case I want to use the task [97:20] sentiment- analysis. Then let's go ahead [97:23] and and you can specify the model here. [97:25] Feel free to choose any model. You can [97:27] go to models and choose whatever model [97:30] that is used for sentiment analysis. You [97:32] can go to tasks NLB and then go to text [97:36] classification. It's the same as [97:38] sentiment analysis. Then you can choose [97:41] any model. If you go to to sort the [97:44] models here and choose most downloads, [97:46] you will see that it will it will be [97:47] dist which is the default. So let's keep [97:51] the default here. [97:52] And straight away run this as this like [97:56] it like this. You can see now the model [97:58] is being downloaded model. Tenses [98:01] tokenizer_config.json [98:03] and we have vocabulary text device set [98:06] to CPU. So now the model is loaded. [98:08] Let's go ahead and use this model on our [98:10] text. So we have so we can go classifier [98:13] and then pass into here my text as this [98:17] day is great. [98:22] the classifier will work and it will [98:24] label it as positive. So you can see now [98:26] we have a positive label for this string [98:29] and the score the confidence rate of the [98:32] model is 99%. [98:34] So let's go ahead and also uh do another [98:36] example classifier [98:38] and then pass here the string this day [98:41] is terrible [98:43] is terrible [98:45] and I am so sad [98:48] and the model will classify it as [98:50] negative. Hooray we have label negative [98:52] and the score is 99%. [98:55] So you now you have a ready classifier [98:59] that is uh performing really well pretty [99:02] well. You can see uh the performance [99:04] rate here is really high. The model is [99:06] really is is really confident. Now let's [99:10] use it on our uh text data set which is [99:12] IMDb from Stanford NLB. [99:16] Let's create a function called def [99:18] score. This function take takes from me [99:21] a text a sample text. You can name it [99:25] review text as review text. Then this [99:28] function will return the classifier and [99:32] this classifier will work on what on the [99:33] review text. But we need to specify the [99:37] limit here as to 500 because this will [99:41] beert the maximum length is 512. Then we [99:44] can go at index zero. Then we can fetch [99:46] the label. We need just the label. So [99:48] here the same idea you have a list [99:51] inside this is we have a dictionary. [99:53] What if you want to just return the [99:55] label? Well that's easy. index at index [99:57] zero. It will fetch only the dictionary. [100:00] Then after that we have a dictionary. [100:02] Now how to grab the label? We can go [100:04] label and it will step into that label [100:07] as negative. Now here we need to pass [100:10] this function by doing data set or my [100:13] data set [100:15] data frame and then choose the column [100:17] text or let's name a new column called [100:20] model prediction. the model prediction [100:24] equals [100:25] the data set or my data set [100:29] data frame and then choose the column [100:32] text dot apply. We have to apply the [100:35] function score. So here all we are doing [100:37] is we are applying this little function [100:40] score. This is a really helpful [100:42] function. we are applying it on our text [100:45] column and assigning the value of this [100:48] function the return value which is the [100:50] label whether it was negative or or [100:52] positive we are assigning it to a new [100:55] column called model prediction. So let's [100:57] straight away go ahead and run that. Now [100:59] keep in mind it might take a little bit [101:01] of time because we have so many samples. [101:03] As we saw we have 25 rows. So it might [101:06] take a little bit time. We have to wait [101:09] until it it generates those predictions. [101:14] Now in the meantime we can go to distal [101:16] B to the paper distal B paper we have [101:21] here this B you can see a distilled [101:23] version of BERT smaller faster cheaper [101:26] and lighter as transfer learning from [101:28] large scale pre-trained models becomes [101:30] more prevalent in NLP operating these [101:33] large models in on the edge. So keep in [101:36] mind this this DLER is really a small [101:38] model but it's faster, smaller, cheaper [101:41] and lighter. So it is it has a [101:44] state-of-the-art performance but also it [101:46] is pretty fast. So it can be uh it can [101:49] be it can operate on the edge on the [101:51] edge. Okay. Under constrained [101:54] computational training or inference [101:56] budget remains challenging. In this [101:58] work, we propose a method to pre-train a [102:01] smaller general purpose language [102:03] representation model called distlbert [102:05] which can then be fine-tuned with good [102:07] performance on a wide range of tasks [102:10] like its larger counterparts and we saw [102:12] that it is performing already pretty [102:14] well. While most pry work investigated [102:16] the use of distillation for building [102:18] task specific models, we leveraged [102:20] knowledge distillation. So distillation [102:23] is a one of the brilliant ideas in the [102:26] world of deep learning. It was proposed [102:28] by Jeffrey Hinton. Distillation is you [102:31] train a student model to behave like a [102:33] teacher model. So rather than using a [102:36] large model, a pretty massive model, you [102:39] train a student model that tries to [102:41] mimic the output layer of a teacher [102:44] model during the pre-training phase and [102:46] showed that it's possible to reduce the [102:48] size of of of a B model by 40%. So they [102:52] reduced they reduced the size of a BERT [102:54] model of the original BERT model by 40%. [102:57] While retaining 97% of its language [103:01] understanding capabilities and being 60% [103:04] faster to leverage the inductive biases [103:07] learned by larger models during [103:08] pre-training. We introduce a triple loss [103:11] combining language modeling distillation [103:13] and cosine distance losses. Our smaller, [103:16] faster, and lighter model is cheaper to [103:18] pre-train when and we demonstrate its [103:21] capabilities of for ondevice [103:23] computations in a proof of concept [103:26] experiment and a comparative ondevice [103:28] study. So by the way feel free to read [103:30] through this paper. You have this paper [103:32] here. You can see here this [103:34] visualization they are comparing distal [103:36] bird of hugging face by other models [103:39] like we have GBT. We have here Bert [103:41] large transformer Elmo GBT2. We have XLM [103:45] by Facebook or Meta and we have your [103:47] Nvidia. You can see dist is a really [103:50] fast model and really performing well. [103:52] Okay. [103:54] And you can see here also uh they they [103:57] present their work here in in the [104:00] present work. We the student Bert has [104:02] the same general architecture uh as [104:05] BERT. The token type embeddings and the [104:08] pooler are removed while the number of [104:10] layers is reduced by a factor of two. [104:13] And you can read this in your own time. [104:15] But now let's get back to our collab. [104:17] You can see it's still running. Now the [104:20] loading has finished. We have applied [104:23] the function score on the column text [104:25] successfully. And as you can see here, [104:28] we have now a column called model [104:30] prediction ready to go. Let's now go [104:32] ahead and my data set data frame and [104:35] show this data frame. Now, hooray. We [104:38] have a column called model prediction [104:41] that shows negative, negative, negative, [104:43] positive. Shows the labels that the [104:46] model has made on our text. So, let's [104:49] compare the model predictions to the [104:52] ground truth labels by going choose the [104:55] model prediction. We can choose both [104:59] both columns [105:01] label comma model prediction. [105:07] Now you can see here we have label and [105:09] we have model prediction. Now let's show [105:13] the first 20 rows. [105:17] As you can see here we have label zero. [105:19] The model has got it right as negative. [105:22] 0 negative zero negative. Then we have [105:24] here zero. The model hm has made it [105:27] wrong. The model get it wrong and [105:29] classified positive. But in fact the [105:32] label is zero. But the model prediction [105:34] here is positive. And here we have 0 [105:36] negative 0 negative negative. And feel [105:39] free to compare the labels to the ground [105:43] truth or compare the model prediction to [105:45] the ground truth label in your own time. [105:47] But here as you can see we have the this [105:50] data frame my depth set data frame and [105:54] it has two columns. It has the label and [105:57] model prediction. And here we have the [105:59] text. For example, if you show the first [106:01] row by going do I lock zero. [106:06] Here we have 0 comma 1. [106:10] If you go zero here we have the text. [106:12] Let's get the text. [106:14] This is the text of the first sample. We [106:17] can print it to see what it what is what [106:19] what it looks like. I rented I am [106:22] curious hello from my video store [106:24] because of all the blah blah blah blah. [106:27] Now what if we pass this text? So this [106:30] is a review. Let's pass it to my model [106:33] by going classifier and then pass into [106:36] here the review. You can see [106:40] the review. Yeah, we need to get rid of [106:42] this print function. [106:45] Now we pass in the review to the [106:47] classifier and the classifier now took [106:50] care of it and predicted the label as [106:53] positive. The score is 78%. Now to get [106:56] the label we can go label. We can go add [106:59] index zero to fetch or retrieve this [107:02] dictionary. Then we can tab into the [107:04] column or the key label. [107:08] It is positive. Okay. The ground truth [107:11] for it. If we go to my data set data [107:14] frame, this is the data frame. Then we [107:17] can choose add index zero. We get the [107:19] sample zero dot eyelock. [107:24] Here we have the the text sample the [107:26] first text sample. The label is what? [107:28] Zero. And the model has predicted that [107:30] it is positive. Okay. So you can see the [107:32] model is getting some are right and some [107:34] are wrong. Okay. But here in this frame [107:37] we have made the predictions [107:38] successfully. Now let's go to the next [107:41] step where we import another model. So [107:44] if you go to hugging face [107:48] here in the tab models if you search for [107:51] a model called finnbert. Now I was [107:54] exploring the uh models for sentiment [107:57] analysis and I found this model which is [108:00] called Finn Bert for financial B. You [108:04] can see Fenbert is a pre-trained NLB [108:06] model to analyze sentiment of financial [108:09] text. It's built by further training of [108:12] the BERT language model in the finance [108:14] domain. So the author of this model uh [108:18] took Bert architecture the same BERT [108:20] architecture but fine-tuned BERT in the [108:23] finance domain used some financial [108:25] documents and fine-tuned it on this [108:28] financial documents and uh financial [108:32] documents using a large financial corpus [108:34] and thereby fine-tuning it for financial [108:37] sentiment classification. You can see [108:39] now we are in the in the field or in the [108:42] task sentiment analysis. Let's go ahead [108:44] and use this model from the [108:46] transformers. Here we have the higher [108:48] level helper and we have the the [108:51] specified uh class and functions. Let's [108:53] use the higher level helper. So to use [108:56] this we can go here. Then we can say [108:58] from transformers [109:00] import pipeline. [109:03] Then we can go pin [109:06] equals pipeline and then pass into here [109:10] the name of the task. As we said [109:12] sentiment- [109:14] analysis then we can specify the model [109:16] here as the name of the user slash the [109:21] name of the model. So here we can copy [109:23] this and paste it over here. And that's [109:26] it. Now it's loading Finn Bert a B model [109:31] archite [109:35] in the financial domain and it's capable [109:38] of classifying the financial documents. [109:40] So let's go ahead and [109:44] and try some to to predict on some [109:47] financial sentences by going here. You [109:49] can see for uh we have the the sentence [109:54] equals the company [109:57] reported [109:58] a strong [110:01] increase [110:02] in quarterly in quarterly. So you can [110:06] quarterly [110:08] revenue revenue comma exceeding [110:12] exceeding market expectations. So this [110:16] is a sentence and you can see it's [110:18] clearly its meaning is indicating that [110:21] it's a positive financial sentiment [110:24] analysis. So the company reported a [110:26] strong increase in quarterly revenue [110:29] exceeding market expectations. If you [110:32] pass this sentence to Fenbert. So now [110:34] use Finnbert model and then pass my [110:37] sentence sentence and then go ahead [110:40] Finnbert will classify it label as [110:43] positive because why? Because this [110:46] sentence is referring to a positive [110:48] indication or positive intent. So the [110:50] Finbert is using its fine-tuned [110:53] parameters to classify this sentence and [110:56] it got it right. This is a positive [110:58] sentence. The score is 95%. And the [111:01] paper for Fbert is this paper. You can [111:04] see Finn Bert [111:06] by Dogo Arashi. Financial sentiment [111:09] analysis is a challenging task due to [111:11] the specialized language or and lack of [111:13] able data in that domain. General [111:16] purpose models are not effective enough [111:18] because of the specialized language used [111:20] in the financial context. We hypothesize [111:23] that pre-trained language models can [111:25] help with this problem because they [111:27] require fewer labeled examples and you [111:30] can and you can feel free to read that [111:31] in your own time. But the most important [111:33] idea we introduce Finn Bert, a language [111:36] model based on BERT to tackle NLB tasks [111:40] in the financial domain and you can find [111:42] it here on hugging face. Let's try [111:44] another example with a with a negative [111:46] sentence by going sentence equals [111:50] shares [111:51] fell after the firm reported [111:56] lower than expected. So you can see the [112:00] the fine-tuned language here lower than [112:04] expected. So you need a fine-tuned model [112:06] trained on massive corpus of financial [112:09] documents to be able to work and deal [112:11] with these uh kind of uh very particular [112:15] terms terminology and then earnings. So [112:18] here shares fell after the firm reported [112:22] lower than expected earnings. Let's pass [112:24] this to my friend Fenbert and then pass [112:27] my sentence then go ahead. You can see [112:30] the label is negative because why? [112:32] Shares fell after the firm reported [112:35] lower than expected earnings. And the [112:38] and the score here is 97%. The model is [112:41] confident by 97% [112:43] that this sentence belongs to the [112:46] negative class or the negative uh group. [112:49] And keep in mind you can also deal with [112:52] batch deal with several sentences in one [112:55] in one time by going sentences equals a [112:59] list. Then let's pass here several [113:01] sentences. The first sentence is strong [113:04] consumer [113:06] demand drove record [113:10] sales across all regions. [113:17] The next sentence is supply chain [113:21] disruptions disruptions [113:24] severely [113:25] severely [113:28] affected [113:29] production [113:31] output. And let's pass these uh this [113:34] list into our fin classifier. Financial [113:37] BERT pass my sentences and 3 2 1. You [113:42] can see now we have two different [113:45] predictions because why? because we have [113:47] provided the model with two different [113:49] sentences. Hence the model took that [113:52] took those sentences and made a [113:54] prediction for every single sentence on [113:56] its own. The first is positive and the [113:59] second is negative. With that being [114:01] said, let's move on to a third task [114:04] which is in this case named entity [114:08] entity recognition recognition. [114:11] So this task is one of the most common [114:13] tasks in the world of NLB, natural [114:16] language processing which is the field [114:18] of detecting and identifying the named [114:21] entity recognition for example [114:23] locations, countries, organizations and [114:27] the names of persons. Let's go ahead and [114:30] create our sentence. So sentence equals [114:32] Apple. So Apple is a company announced [114:37] announced. Then let's go ahead and [114:40] record [114:41] earnings [114:44] in the United States. Here we have a [114:47] name for a location on Monday. [114:51] So here we have a sentence that says [114:53] Apple announced record earnings in the [114:56] United States on Monday. Let's pass this [114:58] to a named entity recognition model [115:04] by going neer. So ne named entity [115:09] recognition equals [115:12] equals pipeline pipeline. So we can use [115:15] the same function and then pass into [115:16] here neer. Now of course you can pass [115:19] here a model. So feel free to go into [115:22] hugging face go to models and then [115:24] scroll down to tasks and you can choose [115:26] from NLP the class of or the field of uh [115:31] named entity recognition. But here we [115:34] can see we can explore what is the [115:37] default model is. So let's just explore [115:39] go ahead and run this straight away and [115:43] see and detect what is the default [115:45] model. [115:46] So the default model here is what is [115:48] called dbmdz/bbert [115:51] large case fine-tuned. So if you take [115:54] this model and search on it on hugging [115:57] face you can definitely find it by going [116:01] scroll up filter by name and there we [116:04] have it dbmdz and you can see this is [116:08] the model uh dbmdz/bbert [116:11] and and it's used for token [116:13] classification it's built upon the [116:16] library transformers pytorch tensorflow [116:18] jax rust and so on you can see downloads [116:22] last month is 1 million and 700 and [116:25] about it. And you can see you can go [116:27] ahead and use this model using [116:29] transformers and feel free to uh pass [116:31] that model alongside with the task [116:35] specified as token classification. But [116:38] now let's go ahead and use this model [116:40] straight away by passing my sentence to [116:43] the neer tag pipe by going named entity [116:47] recognition and then just pass in my [116:49] sentence and let's the magic trigger. [116:55] Here you have a list of dictionaries. [116:59] You can see the vest dictionary. Let me [117:00] just show up the sentence here to [117:03] compare and see. We have apple. So the [117:06] first thing is we have or org is short [117:08] for organization company. So here we [117:11] have Apple. Apple has been detected as [117:14] an organization. That's perfect. Then we [117:17] have location which is United. Then we [117:20] have another location which is states. [117:22] So you can see United States is a [117:24] location is a country. Then if you go [117:26] ahead and try another sentence here you [117:28] can see location is state. We can go and [117:30] try another sentence together. So [117:33] sentence equals for example I worked [117:38] at Facebook [117:40] after graduating [117:44] from Harvard. [117:46] Then we can pass the sentence to my neer [117:49] sentence. You can see here we have [117:51] organization Facebook and have another [117:54] organization as Harford. And you can see [117:56] if we pass here for example I [118:00] for example live in for example UK. [118:05] Here we have also a location I think [118:08] called UK. [118:10] So UK yeah United Kingdom is a location. [118:14] Okay. I live in UK worked at Facebook [118:16] after graduating from Harvard and it [118:18] detected that so beautifully and in [118:21] pretty well performance. Now let's move [118:23] on to another task. It's called question [118:26] answering. Question answering. And this [118:30] field is also one of the most important [118:33] fields in the world of NLP where you [118:36] provide the model with some context, [118:38] some text and the model tries to find [118:42] some answers in that text and when you [118:44] ask it a question, it will answer that [118:46] question from the text. [118:50] Let's go ahead and Q&A bot. Let's call [118:52] it question answering bot equals pipe [118:55] pipeline and then pass into here the [118:58] task. So remember you can pass here [119:01] sentiment- analysis you can pass into [119:04] here text-cl classification you can pass [119:07] here named entity recognition neer and [119:11] also you can pass into here the task as [119:16] question answering so question dashans [119:19] answering let's write a answering [119:23] and see what is the default model that [119:25] it will provide us Let's [119:31] let's write here R and then CL enter. [119:34] You can see the default model here is [119:35] also distilled / this base case [119:39] distilled- squad. And now it's it [119:42] downloaded the model successfully. Let's [119:44] go ahead and use this model. So we need [119:48] the text context [119:51] equals [119:53] some context. then it can find some [119:56] answers. So let's copy this context for [119:59] example and pass it into here context. [120:03] Let's make that pretty uh uh pretty well [120:06] visualized by going ahead and when the [120:09] word is reaching this line it should cut [120:13] off and go down. [120:15] The line should cut off and go down [120:21] like that. So now we have context. Okay, [120:24] let's go ahead and run that. Then let's [120:26] provide this model which is QA-bot_bot. [120:31] Let's provide context equals context and [120:35] also we have to provide into here some [120:37] question by going question equals [120:40] question [120:42] question and we have also to provide we [120:44] have to create a question variable [120:46] equals for example here we have [120:48] financial sentiment analysis is a [120:50] challenging task. Let's let's ask it. [120:52] What is financial financial [120:56] sentiment [120:57] analysis like that? [121:01] This is my question. Let me pass my [121:03] question. The question and the context [121:05] as this 3 2 1 let's ask it. Now it works [121:10] and you can see the score is 30%, answer [121:14] is a challenging task. So here if we go [121:17] up you can remember that financial [121:19] sentiment analysis is a challenging task [121:22] and it responded with that and it told [121:24] us that that it is a challenging task. [121:26] So this is this idea is mimicking the [121:29] idea of rag retrieval augmented [121:33] generation. Okay. Also we can ask [121:35] another question. We can ask question as [121:38] for example let's read here. We can see [121:42] general purpose models are not effective [121:44] enough because of the specialized. [121:46] Let's ask why why generative [121:51] models are not effective [121:55] and then pass this question to my model [121:57] by giving Q a Q question answering_bot [122:02] question [122:03] equals question, [122:07] context equals context. [122:11] Then let's go ahead and here we have [122:14] quest question. Yeah, let's [122:19] run that. You can see we have score 41% [122:22] answer specialized language used in a [122:25] financial back slash in context. And you [122:28] can see we have here back slashn. If you [122:30] want to show it in a pretty formatted [122:32] way, we can sort this output into a [122:34] variable called result. For example, [122:37] then we can show this result. Then we [122:40] can index on the this this dictionary [122:43] and access the answer key answer. Then [122:48] we can print it. Print it. You can see [122:50] print. So specialized language model or [122:53] specialized language used in financial [122:55] back slash in the mail context. For [122:58] example, let's go ahead and here find [123:00] some we introduce Finnbert. Let's ask it [123:03] what is Finnbert? [123:05] What is Finnbert? [123:09] Fen bird [123:11] a language model based on bird. You can [123:14] see I asked it what is fenb bird. It [123:16] said it said a language model based on [123:18] bird. And that's it. Feel free to uh [123:21] grab or fetch another kind of kind of [123:24] context or text and ask this model [123:28] whatever you want. But keep in mind this [123:30] model is the default model which is this [123:33] still birth. And remember there are some [123:35] better models, some very powerful models [123:39] out there in the hugging face platform. [123:41] You can feel free and go ahead and [123:43] explore them. Maybe they are better than [123:45] this subert. You can go ahead and [123:48] specify the model name as you would like [123:50] and use it. Okay, but now let's go ahead [123:53] and move on to the last task which is [123:56] machine translation. [124:00] to the task of translating from one [124:02] language into another converting from [124:04] one language into another language. So [124:06] the first thing is we need to define a [124:08] variable called translate [124:11] I will name it translator equals [124:13] pipeline pipeline and then pass into [124:16] here translation [124:18] translation_en_2 [124:22] French. So we have the task of [124:25] translating from English to French. [124:27] Let's go ahead and run that. Here you [124:30] can you can see we are downloading now [124:31] the translator here we have Google T5. [124:34] So the default model is Google-T5 [124:37] which is also if you search here T5 [124:39] paper [124:41] this is the paper behind the T5 model [124:44] which is the default model at hugging [124:46] face for translating exploring the [124:48] limits of transfer learning with a [124:50] unified texttoext transformer by coin [124:53] rafael and it all. So here you can see [124:56] they leverage the whole transform [124:58] architecture. Unlike Bert which is [125:00] encoder only and unlike Gibb which is [125:02] decoder only, T5 is using both [125:06] both uh both parts encoder and decoder. [125:10] Okay. And they performed very well on [125:13] the transition task. [125:15] Okay. So feel free to read that in your [125:16] own time. Let's go ahead now and use [125:18] this model by going code and then [125:22] creating a text or sentence. So [125:24] translate. So all you need to do is just [125:28] call your translator your variable [125:30] translator and then pass in some word. [125:33] For example, hello. You you know hello [125:35] in in French is bonjour. So let's go [125:38] ahead and translator. Hello it's [125:42] bonjour. And if we if we also translate [125:45] translator [125:47] and then pass into here thanks it will [125:49] say mercy you can see mercy and feel [125:53] feel free to pass in whatever text you [125:55] want and it will translate. So you can [125:57] see how easy it is to translate how easy [126:00] it is to to find named entity [126:03] recognition in text. how easy it is to [126:06] grab some context and ask some questions [126:08] about it without even training these [126:10] models without caring about what how to [126:13] build these models what are these [126:14] architectures. So hugging face is an [126:16] open-source platform that hosts millions [126:19] of models and you can access them [126:21] freely. So last thing let's try some [126:24] sentence here for example sentence what [126:27] is your name what is your name and try [126:31] to translate it to French translator [126:35] pass here my sentence [126:37] go ahead call est vector of veteran num [126:41] okay so this is how you can translate [126:43] and feel free to access the dictionary [126:45] you can you can go at index zero gives [126:48] you a dictionary then we can access the [126:49] key translation [126:52] underscore text [126:54] and then we have it. Okay, so the the [126:57] last thing we can go to hugging face [127:00] like that. Go to models. [127:02] Here we have tasks. Go to tasks. Go to [127:05] NLB. Scroll down NLB. Here we have the [127:08] task of machine translation. [127:11] Uh translation. Yeah, here we have a [127:13] plethora of models that are all about [127:16] trans. You can see the first model is T5 [127:18] Google T5 which we used before. We have [127:21] Facebook NLB. [127:23] So no language left behind. Okay. 200 [127:28] languages distilled. Then we have Google [127:31] M A L and feel free to to use whatever [127:34] you want. If you choose this model, use [127:36] this model transformers. Then we can [127:38] copy this model. So this is how easy it [127:40] is to grab a model from hugging face. [127:43] Then we can just run it. So this is [127:46] Facebook NLB model 200 distilled 600m. [127:51] I think 600m is referring to 600 million [127:54] parameters. And here we have the model [127:57] card. You can also read here. This is [128:00] the model card of NLLB. Here are the [128:02] metrics. You can read the metrics. Uh [128:04] there are some intended use. So how we [128:07] want to use the model. We have some [128:09] training data, ethical considerations. [128:11] And this is the model card. And also you [128:14] can see here we have it's built upon [128:15] PyTorch transformers and here the number [128:19] of downloads. Okay, let's go here. You [128:21] can see the model has been downloaded [128:22] successfully. Welcome to this section [128:25] where we will discuss audio processing [128:27] using Python and how to use models from [128:30] the hugging face platform to process and [128:33] model audio. So the first thing here [128:36] let's name this as audio processing [128:41] using Python. [128:43] So how to import an audio in Python? [128:46] First we need to import some libraries [128:48] like numpy, mattplot liib and librosa. [128:51] So let's go import numpy as np import [128:55] mattplot lib.pipplot [128:58] as plt and then import librosa. So why [129:01] do we need these libraries? First numpy [129:04] because we we want to convert the audio [129:06] into an array of numbers. Secondly, math [129:08] plot lip because we need to plot some [129:10] some plots and visualizations of the [129:12] audio of the audio amplitude of the [129:14] audio waves sound waves. Then we have [129:16] librosa for processing and reading in [129:19] the audio file. Librosa is the standard [129:22] library in Python for dealing with and [129:24] controlling audio files. Secondly, let's [129:27] go ahead and create a variable called [129:29] audio path equals and then I already [129:32] have here a audio file. You can see it's [129:35] called audio.mpp3. [129:37] The extension is MP3. Let's pass it [129:40] here. Audio.mpp3. [129:42] Then run it away straight away. The [129:46] third step is here. We have to unpack. [129:49] We have to call librosa librosa dot [129:52] load. Call the function load and then [129:55] pass in here my audio path comma. Then [129:58] we have here the parameter that is [130:01] called SR short for sampling rate. The [130:04] sampling rate is an important concept [130:07] when dealing with audio. Every single [130:09] digital audio is associated with a [130:11] sampling rate. In this case, let's set [130:13] it to none. [130:15] This function would return two values. I [130:18] would unpack them and store them in two [130:20] different variables. The first variable [130:22] is y, [130:24] so my audio. And the second is SR, so [130:27] sampling rate. So I will store the audio [130:30] file inside y. and I will store the [130:33] sampling rate inside this variable SR. [130:35] Let's go ahead and run that. Now it's [130:38] trying to load it. And it is already [130:40] loaded. If we print here why, you can [130:43] see it's an array. It's a massive array [130:45] with a lot of numbers, a lot of digital [130:48] numerical values. [130:51] Why? Why? Because we have converted and [130:52] read in the audio into an array. So now [130:55] we have successfully converted our audio [130:57] file, our audio path into an array of [131:00] numbers. And feel free to visualize it [131:03] like that. For example, the first 100 [131:05] samples, these are all the numerical [131:08] numbers, the numerical digits when we [131:10] converted our audio into an array of [131:13] numbers. Second, we have here y dot [131:15] shape. So let's see what is the shape of [131:17] my y. The shape will be as this. You can [131:19] see we have one dimension. The audio [131:22] lives in one dimension and it contains [131:24] 293,280 [131:27] values. [131:30] Then let's go ahead and let's explore [131:32] what is SR. SR is the sampling rate and [131:36] it is currently 16,000. [131:39] Then if you wanted if you wanted to [131:41] visualize the audio waves, the sound [131:44] waves, let's go ahead and create a nice [131:46] figure by going pld.f figure and then [131:48] pass in here fix size equals then let's [131:52] go ahead and set it as 14 comma 6 or [131:55] comma 5 comma dpi let's set it to for [131:58] example 150 then let's go ahead and go [132:01] pltot let's plot the audio stored in the [132:06] variable y let's go ahead and put a [132:10] beautiful x table as time dash samples [132:14] so the xaxis represents the time the [132:18] samples the samples would go on the [132:20] x-axis then we have plyy label the label [132:23] of the y-axis as amplitude amplitude of [132:27] the audio or of the sound okay [132:30] then let's go ahead and run that and [132:32] then cool we have here the sound waves [132:34] you can see these are the sound waves [132:36] the sound goes like that okay you can [132:39] you can see here we have sharp sounds [132:41] and we have then soft sounds then it [132:43] goes up goes down up down up down like [132:46] the sinusal waves [132:48] You can see this is the audio waves. [132:50] Okay. Then let's go ahead and do the [132:54] next step. Then we need to divide the [132:56] number of data points in the sound by [132:59] the sampling rate yields to yield the [133:01] total time of the audio. So if you [133:03] wanted to know or if you wanted to [133:05] discover what is the total length of the [133:07] audio all you have to do is to grab here [133:10] the y the array then get the length of [133:14] it by passing the y to the length [133:16] function here we have this number this [133:18] huge number let's divide it by the [133:20] sampling rate so the total time of the [133:23] audio is 18.3 [133:25] you can see here it's 18.33 okay let's [133:29] go ahead and so it's it's 18 seconds [133:31] okay this is in seconds, not minutes, [133:33] not hours. 18 seconds.33. [133:38] Then let's go ahead and divide here. You [133:40] can see from ipython [133:43] dot display [133:46] import audio. So in order to listen to [133:50] this audio, if you wanted to to know [133:52] what is what this audio actually [133:54] represents, you can import the audio [133:56] class from within the display module [133:59] from within the IPython library. So from [134:01] IPython note the I is capital here dot [134:04] display import audio class then run it. [134:08] Now all you have to do is just call [134:11] audio and then pass data equals Y comma [134:14] the rate equals SR the sampling rate. [134:17] Then run that again. You can see here we [134:19] have this audio. Maybe you won't be able [134:21] to listen it but I will run it now. It [134:24] seems to me more and more as I live [134:26] longer that most poetry and most [134:29] literature and particularly the [134:32] literature of the past is discorded with [134:34] the vastness and variety. [134:36] >> Let's move on to the next step because [134:38] I'm recording my audio now. Then let's [134:41] go ahead and we have a an important [134:43] concept which is DFT. And if you don't [134:46] know this concept, this concept is [134:49] important in the world of audio. It's [134:51] short for discrete [134:53] foryear for your transform transform. [134:58] And if you wanted to visualize this [135:00] discrete for your transform function or [135:03] integral, you can go ahead and go window [135:05] equals numpy.hanning. So numpy is a [135:08] pretty uh pretty powerful library. It [135:11] contains so so many functions. One of [135:14] these functions is hanning function. The [135:17] hanning function computes the hanging [135:18] window. You can see here return the [135:20] heading window. The heading window is a [135:22] taper or tapper formed by using a [135:24] weighted cosine. Let's just pass my [135:27] audio here. So you have to pass in the [135:29] length of the of the y. Then we have to [135:31] go and windowed underscore input equals [135:36] y * window. So all you have to do is [135:38] just multiply the y by this window that [135:43] has created or has calculated for us by [135:45] using the handling function. And then [135:47] dft. So discrete for transform equals [135:50] numpy. FFT. So FFT is short for fast for [135:55] transform RFT and RFT I believe it's [135:59] short for real valued [136:02] uh it's a real valued transform for [136:06] transform and then pass into here [136:08] windowed_input. [136:10] Okay. Then let's run that. You can see [136:13] here an FFT. [136:16] So it's called FFT dot R RF yeah RFT [136:22] RFT is short for real valued fast for [136:26] transform okay I missed I missed a typo [136:29] there you can run that again you can see [136:31] here we have DFT dft now is what is an [136:34] array array of values okay so again [136:38] numpy fft [136:40] is for fast for transform dot real [136:45] valued vast foria transform and if you [136:49] search here numpy. FFT [136:52] dot RFT you can see we have here the [136:55] function showing up you can see compute [136:57] the one-dimensional discrete foryear [137:00] transform for real input real you can [137:02] see real R is for real this function [137:05] computes the one-dimensional endpoint [137:07] discrete 4year transform DFT of a real [137:11] valued array by means of an efficient [137:13] algorithm called the fast 4year [137:15] transform FFT. So f is fast this f short [137:19] for for t transform and you pass it you [137:23] provide it with an array input array [137:26] then you specify the n here we kept we [137:28] kept the n as optional value you can see [137:31] here we have a lot of expressions feel [137:33] free to read this in your own time but [137:35] now let's go ahead and do the exciting [137:37] thing of plt.plot plot and then we have [137:40] to pass in here the DFT. So the discrete [137:44] for your transform. Then let's go ahead [137:46] and plt.title. Set a nice little title [137:49] here as discrete for your transform. [137:54] Then let's go ahead 3 2 1. Hope it [137:57] works. And now it worked. You can see we [138:00] have the discrete for transform here [138:02] showing up in a nice visualization. [138:05] And you can see we have sound waves in a [138:08] discrete for transform function. [138:11] Then let's go ahead and do the [138:13] amplitude. Amplitude equals [138:16] numpy.absolute. So if you wanted for for [138:20] whatever reason to get the amplitude of [138:22] this discrete for your transform sound [138:25] waves, you can pass it to the absolute [138:27] function to get the absolute value by [138:30] passing in a dft discrete for transform [138:33] and easily go ahead and just pl.plot [138:36] pl.plot plot then pass in here the [138:39] amplitude. [138:42] You can see here we have the amplitude. [138:44] You can see the difference between this [138:46] waves and this wave. [138:49] Here we have the absolute version of [138:51] those waves. Then let's go ahead and [138:53] continue our process by going amplitude [138:58] dB. So dB is short for decibel. Decibel [139:01] is the unit of measuring audio in the [139:04] universe. So we have librosa [139:07] dot librosa.mplitude amplitude [139:10] amplitude_2 [139:13] dB which is a nice helpful function to [139:15] convert amplitude to decibel and then [139:18] pass into here amplitude [139:20] amplitude comma ref the reference equals [139:24] numpy domax and then all I have to do is [139:27] just go frequency frequency equals [139:31] librosa dot fftft [139:34] frequencies so we have here a function [139:36] called fft fast for transform [139:40] frequencies and then pass into here SR [139:42] the sampling rate equals SR sampling [139:45] rate comma NFTt [139:47] FFT equals length of Y [139:51] and then we have we can go easily [139:53] pligure create a figure and then set the [139:56] fig size to make it look pretty as 515A [140:00] 4 and then the DPI as our faithful value [140:04] 150. Then let's go ahead and plt dot [140:07] plot and then pass in my frequency comma [140:11] amplitude [140:13] db. So remember we have converted our [140:15] amplitude unit to to to decibel and [140:18] stored it in the variable amplitude db [140:20] plt.x label and then set it as frequency [140:24] in heads in the unit of heads. Let's go [140:27] plt.y label and then pass in here [140:30] amplitude in decibel. You can see [140:32] decible is written as B small B capital. [140:35] Then we have here plt.x scale. So you [140:38] have to if you if we go ahead and just [140:41] run that as that you can see [140:46] it's now it's now the x scale isn't [140:49] measured in a logarithmic measure. It's [140:51] measured in linearly. But once we set [140:53] the x scale and then pass in log short [140:57] for logarithm logarithm function [141:02] you can see now the measure changes. [141:04] It's now uh showed in a logarithmic [141:06] fashion. So you have to set the x scale [141:09] as log to show the x scale as in a [141:12] logarithmic fashion. And keep in mind [141:14] you can see here the scale 10 raised to [141:16] ^ 1 10 raised to the power 0 10 raised [141:20] to ^ 1. If we now get rid of this log, [141:23] it will be measured in linear units 0 [141:25] 1,00 2,000 3,000 4,000 and so on. But [141:29] when once we open this comment or [141:32] uncomment this and measure the X scale [141:35] logarithmically, this these numbers will [141:38] be in logarithmic world. You can see. [141:41] Okay. Now let's go ahead and continue [141:44] our process. Here we have a a code for [141:47] converting the audio into STFT by going [141:51] D. Librosa STFT and then pass in my Y. [141:56] Now you can see this D what it looks [141:58] like. This is the S stft version. Then [142:02] we have to go ahead and just go D DB. So [142:08] convert it to decibel by going [142:09] librosa.mplitude_2 amplitude_2 [142:13] decibel and then pass in numpy.absolute [142:16] and then pass in my d. Just we have to [142:18] pass in my d inside of wrapped within [142:21] the function absolute and then wrap this [142:23] function within the amplitude to db to [142:27] convert from amplitude to decibel and [142:29] then set reference equals numpy.mmax. [142:33] Now all I have to do is just create a [142:35] figure plt.figure figure and then pass [142:38] into my figure here the fixed size to [142:40] set the fixed size as 15 or 14a 5 dpi = [142:45] 150 then we have plt dot or librosa [142:48] let's go librosa now librosa dot display [142:52] so please display dot spec so if you [142:56] know spectrogram spectrogram is the [142:58] standard way to plot audio spectro and [143:02] then just pass d db comma sampling rate [143:05] equals sampling rate comma x-axis equals [143:09] sets it as the value time so name it as [143:12] time and the yaxis represents the [143:16] log logarithmic then we have pltc [143:19] calibbar so if you want to make it [143:21] pretty and visualize the calabar you can [143:23] set it as format equals percent sign [143:27] plus 2 [143:29] then you can dot 0f [143:33] db so this is the format of the numbers [143:36] we want just to show just two decimal [143:38] points after the zero. [143:42] You can see here this is the [143:44] visualization. It's a it a spectrogram [143:47] for visualizing the audio. The x-axis [143:50] represents the time. The y-axis [143:52] represents the hertz. You can see in the [143:54] unit of hertz and here we have the [143:55] calaba showing from 80 decel to zero [143:59] decibel. So the closer the color to this [144:02] color lighter it's close to zero decibel [144:05] and then the closer the color here. So [144:07] these regions here represents the zero [144:10] decibb. When we reach this color this [144:12] darker the darker the color the more the [144:15] value of the this sound wave or this [144:17] sound point the closer it is to 80 [144:19] decel. You can see here we have very 80 [144:22] decel. Okay. [144:24] Then let's move on to the next [144:26] visualization. Let's go. S equals [144:28] labrosa dot feature dot male [144:33] specttogram. Also we have another type [144:35] of specttogram. It's called MEL [144:37] specttogram. [144:39] So we can go to librosa. This is the [144:42] library of librosa. You can see here. [144:45] This is the homepage of librosa audio [144:47] and music processing in Python. And here [144:49] we have the documentation. You can see [144:51] this is the logo of of labrosa. It's the [144:54] standard library in Python for dealing [144:57] with audio files. You can easily pip [145:00] install it. If you don't have it, for [145:02] example, if you are on a local Jupiter [145:03] and you don't have this library already [145:06] installed, you can go pip install [145:08] librosa. Then we have librosa is also [145:10] available on package. You can install it [145:12] by saying install Conda for librosa. And [145:16] don't forget here the second website [145:18] here is the documentation. You can feel [145:20] free to read about the functions. Uh [145:22] troubleshooting. We have display. You [145:25] can see display here. I think they will [145:27] display the function as we did now. The [145:29] spectrogram and also they will teach you [145:32] a lot about about hertz format, chroma [145:35] formatter uh and and a bunch of other [145:38] functions that if you want to dive [145:40] deeper into the browser. But here let's [145:42] keep the things simple and just pass [145:43] into here y = y comma sampling rate [145:47] equals sampling rate comma n millls. So [145:50] what is the number of mills? Let's set [145:52] it as 125 125 and then or 128 for [145:56] example. Then comma f max equals 8,000. [146:02] Then let's go ahead and convert it to [146:03] decibel by going db. So remember the B [146:07] is capital equals librosa dot power_2 [146:12] decibel. So convert the power to decibel [146:15] and then we have we have to pass in s [146:17] comma reference equals numpy domax [146:20] then plt dot figure to to create a [146:23] figure a nice pretty figure and then set [146:26] the fix size as 14a 5 and then dpi [146:31] equals 150 and remember we have to make [146:34] our code pretty and aesthetically [146:36] appealing. Then librosa dot display [146:40] dotspec show spectrogram. So spec show [146:45] and then pass into here s db s in the in [146:49] the unit of decel. We have sampling rate [146:51] equals sampling rate comma xaxis equals [146:54] let's make it as time comma y_axis [146:58] let's set it as log log short for [147:01] logarithm. Then we have f max equals [147:04] 8,000. Then on the last here you have to [147:06] show a color bar by giving penalty doc [147:08] color bar and then pass into your format [147:12] equals percent sign plus two dot 0f then [147:16] db 3 2 1 go ahead run. You can see now [147:21] the code works fine and it shows us this [147:25] another spectrogram which is by [147:27] converting the array or the sound wave [147:30] to STFT or MEL spectrogram function. [147:35] So it's kind of look the same. This is a [147:38] normal spectrogram and this is called [147:40] MEL spectrogram. If you wanted to know [147:42] the difference, you can feel free search [147:44] that on Google. But now let's go ahead [147:46] and just do the the other part which is [147:50] now get a model from hugging face and [147:53] try to classify it uh to classify this [147:55] audio based on a model from hugging [147:58] face. So we have to import required [148:01] required libraries [148:05] and remember this task is called audio [148:08] audio classification. [148:12] So the first thing is we have to go [148:13] ahead and just import those libraries by [148:15] going import torch comma transforms [148:20] comma uh transformers. You need [148:22] transformers because remember the [148:25] default library in hugging face for [148:27] dealing with audio is the same as the [148:29] text. It's called transformers. So the [148:32] same library transformers works for text [148:35] and works for audio. Then we have to [148:37] import here torch audio. [148:39] Okay. Then let's print the versions of [148:41] them. Torch dot just for fun version. [148:45] Then we have here print torch or [148:47] transformers transformers dot underscore [148:50] underscore version underscore. Then we [148:53] have to print here print torch audio [148:57] version [148:59] run it away. You can see here torch. [149:03] Transforms [149:04] torch audio version. It will show us [149:06] these versions. So my torch version at [149:09] the time of recording this video 2.9.0 [149:11] plus CUDA 12.6 remember in the future [149:14] you might have a slightly bigger number. [149:16] Then we have here transformers version. [149:18] Currently it's 4.57.3 [149:21] and here we have torch audio. It's 2.9.0 [149:25] plus cuda 12.6. [149:28] So let's go ahead now and do the audio [149:32] classification [149:34] process. The first thing the first thing [149:36] is we need to import from transformers. [149:39] So from the library transformers inside [149:42] hugging face import auto picture [149:47] extraction or extractor, [149:50] a st for audio classification. You can [149:54] see the naming are here uh long. So you [149:57] have to keep in mind the first letter of [149:59] every word is capital. So a is all [150:03] capitals for audio classification and [150:05] you have the class auto feature [150:07] extractor. Run it uh straight away. Then [150:10] you can see now it's trying to import [150:12] and fetch those classes from the [150:14] transformers library. The second step is [150:16] that we have to import leosa again. You [150:18] don't have to because you have it we [150:20] already have it like but we we can [150:22] import it again. Then we have audio [150:25] path. So maybe you need to keep this [150:28] section separate from this section by [150:32] importing all the necessary libraries [150:34] again. So sometimes you don't want to [150:37] run the whole collab. So you might in [150:39] some point just run from the cell but [150:42] have all the codes below the cell [150:45] working. So you have to sometimes [150:48] reimpport the libraries as we saw here. [150:50] Then the audio path equals audiompp3 [150:55] the name of my audio. Remember I have I [150:57] already have an audio here file called [150:59] audio.mpp3. Let's pass it to my variable [151:01] here. Audio path. Then let's go ahead [151:04] and we have y, sr equals librosa.load. [151:09] Just have some practice. Then we have [151:11] here we have we have to pass the audio [151:13] path comma the sr equals none. So here [151:17] all we are doing is just we are passing [151:20] the audio path to the load function. The [151:22] load function will take care of this [151:24] audio path and read the the audio path [151:26] in. It will read this audio file in and [151:29] convert it from audio waves from audio [151:31] file into an array stored in this [151:34] variable. Y. You can see this array. [151:36] It's a huge array. And every single [151:38] audio is connected with a sampling rate. [151:42] We store it in this variable called SR [151:45] short for sampling rate. And it's [151:47] currently 16,000. But now let's dive [151:49] into the most important idea which is [151:52] and and also you can visualize it using [151:54] from ipython of course from ipython [151:57] display import audio class. Now let's [152:02] call this class audio and then pass in [152:05] my audio path by going data data [152:08] parameter equals audio path then run it. [152:12] You can see now this is my audio. I will [152:14] play it and you can listen it now. It [152:17] seems to me more and more as I live [152:20] longer that most poetry and most [152:23] literature and particularly the [152:25] literature of the past is discordant [152:27] with the vastness and variety, the [152:30] reserves and resources and recuperations [152:32] of life as we live it today. [152:36] Now the audio has finished. You can see [152:38] it's finished. We can continue our [152:40] processing and coding. So the first [152:43] thing is we have to go picture extractor [152:46] picture extractor this my this is a [152:48] variable well where I will call the [152:51] class auto picture extractor [152:55] dot from pre-trained. So here we are [153:00] calling the method from pre-trained and [153:03] then pass into here the name of the [153:06] model. So you can see it's from the MIT [153:08] University-finetuned [153:12] fine-tuned [153:15] audio set audio set- 10-10-0 [153:20] the version 4598 [153:23] so 4593 [153:26] okay this is the name of the model okay [153:29] we are passing this string to the method [153:31] from underscore pre-trained coming from [153:34] the discuss auto feature feature [153:36] extractor and then we are instantiating [153:38] a new object and convert and sorting it [153:42] inside this variable feature extractor. [153:44] Let's run that. You can see now it's [153:46] trying to instantiate. Then the [153:48] processor the pre-processor_conig.json [153:51] has been loaded 100%. Okay. Now let's go [153:54] ahead and uh see the see here you can [153:57] see we have a result. Let's create the [153:58] result variable equals feature extract. [154:01] So let's call let's now leverage the [154:04] benefit of using this object feature [154:06] extractor then and just pass into here [154:09] my y the array of the audio comma return [154:14] tenses. So return tenses set it as pt [154:17] short for pytorch. So please return the [154:21] tenses in the form of pytorch tenses and [154:24] then let's run that. You can see here [154:27] this is uh the running of this cell. [154:29] Let's go ahead and visualize my result. [154:32] You can see and hooray. Here we have my [154:35] result. We have a dictionary. We have a [154:37] key called input values and the value as [154:40] a tensor. Here this class has worked on [154:43] my array of values, the array of the [154:46] audio and produced this tensor that [154:49] contains [154:50] these numbers. You can see we can access [154:52] this dictionary by going result in the [154:54] key input values. Then we have this [154:57] tensor. Then you can also visualize the [154:59] shape what it looks like. We have one. [155:01] So we have a batch of one. Then we have [155:04] 124. Then we have 128. Okay. Then let's [155:07] go ahead and dive into uh calling in the [155:11] model loading in the model by going a [155:14] for [155:16] audio classification. So we have [155:19] imported discuss earlier. It's called a [155:22] for audio classification. Remember my [155:24] task here is to classify audio. We will [155:27] not going to uh produce audio here in [155:30] this specific task. We are going to [155:31] classify audio by dot from pre-trained [155:35] and then pass in here MIT/-finetuned [155:41] dash audio set- [155:45] 10-10-0.4593. [155:50] Then we can also keep in mind you can [155:51] copy this name go to hugging face and [155:55] search for it. by going to models tab [155:58] then paste the name into here you can [156:00] see we have this model MIT and the task [156:02] here is audio classification and here we [156:05] have 86.6 6 million updated September 6, [156:09] 2023. You can go into it. You can see [156:11] this this is the name. You can copy it [156:13] if you don't want to write these [156:15] numbers. You can copy from here and just [156:17] paste it over here. [156:19] Here you have you can see here you have [156:21] uh how to use this model. Use this model [156:23] and go to transformers. Here we have [156:25] from transformers import pipeline. You [156:27] can use the higher level the highle [156:29] helper and also you can use load model [156:31] directory as we did together. Auto [156:34] feature extractor. we have imported from [156:36] transformers and also we have here [156:37] automodel for audio classification. If [156:39] you don't like to type code you can copy [156:41] that and just paste over over there and [156:44] and you will have the extractor and also [156:47] you will have then the model which is [156:49] automodel for audio classification from [156:51] pre-trained and then pass in the [156:53] username of the account/ the name of the [156:55] model and keep in mind you have you you [156:57] might need to read here on how to use [156:59] this model okay later. So the model [157:02] description the audio specttogram [157:03] transformer is equivalent to vit but [157:05] applied to audio and blah blah blah. [157:08] Let's go ahead and just run this cell. [157:10] You can see here we have some typo a [157:14] for audio [157:16] audio classification. Let's just choose [157:19] it. You can see the model is running [157:22] with no any single error. You can see we [157:25] have here models save tensors. It's [157:26] loaded successfully. And then all we [157:28] have to do now we have a model. Now [157:30] actually you can visualize this model by [157:32] going model. [157:34] This is the model architecture. Feel [157:36] free to uh look look at this model [157:38] architecture to learn from it. We have [157:40] as embeddings as P patch embeddings. So [157:43] you can see we have patches here. So [157:45] this paper I think here they will direct [157:48] us to the paper. You can see in this [157:51] repository actually we can search for [157:54] this and write paper. So yeah this is uh [157:58] that's it. It's called AST audio [158:01] specttogram transformer by Yuan Gong and [158:04] it all in in the past decade convolution [158:06] neuron X CNN's have been widely adopted [158:10] as the main building block for end to [158:12] end audio classification models which [158:14] aim to learn a direct mapping from audio [158:17] spectrograms to corresponding neighbors [158:19] to better capture long range global [158:22] context. A recent trend is to add a [158:25] self- attention mechanism on top of the [158:27] CNN forming a CNN attention hybrid [158:30] model. So in the past they used to use a [158:33] self attention mechanism on top of the [158:35] CNN forming a CNN attention hybrid [158:39] model. However, it's unclear whether the [158:42] reliance on a CNN is necessary and if [158:44] neuronets purely based on attention are [158:47] sufficient to obtain good performance on [158:49] audio classification. In this paper, we [158:51] answer the question by introducing the [158:54] audio specttogram transformer A, the [158:57] first convolution free purely [158:59] attentionbased model for audio [159:01] classification. We evaluate a on various [159:04] audio cast benchmarks where it achieves [159:06] new state-of-the-art results on uh of [159:09] 0.485 AMAP on audio set 95%.6 accuracy [159:14] on SC 50 and 98.1% accuracy on speech [159:19] commands v2. And here if you visualize [159:21] it you can see the architecture is very [159:23] simple. They have here some input [159:25] spectrogram. Then they patchify it patch [159:28] split with overlap 1 2 3 4 5 6 7. Then [159:31] they flattened it using a linear [159:33] projection layer. Linear projection [159:36] layer. Then they embed it. So we have E. [159:38] This E is short for embedding. So [159:40] embedding one, embedding two, embedding [159:42] three and the rest. Then we have [159:44] positional encoding P 0, P1, P3. So here [159:48] they have uh pre-processed the audio to [159:51] add to map embeddings to map these [159:54] patches into embeddings and then insert [159:56] or inject some positional encodings to [159:59] to preserve the positional of the audio. [160:01] Then they straight away pass it into a [160:03] transformer encoder block. Then they [160:06] have a line layer to classify and then [160:08] produce some logits to classify. Now [160:10] let's go ahead and use this giant [160:11] architecture. This pretty powerful [160:13] archite [160:15] which is uploaded on hugging on hugging [160:17] face. You can see here we have a then [160:20] let's use it. So this is the [160:21] architecture we have already came [160:23] through it. Now let's go ahead and go to [160:25] the next step where we will uh see here [160:27] result and then choose the key input [160:30] values. So now remember I had an audio [160:34] and I passed it to my auto feature [160:38] extractor here feature extractor. So I [160:41] inserted a new object or feature [160:43] extractor. Then I use that feature [160:45] extractor object to pass my audio. Then [160:48] it produced a really really powerful [160:51] tensor. This tensor is ready to be [160:54] passed in this architecture and to be [160:57] processed further further further until [160:59] the model decides what is the actual uh [161:03] what is the actual class of this audio. [161:05] Then let's go ahead and just use this [161:08] tensor this magical tensor and pass it [161:10] to here. So prediction logs I really [161:13] love the logits equals model and then [161:15] pass into here result and then also [161:19] step into input values dot loggets. [161:24] So here all we are doing is we are [161:26] fetching retrieving that input values [161:29] tensor passing it to our model this [161:31] model as then this model will go through [161:34] some several steps thanks to these [161:37] authors. These are really kind authors. [161:39] They have created a really powerful [161:40] model here. Then we can just go ahead [161:43] and get or grab the logs. Then we can go [161:45] ahead and get the logs. Now you have [161:48] logits prediction and the score logs. [161:50] And these are the logits. You can see [161:52] these are unnormalized scores the model [161:55] has produced and they represents how [161:58] much the model or how much the model [162:00] believes that this audio belongs to [162:03] every new class. though this number [162:05] represents how much ofeness the model [162:09] thinks this audio should belong to this [162:12] class and so on for every single class. [162:14] Now let's go ahead and create a variable [162:18] called predicted [162:21] class ids and then go ahead and [162:24] torch.orgmax [162:26] argmax and then pass in prediction [162:29] logits comma dimension equals one. So [162:32] all we're doing here is we are dimension [162:34] equals minus one. You have to grab or [162:36] fetch the last dimension. Here we are [162:38] passing my logits to the arc max [162:41] function. The arc max function will [162:43] return the index at which the highest [162:46] value would be the highest probability [162:48] value is. And then just give me that [162:50] item and that's simple. Then we have [162:53] this predicted underscore class [162:55] underscore uh ids. Let me just yeah [162:59] write ids. Then you can see here we have [163:02] ids. It's now zero. But I wonder what is [163:04] this class? What does zero represents? [163:07] What is the corresponding human readable [163:09] class name that corresponds to zero? [163:12] Well, you can get it by going [163:13] model.config. [163:15] So short for configuration do ID2 label [163:19] and then all you have to do is just uh [163:21] pass you can see here these are all the [163:23] classes the available classes within [163:25] this model. So here you can see zero is [163:28] speech currently this audio is speech [163:31] one male speech second female speech [163:34] woman speaking three child speech uh [163:36] conversation they have via babbling uh [163:39] speech synthesizer shout bellow whoop [163:42] yell yell children shouting screaming [163:45] these are all the different classes and [163:47] different categories that exist in this [163:49] model. You can see here also we have via [163:52] uh goose, honk, wild animals, uh bird. [163:55] You can uh fetch some bird sound from [163:58] the internet and classify it and it will [164:00] classify it. Here the most important [164:02] idea is the our model predicted that the [164:05] most believable loget should be at this [164:09] position here at position zero. Okay, so [164:12] the predicted class ids is zero and we [164:14] know that zero is speech. So let's just [164:16] go ahead and index on this by going [164:19] model.config [164:20] do id2 label and then just pass in here [164:24] predicted class ids. Then run that and [164:29] the model successfully told us that this [164:32] audio belongs to the speech class. [164:35] So what a surprise. Okay. So the model [164:37] is successfully. So look at look at how [164:40] much easy it is to use a model from a [164:42] hugging face to classify audios. Feel [164:45] free to to retrieve or grab your own [164:48] audio from the internet or your your [164:51] your favorite music or whatever you want [164:53] and try to run this model on it and this [164:56] model will classify it and return you a [164:58] class name for what this audio belongs [165:01] to. So you can see hugging face is now a [165:04] truth treasure. Hugging face contains 2 [165:07] million models and they are all open [165:09] source to us. They are all available to [165:11] us. You can easily within five lines of [165:14] code I think here if you compress them [165:16] there are actually two lines of code to [165:19] have a model to classify for you to [165:21] classify audio without worrying about [165:23] without you don't have to worry about [165:25] how to train it you don't have to worry [165:27] about the about the hyperparameters you [165:29] don't have to worry about what the [165:30] arisha how to complete how to combine [165:33] this arisha what are the embeddings what [165:35] are the the position coding how to [165:37] construct a transformer they they have [165:39] done all the work for you they you have [165:41] uploaded this architecture onto hugging [165:43] face trained on a massive data set. The [165:47] model has really powerful parameters [165:49] then you can easily leverage them for [165:52] free. Okay, so this is how to classify [165:54] audio and let's dive into another task [165:56] in the audio world. And also keep in [165:58] mind you have you can go to hugging [165:59] face, you can go to tasks, models, [166:03] tasks, then go to audio here. Here we [166:05] have audio under the NLP we have audio, [166:08] we have text to speech, text to audio. [166:10] Uh we will do this. We will do text to [166:12] speech and also we will do text to audio [166:15] and uh we already have done uh audio [166:18] classification, audio classification. So [166:20] feel free to play around this. And also [166:22] we have automatic speech recognition. [166:25] Now let's dive into another task of the [166:28] world of audio in machine learning which [166:30] is converting [166:33] converting audio to text. So this is the [166:37] modality converting the model audio into [166:40] the model text. The first thing is we [166:43] have to from transformers [166:46] from transformers import pipeline. So we [166:49] have to import the pipeline function [166:51] from the transformers library inside [166:54] hugging face. Then let's go ahead and [166:55] create a pipe equals pipeline and then [166:58] pass into here automatic- [167:00] speech dash recognition recognition. [167:04] So this is the task of converting audio [167:07] to to text. It's called automatic speech [167:11] recognition ASR. [167:14] Yeah, you can see we have a typo, a [167:17] classic error. Now the model is running [167:19] successfully. You can see here it's [167:21] downloading model safe tenses uh same [167:24] weights some weights we have vocabulary [167:26] special tokens they are all loaded and [167:29] downloaded successfully. The next step [167:31] is let's use this model pipe and [167:36] try to convert the audio to text. So the [167:38] first thing is let's go pipe. You can [167:40] see here the pipe. This is my current [167:42] pipe transformers.pipelinesautomatic [167:44] pipelines dotautomatic speech [167:46] recognition dotautomatic speech [167:49] recognition pipeline at this stored at [167:51] this location in the memory. Then let's [167:53] go ahead and dive into pass uh pass in [167:57] here my audio.mpp3mpp3. [168:01] So currently I have an audio file here [168:03] stored here. It's called audio.mpp3. [168:06] Let's pass th this file path into my [168:09] pipe which is this model that I have [168:12] loaded previously automatic speech [168:15] recognition and then we uh loaded a [168:17] pipeline and stored it in the variable [168:19] pipe. Now let's run this command pipe [168:22] and then pass in my audio. Now you can [168:25] see this is my audio and it converted [168:27] that audio into text. You can see here [168:28] we have a dictionary text. It seems to [168:31] me more and more as I live longer that [168:34] most poetry and most literature and [168:37] particularly the literature of the past [168:39] is discordant with a with a vastness and [168:43] variety that uh the reverses and [168:47] resources and recorruptions of life or [168:50] recap recapations of life as we live uh [168:54] today. So this is the text we have [168:56] converted the model audio into the model [168:58] text successfully. You can see feel free [169:00] to change this audio into whatever audio [169:03] file you have. For example, you can [169:05] record your own voice and pass it to [169:07] here. You can download or load the file [169:11] here. Then you can pass it to the model [169:13] and the model will transcribe that audio [169:15] into the text. Now let's dive into [169:17] another task which is converting text to [169:23] audio. So now let's do the reverse. [169:25] Let's do the reverse option which is [169:28] converting text to audio. What if you [169:31] have some text and you want to convert [169:33] it into some audio file? Well, you can [169:35] do that easily using hugging face [169:37] platform. The first thing is let's go [169:39] from transformers. So remember when you [169:42] deal with audio you have to deal with [169:44] transformers not diffusers. From [169:47] transformers import [169:50] pipeline. This is a function a highle [169:53] helper. a high level helper that you [169:55] want to use or that you will use to load [169:58] in models from the platform hugging [170:00] face. Then let's go pipe equals pipe [170:04] equals pipeline. Then let's pass into [170:06] here text to -ash2- speech. So convert [170:11] text to speech the reverse oper uh the [170:15] reverse process. You can see the model [170:16] here is loading. It's it's trying to [170:18] fetch it's trying to fetch that model [170:20] from the pipeline of hugging face using [170:23] the library transformers. [170:26] Now let it download on its own and then [170:29] once it's finished we can go and write [170:31] some text. For example, I like coding in [170:35] Python. For example, this is my text. [170:37] Then we can go ahead and show it. You [170:40] can see this is my text. I like coding [170:42] in Python. Let's pass it to my pipeline [170:45] pipe. Here we can go. You can see [170:47] output. Let's create a variable called [170:49] output equals pipe and then pass in here [170:52] text. So all you have to do is just [170:54] create some text. You don't have to [170:56] store it in variable. I stored it in [170:58] variable to make my code neat. Then we [171:00] pass that variable text into the pipe [171:04] function. Then it will produce an an [171:06] audio out of this text. Let's go ahead [171:09] and run that. You can see now it's [171:11] trying to create and produce an audio [171:13] file audio waves from this text and it's [171:17] finished. Now we have output. This is my [171:20] output. You can see it outputed a [171:22] dictionary of audio and sampling rate. [171:26] As I told you, every single digital [171:28] audio in the computer is connected to a [171:32] sampling rate. And the sampling rate [171:34] here is 24,000. [171:37] The audio here is in the form of array. [171:39] So this model as we know machine [171:42] learning is the art of converting data [171:45] into numbers and then passing these [171:47] numbers into models. Then the model will [171:49] produce also numbers. So the model will [171:51] not produce here straight away a audio [171:53] file path or audio file but rather it [171:57] will produce an array of numbers. Let's [172:00] go ahead now and uh index to it and have [172:02] this output. You can say we can see here [172:04] output and then get the key audio. This [172:08] is my array. Then also we can get the [172:11] shape of it. We can go here and then at [172:13] the new cell output and then audio. Then [172:16] let's go and print the shape out of it. [172:18] The shape is 1 comma 50,90 [172:22] 59 59,840. [172:25] So it contains 59,840 [172:29] values. If you go ahead now and do like [172:32] that output [172:34] and then index on the audio dot squeeze. [172:37] If you squeeze it, we will get rid of [172:40] this one. [172:42] You can see our output here is written [172:44] output audio and then do squeeze. Now we [172:47] have this shape without the one here. [172:49] You can see just we have this value. We [172:51] can pass this. So here we can we can get [172:53] this and pass this array to plt.plot. [172:57] So plot it using mattplot lip. Okay, [173:00] then we can go ahead and run that. You [173:02] can see these are the audio the audio [173:04] waves the sound waves that the model has [173:07] produced. So this audio didn't exist. [173:09] The model we provided the model with [173:11] this text I like coding in Python. Then [173:14] the model has produced these waves and [173:18] stored it in an array that is like that [173:22] a massive array that contains 59,000 [173:25] and and it's about values. [173:30] Now let's go ahead and from IPython. So [173:32] if you wanted to listen to this audio [173:34] you can do that easily by going from I [173:37] from ipython dot display import audio. [173:41] So you have to import the class audio [173:43] and all you have to provide to this [173:45] class is the audio. Then go ahead and [173:49] you can see data pass in data equals [173:52] output and then index on the audio key, [173:57] rate equals output and then pass in here [174:00] sampling rate because remember the [174:03] output of the model is a dictionary as [174:06] you saw here that contains two different [174:08] keys two distinct keys audio and [174:11] sampling rate. To access sampling rate [174:13] you have to go output this is the name [174:15] of the dictionary and then access the [174:17] sampling underscore rate and it will [174:19] provide you with the sampling rate and [174:22] the same with the audio it will provide [174:24] you with the array with those with these [174:26] values and if you show output it will be [174:28] a whole dictionary. Now let's go ahead [174:30] and run this as it is. You can see now [174:34] this is my audio. Let's play let's play [174:36] it. [174:37] >> I like coding in Python. [174:40] And keep in mind these models uh these [174:42] models are open source on hugging face. [174:45] They are not the most perfect models out [174:48] there in the world. So maybe the sound [174:50] will be uh noisy a little bit. Maybe it [174:53] contains some noises in the background, [174:55] but that's fine. All you have to do is [174:58] go to hugging face, go to tasks here, [175:01] then scroll down until you go to audio, [175:04] then choose text to speech. This task [175:07] converting text to speech. Then you can [175:09] choose uh the best models from here. So [175:12] maybe try to use Microsoft vibe voice [175:15] vibe voice maybe you want to use Araco [175:18] T5 GMA and so on. So feel free to choose [175:21] whatever model you you would like to use [175:24] you prefer to use from here. Here we [175:26] have you can see 3,000 models dedicated [175:29] to this task which is text to speech and [175:32] also we have text to audio which is the [175:34] task of converting text to audio. You [175:36] have models 2,623 [175:39] models. You can see stability also has a [175:41] model called stable audio open. Also [175:45] have Facebook music gen large. We have [175:47] also Marvis AI/ Marvis. And you can see [175:50] we have tons and tons of models. Then [175:53] also we can go to here ASR automatic [175:56] speech recognition. Then we can find [175:59] heaps of models. How to use it? You can [176:01] go to here for example open eye whisper. [176:03] Then all you have to do is just go to [176:05] transformers then copy this code or this [176:08] code into the environment collab and it [176:10] will work for you. Just as easy as that. [176:13] Here we can go ahead and so by uh with [176:18] that being said we finished the model [176:21] audio. We have finished this section of [176:23] audio. You can see we have done three [176:25] different tasks. The task of audio [176:27] classification [176:29] where we classified some audio into [176:31] speech. Here it told us that it's [176:32] speech. You can see here these are the [176:34] logists that the model has produced and [176:36] then the we have provided with some [176:39] speech and it says that it is zero which [176:41] means it's a speech [176:44] here you can see speech and then we have [176:46] another task which is converting audio [176:48] to text the process of converting audio [176:51] into text and we have provided it with [176:53] some audio file you can see here we have [176:55] loaded a model called automatic speech [176:58] recognition and be careful about the [177:00] typos previously I written here auto in [177:04] a reverse way and it did load. You have [177:06] to be careful when you write these [177:08] words. Then here we have converted the [177:12] audio into text. You can see it seems to [177:14] me more and more as I live longer. So we [177:17] have provided an audio and it converted [177:19] it into some text. At the last we have [177:22] the last task which is converting text [177:24] to audio. The reverse you provide the [177:26] model with some text and the model tries [177:28] to tries its best to convert this text [177:31] into audio using some deep learning [177:33] mechanism. And you can see these are the [177:35] sound waves. And then if you want to [177:37] save this audio file path you can go [177:40] with that. You can go with that easily [177:42] by going you can see here we have a from [177:45] ipython. So let's go from or from pi dub [177:49] from python dub import audio segment. [177:54] This is a class. Then we have to go [177:56] ahead and create an instance called [177:58] audio. Audio seg equals audio segment [178:05] the class audio segment. You can see [178:08] segment and then provide here the output [178:11] and then index on what on the audio to [178:13] get the the array with the with these [178:15] values dot two bytes. Then let's go [178:19] ahead and comma provide the frame rate [178:23] equals. So the frame rate is the same as [178:25] the sampling rate but another name. So [178:27] output and then index on the sampling [178:30] underscore rate [178:32] and then you have to provide or supply [178:34] this function with the sample width [178:37] equals output and then index on audio [178:41] dype [178:43] dot item size comma. So here you can see [178:47] we have here the type. So the data type [178:50] okay then at the last we have to provide [178:52] the parameter channels. So how many [178:54] channels are there? We have one channel [178:57] 3 2 1 run that. And now we have [179:00] successfully uh saved our model. Let's [179:02] go ahead and audio seg. Let's use that [179:06] object [179:07] or that instance dot export. So we have [179:11] a function called export that will [179:13] export my audio and then pass in the [179:15] name of the audio. For example, my audio [179:18] saved dot and you have to provide the [179:21] extension as MP3, [179:24] the format of the audio should be in [179:27] this format MP3. Now let's run that and [179:30] you can see it saved the model with this [179:32] name my audio saved.mpp3 using this [179:35] name. If you open this file section in [179:37] collab, you will see that I have now a [179:39] new file my audio saved mp3. And to [179:42] download it, just double click on it and [179:45] it will download straight away. In this [179:48] section, let's dive into diffuses models [179:51] for images and generating images using [179:54] hugging face platform. The first thing [179:57] let's name this cell by going image [180:00] processing. So let's take the first some [180:03] techniques to process images in the [180:05] Python language using Python programming [180:08] language and then let's convert it into [180:10] Macdown. You can see image processing [180:12] using Python. So the first thing we need [180:15] an image. You can see here I already [180:17] have an image and it's called [180:18] whiteflower.jpg. [180:21] You can see it here. Now let's process [180:23] this image using Python. The first thing [180:26] is let's go from pill import [180:31] py import image by going from pill [180:35] import image. So from Python image [180:37] library [180:39] import the class image and the class [180:41] image will read in our image file and [180:44] then convert it into an array full of [180:46] numbers by going pick equals image dot [180:52] open. Then let's pass into here let's [180:54] pass into here my image path by going [180:58] flower dot. You can see here my [181:00] extension is jpg. So let's write white [181:03] dash [181:05] blower.jpg. [181:09] And now it's read successfully. If you [181:12] go ahead and show this pic short for [181:15] picture here we have this picture. [181:18] Then let's go ahead the next step. So [181:20] now we have done the the reading of the [181:24] image. The next step is what if you want [181:26] to check the type of the picture. What [181:28] is the type of the picture? The tab is [181:31] JPEG image file. And keep in mind if you [181:34] read in a PNG image, it will the type of [181:37] it will be PNG image file. And if you [181:40] read a GIF image, it will be GIF image [181:43] and so on. So whatever the extension we [181:46] have, this end part, this ending part [181:49] will change depending on the extension [181:51] of the image. [181:53] Let's go ahead. Then we need to to [181:55] import [181:57] numpy as np to convert our image into an [182:01] array into a numpy array. And remember [182:04] numpy is the fundamental computing [182:07] library inside the python ecosystem for [182:10] data science. Let's go ahead and do [182:12] picture array equals numpy as array. So [182:16] we have a helpful function inside numpy. [182:19] So within numpy we have a function [182:21] called you can see numpy numerical [182:22] python we have a function called as [182:24] array. This function is responsible for [182:27] converting the image into an array in in [182:30] numpy by passing in my picture. Then [182:33] let's go ahead and show my picture [182:34] array. You can see we have this [182:37] we can show data and then convert this [182:39] image or this picture or this image into [182:43] an array full of numbers. You can see [182:45] these are 255 255 255. It's a white [182:49] pixel corresponding to this region where [182:51] you can see I have here so many white [182:53] pixels and these are all white pixels. [182:56] Okay. Then we have here some yellow [182:58] pixels and so on. Now let's go ahead and [183:02] retrieve the shape of this array by [183:04] going pick array dotshape and the shape [183:07] of it is as you can see we have 360 [183:11] comma 360 comma 3. So here we have the [183:15] height, the width and the color [183:17] channels. So every single image is [183:21] containing a height dimension, a width [183:23] dimension and a color channels [183:25] dimension. The number of color channels, [183:27] if the image is colored, so it's a if [183:30] it's a colorful image, then the number [183:32] of channels would be three. And if the [183:34] image is a grayscale image, so it's a [183:37] silver image, or a gray image, then you [183:39] have just one channel. In this case, [183:41] it's a colored image. So here we have [183:43] three different color channels and these [183:46] are red, green, blue. Now if you take a [183:49] a closer look at my picture array, it [183:52] contains integer values. These numbers [183:56] are integer. These are a whole numbers. [183:58] If you want to normalize the image, we [184:01] have to divide it by 255. So 255. [184:06] Then it will be normalized. As you can [184:07] see the where the most white color would [184:12] correspond to one and the most dark [184:15] color so the most black color would [184:17] correspond to zero and every other color [184:20] between zero and one. So you can see [184:22] here once we go ahead and import [184:25] mattplot liib import mattplot [184:28] liib.pipplot [184:30] as plt. So mathlip is the default [184:33] library in Python or the standard [184:35] library in Python for visualizing and [184:37] plotting some visualizations. Let's go [184:39] ahead and pltim show and then pass in my [184:41] picture array. So you can see here by [184:45] passing my picture array to the im show [184:47] function. IM show is short for image [184:49] show. It shows our image the flower [184:52] image. You can see and here we have some [184:53] axes. If you want to get rid of these [184:55] axes, you can do that easily by going [184:58] plt dot access and then pass here false. [185:02] You can see here it will turn off the [185:04] axes, the x-axis and the y-axis. [185:08] Now let's go straight away and picture [185:10] red. So what if you want just to [185:14] view or to display just the red channel? [185:16] You can do that easily by going picture [185:18] red equals [185:20] picture array.copy. So first I advise [185:24] you to copy the picture array into a new [185:27] variable. So rather than manipulating [185:30] and damaging the actual array here where [185:33] you could you could do that but I advise [185:35] you I recommend I highly recommend that [185:37] you copy this array and store it into a [185:40] variable called picture red. Then you [185:42] can see here we have the same exact [185:44] copy. So here pick red currently is the [185:48] same exact copy because we have done [185:50] nothing but copied the array into a new [185:52] variable called p pick red short for [185:55] picture red. Now let's do the [185:57] manipulation part where you will go [186:00] ahead and you can remember we have RGB. [186:04] Now we want to zero the green channel, [186:07] zero the blue channel and just keep the [186:09] red channel. And look what happens here. [186:11] We can see pick red and then give me all [186:15] the height dimension. No problem. Comma. [186:18] Also give me all the width dimension. I [186:20] don't want to get rid of any the width [186:22] or height. I don't want to cut anything [186:24] of the image. But just set the one [186:28] channel which is G. So remember R zero, [186:32] G is one, B is two. Please set the one [186:36] channel. This green channel set it to [186:38] zero. So here you can see green [186:42] channel green channel set to zero. So [186:46] zero it then the same with the blue [186:48] channel give me all the height all the [186:50] width and then the second channel which [186:52] is B. Remember in programming we start [186:55] counting from zero set it to zero. So [186:59] zero out the blue channel. Then here we [187:02] have blue channel [187:05] set as zero. [187:09] You can see here my picture red now is a [187:13] red image. So you can see now we have [187:16] got rid we have removed the blue channel [187:19] the green channel and only kept the red [187:21] channel. You can see here we have this [187:23] array. It just contains you can see here [187:25] every single blue value and every single [187:28] green value is now zeroed out. The green [187:31] channel is zero. The blue channel is [187:32] zero for all pixels for all values. just [187:36] the red channel is kept and you can see [187:39] now we have a very reddish picture. Now [187:43] let's go ahead and do the same with the [187:45] the rest and the remaining of the [187:47] channels by going RGB. Remember here we [187:50] have to go ahead and copy the array by [187:52] going pick green equals pick array. So [187:56] give me the picture array the original [187:58] array but copy it dotcopy. So make a [188:02] copy of it. Then let's go ahead and say [188:04] pick underscore green and then give me [188:07] all the height all the width but please [188:10] go ahead and for the red channel zero it [188:12] out and the same here pick underscore [188:14] green give me all the height all the [188:16] width but please for the blue channel [188:20] zero it out here we have remember zero [188:23] corresponds to what the red channel so [188:26] red channel [188:30] is zeroed out is zeroed And here the [188:33] blue channel, blue channel [188:38] is zeroed. [188:40] Set to zero. And here also set to zero. [188:43] And now let's see what would happen here [188:45] by printing out my picture underscore [188:48] green. 3 2 1 run that. And here you have [188:53] a green like image. So you can see the [188:56] red channel is zero out is zeroed out. [188:59] The blue channel is zeroed. While we [189:01] have kept all the green channels, you [189:03] can see now we have a green image. Let's [189:06] do the same with the last channel by [189:09] going ahead and copying out the picture [189:12] blue equals pick ray dotcopy. Let's go. [189:17] Picture blue. And then just set the zero [189:22] channel to zero which is red. And then [189:25] do the same [189:27] with the one channel which is green to [189:30] zero. Then let's visualize my picture [189:32] blue where you have a blue image. Okay. [189:35] So we have red, green, blue. We have [189:37] three color channels. Red, green, blue. [189:41] Now you can see that these are the [189:43] manipulation of images. We can go ahead [189:45] and also do some feather processing [189:47] using now CV2. Import CV2. So CV2 is a [189:51] library inside Python and it's short for [189:55] computer vision 2. So CV is has also [189:59] pretty much functionalities to [190:01] manipulate and to process images in [190:04] Python. So let's go ahead image equals [190:07] CV2 im read. So please image read the [190:12] image by passing here the path of the [190:14] image. So let's copy here the path of [190:16] the image. You can go ahead and just [190:18] copy the path and pass it to this [190:21] function m read. Here we have image. You [190:24] can see it read the image here. But you [190:27] can see there's a difference. Why? When [190:29] we read the image using the normal pil [190:32] library in Python, Python image library, [190:35] it read that image and it displayed that [190:38] image as this. You can see here these [190:42] colors are yellow. But here when we read [190:45] the image using this library CV2, you [190:49] can see here we have different color. We [190:51] have different color hues. Why? Because [190:54] if you notice the real image or the [190:56] actual image actually is the same as the [190:59] way that the PIL image has read it in. [191:03] So CV2 keep in mind that CV2 library [191:07] here it changes the order of the [191:11] channels. So rather than red green blue [191:14] it's blue green red. So let's go ahead [191:18] and change that. Here you can see image [191:22] RGB equals CV2. If you want to get it in [191:26] the in the original format, if you want [191:28] to get back to the original format and [191:30] the original form, you can do that [191:31] easily by going CV2 dot CVT color. So [191:37] you have a nice function here. It's an [191:39] amazing function. It's called CVT [191:42] color. And then pass into here image [191:44] comma CV2 dot you can see here color uh [191:49] write it all in caps so all in capitals [191:52] color underscore B G R so blue green [191:57] blue uh blue green red to [192:01] red green blue so this is the the name [192:05] that you want to write color underscore [192:08] blue green red two red green blue as I [192:12] told you this library or this package [192:14] that uh which is called CV2 when you [192:17] read an image using it it reads that [192:20] image and it manipulates and damages the [192:23] color schemes how to solve it well [192:26] there's a helpful function a very [192:29] amazing function you can use it just [192:31] pass in the image and then write this [192:34] flag cv2 color_bgr2gb [192:39] then let's go ahead and run that you [192:41] have a image underscore RGB. So you can [192:44] see we are now having the same image. [192:47] You can see we have returned that image [192:50] to its original format to its original [192:52] form. You can see now this region is now [192:56] not anymore blue but now is the is the [192:58] original form which is yellow. Okay. So [193:01] this is how you can get back to the [193:03] original format. If you read an image [193:05] using CV2, [193:07] please keep in mind you have to call the [193:09] function CVT color. Pass in the image [193:11] you want to apply the manipulation on [193:14] it. Then write this flag CV2 dot CV2C [193:18] color blue green red to RGB red green [193:22] blue to the normal and regular format. [193:26] So now let's convert this image into a [193:28] grayscale image by going image gray and [193:32] then equals cv2 dot m read. So we have [193:36] to read in the image. So remember m read [193:39] short for image read. So please read in [193:41] the image and we have to pass into here [193:43] my image file path by copying the path [193:46] and then pass into the my function here. [193:49] Then let's go ahead and also provide [193:51] this parameter cv2 dot [193:55] im read underscore. So image readers [194:00] gray scale. So if you want to convert [194:02] this image into a gray scale you have to [194:04] provide this flag cv2 [194:07] mread [194:09] grayscale. Let's go ahead and run that. [194:11] And now we can easily go ahead pltim [194:15] show and then just pass in my image gray [194:19] comma cmap set the color map equals to [194:22] gray. So you have to set the color map [194:24] as gray in order to display the image in [194:27] a gray format or a gray mode. You can [194:29] see now we have a gray scale image. [194:32] So an image in the fashion of grayscale [194:34] it has gray colors. Okay. Now let's go [194:38] ahead and do the next step which is [194:41] image resize. So it's an important skill [194:44] of how to resize images. You will see it [194:47] a lot especially in hugging face models. [194:50] There are some cases where you need to [194:52] resize some image in order to meet the [194:55] predefined criteria or the assumptions [194:57] of the model. So it's an a really [194:59] important skill to have. Image resize [195:03] cv2.resize. [195:05] So just use the function resize then [195:07] pass in my image. Then you have to [195:08] define the size 200 comma 400. For [195:11] example, set it as 200 comma 400. These [195:15] are the the dimensions and the [195:17] dimensions of the of the image. So let's [195:19] go ahead and just go plt.mshow [195:23] dot imshow and then pass in into here [195:26] image resize. You can see it will resize [195:28] my image using this function resize. Go [195:31] ahead and run that. And now we have a [195:33] resized image. You can see here the [195:36] x-axis. So the width here spans until it [195:40] reaches 200. [195:42] So you can see here 200. This [195:44] corresponds to the x-axis. Here we have [195:46] from 0 to 200. So here we have 200. And [195:49] the yaxis spans from zero until it [195:53] reaches 400. So you can see at 350 it [195:56] reaches 400. So these are the [195:57] dimensions. If you change it for example [195:59] 150 by 150 then it will change into 150 [196:03] by 150. You can see we have a squared [196:06] image 150 and here we have 150. Okay [196:09] feel free to play around and have fun [196:11] with that with those values. Now let's [196:13] go ahead. W ratio equals 0.5. Then we [196:17] have height ratio equals ratio equals [196:20] 0.5. [196:22] Here remember when we resized the image [196:24] into 200 [196:26] comma 400 the image scaling wasn't [196:30] equal. You can see the image here is [196:32] squeezed is squashed it's not in equal [196:35] dimensions it look like a real image. So [196:38] in order to have a re in order to resize [196:41] the image without damaging its scaling [196:44] without damaging its dimensionality [196:46] keeping the image uh to be in equal [196:49] dimensions you have to specify W ratio [196:51] as 0.5 height ratio 0.5. So the half [196:56] let's go ahead new image or new image [196:58] equals cv2 dot resize [197:02] and then pass into here my image and [197:05] then comma z uh pass here 0 comma 0. So [197:09] for the D size then comma FX equals W [197:14] ratio. So W ratio stands for width ratio [197:18] comma FY so the Y-axis corresponds to [197:21] the height ratio [197:23] H ratio. [197:26] Then now let's go ahead and just just [197:28] display my image using the new image. [197:31] You can see here we have resized our [197:33] image successfully but keeping the [197:36] dimensions equal. keeping the dimensions [197:38] to be in a not damaged way. You can see [197:41] here we can set a W ratio, height ratio, [197:45] pass these variables to the parameters [197:48] f_sub_x, f_sub_y to the resize function. [197:51] Rather than specifying these dimensions [197:52] here, we passed here 0 comma 0 and then [197:55] determine f_sub_x as w ratio, fy as the [197:59] height ratio. Then we have this image [198:01] with a neat scaling. You can see it's [198:03] now 180 by 180. So it halves the image [198:06] width and height. Then let's go ahead [198:08] and generate. Now let's dive into the [198:11] actual generation. So that's enough with [198:13] how to pre-process images, how to deal [198:16] with images. You can see here we have [198:17] done uh plenty of skills of reading the [198:20] image, converting it into array, then [198:22] how to get the red channel, green [198:24] channel, blue channel. And you can see [198:26] here we have used the library CV2 as an [198:28] alternative. Then we have convert the [198:31] the format from blue green red into red [198:36] green blue. You can see here from blue [198:38] green red to red green blue. And then [198:41] you have here you can see the way to to [198:44] convert it into grayscale. And the last [198:46] thing here how to resize it with unequal [198:49] dimensions and with equal dimensions. [198:51] That's enough with the skills and on how [198:53] to pre-process images. Let's dive into [198:55] how to use im uh models diffused models [198:58] from hugging face to actually generate [199:01] images by going let's name this [199:03] generating faces using diffusion models [199:07] or diffuses diffusion model. [199:10] So here you can see this task is one of [199:14] the most tasks in the world of [199:16] generating images and it's pretty well [199:18] served. You can see here from diffuses [199:21] import the DPM [199:25] pipeline. So DP [199:28] DDP PM pipeline pipeline. So from [199:33] diffuses import DDPM pipeline. If you [199:36] wonder what is DDPM, [199:39] DDPM is a is a diffusion model and it [199:42] stands for denoising diffusion [199:44] probabilistic model. And if you we [199:47] current here d noising diffusion [199:51] probabilistic model. [199:54] Okay, this is a DDBM pipeline. We want [199:57] to import that pipeline from the diffuse [200:00] library. Go ahead and import. You can [200:02] see now it's importing. This model has a [200:04] paper. If you search here [200:07] DDPM model paper, this is the model D [200:11] noising diffusion probabilistic models. [200:14] And you can go here you can see it's by [200:16] Jonathan Ho and it all. So here you can [200:19] see we present high quality image [200:21] synthesis. So image synthesis is a fancy [200:23] word for image generation results using [200:26] diffusion probabilistic models a class [200:28] of latent variable models inspired by [200:32] considerations from non-equilibrium [200:34] thermodynamics. So remember the [200:36] diffusion model is inspired by the [200:39] physics science specifically [200:41] non-equilibrium thermodynamics. So [200:44] particularly this field has inspired [200:47] this deep learning model diffusion. Our [200:49] best results are obtained by training on [200:52] a weighted variational bound designed [200:54] according to a novel connection between [200:57] diffusion probabilistic models and [200:59] denoising score matching with lang [201:02] dynamics. And our models naturally admit [201:05] a progressive lossy decompression scheme [201:08] that can be interpreted as a transition [201:10] of auto reggressive decoding on the [201:13] unconditional cipher 10 data set. We [201:15] obtain an inception score of 9.46 and a [201:19] state-of-the-art FID score of 3.17 and [201:22] you can read that in your own time. So [201:24] this the paper behind this class. Feel [201:26] free to read it in your own time. But [201:28] now let's go ahead and use this class. [201:29] It's called DDPM pipeline from diffuse [201:33] library and you can see here the paper [201:34] is loading [201:38] where they generated faces. So the [201:41] actual interest of this paper is to [201:42] generate faces and we will do the same [201:44] as they did. We will go we're going to [201:47] generate faces and you can see here they [201:50] have so many equations the paper you can [201:53] see also not faces but also images. [201:58] So you can see you can see from here [202:00] these are uh this is the for the forward [202:03] process where they start with non uh [202:07] non- noisy image then they slowly add [202:09] noisy or they start from random noise [202:12] then the model slowly but gradually den [202:15] noises that noise until it reaches a [202:18] point where we have a coherent logical [202:21] object. [202:23] We have a forward process and we have [202:25] also a reverse process. Okay. [202:29] So now let's go ahead and use that [202:32] model. [202:34] You can see here DDPM [202:37] equals [202:39] DDPM [202:41] pipeline [202:42] then from pre-trained and then pass in [202:47] Google/ [202:48] DDPM-sellip [202:52] ahq-256. [202:56] Then we have two cuda. So here we have [202:59] from pre-trained method and then pass in [203:01] Google. So the provider of this model on [203:05] hugging face is Google/ [203:07] denoising diffusion probabilistic model- [203:10] celib is a data set of faces. Celip [203:15] ahq256. [203:16] If you search here and say here data set [203:20] it's a huge det. It's also provided on [203:23] tensorflow. You can see tensflow library [203:25] is providing this data set celip a_hq. [203:29] So in case you don't know what this is, [203:31] this is a data set for faces. You can [203:33] see here high quality version of the [203:36] cilip a data set consisting of 30,000 [203:40] images in 1x,024 [203:44] resolution. And you can see here you can [203:46] download the image. You can download it [203:48] from this GitHub uh repository. And you [203:51] can go down this some code to download [203:53] and get the data set. And you can see [203:56] these are faces faces for people. So [203:59] CLIP is a data set of faces containing [204:03] tons and tons of human faces. You can [204:05] see DDPM-sip [204:07] ahq-256. [204:10] You can see now it's downloading. [204:15] So it's trying to load in this model [204:19] from the hugging face platform. Now [204:21] let's go ahead and just use this model [204:24] by going. So you can see here we have [204:25] this model DDPM. This is the model DDBM [204:28] pipeline. It's a pipeline. Let's go [204:30] ahead and just generate an image equals [204:33] DDPM and then pass in num_inference [204:38] steps equals 30 dot images at index [204:41] zero. So all we're doing here is just we [204:44] are calling the ddpm dn noising [204:47] diffusion probabilistic model passing in [204:50] num inference steps equals 30. Then we [204:53] are stepping into the images property [204:56] and then getting or retrieving at index [204:59] zero. Let's run that. You can see it's [205:01] now trying to generate that image and [205:03] you can see wa you are going very fast. [205:05] Why? Because I have here a GPU. Go to [205:07] runtime change runtime type. You can see [205:10] here I am using currently L for GPU. So [205:13] don't be surprised if you are using CPU [205:15] and you are slow or your model is going [205:18] slow. Currently I am using GPU. Okay, [205:21] that's why my model has done that [205:23] quickly. Now let's go ahead and just [205:27] show this image. You can see this is a [205:30] face for a human. Okay. So this is the [205:32] this is the diffusion model the same as [205:35] they did in the paper. So remember in [205:37] the paper here they have generated [205:39] faces. Okay. Here you can go down or up. [205:44] Yeah. Here you can see they have a lot [205:46] of faces. They generated a lot of faces. [205:49] And one of these faces now is showing up [205:52] in our collab environment because why? [205:53] Because we used the model DDPM-sip [205:57] ahq-256. [205:59] Now let's go to the next step and go [206:02] from diffuses [206:05] import from diffuses import DDPMuler [206:10] now you can see we have here DDBM [206:12] scheduleuler comma unit to the model and [206:17] also we need to import the unit 2D model [206:19] run that now let's go [206:23] equals ddpmuler [206:28] then let's go dot [206:29] from pre-trained and then pass into this [206:33] from pre-trained the name of the model [206:36] by going here you can see passing Google [206:40] slash DDBM so remember dn noising dn [206:44] noising diffusion probabilistic model [206:47] dash celib [206:50] or ahq-256 [206:54] just like that we are passing here the [206:56] name of the model to the method from [206:58] underscore pre-trained coming from the [207:00] DDPMuler. [207:01] Let's run that. Now we have a [207:02] scheduleuler. We can check it outer. [207:07] Okay, this is my scheduleuler. You can [207:09] see it has a bunch of info. Class name, [207:12] DDBMuler, diffuses version 0.56.0, [207:16] beta end, betaul, uh beta start, clip [207:19] sample, clip sample range, and a lot of [207:22] other properties and attributes. But now [207:24] let's go ahead and focus on how to use [207:27] the the model unit by going model equals [207:30] u net 2D 2D model. So the unit 2D model [207:35] two-dimensional model dot underscore uh [207:38] dot from pre-trained. So use from [207:41] pre-trained and then pass in google ddbm [207:47] ahq-256.2 [207:50] cuda. Keep in mind you have to to move [207:53] this model to the to CUDA to CUDA [207:55] environment or to CUDA device because [207:57] otherwise CUDA otherwise this model will [208:00] live on the CPU and it will take forever [208:03] generating one one simple image. Okay, [208:05] run that. Now it's getting my model. We [208:08] can check out my model and this is my [208:10] model. These are all the layers. You can [208:12] see this is the deep learning. You go [208:14] deep. You you stack deep layers on top [208:18] of each other. You can see we have here [208:20] cellu activation function. We have [208:21] convolution drop out. This is the unit [208:24] model. Okay. Module list. We have reset [208:26] block 2D group addition convolution 2D [208:29] linear group norm drop out convolution. [208:31] And they replicated these blocks many [208:34] many times. Okay. You can see here the [208:36] scroll goes forever. We have so many [208:40] layers. Okay. Feel free to explore this [208:42] architecture in your own time. But now [208:44] let's go ahead and use theuler. [208:48] scheduler then dot set time steps and [208:52] then pass 50. So I want to set the time [208:54] steps as 50 [208:57] run that now if you check out the [208:59] scheduleuler dot time steps. So how many [209:01] time steps do I have here? You can see [209:04] we have 50 time steps and these are as [209:08] 980, 960, 940, 920, 900, 80, 880, 860, [209:15] 840, 820 and so on so forth. Now let's [209:18] go ahead and import torch. [209:22] Then let's go sample size. set the [209:25] sample size variable equals to model dot [209:29] configuration dot sample size and then [209:33] show the sample size. So here we have [209:37] tapped into the configuration of the [209:39] model dot sample size. [209:42] Then we have here the variable sample [209:44] size. You can see we have here 256. [209:48] Okay, this is the sample size. [209:52] So all we have done is model.configur [209:54] configuration model sample size then all [209:58] of a sudden the sample size is 256. Now [210:01] let's go ahead add noise equals [210:05] torch dot rand in rand n and then we [210:10] have here 1 comma 3 [210:13] comma sample size. So we are passing the [210:17] sample size inside the tpple. Here we [210:20] have a touble the uh one represents the [210:23] batch size. So how many image here [210:26] inside this batch only we have one image [210:28] we have just one image comma then we [210:30] have here three color channels red green [210:33] blue comma we are passing the sample [210:35] size here as a value. Then let's go [210:37] ahead comma device equals cuda. [210:42] So now we have a variable called noise [210:44] and it stores a random torch random [210:48] tensor. We can show it here. You can go [210:51] noise. This is my noise. So just noisy [210:54] numbers, noisy values. Okay. Rand n pass [210:57] here 1 comma 3 comma sample size. Then [211:00] device ascoda. Now let's go ahead and [211:03] create a variable. You can see here we [211:06] need to duplicate the sample size. [211:08] Sample size two times. Okay. We have to [211:11] pass them twice like that. Okay. Because [211:14] here we have color channels height [211:17] width. Okay. [211:20] Now let's go to the next step where [211:22] input equals noise. Then let's go a for [211:26] loop for t. So for every single t. So [211:29] time step for every single time step [211:31] inuler [211:34] dot time steps. So loop through the time [211:36] steps. Then do the following with [211:39] torch.inference mode. So we need to [211:42] enable the inference mode. We have to [211:45] enable and turn on inference mode. And [211:49] when you enable and turn on the [211:51] inference mode using PyTorch, then you [211:54] are basically turning off the back [211:56] propagation algorithm. You are turning [211:58] off the gradient tracking in in PyTorch [212:01] and you are uh you are in the inference [212:04] mode. Okay, let's go. noisy underscore [212:08] residual residual [212:11] equals model and then pass input comma t [212:14] dot sample and here let's call it noisy [212:17] noisy residual equals model and then [212:19] pass input comma t. So the time step [212:22] that we are looping through in scheduled [212:24] time steps and remember this time steps [212:26] is just a tensor with these values. It [212:30] contains here 50 values. [212:33] Okay, we have we have 50 different [212:35] values. They are representing the time [212:37] steps. We are passing every single time [212:39] step alongside with the input to the [212:41] model and the input is what is random [212:44] noise. So we are taking the random noise [212:46] passing to the to my model the unit [212:48] model and providing also the time step [212:51] here and then taking the sample. Then we [212:53] have your previous underscore noisy [212:57] noisy sample equals scheduleuler dot [213:02] step and then pass into here noisy [213:04] residual comma t comma input and then [213:08] dot previous sample. So here we are [213:10] getting the previous sample the previous [213:12] step. [213:14] Then at the last you can see we need to [213:17] set the input variable input equals [213:20] previous [213:23] noisy sample and run that. Now it's [213:27] running. So now it's looping through the [213:28] time steps and now it's finished. It's [213:30] done this kind of steps with torch [213:34] inference mode. Please enable the [213:36] inference mode. Then call the model pass [213:39] in my input. So input is what here my [213:42] input is just random tensor if you want [213:45] to show it here you can go dot shape you [213:48] can see here this my shape let's squeeze [213:50] it by going squeeze [213:52] here squeeze function so now we don't [213:55] have anymore that one dimension now [213:58] let's permute it to be to be compatible [214:00] with math lip permute pass one to zero [214:04] why we need to pass here the height here [214:07] so height has one put it at the the [214:10] width put it here. Then the the color [214:12] channels move it to the last dimension. [214:14] Now we have a compatible tensor. Let's [214:16] go dot numpy. [214:19] Now dot detach first. Here we have dot [214:23] detach [214:25] then numpy because we need to detach it [214:27] and then numpy. You can see [214:33] dot and then convert it [214:37] numpy version. [214:39] device. Yeah, we need to [214:42] CPU then numpy. Okay, then we need to [214:45] pass this to plt dot im show. [214:54] You can see here we have random noisy [214:56] image. So we are starting from this [214:58] noisy image. Okay. So the the error here [215:02] was you are uh we were converting this [215:05] tensor which is living on the CUDA [215:07] device. We were converting it to numpy. [215:11] If you see here, this is a pytor torch [215:13] tensor. And then once you convert once [215:16] you turn it into numpy, okay, by going [215:18] numpy, it gives you a huge error saying [215:22] that hey, you are sorting that tensa on [215:25] CUDA can't convert CUDA device to numpy [215:28] because numpy lives on the CPU. Numpy [215:31] always always works and deals with [215:34] arrays stored in the CPU. So use here [215:38] they gives they give you a hint please [215:40] use tensor CPU or two CPU you can see [215:46] now you can go ahead and CPU okay [215:51] you can see now we are moving our uh [215:54] tensor CPU then we can go [215:57] dot CPU okay it turns it into CPU then [216:01] we can go numpy and it returns a numpy [216:04] version of it then we can pass that to [216:06] our uh plt.m show function and it will [216:09] show random noise image. So you can see [216:11] here we have random random noisy pixels. [216:14] So nothing nothing to infer from this [216:16] image just random pixels and every time [216:18] you run that you get random shuffle of [216:22] the pixels. Okay, we are starting from [216:24] this noise and then passing the t which [216:27] is the the the [216:30] time step alongside this input noise [216:32] input and then getting the previous [216:34] sample. Then we are taking this previous [216:36] noise sample and assigning it to the [216:38] input variable. Okay, [216:41] let's go now from pil. [216:47] So let's go ahead and we have another [216:51] another idea of showing a distribution [216:54] or another concept of showing an a the [216:57] distribution of this array. So all you [216:59] need to do is go ahead and copy this [217:01] array. Then just show its distribution [217:04] by going here. You can see we have this [217:06] numpy array. It contains all these [217:08] dimensions. We can go do flatten flatten [217:12] it. You can see we have now one [217:14] dimension because the histogram just [217:16] deals with one dimension. Then we can go [217:18] and plt dot hist to show a histogram. [217:22] Here if we put semicolon you can see we [217:25] have a nice little amazing histogram and [217:28] it's showing this histogram. You can see [217:30] it's like Gaussian distribution. Okay, [217:33] it's like the normal distribution. So [217:35] this is noise image and it is [217:38] representing Gaussian distribution. [217:40] Okay, so random Gaussian distribution. [217:42] If you search here Gaussian distribution [217:45] images, it's like that. You can see we [217:47] have this like of shape the same as [217:50] here. Okay. So let's go ahead and [217:56] input or from pil import image the image [218:01] class then let's go import numpy as np [218:05] you don't have to but let's recreate it [218:08] then you can then we can go image equals [218:10] input divided by two plus 0.5 so we are [218:15] dividing our image by two then we are [218:17] adding 0.5 [218:20] dot clamp and then also need to call the [218:22] clamp function pass in 0 comma 1 then [218:25] dots squeeze. [218:27] So you can see here we are again [218:29] squeezing our image. Why? To get rid of [218:31] that batch dimension. [218:33] Then we have here image equals image dot [218:36] permute. So we can see permute we are [218:39] again permuting and reordering the the [218:42] order of the height width dimensions by [218:45] going 1 2 0 time 255 dot round. Then we [218:50] are rounding here two. Then we are [218:52] moving to the torch dot torch dot u [218:57] integer 8 dot CPU you can see and then [219:00] numpy. [219:05] Then we can go image equals image dot [219:09] from array and then pass in my image. [219:12] And here we have my image. So this is [219:15] the image of the face. As you can see, [219:20] we need to run this again. You can see [219:23] here. Let's let's write another face [219:26] image. [219:29] Then another face. Then we can go down [219:32] and use this code again. You can see [219:36] this for loop and then [219:39] from p import image then divide the [219:41] image. Okay, you can see this face. [219:44] Okay. So because we have overwritten [219:47] over the variable image using this you [219:50] can see using this code. Okay. So in [219:52] order to get this image you have to [219:55] rerun the codes above. Okay. So you can [219:58] see here image it's showing this image. [220:00] And if you run again we have the same [220:01] image but here we have done it using [220:04] auler from the library diffuses. So you [220:08] can see here from diffuses we have [220:10] imported the scheduleuler function [220:12] DDPMuler and unit 2D model unit 2D [220:16] model. So we've done here manually [220:18] instead of this doing it automatically [220:20] here we have done it manually using a [220:22] for loop we started from this random [220:24] noise used a for loop as a loop across [220:28] the time steps here. So we are looking [220:30] here through this time steps and then [220:32] slowly but gradually den noising that [220:35] that noise and recrafting this image. [220:38] You can see we have crafted this image [220:39] from scratch from pure noise. So we have [220:42] we had pure noise then we turned it into [220:45] this face humanlike face by doing this [220:49] code. We use the unit 2D model. You can [220:52] see here this model is representing the [220:54] unit 2D model. Then we pass input comma [220:57] t. Okay. Now let's go to the last [221:01] task which is generating generating [221:06] images using a prompt a prompt with [221:11] diffusion. So this task is one of my [221:14] favorite tasks because we are we're [221:16] going to generate images using a prompt [221:19] with diffusion models or using diffusion [221:21] models. So here remember we need to [221:24] import torch then we have to go to from [221:28] diffusers from diffusers [221:32] import diffusion [221:35] pipeline pipeline. So we need to import [221:38] diffusion pipeline from within the [221:40] diffuser library. Let's go ahead and uh [221:43] set a pipe here. So pipe equals you can [221:46] name it whatever you want. I will name [221:48] it I will name it as image generator [221:51] equals diffusion [221:54] pipeline [221:56] and then dot from pre-trained then we [221:59] have to pass in here the model of the [222:00] stability I so if you go to hugging face [222:03] here hugging face we can go to models [222:07] tab then go to tasks then choose the [222:10] task which is converting text to image [222:13] text to image you can see we have it [222:15] here computer vision It's under cubit [222:17] vision which is text to image. Choose [222:20] that. Here you can see the first if you [222:23] choose or if you sort the models based [222:25] on most downloads. Here we have the [222:28] stability AI stability AI/stable [222:31] diffusion-xl [222:34] 1.0. The same model that we want to use. [222:36] If you go to use this model and then [222:39] diffuse this. You can see you have a [222:41] plethora of images here. Let's copy this [222:43] code. You can see the same code. import [222:45] torch from diffuses import diffusion [222:47] pipeline. Then we have here some [222:48] diffusion pipeline from pre-trained and [222:50] then pass in stability [222:52] aai/stable-defusion-xl. [222:55] So xar-base-1 [222:56] then the d type here is torchbe float 16 [223:00] device map equals cuda. Then here we [223:02] have some prompt aster in a jungle cold [223:05] color palette muted colors detailed 8k [223:08] let's copy that code and then go here. [223:10] So rather than writing this code you can [223:13] easily paste that code here. So feel [223:15] free to paste it here and then run that [223:18] again. You can see now it's trying to [223:20] download that im that model and it's now [223:22] generating the image which is astronaut [223:25] in a jungle. You can see here we have [223:27] astronaut in a jungle cold color palette [223:29] muted muted colors detailed 8K. [223:38] So here in this model card you can see [223:40] they have SD XL base model card. You can [223:43] scroll down here. We have the model [223:45] architecture. We provide a prompt. So [223:47] add the prompt here. Then we have some [223:49] base architecture. Then we have some [223:52] latent space. Then we have a refiner. So [223:54] there's an internal refiner that [223:56] refineses your prompt. So your prompt [223:58] doesn't get inputed the same as your [224:01] prompt. There is some refiner text [224:03] refiner that refineses your prompt or or [224:07] your instructions. Then we have some [224:09] image of the size 1 to4 by 1 to4. So [224:12] here you can see uh model description [224:15] developed by stability AI model type [224:18] diffusion based textto image generative [224:21] model license. This is license model [224:23] description. This is a model that can be [224:25] used to generate and modify images based [224:27] on text prompts. So based on a text [224:29] prompt, you can generate a model. It's a [224:31] latent diffusion model that uses two [224:33] fixed pre-trained text encoders. And [224:36] here if you go to polo cup polo cup [224:39] diffusion model it's a an explanation [224:42] for stable diffusion. So you can see [224:44] here diffusion. If you go to outside [224:46] here you can see it's called stable [224:48] diffusion explained with visualization. [224:51] If you go here you can see first we have [224:54] some prompt a cute and adorable bunny [224:57] with huge clear eyes holding a bunch of [225:00] flowers in the style of cute Pixar [225:03] character. So then we have some text [225:05] representation generator. If you open [225:07] this here we have a tokenizer. So first [225:09] your text gets tokenized and as you know [225:13] tokenization is the process of [225:15] segmenting breaking down or breaking [225:19] down your text into several tokens and [225:22] every single token represents a single [225:24] chunk. So it's the process of [225:26] chunkifying your text. Then we have here [225:29] 77 tokens. Start token that triggers and [225:32] motivates the model. Then we have a cute [225:35] cute is a single token and is a token. [225:37] You can see adorable is a single token. [225:39] Bunny is a token with huge clear. Then [225:42] we have some text encoder. The text [225:44] encoder here as you can see we have [225:46] image encoder and a text encoder. So you [225:49] can see photo of white dog you have [225:51] image encoder text encoder. It tries to [225:53] understand the text and then tries to [225:55] generate an image. Okay. So you can see [225:57] stable diffusion uses a special text [225:59] encoder that connects text with image [226:02] for an image caption pair. The text [226:04] encoder converts the captions tokens [226:07] into a vector that is close to the [226:09] images vector produced by a pad image [226:11] encoder not used in simple diffusion in [226:14] hybrid text image space. Then after that [226:18] these tokens gets mapped into a vector [226:20] for each token. So these tokens are [226:23] mapped to ids. Then for every single ID [226:25] we map it we map it into a vector into a [226:28] vector. So you can see vector for each [226:30] token start has its own vector a has its [226:33] own vector and every single vector has [226:36] some numerical values 0.03 minus 1.33 [226:41] 0.31 and so on. So for every single [226:44] token we have a single vector and these [226:46] vectors contains numerical values. After [226:49] that after the text representation data [226:52] has finished here we can say we can see [226:54] we have the image representation refiner [226:57] so we have the unit so we have the same [226:59] archition that we used earlier which is [227:01] the unit model the unit is a convolution [227:04] network you can see predicts noise you [227:05] can see here [227:08] this is the unit here image [227:09] representation it starts with random [227:11] noise then it does several steps several [227:14] many many steps you can see 8 9 20 27 20 [227:18] 30 until it gets to the coherent last [227:22] image. Okay, until the clear image. And [227:24] here after this image refiner, we have [227:27] the upscale. You can see here we have [227:29] this time step. We can go to one. You [227:32] can see here we start with random noise. [227:34] We start with pure noise and then slowly [227:37] but gradually [227:39] the diffusion model dinoises it denoise [227:41] it and gets rid of the of the noise [227:43] until it creates this really clear clear [227:46] image. It's a very clear image. Okay. So [227:49] this is the process of diffusion. [227:51] Diffusion is one of the greatest ideas [227:53] in the world of deep learning. It starts [227:56] with random noise and then slowly it [227:59] tries to remove that noise. It tries to [228:02] delete that noise to get rid of it and [228:04] then slowly build coherent objects like [228:07] eyes, like hair, like airs and then it [228:11] shows us this nice really appealing [228:13] image which is an adorable bunny. Okay. [228:17] So now the model has finished. You can [228:19] we can visualize the image by going [228:21] image. You can see here it's an astrand [228:24] in a jungle with cold colors muted 8K [228:28] and detailed [228:31] you can see we have this astrand in a [228:33] jungle. Okay. [228:36] This is the the the image from the [228:39] stability AI stable division and that is [228:41] how to generate images using stability [228:44] AI diffusion model. So that is how easy [228:47] you can see it's really easy how to use [228:49] a model from hugging face. All we've [228:51] done actually is just we've gone to from [228:54] diffuses import diffusion pipeline class [228:57] dot from pre-trained method and then [229:00] pass in stability AI/stable [229:03] diffusion xl base then just created and [229:06] crafted some prompt and then pass that [229:08] prompt into a pipe dot images at index [229:11] zero. So let's go to sub AI or to the [229:14] expression here we can see section the [229:16] prompt we can actually copy this [229:19] prompt and then go to [229:23] here. So all we've done is prompt [229:26] prompt equals this prompt a cute and [229:30] adorable bunny with huge clear eyes [229:33] holding a bunch of flowers in the style [229:35] of cute Pixar character. Then we have [229:37] here image equals pipe and then pass my [229:40] prompt into here inside the pipe dot [229:43] images at index zero. Then we have here [229:46] image. So all we are doing here is just [229:50] crafting some prompt passing it to my [229:52] pipe and then images. Then we are here [229:54] you can see we are generating the image. [229:58] Okay. Now behind the scenes it's [230:00] starting with random noise. Then slowly [230:02] it's trying to remove that noise and [230:05] trying to draw that image, trying to [230:07] print to paint that image for us, which [230:09] is a cute and adorable bunny. [230:13] So if you go to here, you can see this [230:15] is the resulted image in the polar club [230:18] tool. Let's compare it to what will be [230:20] outputed in our collab environment using [230:22] this model SAI. And if you choose [230:25] another prompt here, for example, a very [230:27] very very beautiful titcape, you can see [230:29] run that again. It starts here from [230:32] random noise. [230:35] Then slowly it's trying to remove that [230:37] noise and shows a really [230:40] clear city. [230:44] You can see here we have the cityscape. [230:46] Okay. [230:50] And if you select another prompt for [230:51] example a [230:54] a for example here you can go [230:57] to a cute and lovely 4year-old baby. You [231:02] can see this is the clear image the end. [231:04] Then if you time if we play the time [231:07] steps you can see now it's trying to add [231:10] some noise. You can see starting with [231:12] noise then slowly denoising it until it [231:15] creates that logically coherent and [231:18] logically and appealing image. Now if [231:21] you go to here you can see here we have [231:23] an adorable bunny image. You can see we [231:26] have this rabbit. Okay using that prompt [231:30] a cute and adorable bunny with huge [231:32] clear eyes. So you can see it he he it [231:35] has huge clear eyes holding a bunch of [231:38] flowers. So it's holding here you can [231:40] see a bunch of flowers [231:43] orange flowers. Then we have here in the [231:45] style of cute Pixar character and it's [231:48] in the format of Pixar character. If you [231:50] know a Pixar character this is one of [231:51] the this is one of the Pixar characters. [231:55] Then let's go and copy that prompt here [231:59] which is a cute and lovely 40year-old [232:01] baby and paste it over here. We have [232:04] here prompt paste it over here. Then [232:07] just all all you have to do is to go [232:09] ahead and image equals pipe. So go the [232:12] pipe and then pass into here images or [232:15] prompt pass the prompt dot images then [232:19] access index zero then actually show the [232:21] image. [232:25] So if you know the TV static TV static [232:28] images it's the same as the random [232:31] noise. It's starting from random noise [232:33] and then slowly getting rid of that [232:35] noise and painting or drawing the image [232:39] that you ask here. And this model is [232:41] considered a a conditioned latent [232:43] division model because why? Because it's [232:45] conditioned upon our text prompt. You [232:48] instruct it and it's conditioned based [232:50] on the prompt. [233:00] So if you go to here to the polo club [233:03] tool and you can see here what is a [233:06] stable diffusion model. Stable diffusion [233:08] is a text to image. So it's a model that [233:10] converts text to image model that [233:12] transforms a text prompt into a high [233:14] resolution image. For example, if you [233:16] type a cute and adorable bunny, stable [233:18] diffusion generates high resolution [233:20] images depicting that a cute and [233:22] adorable bunny in a few seconds. Click [233:24] select on the prompt here to change it. [233:26] And there they have discussed how it [233:28] works. Stable diffusion first changes [233:30] the text prompt into a text [233:32] representation. As we said, it first [233:34] tokenizes our prompt. So here this [233:36] prompt is first gets tokenized. Then [233:39] every single word gets mapped into a [233:41] vector. Then it tries to understand that [233:44] vector semantically and contextually. [233:47] Then it tries to generate an image. You [233:48] can see here image. This is an image of [233:51] a cute girl for and cute girl. Okay, as [233:55] we asked in the prompt and the last here [233:58] we have a prompt. Let's try a prompt [234:00] that we write here which is a giraffe a [234:02] giraffe [234:04] with top hat and then pass this prompt [234:08] into my pipe the model by going model [234:11] and then pass in my prompt then dot [234:15] images at index zero. Then let's show [234:19] that image [234:24] and here we have called it pipe. [234:27] Okay. So image equals pipe pass in my [234:29] prompt do images at index zero then show [234:32] that image. We can go ahead and run [234:34] that. Now it's trying to generate or [234:36] trying to produce a giraffe with top [234:39] hat. [234:40] So as we said here you can see in the [234:43] polo club tool they have said that they [234:45] have a tokenizer. So first this prompt [234:48] gets tokenized. So for example like that [234:51] a [234:52] giraffe [234:55] is a single token with is the token and [234:57] then every single token gets mapped into [235:00] a vector like this [235:02] 0.5 0.3 0.2 and then this vector [235:07] represents the contextual meaning the [235:09] semantic meaning the spelling and so [235:12] many different perspectives about this [235:14] word. Then we pass it into a text [235:16] encoder. Then it understands the meaning [235:18] of this text and pass this rich [235:21] representation vectors with its [235:23] understanding to an image encoder. Then [235:25] the image encoder will take this and [235:26] understand it and tries to direct the [235:28] image using a diffusion model. [235:33] So the main idea here you can see we [235:35] have text representation. Stable [235:36] diffusion converts a text prompt into a [235:38] text vector representation. Then image [235:40] representation starting with random [235:42] noise. Stable diffusion refineses the [235:44] image representation little by little [235:47] with the guidance of the text [235:48] representation. Stable diffusion repeats [235:51] the refining over multiple time steps. [235:53] For example, 50 in our diffusion [235:55] explainer. Then we have image upscaling. [235:57] Stable diffusion upscales the image [235:59] representation into a high resolution [236:02] image. And feel free to read through [236:03] that in your own time. But this is the [236:05] main idea of the diffusion. And you can [236:08] see here we have this image of a [236:10] giraffe. You can see with top hat. Okay. [236:15] Okay. So that's it. That's how to use [236:19] diffusion models in hugging face [236:21] diffuses models and how to generate [236:24] images with a prompt using a text prompt [236:27] and also how to generate faces without [236:30] impromptising [236:32] diffusion probabilistic models and we [236:35] have taught some skills on how to [236:37] process images using the Python [236:39] programming language ecosystem. [236:44] Let's dive into another modality and [236:47] section of this course which is video [236:49] models and how to use video models from [236:52] hugging face platform. The first thing [236:54] is let's go and name this as video [236:57] models. Let's import the the required [237:01] libraries that for example torch import [237:05] diffuses. The diffusers library is a [237:08] really important library in the world of [237:10] video and images. Whenever you want to [237:13] deal with video modality or video files [237:16] or to generate videos using deep [237:17] learning, you have to import the [237:19] diffusers library from hugging face. [237:22] Then let's go import transformers [237:26] and then at the last import. [237:32] Now let's run that. You can see now it's [237:34] importing those libraries including [237:36] torch, diffusers, transformers and CV2. [237:40] The next step is that let's go ahead and [237:42] name this as generating [237:45] start new section as generating video [237:48] with diffusion. So now let's take how [237:51] would you generate a video using a [237:53] diffusion model diffusion model from [237:55] hugging face. The first thing let's go [237:57] from diffusers [237:59] import [238:01] stable video [238:04] stable video diffusion [238:07] pipeline. So here we have this class. [238:09] You can see it's called stable video [238:12] diffusion pipeline. Let's import this [238:15] class from the diffuses library. You can [238:18] see now it's importing this class. [238:27] And let's wait until the class is [238:29] imported. You can see now the class is [238:31] imported. The third step, let's go ahead [238:33] and dive into it. Let's construct a new [238:36] instance from this class by going pipe [238:39] equals and then let's call that class [238:42] which is called stable diffusion [238:46] or stable video diffusion [238:49] pipeline. [238:52] Let's use that class and then pass into [238:54] here. You can see dot from pre-trained. [238:58] Let's call the method from pre-trained [238:59] and then pass stability [239:02] AI slash stable dash video diffusion [239:09] dash image to vit. So vit is short for [239:12] video xt. So you can see here this is [239:16] the name of the model. Then let's set [239:18] the torch [239:20] type. So the data type of the torch [239:22] should be as torch dot float 16. So why [239:27] you have to set this parameter? If you [239:29] don't set this par parameter, you will [239:32] get the error out of memory because [239:35] remember or keep in mind videos are [239:38] really large files and generating them [239:41] isn't easy. So you have to change the [239:44] precision of the numbers to be float 16. [239:47] Okay, this a really really really [239:50] important setting you have to care about [239:52] and you have to provide it otherwise you [239:55] will get out of memory error variant [240:00] equals FP16 so short for a floating [240:03] point 16 and here keep in mind you have [240:06] to write stability AI/Stable [240:09] video diffusion stable- video diffusion- [240:12] image tovide-xt [240:14] then let's go ahead and load this pipe [240:17] Let's run that. You can see now it's [240:19] trying to load in our model. You can see [240:21] the model is now loading model and index [240:23] of JSON. It's now trying to download [240:25] that model. [240:29] So all we did is just call this class [240:33] stable video diffusion and pipeline dot [240:36] from pre-trained and then pass in the [240:38] name of the model, torch dype as float [240:41] 16 and then set the variant setting as [240:44] FP16. [240:45] Okay, like that. And then we have the [240:47] model here or the pipe. Let's now go the [240:50] pipe to see it. [240:53] Now this is the pipe. You can see here [240:54] we have this stable video diffusion [240:57] pipeline. We have class name stable [240:59] video diffusion pipeline. Diffuses [241:01] version 0.36.0 [241:03] name or path stability AI/stable [241:06] video diffusion- image to video-ext [241:09] feature extractor transformers clip [241:12] image processor and so on. Now let's go [241:14] ahead and you can see here I have an [241:17] image. I already have an image here [241:19] called Elmo.jbg. If you open it, you can [241:22] see this is an image of Elmo. Let's [241:25] let's take the copy of this image and [241:27] paste it inside the variable called my [241:29] image or for example path equals [241:32] Elmo.jpg. [241:34] So this is the path. This is the path of [241:37] Elmo.jpg image stored inside a variable [241:41] called path. Run that. You can see a [241:43] path is a string a normal string a [241:46] regular string containing the word [241:48] elmo.jpg jpg. Now to show this image [241:52] let's go ahead and [241:54] from diffuses diffusers dot utils. So [241:58] inside the utils library or utils [242:01] package which is short for utilities [242:04] import the following. The first is load [242:07] underscore image. So this function is [242:09] really really helpful. It's from the [242:12] utilities library from within diffusers. [242:15] It helps you to load the image and show [242:17] it. Then comma we have this also [242:19] function export to video to later export [242:23] our video frames into a video. Okay. So [242:27] let's from diffusers dot utils import [242:30] load image, export video and let's go [242:32] ahead and image equals. So here we have [242:36] a path. Remember we sort the path of the [242:38] image inside a variable called path. [242:40] Let's use this function load image and [242:44] pass into here the path of the Elmo [242:46] image. [242:49] Now once you write here the name of the [242:51] variable image, we have a surprise. [242:55] The Elmo image is loaded [242:58] into the environment of Google Collab of [243:01] Google Collab. You can see we have Elmo [243:04] image loaded inside the collab [243:06] environment successfully without any [243:09] problems. [243:11] So the idea here all we have done is [243:13] from diffuses utils import load image [243:16] comma export video then we pass the path [243:18] to load image and we showed that image [243:21] successfully. Here you can see we have [243:23] an important setting that we have to do [243:26] which is pipe dot enabled_model [243:32] CPU offload. This is a really important [243:37] setting that you have to use this [243:39] function enable model CPU offload. You [243:41] have to call this function on top of [243:44] pipe. Why? Because to not get the error [243:47] of out of memory [243:49] as you can see. So keep in mind you have [243:52] to run this code. This code is [243:54] necessary. Then let's go ahead and [243:58] dive straight away into how to use this [244:01] model. First let's set a generator [244:06] equals torch dot manual seed to set a [244:09] manual seed to get reproducible results. [244:13] So in order to get reproducible results, [244:16] the same results, you and I have the [244:18] same results, the same video frames, you [244:20] should have to set u manual seed as 42. [244:24] So now we have a generator generator. [244:27] You can see it's a normal uh torch [244:30] manual seed function set as 42. So feel [244:33] free to play around this value. You can [244:35] set it whatever you want, but in this [244:37] case I will keep it as the regular [244:39] normal seed 42. Now let's dive into the [244:44] step of of creating the actual frames by [244:47] going frames equals pipe. So remember we [244:49] have created this pipe earlier. You can [244:51] see here pipe at the top here we have [244:54] here pipe equals [244:57] the class that we have imported earlier [245:00] stable video diffusion pipeline dot the [245:02] method from pre-trained and then pass [245:05] here the information about the model. [245:07] Now let's use this pipe and pass in my [245:10] image and then the model will work and [245:14] produce a video and generate a video out [245:17] of this image by going image. So pass my [245:20] Elmo image. Here we have image. You can [245:22] see it's an [245:25] it's an Elmo image. Let's pass it to our [245:28] pipe. Then comma decode trunk [245:34] size equals 8. So set the decode chunk [245:37] size as eight. This is a really [245:39] important parameter. You have to set it [245:41] set it as eight comma generator and then [245:44] pass in my generator which is if you [245:47] remember our seed okay dot frames at [245:51] index zero. Why? Because this class will [245:54] return a a kind of object. You have to [245:58] access the attribute frames and get the [246:00] frames at the index zero. So this is a [246:03] really important chunk or really [246:04] important piece that you have to provide [246:06] in order to get the frames straight [246:08] away. Let's go ahead and run that. You [246:10] can see now it's trying to generate a [246:14] actual an actual video out of this image [246:16] which is Elmo image. [246:20] Now the model is working. You have to [246:21] keep in mind I got I went to runtime [246:24] change runtime type and then chose [246:26] L4GPU. Otherwise, if you are working on [246:28] CPU, it will take at least 40 minutes to [246:33] generate a video. So, keep in mind you [246:35] have to rent GPUs. You have to buy GPS [246:38] from Google Collab and then use L4 GPU [246:41] or A100 GPU or whatever GPU you have [246:44] available. The the more powerful GPU, [246:46] for example, L4, A100 will take less [246:48] than 1 minute to generate a video. But [246:51] otherwise, if you are using CPU, keep in [246:53] mind it might it might take at least 40 [246:56] minutes. [246:59] You can see here we have little nice [247:01] progress bar showing us the progress of [247:03] this model. Now this pipe is running [247:06] behind the scenes running behind the [247:08] scenes a diffusion model. The diffusion [247:10] model is the model that is responsible [247:12] for generating pixels and generating the [247:16] video frames from this image. So here [247:18] only we are providing the image. We are [247:21] not providing a prompt. Later in this [247:22] lecture we will see that we will use [247:25] another model from hugging face. Then we [247:27] will provide an image alongside with a [247:30] prompt. Then the model will be able to [247:32] generate a video. But here this model [247:34] which is the default model here that we [247:36] have used which is called stable video [247:39] diffusion pipeline. We have imported it [247:41] from this library from diffuses import [247:44] stable video diffusion pipeline. This [247:46] model is generating a video out of a [247:50] single image. We are not providing any [247:52] prompt. And if you go to here and search [247:54] stable video diffusion paper, this is [247:57] the the real paper or the actual paper [247:59] that introduced and proposed this model [248:02] from stability AI. You can see stable [248:04] video diffusion scaling latent video [248:07] diffusion models to large [248:10] on nove on 25th November 2023. And if [248:13] you go to VPDF, it will show us that [248:16] here we have stability AI. This is the [248:18] the paper that they proposed this model [248:21] stable video diffusion [248:25] and feel free in your own time to read [248:27] this paper at your own pace. But here in [248:30] the abstract they said we present stable [248:33] video diffusion a latent video diffusion [248:36] model for high resolution [248:37] state-of-the-art text to video and image [248:40] to video. Here we are using this [248:42] modality image to video. We are using [248:44] this paradigm converting an image into a [248:47] video but also you can use text to [248:48] video. So you provide a text prompt then [248:51] it converted into a video. Recently [248:53] latent diffusion models trained for 2D [248:56] images synthesis have been turned into [248:58] generative video models by inserting [249:01] temporal layers and fine-tuning them on [249:03] small highquality video data sets. [249:05] However, training methods in literature [249:08] vary widely and the field has yet to [249:10] agree on a unified strategy for for for [249:14] curating video data. And if you go down, [249:17] you can see they generated so many [249:19] images and so many you can see here we [249:22] have so many images and so many videos. [249:24] These are video frames. So one frame, [249:26] one second frame, third frame, fourth [249:28] frame. So this is a video. Okay, you can [249:30] see they provide one image and then the [249:32] model tries to manipulate that image and [249:35] create many many many frames. Here the [249:37] model has finished. You can go and show [249:40] the frames. You can see here we have a [249:42] nice list whoa containing so many images [249:46] and every single image is in the form of [249:48] pill. And if you don't know pill, pill [249:51] is short for python image library or [249:53] python imaging library. So if you wanted [249:55] to access one of these images, you can [249:57] go at index zero. Then it shows us that [250:01] image. You can see this is the first of [250:03] this is the first image of Elmo. This is [250:05] the second image of Elmo. [250:08] This is the third at index two. The [250:11] fourth image. [250:14] You can see Elmo is going down. [250:16] Every time Elmo is going down. Okay. And [250:20] these are the frames. So feel free to [250:23] play around with these images. Here you [250:24] have a nice list containing of if you [250:28] take the length of it here we have how [250:29] many frames do we have? We have 25 [250:32] frames. We have 25 images different [250:35] images inside this list and every string [250:37] image represents a single scene [250:39] represents a single frame. [250:42] Now let's go ahead and export it by [250:44] going export to video. Remember from the [250:48] utilities library or from the utilities [250:50] package we imported here a function [250:53] called export to video that will help us [250:57] to export these frames into a combined [251:00] video. It will combine these images into [251:02] one video file and store it in your [251:05] collab environment. Then you will be [251:07] able to download it and visualize it. So [251:10] let's use that function. Export to [251:11] video. Then just pass in the frames. [251:14] Pass in the list. this list this great [251:17] list of frames pass it into this [251:18] function frames then we will [251:22] you have to name this video I will name [251:24] it as Elmo Elmo do [251:27] mp4 elmompp4 [251:30] then all we have to do is just set the [251:32] fps equ= 7 so here frames per second [251:36] this parameter is short for frames per [251:39] second so how many frames do you want to [251:41] show per second I want to show seven [251:44] frames per a second. Okay. Now let's [251:47] open this section and 3 2 1 run the [251:52] cell. [251:54] And you can see here when we refresh we [251:57] will get a video. You can see it's [251:59] called Elmo.mpp4. [252:01] Now let's click that video. Double click [252:04] it will download that video [252:05] automatically. You can see it takes a [252:08] little bit of time but no worries it [252:10] will download that in any time. Now [252:12] let's click on that video. You can see [252:19] this is a video of El of Elmo jumping up [252:23] and down. You can see Elmo now is a [252:25] video. Elmo isn't anymore one single [252:28] image. It's now a moving video jumping [252:30] up and down. Okay. So, feel free to grab [252:34] any image to to fetch any image of your [252:36] of yours and try to pass it to this [252:39] model. It's called stable video [252:41] diffusion. And you can see how easy it [252:44] is to use it just in four lines of code [252:46] or five lines of code. You have the [252:49] ability to use a deep learning model to [252:52] convert one image into a video. As they [252:55] mentioned here, it's easy to convert one [252:57] image to a video or the second modality [252:59] text to a video. So you can see here we [253:02] have here two options text to video or [253:06] image to video. Now we have used this [253:07] paradigm image to video. We provide it [253:10] with one single image. [253:12] with one single image which is this Elmo [253:14] image a static Elmo. Then we convert it [253:18] into a video using this pipe our model. [253:21] Pass in the image decode chunk size as [253:24] eight and then set generator as [253:26] generator to get reproducible results. [253:28] Then get the frames add index zero. Then [253:31] all of a sudden all the surprise you [253:33] have a massive list of all those images. [253:36] Then we tried to export it to a video [253:39] and it did that. And then we have a [253:41] video here called LMP MP4. Now let's go [253:44] to hugging face [253:47] here. You can go to tasks or models [253:51] tasks. Then we have here multimodel. We [253:54] have here computer vision. We have NLB [253:56] and we have audio. Let's go to here to [254:00] multimodel. You have the option to [254:02] convert image to a text. So image text [254:06] to it's called image text to video or [254:09] let's call it image to video. Yeah. Yes. [254:13] This is the option that I want to point [254:16] out. Image to video like we did here. We [254:19] have a plethora of models. You can see [254:21] we have so many models. We have [254:23] mountains of models. You can see 4484 [254:25] models. A big number. Uh the first model [254:28] is by Quark Vision Live Avatar. Uh [254:31] updated 7 days ago. You can see it has [254:34] 151 likes. Here we have also stability [254:37] AI stable video diffusion model image to [254:40] video. And I believe if you go to saw [254:43] and then choose most downloads here the [254:46] most or most likes most likes here [254:49] stability AI stability video diffusion [254:52] is the most model that has the the most [254:55] amount of likes. So if you sort by most [254:58] likes here the first model is the model [255:01] that we have used previously stability [255:04] AI/stable video diffusion image video [255:07] and if you click on it these are the [255:09] images that they have made. You can see [255:10] these are some videos. If you grab some [255:12] earth earth image and pass to the model [255:15] it will round that earth make it moving. [255:17] The same with the with the clouds and [255:19] the sea. You can see we will try with [255:21] the sea. We will try to pass an image of [255:23] C and it will animate the sea and make [255:27] the waves animating and there we have we [255:29] have these buildings. So feel free to [255:32] play around with these models to explore [255:34] explore explore experiment experiment [255:37] experiment. Here we have use this model [255:40] diffuses. So you can remember that [255:42] previously when we clicked on use this [255:44] model it was here transformers. It was [255:48] pointing out to transforms. But now once [255:50] we got into the models of video, hugging [255:53] face automatically changes this option [255:55] to diffuses. And you can see we have [255:57] here we have diffuses library. If you [255:59] click on it, it will show us a model. [256:02] Then here is the uh the guide on how to [256:05] use this model. First from diffusers [256:08] import diffusion pipeline. Then [256:10] diffusion pipeline. From pre-trained [256:12] pass in here stability AI/Stable [256:15] diffusion. Then we have here a prompt. A [256:18] man with short gray hair plays a red [256:20] electric guitar. Then we have this [256:22] image. So they provided here with prompt [256:25] and image. Then the pipeline will work [256:28] and produce output a MP4 video. Now [256:32] let's get back to our notebook and [256:34] complete our lesson. [256:36] Let's jump into another topic here which [256:38] is generating [256:40] video using [256:44] another diffusion diffusion [256:47] model. [256:49] This time using image and prompt. So not [256:53] only an image but also a prompt. Provide [256:56] the the model with the image and prompt [256:58] by going import torch. Then let's go [257:02] torch.cuda CUDA dot empty cache. [257:07] Then let's call that function. You can [257:09] see here we have empty the cache. So you [257:11] have to empty the cache or empty the [257:13] cache before you move on to the next [257:15] step. Why? Because torch. CUDA empty [257:18] cache makes sure that our our CUDA isn't [257:22] busy with any other task. Let's go from [257:25] diffusers from diffusers import [257:29] a function or a class called I2 to V gen [257:34] XL pipeline. So what basically this [257:37] class is short for I image to video. So [257:41] converting image to video generator or [257:45] generation Xlarge model pipeline. We [257:48] need to import this class from the [257:50] diffuses library. So from diffusers [257:53] import I2V gen XL pipeline. Then let's [257:57] go ahead and here you can see from [257:58] diffusers from diffusers import dot [258:02] utils. So from the utility functions or [258:04] from the utility library please go ahead [258:07] and import the following. Load image, [258:10] export video. And here you can see oh we [258:13] used this function before. Load image is [258:16] used to load an image. Export video is [258:18] used to let us export a video. So we've [258:22] seen that before. Then we have uh to [258:24] create a pipeline. Pipeline equals diff [258:27] I. So image to video generator [258:31] XL [258:33] pipeline. So I [258:37] I2V generator XL pipeline. [258:41] Then let's go ahead and pass into here [258:43] dot from pre-trained. [258:45] We have to provide into here the repo [258:48] ID. So this time let's store it in a [258:50] variable called repo [258:54] id equals ali dash vl vl [259:00] / i2v [259:02] it's its meaning is image to video [259:06] generation or generator dash xl xar this [259:10] is my repo ID so a li-v [259:14] i l/ [259:16] i2 video generator - XL then let's pass [259:20] here the ripple id. So instead of [259:21] manually writing the ripple ID here what [259:24] if we store it in a variable and use [259:27] that variable it will make your code a [259:28] lot neater. Then you have here torch dt [259:31] type equals torch.flat6. [259:34] Please keep in mind you have to set it [259:36] like that otherwise you will get a out [259:39] of memory error because as we told you [259:42] we are dealing with video data video [259:44] modality which is a really massive or [259:47] intensive computing modality. When you [259:49] compute on images on videos on audio the [259:52] model will be heavy in computing will be [259:55] in compute computationally intensive. So [259:58] you have to lower down the precision to [260:00] be float 16. If you use float 32, you [260:03] will be out of memory. So we have keep [260:05] keep in mind make sure you are using [260:06] float 16. Then let's go ahead and set [260:10] the variant equals to FP16 floating [260:14] point 16. Now let's run that. You can [260:17] see now it's trying to download in the [260:19] model to load the model in which is this [260:22] model I2V generator XL pipeline from [260:26] pre-trained. [260:28] So as you can see the progress bars are [260:30] going are moving [260:33] forwardly. Then let's go ahead and set [260:35] the pipeline. So by by by going pipeline [260:40] dot enable underscore model cpu offload. [260:46] This function is really is really [260:48] important because it will make sure that [260:50] we not that we are not getting the out [260:52] of memory error. You have to enable [260:55] model CPU offload. Then let's go ahead [260:58] and here I have you can see [261:02] an image of C. If you open it, you can [261:04] see we have now another image which is a [261:07] view of a C or an image or a picture of [261:09] C. Let's now use this picture to [261:11] generate a video using another model [261:14] which is this model from the repo ID. [261:17] This it's called I2V gen XL pipeline by [261:20] going ahead and loading the model as we [261:23] did before. Here we can we can go C [261:26] equals load image and then pass in my [261:30] C.jpg. [261:32] So all you have to do is just grab an [261:35] image, store it over here, call the path [261:38] of this image, pass it to this function, [261:40] load underscore image, then you have to [261:42] store it inside some variable. I called [261:44] it C. If you now visualize or print out [261:48] the C variable, it will show us that we [261:50] have now a picture or a view or a scene [261:53] of a C. You can see we have this image [261:56] of a C, normal image. It's a static [261:59] image. Note that when we pass it to a [262:01] model, it will animate that C. It will [262:04] make this C be animated, moving. It's [262:07] not static anymore. So by going prompt [262:10] equals [262:12] the sea for example and uh you can be as [262:15] as creative as you want. So the sea [262:18] waves heavily. [262:21] I will keep it simple here. You can feel [262:23] free to prompt and to to ask or to [262:25] instruct the model as you would like. [262:28] Here I just told the model, hey take [262:30] this image and try to make the sea waves [262:33] heavily. You can get as creative as you [262:36] would like here. But now let's go ahead [262:38] and just set the generator equals torch [262:41] dot manual seed as 42. Then frames. So [262:46] get back the frames. This pipe will [262:49] return back a frames a set of frames [262:51] stored inside the form of a list by [262:54] going pipeline and then pass in prompt [262:57] equals the prompt [263:00] equals prompt, [263:02] image= c, comm, num frames. So, how many [263:06] frames do you want to generate? In this [263:08] case, I want to generate 16 frames. Keep [263:10] in mind, the higher the number, the more [263:13] time it will take to produce that video. [263:15] And the lower the number of frames, the [263:18] faster the model will run to generate [263:20] your video. So, you have to keep it [263:22] balanced. But if you have infinite [263:23] amount of resource, if you have infinite [263:25] amount of computing, feel free to set it [263:28] as to for example [263:30] 164. But let's keep it simple and quick [263:33] by setting this to 16. Then we have here [263:35] generator equals generator [263:39] dot frames and then get or fetch the [263:41] frames at index zero. So you don't have [263:44] then to access it later. Here we are [263:46] accessing straight away dot frames fetch [263:49] at index zero. [263:51] Now let the model run. [263:54] So by the way this model which is called [263:57] I2V gen XL pipeline if you copy this [264:00] name go to hugging face this is the [264:03] stability AI model don't forget go to [264:06] models and then filter by name you can [264:09] see we have this model I2V [264:13] image to video [264:18] it's it's called this ripple you can [264:20] copy the name of the ripple from Yeah, [264:26] you can see this is the model. Okay. So [264:29] this is the model V gen. It's called [264:31] it's from Ali Alibaba company-VAB/I2 [264:37] imagetovideo generator-XL. You can copy [264:39] the name from here. If you don't want to [264:41] type it later, you can give it a like if [264:44] you want. You can follow this account. [264:46] Here you can see downloads last month. [264:48] They have 2,729 [264:52] downloads in the last month. Here you [264:55] can see VGEN is an open- source video [264:57] census codebase developed by Tongi lab [265:00] of Alibaba group featuring the [265:01] state-of-the-art video generative [265:03] models. This repository includes [265:05] implementations of the following methods [265:08] and you can read through that in your [265:10] own time. But here the most important [265:11] idea is if you go down they have here [265:14] some guides on how to use this model. [265:17] You can see from diffuses import I2V gen [265:20] XL pipeline. So that's what what we did [265:22] before and then you can provide it with [265:24] image URL you can provide this model [265:27] with a you can see image then prompt [265:31] papers were floating in the air on a [265:33] table in the library then they provide [265:35] the image they supply the model with [265:38] image and prompt then they pass it as we [265:40] did you can see frames so this pipeline [265:43] returns frames they pass here prompt [265:45] equals prompt so as we did here you can [265:47] see if you go down scroll down here we [265:50] have prompt equals prompt image equals C [265:53] and they passed it like that. Image [265:55] equals image generator equals generator [265:58] then dot frames at index zero. [266:01] You can see here they did what we did [266:03] exactly. We have here dot frames and [266:05] then they accessed at index zero. Then [266:08] at the last they export this uh frames [266:10] into a GIF. In this case we will export [266:12] it into a video and then they passed [266:15] straight away the frames. Okay. So you [266:18] have here a plenty of explanations on [266:20] how to use this model, how to use it. [266:23] Okay. And this and this is the result. [266:24] You can see papers. They ask the model [266:27] to generate papers were floating in the [266:29] air. So here you can see the papers are [266:31] floating in the air. And also this uh [266:34] prompt is really important on a table. [266:36] You can see here on a table. So they [266:39] have a table. So you can see how how [266:41] great this model is at capturing the [266:43] meaning and the contextual meaning of [266:45] the prompt. understanding the latent of [266:48] the text and then combining that [266:50] understanding with a visual [266:51] understanding of the image and then [266:53] using a diffusion model they generated [266:55] this great masterpiece of art. Okay. So [266:59] feel free to go as creative as as you [267:02] would like. Now the diffusion model the [267:05] first paper that introduced it was by [267:07] Josh which is titled deep unsupervised [267:12] learning using non-equilibrium [267:14] thermodynamics. This is the original [267:16] paper that introduced the model [267:18] diffusion by Josha S Dixon. You can see [267:20] a central problem in machine learning [267:22] involves modeling complex dead sets [267:24] using highly flexible families of [267:27] probability distributions in which [267:29] learning sampling. Sampling is another [267:32] word or is a fancy word for generating [267:34] inference and evaluation are still [267:36] analytically or computationally [267:38] tractable. Here we develop an approach [267:41] that simultaneously achieves both [267:43] flexibility and tractability. So all the [267:46] models like VAE GANs and Bosma machines [267:49] they are all generative models but they [267:52] are intractable. But diffusion is a [267:54] model that is both tractable and [267:56] flexible capable of modeling complex [267:59] probability distributions and at the [268:01] same time it's yet tractable. The [268:04] essential data inspired by [268:05] non-equilibrium statistical physics is [268:08] to is to systematically and slowly [268:10] destroy structure in a distribution. So [268:14] so the way the diffusion model operates [268:16] is it it starts with a with a random [268:19] noise. It starts with an initial random [268:22] noise. Then it slowly slowly uh uh [268:26] during the training phase it starts with [268:28] a the original image. Then it tries to [268:31] destroy that structure in the image. [268:34] Then at the inference time it starts [268:36] from random noise. Then it tries to [268:38] generate this [268:41] image. For example, a coherent and [268:44] logically inspiring and logically [268:46] appealing image. So it starts with [268:49] random structure. It destroys structure [268:51] in a data distribution through an [268:53] iterative forward diffusion process. We [268:55] then learn a reverse diffusion process [268:58] that restores structure in data. So as I [269:00] told you during the training phase, the [269:03] diffusion model starts with a logically [269:06] coherent image. Then it slowly adds [269:08] noise adds noise until it destroys [269:10] structure in the image. Once the image [269:12] is destroyed, there's no any meaningful [269:15] logical coherence or there's nothing to [269:18] infer from it. the diffusion model [269:21] starts to redraw that image or to get [269:24] how to reverse how to do a reverse [269:26] diffusion uh process to get to the [269:28] original image. Then during the [269:30] inference mode, the the the model is [269:33] asked with a prompt to generate a new [269:36] image or a specified image and it starts [269:39] from random noise then it tries to [269:41] generate that image and as we saw here [269:44] with the Elmo video we give it an image [269:46] then it tried to move and animate Elmo [269:50] image. So this these are all diffusion [269:52] models and evaluate probabilities in [269:54] deep joint models with thousands of [269:56] layers on or time steps as well as to [269:59] compute conditional and posterior [270:01] probabilities under the land model. We [270:04] additionally release an open-source [270:05] reference implementation of the [270:07] algorithm and later there's a paper by [270:11] Yang diffusion paper you can see by [270:16] Yangsung the principles of diffusion [270:18] model. So if we go to [270:24] yeah this is Yangsung here we have [270:26] scorebased. So Yangong here what he did [270:28] in this paper with his friends [270:33] they [270:36] avoided modeling the pro distribution by [270:39] computing a score based generative [270:42] modeling through stocastic differential [270:44] equations. So in this paper which is was [270:47] in 2021 [270:49] they avoided modeling the complex [270:52] probability distribution. So it is it [270:54] was submitted on 26th November 2022 uh [270:57] 2020 and last revised 10 Febru uh 10 [271:01] February 2021. [271:03] They here in this paper they proposed a [271:05] way to avoid modeling complex complex [271:09] and in highdimensional probability [271:11] distributions via using a scorebased [271:14] generative modeling through stocastic [271:16] differential equations. So you can see [271:18] they have this we present a stocastic [271:20] differential equation SDE that smoothly [271:22] transforms a complex data distribution [271:25] to a known prime distribution by slowly [271:27] injecting noise. So you can see here by [271:30] slowly injecting noise. The the [271:32] deficient model operates by injecting [271:34] noise and destroying that image [271:36] structure, then slowly getting back and [271:39] returning back, learning how to return [271:41] back to this to to the image and a [271:43] corresponding reverse time SDE that [271:46] transforms the prior distribution back [271:48] into this distribution by slowly [271:50] removing the noise. Crucially, the [271:53] reverse time SDE depends only on the [271:55] time dependent gradient field aka score. [271:59] So they named it as score of the of the [272:01] perturbed data distribution by [272:03] leveraging advances in score-based data [272:06] modeling. We can accurately estimate [272:08] these scores with neural networks and [272:11] use numerical SDE solvers to generate [272:14] samples. We showed that this network or [272:16] this framework encapsulates previous [272:19] approaches in score-based generative [272:21] modeling and diffusion proistic modeling [272:23] allowing for new sampling procedures and [272:26] new modeling capabilities. So feel free [272:28] to read that in your own time. Here just [272:30] we are uh giving an introduction to [272:32] diffusion model and what is the core [272:34] band diffusion. So the core idea is that [272:36] diffusion model is a is a genative model [272:39] mostly used to generate videos and [272:42] images. It starts with random noise and [272:45] slowly building structure and coherence [272:47] and coherent meaning in the image or in [272:50] the pixels by reordering the pixels by [272:52] shuffling the pixels around. Okay, you [272:54] can see it has heavy math. Then you can [272:57] you can go down you can see the fields. [272:58] These are these are the gradient fields [273:01] that the score you can see overview of [273:03] score-based generative modeling through [273:05] SDS. And if you scroll down you can see [273:08] here they generated some faces [273:11] using a diffusion model. [273:13] Now let's get back to our collab. You [273:16] can see now the model has finished uh [273:19] generating. Now we have here frames. [273:25] These are the frames stored inside the [273:28] list. And if you wonder what is the [273:29] length of this of the frames, we have 16 [273:32] frames, 16 scenes, 16 images as we have [273:36] specified over here. So feel free to [273:40] play around [273:42] and it's fun to play around with these [273:44] values to play around with the number of [273:47] frames. [273:48] Here the number is 16. The same as we [273:51] did here. Now let's go ahead and export [273:54] it by going export to video and then [273:56] pass in my frames, [273:58] C [274:00] name it whatever you would like. I will [274:02] name it as C waves_mpp4. [274:07] Then let's go frames per second equals [274:10] 4. So I want to show four frames per [274:13] second. [274:20] Here you can see cwaves.mpp4. Let's go [274:23] ahead and download that video. You can [274:26] see it's a video.mpp4. [274:35] And these are the frames. You can see [274:37] now the C is moving. It's now animating [274:40] that image. It's now it's no more it's [274:42] no longer a single image. It's now a an [274:45] a a an animating video animation. And [274:48] you can play around this value. FPS make [274:51] it as eight. You can see now we will [274:54] visualize eight frames per second. And [274:56] then [274:58] download that video again. You can see [275:01] it will show us eight frames per second. [275:09] Okay. Now it's showing us eight frames [275:11] per second. [275:13] So here we have reached the end of the [275:16] video models. You can see we have used [275:18] both modalities using an image or a [275:21] single image to generate a video and [275:23] also how to instruct a model via a [275:25] prompt plus an image to generate a [275:29] video. [275:30] Keep in mind hugging face has a plethora [275:32] of models. You can go to models then [275:35] tasks then choose here the task image to [275:39] you can see here we have image to video [275:42] and also we have here text to video as [275:44] we did here we have 6 863 models [275:48] dedicated to the field of text to video [275:50] for example 10 cent huan video [275:54] it has here a way on how and on how to [275:58] use this model and here we have the they [276:00] have the model card okay so Feel free to [276:03] play around with any of these models in [276:05] your own time. But now we have taught [276:07] you the way on how to use video models [276:10] from hugging face. See you in the next [276:12] section. [276:15] Now in this section, let's dive into [276:17] Gradio. Gradio library allows you to [276:20] create graphical user interfaces for [276:23] your machine learning models. So enables [276:26] you to create GUIs for your models. You [276:30] don't have to worry how to create the [276:32] front end web development only you need [276:35] to just to create a model wrap it inside [276:38] a function then pass that function into [276:40] gradio library and it will do all the [276:43] job for you and create a nice front end [276:46] interface for you. The first thing is [276:49] let's name this as gradio [276:52] or gradio and then import it. Import [276:55] gradio as G. Then import torch. [277:00] Import numpy as np. And then from pil [277:05] from python image library import image. [277:10] Then if you need to check the gradio [277:12] version, you can go that you can go with [277:14] that import gradio and then tab into [277:18] gradio dot version underscore [277:21] underscore. [277:26] So we can go gradio and dot [277:30] version [277:31] and it will retrieve the version of [277:33] gradio and currently at the time of [277:36] recording this video it's 550.0. zero. [277:40] Maybe in the future you will see a [277:41] higher number here but it will work for [277:44] you as we do here. So the first thing [277:48] let's go and deal with gradial numbers. [277:51] So how to deal with gradual numbers? How [277:54] do you create a number field or a number [277:57] input in gradio? We have to create a [277:59] function define called add underscore [278:03] numbers and then pass in x comma y. Then [278:06] all this function does is just return x [278:10] + y. So this function is a really simple [278:13] function. It's take it takes here x [278:16] comma y. It takes two parameters then it [278:19] adds these parameters and returns them. [278:21] So return x + y. And if you call this [278:24] function add underscore numbers and then [278:26] pass 10, 20, it will add these two [278:29] numbers and return 30. And if you pass [278:33] here 50, 20, it will add them. So 50 + [278:37] 20, we will have 70. Okay, so it's a [278:41] really simple function. Now let's create [278:43] a gradial interface by wrapping this [278:46] function inside the interface class. So [278:49] all you have to do is just define a [278:51] variable iface or interface. Let's call [278:54] it interface equals gradio.inface. [278:59] Then we call the class interface. So you [279:01] can see here we have the class [279:02] interface. This is the most common class [279:05] that you will use inside the gradual [279:07] library. It's the main class, the major [279:10] class interface. It takes from you three [279:12] parameters. Fn short for function. And [279:16] as you can see the function is add [279:19] numbers add numbers, [279:22] let's pass in some inputs equals a list. [279:25] Here you can see we have two inputs. So [279:28] if you have multiple inputs you have to [279:30] pass them inside a list and then inside [279:32] this list let's define grado dot number [279:37] grado dot number. So this function takes [279:40] from me two inputs x and y and we have [279:44] to provide two input fields. This is x [279:47] and this is y. Then comma let's pass in [279:51] outputs equals gradio dot number. And [279:54] this function also returns a number. So [279:56] the addition of x + y will be also here [279:59] a number. So we have to return that as [280:03] in the form of a number using the class [280:05] number. You can see here n is capital [280:07] number from the gradio library. Now the [280:11] the last step is interface. [280:14] We have to launch it. So call the method [280:17] launch to launch this interface and to [280:19] have it ready. Now let's run that. You [280:22] can see now it's preparing a gradio [280:25] interface. And it will show us these [280:27] inputs. You can see we have now x. We [280:30] can type here whatever number you want. [280:32] For example, five. 5 + 10. Submit. It [280:37] will return here 15. And if you type [280:39] here, for example, x as 10, y as 30, 10 [280:44] + 30, it will be 40. Okay? Feel free to [280:48] try whatever numbers you would like. [280:50] Here we have it looks like you are [280:52] running gradial on a hosted Jupiter [280:55] notebook which requires share equals [280:57] true. If you are running this code on a [281:00] local Jupyter notebook it will not give [281:03] you this warning or this kind of [281:05] information. But if you are running it [281:07] on collab then you have to provide here [281:10] the parameter share share equals true [281:14] then it will disappear. This info will [281:17] will disappear. Okay. So you can see [281:20] three two one run that this notification [281:23] will disappear. You can see now it [281:25] disappear. Here we have another kind of [281:28] info collab notebook detected. So now it [281:31] detects that we are using collab to show [281:34] errors. If you want if you want to show [281:36] errors if you have some errors here and [281:38] you want to debug those errors set debug [281:41] equals true. But in this case we don't [281:42] have any errors. So this is a little [281:44] notification. Okay. If you want to debug [281:47] just set debug equals true. In this case [281:49] we don't. Then here we have some kind of [281:52] important information. This share link [281:54] expires in one week. So we have here a [281:57] live link. If you open it in a new tab [281:59] it will show the gradio app. You can see [282:03] here we have another uh interface GUI [282:06] graphical user interface using gradio [282:09] library or gradio package. If you pass [282:11] here 100 plus 200 it will be 300. [282:16] Submit. You can see it works. Okay. So [282:19] this link here expires in one week for [282:22] free permanent hosting. If you want to [282:24] have a permanent hosting forever and GPU [282:28] upgrades run gradio deploy then you can [282:31] deploy it on hugging face to hugging [282:33] face spaces and we'll see that later. So [282:36] now let's go ahead and continue learning [282:38] about what are the different [282:39] capabilities and features offered by [282:41] gradial. Also we can do the same thing [282:43] by the function sub numbers which [282:46] subtracts numbers x comma y. Now we we [282:49] can go faster than earlier return x [282:52] minus y. Here this is a function called [282:55] sub numbers. So subtract numbers. It [282:58] takes from me two different numbers x [283:01] and y. Then it returns x minus y. Now [283:05] let's go ahead and create an interface [283:06] by going interface equals gr. So [283:09] gradinface [283:11] and as I told you this is the main class [283:14] the most important class inside gradio [283:17] it creates an interface a graphical user [283:20] interface for you a user friendly [283:23] interface function equals sub numbers so [283:27] the name of the function comma inputs [283:30] equals x list and then pass grado dot [283:34] number and then grado dot number [283:38] and we have also Now it puts equals [283:40] gradio dot number. [283:42] So it takes from me it receives two [283:45] different numbers and it returns the the [283:48] subtraction of that of those two numbers [283:51] in the in the form of a number. Now [283:53] let's launch it interface.launch [283:58] and then all you have to do is just run [284:00] the cell. [284:03] You can see now it's showing the [284:05] interface. [284:11] Here we have X if you provide 8 and then [284:15] Y 4. 8 - 4 we have here four. If you [284:19] provide 12, 12 minus 10 is 2 and so on. [284:26] You can see how easy it is to create a [284:28] really pretty web interface or front-end [284:32] interface using gradial. Now let's go [284:35] ahead and do the another task of [284:38] dividing numbers. So divore [284:42] numbers and then it takes from me x [284:44] comma y. Then it returns return x / y. [284:49] Then let's create an interface interface [284:52] equals gradio. So grin interface pass in [284:57] fn function equals this function div [284:59] underscore numbers or div number we can [285:02] call it div numbers then we can pass in [285:04] here div numbers comma inputs equals a [285:07] list so it takes from me multiple inputs [285:10] x and y equals grado dot number and then [285:13] we have to put here parentheses or open [285:15] and close parenthesis and also we have [285:17] here grado dot number [285:20] then we can get some outputs by going [285:22] grad number and then at the last don't [285:26] forget to launch it. dot launch launch [285:29] start then run that again. You can see [285:31] we will have another gradio interface [285:36] and it allows us to divide different [285:38] numbers. For example, if you have here x [285:41] as 15ide by 5, 15 / 5 will be 3. 3 * 5 [285:48] it's 15. Okay. So this is how to [285:51] calculate or create a mini calculator [285:54] using radial. Now let's go to the text. [285:56] So what if a what if you want to [285:59] manipulate text or to deal with text? [286:01] How gradio offers us the text [286:04] capabilities and how can we make the [286:06] most out of it? We can go def and then [286:09] name this function. I will name it as [286:11] reverse text. So I will provide the [286:14] gradio interface with some text and I [286:17] need this interface to reverse my text. [286:20] So provide here text then it returns [286:25] input text we can call it input text [286:29] input text then we can reverse it as [286:32] colon colon minus one. So the most [286:35] simple way or the simplest way to [286:37] reverse any text in Python is just [286:40] passing here square brackets then colon [286:43] colon. So start from the start and end [286:46] at the end but just pass here minus one [286:50] to reverse the order. So if you have [286:51] here some text for example python this [286:54] is a normal text in python we can pass [286:57] into here or index on it and then pass [287:00] give me all python. You can see here we [287:02] have all Python. It's the same. Start [287:03] from the start until the end. And then [287:05] provide here another colon. Then pass [287:08] into here minus one. So start from the [287:10] start and you can stop at the end but [287:14] start from the end. Okay. You have minus [287:17] one. You can see Python in reverse [287:20] order. You can see py. [287:23] Okay. In a reverse order. You can also [287:25] pass here language. [287:28] You can see language in a reverse order. [287:30] Okay, so this function just receives [287:33] some text and returns the reversed [287:36] version of it. So returns returns the [287:39] reversed [287:41] version of my text. It's a really simple [287:46] function. Now we can go reverse function [287:49] or reverse text and then pass in some [287:52] text. For example, I like Python. Then [287:55] it returns I like Python in a reverse [287:58] version. Now to create a grad interface [288:00] we can do that easily by going interface [288:04] equals gradio dot interface. So create [288:08] an interface then pass in here a [288:10] function. So you need to provide the [288:12] interface with a function to perform it. [288:15] So you have some inputs and you need to [288:17] create some outputs. Then what is the [288:19] link? What is the linking process [288:21] between my inputs and outputs? This [288:23] function represents the linking and the [288:26] connection between my x and y equals [288:29] reverse equals text, [288:32] inputs equals a list of so here we can [288:35] pass here gradio.ext. So this time we [288:38] don't need to pass multiple inputs but [288:39] rather just pass one text. Okay. So [288:42] gradio text and you can see we can pass [288:45] in some text later and it will be [288:47] reversed. Comma outputs equals gradio [288:50] dot text. So we are setting the inputs [288:54] as a text and setting the outputs as a [288:56] text. Then we can go interface dot [289:00] launch. We can launch this app [289:06] and and remember we have a public URL [289:08] here that will be expired after one [289:11] week. Here we have some input text. You [289:13] can see now we have input text. We can [289:15] write I really love the programming [289:20] language Python. Then we can submit it [289:23] and it will be here reversed. I really [289:26] love the programming language Python. [289:30] Okay. If you provide us some word for [289:32] example the word machine, it will be [289:36] reversed. [289:38] You can see machine. Okay. So this is [289:40] how you can create a grad interface to [289:42] deal with text inputs. Now let's dive [289:45] into grad sliders. How do you deal with [289:48] sliders and the the slider input or [289:53] component? First we have to go def and [289:55] then define a function called slider [289:58] example and then pass in a value. Then [290:02] return f. So let's return some string [290:05] formatted string as slider slider [290:10] current value is. So let's just keep it [290:13] keep it simple and just return a string [290:16] that is saying slide the current value [290:19] is as this value. So this function will [290:23] be passed to interface and then here we [290:25] have some value. We inject this value [290:28] inside this string and then we we are [290:30] printing slider current value is value. [290:33] Now let's go ahead and use that [290:35] function. We can go slider example and [290:38] then pass in for example 20. Now it will [290:40] return the text slider current value is [290:42] 20. But if but if we want to pass it to [290:47] interface gradio we can go gradio or [290:49] interface equals gradio.inface [290:54] and then pass here fn. So fn is short [290:57] for function. Provide this interface [290:59] with a function as slider example. And [291:02] note in here you don't need to pass here [291:06] two parenthesis. Okay. Because why? You [291:09] only need to execute this function when [291:12] the gradu interface is being displayed. [291:14] So you don't need or you don't want to [291:17] execute it when the interpreter reads [291:19] that function. Then you have your inputs [291:21] equals [291:23] grad slider then pass minimum equals [291:28] zero. So what is the minimum value you [291:30] want to set as maximum equals 100. So [291:33] that's up to you. whatever the minimum [291:35] value you want to set and the maximum [291:37] whatever maximum value you want to set. [291:39] Then we have here step equals for [291:41] example one. Let's set it to one. Then [291:44] we have here outputs equals gradial.ext. [291:49] Let's go ahead interface launch dot [291:53] launch. So launch the app and we can run [291:56] that. Feel free to pass here share [291:58] equals true to not display that [292:00] notification anymore. [292:04] So you can see we don't have that [292:05] notification and here we have a small [292:08] info to show errors set debug to true [292:10] but now we don't have errors you can see [292:12] we have now a slider you can choose [292:14] whatever value you want then for example [292:17] 46 if you submit it will return this [292:19] text slider current value is 46 if you [292:23] set it to 100 it will be slider current [292:26] value is 100 and you can see now the [292:27] step size is one so when you decrease it [292:30] will be decreased by one 99 98 98 94. [292:34] You can see 93. But if you want to [292:37] change the slider step, you can do that [292:40] by setting by resetting the step [292:43] parameter for example into two. Now you [292:45] set it to two, it will jump by two [292:49] steps. It will increase or decrease by [292:52] two steps. So you can see here [292:54] increasing by two steps. You can see two [292:57] 4 6 8. Now the scale is increasing by [293:01] two. And also when you decrease it will [293:03] be decreased by two. Okay. So now the [293:06] range here is by twos. Okay. [293:10] So that's how to deal with sliders. Now [293:13] let's move on to gradio [293:15] drop-down dropdown [293:18] menu. So how to create a drop-own menu [293:21] in gradial? First we need to define a [293:24] function as always def drop-down [293:27] underscore [293:28] example and then pass in some value [293:33] then return f string current value or [293:38] current value is as value and then let's [293:43] go ahead and call it dropown example [293:46] pass in some value for example the value [293:48] python so I choose python language You [293:52] can see current value is Python. Now [293:55] let's wrap it inside interface gradial [293:57] the grad interface by going i iface [294:00] equals for example gradio.inface [294:04] dot interface. So creating an interface [294:07] a GUI then pass in a function equals [294:09] drop-down [294:11] example example [294:14] inputs. What other inputs you want to [294:17] input? For example here gradio dot [294:19] dropdown. So the class here you can see [294:22] it's called drop-down. These are [294:24] different components are all available [294:26] for us in the gradial library. So the [294:29] grad library has so many components. It [294:31] has buttons, drop-downs, sliders, text [294:34] boxes, number boxes, number inputs and [294:38] so many components. Here we can pass [294:41] choices equals a list cause called [294:44] options. So this options list we have to [294:47] create by going options equals a list [294:50] that we can list here a b c. So here we [294:55] have some options for example a the [294:57] option a option b option c. So for [294:59] example, if you want to customize a [295:01] habarameter, if you are hosting some [295:04] machineing model, then you then you have [295:05] some different values for some [295:07] habarameter. For example, learning rate, [295:09] you set some different values. Then you [295:11] let the user choose whatever learning [295:13] rate they want to use. This is one idea [295:16] on how to use and leverage dropdowns in [295:19] gradio. Then we can go comma outputs [295:22] outputs equals gradio.ext. [295:26] So it returns a text. Why? Remember in [295:29] this function we we are returning what [295:30] text. So you have to keep in mind what [295:32] are my input shape and what are my [295:35] output shape and also how do you present [295:38] your output and how do you present your [295:40] input and let's go ahead and then call [295:44] iface equals launch. [295:47] So you can see this time I called it [295:48] IPAC here because since we have just one [295:52] one input we don't need to pass it into [295:55] a list although it doesn't produce any [295:58] error. So let's run that. You can see [296:00] options. Yeah here I called it options [296:03] and I didn't run that. You have to run [296:05] this list then run this list. [296:08] So you can see this is my radio [296:12] interface is being loaded. Here we have [296:13] some dropown menu. You have a b you can [296:16] switch to c and then for example choose [296:19] b submit it you can see current value is [296:22] b so you can see how easy it is to [296:24] create dropown menu you don't have to [296:26] worry about the html thes javascript to [296:30] create this front-end web development as [296:33] they say in the in the tool gradio [296:36] here and then go to the website [296:40] build machine learning apps in python [296:42] create web interface faces web you can [296:45] see web is using HTML so behind the [296:47] scenes there there are some HTML files [296:50] CSS files JS files are being run and [296:53] loaded behind the scenes and you don't [296:56] know what's happening behind the scenes [296:57] so create web interfaces for your ML [297:00] models in minutes deploy anywhere share [297:03] with anyone and you can see here down [297:05] they have everything you need to build [297:07] so they have uh so many components you [297:10] can see here import grado SGR Then we [297:14] have greet to greet someone. Name hello [297:17] plus name. Then you pass the function as [297:19] we did fn equals greet. This is a simple [297:21] function like us. Then inputs equals [297:23] text. We have a text input. We can write [297:26] here my name is Muhammad. Then submit. [297:28] You can see the output here. Hello [297:31] Muhammad. And then the outputs here as [297:33] text. So feel free to read here in your [297:35] own time. You can see so many [297:37] documentations as I have here IPI. But [297:40] let's now go to our notebook and [297:42] complete the codes. [297:45] Now if you wanted to change for example [297:47] here the options you can do that easily [297:49] by going A change for example into [297:52] Python Python B change it into P ph PHP [297:57] and for example C change into SQL and [298:01] you can run that again. You can see now [298:03] our gradio interface is being loaded and [298:06] it will show us the names of the Python [298:09] or PHP. If you submit you can see [298:12] current value is PHP SQL you can see [298:14] here current value is SQL structured [298:17] query language. Now the the next step is [298:21] if you want to create gradio image how [298:23] do you deal how do you deal with images? [298:27] Well you can do that in easy steps. [298:29] First from pill import image. We have to [298:32] import the image class. Then def convert [298:35] and then pass in image path. So this [298:40] function will convert my image from a [298:42] normal image from a regular image into a [298:44] grayscale image which is amazing. You [298:46] can see load the image and then we have [298:50] image equals the image class dot open [298:53] and then pass in this function image [298:56] path. So we are receiving this function [298:59] receives image path the path to an [299:01] image. Then we are passing that image [299:03] path into the open function. It will [299:06] open that image and return a pill image [299:08] instance into this variable. Then all we [299:10] need to do is just go ahead and convert [299:13] the image to into a grayscale format by [299:17] going return image.con convert the [299:20] function convert and pass in L capital L [299:22] to convert any image you would like into [299:25] a grayscale image. Now if you have here [299:28] some images I already have some images [299:30] here you can see I have this image of C [299:33] a picture of C. We can pass this image [299:36] copy this path. We can pass the path of [299:39] this image into this function. As you [299:41] can see here it's called convert. So we [299:43] can say convert and then pass in my [299:45] image path. Here they ask for image [299:47] path. We can run that. It returns an [299:50] image converted into a grayscale image. [299:52] You can see now the image is a [299:54] grayscale. So to to incorporate this [299:57] into a gradio interface that's easy. We [299:59] can go iface equals gradio.inface [300:03] then pass in the function the function [300:05] that you want to execute the function [300:07] that you want to run behind the scenes. [300:10] Here we have convert, [300:12] then we have input equals gradio dot [300:16] image. So we have the image component [300:18] then pass type as a file path. So the [300:21] type of this image will be a file path a [300:24] path to the image file. They have [300:25] outputs equals gradial dot image. It [300:28] receives an image and it returns an [300:31] image but in a grayscale format. [300:34] Sometimes you will receive an image and [300:36] return a label to classify the image. [300:38] But here we are inputting an image and [300:40] also outputting another image. Here you [300:42] can see iface.launch launch then pass [300:45] here S share equals true to not to get [300:47] that to to to not to get that error or [300:50] notification [300:52] share equals true. You can see now it's [300:54] loading a gradial interface. Here we [300:57] have drop image here. We can for example [300:59] take this image and drop it here [301:04] or you can go to Google search for C [301:07] images. Here you have a plethora of C [301:10] images. We can for example take this [301:12] image [301:15] go to gradio there you can see to the [301:18] collab [301:19] drop it here now it's dropped submit [301:24] now we have a grayscale version of the [301:26] image so you can see here we are [301:28] inputting a colored image we have colors [301:31] submitted then we have a grayscale image [301:34] converted successfully so now we have a [301:36] nice app and we can also present it [301:38] using this public URL So running now [301:41] this app is being running on a public [301:43] URL for a duration of one week. After [301:47] one week this app this link will be [301:49] expired. So you can see here here we [301:52] have this app. You can drag any image. [301:54] For example if you search for flower [301:56] images flower images [301:59] this image for example. [302:05] If you go to here drop image here. Now [302:07] this is the image of a flower. submit [302:10] it. You can see now we have a grayscale [302:12] image converted. [302:16] Let's dive into gradio JSON. [302:20] So let's put a title here gradio JSON. [302:23] And then let's define a function called [302:28] number [302:29] details. So number details to retain [302:33] some details about a number. it takes [302:36] from me or it receives a number and then [302:39] you can see it creates a details [302:42] dictionary. This dictionary contains [302:45] several information and details about [302:47] this number. The first information is [302:50] original [302:52] and it contains the original form of the [302:54] number the same number. So we the number [302:57] that we pass in will be stored inside [302:59] the key original original comma we have [303:02] the squared version of the number which [303:05] is number then double asterisk two. So [303:08] we raise this number to the power two [303:12] then we have square root where we take [303:15] the square root of the number by raising [303:18] this number to the 0.0.5 [303:21] half. So if you raise any number to the [303:24] power 0.5 then you are basically getting [303:28] the square root of that number and also [303:30] at the finally is and the score even is [303:34] the number even or is the number odd in [303:36] this case you can discover that by going [303:39] number modulus 2 and then equals equals [303:42] zero. So you can see now we have a [303:44] dictionary that contains several [303:46] information and details about this [303:47] number. For example, here if you have [303:49] the number 10 and if you put double [303:52] asterisk sign then two then basically [303:56] you are rising raising 10 to the power [303:58] two and then we have here 100. So we [304:01] have 10 * 10 then we have 100 okay 10 * [304:07] 10 the same number double twice and here [304:11] we have square root where you raise 10 [304:14] to the power 0.5 then you have 3.16 and [304:18] if you raise here 100 to the power 0.5 [304:22] it will be 10 you are taking the square [304:24] root and also at the end you have 100 [304:27] modulus 2 then you have here zero Does [304:31] this equal equal zero? In that case will [304:33] be true. So this modulus sign in [304:37] addition to double equal sign zero will [304:40] return true or false. Whether this [304:42] number is an even number or odd number. [304:44] For example, 50 is considered an even [304:47] number, it will be true. Whereas 51 is [304:50] an odd number. So you can see here we [304:52] have false. Okay. It's a it's a clever [304:54] trick to retain to return and discover [304:58] whether this number is even or not. Then [305:01] at the end return details. [305:03] As you can see now we have a nice [305:05] function an amazing function that [305:08] returns some information and details [305:10] about whatever number. So you can call [305:12] that number underscore details and then [305:14] pass in for example the number 400. [305:17] There you can see it returns here some [305:19] information original 400 squared 16,000 [305:24] square<unk> 20 is even true if you raise [305:28] here for example 20 you can see it [305:30] changes dynamically based on what the [305:32] number you provide to the function. Now [305:35] let's go ahead and wrap this function [305:36] inside a gradio interface by going [305:39] interface equals gradio dot interface [305:43] dot interface and then pass in here fn [305:46] equals number details. This is uh this [305:50] function and then comma inputs equals [305:53] gradio number. So remember this function [305:55] receives from me a number a single [305:58] number then it outputs so the outputs [306:01] equals gradio.json JSON. [306:04] Then at the end, let's go [306:06] interface.launch. Let's launch this [306:08] interface. [306:13] Now we have this fascinating gradio [306:16] layout. You can see it's loading. Here [306:18] we have number. If you for example input [306:20] the number 70 submit, then it will [306:23] return a JSON format data. You can see [306:26] original is 70 squared 4,900 square<unk> [306:31] 8.3 is even true. So you are retaining [306:34] some information some really helpful and [306:36] rich information about the number you [306:38] are providing in. If you change it for [306:40] example to 99 you can see submit [306:43] it changes based on the number that you [306:45] are providing. If you input here 100 to [306:47] see whole numbers, you can see here [306:50] squared is 10,000 square root is 10 is [306:54] even true and original is 100. Okay, so [306:59] that's it dealing with grad JSON. Now [307:02] let's go to the next step where we will [307:04] deal with another [307:08] label. So the gradio label [307:13] labels in gradio. So the label is the [307:16] name of the class. So imagine you are [307:19] creating a classification model and you [307:21] want to classify into different [307:23] categories. You will return then a label [307:26] that labels whether this data point [307:29] belongs to one class or another class. [307:31] For example, you are classifying several [307:34] different types of flowers or this [307:36] flower is a red flower or a yellow [307:38] flower. So let's define a function [307:40] called classify underscore [307:43] number. It receives from me a number [307:46] then it's it checks if condition if [307:51] number is larger than zero then return [307:55] positive. So that's it. If this number [307:58] is larger than zero if this number [308:01] greater than zero then return positive. [308:04] L if so else if L if this number is less [308:08] than zero. So this number is smaller [308:10] than zero then return negative. [308:15] Else [308:17] this condition is false. This condition [308:19] is false then basically return zero. So [308:22] if none of these conditions are met then [308:24] at the end we do an else statement and [308:27] then return zero. But if this condition [308:30] has met has been met so if number is [308:33] greater than zero return positive. L if [308:36] number less than zero then return [308:38] negative. So here we are basically [308:40] classifying numbers into positive [308:42] negative zero. Now let's go ahead and [308:45] use this function. You can see here we [308:47] have [308:50] indentation error. So [308:53] expected colon. [308:59] So you can see here we have yeah here we [309:01] need to put semic colon. Okay you can [309:04] see now we have a classify function [309:07] ready to go. Let's go classify [309:09] underscore number and then pass into [309:11] here for example 50. 50 is positive then [309:14] it classifies it as positive. If we pass [309:17] here -1 then it classifies it as [309:20] negative. If you pass here -50 it will [309:24] be as negative but if you pass here zero [309:26] it will be classified as zero and so on. [309:29] So here we have a nice function that [309:31] classifies into positive num negative [309:34] zero. It classifies and categorizes [309:37] different numbers into these different [309:39] categories. Let's wrap it inside the [309:41] gradio interface [309:43] by going interface equals gradio dot [309:46] interface and then pass in the function [309:49] fn equals classify underscore number. So [309:52] we pass in the name of the function. We [309:55] are taking the name function and passing [309:57] it to the fn parameter. Let's insert an [309:59] inputs here. Inputs equals a list or a [310:02] graded number. Let's pass in just one [310:05] number a grad number component, outputs [310:09] equals gradio dot label. So the output [310:13] here is a label. You can see now we are [310:15] using the component label to label the [310:18] class. Then let's go ahead and go [310:20] interface.launch to launch our app. Run [310:24] that. And you can see now we have this [310:27] interesting layout is being loaded. You [310:30] can see loading. Now we have a number. [310:32] If you insert 80, it's a positive number [310:34] and it will be classified as positive. [310:36] You can see now we have a label. This is [310:38] the label component. It's different from [310:40] the text component. You can see here we [310:43] have the text component at the top [310:46] where you type something and it produces [310:49] you can see here [310:54] you insert some text Python that it [310:56] produces some text. Okay, it's still [310:59] within the text input. Here we have a [311:02] label. A label is different. Here you [311:04] don't have an input but a label but [311:06] rather a label. So you can put here [311:08] whatever number for example minus five [311:10] then submit it. Then you have here [311:12] negative. And if you pass here zero it [311:15] will be classified as zero. And feel [311:18] free to try whatever number you would [311:19] like. Now let's go ahead and dive into [311:22] another component of gradial which is [311:25] grad layout. So gradial layout. So now [311:28] basically we have finished. So [311:30] congratulations we have finished all the [311:32] components the most important and common [311:35] components that you will use inside [311:38] gradial. If you go ahead and search here [311:40] gradio or gradio components components [311:44] documentation [311:47] then here you can see components here we [311:49] have a great list a massive list of all [311:52] the components that you can use. So it's [311:54] a huge list. You can see we have here [311:57] introduction, audio, you can set some [311:59] audio, bar plot, button, chatbot, [312:03] checkbox. You can see have clear button. [312:05] We have here the data set date time and [312:07] also you can see image that we have used [312:09] earlier. So remember image grid image [312:12] where you pass in an image and then it [312:14] will opens that image. And remember we [312:16] have specified the value here as or the [312:19] type here as file path. Okay. Then you [312:21] can also search here for whatever [312:24] component you would like to use. Also [312:26] here we have JSON that we used earlier. [312:27] You can see JSON. Also here we have the [312:30] markdown dropown menu. And feel free to [312:33] explore that in your own time. But now [312:35] let's go ahead and dive into the gradual [312:38] layout code. So now basically what we [312:41] are what we we were doing is just [312:44] creating components and throwing them [312:47] straight away into this interface. But [312:50] how do you lay out? How do you organize [312:53] these items? How do you organize them [312:55] and control the layout and the way of [312:58] displaying these items? So rather than [313:00] just displaying them in random order, [313:03] why don't we just organize the way of [313:06] displaying them, making them organized, [313:09] neat, and a lot more uh beautiful way. [313:13] So render it in a beautiful way by going [313:15] with gradial do.blocks. So now you can [313:18] see this class interface [313:22] controls the rendering process. So [313:25] interface is an automatic class that [313:28] automatically sets the organization way [313:32] of the items. So the interface is an [313:35] automatic way. But if you want to use a [313:37] manually created layout way, if you want [313:40] to create layouts manually, you define [313:42] you the you are the person who defines [313:44] how to display things, then you would [313:47] need to go ahead and with so use a with [313:51] context in Python grad. [313:54] Then open and close two parenthesis as [313:56] demo. Then we have here with gradio dot [314:00] row. So create a row in this layout. So [314:04] you can see we have a class row. So now [314:06] we are manually defining the layout. How [314:09] do you want to display these items [314:10] inside this row here? Here goes whatever [314:15] you want to put inside the first row. [314:21] For example, the first thing is text one [314:24] equals gradio.ext and then pass into [314:27] here the value [314:29] equals output one or output one. Okay. [314:33] Then we have text two. We need to insert [314:35] another text item equals gradio.ext [314:38] another text component and pass value [314:41] equals output two [314:45] with gradio row. So please insert [314:48] another row. Please add another row [314:50] here. A second row. And inside that row [314:53] go ahead and insert whatever item you [314:55] would like. For example, text 3 equals [314:58] gradio.ext. and then pass in value [315:01] equals our bottom row. So we have bottom [315:06] row and then let's go ahead and do demo [315:10] launch. So demo.launch. [315:14] So here these values are just really [315:17] strings. You can define whatever string [315:19] you you would need to add. Now let's [315:21] launch our app. Make sure you pass in SH [315:24] equals true if you don't want to get [315:26] that notification anymore. You can see [315:28] here we have this gradial layout [315:32] and beautiful. Now we have two rows. You [315:36] can see now you are controlling the way [315:38] of displaying the item. See we have two [315:41] rows. The first row contains two text [315:44] boxes. You can see here we have output [315:46] one output two. Then the second row [315:48] contains a text box with bottom row. So [315:52] here again we are instantiating a new [315:55] instance of blocks by going with with [315:58] grid.blocks [316:00] as demo. Then inside this blocks define [316:03] two rows row and row. Inside this row [316:07] insert or inject two text boxes and you [316:09] can see them here. Here we have two text [316:12] boxes. They are siblings. Inside this [316:15] first row we have injected two input [316:19] boxes. Then we have inside the next row [316:21] we have one text element that is say [316:24] that says bottom row. As you can see now [316:27] we have this beautiful layout being [316:29] rendered in our customized in our [316:32] customized way. Now let's go and move on [316:35] to the next step. Import gradio asg [316:39] with gradio.blocks [316:43] as gra as demo. So you don't need to [316:46] import gradio again. But if you want to [316:49] run this cell as the starting cell [316:51] again, you need to import it again. So [316:54] if you if you don't want to run all the [316:57] cells later when you come back later, [316:58] for example, after 1 hour and the [317:00] runtime has disconnected the when you [317:03] import from here, you don't need to run [317:05] all the cells from above. You just start [317:07] from here. Okay. With gr.blocks [317:10] as demo, let's go ahead with gradio row. [317:14] So insert a new row. Then inside this [317:17] row, insert text one equals gradio.ext. [317:21] And then pass in value equals output [317:24] one. And then we have here text two [317:27] equals gradio.ext. So insert a text [317:30] component. And then pass here a value [317:32] equals output 2. Then we have here with [317:36] gradio. So with gradio. So with is a [317:39] context manager in Python. With grad [317:42] row, it manages the context. Then text [317:45] three equals gradio.ext and then pass in [317:48] the value as bottom row. Let's go ahead [317:51] and demo.launch [317:54] to launch that app. And here we you can [317:56] pass share equals true or without it. [317:59] You can see now we have [318:02] this gradio layout is being displayed [318:04] and rendered for us. You can see we have [318:06] two rows. The first row contains two [318:09] text boxes. The second row contains one [318:12] text box. And you can see this because [318:14] the first row contains two different two [318:17] distinct text boxes. It shares the [318:20] spacing between them. So you can see [318:23] these are equally spaced. Whereas the [318:26] second row here contains one text box [318:29] and it's fully it it spans across the [318:32] full width. It reaches the full width of [318:35] this row. Okay? It takes up to the full [318:38] width. [318:40] It fills the full row. Now let's go [318:42] ahead and add another code cell and also [318:47] explore some more different option [318:48] options with gradio.blocks to customize [318:51] our layout. So here we have to go with [318:55] gr.blocks as demo. [318:57] Let's go ahead and add with gradio row. [319:01] So add a new row. Then inside this row [319:04] note here we have a new column [319:06] withgrad.com. So here we are adding a [319:09] column. Now sometimes you don't need [319:12] necess it doesn't it doesn't necessate [319:15] so in some cases it doesn't necessate to [319:18] add a new column but in this case you [319:20] will need to add a column then inside [319:22] this column text one equals grtext and [319:27] then pass into here value equals row 0 [319:31] column zero. So this is a row zero call [319:34] zero dash component one and as I told [319:38] you in some cases it doesn't necessitate [319:41] to add columns in some cases it will be [319:44] necessary in case you need to add [319:46] columns note we are adding the columns [319:49] inside the rows. So first you start by [319:51] adding a row inside the row you add [319:54] columns and you cannot swap that order. [319:56] Okay. Then we have here text tool equals [319:59] gradio.ext and then insert some value [320:01] here as row zero column zero dash [320:06] component one or component two here. [320:08] Okay. So here we have component one the [320:11] row number zero the column number zero [320:14] but component one and then component [320:16] two. After this column let's add another [320:19] column with gradio column. So note the C [320:21] here is capital. It's a class. It's [320:24] called column. Inside this column, go [320:27] ahead and insert a text three component [320:29] equals gr.ext and then pass in a value [320:32] parameter and then specify row zero [320:36] column one. Then after that let's add so [320:40] you have to to de-indent here two times [320:43] to get rid of this row context manager [320:46] and then add another row. So with [320:48] gradial row outside the scope of this [320:52] row. So here we have this row scope. [320:55] Inside the scope we have two columns. [320:57] Inside the first column scope we have [320:59] two text components. So you have to be [321:02] you have to be careful when you space [321:04] into the columns the rows. Inside this [321:07] column scope we have two text [321:08] components. Inside the second column [321:11] scope we have one text component. Then [321:14] let's create another row and insert here [321:17] text for. So without inserting any [321:19] column gr.ext text and then pass here [321:23] value equals bottom row. And you can see [321:25] here the flexibility here. Not when you [321:28] when you add one column inside one row, [321:31] you don't need to add columns inside the [321:33] next row. So it's really it's really [321:36] flexible here. The way you organize it [321:38] is up to you. You are the person who is [321:41] responsible for displaying how is your [321:44] gradio layout is being rendered and [321:46] displayed to others. You define the [321:48] columns. Inside these columns we put [321:50] components. These columns are inserted [321:52] inside rows and not every single row [321:56] needs to contain columns. Maybe there [321:57] are some rows that doesn't contain any [321:59] column. Now let's go ahead. So you can [322:02] see the flexibility and how easy it is [322:05] to construct layouts. [322:09] So you can see now we have layout and [322:11] note now we have columns. So now we have [322:14] one row inside this row we have one [322:17] column. This column contains two [322:20] components. So component one and the [322:22] component two. Here we have another row. [322:24] You can see bottom row. So we can see [322:27] here we have one row inside the two [322:30] columns. Column column. The first column [322:33] contains two different text boxes. The [322:35] first column contains two different text [322:38] boxes. The second column inside the [322:40] first row contains one text component. [322:44] And then the second row contains one [322:46] text component. And it takes up until it [322:49] fills the full width. Until it fulfills [322:51] the full width. Okay. Okay. Now let's [322:54] move on to the next step [322:57] with gradio.blocks [323:02] as demo. [323:04] Let's go ahead and insert here a row. So [323:07] with gr row insert a row here. And then [323:11] we have with gradioc column. And note [323:14] here the the two columns you can notice [323:17] that the two columns here are taking the [323:20] same amount of spacing. They are taking [323:23] the same amount of width. You can see [323:25] they are equally spaced. Whereas if you [323:27] want to to to differentiate or to assign [323:31] different width scales, you can do that [323:34] easily by passing here a scale [323:36] parameter. So you can go scale equals [323:38] two. So here we are scaling this column [323:41] to take up 2/3 of the width and then [323:46] text one equals gradio.ext then pass in [323:50] value equals row zero column zero dash [323:55] component one. They have they need to [323:58] insert another text component by going [324:01] gr.ext pass value equals row zero column [324:07] zero dash component 2 [324:11] and this column is taking scale [324:14] parameter equals 2. Let's now insert and [324:17] add another column by going with [324:20] gradient column. [324:23] Let's go ahead and insert the scale [324:27] parameter equals to one. So this column [324:32] will take just one/3. You can note here [324:35] 1/3. This column will take here 2/3 of [324:38] the total width. Okay. Then inside this [324:42] column context manager let's insert text [324:44] three equals gr.ext [324:46] and pass in value equals row zero column [324:50] 1. [324:52] Then outside of this row scope, let's [324:55] add another row by going with gr row [325:00] and then pass in here text for or add [325:03] text for equals gradio.ext [325:06] value equals bottom row. And then let's [325:11] launch this app and see how this differs [325:13] from the previous app. And remember to [325:17] pass share equals true. So you need to [325:19] make sure that you pass in share equals [325:21] true. You can see here now this column [325:26] takes up to 2/3 of the total spacing. If [325:30] you go up scroll up here you can see [325:32] here the the the difference between that [325:35] layout and this layout. This layout the [325:37] spacing is shared is the spacing is [325:41] distributed equally between these two [325:44] columns. The spacing is distributed [325:47] equally between them. But here the [325:49] spacing [325:51] is 2/3 of the space is inserted to the [325:54] first column. One/3 of the space or of [325:57] the width is inserted to the next [325:59] column. And you can see how the [326:01] different between that all that just [326:03] with one parameter. Scale equals 2, [326:06] scale equals 1. You can see how easy it [326:08] is to manipulate and to play around with [326:11] the layouts. It's actually quite fun to [326:14] play around with layouts in gradio. [326:19] Now let's go ahead and [326:21] do some more complex layouts by going [326:24] with gr.blocks [326:26] as demo inside this blocks. You can see [326:30] first we instantiate a blocks object and [326:34] insert an alias name via as demo. So [326:37] adding an an alias name then with gr row [326:41] or tab. Now let's add a tab. So here we [326:44] have the tabs tab and then we have tab [326:46] one. [326:48] So this is the first tab. Actually you [326:50] can insert tabs in gradio. Then we have [326:52] here with gradial dot row [326:56] with gradio dot column. So inside this [326:59] row let's add a column. Inside this [327:01] column let's insert text one equals grad [327:04] text. Then pass in some value equals row [327:07] zero column zero dash component one. We [327:12] have text two equals gradio text pass in [327:15] value equals row zero column zero- [327:19] component 2. Let's go ahead and insert [327:22] another column with grt column [327:25] and then add another column inside this [327:27] column. Let's insert text three equals [327:30] gradual text and then insert a value of [327:34] row zero [327:36] column one and after this column let's [327:40] add so outside the scope of this row [327:44] context manager let's add another row we [327:46] can see and then inside this row insert [327:49] text for equals gradio text and then [327:52] pass in value equals bottom row. [327:58] At the last, let's add another tab by. [328:01] So you can see here outside the scope of [328:03] the tab class here, the tab class of the [328:05] tab object. Let's add another with [328:07] context manager with graduate.tab and [328:10] then pass in here tab two. So you will [328:13] see how nice it is these tabs. These [328:15] tabs are really nice. They organize the [328:18] display of the items text. But then with [328:21] gradio dot row let's add a row inside [328:24] this tab gradio text and actually you [328:27] can add here columns but let's keep it [328:29] simple text and then pass here welcome [328:32] to the new tab. [328:36] Let's go ahead and run the demo by [328:38] demo.launch [328:40] dot launch every single grade you run [328:44] you have to launch the demo. Okay. And [328:46] then pass a share equals true. Let's run [328:49] that. You can see now it's running. [328:53] Now it will show us the layout. You can [328:55] see the the layout has shown up here. We [328:57] have tab one. You can see we have one [329:00] row inside this row. It contains two [329:02] columns. The first column, the second [329:04] column. The first column contains two [329:07] text components. And you can see that [329:09] the spacing is distributed equally [329:12] between these two columns. We have the [329:14] second row. In the tab two here we have [329:16] just one row that contains this text box [329:18] welcome to the new tab. So these tabs [329:21] are really nice to organize and to make [329:24] the the display neat. [329:27] Here we can also insert column equals [329:30] scale equals scale equals 2 and here we [329:35] can insert a parameter scale [329:39] equals 1. Okay. Now if you go to tab one [329:42] you can see here the the spacing is [329:45] distributed equally. When you run that [329:47] again you can see this column will take [329:50] 2/3 of the space 2/3. And if you stop [329:55] the order you can do that easily by [329:57] going two and here one. [330:00] So feel free to customize your your [330:02] layouts whichever way you want. Okay. [330:06] You can see this column takes 2/3 and [330:08] this column takes just 1/3. even though [330:10] it contains two components. [330:17] So we can also use [330:28] so here the name of this is called as [330:32] let's name it as gradial [330:35] demo [330:37] tabs. Okay, these are the the tabs that [330:40] we add to to a gradio. Okay, [330:45] now let's move on to the next step which [330:47] is the idea of [330:51] using a gradual blocks and then adding [330:53] an accordion. So, how do you add an [330:55] accordion? Yeah, gradio accordion. [330:58] How do you add an accordion into a [331:01] gradio and insert a new component inside [331:04] an accordion? So, let's go with [331:06] gradio.blocks. blocks as demo. Let's go [331:10] ahead add gradio label and then insert [331:13] here label here. So you can add a label [331:16] here. After this label, let's go with [331:18] gradio accordion. So you can see the [331:21] accordion object or the accordion [331:23] component uses this with with keyword. [331:26] So you have to add with keyword in [331:28] Python grad accordion and then pass into [331:32] here accordion [331:35] here, [331:37] open equals false. So if you want to [331:40] this accordion this accordion to be [331:42] opened by default, you have to pass in [331:45] here true. And if you want this if you [331:47] want this accordion to be closed by [331:49] default, pass in here false. [331:52] And that's up to you. And then we have [331:54] here gr image. So inside this accordion [331:57] we want to open an image object or image [332:00] component. Let's go demo.launch [332:06] you can see here [332:09] it's trying to load in the layout and it [332:11] will have a an accordion object. You can [332:14] see accordion here we can open it close [332:16] it and if you open this accordion we [332:18] have this drop image here. We can open [332:21] our images. Choose Elmo. You can see [332:23] here we have Elmo image. Okay, it's [332:27] trying to load that. So inside this [332:29] accordion, if you close it, it will be [332:31] closed. Open it, it will be opened. And [332:34] remember, we have a nice little [332:37] parameter called open. If you pass here, [332:40] it will be open by default. So this this [332:42] accordion item is one of my favorite [332:45] items in gradient library. In grad [332:47] library, [332:49] you can see now it will be open by [332:51] default. [332:54] Okay, choose image flower. [332:58] It's loaded successfully and you can [333:01] close it if you want. [333:03] Okay, and that is accordion. Accordion [333:06] is short is uh one of the most important [333:10] items inside gridio and feel free to go [333:12] to documentation to refer back and [333:14] reference some more opportunities and [333:17] some more option ways to use that. So [333:20] you can see here methods they have [333:21] expand if you want to expand it [333:23] description. This listener is triggered [333:26] when the accordion is expanded. So you [333:28] have also a listener that could execute [333:30] some code when the accordion item is [333:33] triggered when it is motivated and [333:35] opened. You can trigger some code to be [333:37] happened and to be executed. We have [333:39] collapse. So the collapse event. We have [333:42] description. This listener is triggered [333:44] when the accordion is collapsed. And [333:46] feel free to play around with that in [333:48] your own time. It's actually quite fun [333:50] to play around with with gradio. So you [333:53] can see how easy the syntax is of [333:55] gradio. Let's go to gradio buttons. So [333:59] how do you add buttons using gradio? The [334:02] first thing let's go with [334:05] gradio.blocks [334:07] as demo. [334:09] Let's go ahead and insert here some [334:12] buttons by going grutton and then pass [334:14] here button [334:17] component one. Then we have here [334:19] grbutton and then pass here button [334:22] component two. And also we have here [334:26] gradio image. Then after that we have [334:28] gradio [334:30] demo.launch. [334:31] Okay. So we can launch our app. Now pass [334:34] in here share equals true to be able to [334:37] share it on a public URL. Then you can [334:40] see now it's loading our layout and now [334:43] it contains some buttons. [334:49] So we have button component one button [334:51] component two drop image here or click [334:54] to upload. You can click here and upload [334:56] whatever image you would like. You can [334:58] see now we have an image. [335:00] Currently these buttons when you click [335:02] on them they will not do anything. In [335:05] later section we will learn how to make [335:08] these buttons work. Okay. Now let's move [335:11] on to next step. Next how to insert some [335:15] CSS into gradio. That's easy. We can go [335:18] ahead and gradio CSS. Put a nice little [335:21] name here. Then let's go ahead and [335:24] create some CSS. So we can we can go CSS [335:27] variable equals and then insert three [335:30] double quotation marks. So we need to [335:33] add one 2 3 then also close by three [335:37] double quotation marks 1 2 3 inside [335:40] these double quotation marks let's add [335:42] dot your class for example here you can [335:46] name your class whatever name you want. [335:48] Then let's go ahead and open and close [335:50] parenthesis. Inside this parenthesis, [335:53] add whatever custom CSS you would like. [335:56] CSS is short for cascading stylesheets. [335:59] Let's go ahead and insert some height [336:01] property. So the height attribute as [336:04] 1,000 pixels, [336:07] background dash color, and then set it [336:10] as for example red. [336:12] You can customize whatever codes you [336:14] would like. Then let's go with [336:16] gradio.blocks blocks and then add here [336:18] CSS equals CSS. So insert the CSS [336:21] variable inside the parameter or as a [336:24] value to the parameter CSS as demo. And [336:28] feel free to name this variable whatever [336:29] you want. I will name it in this case [336:31] for example my own demo. Then let's go [336:34] ahead and with gradio row and then [336:38] element_classes. [336:40] So this is a parameter called lm classes [336:44] and then pass in square brackets pass in [336:47] the name of the class in this case your [336:50] class and then let's go ahead and add [336:53] gradio image as a component inside this [336:56] row by going ahead and setting the [336:59] height as 900 width equals 20. Now let's [337:04] go and demo or my own demo in this case [337:08] my own demo [337:10] dot launch and then pass here share [337:13] equals true. [337:16] Now 3 2 1 run that. You can see now we [337:19] have a custom CSS code that will run. [337:32] Now this is the layout. It's being [337:33] loaded. You can see now the height is [337:36] longer. So you might not notice that but [337:38] you can see the height here. Before it [337:41] wasn't that height, but here now we have [337:43] a much longer height. You can see the [337:45] scroll. Now you can go down further. And [337:48] then we have a background as red. You [337:50] can customize whatever CSS you would [337:52] like. For example, changing the [337:53] background rate as yellow. It will [337:56] change it to yellow. [338:00] It's all up to you. the custom settings, [338:02] the custom modifications that you want [338:05] to that you want to apply to your custom [338:08] layouts. Here we have def multiply. Now, [338:11] what if you want to if you want to [338:14] actually make these buttons work? How do [338:17] you trigger some code when you click on [338:19] a button? How do you execute some code [338:22] when you click on a button? That's [338:24] really simple. By going ahead and [338:26] defining a function, for example, [338:28] multiply. So def is short for define a [338:30] function called multiply pass in x comma [338:33] y then just return it returns x * y. [338:40] This is one of the most simplest [338:41] functions that we have created. If you [338:43] write multiply and then pass in 5, 5, it [338:47] will just multiply 5 by 5 25. If you [338:50] pass here for example 10, it will be [338:52] here 50. [338:54] Let's wrap it inside gradio [338:57] interface by going with gr.blocks [339:02] as demo. So instantiate a new instance [339:05] of the blocks class as and store it [339:08] inside the the alias name demo. Go ahead [339:12] and create a row to organize our display [339:16] and then create two sliders. Now we have [339:19] two events. The first event is called [339:22] here change event. So you want to [339:26] trigger some code in a change event or [339:29] during a change event. [339:32] So basically we have two different types [339:34] of events. If you go here and right here [339:37] there are two different types [339:42] of events. We have one click events. [339:46] Click events. We have two change events. [339:50] Let's start with change events. So the [339:52] first thing is change event. How do you [339:55] trigger code when you for example change [339:57] a slider value? X slider equals gr [340:01] slider and then pass nothing. We have [340:03] here y slider equals grio slider. So [340:08] here we have two different sliders. How [340:10] do you trigger some code whenever you [340:13] change the so change event whenever you [340:15] change the value of the slider. And then [340:18] we have here with gradio dot row add [340:21] another row and the result this row will [340:23] contain the result equals gradio.ext. [340:27] Let's go ahead and here add y cider dot [340:31] change. So let's trigger some code when [340:34] we change the value of the slider. Here [340:37] we are applying this we are applying [340:39] this event upon the y slider. Okay, not [340:43] upon the x slider. you will see later I [340:46] will tell you that X slider down will [340:48] not when we change it it will not [340:50] trigger any code whereas Y slider when [340:53] you change it when you play around with [340:55] this value it will trigger this code [340:57] that I'm about to write in into here fn [341:00] function equals multiply so here we are [341:03] saying hey please y slider when I change [341:05] your value execute the code multiply and [341:09] then take the inputs as the following [341:11] you can see here inputs equals a list of [341:14] two numbers x slider comma y slider. So [341:18] multiply the x slider value by the y [341:20] slider value. Then go ahead and just [341:22] display the outputs as the following [341:25] gradio or result. So we have to pass in [341:29] result. This result variable contains [341:31] gradio.ext. Okay. The output will be in [341:34] a text form. Now if if we go ahead and [341:37] just app or demo dot launch. [341:43] So as I told you, you can feel free to [341:45] play around with this name. It's called [341:47] now demo. I can change it to app and it [341:50] will work successfully. You can set sh [341:53] equals true. [341:58] Now we will have a really exciting app [342:00] where you can see here when we change [342:02] the x slider, okay, [342:05] the code doesn't get triggered. So feel [342:08] free to change that. Nothing happens. [342:11] But when you change the Y slider, grad [342:14] will work and it will execute. So [342:18] there's a hidden listener that listens [342:20] for this event, the change event. And [342:23] whenever you change the value of this, [342:25] so it multiplies 15 by 42. You can see [342:28] here we have 630. If you reduce this [342:31] value by 10. Now 15 * 10. If you change [342:34] this [342:39] 15 * 10 here we have 150. Okay. So [342:42] whenever you change the value of the y [342:45] slider there is a hidden listener there [342:47] is an a latent listener that listens for [342:50] this event and tries to execute this [342:53] code multiply. So this function this [342:55] nice function it will be executed [342:58] whenever you change the value of the y [343:01] slider here. Y slider change pass in the [343:04] function multiply pass in the inputs X [343:06] slider Y slider multiply the Y-sided [343:09] value by the X slider value and they [343:10] return the output in the form of a [343:12] result which is a gradio text a normal [343:14] regular gradio text. Okay. And you can [343:17] see here this app is fully working. It's [343:20] a multiplication app that multiplies the [343:22] y value by the x value. Now let's go [343:24] ahead and apply some more examples to [343:26] practice more. Here we have with [343:28] gradual.blocks blocks [343:31] as demo. Then let's go ahead with [343:34] gradial [343:36] dot row add a new row or insert a new [343:39] row. Then we can go x slider equals [343:43] gradios slider and then pass in label [343:47] equals x and then also we have here y [343:50] slider equals gradios slider. So a [343:53] slider is a component inside gradio [343:55] which is a normal slider that you can [343:57] slide and change the values. Then we [343:59] have label equals y. [344:02] Then we have another row that displays [344:04] the value displays the the the result by [344:07] going result [344:10] equals gradio.ext a normal text input. [344:14] Then here we have x slider. So now let's [344:17] step things up and apply the listener [344:20] not only to the Y but also to the X by [344:24] going X slider dot change. So execute a [344:27] change event listener to pass in the [344:30] function as multiply multiply inputs [344:35] equals a list of two inputs. So here we [344:39] have x slider comma y slider and then [344:43] the last finally we have outputs equals [344:46] result and here we we have to insert [344:48] equal sign and also here here the change [344:52] event will work on the x slider. Let's [344:54] do the same on the y slider by going y [344:56] slider dot change pass in function as [344:59] multiply. You can see how fast we can go [345:01] here. inputs equals X slider, comma, Y [345:04] slider. And then we have at the last [345:06] here outputs equals result that we have [345:10] the same component for the row. The [345:14] second row here you can see multiply [345:17] multiply. [345:20] Okay, X slider is called X [345:23] slider. Okay, now let's run that after [345:27] we launch this demo by going demo.launch [345:30] launch [345:33] launch our app and then pass and then [345:35] pass share equals true [345:38] run that now when we change the value of [345:41] x slider it will multiply its value by [345:43] the the y value so we have here if you [345:46] multiply you can see now whenever we [345:49] change the value it will execute and [345:52] it's multiplying 13 by 0 it's zero so [345:55] here the y is always zero so the result [345:57] is always zero remember when you [345:59] multiply anything by zero the result is [346:02] zero. Now let's change the value here [346:03] make it for example 3 3 * 13 is 93 by [346:07] 39. And when we change the value here [346:10] for example here 2 * 2 is 4. [346:14] If you increase that 2 * 2 * 3 is 6. [346:20] Okay. And whenever change whenever you [346:22] change the value of y it will display [346:24] different result. [346:28] Let's go ahead and dive into another [346:31] example where where we have the button [346:33] and click. So now we have click event [346:39] button click triggers code. [346:44] So when you click on a button it [346:45] triggers some code [346:48] with gradio.blocks. So let's create [346:50] another blocks as demo or as app. Let's [346:54] go ahead and insert a row a row. Inside [346:57] this row, let's add an a text x slider [347:02] equals gradio dot slider. And then pass [347:05] in the label as x. Then let's also [347:09] create another slider by going y slider [347:14] gradial dot slider and then insert the [347:16] label as y. [347:20] Then we need another row that shows us [347:21] the output by going [347:25] result equals gradio.ext [347:28] that shows us the result of our [347:31] multiplication process. Then we have [347:33] here with gradio dot row insert a [347:36] button. So you can see now we need a [347:38] button element by by going button equals [347:41] gradio dot button and then insert here [347:44] multiply. So put a catchy and also [347:48] motivating [347:49] text here to let the user or to motivate [347:53] the user to click on the button. Okay. [347:56] Then we have button here dotclick. Now [347:59] we have you can see we are using the [348:00] click function to execute some code that [348:03] will be triggered once you click on the [348:05] button. This code or this function will [348:08] be executed. pass into here multiply [348:11] comma inputs equals a list of X slider [348:15] comma Y slider. [348:18] Then we have here [348:20] outputs equals. So the outputs equals [348:23] here [348:28] one value which is the result. Okay. Now [348:32] let's go ahead and just launch our app [348:34] by going demo.launch launch [348:38] and then pass into here share equals [348:41] true. [348:43] So all we're doing here is just we are [348:46] creating some rows. The first row [348:48] contains two sliders. [348:50] The second row contains one text [348:52] component. The third row contains a [348:55] button that contains this text multiply. [348:58] It motivates the user to click on this [348:59] button because it it uh labels multiply. [349:03] Then we have a click function that will [349:06] execute this function multiply upon the [349:08] clicking event. So when you click upon [349:10] this button, it will run this function [349:12] multiply with inputs X slider Y slider. [349:16] It will take these values and multiply [349:17] them with each other. Then it will [349:19] render the output at the results at the [349:22] result component which is the the normal [349:24] text. You can see here we have X slider. [349:29] We can see when when we play around this [349:31] values, it doesn't run anything [349:34] and the y also when you run here [349:39] we have some values. Now let's go ahead [349:41] and multiply it will multiply these [349:43] values and give you the result. So when [349:45] you specify for example here four here [349:47] for example uh uh 6 6 * 4 is [349:52] 24 and also change here for example 3 3 [349:55] * 4 is 12. [349:58] Okay, so this is how to execute some [350:01] code upon the clicking event. Now let's [350:04] create an app that takes one input and [350:06] returns multiple different outputs. So [350:10] so let's write returning different [350:13] outputs. [350:16] Here the first thing that we need to add [350:18] or we need to create is the idea that [350:21] creating a function called [350:22] make_gayscale. [350:25] This function takes from me image path [350:28] and then and then let's open the image. [350:31] So open the image. Let's go image equals [350:35] image class dot open. Then pass into [350:38] here image path. Then let's go ahead and [350:42] convert the [350:45] image to grayscale format. And let's go [350:49] image grayscale equals image.con convert [350:55] pass in L then let's return image [350:58] grayscale comma a text output called [351:03] image unage convert image converted [351:08] let's run that you can see now this [351:10] image when you pass in some image for [351:12] example make grayscale this this [351:16] function it takes from me an image path [351:19] then it returns image converted into [351:21] grayscale comma [351:23] string. So let's go ahead and pass here [351:26] some image for example the C image copy [351:29] path pass it to our function make [351:32] grayscale it will return back a tuple [351:35] containing a pill image and the the the [351:39] text image converted let's unpack that [351:41] into two variables you have the gray [351:44] image log the log okay here we have two [351:48] variables the first variable is image [351:50] gray image you can see gray image [351:53] It's now a gray image. [351:59] You can see a gray image. The same [352:00] variable here. And we have the log [352:02] variable that contains the text image [352:05] converted. Okay. So here we are dealing [352:07] with multi modalities. Let's now wrap [352:10] that into a gradio interface by going [352:12] with gr.blocks [352:16] as demo. Let's go ahead and create a new [352:20] row with gr row. And then inside this [352:23] row, let's insert input image. So we add [352:26] we need to add an input image equals gr [352:29] image component dot image component. [352:31] Then pass into here type equals file [352:34] path. And then we have also output image [352:37] equals gradio image insert an image [352:40] component. And let these parentheses [352:42] empty. Let's go ahead with gr row pass [352:46] inside this row log equals gradio.ext. [352:50] Then inside this text leave it as it is [352:52] empty. Then let's create a submit button [352:55] equals gradioton. And then we have your [352:58] value equals convert convert to [353:01] grayscale. So this is the call to action [353:04] call to action of the button that you [353:06] need to make it motivated motivating and [353:09] catchy in order for the user to click [353:11] it. Then we have here submit dot click. [353:15] Then we have here fn equals make [353:18] grayscale. This function remembers takes [353:21] for me one input and returns two [353:23] outputs. Then we have inputs equals a [353:26] inputs equals input image. Input image. [353:30] Then we have here outputs equals a list [353:32] of output image also comma log. So the [353:37] the log. Okay. We have here demo launch [353:40] and then pass here SH equals true. So [353:43] the log is the status and it's simply [353:45] this text which is image converted. [353:48] Let's launch my app. You can see here we [353:50] have a gradial app now running. We have [353:53] a drop image here. Let's choose the [353:54] image of Elmo. Let's choose it. You can [353:57] see now it's loading. Here we have Elmo. [353:59] Let's click on convert to grayscale. It [354:02] will return here this text image [354:04] converted. in addition to with this [354:08] image of Elmo converted into a grayscale [354:10] image. So here we have multiple outputs [354:12] being retained and this is the idea of [354:16] returning different outputs taking in [354:18] one input and returning different [354:20] outputs. [354:26] Let's dive into gradio themes to change [354:29] the theme of a gradio layout using some [354:31] of the available themes on the gradio [354:34] list. We can go help and then pass in [354:36] gradio themes to check what are the the [354:40] different themes that we have. [354:43] You can see now we have a massive [354:45] output. If you scroll up here you can [354:48] see all the available themes the names [354:50] of the themes that are available for you [354:52] to check out. [354:54] You can see here these are the themes. [354:56] We have the glass theme, monochrome [354:58] theme, ocean theme, origin theme, soft [355:01] theme and the color. Now let's go ahead [355:04] and try and try some of these themes. We [355:07] can go def echo. So this function will [355:10] just echo what we pass into yet text or [355:13] num. Then it returns let's make it num [355:16] better than text. Then let's return num. [355:19] So we pass in five it returns five. If [355:22] you echo six it returns here the number [355:25] six. Then let's go ahead and then I face [355:29] create an interface that will be [355:31] gradio.inface. interface to instantiate [355:34] an interface object. It takes from me fn [355:36] equals echo. The function echo inputs [355:38] equals gradio number. It takes from me a [355:42] number and it returns to me a number. It [355:45] does nothing but returns the same exact [355:47] same input that we have passed in [355:49] earlier. The theme now we can see we [355:51] have a new parameter here theme equals [355:53] gradio do themes do themes do glass. So [355:58] we have a glass theme. Let's try it. We [356:01] can go ahead and run that. After we [356:03] launch this interface, we can go [356:04] interface.launch. [356:08] Then set share equals true. [356:13] Now the the layout is being loaded. You [356:15] can see now we will have a brand new [356:17] layout. You can see the glass theme. [356:19] Here we have number five. Then submit. [356:21] It will submit the same number but with [356:23] a nice beautiful pretty theme. Okay. The [356:26] glass theme. We have also different [356:28] options. If you want to try them, we can [356:30] go ahead and copy this code, the same [356:32] exact code, so we don't rewrite the [356:35] code, duplicate it, and then change the [356:38] theme to soft. Okay, so all you need to [356:42] do is just change the theme from glass [356:44] to soft and then see the magic happens. [356:47] You can see now we have a soft theme [356:49] different than what what we have [356:51] earlier. You can see here the theme this [356:53] theme. Okay, we can pass in 10 then it [356:56] will submit and returns the just 10 echo [356:58] it. Then let's try the last theme here [357:02] which is called also monochrome. Mono [357:06] Chrome also it's a it's a nice theme. [357:09] You can see [357:13] it's an exciting theme. We can pass in [357:15] here 40 submit. It submits and returns [357:18] 40. Okay. And feel free to play around [357:20] with different themes. Also you can use [357:22] the different method which is gradu [357:25] setting a blocks [357:28] as demo. Then we can go and create a new [357:32] row by going with gradio dot row and [357:36] then append into here gradio number. [357:39] Here we can pass inside the blocks theme [357:42] equals gradio themes dot monochrome. [357:47] So we have two different methods [357:49] remember to create a layout. We have the [357:51] interface class, the automatic interface [357:54] class and also we have this manual way [357:56] of constructing the the layout here. In [357:59] this way also you can play around with [358:01] themes by passing the theme parameter to [358:03] the blocks class. So rather than passing [358:05] theme parameter to the interface class, [358:08] we can pass this theme parameter to the [358:10] blocks class. You can see here [358:13] like that. Then we can call this demo [358:16] and then launch [358:18] the app. [358:22] You can see now we have a monochrome [358:25] theme. Okay, just with one input. Let's [358:28] dive into another topic which is gradio [358:31] integration with machine learning. [358:33] Integration with machine learning. So so [358:36] far we have building layouts but with [358:39] with no in without incorporating any [358:41] machine learning into the pipeline into [358:43] the process. Now let's have machine [358:45] learning from transformers import auto [358:48] image processor. So we have this class [358:51] which is called auto image processor and [358:54] this class is used to process an image [358:57] to be able to pass it into the resonant [359:00] model. We'll use reset model then we [359:02] have a comma automodel for image [359:06] classification to classify different [359:08] images for image classification. [359:12] Let's go ahead and use that. First we [359:15] have the processor equals processor [359:17] equals here our instance from the class [359:20] auto image processor that we can dot [359:24] from pre-trained. So from the [359:26] pre-trained model which is Microsoft [359:29] /resnet-18 [359:32] so 18 layers here we can load the model [359:35] into the processor instance. Then we [359:37] have here model equals automodel [359:43] automodel for image classification [359:47] classification dot from pre-trained and [359:51] then pass into here Microsoft [359:54] Microsoft/resNet- [359:58] 18. So, so this model you can search on [360:02] hugging face and you can find it [360:05] on hugging face under the name [360:08] Microsoft. [360:13] Go to models then search filter by name. [360:16] You can see here we have Microsoft [360:17] resent 50 and also resent 18. You can [360:20] see here the model card on how to use [360:22] this model transforms. You can see from [360:24] transforms import pipeline or from [360:26] transforms the way that we used from [360:28] transforms import auto image processor [360:31] comma automodel for automodel for image [360:34] classification and then from pre-trained [360:36] pass in Microsoft/resent 50 or 18 and we [360:39] have from pre-trained pass in [360:41] Microsoft/resNet [360:43] 50 [360:44] and you can see here we have the reset [360:46] archetype you have residual connections [360:49] this is the this is the residual [360:50] architecture this is the the plane [360:52] architeia And this is the VG19. They [360:55] convert it to the VG19. And keep in mind [360:57] this architecture is used to classify [360:59] images. So let's go ahead and load that [361:01] model in. [361:06] Now the next step is to visual is to [361:08] visualize the model architecture by [361:09] printing the model. And you can see the [361:11] architecture of the model [361:19] here. If you search for ResNet model [361:22] architecture, you can see a plethora of [361:25] images on what is the residual net. You [361:27] can see here the residual neurotic. This [361:30] is the main idea behind ResNet. The main [361:34] novel idea that they have proposed in [361:36] this archae is the residual connection. [361:38] The residual connection makes the [361:40] neuronet training more stable and also [361:43] it is it eases the flow of the residual. [361:46] So it ease the flow of the data. It [361:49] makes the flow of data more stable and [361:51] easy without any degradation in the [361:53] performance. And you can see the idea [361:55] here is incorporating the input to the [361:58] layer into the output. And if you search [362:00] here ResNet paper, you can see deep [362:03] residual learning for image recognition [362:06] by coming here it all. This is the most [362:08] cited paper in the world uh by 300 above [362:13] thousand citations. You can see here [362:15] view PDF and scroll down. Here we have [362:18] the the image the same image basically [362:20] in the hugging face platform the [362:22] residual connections. Now I think our [362:24] model is loaded. We can we can show its [362:26] architecture and that is and that is [362:28] this is our architecture. You can see we [362:30] have now reset stage sequential reset [362:34] basic layer identity. So they are [362:36] preserving the identity of the layers. [362:38] ResNet convolution layer, convolution [362:40] 2D, batch normalization layer, relu. You [362:43] can see they have so many layers. It's [362:46] going deep. You can see they have deep [362:48] layers. And the main idea is that they [362:50] are using here reset shortcut. [362:53] Now let's move on and use this model [362:56] into our gradio interface. The first [362:58] thing is create a function called [363:00] classify [363:02] image. This function takes from me an [363:04] image. It receives an image. Then we [363:06] need to process that image. So in order [363:08] to use this model we need to process our [363:11] image and make it compatible with with [363:13] our model by going ahead and image [363:15] equals processor that we have created [363:18] earlier by going processor then pass in [363:21] our image return tenses equals to [363:24] pytorch to set the form of the return [363:28] tensor should be in the form of pytorch. [363:30] Let's go ahead and access the key pixel [363:34] values. Then let's go ahead and do the [363:36] next step which is passing the image [363:39] passing our image to the model to the [363:43] model and then it returns logistics. [363:45] Keep in mind any classifier in the world [363:48] would return logits and you have to [363:51] select or choose the most highest value [363:54] of these logits and this is a predicted [363:56] class. Here we have 0 to 1,000 because [363:58] they are trained on this model is [364:01] trained on imageet consisting of 1,000 [364:04] different classes. Here we have loits [364:05] equals model. So let's use our magical [364:09] model and then pass in our image dot [364:11] logets. So get in the logits. Then we [364:13] have via gra grab grab the index of the [364:17] largest value or of the highest value [364:21] between these logets [364:24] probability. So here we have so many [364:26] probabilities. We need to select the [364:28] highest probability value by going ahead [364:30] and predicted [364:32] label equals lojets.ax [364:35] and then pass in here minus one [364:38] dimension minus one dot item to select [364:40] that singular item then return [364:43] model.configuration [364:44] do ID2 label. So just index and on the [364:48] this list ID to label basically if you [364:51] go ahead here and then tap into [364:53] model.configuration [364:54] do ID to label. So map the ID into a [364:58] label. It's a huge dictionary or a huge [365:01] list. You can see here all the different [365:03] classes and categories and groups that [365:05] are existing inside imageet. We have [365:07] tench, we have goldfish, great white [365:10] shark, tiger shark, and so on. There are [365:13] 1,000 different classes here inside the [365:16] imageet. That's it. Let's go ahead and [365:19] pass in our predicted label to get the [365:21] predicted label. Then let's go ahead and [365:24] just run this function. This function [365:26] will help us to use our model and [365:32] go ahead like that. Now let's [365:36] interface equals gradio dot interface. [365:38] Let's create and construct a new [365:40] interface. Let's pass in function equals [365:42] classify [365:44] image. Feel free to name it whatever you [365:46] want. We have inputs equals grandio [365:50] image. This is the image component, [365:51] comma, outputs equals gradio.label. So [365:55] keep in mind this is a classification [365:57] model. It receives an image object and [366:00] it produces a label for for which class [366:03] this image belongs to. Then we have via [366:06] the description [366:08] nice little description that describes [366:11] what this model does by going [366:13] description upload an image. Upload an [366:16] image. Then let's go ahead and launch [366:19] that straight away. You can see now we [366:21] will have a gradio demo app that is [366:24] capable of predicting or classifying the [366:28] image by just passing here an image. So [366:31] here we have an image an area to to pass [366:33] an image. Then we can submit and then [366:35] classify it. And keep in mind this there [366:37] are there is a public URL that that will [366:40] expire after one week. But now we have [366:42] one week available to use this URL link. [366:45] Let's go ahead and search for bird for [366:47] example images bad images. Here we have [366:50] plenty of birds. Let's select one of [366:52] them. Let's copy that image or grab that [366:54] image. Go to gradio. [366:57] Drop it like there. Then let's submit. [367:00] It will classify here as be eater. Uh [367:03] bee eater. Okay. Let's go ahead and for [367:06] example grab another image of shark [367:08] image. [367:15] This is a shark image. Let's go here. [367:17] Graio submit. You can see here great [367:21] white shark. Nice. And the last search [367:24] here. Volcano [367:27] image. You can see here we have volcano [367:29] image. Let's get rid of this image. Pass [367:32] in the volcano image. It will classify [367:34] it as a volcano. [367:38] And hooray. Here we have volcano. So you [367:41] can see resident now classifies that [367:43] image as a volcano image. Now let's move [367:45] on to the next step. Now you can see how [367:47] easy it is to build gradio layouts or to [367:49] build gradio apps incorporating some [367:52] state-of-the-art machine learning [367:54] models. Easy how easy it is. Now let's [367:57] go to gradio [368:00] sentiment analysis to incorporate [368:03] another model which is the sentiment [368:04] analysis model. [368:07] Let's go from transformers import [368:12] pipeline. So we need the pipeline here. [368:15] Remember this is a highle pipeline that [368:17] allows us to load in a model. So let's [368:19] go ahead and sentiment [368:23] analysis equals [368:25] analysis [368:28] equals pipeline and then pass in [368:31] sentiment underscore anal dash analysis. [368:35] You can see here analysis. Okay, [368:42] run that. Here we have the pipeline and [368:45] then passed into here as sentiment [368:46] analysis. And remember this is defaulted [368:49] to dist the default model to classify [368:52] the sentiment or to classify text. Now [368:56] our model here is loading. No model is [368:58] supplied because we are we want to use [369:00] the default model. subbert and you can [369:02] see here it's called also this subbert [369:05] base and case fine-tuned SST to English [369:08] now my model is loaded let's go ahead [369:10] and see here model or it's called a pipe [369:15] you have sentiment analysis sentiment [369:19] analysis and this is my model you can [369:21] see the architecture of the model or the [369:23] pipeline okay text classification [369:25] pipeline now let's move on to the next [369:28] step which is using this model to [369:30] classify the sentiment by going [369:33] sentiment underscore analysis then pass [369:36] in I am sad [369:38] run that sentiment uners sentiment [369:42] underscore analysis run that I am sad it [369:45] says negative if you choose here I am [369:48] happy [369:49] it says positive my day was great you [369:52] can see my day was great [369:56] positive now let's move on to the next [369:58] thing which is incorporating this [370:00] sentiment analysis model into the gradio [370:02] layout by going defaf and then name my [370:04] function predict underscore predict [370:07] underscore sentiment. So we have to [370:10] predict the sentiment [370:12] and then pass into here a text then it [370:16] returns a result. So result equals [370:20] sentiment [370:21] analysis and then pass into here text [370:23] some nice text then return result at [370:26] index zero. then grab the label and then [370:31] return also the result at index zero and [370:34] then score. So here we are returning [370:37] both things the label negative positive [370:40] or whatever it was then returning the [370:42] score the confidence rate of the model [370:44] that the model how many how much the [370:46] model thinks that this or believes that [370:48] this text belongs to one class. Let's go [370:52] predict underscore sentiment. So this [370:55] function predicts a score sentiment and [370:57] then pass in hello I am happy this [371:02] course [371:03] is great [371:06] run that [371:09] here we have a typo so it's called [371:11] predict sentiment no worries sentiment [371:20] copy this name paste it over Here [371:25] you can see uh yeah I have a typo here [371:28] here sentiment you can see here we have [371:31] forgot an end. Now this function returns [371:33] positive comma 0.999. [371:36] So the model is confident that 99% this [371:40] sentence hello I am happy this course is [371:42] great belongs to the positive class. [371:44] Let's move on to the gradio interface [371:47] and see how easy it is to incorporate [371:49] into to incorporate it into gradio by [371:51] going gradio interface and then pass in [371:54] here fn equals predict. So the function [371:58] called predict [372:00] sentiment and then pass into here comma [372:03] inputs equals gradio [372:06] text box and then pass into this [372:08] function lines equals to comma [372:11] placeholder. You can put a placeholder [372:13] as type your text here. Type your text [372:17] here. [372:19] Then we have outputs equals a gr a list [372:22] of gradio.ext [372:24] and then pass in the label equals [372:27] sentiment [372:28] sentiment. Then we have here comma [372:31] gradio.ext and then pass into here label [372:34] equals score. So one text for the [372:37] sentiment and one text box for the [372:40] score. Let's go. iFace.launch [372:43] and start my app by passing share equals [372:46] true. And you can see now we have a cool [372:52] layout or a cool gradio app that you can [372:54] type here. Hello, I am terrible [372:59] and then submit. You can see it will say [373:01] here negative by 93%. And you can pass [373:04] in hello I am amazing. [373:07] It will say that it is positive. because [373:10] cinema is positive score 99%. And that's [373:13] it. That's how to incorporate machine [373:16] learning models from hugging face into [373:18] the gradio [373:20] layout interface GUI graphical user [373:22] interface. Now let's go ahead and then [373:25] uh dive into gradio errors. So this is [373:28] the last topic I want to discuss gradio [373:30] errors. How to show some errors in [373:32] gradio [373:34] by going defaf underscore. So define add [373:38] underscore even. So imagine a function [373:40] that you just you want to add or sum [373:43] even numbers. So let's pass in a comm, [373:46] b. So for whatever reason you just want [373:49] to add even numbers otherwise you want [373:51] to raise an error. You want to tell the [373:53] user that I don't want to sum odd [373:56] numbers. So here if not validate even. [374:00] So let's create a function called [374:01] validate even. First validate even. [374:06] This function will check whether my [374:07] number is even or odd. And based on that [374:11] information, I want to incorporate it [374:13] and check. It takes from me num. It [374:15] receives from me a num. Then it returns [374:18] return [374:20] return num. Then modulus 2 equals equals [374:24] zero. [374:26] Then let's go ahead and call that [374:28] function validate even. So here if you [374:31] want to check that we can call this [374:33] function validate even and then pass [374:35] here for example 10 it will say that [374:37] true. Yes my number here is even. If you [374:39] pass here 11 which is odd it will say [374:41] false not even 12 it will be true. So if [374:46] not validate even then pass into this [374:49] function the a parameter then raise so [374:53] raise a gradual error and you'll see [374:55] that this error will be displayed in a [374:57] nice format pass into here first value [375:01] is not even I am angry at you then we [375:04] have here if not validate underscore [375:08] even so if the B value is not even then [375:11] please raise an error so raise a grandio [375:15] error also [375:16] telling the user that [375:19] first or second second value isn't even [375:22] please update it to be able to sum those [375:26] numbers then we can go ahead else if [375:28] there's if there's no any error if [375:30] everything is fine return a plus b so [375:34] this is the main logical thing logical [375:36] process that I want to use let's go [375:38] ahead and I if create an interface by [375:41] going graddioface [375:43] then let's pass in here F function [375:45] equals add even comma inputs equals a [375:50] list of gradio dot number comma gradio [375:55] dot number. [375:57] So we are here passing two number inputs [376:00] comma outputs equals gradio dot number [376:04] then iace launch launch my app and let's [376:08] see what happens. [376:13] So now we have here a gradual layout. We [376:17] have two inputs. A let's pass in here [376:19] four plus 8. You can see four is even 8 [376:22] is an even number. Submit it. We don't [376:25] have any issues. But once we pass the a [376:28] input as seven then [376:32] now gradio is angry at you telling you [376:34] hey error first value is not even. You [376:38] have to make sure that it is even. Once [376:40] we make it even you don't have any [376:44] error. Then the second value B here if [376:46] you pass here seven or 9 submit then B [376:51] is not even it says error second value [376:54] is not even update it and then go ahead. [376:57] Okay. So this is how you can raise [376:59] errors using gradio and in a nice [377:01] format. Now let's if you want a lower [377:04] degree of uh of danger for example you [377:07] don't need you don't want to display [377:08] this type of danger which is red if you [377:11] pass here 11 you can see it's a red [377:14] error it indicates that this is there is [377:16] something dangerous if you want to lower [377:19] that degree we can easily go ahead and [377:21] just update this function here of error [377:25] error into warning [377:28] and here warning and then since This is [377:32] a warning. You don't have to raise it. [377:33] So you don't have to raise this warning. [377:36] Okay. Now we have a normal warning. [377:38] Let's run that again. Run this again. [377:41] You can see now [377:46] here we have a. Let's pass for example 8 [377:49] + 20. They are all even numbers. Submit. [377:53] You can see here there's nothing to [377:55] worry about. Once we make a for example [377:58] 11, submit it. You can see now there's a [378:01] warning first value is not even. So this [378:04] is a warning. Then we have here B once [378:06] we make B for example 13. Okay. And and [378:10] also A is 11. So none of these numbers [378:13] are are are even. We can go submit. And [378:16] you can see here we have two warnings. [378:17] Now first value is not even. Second [378:20] value is not even. And at the last you [378:22] can also raise this is a warning. You [378:24] can see there's some type of danger [378:27] still. But if you want to raise some [378:28] info telling the user everything is [378:31] fine. But keep in mind keep in mind that [378:34] this information is whatever you want to [378:37] tell the user is [378:40] pass here for example five pass here [378:43] six. You can see now [378:46] there's an info message first value is [378:48] not even. And that's it. That's gradio. [378:51] I hope this gradual tutorial is fun. So [378:55] you know you should a you should be able [378:56] to know you should you should know how [378:58] to use gradio to host or to create [379:02] layouts and also incorporate machining [379:04] models into it. You can see we've done a [379:07] lot of the layouts how to process images [379:09] and also a lot of the stuff about gradu [379:17] section. In this section we'll discuss [379:19] the spaces tab inside hugging face [379:23] website. Here when you go to your [379:25] profile and then move to your profile it [379:28] will direct you to the first section [379:30] which is called spaces and currently I [379:33] have four spaces. Hugging face spaces is [379:36] a free public platform for hosting and [379:39] sharing interactive machine learning ML [379:42] applications and demos directly on your [379:44] profile. It functions similarly to [379:47] GitHub but is specifically designed for [379:50] the ML ecosystem allowing users to build [379:53] a project portfolio, showcase work and [379:56] collaborate with others. Key features [379:58] and functionality of spaces are as the [380:01] following easy deployment, interactive [380:03] demos, version control, hardware options [380:07] and flexibility and many many more [380:10] community and collaborations and many [380:11] more. So let's dive into how to create a [380:15] hugging face and deploy your own model. [380:18] So here you can go to your profile and [380:20] then it will show us this drop-own menu. [380:23] Let's go to new space. Then it will [380:26] direct you to this page create a new [380:28] space. Spaces are git repositories that [380:31] host application code for machine [380:33] learning demos. You can build spaces [380:35] with Python libraries with like gradio [380:38] or using docker images. So spaces are [380:41] git repositories. So they are like the [380:44] GitHub repositories that host [380:46] application code for machine learning [380:48] nodes. They are dedicated for machine [380:50] learning. First here we have the owner. [380:52] You can choose your own profile or other [380:54] profile that you have. Then let's name [380:57] our space here in this input. New space [381:00] name. Let's name it as diffusion model. [381:04] So this is my space name. Then here we [381:07] have an input that says put some short [381:10] description that describes the purpose [381:12] and the functionality of this model. [381:14] Let's go for example diffusion model to [381:18] generate [381:20] emnest amnest numbers or emnest detest [381:25] numbers. Yeah, we have some license. [381:28] Feel free to choose whatever license you [381:30] would like. In this case, I will choose [381:31] the most common one which is MIT the MIT [381:34] license. Then let's scroll down here. [381:36] You can see select the space SDK. Let's [381:38] choose gradio. Actually we will use [381:41] gradio to build this interface and then [381:43] deploy it or host it to hugging face [381:46] spaces. Then let's go to here. You can [381:48] see we have space hardware. You can [381:50] choose CPU basic or zero GPU or custom [381:53] hardware. In this case I will stick to [381:55] the default CPU basic. And here also we [381:58] have choose a gradual template whether [382:00] you want blank chatbot text to image [382:03] leaderboard or trackio. I will keep it [382:05] the same or the default the default [382:07] option as blank. [382:10] Then here you can see we have public or [382:12] private. This depends on your your own [382:14] preferences. I will keep it as public to [382:16] be able uh to be able to share it with [382:19] other people. Let's go and click on the [382:22] button create space. You can see now we [382:24] have a space here. Created successfully. [382:27] Successfully. Let's go to files. [382:32] You can see here we have get attributes [382:34] and we have here readme.md. [382:36] These are the default files that hugging [382:39] face creates it for you. And you can see [382:42] if you go and click get attributes, it [382:44] will show you all these different [382:46] options. You can see we have the dot. [382:48] PTH short for PyTorch. PT short for [382:52] PyTorch. Also another extension for [382:55] storing model weights for PyTorch. Also [382:58] we have PKL pickle and also we have this [383:02] NNx short for open neural network [383:05] exchange and we have so many extension [383:08] here extensions and if you go to here [383:10] diffusion model it will be direct it [383:12] will direct you to the same page and [383:14] also we have rhymed.mmockdown [383:17] where it contains some meta information [383:19] about your project or about your haggus [383:22] space title diffusion model emoji they [383:25] choose this emoji for you and We have [383:27] also color from red, color to blue, SDK [383:30] gradio, the SDK version 6.1.0. And if [383:34] you want to change this emoji here, you [383:36] can go to edit. So feel free to change [383:38] it. And then you can scroll down and [383:40] then change this emoji to whatever you [383:42] want. Then you simply just go ahead and [383:44] commit the changes here. After you [383:47] change it, you can commit it and go [383:49] ahead. But now let's go ahead and just [383:52] use our diffusion model to deploy it on [383:54] Hag spaces. [383:58] So the first step is we need a command [384:00] line terminal. Let's go and search here [384:02] for terminal. You can see here we have [384:04] the terminal [384:06] window opened. The first thing is we [384:08] have to check our git version [384:12] get and then let's go zoom in a little [384:14] bit dash version. So now so now let's [384:18] check what is the version what is my [384:20] current version of the g of the g [384:23] package. You can see here have g version [384:25] as 2.52.0 [384:28] windows.1. [384:29] So if you don't have git this running [384:32] this command will result into an error [384:35] because it don't have git. And if you [384:37] don't have git please go here to Google [384:39] and then search install git. And here [384:42] you have you have the main website the [384:45] first website that shows it's called git [384:48] install. Click on it and it will direct [384:50] you to this web page that says install. [384:53] Then you have here many options there. [384:55] There are different OS systems. We have [384:57] Windows, we have also Mac OS and we have [384:59] also Linux. If you are on Windows, click [385:02] on Windows and then download here. If [385:04] you are on Mac OS, you can download from [385:06] here. And if you are on Linux, you can [385:08] download from here. Okay, I already have [385:11] I I already have installed. So you can [385:14] see here the version has been printed [385:16] successfully. Success success [385:19] successfully. And let's go ahead and [385:21] clear our tab here. Let's go to MK [385:24] directory. Then let's go and name some [385:27] of my folders. So we So feel free to [385:30] name it whatever you want. We can go and [385:31] name it as for example my diffusion [385:37] model. And now it created that folder [385:40] for me. Okay. And you can check that by [385:43] going ls. Here you can see we have so [385:46] many folders here. One of them is called [385:49] my diffusion model. So this so this is [385:53] my folder my diffusion model the same [385:56] that I have created over here my [385:58] diffusion_model. [386:00] Now let's go ahead and clear that. The [386:03] next step is ls dot dot/d downloads then [386:08] / diffusion. So whenever or when you [386:12] want to deploy a model on hugging face [386:14] spaces you need a custom or a particular [386:17] structure that you have to construct and [386:19] build. For example here I have in this [386:22] notebook I trained and built a model [386:25] diffusion model to convert from text [386:28] into an image or to draw an image out [386:31] from text. So you provide some text for [386:33] example three four five and then it [386:35] generates those emnest numbers. First we [386:38] have some imports here. Import the [386:40] necessary libraries. Then also here we [386:42] have set up the CUDA if torch CUDA is [386:44] available otherwise use CPU. And then we [386:46] have the batch size embedded dimension [386:48] time steps epox [386:50] as 100. And I already trained that model [386:52] for 100 epochs. You can see here also we [386:55] have the transformations. We have to two [386:57] tensa to convert the mnest images into [386:59] tensa. And also we have lambda that [387:01] performs a specific a specific function [387:04] on the on our images. [387:07] Here we have conver here we have stored [387:10] the mnest data set class inside a [387:13] variable called train data and the same [387:16] we have passed this train data variable [387:18] into the data loader class. Then we have [387:21] a train loader variable that contains a [387:24] data loader object that is compatible to [387:27] be passed to the PyTorch model and the [387:29] PyTorch model will will be able to train [387:32] on it. [387:36] Here we have also the unit class. You [387:38] can see we have the archae of the model. [387:40] We have the first encoder convolution [387:42] 2D. We have some convolution to DD [387:44] layers that performs the operation of [387:47] convolution. Also here we have some [387:48] decoder that does the transpose [387:51] operation of convolution and also here [387:53] we have the for method that calls these [387:56] layers step by step and then it defines [387:58] the forward computation of the neuron [388:01] net [388:03] after that here we have set the [388:05] optimizer as Adam to optimize our [388:07] weights and here we I've built and [388:09] written a training loop that trains our [388:12] model you can see we have the foreign [388:13] pass to pass into the model and then we [388:16] have here you can see we have for pass [388:18] uh loss calculation uh optimize a zero [388:21] grad doing some back propagation and [388:23] then optimize the step to optimize and [388:25] improve our weights and then I let the [388:28] model train for 100 epoch you can see [388:31] here we have epoch 100 out of 100 and [388:34] the loss became 0.06 so we started the [388:38] loss from here 0.09 until it decreased [388:41] decreased decreased until it became 0.06 [388:44] 06. Then we have here the loss curve. [388:46] You can see it's going down and down. [388:52] So the loss curve is is getting lower [388:54] and lower. After that here we have a [388:57] sampling function. So this function is [388:59] really important. It it is considered [389:01] the prediction function. So here it [389:03] takes from me you can see word a word a [389:06] prompt then n= 8 steps equals 100 [389:10] guidance scale equals 3 and then it [389:13] performs this unconditional labels for [389:15] cg for cfg and then it turns it returns [389:19] out an image as you can see here if we [389:22] type over here for example the number [389:25] seven [389:28] then it gen it generates here the number [389:30] seven you can see here it's it looks [389:32] looks like a seven number. If you run [389:34] again, here we have a seven a an image [389:37] that looks like seven. Okay, so the [389:39] model here is trying the model is trying [389:41] to generate and to synthesize the image [389:44] of seven. You can see here we have seven [389:46] and if we write here one in natural [389:50] language uh way it will turn it into an [389:53] image. You can see we have one here the [389:55] number one and also if we provide here [389:57] two it will create and it will generate [390:00] the number two. If we go again here you [390:02] can see [390:04] we have two number. So we have a [390:06] diffusion model we have trained it for [390:08] 100 epochs and we use this architecture [390:11] that is leveraging the unit model. You [390:13] can see here we have the class unit [390:17] and this class here has so many [390:18] convolution layers and transpose [390:21] convolution layers that is working on [390:22] our images and trying to convert and [390:25] trying to turn our prompt from a natural [390:28] language into text into image. So [390:30] converting text to image. [390:34] After that we have another sample here. [390:37] If you pass here for example the number [390:39] six then it will generate here the [390:42] number six. You can see here we have a [390:43] look lookalike number six. [390:48] So here you can see we have here an [390:50] image that looks like the number six. [390:54] And if you type over here also for [390:56] example four then it will generate here [390:59] some numbers that looks like four. [391:03] So you can see here [391:09] you can see here we have four number [391:10] four. Okay. [391:14] And let's go move on to the next step. [391:16] And then I saved my model into some [391:18] files by going import OS osmake [391:22] directories. So create a directory [391:25] called checkpoints and then set the [391:27] parameter exist okay equals true. Then [391:29] save the model dot state dictionary [391:32] inside this file that has extension. PT [391:36] and also save this model the model [391:39] weights save it into a file with the [391:41] extension PTH. So PyTorch [391:46] and then I saved it. You can see here we [391:48] have here the model the model saved. If [391:50] you go to here, you can see I have here [391:52] two files. The first has the name [391:55] texttodigit function_mnest.pt [391:58] so PyTorch and the second is text to [392:01] digit diffusion emnest. Pth and they are [392:04] they are all the same. So differing with [392:07] different extensions but they represent [392:09] the same set of weights [392:13] because remember PyTorch library [392:15] supports two different types of [392:17] extensions PT and PTH and they are [392:20] considered the same. Then we have here [392:23] you can see model.load set load set [392:26] dictionary then pass into here the [392:27] function torch.load and then pass here [392:30] one of these files the path that that is [392:33] pointing to one of these files. it will [392:34] load in the weights and create a new [392:37] instance of the model with the same set [392:39] of the weights as our trained model [392:41] earlier. And here we have also the [392:44] function of the of the loaded model. If [392:47] you pass here the number for example [392:49] number one then it will generate some [392:51] images that is representing the number [392:53] one that is representing number one. And [392:56] if you pass here for example eight it [392:57] will generate the number eight. [393:00] You can see here we have some images [393:01] that looks like number eight. [393:05] Let's move on to here. And then we have [393:07] some function to normalize input. So you [393:09] pass in some text. Then we have here [393:10] return the text and also raise some [393:12] value error. Please enter a number. So [393:14] if you don't provide the number, it will [393:16] raise an error. Then we have also here [393:18] you can see a function gradio generate [393:21] that creates some try and accept block. [393:25] So try word then normalize the text [393:29] except exception. We have if you have [393:30] some error then return none and return [393:33] the string of the error and also here we [393:35] are returning the image and returning a [393:37] string indicating that generated digit [393:40] as word [393:42] and at the last we created a gradio [393:44] interface that takes from me a gradio [393:47] interface the input in the form of a [393:49] text box. We enter some text and then [393:51] the model tries to generate and convert [393:54] this text into an image and then it [393:56] returns that that output in the form of [393:58] image and you can see the label as [394:00] generated emnest digit type equals numpy [394:04] and then if we provide here a number for [394:07] example number two [394:10] submit then it here number two you can [394:13] see it tries to generate number two [394:17] and if you provide here one submit it [394:20] tries to join number one. So you can see [394:21] here we have one submit again one and if [394:25] you provide for example number six it [394:28] tries to join number six. [394:36] So you can see some images looks like [394:38] number six. Okay [394:42] you can see here we have number six. Now [394:44] let's move on to the next step where [394:46] here we have built also another grid [394:48] interface but using the manual way of [394:52] doing that. Now let's move on to the [394:53] next step. So once you train a model [394:56] once you have a model and build it train [394:58] it you have to construct a file that is [395:02] called model.python model.python this [395:05] file will instantiate the backbone [395:07] architecture of your model. So here you [395:09] have the same code as the unit class [395:11] before but we have put it inside a [395:14] separate notebook file inside a separate [395:16] cell and then you can and you can see [395:18] here after that we have defined a [395:20] function called create diffusion model. [395:23] This function takes from me several [395:25] parameters that sets the values the [395:28] default values for the for some [395:29] hyperparameters. And these [395:31] hyperparameters will be defined and used [395:33] inside the architecture here the unit. [395:36] For example, embedding dimension number [395:38] of classes, time steps and whatever [395:41] hypamearameters you need to set. At the [395:43] last or finally we return a a a version [395:46] of the model with no in weights just a [395:49] model the model architecture. And also [395:51] we need a file then feel free to go [395:54] ahead and file download and then [395:56] download file in the form of python. So [395:59] you have to have a copy or you need to [396:01] have a copy with the extension with the [396:03] extension python [396:06] that is containing a python script that [396:09] creates for you a model architecture a [396:12] plain model architecture without any [396:14] weights. Then we plug those weights to [396:17] this model architecture. And here we [396:19] have a notebook called app.py.i [396:22] python notebook. This is also a notebook [396:25] that contains some code for constructing [396:29] the app for gradial. So here we are [396:31] putting all the code together. So you [396:34] can see we are from model. So from this [396:37] module model from this sub package [396:40] called model please go ahead and import [396:42] the function create diffusion model. And [396:45] that's it. This is our function create [396:48] diffusion model. So we are keeping our [396:50] code here neat, beautiful and easy to [396:53] organize. So from model import this [396:55] function also from time it import [396:58] default timer as timer because we need [396:59] to time our model uh our model time the [397:03] time it takes to make a prediction or to [397:04] gener image and also from typing [397:06] importable and then we have here CUDA [397:09] device to set the device as CUDA if if [397:13] CUDA is available otherwise it set it as [397:15] CPU and we have the same logical code as [397:18] you can see but at the last we are [397:21] creating the grad interface you can See [397:24] here we have gradio.inface then pass [397:26] function equals predict inputs as a [397:28] textbox label into some digit and we [397:31] have outputs in the form of an image [397:33] then we have this gradual interface. So [397:35] here we have a file for training and [397:38] building the model and also we have a [397:40] notebook for [397:42] storing in a code that instantiates the [397:45] model architecture and here we have a [397:47] file or app. Python that contains all [397:52] the code and combines it into one file [397:54] to create and instantiate an app for [397:57] hugging face spaces to work properly and [398:00] to work in a proper way. Here you have [398:03] to go file and then download then [398:05] download this file in the form of [398:08] Python. Then you you have to have a [398:11] folder called for example diffusion or [398:13] whatever name you would like. This [398:15] diffusion folder will contain the [398:17] app.python Python that we saw earlier. [398:19] Also, it will contain model.py. [398:23] So, you can see we have the file [398:24] app.python. Also, we have model.python [398:28] that contains a code to create an object [398:31] of the model to create an instance of [398:33] the model. And also, we have a file, a [398:35] special file special called [398:37] requirements. .txt that contains the [398:40] requirements and the necessary libraries [398:42] that you need to require when your [398:44] Gradial app runs. So when your hugging [398:47] face runs, it will require some [398:50] libraries, some packages, some modules [398:52] to be installed. And here you are [398:54] telling hugging face, hey hugging face, [398:56] I will upload to you this model. Please [398:58] make sure you have these libraries [399:00] installed already. Torch with this [399:02] version, torch vision with this version [399:05] 0.24 and gradio with this version 550.0. [399:10] And also here we have a two files. [399:13] Actually you don't need two files but [399:15] here we are providing hugging face with [399:17] two files. The first ends with the [399:20] extension PTH and the second ends with [399:23] the extension PT. They are all the same. [399:26] Feel free to choose whatever you [399:28] preferring. If you prefer PTH, use that. [399:30] If you prefer PT, use that. You don't [399:32] need to have two files. But here we have [399:34] the weights. We plug these weights to [399:36] the architecture. Then we trigger that [399:39] code inside the app. And we have some [399:41] requirements to tell Gradio and hugging [399:44] face that we require these libraries to [399:46] be installed in order for our app to [399:49] work in a proper way. [399:52] Let's continue with our prompts to [399:54] deploy our app or our model to hugging [399:56] face spaces. First you can see we are [399:59] inside the directory my diffusion_model [400:02] that we have that we have done earlier. [400:05] Let's go ahead and ls so list everything [400:09] inside the dot dot slashd downloads [400:13] slashde diffusion. So the the diffusion [400:17] directory and it will show us what is [400:20] the content inside the diffusion [400:22] directory. You can see we have app model [400:24] requirements txt text dot_2 digit [400:28] diffusion model_mnest.pytorch [400:31] and the other extension. PTH the same [400:33] files that we have inside our diffusion [400:36] directory inside downloads. So remember [400:39] I downloaded these files so they are now [400:41] exist in the downloads directory and it [400:43] should be the same. Now let's move on to [400:46] the next step where we have to go to [400:49] copy these files. So remember currently [400:52] if you do ls it will show here nothing. [400:56] Let's go ahead and copy. So we have to [400:59] get get clone. Then we have to pass in [401:02] the path of the repo of the hugging [401:05] face. We can go here app and then we can [401:08] copy this URL. [401:11] Go into my command line. You can see now [401:13] once I do ls here we have nothing. But [401:16] once we get clone and then pass in my [401:20] path of the repo and then clone the [401:22] repo. You can see now it's cloning into [401:24] diffusion-model. [401:26] And you can see here we have also [401:28] enumerating objects. And now once we go [401:31] ls now we have this folder which is [401:34] called diffusion-model. [401:35] And if you go change directory [401:37] diffusion-model [401:39] it will direct us to that directory and [401:41] we have we could have ls and it will [401:44] show us that now we have cloned our repo [401:46] and you can see here currently my repo [401:49] just contains two files readme.md.get [401:53] attributes and they are showed over [401:55] here. You can see now we have these two [401:57] files readme and get attributes. So we [402:00] have cloned our ribbo successfully. [402:03] Let's go ahead and clear my page and [402:05] move on to the next step. Now the next [402:07] step is remember that ls do/d downloads. [402:10] So I need now double then we go [402:13] downloads [402:14] /deusion. I have here the contents app [402:18] model. I need to part to copy these [402:20] contents and paste them over the [402:22] diffusion-model directory by going cp [402:25] short for copy-r recursively and we have [402:29] here dot dot slash dot dot slash [402:31] downloads and then we have here / [402:34] diffusion and then move everything [402:36] inside the diffusion by putting star [402:38] then space dot. So we move everything [402:42] here into our current working directory [402:45] that is representing the fusion-model [402:48] directory or folder. Let's go ahead. You [402:50] can see now it's copied it successfully. [402:53] And if you go and show ls now hooray, we [402:56] have app.python model.python [402:59] readme.markdown [403:01] requirements.txt [403:03] and my weights files. So now we have got [403:06] all the necessary parts, the necessary [403:09] files that we need to run our gradio app [403:13] on hugging face. Now all we have to do [403:15] is just go ahead and try to [403:19] uh try to commit or push these files [403:21] into hugging face. So the first thing is [403:23] go get space remote-v. So what is my [403:27] remote origin? What is my origin? You [403:29] can see here I have origin [403:31] https/huggingface.co. [403:34] cospaces/my [403:37] username/defusion-model [403:39] and we have two permissions available. [403:42] We have fetch. So the permission fetch [403:44] and also push you can also push to the [403:47] hugging face spaces and here you can [403:49] also go ahead and get lfs dash lfs and [403:54] then install. So we need the get lfs. [403:57] Why? Because [403:59] currently my file my model file is [404:02] larger than 1 mgabyte. So if you have [404:05] large files at least they are above 1 [404:08] megaby or above 10 megabytes you need to [404:11] install the lfs package by going to [404:14] Google and then get LFS installed. If [404:17] you don't have it already you can [404:19] install the gfs. You can see get large [404:22] file storage an open-source git [404:24] extension for versioning large files. [404:27] And you can download that from here to [404:29] be able to upload large files onto [404:32] hugging face. And you can see here you [404:34] can go easily get LFS large file storage [404:38] install. Then we can also get uh track [404:41] get LFS track and then have this file. [404:44] So let's move on to our command line. [404:47] Get LFS install. You can see now I have [404:50] it already installed. Updated git hooks [404:52] get LFS initialized. And you can track [404:55] files easily by doing this get space LFS [404:58] then track track every file for example [405:02] my model file and we can go here to my [405:05] here folder and then copy the name of [405:08] the of the model file. So remember we [405:10] don't need the model python but rather [405:13] the weights file because the weights [405:15] file could get large and large. Then we [405:17] can go here and paste and paste it over [405:19] here. Now we can go you can see now get [405:22] LFS track. You can see here the path [405:25] phth is tracking text to digit [405:29] diffusion_mness. [405:31] PTH. [405:33] So now our model file weights is tracked [405:36] in a proper way. We can move ahead and [405:38] also uh continue with the prompt. So get [405:42] space lfs track every single file that [405:45] ends in the extension. PTH we can go [405:49] ahead and you can see PTH is already [405:52] supported. So now we have we have [405:54] tracked our files. We can go ahead and [405:56] do the next step. Get add.get [406:00] attributes and feel free to read here [406:02] into the documentation. You can see here [406:04] we have also get add.get attributes. You [406:07] can run that command and it will execute [406:09] over here. You can see here we can run [406:12] and it added the get attributes and also [406:15] let's get add star. So add everything [406:19] and then if you check the status by [406:21] going get status or status you can see [406:23] now all our all of our files are green [406:27] that means we have tracked every single [406:30] file we have inside our folder [406:32] diffusion-model. [406:34] So now all we need to do is just go [406:36] ahead and commit that by going get [406:39] commit and then here dash m. So pose a [406:42] message or send a message that says [406:44] initial commit. So feel free to read [406:47] here to put here or write here whatever [406:50] message you want to put or write but in [406:53] the in the first commit let's keep it [406:55] simple and write here initial commit. [406:57] You can see now it will commit that to [407:00] the hugging face and then let's go ahead [407:01] and get push. It will push that to the [407:04] hugging face spaces. You can see now [407:06] it's trying to push my files and now [407:08] it's uploading. So keep in mind [407:11] sometimes it will ask you for a token ID [407:15] or a token that you need to provide here [407:17] and that is easy. All you have to do is [407:19] just write the command get remote and [407:23] then set URL. So you can go get and then [407:26] write here remote set dash URL then pass [407:30] into here origin https and then put here [407:34] your username. So your user dash name [407:36] you can put here whatever username you [407:38] have. Then we have colon token. So here [407:41] you can replace the token your token ID [407:44] at huggingface.co/reopath. [407:49] And keep in mind this repo path should [407:51] contain your us your spaces like that [407:55] spaces slash your username slash your [407:59] repo name like that. Okay. Then you can [408:03] run that and it will ask you to provide [408:06] the the token and it will authenticate [408:08] and also then you can run the next [408:10] command which is which says get pull [408:14] origin and it will push your files into [408:16] hugging face spaces. [408:18] With that being said now we have we have [408:20] done we've done we are done with the [408:23] command line. Let's go to here and check [408:25] our you can see now our hugging face is [408:29] now starting. You can see now it's [408:31] starting and trying to start and launch [408:33] our app on hugging face spaces. So it [408:36] might take some minutes for example 2 [408:38] minutes to 30 minutes to to 3 minutes [408:41] and you have to wait until it finishes [408:44] building and starting then you then you [408:46] can be able or then you can review your [408:49] app and view it and also actually start [408:51] using it. Now you can see my gr my [408:56] hugging face is now running [408:57] successfully. We don't have any errors. [409:00] We don't have any issues. Everything has [409:02] run smoothly. And if you go to your [409:04] username here, you can see we have now [409:07] many spaces. And our space that we have [409:10] done in this section or in this course [409:12] is this space diffusion model. Diffusion [409:15] model to generate eminent numbers. And [409:18] if you click on it, you can see uh it is [409:21] committed 4 minutes ago or it has run [409:24] for 4 minutes now ago. And here we have [409:27] running. [409:29] You can see now my ripple or my hugging [409:32] face is now running smoothly. And this [409:35] is my dashboard or this is my grandio [409:37] interface. Here we have some input text [409:40] box where you can write here a human [409:42] readable number name for example three [409:45] or for example one. Then you can go [409:46] ahead and submit it and it will generate [409:49] here some numbers. As you can see [409:55] submit again [409:57] if you pass here three submit [410:01] it tries to write here the uh it's it [410:04] tries to draw the number three you can [410:06] see here it's trying to draw number [410:08] three and if you pass here number four [410:10] for example submit [410:14] you can see now it is trying to draw [410:17] number four and if you pass here two for [410:19] example [410:22] it tries to draw or to write number two. [410:26] So this is my diffusion model. You can [410:27] see here it wrote or it draw here number [410:31] two. And feel free to upscale this model [410:33] to train it further or to upscale the [410:35] architecture or even use transfer [410:38] learning techniques to better make this [410:40] model work properly. Okay. And here you [410:42] can see we have the the generation time [410:44] in seconds. So you can you can see here [410:47] every time we generate here we have a [410:50] differing amount of time that the model [410:52] has taken to produce or to draw this [410:54] image. Here we have the generation time [410:56] as 0.34. [410:58] And if you pass here for example number [410:59] six it also will generate and submit [411:02] number six. You can see [411:06] a draw number six [411:09] an image that looks like number six. [411:10] Okay. Now let's move on to you can see [411:13] here my gradual interface. now is [411:15] working properly and that's it. That's [411:17] how you can upload and also here we have [411:19] the share via link. You can you have the [411:21] ability to share this app or to share [411:24] this gradio interface via a link. You [411:26] can go here, open this link and it will [411:29] direct you to the same gradio web [411:31] interface hosted on the on hugging face [411:34] spaces and you can see it belongs to my [411:38] username my account and it's named [411:40] diffusion-model and currently I have [411:43] here zero likes hopefully give me a like [411:45] here and then we have this text box and [411:47] it draws some our output image here and [411:49] then also I have some other spaces here [411:52] that I would like you to see [411:55] if you Go to hugging face and then my [411:58] account. You can see here I have so many [412:00] spaces. I have the generation space GANs [412:03] model food vision big and also I have [412:05] food vision mini. Let's go to food [412:07] vision big. Now you can see my my app [412:10] here or my grado interface is working [412:12] smoothly. You can see it's running and [412:14] here we have this area where you can [412:16] drop in here an image and it will [412:18] classify this image for you into 101 [412:22] different classes of food categories. If [412:24] we submit it here we have pizza and also [412:26] if we grab another image for example you [412:28] can see here I have examples. If you [412:29] choose it it will use that example to [412:32] demonstrate how the the workflow works. [412:35] And if you go to Google and search over [412:37] here trau for example [412:40] images it will show you some tiramisu [412:43] images. Let's copy let's drag some of [412:46] these images and drop it over there. We [412:48] can go like that. Drag it. Drop over [412:51] here. Drop the image here. You can see [412:54] now we have a term image. We can go and [412:56] submit it. [412:59] Then it classified that as termisu. [413:04] That's it. I hope you find this tutorial [413:07] helpful and thanks for watching.