Remember that most AI systems are proprietary. We have to pay a subscription for them, and no one knows how they work or what data they were trained on. Well, now hold on to your papers, fellow scholars, and check out this incredible work. And when I first saw it, my jaw hit the floor. They absolutely knocked it out of the park. They spilled all the secrets. This is an AI assistant that is free for all of us forever. But not just the model itself, they also gave us a 51-page research paper, which might be the holy Bible for creating such a system for now. Why is that? Well, they show us every step of the way of how it was done and the data set it was trained on as well. That is extraordinary. Usually, something is always missing, but not here. They call it NeMo-Tron 3 super. And we are going to find out whether it is indeed super or not. Okay, so in goes 25 trillion tokens as training data, and out comes a 120 billion parameter AI assistant that is how smart exactly? Well, it roughly matches the best closed frontier models from about a year and a half ago. Note that these models cost billions of dollars to train, and every detail about them was kept in secret. And now, we just get this kind of stuff for free. That is mind-blowing. This is amazing for us consumers and fellow scholars. So, as you see, it is really smart. Up with some of the best open models out there in most tests, but note that it's still a bit behind some areas. But here's something that really surprised me. In this result, they showcase two versions of the new model, BF16 and NVFP4. They perform roughly the same in terms of accuracy. So, why the big fuss about it? Well, look at this. Holy mother of papers. Wow. Well, the NVFP4 version is about three and a half times faster than their other model, and it is up to seven times faster than similarly smart open models. So, the story is not just the similarly smart part. No, no. The story is that it is seven times faster while it is similarly smart. Goodness. Okay, so how on earth did they do that? Well, here are four secrets they gave us from the paper in very simple words. Dear fellow scholars, this is 2-Minute Papers with Dr. Karoly Zsolnai-Feher. Okay. NVFP4. What is that? This is a way for speeding up the AI to run a great deal faster by essentially compressing the mathematics it uses. Seeing a long number and rounding off a few digits, you get a smaller format. Less work. What's wrong with that? Well, everything. Normally, if you do that, you lose too much accuracy, and the system will output nonsense. However, here scientists did it the smart way. They left the most sensitive calculations alone and did this rounding for the rest, where it does not cause trouble. The result is that it runs up to seven times faster than many other techniques, and we saw that it gives us no meaningful loss in accuracy. Magic. But, there is more magic. When the other AI techniques write their answer, they write it token by token. Let's simplify by saying word by word, writing one word at a time. But, not this one. This one calculates several future words at once, a whole sentence. Almost. Specifically, seven tokens. And then, the system verifies the seven tokens in one go. Another massive speed up. They call it multi-token prediction. But, why stop here? Let's add even more magic. They showcased these weird things they call the member layers. What do these do? Well, traditional AI systems have a bit of a memory problem. They work like a student who constantly rereads the textbook over and over again when they are given a question. Scientists at Nvidia say that's not the way to go. Memory is precious. So, instead, read the book only once and take highly compressed notes. So, this kind of memory remembers important details about the conversation. However, it is still smart enough to throw away the filler words. Thus, this system can process massive amounts of data efficiently. Okay, all this sounds glorious, but this still does not give us a working system. Why is that? Well, this is why. You see that there is a lot of addition here. That is the problem. The AI generates your answer step by step. And because we rounded off the numbers, there is a little error. That's not a problem. Here is the problem. There are many steps, and the error is magnified through each step. Imagine trying to walk to your car, which is 100 steps away. But, you feel a bit sluggish today, and every single one of your steps is a bit smaller than it was before. What's the result? Well, of course, after a hundred steps, you are still really far away from your car. So, what is the solution? Well, scientists solve this by adding back some random noise to the system. But wait, this noise is carefully crafted in a way that it averages to zero. So, your new steps are sometimes smaller and sometimes bigger than they used to be. But, if you average them out over a hundred steps, you will be exactly at your car. So good. They call this stochastic rounding, and it is a genius idea. Now, not even this technique is perfect. For instance, when I give it my favorite question about assembling robotic cows with lots of math. I like this guy a lot, but it thinks for almost an hour to get me an answer for that one. That's a lot. So, if I have workloads like that, I like to run it on a much faster Lambda instance. But, I still think the AI game has suddenly changed. Closed systems used to dominate. Now, not anymore. It seems to me that Jensen at Nvidia is not playing games here. It's in the news that they are going to invest tens of billions of dollars into fully open systems like this. Look, I am not a money person. I don't know how that works exactly. But, if we get to own more amazing free AI systems like this, well, sign me up for this one. What a time to be alive. And there is so much more in the paper. I would definitely love to come back for at least another video on it. Let me know in the comments if you would like that. And if you enjoyed this, subscribe and hit the bell. Here you see me running the full DeepSeek AI model through Lambda GPU Cloud. 671 billion parameters running super fast and super reliably. This is insane. I love it and I use it on a regular basis. Lambda provides you with powerful Nvidia GPUs to run your own chatbots and experiments. Seriously, try it out now at lambda.ai/papers or click the link in the description.
Generated algorithmically for Search Engine
Indexing.