Illustrated Guide to Transformers Neural Network: A step by step explanation

this is great but would've loved if you could have taken a sample sentence as an input and show us how it transforms as it moves through the different parts of the transformer.
Perhaps an idea for the next video!

I must say you’ve given the best explanation on transformers that’ve saved me lots of time studying the original paper. Please produce more vids like this, I would recommend the BERT family and the GPT family as well 👏👍

Wow , this was great, I have watched a no of videos on the transformer models, and they have all contributed to my understanding, but this puts everything together so neatly. Amazing, please keep making more such videos.

I used multiple sources to learn about the transformer architecture. Regarding the decoder part, you really helped me understanding what was the input and how the different operations are performed ! Thanks a lot :)

This is incredible. I've been watching videos and reading papers about transformer and attention for days, this is the best material so far.

There is actually a small mistake at 12:56: The Ecnoders Output are the VALUES and keys! for the Decoders second self attention.
So it is: Value and Key from Encoder with Query from Decoder are combined.
from the "Self Attention is all you need" paper: "In "encoder-decoder attention" layers, the queries come from the previous decoder layer,
and the memory keys and values come from the output of the encoder."

I have been struggling with this architecture for an eternity now and this is the first time I really understood what's going on in this graphic. Thank you so much for this nice and clear explanation!

This video marks an end to my search for one place explanation of Transformers. Thanks a lot for putting this up! :)

Man thanks for this video, reading a paper for newbie is super difficult, but such explanations like you've posted for key, value and query as well as reasoning for masking is very, very helpful. I subscribed to your channel and am looking forward for new stuff.

This tutorial is absolute brilliant, I have to see it again and read the illustrated guide, there are so many infos!! Thank you!!!

Best explanation I've seen so far, thanks so much! 😃

This is literally the best explanation of Transformers I have ever seen!

Thanks for your explanation, very clean and well built in every argument about transformers. I was so lucky to get this video randomly on KRplus. Good job!

Brilliant explanation with visually intuitive animations ! I rarely comment or subscribe to anything but this time I instantly do both after watching the video. And how coincidental it is that this was uploaded on my birthday. Hope to see more videos from you.

This is by far the best explanation I’ve ever seen on Transformer Networks. Very very well done

12:56 encoder has hiddens state of key-value pairs, and in the decoder, the previous output is compressed into a query. The next output is produced by mapping this query and the set of keys and values.

Half of it went through my head. Just beautiful. I'll watch it many more times.. That's how I know the content is gooood.

Your videos are the BEST, I understood RNNs, LSTMs, GRUs and Transformers in less than an hour.
Thankyou.

This explanation is INCREDIBLE!!!

This is THE BEST transformer video I have encountered.

This is the best video I have seen about transformers. Very articulate and concise. Great job

Best video ever on transformers, trust me I tried others, just positional encoding is missing, but rest is gold.
Thank you.

amazing explanation, honestly this is the first time i understand how Transformers work.

This seems to be one of the best videos on Transformers

This is one of the best introductory videos I've seen on this subject. Thank you!

This illustrated explanation is just so well done :OOOOO
I'm a novice at Deep neuronal networks and just by looking at the video, I just understood everything !
Completely recommended to understand Transformers :)
Good work :D

Favourite video on the topic! I'm reasonably knowledgeable on ML, but the other 5-10 videos I've tried so far all resulted in increased confusion. This is clear.Nice one 👍🏿

I watched a second time not for better understand the video, but to appreciate it. It is very well done and pretty clear. Thank you.

You did an amazing job explaining the workflow … looked for more similar stuff… please continue … I hope you will be back to help people like me

Wow. This is some really good explanation! I don't have much NLP background except RNN/LSTM and things before DL (N-gram), but wanted to know more about Attention mechanism for robotics application. (my field) Most other explanation either skimmed over the mathematics, or used NLP specific nomenclature/concepts that made it hard to understand for non-NLP people.
This was some good stuff! Much appreciated and Keep up the good work!

That's a very good explanation imo! Thanks for taking the time to produce such a gem.

My first read about Micheal Phi was "Stop Installing Tensorflow using pip for performance sake!" in TowardDataScience blog (as I recall you was "Micheal Nguyen" at that time). My first impression was like "oh this guy was good at explanation". Then I read his several blogs, and now here I am. I never knew that you have a channel. You are one of the best educator I've ever known. Thanks so much.

A complex process- I need to listen to this multiple times to fully understand this.

Now i have two great heroes that explain complex concept using mindblowin visualization, first is 3b1b for complex math topics, then Michael Phi for complex machine learning architecture! Just wow ... salute sir! thank you so much!

This was fantastic, thanks!

Honestly this is the best explanation I've ever seen on transformers and attention

This was super helpful thank you. I read the original paper and absorbed like 70% of it but this clarified several things.

Great explanation!

Thanks, I particularly liked that you went into as much detail for the decoder as for the encoder.

Incredibly interesting. It is amazing how much processing and storage is required to achieve this.

This was beautiful.
This was the best explanation out there. You Sir, are a person of highest quality.

Amazing. I still don’t really understand how the Q K and V values are calculated but I learnt a lot more about this seminal paper than others provided - thank you! 🙏

The best explanation so far. Loved the animated illustration.

the best explanation that I've seen. 👏

This is the best explaination on transformers anywhere on the web

Thank you for this amazing explanation. It really helped after an insufficient explanation from my DL lecture. The prof did not even mention that the final part is a classification over the vocabulary for an nlp task!

Thank you so much for the step by step explanation. This is a good starting point for ML dummies like me to get a grasp on the transformer model architecture.

The explanation about Transformer architecture is clear, and the animation in presentation is really good, it catches my attention :)

Thanks a lot. This is by far the most clear explanation of the paper. Kudos. Hope you can do similar videos for say Bert, XLNET architectures as well.

The best video on this channel Michael. Do you think you can make a bunch more like this... with this visual style (white over black drawings), and clear and calm explanation of the diagrams.

This was not just the best transformer explanation, but the best explanation in general that I've ever seen. You know what to abstract for a fluid and clear explanation. Congratulations. You should do more videos like this.

That's absolute genius ! You made it so easy to understand such a mighty concept.

Nice work! Love the visuals for this abstract topic. Just found your channel. Keep em coming!!

Wow Michael, this is a superb explanation of the transformer architecture. You even went into detail about the meaning of the Q,K,V vectors and masking concepts which were hard for me to grasp. I bounced around through 3-4 videos about the transformer arch, and for each one I claimed it was the best explanation on the topic. But your video takes the cake and explains it in half the time as the others. Thank you for sharing! Also, great job on the visuals which are on par with 3blue1brown's animations.

Best explanation I've seen - thanks !

Thanks for the effort you put into making the animation on the slide.

Amazing! thank you so much - great quality of the video and content

While I still have some questions, this is a pretty good explanation, I mean I actually have an idea of how this works! Gonna watch it like 2 more times.

This is by far the best explanation that I have seen.

Simple and coherent explanations. Brilliant

Brooo you are seriously my god😭😭🙏🙏...thanks a lot for this video...no one... literally no one could teach me transformer and your video just got drilled into my mind...please make other videos like this for bert gpt xlnet xlm etc etc... I'm really thankful to you

Such a lucid explanation it is. Thanks for posting!!

Amazing explanation Michael! Thank you for your time!!!!

Incredible video! I hope you are doing well and find the time to make more, especially with the recent popularity explosion of AI.

Very Very thanks for this gem of an explanation.

This is the best explanation ever! So genius! Need more videos like this

Perfect explanation of the Transformes !!! Thanks.

This was the best video i saw on attention . Thank you so much

感谢！ Thank you so much, I've looked up all the resources on the Internet but still messed up with the mechanism. It's really a clear and detailed explanation.

Great deep dive into transformers. Helped me understand this architecture.

Wow..one of the best videos I have watched on transformers...so simple to grasp. Please make more videos.

Best video on transformers on KRplus, thank you so much

Thank you Michael. This is awesome video. Much appreciated

Best examplanation I have seen so far. Great job! You destroyed almost all my questions.

Perfect explanation of the concept, thank you!

This is the best explanation on transformers. Thank you so much for the video.

Best illustration on transformer.
Subscribed for many more to come.

Very deep explanation, brilliant talent to give somebody an intuition

A really great explanation! Thank you very much!!

thanks for this clear video, very well explained

AMAZING!!!!!THANK YOU SO MUCH!

Fantastic!
Thank you for a marvelous presentation.

hands down the best explation for transformer models !

best ever explanation i came across, you made it very useful as well as fun to watch, thanks

Your explanation of q k and v is the thing that finally did it for me, I get it!

sir u r superExplanation teacher ever

Best Transformers Explanation I have seen thank you very much, Liked the video and Subscribed !! Keep it up :))

Good explanation. What boggles my mind is that this architecture can not only produce reasonable sentences, but there can be some logic going on behind the sequence of sentences as we've seen in chatGPT. It is mind-boggling that there must be some amount of deeper conceptualization represented in the embeddings too! amazing

what an awesome explanation! saved me so much time, thanks for the great effort!

whoa bro, hats off what a nice explanation and one of the best explanation video for transformers.

One of the best explanation of attention mechanism

Great work here!! Thank you for this excellent explanation!

I must say this is the best explanation I had ever seen...no confusion no doubt left in mind.. Thanks a lot sir.. It will be helpful if you kindly do a vedio on language models like IBM GPT BERT..

I watched many videos, but I'm 100% sure that this video is only the best among those in explaining the transformers

One word excellent , none of the explanation will match this in KRplus ....stunned

In "encoder-decoder attention" layers, the queries come from the previous decoder layer, and the memory keys and values come from the output of the encoder.

wow! you explained it so clearly and really helps my understanding, thanks

This is very well explained! Thank you!

Wow, by far the best explanation. Teaching really is an art where even experts with their blind spots can be very bad at explaining (even if they know it well themselves). Well done. It's really too bad your previous video was a year ago. I really hope you make more videos explaining other deep learning papers such as Faster RCNN etc. Thanks for posting this.

Love you bro. you made my life easier!