{"id":6218,"date":"2022-12-01T11:44:54","date_gmt":"2022-12-01T08:44:54","guid":{"rendered":"https:\/\/www.newworldai.com\/?p=6218"},"modified":"2023-01-13T00:45:39","modified_gmt":"2023-01-12T21:45:39","slug":"illustrated-guide-lstms-grus-step-step-explanation","status":"publish","type":"post","link":"https:\/\/www.newworldai.com\/illustrated-guide-lstms-grus-step-step-explanation\/","title":{"rendered":"Illustrated Guide to LSTM\u2019s and GRU\u2019s: A step by step explanation"},"content":{"rendered":"

Hi and welcome to an Illustrated Guide to Long Short-Term Memory (LSTM) and Gated Recurrent Units (GRU). I\u2019m Michael, and I\u2019m a Machine Learning Engineer in the AI voice assistant space.<\/strong><\/h1>\n

In this post, we\u2019ll start with the intuition behind LSTM \u2019s and GRU\u2019s. Then I\u2019ll explain the internal mechanisms that allow LSTM\u2019s and GRU\u2019s to perform so well. If you want to understand what\u2019s happening under the hood for these two networks, then this post is for you.<\/span><\/p>\n

You can also watch the video version of this post on youtube if you prefer.<\/span><\/p>\n

The Problem, Short-term Memory<\/span><\/strong><\/span><\/p>\n

Recurrent Neural Networks suffer from short-term memory. If a sequence is long enough, they\u2019ll have a hard time carrying information from earlier time steps to later ones. So if you are trying to process a paragraph of text to do predictions, RNN\u2019s may leave out important information from the beginning.<\/span><\/p>\n

During back propagation, recurrent neural networks suffer from the vanishing gradient problem. Gradients are values used to update a neural networks weights. The vanishing gradient problem is when the gradient shrinks as it back propagates through time. If a gradient value becomes extremely small, it doesn\u2019t contribute too much learning.<\/span><\/p>\n

READ MORE<\/a><\/span><\/p>\n

https:\/\/towardsdatascience.com\/illustrated-guide-to-lstms-and-gru-s-a-step-by-step-explanation-44e9eb85bf21<\/a><\/span><\/p>\n","protected":false},"excerpt":{"rendered":"

Hi and welcome to an Illustrated Guide to Long Short-Term Memory (LSTM) and Gated Recurrent Units (GRU). I\u2019m Michael, and I\u2019m a Machine Learning<\/p>\n","protected":false},"author":2,"featured_media":6219,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":[],"categories":[1401],"tags":[741,252],"yoast_head":"\nIllustrated Guide to LSTM\u2019s and GRU\u2019s: A step by step explanation - New World : Artificial Intelligence<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/www.newworldai.com\/illustrated-guide-lstms-grus-step-step-explanation\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Illustrated Guide to LSTM\u2019s and GRU\u2019s: A step by step explanation - New World : Artificial Intelligence\" \/>\n<meta property=\"og:description\" content=\"Hi and welcome to an Illustrated Guide to Long Short-Term Memory (LSTM) and Gated Recurrent Units (GRU). I\u2019m Michael, and I\u2019m a Machine Learning\" \/>\n<meta property=\"og:url\" content=\"https:\/\/www.newworldai.com\/illustrated-guide-lstms-grus-step-step-explanation\/\" \/>\n<meta property=\"og:site_name\" content=\"New World : Artificial Intelligence\" \/>\n<meta property=\"article:publisher\" content=\"https:\/\/www.facebook.com\/groups\/952169468169594\/\" \/>\n<meta property=\"article:published_time\" content=\"2022-12-01T08:44:54+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2023-01-12T21:45:39+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/www.newworldai.com\/wp-content\/uploads\/1_n-IgHZM5baBUjq0T7RYDBw.gif\" \/>\n\t<meta property=\"og:image:width\" content=\"960\" \/>\n\t<meta property=\"og:image:height\" content=\"540\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/gif\" \/>\n<meta name=\"author\" content=\"mussem\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:creator\" content=\"@NewWorldAI1\" \/>\n<meta name=\"twitter:site\" content=\"@NewWorldAI1\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"mussem\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"1 minute\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"WebPage\",\"@id\":\"https:\/\/www.newworldai.com\/illustrated-guide-lstms-grus-step-step-explanation\/\",\"url\":\"https:\/\/www.newworldai.com\/illustrated-guide-lstms-grus-step-step-explanation\/\",\"name\":\"Illustrated Guide to LSTM\u2019s and GRU\u2019s: A step by step explanation - New World : Artificial Intelligence\",\"isPartOf\":{\"@id\":\"https:\/\/www.newworldai.com\/#website\"},\"datePublished\":\"2022-12-01T08:44:54+00:00\",\"dateModified\":\"2023-01-12T21:45:39+00:00\",\"author\":{\"@id\":\"https:\/\/www.newworldai.com\/#\/schema\/person\/9d8dd21ba4efacdbf5a5a7700ac3ec1f\"},\"breadcrumb\":{\"@id\":\"https:\/\/www.newworldai.com\/illustrated-guide-lstms-grus-step-step-explanation\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/www.newworldai.com\/illustrated-guide-lstms-grus-step-step-explanation\/\"]}]},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/www.newworldai.com\/illustrated-guide-lstms-grus-step-step-explanation\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/www.newworldai.com\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Illustrated Guide to LSTM\u2019s and GRU\u2019s: A step by step explanation\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/www.newworldai.com\/#website\",\"url\":\"https:\/\/www.newworldai.com\/\",\"name\":\"New World : Artificial Intelligence\",\"description\":\"Artificial Intelligence, Deep Learning, Machine Learning, AI Lectures, AI Conferences, AI TED Talks, AI Movies, AI Books\",\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/www.newworldai.com\/?s={search_term_string}\"},\"query-input\":\"required name=search_term_string\"}],\"inLanguage\":\"en-US\"},{\"@type\":\"Person\",\"@id\":\"https:\/\/www.newworldai.com\/#\/schema\/person\/9d8dd21ba4efacdbf5a5a7700ac3ec1f\",\"name\":\"mussem\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/www.newworldai.com\/#\/schema\/person\/image\/\",\"url\":\"https:\/\/secure.gravatar.com\/avatar\/b2359d117d746a042c3d719211039fe2?s=96&d=mm&r=g\",\"contentUrl\":\"https:\/\/secure.gravatar.com\/avatar\/b2359d117d746a042c3d719211039fe2?s=96&d=mm&r=g\",\"caption\":\"mussem\"}}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Illustrated Guide to LSTM\u2019s and GRU\u2019s: A step by step explanation - New World : Artificial Intelligence","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/www.newworldai.com\/illustrated-guide-lstms-grus-step-step-explanation\/","og_locale":"en_US","og_type":"article","og_title":"Illustrated Guide to LSTM\u2019s and GRU\u2019s: A step by step explanation - New World : Artificial Intelligence","og_description":"Hi and welcome to an Illustrated Guide to Long Short-Term Memory (LSTM) and Gated Recurrent Units (GRU). I\u2019m Michael, and I\u2019m a Machine Learning","og_url":"https:\/\/www.newworldai.com\/illustrated-guide-lstms-grus-step-step-explanation\/","og_site_name":"New World : Artificial Intelligence","article_publisher":"https:\/\/www.facebook.com\/groups\/952169468169594\/","article_published_time":"2022-12-01T08:44:54+00:00","article_modified_time":"2023-01-12T21:45:39+00:00","og_image":[{"width":960,"height":540,"url":"https:\/\/www.newworldai.com\/wp-content\/uploads\/1_n-IgHZM5baBUjq0T7RYDBw.gif","type":"image\/gif"}],"author":"mussem","twitter_card":"summary_large_image","twitter_creator":"@NewWorldAI1","twitter_site":"@NewWorldAI1","twitter_misc":{"Written by":"mussem","Est. reading time":"1 minute"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"WebPage","@id":"https:\/\/www.newworldai.com\/illustrated-guide-lstms-grus-step-step-explanation\/","url":"https:\/\/www.newworldai.com\/illustrated-guide-lstms-grus-step-step-explanation\/","name":"Illustrated Guide to LSTM\u2019s and GRU\u2019s: A step by step explanation - New World : Artificial Intelligence","isPartOf":{"@id":"https:\/\/www.newworldai.com\/#website"},"datePublished":"2022-12-01T08:44:54+00:00","dateModified":"2023-01-12T21:45:39+00:00","author":{"@id":"https:\/\/www.newworldai.com\/#\/schema\/person\/9d8dd21ba4efacdbf5a5a7700ac3ec1f"},"breadcrumb":{"@id":"https:\/\/www.newworldai.com\/illustrated-guide-lstms-grus-step-step-explanation\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/www.newworldai.com\/illustrated-guide-lstms-grus-step-step-explanation\/"]}]},{"@type":"BreadcrumbList","@id":"https:\/\/www.newworldai.com\/illustrated-guide-lstms-grus-step-step-explanation\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/www.newworldai.com\/"},{"@type":"ListItem","position":2,"name":"Illustrated Guide to LSTM\u2019s and GRU\u2019s: A step by step explanation"}]},{"@type":"WebSite","@id":"https:\/\/www.newworldai.com\/#website","url":"https:\/\/www.newworldai.com\/","name":"New World : Artificial Intelligence","description":"Artificial Intelligence, Deep Learning, Machine Learning, AI Lectures, AI Conferences, AI TED Talks, AI Movies, AI Books","potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/www.newworldai.com\/?s={search_term_string}"},"query-input":"required name=search_term_string"}],"inLanguage":"en-US"},{"@type":"Person","@id":"https:\/\/www.newworldai.com\/#\/schema\/person\/9d8dd21ba4efacdbf5a5a7700ac3ec1f","name":"mussem","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.newworldai.com\/#\/schema\/person\/image\/","url":"https:\/\/secure.gravatar.com\/avatar\/b2359d117d746a042c3d719211039fe2?s=96&d=mm&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/b2359d117d746a042c3d719211039fe2?s=96&d=mm&r=g","caption":"mussem"}}]}},"_links":{"self":[{"href":"https:\/\/www.newworldai.com\/wp-json\/wp\/v2\/posts\/6218"}],"collection":[{"href":"https:\/\/www.newworldai.com\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.newworldai.com\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.newworldai.com\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/www.newworldai.com\/wp-json\/wp\/v2\/comments?post=6218"}],"version-history":[{"count":5,"href":"https:\/\/www.newworldai.com\/wp-json\/wp\/v2\/posts\/6218\/revisions"}],"predecessor-version":[{"id":6224,"href":"https:\/\/www.newworldai.com\/wp-json\/wp\/v2\/posts\/6218\/revisions\/6224"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.newworldai.com\/wp-json\/wp\/v2\/media\/6219"}],"wp:attachment":[{"href":"https:\/\/www.newworldai.com\/wp-json\/wp\/v2\/media?parent=6218"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.newworldai.com\/wp-json\/wp\/v2\/categories?post=6218"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.newworldai.com\/wp-json\/wp\/v2\/tags?post=6218"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}