{"id":5612,"date":"2020-05-23T00:00:22","date_gmt":"2020-05-22T21:00:22","guid":{"rendered":"https:\/\/www.newworldai.com\/?p=5612"},"modified":"2021-08-03T15:34:59","modified_gmt":"2021-08-03T12:34:59","slug":"system-1-deep-learning-system-2-deep-learning-yoshua-bengio","status":"publish","type":"post","link":"https:\/\/www.newworldai.com\/system-1-deep-learning-system-2-deep-learning-yoshua-bengio\/","title":{"rendered":"From System 1 Deep Learning to System 2 Deep Learning | Yoshua Bengio"},"content":{"rendered":"<h1 class=\"title style-scope ytd-video-primary-info-renderer\">Yoshua Bengio: From System 1 Deep Learning to System 2 Deep Learning (NeurIPS 2019)<\/h1>\n<p style=\"text-align: justify;\"><span style=\"font-family: helvetica, arial, sans-serif; font-size: 12pt;\">The purpose of the Neural Information Processing Systems&#8217; annual meeting is to foster the exchange of research on neural information processing systems in their biological, technological, mathematical, and theoretical aspects. The core focus is peer-reviewed novel research which is presented and discussed in the general session, along with invited talks by leaders in their field.<\/span><\/p>\n<p><iframe loading=\"lazy\" src=\"https:\/\/www.youtube.com\/embed\/T3sxeTgT4qc\" width=\"760\" height=\"315\" frameborder=\"0\" allowfullscreen=\"allowfullscreen\"><span data-mce-type=\"bookmark\" style=\"display: inline-block; width: 0px; overflow: hidden; line-height: 0;\" class=\"mce_SELRES_start\">\ufeff<\/span><\/iframe><\/p>\n<p>NeurIPS&#8217;2019 Posner Lecture<br \/>\nDecember 11th, 2019, Vancouver BC<\/p>\n<p style=\"text-align: justify;\"><span style=\"font-family: helvetica, arial, sans-serif; font-size: 12pt;\">Past progress in deep learning has concentrated mostly on learning from a static dataset, mostly for perception tasks and other System 1 tasks which are done intuitively and unconsciously by humans. However, in recent years, a shift in research direction and new tools such as soft-attention and progress in deep reinforcement learning are opening the door to the development of novel deep architectures and training frameworks for addressing System 2 tasks (which are done consciously), such as reasoning, planning, capturing causality and obtaining systematic generalization in natural language processing and other applications. Such an expansion of deep learning from System 1 tasks to System 2 tasks is important to achieve the old deep learning goal of discovering high-level abstract representations because we argue that System 2 requirements will put pressure on representation learning to discover the kind of high-level concepts which humans manipulate with language. <\/span><\/p>\n<p style=\"text-align: justify;\"><span style=\"font-family: helvetica, arial, sans-serif; font-size: 12pt;\">We argue that towards this objective, soft attention mechanisms constitute a key ingredient to focus computation on a few concepts at a time (a &#8220;conscious thought&#8221;) as per the consciousness prior and its associated assumption that many high-level dependencies can be approximately captured by a sparse factor graph. We also argue how the agent perspective in deep learning can help put more constraints on the learned representations to capture affordances, causal variables, and model transitions in the environment. Finally, we propose that meta-learning, the modularization aspect of the consciousness prior and the agent perspective on representation learning should facilitate re-use of learned components in novel ways (even if statistically improbable, as in counterfactuals), enabling more powerful forms of compositional generalization, i.e., out-of-distribution generalization based on the hypothesis of localized (in time, space, and concept space) changes in the environment due to interventions of agents.<\/span><\/p>\n<p><span style=\"font-size: 14pt;\"><strong><a href=\"https:\/\/slideslive.com\/38921750\/from-system-1-deep-learning-to-system-2-deep-learning\">Slides<\/a><\/strong><\/span><\/p>\n<p><strong>Source:<\/strong>\u00a0<a class=\"yt-simple-endpoint style-scope ytd-video-owner-renderer\" tabindex=\"-1\" href=\"https:\/\/www.youtube.com\/channel\/UCJIfeSCssxSC_Dhc5s7woww\"><img decoding=\"async\" id=\"img\" class=\"style-scope yt-img-shadow\" src=\"https:\/\/yt3.ggpht.com\/a\/AGF-l78r2Vv5Rj4X31A51yp4qFDiVOnAQ1u4v37NYA=s48-c-k-c0xffffffff-no-rj-mo\" alt=\"\" width=\"48\" \/><\/a><\/p>\n<div id=\"upload-info\" class=\"style-scope ytd-video-owner-renderer\">\n<div id=\"container\" class=\"style-scope ytd-channel-name\">\n<div id=\"text-container\" class=\"style-scope ytd-channel-name\"><em><strong><a class=\"yt-simple-endpoint style-scope yt-formatted-string\" spellcheck=\"false\" href=\"https:\/\/www.youtube.com\/channel\/UCJIfeSCssxSC_Dhc5s7woww\">Lex Clips<\/a><\/strong><\/em><\/div>\n<\/div>\n<\/div>\n","protected":false},"excerpt":{"rendered":"<p>Yoshua Bengio: From System 1 Deep Learning to System 2 Deep Learning (NeurIPS 2019) The purpose of the Neural Information Processing Systems&#8217; annual meeting<\/p>\n","protected":false},"author":2,"featured_media":5613,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":[],"categories":[1476,3],"tags":[18,162],"yoast_head":"<!-- This site is optimized with the Yoast SEO plugin v20.2.1 - https:\/\/yoast.com\/wordpress\/plugins\/seo\/ -->\n<title>From System 1 Deep Learning to System 2 Deep Learning | Yoshua Bengio - New World : Artificial Intelligence<\/title>\n<meta name=\"description\" content=\"Yoshua Bengio: From System 1 Deep Learning to System 2 Deep Learning (NeurIPS 2019)\" \/>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/www.newworldai.com\/system-1-deep-learning-system-2-deep-learning-yoshua-bengio\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"From System 1 Deep Learning to System 2 Deep Learning | Yoshua Bengio - New World : Artificial Intelligence\" \/>\n<meta property=\"og:description\" content=\"Yoshua Bengio: From System 1 Deep Learning to System 2 Deep Learning (NeurIPS 2019)\" \/>\n<meta property=\"og:url\" content=\"https:\/\/www.newworldai.com\/system-1-deep-learning-system-2-deep-learning-yoshua-bengio\/\" \/>\n<meta property=\"og:site_name\" content=\"New World : Artificial Intelligence\" \/>\n<meta property=\"article:publisher\" content=\"https:\/\/www.facebook.com\/groups\/952169468169594\/\" \/>\n<meta property=\"article:published_time\" content=\"2020-05-22T21:00:22+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2021-08-03T12:34:59+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/www.newworldai.com\/wp-content\/uploads\/Yoshua.jpg\" \/>\n\t<meta property=\"og:image:width\" content=\"1280\" \/>\n\t<meta property=\"og:image:height\" content=\"720\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/jpeg\" \/>\n<meta name=\"author\" content=\"mussem\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:creator\" content=\"@NewWorldAI1\" \/>\n<meta name=\"twitter:site\" content=\"@NewWorldAI1\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"mussem\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"2 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"WebPage\",\"@id\":\"https:\/\/www.newworldai.com\/system-1-deep-learning-system-2-deep-learning-yoshua-bengio\/\",\"url\":\"https:\/\/www.newworldai.com\/system-1-deep-learning-system-2-deep-learning-yoshua-bengio\/\",\"name\":\"From System 1 Deep Learning to System 2 Deep Learning | Yoshua Bengio - New World : Artificial Intelligence\",\"isPartOf\":{\"@id\":\"https:\/\/www.newworldai.com\/#website\"},\"datePublished\":\"2020-05-22T21:00:22+00:00\",\"dateModified\":\"2021-08-03T12:34:59+00:00\",\"author\":{\"@id\":\"https:\/\/www.newworldai.com\/#\/schema\/person\/9d8dd21ba4efacdbf5a5a7700ac3ec1f\"},\"description\":\"Yoshua Bengio: From System 1 Deep Learning to System 2 Deep Learning (NeurIPS 2019)\",\"breadcrumb\":{\"@id\":\"https:\/\/www.newworldai.com\/system-1-deep-learning-system-2-deep-learning-yoshua-bengio\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/www.newworldai.com\/system-1-deep-learning-system-2-deep-learning-yoshua-bengio\/\"]}]},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/www.newworldai.com\/system-1-deep-learning-system-2-deep-learning-yoshua-bengio\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/www.newworldai.com\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"From System 1 Deep Learning to System 2 Deep Learning | Yoshua Bengio\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/www.newworldai.com\/#website\",\"url\":\"https:\/\/www.newworldai.com\/\",\"name\":\"New World : Artificial Intelligence\",\"description\":\"Artificial Intelligence, Deep Learning, Machine Learning, AI Lectures, AI Conferences, AI TED Talks, AI Movies, AI Books\",\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/www.newworldai.com\/?s={search_term_string}\"},\"query-input\":\"required name=search_term_string\"}],\"inLanguage\":\"en-US\"},{\"@type\":\"Person\",\"@id\":\"https:\/\/www.newworldai.com\/#\/schema\/person\/9d8dd21ba4efacdbf5a5a7700ac3ec1f\",\"name\":\"mussem\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/www.newworldai.com\/#\/schema\/person\/image\/\",\"url\":\"https:\/\/secure.gravatar.com\/avatar\/b2359d117d746a042c3d719211039fe2?s=96&d=mm&r=g\",\"contentUrl\":\"https:\/\/secure.gravatar.com\/avatar\/b2359d117d746a042c3d719211039fe2?s=96&d=mm&r=g\",\"caption\":\"mussem\"}}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"From System 1 Deep Learning to System 2 Deep Learning | Yoshua Bengio - New World : Artificial Intelligence","description":"Yoshua Bengio: From System 1 Deep Learning to System 2 Deep Learning (NeurIPS 2019)","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/www.newworldai.com\/system-1-deep-learning-system-2-deep-learning-yoshua-bengio\/","og_locale":"en_US","og_type":"article","og_title":"From System 1 Deep Learning to System 2 Deep Learning | Yoshua Bengio - New World : Artificial Intelligence","og_description":"Yoshua Bengio: From System 1 Deep Learning to System 2 Deep Learning (NeurIPS 2019)","og_url":"https:\/\/www.newworldai.com\/system-1-deep-learning-system-2-deep-learning-yoshua-bengio\/","og_site_name":"New World : Artificial Intelligence","article_publisher":"https:\/\/www.facebook.com\/groups\/952169468169594\/","article_published_time":"2020-05-22T21:00:22+00:00","article_modified_time":"2021-08-03T12:34:59+00:00","og_image":[{"width":1280,"height":720,"url":"https:\/\/www.newworldai.com\/wp-content\/uploads\/Yoshua.jpg","type":"image\/jpeg"}],"author":"mussem","twitter_card":"summary_large_image","twitter_creator":"@NewWorldAI1","twitter_site":"@NewWorldAI1","twitter_misc":{"Written by":"mussem","Est. reading time":"2 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"WebPage","@id":"https:\/\/www.newworldai.com\/system-1-deep-learning-system-2-deep-learning-yoshua-bengio\/","url":"https:\/\/www.newworldai.com\/system-1-deep-learning-system-2-deep-learning-yoshua-bengio\/","name":"From System 1 Deep Learning to System 2 Deep Learning | Yoshua Bengio - New World : Artificial Intelligence","isPartOf":{"@id":"https:\/\/www.newworldai.com\/#website"},"datePublished":"2020-05-22T21:00:22+00:00","dateModified":"2021-08-03T12:34:59+00:00","author":{"@id":"https:\/\/www.newworldai.com\/#\/schema\/person\/9d8dd21ba4efacdbf5a5a7700ac3ec1f"},"description":"Yoshua Bengio: From System 1 Deep Learning to System 2 Deep Learning (NeurIPS 2019)","breadcrumb":{"@id":"https:\/\/www.newworldai.com\/system-1-deep-learning-system-2-deep-learning-yoshua-bengio\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/www.newworldai.com\/system-1-deep-learning-system-2-deep-learning-yoshua-bengio\/"]}]},{"@type":"BreadcrumbList","@id":"https:\/\/www.newworldai.com\/system-1-deep-learning-system-2-deep-learning-yoshua-bengio\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/www.newworldai.com\/"},{"@type":"ListItem","position":2,"name":"From System 1 Deep Learning to System 2 Deep Learning | Yoshua Bengio"}]},{"@type":"WebSite","@id":"https:\/\/www.newworldai.com\/#website","url":"https:\/\/www.newworldai.com\/","name":"New World : Artificial Intelligence","description":"Artificial Intelligence, Deep Learning, Machine Learning, AI Lectures, AI Conferences, AI TED Talks, AI Movies, AI Books","potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/www.newworldai.com\/?s={search_term_string}"},"query-input":"required name=search_term_string"}],"inLanguage":"en-US"},{"@type":"Person","@id":"https:\/\/www.newworldai.com\/#\/schema\/person\/9d8dd21ba4efacdbf5a5a7700ac3ec1f","name":"mussem","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.newworldai.com\/#\/schema\/person\/image\/","url":"https:\/\/secure.gravatar.com\/avatar\/b2359d117d746a042c3d719211039fe2?s=96&d=mm&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/b2359d117d746a042c3d719211039fe2?s=96&d=mm&r=g","caption":"mussem"}}]}},"_links":{"self":[{"href":"https:\/\/www.newworldai.com\/wp-json\/wp\/v2\/posts\/5612"}],"collection":[{"href":"https:\/\/www.newworldai.com\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.newworldai.com\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.newworldai.com\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/www.newworldai.com\/wp-json\/wp\/v2\/comments?post=5612"}],"version-history":[{"count":3,"href":"https:\/\/www.newworldai.com\/wp-json\/wp\/v2\/posts\/5612\/revisions"}],"predecessor-version":[{"id":5615,"href":"https:\/\/www.newworldai.com\/wp-json\/wp\/v2\/posts\/5612\/revisions\/5615"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.newworldai.com\/wp-json\/wp\/v2\/media\/5613"}],"wp:attachment":[{"href":"https:\/\/www.newworldai.com\/wp-json\/wp\/v2\/media?parent=5612"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.newworldai.com\/wp-json\/wp\/v2\/categories?post=5612"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.newworldai.com\/wp-json\/wp\/v2\/tags?post=5612"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}