Deep Learning Summer School, Montreal 2016

Recurrent Neural Networks

author: Yoshua Bengio, Department of Computer Science and Operations Research, University of Montreal
published: Aug. 23, 2016, recorded: August 2016, views: 42666

Slides

Report a problem or upload files

If you have found a problem with this lecture or would like to send us extra material, articles, exercises, etc., please use our ticket system to describe your request and upload the data.
Enter your e-mail into the 'Cc' field, and we will keep you updated with your request's status.

Lecture popularity: You need to login to cast your vote.

Description

This lecture will cover recurrent neural networks, the key ingredient in the deep learning toolbox for handling sequential computation and modelling sequences. It will start by explaining how gradients can be computed (by considering the time-unfolded graph) and how different architectures can be designed to summarize a sequence, generate a sequence by ancestral sampling in a fully-observed directed model, or learn to map a vector to a sequence, a sequence to a sequence (of the same or different length) or a sequence to a vector. The issue of long-term dependencies, why it arises, and what has been proposed to alleviate it will be core subject of the discussion in this lecture. This includes changes in the architecture and initialization, as well as how to properly characterize the architecture in terms of recurrent or feedforward depth and its ability to create shortcuts or fast propagation of gradients in the unfolded graph. Open questions regarding the limitations of training by maximum likelihood (teacher forcing) and ideas towards towards making learning online (not requiring backprop through time) will also be discussed.

Link this page

Would you like to put a link to this lecture on your homepage?
Go ahead! Copy the HTML snippet !

Reviews and comments:

1 Kamal Ali, August 28, 2016 at 1:20 a.m.:

Prof Bengio,
next year please ask Audiovisual people to put the camera on all the great stuff you have on your slides rather than slavishly videotaping you! I would have loved to have seen the actual slides. Or maybe a double , where you are in an inset and the slides are also visible.
Kamal.

2 Amelia Jessi, January 17, 2020 at 6:52 p.m.:

Using best techniques to deliver your thinking to other people and that is video. Now a days almost every company or individuals doing business with the help of videos. I am doing marketing work for https://rpseitzancientcoins.com/ and know the importance of video for any business. There are so many techniques are available for marketing and so many other resources to enhance business.

3 Ellie Grayson, July 1, 2020 at 8:47 p.m.:

At Saints 2 Clean, we have professional staff working to assist you through all your cleaning needs. https://saints2clean.com/ We make sure to provide you with services which can add value to the clients as well as their families.

4 Hares Werev, December 14, 2020 at 11:22 p.m.:

Farewell, individuals and teenagers. As of now I need to reveal to you whether you are as of now reasonably pulled in with sports, by then you need an instrument that can without a totally astounding stretch assistance you with managing the issue with back hopelessness or torment in the impacts. You can moreover utilize it Cannabidiol oil https://deeptop10.com/ all specialists support this to treat a resting issue

Write your own review or comment:

Comment:
Name:
Email address:
URL:

make sure you have javascript enabled or clear this field: