A Top-down Approach to Feature Selection in Reinforcement Learning
published: May 28, 2013, recorded: September 2012, views: 3559
Report a problem or upload filesIf you have found a problem with this lecture or would like to send us extra material, articles, exercises, etc., please use our ticket system to describe your request and upload the data.
Enter your e-mail into the 'Cc' field, and we will keep you updated with your request's status.
Feature selection is an important problem in many areas of machine learning including reinforcement learning (RL). A possible approach to feature selection is to solve the machine learning problem in a high dimensional feature space in the hope that relevant features lie there. However, this approach may suffer from overfitting and have poor prediction performance. Two methods that have been used in regression to overcome this problem are regularization (adding l-2 and/or l-1 penalization terms to the objective function) and random projections (solving the problem in a randomly generated low dimensional space). In this talk, we study the use of these two methods in value function approximation in RL In particular, we study the widely-used least-squares temporal difference (LSTD) learning algorithm. We first provide a thorough theoretical analysis of LSTD with random projections and derive performance bounds for the resulting algorithm. We then analyze the performance of Lasso-TO, a modification of LSTD in which the projection operator is defined as a Lasso problem.
Link this pageWould you like to put a link to this lecture on your homepage?
Go ahead! Copy the HTML snippet !