Reinforcement Learning Group Fall 2017
This post is an archive of the hutch_research RL17 workshop page.
Reinforcement Learning is a paradigm of machine learning that is not fully addressed in the curriculum at Western. It is covered only briefly in the AI courses, and can be a difficult topic to break into through independent study (although this is changing). In spite of this, RL is an important aspect of machine learning that allows us to approach problems which are inaccessible to the existing supervised and unsupervised learning techniques. Namely, problems that involve or require complex decision making, control, or more general intelligence.
Schedule
Each workshop will span 1 to 1.5 hours on a selected Wednesday nights of Fall quarter 2017.
Note that the concepts from these workshops largely build off of each other, with the first one being the most substantial. Full slides will be posted for people that miss it, questions can be submitted in the discussion form at the bottom of the page.
The tentative schedule is,
Oct 11 - Reinforcement Learning
Overview
Reinforcement Learning is a paradigm of machine learning where an agent learns by interacting with some environment or a statistical model of the environment. Instead of ground truth labels (supervised learning), or patterns in input data (unsupervised learning), a reward signal is used to steer the optimization. The goal of RL is to learn a policy for executing actions that maximize long term rewards within the environment. By passing through the state space multiple times over, the agent can iteratively update a decision function towards an optima that satisfies this property.
The first workshop in this series introduces the Reinforcement Learning paradigm through selected topics (policy & value functions, Markov decision process, value iteration). We will later touch on some of the popular algorithms used (TD-learning, Q-learning, SARSA, DQN).
Note: This is a lecture/discussion format to get us thinking about the theory behind RL. We will not implement any code. Reinforcement Learning is often taught as an entire class at university, this workshop exposes what I believe to be the most immediately useful concepts to build off of (specifically; model free value iteration) within the context of deep learning.
Sample Slides
Q Learning
Intro
Q-learning is a model-free reinforcement learning technique. We will use the Q function to assign value to state action pairs. Our policy will be to choose the action that maximizes cumulative expected rewards. This method can be used to find an optimal action-selection policy for any (finite) Markov decision process (MDP).
Deep Q Networks (DQN)
Intro
This workshop will include the implementation of the popular DQN algorithm to play Atari games.
Paper: Human-level control through deep reinforcement learning