Confidence-Based Progress-Driven Self-Generated Goals for Skill Acquisition in Developmental Robots

Ngo Quoc, Hung and Luciw, Matthew and Foerster, Alexander Uwe and Schmidhuber, Juergen (2013) Confidence-Based Progress-Driven Self-Generated Goals for Skill Acquisition in Developmental Robots. Frontiers in Psychology, 4 (833). ISSN 1664-1078

Text fip13-ngo.pdf Download (3MB)

Abstract

A reinforcement learning agent that autonomously explores its environment can utilize a curiosity drive to enable continual learning of skills, in the absence of any external rewards. We formulate curiosity-driven exploration, and eventual skill acquisition, as a selective sampling problem. Each environment setting provides the agent with a stream of instances. An instance is a sensory observation that, when queried, causes an outcome that the agent is trying to predict. After an instance is observed, a query condition, derived herein, tells whether its outcome is statistically known or unknown to the agent, based on the confidence interval of an online linear classifier. Upon encountering the first unknown instance, the agent “queries” the environment to observe the outcome, which is expected to improve its confidence in the corresponding predictor. If the environment is in a setting where all instances are known, the agent generates a plan of actions to reach a new setting, where an unknown instance is likely to be encountered. The desired setting is a self-generated goal, and the plan of action, essentially a program to solve a problem, is a skill. The success of the plan depends on the quality of the agent’s predictors, which are improved as mentioned above. For validation, this method is applied to both a simulated and real Katana robot arm in its “blocks-world” environment. Results show that the proposed method generates sample-efficient curious exploration behavior, which exhibits developmental stages, continual learning, and skill acquisition, in an intrinsically-motivated playful agent.

Item Type:	Scientific journal article, Newspaper article or Magazine article
Uncontrolled Keywords:	intrinsic motivation, artificial curiosity, continual learning, developmental robotics, online active learning, markov decision processes, AI planning, systematic explora
Subjects:	Computer sciences > Artificial intelligence > Artificial intelligence not elsewhere classified
Depositing User:	Matthew Luciw
Date Deposited:	11 Feb 2014 13:18
Last Modified:	23 May 2016 08:56
URI:	http://repository.supsi.ch/id/eprint/3702

Actions (login required)

View Item