Learned graphical models for probabilistic planning provide a new class of movement primitives

Elmar A. Rückert; Gerhard Neumann; Marc Toussaint; Wolfgang Maass

doi:10.3389/fncom.2012.00097

Learned graphical models for probabilistic planning provide a new class of movement primitives

Elmar A. Rückert, Gerhard Neumann, Marc Toussaint, Wolfgang Maass

Research output: Contribution to journal › Article › Research › peer-review

Abstract

Biologicalmovementgenerationcombinesthreeinterestingaspects:itsmodularorganizationinmovementprimitives(MPs),itscharacteristicsofstochasticoptimalityunderperturbations,anditsefficiencyintermsoflearning.Acommonapproachtomotorskilllearningistoendowtheprimitiveswithdynamicalsystems.Here,theparametersoftheprimitiveindirectlydefinetheshapeofareferencetrajectory.WeproposeanalternativeMPrepresentationbasedonprobabilisticinferenceinlearnedgraphicalmodelswithnewandinterestingpropertiesthatcomplieswithsalientfeaturesofbiologicalmovementcontrol.Insteadofendowingtheprimitiveswithdynamicalsystems,weproposetoendowMPswithanintrinsicprobabilisticplanningsystem,integratingthepowerofstochasticoptimalcontrol(SOC)methodswithinaMP.Theparameterizationoftheprimitiveisagraphicalmodelthatrepresentsthedynamicsandintrinsiccostfunctionsuchthatinferenceinthisgraphicalmodelyieldsthecontrolpolicy.Weparameterizetheintrinsiccostfunctionusingtask-relevantfeatures,suchastheimportanceofpassingthroughcertainvia-points.Thesystemdynamicsaswellasintrinsiccostfunctionparametersarelearnedinareinforcementlearning(RL)setting.Weevaluateourapproachonacomplex4-linkbalancingtask.Ourexperimentsshowthatourmovementrepresentationfacilitateslearningsignificantlyandleadstobettergeneralizationtonewtasksettingswithoutre-learning.

Original language	English
Article number	97
Number of pages	20
Journal	Frontiers in computational neuroscience
Volume	6.2013
Issue number	January
DOIs	https://doi.org/10.3389/fncom.2012.00097
Publication status	E-pub ahead of print - 2 Jan 2013
Externally published	Yes

Access to Document

10.3389/fncom.2012.00097

Cite this

@article{d5a1aa1170fd4c008780a70a780a1b2b,

title = "Learned graphical models for probabilistic planning provide a new class of movement primitives",

abstract = "Biologicalmovementgenerationcombinesthreeinterestingaspects:itsmodularorganizationinmovementprimitives(MPs),itscharacteristicsofstochasticoptimalityunderperturbations,anditsefficiencyintermsoflearning.Acommonapproachtomotorskilllearningistoendowtheprimitiveswithdynamicalsystems.Here,theparametersoftheprimitiveindirectlydefinetheshapeofareferencetrajectory.WeproposeanalternativeMPrepresentationbasedonprobabilisticinferenceinlearnedgraphicalmodelswithnewandinterestingpropertiesthatcomplieswithsalientfeaturesofbiologicalmovementcontrol.Insteadofendowingtheprimitiveswithdynamicalsystems,weproposetoendowMPswithanintrinsicprobabilisticplanningsystem,integratingthepowerofstochasticoptimalcontrol(SOC)methodswithinaMP.Theparameterizationoftheprimitiveisagraphicalmodelthatrepresentsthedynamicsandintrinsiccostfunctionsuchthatinferenceinthisgraphicalmodelyieldsthecontrolpolicy.Weparameterizetheintrinsiccostfunctionusingtask-relevantfeatures,suchastheimportanceofpassingthroughcertainvia-points.Thesystemdynamicsaswellasintrinsiccostfunctionparametersarelearnedinareinforcementlearning(RL)setting.Weevaluateourapproachonacomplex4-linkbalancingtask.Ourexperimentsshowthatourmovementrepresentationfacilitateslearningsignificantlyandleadstobettergeneralizationtonewtasksettingswithoutre-learning.",

author = "R{\"u}ckert, {Elmar A.} and Gerhard Neumann and Marc Toussaint and Wolfgang Maass",

year = "2013",

month = jan,

day = "2",

doi = "10.3389/fncom.2012.00097",

language = "English",

volume = "6.2013",

journal = "Frontiers in computational neuroscience",

issn = "1662-5188",

publisher = "Frontiers Media SA",

number = "January",

}

TY - JOUR

T1 - Learned graphical models for probabilistic planning provide a new class of movement primitives

AU - Rückert, Elmar A.

AU - Neumann, Gerhard

AU - Toussaint, Marc

AU - Maass, Wolfgang

PY - 2013/1/2

Y1 - 2013/1/2

N2 - Biologicalmovementgenerationcombinesthreeinterestingaspects:itsmodularorganizationinmovementprimitives(MPs),itscharacteristicsofstochasticoptimalityunderperturbations,anditsefficiencyintermsoflearning.Acommonapproachtomotorskilllearningistoendowtheprimitiveswithdynamicalsystems.Here,theparametersoftheprimitiveindirectlydefinetheshapeofareferencetrajectory.WeproposeanalternativeMPrepresentationbasedonprobabilisticinferenceinlearnedgraphicalmodelswithnewandinterestingpropertiesthatcomplieswithsalientfeaturesofbiologicalmovementcontrol.Insteadofendowingtheprimitiveswithdynamicalsystems,weproposetoendowMPswithanintrinsicprobabilisticplanningsystem,integratingthepowerofstochasticoptimalcontrol(SOC)methodswithinaMP.Theparameterizationoftheprimitiveisagraphicalmodelthatrepresentsthedynamicsandintrinsiccostfunctionsuchthatinferenceinthisgraphicalmodelyieldsthecontrolpolicy.Weparameterizetheintrinsiccostfunctionusingtask-relevantfeatures,suchastheimportanceofpassingthroughcertainvia-points.Thesystemdynamicsaswellasintrinsiccostfunctionparametersarelearnedinareinforcementlearning(RL)setting.Weevaluateourapproachonacomplex4-linkbalancingtask.Ourexperimentsshowthatourmovementrepresentationfacilitateslearningsignificantlyandleadstobettergeneralizationtonewtasksettingswithoutre-learning.

AB - Biologicalmovementgenerationcombinesthreeinterestingaspects:itsmodularorganizationinmovementprimitives(MPs),itscharacteristicsofstochasticoptimalityunderperturbations,anditsefficiencyintermsoflearning.Acommonapproachtomotorskilllearningistoendowtheprimitiveswithdynamicalsystems.Here,theparametersoftheprimitiveindirectlydefinetheshapeofareferencetrajectory.WeproposeanalternativeMPrepresentationbasedonprobabilisticinferenceinlearnedgraphicalmodelswithnewandinterestingpropertiesthatcomplieswithsalientfeaturesofbiologicalmovementcontrol.Insteadofendowingtheprimitiveswithdynamicalsystems,weproposetoendowMPswithanintrinsicprobabilisticplanningsystem,integratingthepowerofstochasticoptimalcontrol(SOC)methodswithinaMP.Theparameterizationoftheprimitiveisagraphicalmodelthatrepresentsthedynamicsandintrinsiccostfunctionsuchthatinferenceinthisgraphicalmodelyieldsthecontrolpolicy.Weparameterizetheintrinsiccostfunctionusingtask-relevantfeatures,suchastheimportanceofpassingthroughcertainvia-points.Thesystemdynamicsaswellasintrinsiccostfunctionparametersarelearnedinareinforcementlearning(RL)setting.Weevaluateourapproachonacomplex4-linkbalancingtask.Ourexperimentsshowthatourmovementrepresentationfacilitateslearningsignificantlyandleadstobettergeneralizationtonewtasksettingswithoutre-learning.

U2 - 10.3389/fncom.2012.00097

DO - 10.3389/fncom.2012.00097

M3 - Article

SN - 1662-5188

VL - 6.2013

JO - Frontiers in computational neuroscience

JF - Frontiers in computational neuroscience

IS - January

M1 - 97

ER -