How people initiate energy optimization and converge on their optimal gaits

Jessica C. Selinger; Jeremy D. Wong; Surabhi N. Simha; J. Maxwell Donelan

doi:10.1242/jeb.198234

journal article Jan 01, 2019

How people initiate energy optimization and converge on their optimal gaits

Jessica C. Selinger

Jeremy D. Wong Surabhi N. Simha

J. Maxwell Donelan

Journal of Experimental Biology · The Company of Biologists

View at Publisher Save 10.1242/jeb.198234

Abstract

A central principle in motor control is that the coordination strategies learned by our nervous system are often optimal. Here we combined human experiments with computational reinforcement learning models to study how the nervous system navigates possible movements to arrive at an optimal coordination. Our experiments used robotic exoskeletons to reshape the relationship between how participants walk and how much energy they consume. We found that while some participants used their relatively high natural gait variability to explore the new energetic landscape and spontaneously initiate energy optimization, most participants preferred to exploit their originally preferred, but now suboptimal, gait. We could nevertheless reliably initiate optimization in these exploiters by providing them with the experience of lower cost gaits suggesting that the nervous system benefits from cues about the relevant dimensions along which to re-optimize its coordination. Once optimization was initiated, we found that the nervous system employed a local search process to converge on the new optimum gait over tens of seconds. Once optimization was completed, the nervous system learned to predict this new optimal gait and rapidly returned to it within a few steps if perturbed away. We then use our data to develop reinforcement learning models that can predict experimental behaviours, and these models to inductively reason about how the nervous system optimizes coordination. We conclude that the nervous system optimizes for energy using a prediction of the optimal gait, and then refines this prediction with the cost of each new walking step.

Topics

No keywords indexed for this article. Browse by subject →

References

53

[1]

Energy optimization is a major objective in the real-time control of step width in human walking

Sabrina J. Abram, Jessica C. Selinger, J. Maxwell Donelan

Journal of Biomechanics 2019 10.1016/j.jbiomech.2019.05.010

[2]

Alexander (1996)

[3]

Atzler "Arbeitsphysiologische Studien III" Pflugers Arch. (1928)

[4]

Bastian "Understanding sensorimotor adaptation and learning for rehabilitation" Curr. Opin. Neurol. (2008) 10.1097/wco.0b013e328315a293

[5]

On the Theory of Dynamic Programming

Richard Bellman

Proceedings of the National Academy of Sciences 1952 10.1073/pnas.38.8.716

[6]

Bernstein (1967)

[7]

Collins "Efficient bipedal robots based on passive-dynamic walkers" Science (2005) 10.1126/science.1107799

[8]

Dean "Proprioceptive feedback and preferred patterns of human movement" Exerc. Sport Sci. Rev. (2013) 10.1097/jes.0b013e3182724bb0

[9]

Desmurget "Forward modeling allows feedback control for fast reaching movements" Trends Cogn. Sci. (Regul. Ed.) (2000) 10.1016/s1364-6613(00)01537-0

[10]

Mechanical and metabolic determinants of the preferred step width in human walking

J. Maxwell Donelan, Rodger Kram, Kuo Arthur D.

Proceedings of the Royal Society of London. Series... 2001 10.1098/rspb.2001.1761

[11]

Elftman "Biomechanics of muscle with particular application to studies of gait" J. Bone Joint Surg. Am. (1966) 10.2106/00004623-196648020-00017

[12]

The coordination of arm movements: an experimentally confirmed mathematical model

T Flash, N Hogan

The Journal of Neuroscience 1985 10.1523/jneurosci.05-07-01688.1985

[13]

Franklin "Computational mechanisms of sensorimotor control" Neuron (2011) 10.1016/j.neuron.2011.10.006

[14]

Herzfeld "Motor variability is not noise, but grist for the learning mill" Nat. Neurosci. (2014) 10.1038/nn.3633

[15]

Kording "The dynamics of memory as a consequence of optimal adaptation to a changing body" Nat. Neurosci. (2007) 10.1038/nn1901

[16]

Krakauer "Motor learning: its relevance to stroke recovery and neurorehabilitation" Curr. Opin. Neurol. (2006) 10.1097/01.wco.0000200544.29915.cc

[17]

Krakauer "Human sensorimotor learning: adaptation, skill, and beyond" Curr. Opin. Neurobiol. (2011) 10.1016/j.conb.2011.06.012

[18]

Kuo "Dynamic principles of gait and their clinical implications" Phys. Ther. (2010) 10.2522/ptj.20090125

[19]

Contribution of Feedback and Feedforward Strategies to Locomotor Adaptations

Tania Lam, Martin Anderschitz, Volker Dietz

Journal of Neurophysiology 2006 10.1152/jn.00473.2005

[20]

Lillicrap (2016)

[21]

Minetti "Mechanical determinants of gradient walking energetics in man" J. Physiol. (1993) 10.1113/jphysiol.1993.sp019969

[22]

Molen "Graphic representation of the relationship between oxygen-consumption and characteristics of normal gait of the human male" Proc. K Ned. Akad. Wet. C (1972)

[23]

O'Connor "Fast visual prediction and slow optimization of preferred walking speed" J. Neurophysiol. (2012) 10.1152/jn.00866.2011

[24]

Pagliara "Fast and slow processes underlie the selection of both step frequency and walking speed" J. Exp. Biol. (2014) 10.1242/jeb.105270

[25]

Peters "Reinforcement learning of motor skills with policy gradients" Neural Netw. (2008) 10.1016/j.neunet.2008.02.003

[26]

Ralston "Energy-speed relation and optimal speed during level walking" Int. Z. Angew. Physiol. (1958) 10.1007/bf00698754

[27]

Reinkensmeyer "Robotic gait training: toward more natural movements and optimal training algorithms" Conf. Proc. IEEE Eng. Med. Biol. Soc. (2004) 10.1109/iembs.2004.1404333

[28]

Scholz "The uncontrolled manifold concept: identifying control variables for a functional task" Exp. Brain Res. (1999) 10.1007/s002210050738

[29]

A Neural Substrate of Prediction and Reward

Wolfram Schultz, Peter Dayan, P. Read Montague

Science 1997 10.1126/science.275.5306.1593

[30]

Scott "Optimal feedback control and the neural basis of volitional motor control" Nat. Rev. Neurosci. (2004) 10.1038/nrn1427

[31]

Scott "Computational approaches to motor control and their potential role for interpreting motor dysfunction" Curr. Opin. Neurol. (2003) 10.1097/00019052-200312000-00008

[32]

Humans Can Continuously Optimize Energetic Cost during Walking

Jessica C. Selinger, Shawn M. O’Connor, Jeremy D. Wong et al.

Current Biology 2015 10.1016/j.cub.2015.08.016

[33]

Shadmehr "A computational neuroanatomy for motor control" Exp. Brain Res. (2008) 10.1007/s00221-008-1280-5

[34]

Shadmehr "A representation of effort in decision-making and motor control" Curr. Biol. (2016) 10.1016/j.cub.2016.05.065

[35]

Simha "A mechatronic system for studying energy optimization during walking" IEEE Trans. Neural Syst. Rehabil. Eng. (2019) 10.1109/tnsre.2019.2917424

[36]

Snaterse "Distinct fast and slow processes contribute to the selection of preferred step frequency during human walking" J. Appl. Physiol. (2011) 10.1152/japplphysiol.00536.2010

[37]

Computer optimization of a minimal biped model discovers walking and running

Manoj Srinivasan, ANDY RUINA

Nature 2005 10.1038/nature04113

[38]

Sutton (1998)

[39]

Sutton "Reinforcement learning is direct adaptive optimal control" IEEE Control Syst. (1992) 10.1109/37.126844

[40]

Optimality principles in sensorimotor control

Emanuel Todorov

Nature Neuroscience 2004 10.1038/nn1309

[41]

Todorov "Optimal feedback control as a theory of motor coordination" Nat. Neurosci. (2002) 10.1038/nn963

[42]

Tumer "Performance variability enables adaptive plasticity of “crystallized” adult birdsong" Nature (2007) 10.1038/nature06390

[43]

Umberger "Mechanical power and efficiency of level walking with different stride rates" J. Exp. Biol. (2007) 10.1242/jeb.000950

[44]

Wilson "Humans use directed and random exploration to solve the explore–exploit dilemma" J. Exp. Psychol. (2014) 10.1037/a0038199

[45]

Wolpert "Computational approaches to motor control" Trends Cogn. Sci. (Regul. Ed.) (1997) 10.1016/s1364-6613(97)01070-x

[46]

Wolpert "Computational principles of movement neuroscience" Nat. Neurosci. (2000) 10.1038/81497

[47]

Wolpert "Motor control is decision-making" Curr. Opin. Neurobiol. (2012) 10.1016/j.conb.2012.05.003

[48]

Wolpert "Perspectives and problems in motor learning" Trends Cogn. Sci. (Regul. Ed.) (2001) 10.1016/s1364-6613(00)01773-3

[49]

Wolpert "Principles of sensorimotor learning" Nat. Rev. Neurosci. (2011) 10.1038/nrn3112

[50]

Wong "Contribution of blood oxygen and carbon dioxide sensing to the energetic optimization of human walking" J. Neurophysiol. (2017) 10.1152/jn.00195.2017

Showing 50 of 53 references

Cited By

50

Biomechanical models in the lower-limb exoskeletons development: a review

Vahid Firouzi, Andre Seyfarth · 2025

Journal of NeuroEngineering and Reh...

Metrics

50

Citations

53

References

Details

Published: Jan 01, 2019
License: View

Authors

J

Jessica C. Selinger

School of Kinesiology and Health Studies, Queen's University, Kingston, Ontario, Canada; Department of Biomedical Physiology and Kinesiology, Simon Fraser University, Burnaby, British Columbia, Canada

J

Jeremy D. Wong

School of Kinesiology and Health Studies, Queen's University, Kingston, Ontario, Canada; Faculty of Kinesiology, University of Calgary, Calgary, Alberta, Canada

S

Surabhi N. Simha

Department of Biomedical Physiology and Kinesiology, Simon Fraser University, Burnaby, British Columbia, Canada

J

J. Maxwell Donelan

Department of Biomedical Physiology and Kinesiology, Simon Fraser University, Burnaby, British Columbia, Canada

Funding

Natural Sciences and Engineering Research Council of Canada Award: RGPIN-2019-05677

Cite This Article

Jessica C. Selinger, Jeremy D. Wong, Surabhi N. Simha, et al. (2019). How people initiate energy optimization and converge on their optimal gaits. Journal of Experimental Biology. https://doi.org/10.1242/jeb.198234

How people initiate energy optimization and converge on their optimal gaits

You May Also Like