On the limited memory BFGS method for large scale optimization

Dong C. Liu; Jorge Nocedal

doi:10.1007/bf01589116

journal article Aug 01, 1989

On the limited memory BFGS method for large scale optimization

Dong C. Liu Jorge Nocedal

Mathematical Programming Vol. 45 No. 1-3 pp. 503-528 · Springer Science and Business Media LLC

View at Publisher Save 10.1007/bf01589116

Topics

No keywords indexed for this article. Browse by subject →

References

36

[1]

E.M.L. Beale, “Algorithms for very large nonlinear optimization problems,” in: M.J.D. Powell, ed.,Nonlinear Optimization 1981 (Academic Press, London, 1981) pp. 281–292.

[2]

A. Buckley, “A combined conjugate gradient quasi-Newton minimization algorithm,”Mathematical Programming 15 (1978) 200–210. 10.1007/bf01609018

[3]

A. Buckley, “Update to TOMS Algorithm 630,” Rapports Techniques No. 91, Institut National de Recherche en Informatique et en Automatique, Domaine Voluceau, Rocquencourt, B.P. 105 (Le Chesnay, 1987).

[4]

A. Buckley and A. LeNir, “QN-like variable storage conjugate gradients,”Mathematical Programming 27 (1983) 155–175. 10.1007/bf02591943

[5]

A. Buckley and A. LeNir, “BBVSCG—A variable storage algorithm for function minimization,”ACM Transactions on Mathematical Software 11/2 (1985) 103–119. 10.1145/214392.214395

[6]

R.H. Byrd and J. Nocedal, “A tool for the analysis of quasi-Newton methods with application to unconstrained minimization,”SIAM Journal on Numerical Analysis 26 (1989) 727–739. 10.1137/0726042

[7]

J.E. Dennis Jr. and R.B. Schnabel,Numerical methods for unconstrained optimization and nonlinear equations (Prentice-Hall, 1983).

[8]

J.E. Dennis Jr. and R.B. Schnabel, “A view of unconstrained optimization,” in: G.L. Nemhauser, A.H.G. Rinnooy Kan and M.J. Todd, eds.,Handbooks in Operations Research and Management Science, Vol. 1, Optimization (North-Holland, Amsterdam, 1989) pp. 1–72.

[9]

R. Fletcher,Practical Methods of Optimization, Vol. 1, Unconstrained Optimization (Wiley, New York, 1980).

[10]

J.C. Gilbert and C. Lemaréchal, “Some numerical experiments with variable storage quasi-Newton algorithms,” IIASA Working Paper WP-88, A-2361 (Laxenburg, 1988).

[11]

P.E. Gill and W. Murray, “Conjugate-gradient methods for large-scale nonlinear optimization,” Technical Report SOL 79-15, Department of Operations Research, Stanford University (Stanford, CA, 1979). 10.21236/ada078713

[12]

P.E. Gill, W. Murray and M.H. Wright,Practical Optimization (Academic Press, London, 1981).

[13]

A. Griewank, “The global convergence of partitioned BFGS on semi-smooth problems with convex decompositions,” ANL/MCS-TM-105, Mathematics and Computer Science Division, Argonne National Laboratory (Argonne, IL, 1987).

[14]

A. Griewank and Ph.L. Toint, “Partitioned variable metric updates for large structured optimization problems,”Numerische Mathematik 39 (1982a) 119–137. 10.1007/bf01399316

[15]

A. Griewank and Ph.L. Toint, “Local convergence analysis of partitioned quasi-Newton updates,”Numerische Mathematik 39 (1982b) 429–448. 10.1007/bf01407874

[16]

A. Griewank and Ph.L. Toint, “Numerical experiments with partially separable optimization problems,” in: D.F. Griffiths, ed.,Numerical Analysis: Proceedings Dundee 1983, Lecture Notes in Mathematics, Vol. 1066 (Springer, Berlin, 1984) pp. 203–220. 10.1007/bfb0099526

[17]

D.C. Liu and J. Nocedal, “Test results of two limited memory methods for large scale optimization,” Technical Report NAM 04, Department of Electrical Engineering and Computer Science, Northwestern University (Evanston, IL, 1988).

[18]

J.J. Moré, B.S. Garbow and K.E. Hillstrom, “Testing unconstrained optimization software,”ACM Transactions on Mathematical Software 7 (1981) 17–41. 10.1145/355934.355936

[19]

S.G. Nash, “Preconditioning of truncated-Newton methods,”SIAM Journal on Scientific and Statistical Computing 6 (1985) 599–616. 10.1137/0906042

[20]

L. Nazareth, “A relationship between the BFGS and conjugate gradient algorithms and its implications for new algorithms,”SIAM Journal on Numerical Analysis 16 (1979) 794–800. 10.1137/0716059

[21]

J. Nocedal, “Updating quasi-Newton matrices with limited storage,”Mathematics of Computation 35 (1980) 773–782. 10.1090/s0025-5718-1980-0572855-7

[22]

D.P. O'Leary, “A discrete Newton algorithm for minimizing a function of many variables,”Mathematical Programming 23 (1982) 20–33. 10.1007/bf01583777

[23]

J.D. Pearson, “Variable metric methods of minimization,”Computer Journal 12 (1969) 171–178. 10.1093/comjnl/12.2.171

[24]

J.M. Perry, “A class of conjugate gradient algorithms with a two-step variable-metric memory,” Discussion Paper 269, Center for Mathematical Studies in Economics and Management Science, Northwestern University (Evanston, IL, 1977).

[25]

M.J.D. Powell, “Some global convergence properties of a variable metric algorithm for minimization without exact line search,” in: R.W. Cottle and C.E. Lemke, eds.,Nonlinear Programing, SIAM-AMS Proceedings IX (SIAM, Philadelphia, PA, 1976).

[26]

M.J.D. Powell, “Restart procedures for the conjugate gradient method,”Mathematical Programming 12 (1977) 241–254. 10.1007/bf01593790

[27]

D.F. Shanno, “On the convergence of a new conjugate gradient algorithm,”SIAM Journal on Numerical Analysis 15 (1978a) 1247–1257. 10.1137/0715085

[28]

D.F. Shanno, “Conjugate gradient methods with inexact searches,”Mathematics of Operations Research 3 (1978b) 244–256. 10.1287/moor.3.3.244

[29]

D.F. Shanno and K.H. Phua, “Matrix conditioning and nonlinear optimization,”Mathematical Programming 14 (1978) 149–160. 10.1007/bf01588962

[30]

D.F. Shanno and K.H. Phua, “Remark on algorithm 500: minimization of unconstrained multivariate functions,”ACM Transactions on Mathematical Software 6 (1980) 618–622. 10.1145/355921.355933

[31]

T. Steihaug, “The conjugate gradient method and trust regions in large scale optimization,”SIAM Journal on Numerical Analysis 20 (1983) 626–637. 10.1137/0720042

[32]

Ph.L. Toint, “Some numerical results using a sparse matrix updating formula in unconstrained optimization,”Mathematics of Computation 32 (1978) 839–851. 10.1090/s0025-5718-1978-0483452-7

[33]

Ph.L. Toint, “Towards an efficient sparsity exploiting Newton method for minimization,” in: I.S. Duff, ed.,Sparse Matrices and their Uses (Academic Press, New York, 1981) pp. 57–87.

[34]

Ph.L. Toint, “Test problems for partially separable optimization and results for the routine PSPMIN,” Report Nr 83/4, Department of Mathematics, Facultés Universitaires de Namur (Namur, 1983a).

[35]

Ph.L. Toint, “VE08AD, a routine for partially separable optimization with bounded variables,” Harwell Subroutine Library, A.E.R.E. (UK, 1983b).

[36]

Ph.L. Toint, “A view of nonlinear optimization in a large number of variables,” Report Nr 86/16, Department of Mathematics, Facultés Universitaires de Namur (Namur, 1986).

Cited By

6,311

ASH : A Multi‐Scale, Multi‐Theory Modeling Program

Ragnar Bjornsson · 2026

Journal of Computational Chemistry

Development of Physics-Informed Neural Networks (PINNs) for the natural ventilation of a greenhouse – Part 2: Development of prediction models for variable wind conditions

Young-Bae Choi, In-Bok Lee · 2026

Biosystems Engineering

An adjoint formulation of energetic particle confinement

Christopher J. McDevitt, Jonathan S. Arnaud · 2026

Journal of Plasma Physics

GATRO: GPU-Accelerated Trajectory Optimization With Application to Tractor-Trailers

Bo Yang, Zishuo Li · 2026

IEEE Transactions on Automation Sci...

Theoretical prediction of structural stability and superconductivity in Janus Ti 2 CSH MXene

Jakkapat Seeyangnok, Udomsilp Pinsook · 2026

Physical Review B

A neighbour-aware LSTM-PINN model for simultaneous soil moisture forecasting and retention function estimation

Chenting Jiang, Quan Bai · 2026

Advances in Water Resources

Application of self-adaptive physics-informed neural network for the modeling of deep bottomhole heat exchangers

Ehsan Ghasemi, Mohammad Emami Niri · 2026

Applied Thermal Engineering

SI-PSNN: A Source Iteration based Physics-Specialized Neural Network for solving multi-group neutron diffusion eigenvalue problems

Yuchen Xie, Honghang Chi · 2026

Annals of Nuclear Energy

Multi-view clustering algorithm based on bipartite graph learning and high-order information fusion

Shunyong Li, Mengjiao Zheng · 2026

Information Sciences

Insight into the interfacial adsorption differences of acid retarders on calcite by first-principles calculations

Jian Lan, Jie Qi · 2026

Colloids and Surfaces A: Physicoche...

Deep learning-based probabilistic slope stability analysis considering spatial variability of soils and saturated seepage

Zhengwei Li, Wenping Gong · 2026

Computers and Geotechnics

In vivo blood viscosity estimation from microscopic images by solving an inverse incompressible Navier-Stokes problem

Shota Ito, Moritz Vogel · 2026

Computer Methods in Applied Mechani...

Locally-averaged McCormick relaxations for discretization-regularized inverse problems

Barbara Kaltenbacher, Paul Manns · 2026

Inverse Problems

Sample-efficient counterfactual tuning for compressor pressure control

Margarita A. Guerrero, Rodrigo A. González · 2026

Journal of Process Control

Absence of Barren Plateaus and Scaling of Gradients in the Energy Optimization of Isometric Tensor Network States

Thomas Barthel, Qiang Miao · 2025

Communications in Mathematical Phys...

Physics‐Informed Deep Learning for Estimating the Spatial Distribution of Frictional Parameters in Slow Slip Regions

Rikuto Fukushima, Masayuki Kano · 2025

Journal of Geophysical Research: So...

Physics‐Informed Deep Learning for Forward and Inverse Modeling of Inplane Crustal Deformation

Tomohisa Okazaki, Kazuro Hirahara · 2025

Journal of Geophysical Research: Ma...

Physics-informed neural networks for phase-resolved data assimilation and prediction of nonlinear ocean waves

Svenja Ehlers, Norbert Hoffmann · 2025

Physical Review Fluids

Materials Graph Library (MatGL), an open-source graph deep learning library for materials science and chemistry

Tsz Wai Ko, Bowen Deng · 2025

npj Computational Materials

Persistent Classification: Understanding Adversarial Attacks by Studying Decision Boundary Dynamics

Brian Bell, Michael Geyer · 2025

Statistical Analysis and Data Minin...

Metrics

6,311

Citations

36

References

Details

Published: Aug 01, 1989
Vol/Issue: 45(1-3)
Pages: 503-528
License: View

Authors

Cite This Article

Dong C. Liu, Jorge Nocedal (1989). On the limited memory BFGS method for large scale optimization. Mathematical Programming, 45(1-3), 503-528. https://doi.org/10.1007/bf01589116

On the limited memory BFGS method for large scale optimization

You May Also Like