Absence of Barren Plateaus and Scaling of Gradients in the Energy Optimization of Isometric Tensor Network States

Thomas Barthel; Qiang Miao

doi:10.1007/s00220-024-05217-x

journal article Open Access Mar 19, 2025

Absence of Barren Plateaus and Scaling of Gradients in the Energy Optimization of Isometric Tensor Network States

Thomas Barthel Qiang Miao

Communications in Mathematical Physics Vol. 406 No. 4 · Springer Science and Business Media LLC

View at Publisher Save 10.1007/s00220-024-05217-x

Abstract

Abstract
Vanishing gradients can pose substantial obstacles for high-dimensional optimization problems. Here we consider energy minimization problems for quantum many-body systems with extensive Hamiltonians and finite-range interactions, which can be studied on classical computers or in the form of variational quantum eigensolvers on quantum computers. Barren plateaus correspond to scenarios where the average amplitude of the energy gradient decreases exponentially with increasing system size. This occurs, for example, for quantum neural networks and for brickwall quantum circuits when the depth increases polynomially in the system size. Here we prove that the variational optimization problems for matrix product states, tree tensor networks, and the multiscale entanglement renormalization ansatz are free of barren plateaus. The derived scaling properties for the gradient variance provide an analytical guarantee for the trainability of randomly initialized tensor network states (TNS) and motivate certain initialization schemes. In a suitable representation, unitary tensors that parametrize the TNS are sampled according to the uniform Haar measure. We employ a Riemannian formulation of the gradient based optimizations which simplifies the analytical evaluation.

Topics

No keywords indexed for this article. Browse by subject →

References

106

[1]

Hochreiter, S.: The vanishing gradient problem during learning recurrent neural nets and problem solutions. Int. J. Uncertain. Fuzziness Knowl. Based Syst. 6, 107 (1998). https://doi.org/10.1142/S0218488598000094 10.1142/s0218488598000094

[2]

Fukumizu, K., Amari, S.: Local minima and plateaus in hierarchical structures of multilayer perceptrons. Neural Netw. 13, 317 (2000). https://doi.org/10.1016/S0893-6080(00)00009-5 10.1016/s0893-6080(00)00009-5

[3]

Dauphin, Y.N., Pascanu, R., Gulcehre, C., Cho, K., Ganguli, S., Bengio, Y.: Identifying and attacking the saddle point problem in high-dimensional non-convex optimization. Adv. Neural Inf. Process. Syst. 2, 2933–2941 (2014). https://doi.org/10.48550/arXiv.1406.2572 10.48550/arxiv.1406.2572

[4]

Shalev-Shwartz, S., Shamir, O., Shammah, S.: Failures of gradient-based deep learning. Proc. Mach. Learn. Res. 70, 3067 (2017). https://doi.org/10.48550/arXiv.1703.07950 10.48550/arxiv.1703.07950

[5]

McClean, J.R., Boixo, S., Smelyanskiy, V.N., Babbush, R., Neven, H.: Barren plateaus in quantum neural network training landscapes. Nat. Commun. 9, 4812 (2018). https://doi.org/10.1038/s41467-018-07090-4 10.1038/s41467-018-07090-4

[6]

Cerezo, M., Sone, A., Volkoff, T., Cincio, L., Coles, P.J.: Cost function dependent barren plateaus in shallow parametrized quantum circuits. Nat. Commun. 12, 1791 (2021). https://doi.org/10.1038/s41467-021-21728-w 10.1038/s41467-021-21728-w

[7]

Ortiz Marrero, C., Kieferová, M., Wiebe, N.: Entanglement-induced barren plateaus. PRX Quantum 2, 040316 (2021). https://doi.org/10.1103/PRXQuantum.2.040316 10.1103/prxquantum.2.040316

[8]

Uvarov, A.V., Biamonte, J.D.: On barren plateaus and cost function locality in variational quantum algorithms. J. Phys. A: Math. Theor. 54, 245301 (2021). https://doi.org/10.1088/1751-8121/abfac7 10.1088/1751-8121/abfac7

[9]

Sharma, K., Cerezo, M., Cincio, L., Coles, P.J.: Trainability of dissipative perceptron-based quantum neural networks. Phys. Rev. Lett. 128, 180505 (2022). https://doi.org/10.1103/PhysRevLett.128.180505 10.1103/physrevlett.128.180505

[10]

Napp, J.: Quantifying the barren plateau phenomenon for a model of unstructured variational ansätze. arXiv:2203.06174 (2022)

[11]

Arrasmith, A., Holmes, Z., Cerezo, M., Coles, P.J.: Equivalence of quantum barren plateaus to cost concentration and narrow gorges. Quantum Sci. Technol. 7, 045015 (2022). https://doi.org/10.1088/2058-9565/ac7d06 10.1088/2058-9565/ac7d06

[12]

Miao, Q., Barthel, T.: Equivalence of cost concentration and gradient vanishing for quantum circuits: An elementary proof in the Riemannian formulation. Quantum Sci. Technol. 9, 045039 (2024). https://doi.org/10.1088/2058-9565/ad6fca 10.1088/2058-9565/ad6fca

[13]

Patti, T.L., Najafi, K., Gao, X., Yelin, S.F.: Entanglement devised barren plateau mitigation. Phys. Rev. Research 3, 033090 (2021). https://doi.org/10.1103/PhysRevResearch.3.033090 10.1103/physrevresearch.3.033090

[14]

Holmes, Z., Sharma, K., Cerezo, M., Coles, P.J.: Connecting ansatz expressibility to gradient magnitudes and barren plateaus. PRX Quantum 3, 010313 (2022). https://doi.org/10.1103/PRXQuantum.3.010313 10.1103/prxquantum.3.010313

[15]

Baxter, R.J.: Dimers on a rectangular lattice. J. Math. Phys. 9, 650 (1968). https://doi.org/10.1063/1.1664623 10.1063/1.1664623

[16]

White, S.R.: Density matrix formulation for quantum renormalization groups. Phys. Rev. Lett. 69, 2863 (1992). https://doi.org/10.1103/PhysRevLett.69.2863 10.1103/physrevlett.69.2863

[17]

Niggemann, H., Klümper, A., Zittartz, J.: Quantum phase transition in spin-3/2 systems on the hexagonal lattice - optimum ground state approach. Z. Phys. B 104, 103 (1997). https://doi.org/10.1007/s002570050425 10.1007/s002570050425

[18]

Verstraete, F., Cirac, J.I.: Renormalization algorithms for quantum-many body systems in two and higher dimensions. arXiv:cond-mat/0407066 (2004)

[19]

Vidal, G.: Entanglement renormalization. Phys. Rev. Lett. 99, 220405 (2007). https://doi.org/10.1103/PhysRevLett.99.220405 10.1103/physrevlett.99.220405

[20]

Schollwöck, U.: The density-matrix renormalization group in the age of matrix product states. Ann. Phys. 326, 96 (2011). https://doi.org/10.1016/j.aop.2010.09.012 10.1016/j.aop.2010.09.012

[21]

Orús, R.: A practical introduction to tensor networks: matrix product states and projected entangled pair states. Ann. Phys. 349, 117 (2014). https://doi.org/10.1016/j.aop.2014.06.013 10.1016/j.aop.2014.06.013

[22]

McClean, J.R., Romero, J., Babbush, R., Aspuru-Guzik, A.: The theory of variational hybrid quantum-classical algorithms. New J. Phys. 18, 023023 (2016). https://doi.org/10.1088/1367-2630/18/2/023023 10.1088/1367-2630/18/2/023023

[23]

Liu, J.-G., Zhang, Y.-H., Wan, Y., Wang, L.: Variational quantum eigensolver with fewer qubits. Phys. Rev. Res. 1, 023025 (2019). https://doi.org/10.1103/PhysRevResearch.1.023025 10.1103/physrevresearch.1.023025

[24]

Smith, A., Kim, M., Pollmann, F., Knolle, J.: Simulating quantum many-body dynamics on a current digital quantum computer. Quantum Inf. 5, 106 (2019). https://doi.org/10.1038/s41534-019-0217-0 10.1038/s41534-019-0217-0

[25]

Miao, Q., Barthel, T.: Quantum-classical eigensolver using multiscale entanglement renormalization. Phys. Rev. Res. 5, 033141 (2023). https://doi.org/10.1103/PhysRevResearch.5.033141 10.1103/physrevresearch.5.033141

[26]

Slattery, L., Clark, B.K.: Quantum circuits for two-dimensional isometric tensor networks. arXiv:2108.02792 (2021)

[27]

Barratt, F., Dborin, J., Bal, M., Stojevic, V., Pollmann, F., Green, A.G.: Parallel quantum simulation of large systems on small NISQ computers. npj Quantum Inf. 7, 79 (2021). https://doi.org/10.1038/s41534-021-00420-3 10.1038/s41534-021-00420-3

[28]

Foss-Feig, M., Hayes, D., Dreiling, J.M., Figgatt, C., Gaebler, J.P., Moses, S.A., Pino, J.M., Potter, A.C.: Holographic quantum algorithms for simulating correlated spin systems. Phys. Rev. Research 3, 033002 (2021). https://doi.org/10.1103/PhysRevResearch.3.033002 10.1103/physrevresearch.3.033002

[29]

Niu, D., Haghshenas, R., Zhang, Y., Foss-Feig, M., Chan, G.K.-L., Potter, A.C.: Holographic simulation of correlated electrons on a trapped-ion quantum processor. PRX Quantum 3, 030317 (2022). https://doi.org/10.1103/PRXQuantum.3.030317 10.1103/prxquantum.3.030317

[30]

Chertkov, E., Bohnet, J., Francois, D., Gaebler, J., Gresh, D., Hankin, A., Lee, K., Hayes, D., Neyenhuis, B., Stutz, R., Potter, A.C., Foss-Feig, M.: Holographic dynamics simulations with a trapped-ion quantum computer. Nat. Phys. 18, 1074 (2022). https://doi.org/10.1038/s41567-022-01689-7 10.1038/s41567-022-01689-7

[31]

Srednicki, M.: Entropy and area. Phys. Rev. Lett. 71, 666 (1993). https://doi.org/10.1103/PhysRevLett.71.666 10.1103/physrevlett.71.666

[32]

Callan, C., Wilczek, F.: On geometric entropy. Phys. Lett. B 333, 55 (1994). https://doi.org/10.1016/0370-2693(94)91007-3 10.1016/0370-2693(94)91007-3

[33]

Holzhey, C., Larsen, F., Wilczek, F.: Geometric and renormalized entropy in conformal field theory. Nucl. Phys. B 424, 443 (1994). https://doi.org/10.1016/0550-3213(94)90402-2 10.1016/0550-3213(94)90402-2

[34]

Vidal, G., Latorre, J.I., Rico, E., Kitaev, A.: Entanglement in quantum critical phenomena. Phys. Rev. Lett. 90, 227902 (2003). https://doi.org/10.1103/PhysRevLett.90.227902 10.1103/physrevlett.90.227902

[35]

Jin, B.Q., Korepin, V.E.: Quantum spin chain, Toeplitz determinants and Fisher-Hartwig conjecture. J. Stat. Phys. 116, 79 (2004). https://doi.org/10.1023/B:JOSS.0000037230.37166.42 10.1023/b:joss.0000037230.37166.42

[36]

Latorre, J.I., Rico, E., Vidal, G.: Ground state entanglement in quantum spin chains. Quantum Info. Comput. 4, 48 (2004)

[37]

Calabrese, P., Cardy, J.L.: Entanglement entropy and quantum field theory, J. Stat. Mech. P06002 (2004). https://doi.org/10.1088/1742-5468/2004/06/P06002 10.1088/1742-5468/2004/06/p06002

[38]

Zhou, H.-Q., Barthel, T., Fjærestad, J.O., Schollwöck, U.: Entanglement and boundary critical phenomena. Phys. Rev. A 74, 050305(R) (2006). https://doi.org/10.1103/PhysRevA.74.050305 10.1103/physreva.74.050305

[39]

Plenio, M.B., Eisert, J., Dreißig, J., Cramer, M.: Entropy, entanglement, and area: analytical results for harmonic lattice systems. Phys. Rev. Lett. 94, 060503 (2005). https://doi.org/10.1103/PhysRevLett.94.060503 10.1103/physrevlett.94.060503

[40]

Wolf, M.M.: Violation of the entropic area law for fermions. Phys. Rev. Lett. 96, 010404 (2006). https://doi.org/10.1103/PhysRevLett.96.010404 10.1103/physrevlett.96.010404

[41]

Gioev, D., Klich, I.: Entanglement entropy of fermions in any dimension and the Widom conjecture. Phys. Rev. Lett. 96, 100503 (2006). https://doi.org/10.1103/PhysRevLett.96.100503 10.1103/physrevlett.96.100503

[42]

Barthel, T., Chung, M.-C., Schollwöck, U.: Entanglement scaling in critical two-dimensional fermionic and bosonic systems. Phys. Rev. A 74, 022329 (2006). https://doi.org/10.1103/PhysRevA.74.022329 10.1103/physreva.74.022329

[43]

Li, W., Ding, L., Yu, R., Roscilde, T., Haas, S.: Scaling behavior of entanglement in two- and three-dimensional free-fermion systems. Phys. Rev. B 74, 073103 (2006). https://doi.org/10.1103/PhysRevB.74.073103 10.1103/physrevb.74.073103

[44]

Cramer, M., Eisert, J., Plenio, M.B., Dreißig, J.: Entanglement-area law for general bosonic harmonic lattice systems. Phys. Rev. A 73, 012309 (2006). https://doi.org/10.1103/PhysRevA.73.012309 10.1103/physreva.73.012309

[45]

Hastings, M.B.: Entropy and entanglement in quantum ground states. Phys. Rev. B 76, 035114 (2007). https://doi.org/10.1103/PhysRevB.76.035114 10.1103/physrevb.76.035114

[46]

Brandão, F.G.S.L., Horodecki, M.: An area law for entanglement from exponential decay of correlations. Nat. Phys. 9, 721 (2013). https://doi.org/10.1038/nphys2747 10.1038/nphys2747

[47]

Cho, J.: Realistic area-law bound on entanglement from exponentially decaying correlations. Phys. Rev. X 8, 031009 (2018). https://doi.org/10.1103/PhysRevX.8.031009 10.1103/physrevx.8.031009

[48]

Kuwahara, T., Saito, K.: Area law of noncritical ground states in 1D long-range interacting systems. Nat. Commun. 11, 4478 (2020). https://doi.org/10.1038/s41467-020-18055-x 10.1038/s41467-020-18055-x

[49]

Eisert, J., Cramer, M., Plenio, M.B.: Colloquium: Area laws for the entanglement entropy. Rev. Mod. Phys. 82, 277 (2010). https://doi.org/10.1103/RevModPhys.82.277 10.1103/revmodphys.82.277

[50]

Latorre, J.I., Riera, A.: A short review on entanglement in quantum spin systems. J. Phys. A: Math. Theor. 42, 504002 (2009). https://doi.org/10.1088/1751-8113/42/50/504002 10.1088/1751-8113/42/50/504002

Showing 50 of 106 references

Metrics

8

Citations

106

References

Details

Published: Mar 19, 2025
Vol/Issue: 406(4)
License: View

Authors

Funding

U.S. Department of Energy Award: DE-SC0019449

Cite This Article

Thomas Barthel, Qiang Miao (2025). Absence of Barren Plateaus and Scaling of Gradients in the Energy Optimization of Isometric Tensor Network States. Communications in Mathematical Physics, 406(4). https://doi.org/10.1007/s00220-024-05217-x

Absence of Barren Plateaus and Scaling of Gradients in the Energy Optimization of Isometric Tensor Network States

You May Also Like