Université Grenoble Alpes, CEA List, 38000 Grenoble, France
Find this paper interesting or want to discuss? Scite or leave a comment on SciRate.
Abstract
We propose a method for finding approximate compilations of quantum unitary transformations, based on techniques from policy gradient reinforcement learning. The choice of a stochastic policy allows us to rephrase the optimization problem in terms of probability distributions, rather than variational gates. In this framework, the optimal configuration is found by optimizing over distribution parameters, rather than over free angles. We show numerically that this approach can be more competitive than gradient-free methods, for a comparable amount of resources, both for noiseless and noisy circuits. Another interesting feature of this approach to variational compilation is that it does not need a separate register and long-range interactions to estimate the end-point fidelity, which is an improvement over methods which rely on the Hilbert-Schmidt test. We expect these techniques to be relevant for training variational circuits in other contexts.
► BibTeX data
► References
[1] Nielsen M. A. & Chuang I. Quantum Computation and Quantum Information (2002).
[2] Harrow A. W., Recht B. & Chuang I. L. Efficient discrete approximations of quantum gates. Journal of Mathematical Physics, 43(9), 4445-4451 (2002) https://doi.org/10.1063/1.1495899.
https://doi.org/10.1063/1.1495899
[3] Dawson C. M. & Nielsen M. A. The Solovay-Kitaev algorithm. arXiv preprint quant-ph/0505030 (2005) https://doi.org/10.48550/arXiv.quant-ph/0505030.
https://doi.org/10.48550/arXiv.quant-ph/0505030
arXiv:quant-ph/0505030
[4] Lin H. W. Cayley graphs and complexity geometry. Journal of High Energy Physics, 2019(2), 1-15 (2019) https://doi.org/10.1007/JHEP02%282019%29063.
https://doi.org/10.1007/JHEP02%282019%29063
[5] Krioukov D., Papadopoulos F., Kitsak M., Vahdat A. & Boguná M. Hyperbolic geometry of complex networks. Physical Review E, 82(3), 036106 (2010) https://doi.org/10.1103/PhysRevE.82.036106.
https://doi.org/10.1103/PhysRevE.82.036106
[6] Nielsen M. A., Dowling M. R., Gu M. & Doherty A. C. Quantum computation as geometry. Science, 311(5764), 1133-1135 (2006) https://10.1126/science.1124295.
https://doi.org/10.1126/science.1124295
[7] Preskill J. Quantum computing in the NISQ era and beyond. Quantum, 2, 79 (2018) https://doi.org/10.22331/q-2018-08-06-79.
https://doi.org/10.22331/q-2018-08-06-79
[8] Lloyd S. Quantum approximate optimization is computationally universal. arXiv preprint arXiv:1812.11075 (2018) https://doi.org/10.48550/arXiv.1812.11075.
https://doi.org/10.48550/arXiv.1812.11075
arXiv:1812.11075
[9] Morales M. E., Biamonte J. D. & Zimborás Z. On the universality of the quantum approximate optimization algorithm. Quantum Information Processing, 19(9), 1-26 (2020) https://doi.org/10.1007/s11128-020-02748-9.
https://doi.org/10.1007/s11128-020-02748-9
[10] Kiani B., Maity R. & Lloyd S. Learning unitaries via gradient descent optimization. Bulletin of the American Physical Society, 65 (2020) https://doi.org/10.48550/arXiv.2001.11897.
https://doi.org/10.48550/arXiv.2001.11897
[11] Farhi E. & Harrow A. W. Quantum supremacy through the quantum approximate optimization algorithm. arXiv preprint arXiv:1602.07674 (2016) https://doi.org/10.48550/arXiv.1602.07674.
https://doi.org/10.48550/arXiv.1602.07674
arXiv:1602.07674
[12] Arute F., Arya K., Babbush R., Bacon D., Bardin J. C., Barends R., … & Martinis J. M. Quantum supremacy using a programmable superconducting processor. Nature, 574(7779), 505-510 (2019) https://doi.org/10.1038/s41586-019-1666-5.
https://doi.org/10.1038/s41586-019-1666-5
[13] Zhu Q., Cao S., Chen F., Chen M. C., Chen X., Chung T. H., … & Pan J. W. Quantum Computational Advantage via 60-Qubit 24-Cycle Random Circuit Sampling. arXiv preprint arXiv:2109.03494 (2021) https://doi.org/10.48550/arXiv.2109.03494.
https://doi.org/10.48550/arXiv.2109.03494
arXiv:2109.03494
[14] Bravyi S., Gosset D., & König R. Quantum advantage with shallow circuits. Science, 362(6412), 308-311 (2018) https://doi.org/10.1126/science.aar3106.
https://doi.org/10.1126/science.aar3106
[15] Bravyi S., Gosset D., Koenig R. & Tomamichel, M. Quantum advantage with noisy shallow circuits. Nature Physics, 16(10), 1040-1045 (2020) https://doi.org/10.1038/s41567-020-0948-z.
https://doi.org/10.1038/s41567-020-0948-z
[16] Bauer B., Bravyi S., Motta M. & Chan G. K. L. Quantum algorithms for quantum chemistry and quantum materials science. Chemical Reviews, 120(22), 12685-12717 (2020) https://doi.org/10.1021/acs.chemrev.9b00829.
https://doi.org/10.1021/acs.chemrev.9b00829
[17] O’Malley P. J., Babbush R., Kivlichan I. D., Romero J., McClean J. R., Barends R., … & Martinis J. M. Scalable quantum simulation of molecular energies. Physical Review X, 6(3), 031007 (2016) https://doi.org/10.1103/PhysRevX.6.031007.
https://doi.org/10.1103/PhysRevX.6.031007
[18] Ralli A., Love P. J., Tranter A., & Coveney P. V. Implementation of measurement reduction for the variational quantum eigensolver. Physical Review Research, 3(3), 033195 (2021) https://doi.org/10.1103/PhysRevResearch.3.033195.
https://doi.org/10.1103/PhysRevResearch.3.033195
[19] Hastings M. B. Classical and quantum bounded depth approximation algorithms. arXiv preprint arXiv:1905.07047 (2019) https://doi.org/10.48550/arXiv.1905.07047.
https://doi.org/10.48550/arXiv.1905.07047
arXiv:1905.07047
[20] Bravyi S., Kliesch A., Koenig R, & Tang E. Obstacles to variational quantum optimization from symmetry protection. Physical Review Letters, 125(26), 260505 (2020) https://doi.org/10.1103/PhysRevLett.125.260505.
https://doi.org/10.1103/PhysRevLett.125.260505
[21] Bravyi S., Kliesch A., Koenig R. & Tang E. Hybrid quantum-classical algorithms for approximate graph coloring. Quantum 6, 678 (2022). https://doi.org/10.22331/q-2022-03-30-678.
https://doi.org/10.22331/q-2022-03-30-678
[22] McClean J. R., Boixo S., Smelyanskiy V. N., Babbush R. & Neven, H. Barren plateaus in quantum neural network training landscapes. Nature communications, 9(1) (2018) https://doi.org/10.1038/s41467-018-07090-4.
https://doi.org/10.1038/s41467-018-07090-4
[23] Cerezo M., Sone A., Volkoff T., Cincio L. & Coles P. J. Cost function-dependent barren plateaus in shallow quantum neural networks. Nature communications, 12(1) (2021) https://doi.org/10.1038/s41467-021-21728-w.
https://doi.org/10.1038/s41467-021-21728-w
[24] Grant E., Wossnig L., Ostaszewski M. & Benedetti, M. An initialization strategy for addressing barren plateaus in parameterized quantum circuits. Quantum, 3, 214 (2019) https://doi.org/10.22331/q-2019-12-09-214.
https://doi.org/10.22331/q-2019-12-09-214
[25] Volkoff T. & Coles P. J. Large gradients via correlation in random parameterized quantum circuits. Quantum Science and Technology, 6(2), 025008 (2021) https://doi.org/10.1088/2058-9565/abd891.
https://doi.org/10.1088/2058-9565/abd891
[26] Skolik A., McClean J. R., Mohseni M., van der Smagt P. & Leib, M. Layerwise learning for quantum neural networks. Quantum Machine Intelligence, 3(1), (2021) https://doi.org/10.1007/s42484-020-00036-4.
https://doi.org/10.1007/s42484-020-00036-4
[27] Khatri S., LaRose R., Poremba A., Cincio L., Sornborger A. T., & Coles, P. J. Quantum-assisted quantum compiling. Quantum, 3, 140 (2019) https://doi.org/10.22331/q-2019-05-13-140.
https://doi.org/10.22331/q-2019-05-13-140
[28] Sharma K., Khatri S., Cerezo M. & Coles P. J. Noise resilience of variational quantum compiling. New Journal of Physics, 22(4), 043006 (2020) https://doi.org/10.1088/1367-2630/ab784c.
https://doi.org/10.1088/1367-2630/ab784c
[29] Wang S., Fontana E., Cerezo M., Sharma K., Sone A., Cincio L. & Coles P. J. Noise-induced barren plateaus in variational quantum algorithms. Nature communications, 12(1) (2021) https://doi.org/10.1038/s41467-021-27045-6.
https://doi.org/10.1038/s41467-021-27045-6
[30] Arrasmith A., Cerezo M., Czarnik P., Cincio L. & Coles P. J. Effect of barren plateaus on gradient-free optimization. Quantum, 5, 558 (2021) https://doi.org/10.22331/q-2021-10-05-558.
https://doi.org/10.22331/q-2021-10-05-558
[31] Schuld M., Bergholm V., Gogolin C., Izaac J. & Killoran, N. Evaluating analytic gradients on quantum hardware. Physical Review A, 99(3) (2019) https://doi.org/10.1103/PhysRevA.99.032331.
https://doi.org/10.1103/PhysRevA.99.032331
[32] Holmes Z., Arrasmith A., Yan B., Coles P. J., Albrecht A. & Sornborger A. T. Barren plateaus preclude learning scramblers. Physical Review Letters, 126(19), 190501 (2021) https://doi.org/10.1103/PhysRevLett.126.190501.
https://doi.org/10.1103/PhysRevLett.126.190501
[33] Sutton R. S. & Barto A. G. Reinforcement learning: An introduction. MIT press (2018).
[34] Nautrup H. P., Delfosse N., Dunjko V., Briegel H. J. & Friis N. Optimizing quantum error correction codes with reinforcement learning. Quantum, 3, 215 (2019) https://doi.org/10.22331/q-2019-12-16-215.
https://doi.org/10.22331/q-2019-12-16-215
[35] Moro, L., Paris, M. G., Restelli, M., & Prati, E. Quantum Compiling by Deep Reinforcement Learning. Communications Physics 4 (2021) https://doi.org/10.1038/s42005-021-00684-3.
https://doi.org/10.1038/s42005-021-00684-3
[36] Fösel T., Tighineanu P., Weiss T. & Marquardt F. Reinforcement learning with neural networks for quantum feedback. Physical Review X, 8(3), 031084 (2018) https://doi.org/10.1103/PhysRevX.8.031084.
https://doi.org/10.1103/PhysRevX.8.031084
[37] August M. & Hernández-Lobato, J. M. Taking gradients through experiments: LSTMs and memory proximal policy optimization for black-box quantum control. International Conference on High Performance Computing, Springer (2018) https://doi.org/10.1007/978-3-030-02465-9_43.
https://doi.org/10.1007/978-3-030-02465-9_43
[38] Porotti R., Essig A., Huard B. & Marquardt F. Deep Reinforcement Learning for Quantum State Preparation with Weak Nonlinear Measurements. Quantum 6, 747 (2022) https://doi.org/10.22331/q-2022-06-28-747.
https://doi.org/10.22331/q-2022-06-28-747
[39] Garcia-Saez A. & Riu J. Quantum observables for continuous control of the quantum approximate optimization algorithm via reinforcement learning. arXiv preprint arXiv:1911.09682 (2019) https://doi.org/10.48550/arXiv.1911.09682.
https://doi.org/10.48550/arXiv.1911.09682
arXiv:1911.09682
[40] Yao J., Bukov M. & Lin, L. Policy gradient based quantum approximate optimization algorithm. In Mathematical and Scientific Machine Learning (pp. 605-634). PMLR (2020) https://doi.org/10.48550/arXiv.2002.01068.
https://doi.org/10.48550/arXiv.2002.01068
[41] Yao J., Lin L., & Bukov M. Reinforcement Learning for Many-Body Ground State Preparation based on Counter-Diabatic Driving. Physical Review X, 11(3), 031070 (2021) https://doi.org/10.1103/PhysRevX.11.031070.
https://doi.org/10.1103/PhysRevX.11.031070
[42] He Z., Li L., Zheng S., Li Y. & Situ H. Variational quantum compiling with double Q-learning. New Journal of Physics, 23(3), 033002 (2021) https://doi.org/10.1088/1367-2630/abe0ae.
https://doi.org/10.1088/1367-2630/abe0ae
[43] Barry, J., Barry, D. T., & Aaronson, S. Quantum partially observable Markov decision processes. Physical Review A, 90(3), 032311 (2014) https://doi.org/10.1103/PhysRevA.90.032311.
https://doi.org/10.1103/PhysRevA.90.032311
[44] Blei D. M., Kucukelbir A. & McAuliffe J. D. Variational inference: A review for statisticians. Journal of the American statistical Association, 112(518), 859-877 (2017) https://doi.org/10.1080/01621459.2017.1285773.
https://doi.org/10.1080/01621459.2017.1285773
[45] Koller D. & Friedman N. Probabilistic graphical models: principles and techniques. MIT press (2009).
[46] Williams R. J. Simple statistical gradient-following algorithms for connectionist reinforcement learning. Machine learning, 8(3), 229-256 (1992) https://doi.org/10.1007/BF00992696.
https://doi.org/10.1007/BF00992696
[47] Cirq, a python framework for creating, editing, and invoking noisy intermediate scale quantum NISQ circuits. https://github.com/quantumlib/Cirq.
https://github.com/quantumlib/Cirq
[48] Shahriari B., Swersky K., Wang Z., Adams R. P. & De Freitas N. Taking the human out of the loop: A review of Bayesian optimization. Proceedings of the IEEE, 104(1), 148-175 (2015) https://doi.org/10.1109/JPROC.2015.2494218.
https://doi.org/10.1109/JPROC.2015.2494218
[49] Colless J. I., Ramasesh V. V., Dahlen D., Blok M. S., Kimchi-Schwartz M. E., McClean, J. R., … & Siddiqi I. Computation of molecular spectra on a quantum processor with an error-resilient algorithm. Physical Review X, 8(1), 011021 (2018) https://doi.org/10.1103/PhysRevX.8.011021.
https://doi.org/10.1103/PhysRevX.8.011021
[50] Barends R., Kelly J., Megrant A., Veitia A., Sank D., Jeffrey E., … & Martinis J. M. Superconducting quantum circuits at the surface code threshold for fault tolerance. Nature, 508(7497), 500-503 (2014) https://doi.org/10.1038/nature13171.
https://doi.org/10.1038/nature13171
[51] Yang C. H., Chan K. W., Harper R., Huang W., Evans T., Hwang J. C. C., … & Dzurak A. S. Silicon qubit fidelities approaching incoherent noise limits via pulse engineering. Nature Electronics, 2(4), 151-158 (2019) https://doi.org/10.1038/s41928-019-0234-1.
https://doi.org/10.1038/s41928-019-0234-1
[52] Huang W., Yang C. H., Chan K. W., Tanttu T., Hensen B., Leon R. C. C., … & Dzurak A. S. Fidelity benchmarks for two-qubit gates in silicon. Nature, 569(7757), 532-536 (2019) https://doi.org/10.1038/s41586-019-1197-0.
https://doi.org/10.1038/s41586-019-1197-0
[53] Schäfer V. M., Ballance C. J., Thirumalai K., Stephenson L. J., Ballance T. G., Steane A. M., & Lucas D. M. Fast quantum logic gates with trapped-ion qubits. Nature, 555(7694), 75-78 (2018) https://doi.org/10.1038/nature25737.
https://doi.org/10.1038/nature25737
[54] Goodfellow I., Bengio Y. & Courville, A. Deep Learning. MIT press (2016).
Cited by
[1] Esther Ye and Samuel Yen-Chi Chen, “Quantum Architecture Search via Continual Reinforcement Learning”, arXiv:2112.05779.
The above citations are from SAO/NASA ADS (last updated successfully 2022-09-12 02:03:07). The list may be incomplete as not all publishers provide suitable and complete citation data.
On Crossref’s cited-by service no data on citing works was found (last attempt 2022-09-12 02:03:06).
This Paper is published in Quantum under the Creative Commons Attribution 4.0 International (CC BY 4.0) license. Copyright remains with the original copyright holders such as the authors or their institutions.
