Review ArticleNo Access

Large-dimensional random matrix theory and its applications in deep learning and wireless communications

National Key Laboratory of Science and Technology on Communications, Center for Intelligent Networking and Communications (CINC), University of Electronic Science and Technology of China (UESTC), Chengdu 611731, P. R. China

E-mail Address: gejungang@std.uestc.edu.cn

Search for more papers by this author

Ying-Chang Liang

https://orcid.org/0000-0003-2671-5090

E-mail Address: liangyc@ieee.org

Corresponding author.

Search for more papers by this author

Zhidong Bai

Key Laboratory for Applied Statistics of MOE, School of Mathematics and Statistics, Northeast Normal University, Changchun 130024, P. R. China

E-mail Address: baizd@nenu.edu.cn

Search for more papers by this author

, and

Guangming Pan

School of Physical and Mathematical Sciences, Nanyang Technological University, 637371 Singapore

E-mail Address: gmpan@ntu.edu.sg

Search for more papers by this author

https://doi.org/10.1142/S2010326322300017Cited by:11 (Source: Crossref)

Abstract

Large-dimensional (LD) random matrix theory, RMT for short, which originates from the research field of quantum physics, has shown tremendous capability in providing deep insights into large-dimensional systems. With the fact that we have entered an unprecedented era full of massive amounts of data and large complex systems, RMT is expected to play more important roles in the analysis and design of modern systems. In this paper, we review the key results of RMT and its applications in two emerging fields: wireless communications and deep learning. In wireless communications, we show that RMT can be exploited to design the spectrum sensing algorithms for cognitive radio systems and to perform the design and asymptotic analysis for large communication systems. In deep learning, RMT can be utilized to analyze the Hessian, input–output Jacobian and data covariance matrix of the deep neural networks, thereby to understand and improve the convergence and the learning speed of the neural networks. Finally, we highlight some challenges and opportunities in applying RMT to the practical large-dimensional systems.

Keywords:

AMSC: 15B52, 94A05, 62M45

References

1. B. Adlam, J. Levinson and J. Pennington, A random matrix perspective on mixtures of nonlinearities for deep learning, preprint (2019), arXiv:1912.00827. Google Scholar
2. M. S. Advani, A. M. Saxe and H. Sompolinsky , High-dimensional dynamics of generalization error in neural networks, Neural Netw. 132 (2020) 428–446. Crossref, Google Scholar
3. Z. Bai, Z. Fang and Y.-C. Liang , Spectral Theory of Large Dimensional Random Matrices and Its Applications to Wireless Communications and Finance Statistics : Random Matrix Theory and Its Applications (World Scientific, 2014). Link, Google Scholar
4. Z.-D. Bai and J. W. Silverstein , No eigenvalues outside the support of the limiting spectral distribution of large-dimensional sample covariance matrices, Ann. Probab. 26 (1998) 316–345. Crossref, Google Scholar
5. Z. Bai and J. W. Silverstein , Spectral Analysis of Large Dimensional Random Matrices, Vol. 20 (Springer-Verlag, New York, NY, 2010). Crossref, Google Scholar
6. Z. Bai and J.-F. Yao , Central limit theorems for eigenvalues in a spiked population model, Ann. Inst. H. Poincaré Probab. Statist. 44 (2008) 447–474. Crossref, Google Scholar
7. Z. Bai and J. Yao , On sample eigenvalues in a generalized spiked population model, J. Multivariate Anal. 106 (2012) 167–177. Crossref, Google Scholar
8. J. Baik, G. B. Arous and S. Péché , Phase transition of the largest eigenvalue for nonnull complex sample covariance matrices, Ann. Probab. 33 (2005) 1643–1697. Crossref, Google Scholar
9. J. Baik and J. W. Silverstein , Eigenvalues of large sample covariance matrices of spiked population models, J. Multivariate Anal. 97 (2006) 1382–1408. Crossref, Google Scholar
10. M. Belkin, D. Hsu, S. Ma and S. Mandal , Reconciling modern machine-learning practice and the classical bias–variance trade-off, Proc. Natl. Acad. Sci. USA 116 (2019) 15849–15854. Crossref, Google Scholar
11. M. Belkin, D. Hsu and J. Xu , Two models of double descent for weak features, SIAM J. Math. Data Sci. 2 (2020) 1167–1180. Crossref, Google Scholar
12. M. Belkin, S. Ma and S. Mandal , To understand deep learning we need to understand kernel learning, in Int. Conf. Machine Learning (ICML) (Stockholmsmässan, Stockholm, Sweden, 2018), pp. 541–549. Google Scholar
13. Y. Bengio, A. Courville and P. Vincent , Representation learning: A review and new perspectives, IEEE Trans. Pattern Anal. Mach. Intell. 35 (2013) 1798–1828. Crossref, Google Scholar
14. L. Benigni and S. Péché, Eigenvalue distribution of nonlinear models of random matrices, preprint (2019), arXiv:1904.03090. Google Scholar
15. P. Bianchi, M. Debbah, M. Maïda and J. Najim , Performance of statistical tests for single-source detection using random matrix theory, IEEE Trans. Inform. Theory 57 (2011) 2400–2419. Crossref, Google Scholar
16. P. Bianchi, M. Debbah and J. Najim , Asymptotic independence in the spectrum of the Gaussian unitary ensemble, Electron. Comm. Probab. 15 (2010) 376–395. Crossref, Google Scholar
17. P. Bianchi, J. Najim, M. Maida and M. Debbah , Performance analysis of some eigen-based hypothesis tests for collaborative sensing, in Proc. IEEE Workshop Statistical Signal Processing (Cardiff, UK, 2009), pp. 5–8. Crossref, Google Scholar
18. A. L. Blum and R. L. Rivest , Training a 3-node neural network is NP-complete, Neural Netw. 5 (1992) 117–127. Crossref, Google Scholar
19. K. Bouallegue, I. Dayoub, M. Gharbi and K. Hassan , Blind spectrum sensing using extreme eigenvalues for cognitive radio networks, IEEE Commun. Lett. 22 (2018) 1386–1389. Crossref, Google Scholar
20. A. J. Bray and D. S. Dean , Statistics of critical points of Gaussian fields on large-dimensional spaces, Phys. Rev. Lett. 98 (2007) 150201. Crossref, Google Scholar
21. N. Buduma and N. Locascio , Fundamentals of Deep Learning: Designing Next-generation Machine Intelligence Algorithms (O’Reilly Media, Inc., 2017). Google Scholar
22. D. Cabric, A. Tkachenko and R. W. Brodersen , Spectrum sensing measurements of pilot, energy, and collaborative detection, in Proc. Military Communications Conf. (MILCOM) (Washington, DC, USA, 2006), pp. 1–7. Crossref, Google Scholar
23. L. S. Cardoso, M. Debbah, P. Bianchi and J. Najim , Cooperative spectrum sensing using random matrix theory, in Proc. 3rd Int. Symp. Wireless Pervasive Computing (Santorini, Greece, 2008), pp. 334–338. Crossref, Google Scholar
24. A. L. Caterini and D. E. Chang , Deep Neural Networks in a Mathematical Framework (Springer, 2018). Crossref, Google Scholar
25. A. M. Chan and G. W. Wornell , A class of block-iterative equalizers for intersymbol interference channels: Fixed channel results, IEEE Trans. Commun. 49 (2001) 1966–1976. Crossref, Google Scholar
26. X. Chen, D. W. K. Ng, W. Yu, E. G. Larsson, N. Al-Dhahir and R. Schober, Massive access for 5G and beyond, preprint (2020), arXiv:2002.03491.. Google Scholar
27. M. Chen, J. Pennington and S. Schoenholz , Dynamical isometry and a mean field theory of RNNs: Gating enables signal propagation in recurrent neural networks, in Int. Conf. Machine Learning (ICML) (Stockholmsmässan, Stockholm, Sweden, 2018), pp. 873–882. Google Scholar
28. A. Choromanska, M. Henaff, M. Mathieu, G. B. Arous and Y. LeCun , The loss surfaces of multilayer networks, in Int. Conf. Artificial Intelligence and Statistics (AISTATS) (San Diego, California, USA, 2015), pp. 192–204. Google Scholar
29. R. Couillet and M. Debbah , Random Matrix Methods for Wireless Communications (Cambridge University Press, Cambridge, U.K., 2011). Crossref, Google Scholar
30. R. Couillet, G. Wainrib, H. Sevi and H. T. Ali , The asymptotic performance of linear echo state neural networks, J. Mach. Learn. Res. 17 (2016) 6171–6205. Google Scholar
31. J. Curtiss , On the distribution of the quotient of two chance variables, Ann. Math. Statist. 12 (1941) 409–421. Crossref, Google Scholar
32. M. O. Damen, H. El Gamal and G. Caire , On maximum-likelihood detection and the search for the closest lattice point, IEEE Trans. Inform. Theory 49 (2003) 2389–2402. Crossref, Google Scholar
33. Y. N. Dauphin, R. Pascanu, C. Gulcehre, K. Cho, S. Ganguli and Y. Bengio , Identifying and attacking the saddle point problem in high-dimensional non-convex optimization, in Proc. Advances in Neural Information Processing Systems (NIPS) (Montréal, Canada, 2014), pp. 2933–2941. Google Scholar
34. T. Dupic and I. P. Castillo, Spectral density of products of Wishart dilute random matrices. Part I: The dense case, preprint (2014), arXiv:1401.7802. Google Scholar
35. J. L. Elman , Finding structure in time, Cogn. Sci. 14 (1990) 179–211. Crossref, Google Scholar
36. Z. Fan and Z. Wang, Spectra of the conjugate kernel and neural tangent kernel for linear-width neural networks, preprint (2020), arXiv:2005.11879. Google Scholar
37. O. N. Feldheim and S. Sodin , A universality result for the smallest eigenvalues of certain sample covariance matrices, Geom. Funct. Anal. 20 (2010) 88–123. Crossref, Google Scholar
38. D. Féral and S. Péché , The largest eigenvalues of sample covariance matrices for a spiked population: Diagonal case, J. Math. Phys. 50 (2009) 073302. Crossref, Google Scholar
39. C. D. Freeman and J. Bruna, Topology and geometry of half-rectified network optimization, preprint (2016), arXiv:1611.01540. Google Scholar
40. M. W. Gardner and S. Dorling , Artificial neural networks (the multilayer perceptron) — A review of applications in the atmospheric sciences, Atmos. Environ. 32 (1998) 2627–2636. Crossref, Google Scholar
41. M. Geiger, A. Jacot, S. Spigler, F. Gabriel, L. Sagun, S. d’Ascoli, G. Biroli, C. Hongler and M. Wyart , Scaling description of generalization with number of parameters in deep learning, J. Stat. Mech., Theory Exp. 2020 (2020) 023401. Crossref, Google Scholar
42. S. Geman , A limit theorem for the norm of random matrices, Ann. Probab. 8 (1980) 252–261. Crossref, Google Scholar
43. G. Ginis and J. M. Cioffi , On the relation between V-BLAST and the GDFE, IEEE Commun. Lett. 5 (2001) 364–366. Crossref, Google Scholar
44. D. Granziol, Beyond random matrix theory for deep networks, preprint (2020), arXiv:2006.07721. Google Scholar
45. D. Granziol, T. Garipov, D. Vetrov, S. Zohren, S. Roberts and A. G. Wilson, Towards understanding the true loss surface of deep neural networks using random matrix theory and iterative spectral methods (2019), https://openreview.net/forum?id=H1gza2NtwH. Google Scholar
46. K. Hassan, R. Gautier, I. Dayoub, M. Berbineau and E. Radoi , Multiple-antenna-based blind spectrum sensing in the presence of impulsive noise, IEEE Trans. Veh. Technol. 63 (2013) 2248–2257. Crossref, Google Scholar
47. T. Hastie, A. Montanari, S. Rosset and R. J. Tibshirani, Surprises in high-dimensional ridgeless least squares interpolation, preprint (2019), arXiv:1903.08560. Google Scholar
48. G.-B. Huang, H. Zhou, X. Ding and R. Zhang , Extreme learning machine for regression and multiclass classification, IEEE Trans. Syst. Man Cybern. B, Cybern. 42 (2011) 513–529. Crossref, Google Scholar
49. International Telecommunication Union, IMT traffic estimates for the years 2020 to 2030, Report ITU (2015), https://www.itu.int/dms_pub/itu-r/opb/rep/R-REP-M.2370-2015-PDF-E.pdf. Google Scholar
50. A. Jacot, F. Gabriel and C. Hongler , Neural tangent kernel: Convergence and generalization in neural networks, in Proc. Advances in Neural Information Processing Systems (NIPS) (Montréal, Canada, 2018), pp. 8580–8589. Google Scholar
51. K. Johansson , Shape fluctuations and random matrices, Comm. Math. Phys. 209 (2000) 437–476. Crossref, Google Scholar
52. I. M. Johnstone , On the distribution of the largest eigenvalue in principal components analysis, Ann. Statist. 29 (2001) 295–327. Crossref, Google Scholar
53. N. Kalchbrenner, E. Grefenstette and P. Blunsom, A convolutional neural network for modelling sentences, preprint (2014), arXiv:1404.2188. Google Scholar
54. K. Kawaguchi , Deep learning without poor local minima, in Proc. Advances in Neural Information Processing Systems (NIPS) (Barcelona, Spain, 2016), pp. 586–594. Google Scholar
55. S. M. Kay , Fundamentals of Statistical Signal Processing (Prentice Hall PTR, 1993). Google Scholar
56. A. Krizhevsky, I. Sutskever and G. E. Hinton , ImageNet classification with deep convolutional neural networks, Adv. Neural Inf. Process. Syst. 25 (2012) 1097–1105. Google Scholar
57. Y. LeCun, Y. Bengio and G. Hinton , Deep learning, Nature 521 (2015) 436–444. Crossref, Google Scholar
58. M. Ledoux , The Concentration of Measure Phenomenon (American Mathematical Society, 2001). Google Scholar
59. Y. Li and Y. Liang , Learning overparameterized neural networks via stochastic gradient descent on structured data, in Proc. Advances in Neural Information Processing Systems (NIPS) (Montréal, Canada, 2018), pp. 8168–8177. Google Scholar
60. Y.-C. Liang , Dynamic Spectrum Management: From Cognitive Radio to Blockchain and Artificial Intelligence (Springer, 2020). Crossref, Google Scholar
61. Y.-C. Liang, K.-C. Chen, G. Y. Li and P. Mahonen , Cognitive radio networking and communications: An overview, IEEE Trans. Veh. Technol. 60 (2011) 3386–3407. Crossref, Google Scholar
62. Y.-C. Liang, G. Pan and Z. Bai , Asymptotic performance of MMSE receivers for large systems using random matrix theory, IEEE Trans. Inform. Theory 53 (2007) 4173–4190. Crossref, Google Scholar
63. Y.-C. Liang, S. Sun and C. K. Ho , Block-iterative generalized decision feedback equalizers for large MIMO systems: Algorithm design and asymptotic performance analysis, IEEE Trans. Signal Process. 54 (2006) 2035–2048. Crossref, Google Scholar
64. Y.-C. Liang, Y. Zeng, E. C. Peh and A. T. Hoang , Sensing-throughput tradeoff for cognitive radio networks, IEEE Trans. Wirel. Commun. 7 (2008) 1326–1337. Crossref, Google Scholar
65. Z. Liao and R. Couillet , On the spectrum of random features maps of high dimensional data, in Int. Conf. Machine Learning (ICML) (Stockholmsmässan, Stockholm, Sweden, 2018), pp. 3063–3071. Google Scholar
66. Z. Ling and R. C. Qiu , Spectrum concentration in deep residual learning: A free probability approach, IEEE Access 7 (2019) 105212–105223. Crossref, Google Scholar
67. L. Liu and W. Yu , Massive connectivity with massive MIMO — Part I: Device activity detection and channel estimation, IEEE Trans. Signal Process. 66 (2018) 2933–2946. Crossref, Google Scholar
68. L. Liu and W. Yu , Massive connectivity with massive MIMO — Part II: Achievable rate characterization, IEEE Trans. Signal Process. 66 (2018) 2947–2959. Crossref, Google Scholar
69. C. Louart, Z. Liao and R. Couillet , A random matrix approach to neural networks, Ann. Appl. Probab. 28 (2018) 1190–1248. Crossref, Google Scholar
70. V. A. Marčenko and L. A. Pastur , Distribution of eigenvalues for some sets of random matrices, Math. USSR Sb. 1 (1967) 457. Crossref, Google Scholar
71. S. Mei and A. Montanari, The generalization error of random features regression: Precise asymptotics and double descent curve, preprint (2019), arXiv:1908.05355. Google Scholar
72. T. Mikolov, M. Karafiát, L. Burget, J. Černockỳ and S. Khudanpur , Recurrent neural network based language model, in INTERSPEECH 2010, 11th Annual Conf. Int. Speech Communication Association, Makuhari, Chiba, Japan, 2010, pp. 1045-1048. Crossref, Google Scholar
73. Y. Nesterov , Introductory Lectures on Convex Optimization : A Basic Course, Vol. 87 (Springer Science & Business Media, 2013). Google Scholar
74. T. Neuschel , Plancherel–Rotach formulae for average characteristic polynomials of products of Ginibre random matrices and the Fuss–Catalan distribution, Random Matrices, Theory Appl. 3 (2014) 1450003. Link, Google Scholar
75. S. Oymak and M. Soltanolkotabi , Toward moderate overparameterization: Global convergence guarantees for training shallow neural networks, IEEE J. Sel. Areas Inf. Theory 1 (2020) 84–105. Crossref, Google Scholar
76. L. Pastur, On random matrices arising in deep neural networks. Gaussian case, preprint (2020), arXiv:2001.06188. Google Scholar
77. L. A. Pastur and M. Shcherbina , Eigenvalue Distribution of Large Random Matrices (American Mathematical Society, 2011). Crossref, Google Scholar
78. L. Pastur and V. Slavin, On random matrices arising in deep neural networks: General IID case, preprint (2020), arXiv:2011.11439. Google Scholar
79. F. Penna, R. Garello and M. A. Spirito , Cooperative spectrum sensing based on the limiting eigenvalue ratio distribution in Wishart matrices, IEEE Commun. Lett. 13 (2009) 507–509. Crossref, Google Scholar
80. F. Penna, R. Garello and M. A. Spirito , Probability of missed detection in eigenvalue ratio spectrum sensing, in Proc. IEEE Int. Conf. Wireless and Mobile Computing Networking and Communications (Marrakech, Morocco, 2009), pp. 117–122. Crossref, Google Scholar
81. J. Pennington and Y. Bahri , Geometry of neural network loss surfaces via random matrix theory, in Int. Conf. Machine Learning (ICML) (Sydney, Australia, 2017), pp. 2798–2806. Google Scholar
82. J. Pennington, S. Schoenholz and S. Ganguli , Resurrecting the sigmoid in deep learning through dynamical isometry: Theory and practice, in Proc. Advances in Neural Information Processing Systems (NIPS) (California, USA, 2017), pp. 4785–4795. Google Scholar
83. J. Pennington, S. Schoenholz and S. Ganguli , The emergence of spectral universality in deep networks, Proc. Mach. Learn. Res. 84 (2018) 1924–1932. Google Scholar
84. J. Pennington and P. Worah , Nonlinear random matrix theory for deep learning, in Proc. Advances in Neural Information Processing Systems (NIPS) (California, USA, 2017), pp. 2637–2646. Google Scholar
85. N. Pillay and H. Xu , Blind eigenvalue-based spectrum sensing for cognitive radio networks, IET Commun. 6 (2012) 1388–1396. Crossref, Google Scholar
86. B. Poole, S. Lahiri, M. Raghu, J. Sohl-Dickstein and S. Ganguli , Exponential expressivity in deep neural networks through transient chaos, in Proc. Advances in Neural Information Processing Systems (NIPS) (Barcelona, Spain, 2016), pp. 3360–3368. Google Scholar
87. A. Rahimi and B. Recht , Random features for large-scale kernel machines, in Proc. Advances in Neural Information Processing Systems (NIPS) (Vancouver, British Columbia, Canada, 2007), pp. 1177–1184. Google Scholar
88. D. E. Rumelhart, G. E. Hinton and R. J. Williams , Learning representations by back-propagating errors, Nature 323 (1986) 533–536. Crossref, Google Scholar
89. I. Safran and O. Shamir , On the quality of the initial basin in overspecified neural networks, in Int. Conf. Machine Learning (ICML) (New York, USA, 2016), pp. 774–782. Google Scholar
90. A. M. Saxe, J. L. McClelland and S. Ganguli, Exact solutions to the nonlinear dynamics of learning in deep linear neural networks, preprint (2013), arXiv:1312.6120. Google Scholar
91. J. Schmidhuber , Deep learning in neural networks: An overview, Neural Netw. 61 (2015) 85–117. Crossref, Google Scholar
92. S. S. Schoenholz, J. Gilmer, S. Ganguli and J. Sohl-Dickstein, Deep information propagation, preprint (2016), arXiv:1611.01232. Google Scholar
93. M. E. A. Seddik, C. Louart, M. Tamaazousti and R. Couillet , Random matrix theory proves that deep learning representations of GAN-data behave as Gaussian mixtures, in Int. Conf. Machine Learning (ICML) (Vienna, Austria, 2020), pp. 8573–8582. Google Scholar
94. J. W. Silverstein , The smallest eigenvalue of a large dimensional Wishart matrix, Ann. Probab. 13 (1985) 1364–1368. Crossref, Google Scholar
95. J. W. Silverstein and Z. Bai , On the empirical distribution of eigenvalues of a class of large dimensional random matrices, J. Multivariate Anal. 54 (1995) 175–192. Crossref, Google Scholar
96. A. Sonnenschein and P. M. Fishman , Radiometric detection of spread-spectrum signals in noise of uncertain power, IEEE Trans. Aerosp. Electron. Syst. 28 (1992) 654–660. Crossref, Google Scholar
97. R. Tandra and A. Sahai , Fundamental limits on detection in low SNR under noise uncertainty, in Proc. IEEE Int. Conf. Wireless Networks, Communications and Mobile Computing (Maui, HI, USA, 2005), pp. 464–469. Crossref, Google Scholar
98. T. Tao , Topics in Random Matrix Theory (American Mathematical Society, Providence, RI, 2012). Crossref, Google Scholar
99. C. A. Tracy and H. Widom , On orthogonal and symplectic matrix ensembles, Comm. Math. Phys. 177 (1996) 727–754. Crossref, Google Scholar
100. C. A. Tracy and H. Widom , The distribution of the largest eigenvalue in the Gaussian ensembles: $β = 1$ , 2, 4, in Calogero — Moser — Sutherland Models (Springer, 2000), pp. 461–472. Crossref, Google Scholar
101. D. N. C. Tse and S. V. Hanly , Linear multiuser receivers: Effective interference, effective bandwidth and user capacity, IEEE Trans. Inform. Theory 45 (1999) 641–657. Crossref, Google Scholar
102. D. N. C. Tse and O. Zeitouni , Linear multiuser receivers in random environments, IEEE Trans. Inform. Theory 46 (2000) 171–188. Crossref, Google Scholar
103. A. M. Tulino and S. Verdú , Random Matrix Theory and Wireless Communications (Now Publishers, 2004). Crossref, Google Scholar
104. S. Verdú and S. Shamai , Spectral efficiency of CDMA with random spreading, IEEE Trans. Inform. Theory 45 (1999) 622–640. Crossref, Google Scholar
105. H. Vikalo, B. Hassibi and U. Mitra , Sphere-constrained ML detection for frequency-selective channels, IEEE Trans. Commun. 54 (2006) 1179–1183. Crossref, Google Scholar
106. S. Wagner, R. Couillet, M. Debbah and D. T. Slock , Large system analysis of linear precoding in correlated MISO broadcast channels under limited feedback, IEEE Trans. Inform. Theory 58 (2012) 4509–4537. Crossref, Google Scholar
107. E. P. Wigner , Characteristic vectors of bordered matrices with infinite dimensions, Ann. Math. 62 (1955) 548–564. Crossref, Google Scholar
108. E. P. Wigner , On the distribution of the roots of certain symmetric matrices, Ann. Math. 67 (1958) 325–327. Crossref, Google Scholar
109. J. Wishart , The generalised product moment distribution in samples from a normal multivariate population, Biometrika 20 (1928) 32–52. Crossref, Google Scholar
110. L. Xiao, Y. Bahri, J. Sohl-Dickstein, S. Schoenholz and J. Pennington , Dynamical isometry and a mean field theory of CNNs: How to train 10,000-layer vanilla convolutional neural networks, in Int. Conf. Machine Learning (ICML) (Stockholmsmässan, Stockholm, Sweden, 2018), pp. 5393–5402. Google Scholar
111. L. Xiao, J. Pennington and S. Schoenholz , Disentangling trainability and generalization in deep neural networks, in Int. Conf. Machine Learning (ICML) (Vienna, Austria, 2020), pp. 10462–10472. Google Scholar
112. G. Yang and H. Salman, A fine-grained spectral perspective on neural networks, preprint (2019), arXiv:1907.10599. Google Scholar
113. Y. Yao, L. Rosasco and A. Caponnetto , On early stopping in gradient descent learning, Constr. Approx. 26 (2007) 289–315. Crossref, Google Scholar
114. Y.-Q. Yin, Z.-D. Bai and P. R. Krishnaiah , On the limit of the largest eigenvalue of the large dimensional sample covariance matrix, Probab. Theory Relat. Fields 78 (1988) 509–521. Crossref, Google Scholar
115. W. Zaremba, I. Sutskever and O. Vinyals, Recurrent neural network regularization, preprint (2014), arXiv:1409.2329. Google Scholar
116. Y. Zeng, C. L. Koh and Y.-C. Liang , Maximum eigenvalue detection: Theory and application, in Proc. IEEE International Conference on Communications (ICC) (Beijing, China, 2008), pp. 4160–4164. Crossref, Google Scholar
117. Y. Zeng and Y.-C. Liang , Eigenvalue-based spectrum sensing algorithms for cognitive radio, IEEE Trans. Commun. 57 (2009) 1784–1793. Crossref, Google Scholar
118. Y. Zeng, Y.-C. Liang, A. T. Hoang and R. Zhang , A review on spectrum sensing for cognitive radio: Challenges and solutions, EURASIP J. Adv. Signal Process. 2010 (2010) 1–15. Crossref, Google Scholar
119. W. Zhang, G. Abreu, M. Inamori and Y. Sanada , Spectrum sensing algorithms via finite random matrices, IEEE Trans. Commun. 60 (2011) 164–175. Crossref, Google Scholar
120. C. Zhang, S. Bengio, M. Hardt, B. Recht and O. Vinyals, Understanding deep learning requires rethinking generalization, preprint (2016), arXiv:1611.03530. Google Scholar
121. R. Zhang, T. J. Lim, Y.-C. Liang and Y. Zeng , Multi-antenna based spectrum sensing for cognitive radios: A GLRT approach, IEEE Trans. Commun. 58 (2010) 84–88. Crossref, Google Scholar
122. Z. Zhang and G. Pan, Tracy–Widom law for the extreme eigenvalues of large signal-plus-noise matrices, preprint (2020), arXiv:2009.12031. Google Scholar
123. W. Zhang, C.-X. Wang, X. Tao and P. Patcharamaneepakorn , Exact distributions of finite random matrices and their applications to spectrum sensing, Sensors 16 (2016) 1183. Crossref, Google Scholar
124. Z. Zhang, S. Zheng, G. Pan and P. Zhong, Asymptotic independence of spiked eigenvalues and linear spectral statistics for large sample covariance matrices, preprint (2020), arXiv:2009.11010. Google Scholar
125. H. Zhu and G. B. Giannakis , Exploiting sparse user activity in multiuser detection, IEEE Trans. Commun. 59 (2010) 454–465. Crossref, Google Scholar