\documentclass{SciPost} \usepackage[utf8]{inputenc} \usepackage[T1]{fontenc} \usepackage{amsmath,latexsym,graphicx} \usepackage[bitstream-charter]{mathdesign} \usepackage[dvipsnames]{xcolor} \usepackage{anyfontsize,authblk} \usepackage{xfrac} % sfrac \urlstyle{sf} \fancypagestyle{SPstyle}{ \fancyhf{} \lhead{\colorbox{scipostblue}{\bf \color{white} ~SciPost Physics }} \rhead{{\bf \color{scipostdeepblue} ~Submission }} \renewcommand{\headrulewidth}{1pt} \fancyfoot[C]{\textbf{\thepage}} } % Fix \cal and \mathcal characters look (so it's not the same as \mathscr) \DeclareSymbolFont{usualmathcal}{OMS}{cmsy}{m}{n} \DeclareSymbolFontAlphabet{\mathcal}{usualmathcal} \hypersetup{ colorlinks, linkcolor={red!50!black}, citecolor={blue!50!black}, urlcolor={blue!80!black} } \author{\footnote{\url{jaron.kent-dobias@roma1.infn.it}}} \affil{Istituto Nazionale di Fisica Nucleare, Sezione di Roma I, Italy} \begin{document} \pagestyle{SPstyle} \begin{center}{\Large \textbf{\color{scipostdeepblue}{ On the topology of solutions to random continuous constraint satisfaction problems\\ }}}\end{center} \begin{center} \textbf{Jaron Kent-Dobias$^\star$} \end{center} \begin{center} Istituto Nazionale di Fisica Nucleare, Sezione di Roma I, Italy \\[\baselineskip] $\star$ \href{mailto:jaron.kent-dobias@roma1.infn.it}{\small jaron.kent-dobias@roma1.infn.it} \end{center} \section*{\color{scipostdeepblue}{Abstract}} \textbf{\boldmath{% We consider the set of solutions to $M$ random polynomial equations with independent Gaussian coefficients and a target value $V_0$ on the $(N-1)$-sphere. When solutions exist, they form a manifold. We compute the average Euler characteristic of this manifold in the limit of large $N$, and find different behavior depending on the ratio $\alpha=M/N$. When $\alpha<\alpha_\text{onset}$, the average characteristic is 2 and there is a single connected component, while for $\alpha_\text{onset}<\alpha<\alpha_\text{shatter}$ many large connected components coexist. When $\alpha>\alpha_\text{shatter}$ the large connected components vanish, replaced by small fragments, and the entire manifold vanishes for $\alpha>\alpha_\text{\textsc{sat}}$. In the limit $\alpha\to0$ there is a correspondence between this problem and the topology of constant-energy level sets in the spherical spin glasses. We conjecture that the energy $E_\text{shatter}$ associated with the vanishing of the large connected component corresponds to the asymptotic limit of gradient descent from a random initial condition. } } \vspace{\baselineskip} %%%%%%%%%% BLOCK: Copyright information % This block will be filled during the proof stage, and finilized just before publication. % It exists here only as a placeholder, and should not be modified by authors. \noindent\textcolor{white!90!black}{% \fbox{\parbox{0.975\linewidth}{% \textcolor{white!40!black}{\begin{tabular}{lr}% \begin{minipage}{0.6\textwidth}% {\small Copyright attribution to authors. \newline This work is a submission to SciPost Physics. \newline License information to appear upon publication. \newline Publication information to appear upon publication.} \end{minipage} & \begin{minipage}{0.4\textwidth} {\small Received Date \newline Accepted Date \newline Published Date}% \end{minipage} \end{tabular}} }} } %%%%%%%%%% BLOCK: Copyright information %%%%%%%%%% TODO: LINENO % For convenience during refereeing we turn on line numbers: %\linenumbers % You should run LaTeX twice in order for the line numbers to appear. %%%%%%%%%% END TODO: LINENO %%%%%%%%%% TODO: TOC % Guideline: if your paper is longer that 6 pages, include a TOC % To remove the TOC, simply cut the following block \vspace{10pt} \noindent\rule{\textwidth}{1pt} \tableofcontents \noindent\rule{\textwidth}{1pt} \vspace{10pt} %%%%%%%%%% END TODO: TOC \section{Introduction} Constraint satisfaction problems seek configurations that simultaneously satisfy a set of equations, and form a basis for thinking about problems as diverse as neural networks \cite{Mezard_2009_Constraint}, granular materials \cite{Franz_2017_Universality}, ecosystems \cite{Altieri_2019_Constraint}, and confluent tissues \cite{Urbani_2023_A}. All but the last of these examples deal with sets of inequalities, while the last considers a set of equality constraints. Inequality constraints are familiar in situations like zero-cost solutions in neural networks with ReLu activations and stable equilibrium in the forces between physical objects. Equality constraints naturally appear in the zero-gradient solutions to overparameterized smooth neural networks and in vertex models of tissues. In such problems, there is great interest in characterizing structure in the set of solutions, which can be influential in how algorithms behave when trying to solve them \cite{Baldassi_2016_Unreasonable, Baldassi_2019_Properties, Beneventano_2023_On}. Here, we show how \emph{topological} information about the set of solutions can be calculated in a simple model of satisfying random nonlinear equalities. This allows us to reason about the connectivity of this solution set. The topological properties revealed by this calculation yield surprising results for the well-studied spherical spin glasses, where a topological transition thought to occur at a threshold energy $E_\text{th}$ where marginal minima are dominant is shown to occur at a different energy $E_\text{sh}$. We conjecture that this difference resolves an outstanding problem in gradient descent dynamics in these systems. We consider the problem of finding configurations $\mathbf x\in\mathbb R^N$ lying on the $(N-1)$-sphere $\|\mathbf x\|^2=N$ that simultaneously satisfy $M$ nonlinear constraints $V_k(\mathbf x)=V_0$ for $1\leq k\leq M$ and some constant $V_0\in\mathbb R$. The nonlinear constraints are taken to be centered Gaussian random functions with covariance \begin{equation} \label{eq:covariance} \overline{V_i(\mathbf x)V_j(\mathbf x')} =\delta_{ij}f\left(\frac{\mathbf x\cdot\mathbf x'}N\right) \end{equation} for some choice of function $f$. When the covariance function $f$ is polynomial, the $V_k$ are also polynomial, with a term of degree $p$ in $f$ corresponding to all possible terms of degree $p$ in $V_k$. In particular, taking \begin{equation} V_k(\mathbf x) =\sum_{p=0}^\infty\frac1{p!}\sqrt{\frac{f^{(p)}(0)}{N^p}} \sum_{i_1\cdots i_p}^NJ^{(k,p)}_{i_1\cdots i_p}x_{i_1}\cdots x_{i_p} \end{equation} with the elements of the tensors $J^{(k,p)}$ as independently distributed unit normal random variables satisfies \eqref{eq:covariance}. The size of the series coefficients of $f$ therefore control the variances in the coefficients of random polynomial constraints. When $M=1$, this problem corresponds to the level set of a spherical spin glass with energy density $E=\sqrt{N}V_0$. This problem or small variations thereof have attracted attention recently for their resemblance to encryption, optimization, and vertex models of confluent tissues \cite{Fyodorov_2019_A, Fyodorov_2020_Counting, Fyodorov_2022_Optimization, Urbani_2023_A, Kamali_2023_Dynamical, Kamali_2023_Stochastic, Urbani_2024_Statistical, Montanari_2023_Solving, Montanari_2024_On, Kent-Dobias_2024_Conditioning, Kent-Dobias_2024_Algorithm-independent}. In each of these cases, the authors studied properties of the cost function \begin{equation} \label{eq:cost} \mathscr C(\mathbf x)=\frac12\sum_{k=1}^M\big[V_k(\mathbf x)-V_0\big]^2 \end{equation} which achieves zero only for configurations that satisfy all the constraints. Here we dispense with the cost function and study the set of solutions directly. This set can be written as \begin{equation} \Omega=\big\{\mathbf x\in\mathbb R^N\mid \|\mathbf x\|^2=N,V_k(\mathbf x)=V_0 \;\forall\;k=1,\ldots,M\big\} \end{equation} Because the constraints are all smooth functions, $\Omega$ is almost always a manifold without singular points.\footnote{The conditions for a singular point are that $0=\frac\partial{\partial\mathbf x}V_k(\mathbf x)$ for all $k$. This is equivalent to asking that the constraints $V_k$ all have a stationary point at the same place. When the $V_k$ are independent and random, this is vanishingly unlikely, requiring $NM+1$ independent equations to be simultaneously satisfied. This means that different connected components of the set of solutions do not intersect, nor are there self-intersections, without extraordinary fine-tuning.} We study the topology of the manifold $\Omega$ by two related means: its average Euler characteristic, and the average number of stationary points of a linear height function restricted to the manifold. These measures tell us complementary pieces of information. We find that for the varied cases we study, these two always coincide at the largest exponential order in $N$, putting strong constraints on the resulting topology and geometry. \section{The average Euler characteristic} The Euler characteristic $\chi$ of a manifold is a topological invariant \cite{Hatcher_2002_Algebraic}. It is perhaps most familiar in the context of connected compact orientable surfaces, where it characterizes the number of handles in the surface: $\chi=2(1-\#)$ for $\#$ handles. For general $d$, the Euler characteristic of the $d$-sphere is $2$ if $d$ is even and 0 if $d$ is odd. The canonical method for computing the Euler characteristic is done by defining a complex on the manifold in question, essentially a higher-dimensional generalization of a polygonal tiling. Then $\chi$ is given by an alternating sum over the number of cells of increasing dimension, which for 2-manifolds corresponds to the number of vertices, minus the number of edges, plus the number of faces. Morse theory offers another way to compute the Euler characteristic of a manifold $\Omega$ using the statistics of stationary points in a function $H:\Omega\to\mathbb R$ \cite{Audin_2014_Morse}. For functions $H$ without any symmetries with respect to the manifold, the surfaces of gradient flow between adjacent stationary points form a complex. The alternating sum over cells to compute $\chi$ becomes an alternating sum over the count of stationary points of $H$ with increasing index, or \begin{equation} \chi(\Omega)=\sum_{i=0}^N(-1)^i\mathcal N_H(\text{index}=i) \end{equation} Conveniently, we can express this sum as an integral over the manifold using a small variation on the Kac--Rice formula for counting stationary points \cite{Kac_1943_On, Rice_1939_The}. Since the sign of the determinant of the Hessian matrix of $H$ at a stationary point is equal to its index, if we count stationary points including the sign of the determinant, we arrive at the Euler characteristic, or \begin{equation} \label{eq:kac-rice} \chi(\Omega)=\int_\Omega d\mathbf x\,\delta\big(\nabla H(\mathbf x)\big)\det\operatorname{Hess}H(\mathbf x) \end{equation} When the Kac--Rice formula is used to \emph{count} stationary points, the sign of the determinant is a nuisance that one must take pains to preserve \cite{Fyodorov_2004_Complexity}. Here we are correct to exclude it. We need to choose a function $H$ for our calculation. Because $\chi$ is a topological invariant, any choice will work so long as it does not share some symmetry with the underlying manifold, i.e., that $H$ satisfies the Smale condition. Because our manifold of random constraints has no symmetries, we can take a simple height function $H(\mathbf x)=\mathbf x_0\cdot\mathbf x$ for some $\mathbf x_0\in\mathbb R^N$ with $\|\mathbf x_0\|^2=N$. $H$ is a height function because when $\mathbf x_0$ is used as the polar axis, $H$ gives the height on the sphere. We treat the integral over the implicitly defined manifold $\Omega$ using the method of Lagrange multipliers. We introduce one multiplier $\omega_0$ to enforce the spherical constraint and $M$ multipliers $\omega_k$ to enforce the vanishing of each of the $V_k$, resulting in the Lagrangian \begin{equation} \label{eq:lagrangian} L(\mathbf x,\pmb\omega) =H(\mathbf x)+\frac12\omega_0\big(\|\mathbf x\|^2-N\big) +\sum_{k=1}^M\omega_k\big(V_k(\mathbf x)-V_0\big) \end{equation} The integral over $\Omega$ in \eqref{eq:kac-rice} then becomes \begin{equation} \label{eq:kac-rice.lagrange} \chi(\Omega)=\int_{\mathbb R^N} d\mathbf x\int_{\mathbb R^{M+1}}d\pmb\omega \,\delta\big(\partial L(\mathbf x,\pmb\omega)\big) \det\partial\partial L(\mathbf x,\pmb\omega) \end{equation} where $\partial=[\frac\partial{\partial\mathbf x},\frac\partial{\partial\pmb\omega}]$ is the vector of partial derivatives with respect to all $N+M+1$ variables. This integral is now in a form where standard techniques from mean-field theory can be applied to calculate it. To evaluate the average of $\chi$ over the constraints, we first translate the $\delta$ functions and determinant to integral form, with \begin{align} \label{eq:delta.exp} \delta\big(\partial L(\mathbf x,\pmb\omega)\big) &=\int\frac{d\hat{\mathbf x}}{(2\pi)^N}\frac{d\hat{\pmb\omega}}{(2\pi)^{M+1}} e^{i[\hat{\mathbf x},\hat{\pmb\omega}]\cdot\partial L(\mathbf x,\pmb\omega)} \\ \label{eq:det.exp} \det\partial\partial L(\mathbf x,\pmb\omega) &=\int d\bar{\pmb\eta}\,d\pmb\eta\,d\bar{\pmb\gamma}\,d\pmb\gamma\, e^{-[\bar{\pmb\eta},\bar{\pmb\gamma}]^T\partial\partial L(\mathbf x,\pmb\omega)[\pmb\eta,\pmb\gamma]} \end{align} where $\hat{\mathbf x}$ and $\hat{\pmb\omega}$ are ordinary vectors and $\bar{\pmb\eta}$, $\pmb\eta$, $\bar{\pmb\gamma}$, and $\pmb\gamma$ are Grassmann vectors. With these expressions substituted into \eqref{eq:kac-rice.lagrange}, the result is a integral over an exponential whose argument is linear in the random functions $V_k$. These functions can therefore be averaged over, and the resulting expression treated with standard methods. Details of this calculation can be found in Appendix~\ref{sec:euler}. The result is the reduction of the average Euler characteristic to an expression of the form \begin{equation} \label{eq:pre-saddle.characteristic} \overline{\chi(\Omega)} =\left(\frac N{2\pi}\right)^2\int dR\,dD\,dm\,d\hat m\,g(R,D,m,\hat m)\,e^{N\mathcal S_\Omega(R,D,m,\hat m)} \end{equation} where $g$ is a prefactor of $o(N^0)$, and $\mathcal S_\Omega$ is an effective action defined by \begin{equation} \label{eq:euler.action} \begin{aligned} \mathcal S_\Omega(R,D,m,\hat m\mid\alpha,V_0) &=\hat m-\frac\alpha2\left[ \log\left(1+\frac{f(1)D}{f'(1)R^2}\right) +\frac{V_0^2}{f(1)}\left(1+\frac{f'(1)R^2}{f(1)D}\right)^{-1} \right] \\ &\hspace{7em}+\frac12\log\left( 1+\frac{(1-m^2)D+\hat m^2-2Rm\hat m}{R^2} \right) \end{aligned} \end{equation} The remaining order parameters defined by the scalar products \begin{align} R=-i\frac1N\mathbf x\cdot\hat{\mathbf x} && D=\frac1N\hat{\mathbf x}\cdot\hat{\mathbf x} && m=\frac1N\mathbf x\cdot\mathbf x_0 && \hat m=-i\frac1N\hat {\mathbf x}\cdot\mathbf x_0 \end{align} This integral can be evaluated to leading order by a saddle point method. For reasons we will see, it is best to extremize with respect to $R$, $D$, and $\hat m$, leaving a new effective action of $m$ alone. This can be solved to give \begin{equation} D=-\frac{m+R^*}{1-m^2}R^* \qquad \hat m=0 \end{equation} \begin{equation} \begin{aligned} R^*(m) &=\frac{-m(1-m^2)}{2[f(1)-(1-m^2)f'(1)]^2} \Bigg[ \alpha V_0^2f'(1) +(2-\alpha)f(1)\left(\frac{f(1)}{1-m^2}-f'(1)\right) \\ &\quad+\alpha\sqrt{ \tfrac{4V_0^2}\alpha f(1)f'(1)\left[\tfrac{f(1)}{1-m^2}-f'(1)\right] +\left[\tfrac{f(1)^2}{1-m^2}-\big(V_0^2+f(1)\big)f'(1)\right]^2 } \Bigg] \end{aligned} \end{equation} \begin{equation} \mathcal S_\Omega(m) =-\frac\alpha2\bigg[ \log\left( 1-\frac{f(1)}{f'(1)}\frac{1+m/R^*}{1-m^2} \right) +\frac{V_0^2}{f(1)}\left( 1-\frac{f'(1)}{f(1)}\frac{1-m^2}{1+m/R^*} \right)^{-1} \bigg] +\frac12\log\left(-\frac m{R^*}\right) \end{equation} To finish evaluating the integral, this expression should be maximized with respect to $m$. The order parameter $m$ is both physical and interpretable, as it gives the overlap of the configuration $\mathbf x$ with the height axis $\mathbf x_0$. Therefore, the value $m^*$ that maximizes this action can be understood as the latitude on the sphere where most of the contribution to the Euler characteristic is made. The action $\mathcal S_\Omega$ is extremized with respect to $m$ at $m=0$ or $m=\pm m^*=\mp R^*(m^*)$. In the latter case, $m^*$ takes the value \begin{equation} m^*=\sqrt{1-\frac{\alpha}{f'(1)}\big(V_0^2+f(1)\big)} \end{equation} and $\mathcal S_\Omega(m^*)=0$. If this solution was always well-defined, it would vanish when the argument of the square root vanishes for \begin{equation} V_0^2>V_{\text{\textsc{sat}}\ast}^2\equiv\frac{f'(1)}\alpha-f(1) \end{equation} However, when \begin{equation} V_0^2>V_\text{on}^2\equiv\frac{1-\alpha+\sqrt{1-\alpha}}\alpha f(1) \end{equation} $R^*(m^*)$ becomes complex and the solution is no longer valid. Since \begin{equation} V_\text{on}^2-V_{\text{\textsc{sat}}\ast}^2 =\frac{f'(1)-f(1)}\alpha-\frac{\sqrt{1-\alpha}}\alpha f(1) \end{equation} If $f(q)$ is linear, then $f'(1)=f(1)$ and $V_\text{on}^2>V_{\text{\textsc{sat}}\ast}^2$, so the Likewise, when \begin{equation} V_0^20$, and is positive up to $m_\text{max}$. No stationary points are found with overlap less than $m_\text{min}$. \textbf{Right: the shattered regime.} The action is maximized with $\mathcal S_\Omega(0)>0$, and is positive up to $m_\text{max}$. } \end{figure} \paragraph{The connected regime: \boldmath{$V_0^20$ still exists. The minimum overlap indicates that the solution manifold is exclusively made up of extensive components, because the existence of small components would lead to stationary points near the equator with respect to a randomly chosen axis $\mathbf x_0$. The solution manifold is homeomorphic to a topological space with large Euler characteristic like the product of union of many spheres. In the former case we would have one topologically nontrivial connected component, while in the latter we would have many simple disconnected components; the reality could be a combination of the two. It the framework of this calculation, it is not possible to distinguish between these scenarios. In any case, the minima and maxima of the height on the solution manifold are typically found at latitude $m_\text{min}$ but are found in exponential number up to the latitude $m_\text{max}$. \paragraph{The shattered regime: \boldmath{$V_\text{sh}^2