\documentclass[aps,pre,reprint,longbibliography,floatfix]{revtex4-2} \usepackage[utf8]{inputenc} \usepackage[T1]{fontenc} \usepackage{amsmath,amssymb,latexsym,graphicx} \usepackage{newtxtext,newtxmath} \usepackage{bbold,anyfontsize} \usepackage[dvipsnames]{xcolor} \begin{document} \title{ Conditioning the complexity of random landscapes on marginal optima } \author{Jaron Kent-Dobias} \affiliation{Istituto Nazionale di Fisica Nucleare, Sezione di Roma I, Rome, Italy 00184} \begin{abstract} Marginal optima are minima or maxima of a function with many asymptotically flat directions. In settings with many competing optima, marginal ones tend to attract algorithms and physical dynamics. Often, the important family of marginal attractors are a vanishing minority compared with nonmarginal optima and other unstable stationary points. We introduce a generic technique for conditioning the statistics of stationary points on their marginality, and apply it in three isotropic settings with different typical forms for the Hessian at optima: in the spherical spin-glasses, where the Hessian is GOE; in a multispherical spin glasses, which are Gaussian but non-GOE; and in a model of random nonlinear sum of squares, which is non-Gaussian. In these problems we are able to fully characterize the distribution of marginal optima in the landscape, including when they are in the minority. \end{abstract} \maketitle \section{Introduction} Systems with rugged landscapes are important across many disciplines, from the physics to glasses and spin-glasses to the statistical inference problems. The behavior of these systems is best understood when equilibrium or optimal solutions are studied and averages can be taken statically over all possible configurations. However, such systems are also infamous for their tendency to defy equilibrium and optimal expectations in practice, due to the presence of dynamic transitions or crossovers that leave physical or algorithmic dynamics stuck exploring only a subset of configurations. In some simple models of such landscapes, it was recently found that marginal minima are significant as the attractors of gradient descent dynamics \cite{Folena_2020_Rethinking, Folena_2023_On}. This extends to more novel algorithms, like message passing \cite{Add_me} \textbf{Find out if this is true}. \textbf{Think of other examples.} While it is still not known how to predict which marginal minima will be attractors, this ubiquity of behavior suggests that cartography of marginal minima is a useful step in bounding out-of-equilibrium dynamical behavior. In the traditional methods for analyzing the geometric structure of rugged landscapes, it is not necessarily straightforward to condition an analysis on the marginality of minima. Using the method of a Legendre transformation of the Parisi parameter corresponding to a set of real replicas, one can force the result to be marginal by restricting the value of that parameter, but this results in only the marginal minima at the energy level at which they are the majority of stationary points \cite{Monasson_1995_Structural}. It is now understood that out-of-equilibrium dynamics usually goes to marginal minima at other energy levels \cite{Folena_2023_On}. The alternative, used to great success in the spherical models, is to start by making a detailing understanding of the Hessian matrix at stationary points. Then, one can condition the analysis on whatever properties of the Hessian are necessary to lead to marginal minima. This strategy is so successful in the spherical models because it is very straightforward to implement: a natural parameter in the analysis of these models linearly shifts the spectrum of the Hessian, and so fixing this parameter by whatever means naturally allows one to require that the Hessian spectrum have a pseudogap. Unfortunately this strategy is less straightforward to generalize. Many models of interest, especially in inference problems, have Hessian statistics that are poorly understood. Here, we introduce a generic method for conditioning the statistics of stationary points on their marginality. The technique makes use of a novel way to condition an integral over parameters to select only those that result in a certain value of the smallest eigenvalue of a matrix that is a function of those parameters. By requiring that the smallest eigenvalue of the Hessian at stationary points be zero, we restrict to marginal minima, either those with a pseudogap in their bulk spectrum or those with outlying eigenvectors. We provide a heuristic to distinguish these two cases. We demonstrate the method on the spherical models, where it is unnecessary but instructive, and on extensions of the spherical models with non-GOE Hessians where the technique is more useful. \section{Conditioning on the smallest eigenvalue} An arbitrary function $g$ of the minimum eigenvalue of a matrix $A$ can be expressed as \begin{equation} \label{eq:λmin} g(\lambda_\textrm{min}(A)) =\lim_{\beta\to\infty}\int \frac{d\mathbf s\,\delta(N-\mathbf s^T\mathbf s)e^{-\beta\mathbf s^TA\mathbf s}} {\int d\mathbf s'\,\delta(N-\mathbf s'^T\mathbf s')e^{-\beta\mathbf s'^TA\mathbf s'}} g\left(\frac{\mathbf s^TA\mathbf s}N\right) \end{equation} Assuming \begin{equation} \begin{aligned} &\lim_{\beta\to\infty}\int\frac{ d\mathbf s\,\delta(N-\mathbf s^T\mathbf s)e^{-\beta\mathbf s^TA\mathbf s} }{ \int d\mathbf s'\,\delta(N-\mathbf s'^T\mathbf s')e^{-\beta\mathbf s'^TA\mathbf s'} }g\left(\frac{\mathbf s^TA\mathbf s}N\right) \\ &=\int\frac{ d\mathbf s\,\delta(N-\mathbf s^T\mathbf s)\mathbb 1_{\operatorname{ker}(A-\lambda_\mathrm{min}(A)I)}(\mathbf s) }{ \int d\mathbf s'\,\delta(N-\mathbf s'^T\mathbf s')\mathbb 1_{\operatorname{ker}(A-\lambda_\mathrm{min}(A)I)}(\mathbf s')}g\left(\frac{\mathbf s^TA\mathbf s}N\right) \\ &=g(\lambda_\mathrm{min}(A)) \frac{\int d\mathbf s\,\delta(N-\mathbf s^T\mathbf s)\mathbb 1_{\operatorname{ker}(A-\lambda_\mathrm{min}(A)I)}(\mathbf s)}{\int d\mathbf s'\,\delta(N-\mathbf s'^T\mathbf s')\mathbb 1_{\operatorname{ker}(A-\lambda_\mathrm{min}(A)I)}(\mathbf s')} \\ &=g(\lambda_\mathrm{min}(A)) \end{aligned} \end{equation} The first relation extends a technique first introduced in \cite{Ikeda_2023_Bose-Einstein-like} and used in \cite{Kent-Dobias_2024_Arrangement}. A Boltzmann distribution is introduced over a spherical model whose Hamiltonian is quadratic with interaction matrix given by $A$. In the limit of zero temperature, the measure will concentrate on the ground states of the model, which correspond with the eigenspace of $A$ associated with its minimum eigenvalue $\lambda_\mathrm{min}$. The second relation uses the fact that, once restricted to the sphere $\mathbf s^T\mathbf s=N$ and the minimum eigenspace, $\mathbf s^TA\mathbf s=N\lambda_\mathrm{min}(A)$. The relationship is formal, but we can make use of the fact that the integral expression with a Gibbs distribution can be manipulated with replica techniques, averaged over, and in general treated with a physicist's toolkit. In particular, we have specific interest in using $g(\lambda_\mathrm{min}(A))=\delta(\lambda_\mathrm{min}(A))$, a Dirac delta-function, which can be inserted into averages over ensembles of matrices $A$ (or indeed more complicated averages) in order to condition that the minimum eigenvalue is zero. \subsection{Simple example: shifted GOE} We demonstrate the efficacy of the technique by rederiving a well-known result: the large-deviation function for pulling an eigenvalue from the bulk of the GOE spectrum. Consider an ensemble of $N\times N$ matrices $A=B+\mu I$ for $B$ drawn from the GOE ensemble with entries whose variance is $\sigma^2/N$. We know that the bulk spectrum of $A$ is a Wigner semicircle with radius $2\sigma$ shifted by a constant $\mu$. Therefore, for $\mu=2\sigma$, the minimum eigenvalue will typically be zero, while for $\mu>2\sigma$ the minimum eigenvalue would need to be a large deviation from the typical spectrum and its likelihood will be exponentially suppressed with $N$. For $\mu<2\sigma$, the bulk of the typical spectrum contains zero and therefore a larger $N^2$ deviation, moving an extensive number of eigenvalues, would be necessary. This final case cannot be quantified by this method, but instead the nonexistence of a large deviation linear in $N$ appears as the emergence of an imaginary part in the function. As an example, we compute \begin{equation} \label{eq:large.dev} e^{NG_\lambda^*(\mu)} =P_{\lambda_\mathrm{min}(B+\mu I)=\lambda^*} =\overline{\delta\big(N\lambda^*-N\lambda_\mathrm{min}(B+\mu I)\big)} \end{equation} where the overline is the average over $B$, and we have defined the large deviation function $G_\sigma(\mu)$. Using the representation of $\lambda_\mathrm{min}$ defined in \eqref{eq:λmin}, we have \begin{widetext} \begin{equation} e^{NG_{\lambda^*}(\mu)} =\overline{ \lim_{\beta\to\infty}\int\frac{d\mathbf s\,\delta(N-\mathbf s^T\mathbf s)e^{-\beta\mathbf s^T(B+\mu I)\mathbf s}} {\int d\mathbf s'\,\delta(N-\mathbf s'^T\mathbf s')e^{-\beta\mathbf s'^T(B+\mu I)\mathbf s'}}\,\delta\big(N\lambda^*-\mathbf s^T(B+\mu I)\mathbf s\big) } \end{equation} Using replicas to treat the denominator ($x^{-1}=\lim_{n\to0}x^{n-1}$) and transforming the $\delta$-function to its Fourier representation, we have \begin{equation} e^{NG_{\lambda^*}(\mu)} =\overline{\lim_{\beta\to\infty}\lim_{n\to0}\int d\hat\lambda\prod_{a=1}^n\left[d\mathbf s_a\,\delta(N-\mathbf s_a^T\mathbf s_a)\right] \exp\left\{-\beta\sum_{a=1}^n\mathbf s_a^T(B+\mu I)\mathbf s_a+\hat\lambda\left[N\lambda^*-\mathbf s_1^T(B+\mu I)\mathbf s_1\right]\right\}} \end{equation} having introduced the parameter $\hat\lambda$ in the Fourier representation of the $\delta$-function. The whole expression, so transformed, is a simple exponential integral linear in the matrix $B$. Taking the average over $B$, we have \begin{equation} \begin{aligned} &e^{NG_{\lambda^*}(\mu)} =\lim_{\beta\to\infty}\lim_{n\to0}\int d\hat\lambda\prod_{a=1}^n\left[d\mathbf s_a\,\delta(N-\mathbf s_a^T\mathbf s_a)\right] \\ &\hspace{10em}\exp\left\{N\left[\hat\lambda(\lambda^*-\mu)-n\beta\mu\right]+\frac{\sigma^2}{N}\left[\beta^2\sum_{ab}^n(\mathbf s_a^T\mathbf s_b)^2 +2\beta\hat\lambda\sum_a^n(\mathbf s_a^T\mathbf s_1)^2 +\hat\lambda^2N^2 \right]\right\} \end{aligned} \end{equation} We make the Hubbard--Stratonovich transformation to the matrix field $Q_{ab}=\frac1N\mathbf s_a^T\mathbf s_b$. This gives \begin{equation} e^{NG_{\lambda^*}(\mu)} =\lim_{\beta\to\infty}\lim_{n\to0}\int d\hat\lambda\,dQ\, \exp N\left\{ \hat\lambda(\lambda^*-\mu)-n\beta\mu+\sigma^2\left[\beta^2\sum_{ab}^nQ_{ab}^2 +2\beta\hat\lambda\sum_a^nQ_{1a}^2 +\hat\lambda^2 \right]+\frac12\log\det Q\right\} \end{equation} \end{widetext} where $Q_{aa}=1$ because of the spherical constraint. We can evaluate this integral using the saddle point method. We make a replica symmetric ansatz for $Q$, because this is a 2-spin model, but with the first row singled out because of its unique coupling with $\hat\lambda$. This gives \begin{equation} \label{eq:Q.structure} Q=\begin{bmatrix} 1&\tilde q_0&\tilde q_0&\cdots&\tilde q_0\\ \tilde q_0&1&q_0&\cdots&q_0\\ \tilde q_0&q_0&1&\ddots&q_0\\ \vdots&\vdots&\ddots&\ddots&\vdots\\ \tilde q_0&q_0&q_0&\cdots&q_0 \end{bmatrix} \end{equation} with $\sum_{ab}Q_{ab}^2=n+2(n-1)\tilde q_0^2+(n-1)(n-2)q_0^2$, $\sum_aQ_{1a}^2=1+(n-1)\tilde q_0^2$, and \begin{equation} \log\det Q=(n-2)\log(1-q_0)+\log(1+(n-2)q_0-(n-1)\tilde q_0^2) \end{equation} Inserting these expressions and taking the limit of $n$ to zero, we find \begin{equation} e^{NG_{\lambda^*}(\mu)}=\lim_{\beta\to\infty}\int d\hat\lambda\,dq_0\,d\tilde q_0\,e^{N\mathcal S_\beta(q_0,\tilde q_0,\hat\lambda)} \end{equation} with the effective action \begin{equation} \begin{aligned} &\mathcal S_\beta(q_0,\tilde q_0,\hat\lambda) \\ &\quad=\hat\lambda(\lambda^*-\mu)+\sigma^2\left[ 2\beta^2(q_0^2-\tilde q_0^2)+2\beta\hat\lambda(1-\tilde q_0^2)+\hat\lambda^2 \right] \\ &\qquad-\log(1-q_0)+\frac12\log(1-2q_0+\tilde q_0^2) \end{aligned} \end{equation} We need to evaluate the integral above using the saddle point method, but in the limit of $\beta\to\infty$. We expect the overlaps to concentrate on one as $\beta$ goes to infinity. We therefore take \begin{align} q_0&=1-y\beta^{-1}-z\beta^{-2}+O(\beta^{-3}) \\ \tilde q_0&=1-\tilde y\beta^{-1}-(z+\Delta z)\beta^{-2}+O(\beta^{-3}) \end{align} However, taking the limit with $y\neq\tilde y$ results in an expression for the action that diverges with $\beta$. To cure this, we must take $\tilde y=y$. The result is \begin{equation} \begin{aligned} \mathcal S_\infty(y,\Delta z,\hat\lambda) &=\hat\lambda(\lambda^*-\mu) +\sigma^2\big[ \hat\lambda^2-4(y+\Delta z) \big] \\ &\qquad+\frac12\log\left(1+\frac{2\Delta z}{y^2}\right) \end{aligned} \end{equation} Extremizing this action over the new parameters $y$, $\Delta z$, and $\hat\lambda$, we have \begin{align} \hat\lambda=-\frac1\sigma\sqrt{\frac{(\mu+\lambda^*)^2}{(2\sigma)^2}-1} \\ y=\frac1{2\sigma}\left(\frac{\mu+\lambda^*}{2\sigma}-\sqrt{\frac{(\mu+\lambda^*)^2}{(2\sigma)^2}-1}\right) &\\ \Delta z=\frac1{4\sigma^2}\left(1-\frac{\mu+\lambda^*}{2\sigma}\left(\frac{\mu+\lambda^*}{2\sigma}-\sqrt{\frac{(\mu+\lambda^*)^2}{(2\sigma)^2}-1}\right)\right) \end{align} Inserting this solution into $\mathcal S_\infty$ we find \begin{equation} \label{eq:goe.large.dev} \begin{aligned} &G_{\lambda^*}(\mu) =\mathop{\textrm{extremum}}_{y,\Delta z,\hat\lambda}\mathcal S_\infty(y,\Delta z,\hat\lambda) \\ &=-\tfrac{\mu+\lambda^*}{2\sigma}\sqrt{\Big(\tfrac{\mu+\lambda^*}{2\sigma}\Big)^2-1} +\log\left( \tfrac{\mu+\lambda^*}{2\sigma}+\sqrt{\Big(\tfrac{\mu+\lambda^*}{2\sigma}\Big)^2-1} \right) \end{aligned} \end{equation} This function is plotted in Fig.~\ref{fig:large.dev} for $\lambda^*=0$. For $\mu<2\sigma$ $G_{0}(\mu)$ has an imaginary part. This indicates that the existence of a marginal minimum for this parameter value corresponds with a large deviation that grows faster than $N$, rather like $N^2$, since in this regime the bulk of the typical spectrum is over zero and therefore extensively many eigenvalues have to have large deviations in order for the smallest eigenvalue to be zero. For $\mu\geq2\sigma$ this function gives the large deviation function for the probability of seeing a zero eigenvalue given the shift $\mu$. $\mu=2\sigma$ is the maximum of the function with a real value, and corresponds to the intersection of the average spectrum with zero, i.e., a pseudogap. \begin{figure} \includegraphics[width=\columnwidth]{figs/large_deviation.pdf} \caption{ The large deviation function $G_\sigma(\mu)$ defined in \eqref{eq:large.dev} as a function of the shift $\mu$ to the GOE diagonal. As expected, $G_\sigma(2\sigma)=0$, while for $\mu>2\sigma$ it is negative and for $\mu<2\sigma$ it gains an imaginary part. } \label{fig:large.dev} \end{figure} Marginal spectra with a pseudogap and those with simple isolated eigenvalues are qualitatively different, and more attention may be focused on the former. Here, we see what appears to be a general heuristic for identifying the saddle parameters for which the spectrum is pseudogapped: the equivalent of this large-deviation functions will lie on the singular boundary between a purely real and complex value. \subsection{Conditioning on a pseudogap} We have seen that this method effectively conditions a random matrix ensemble on its lowest eigenvalue being zero. However, this does not correspond on its own to marginal minima. In the previous example, most values of $\mu$ where the calculation was valid correspond to matrices with a single isolated eigenvalue. However, the marginal minima we are concerned with have pseudogapped spectra, where the continuous part of the spectral density has a lower bound at zero. Fortunately, our calculation can be modified to ensure that we consider only pseudogapped spectra. First, we insert a shift $\mu$ by hand into the `natural' spectrum of the problem at hand, conditioning the trace to have a specific value $\mu=\operatorname{Tr}A$. Then, we choose this artificial shift so that the resulting conditioned spectra are pseudogapped. As seen the previous subsection, this can be done by starting from a sufficiently large $\mu$ and decreasing it until the calculation develops an imaginary part, signaling the breakdown of the large-deviation principle at order $N$. In isotropic or zero-signal landscapes, there is another way to condition on a pseudogap. In such landscapes, the typical spectrum does not have an isolated eigenvalue. Therefore, the condition associated with the bulk of the spectrum touching zero, i.e., the pseudogap, will always correspond to the most common configuration. We can therefore choose $\mu=\mu_\textrm m$ such that \begin{equation} 0=\frac\partial{\partial\lambda^*}G_{\lambda^*}(\mu_\mathrm m)\bigg|_{\lambda^*=0} \end{equation} In the previous problem, this corresponds precisely to $\mu_\mathrm m=2\sigma$, the correct marginal shift. Note that when we treat the Dirac $\delta$ function using its Fourier representation with auxiliary parameter $\hat\lambda$, as in the previous subsection, this condition corresponds with choosing $\mu$ such that $\hat\lambda=0$. \section{Marginal complexity in random landscapes} The situation in the study of random landscapes is often as follows: an ensemble of smooth functions $H:\mathbb R^N\to\mathbb R$ define random landscapes, often with their configuration space subject to one or more constraints of the form $g(\mathbf x)=0$ for $\mathbf x\in\mathbb R^N$. The geometry of such landscapes is studied by their complexity, or the average logarithm of the number of stationary points with certain properties, e.g., of marginal minima at a given energy. Such problems can be studied using the method of Lagrange multipliers, with one introduced for every constraint. If the configuration space is defined by $r$ constraints, then the problem is to extremize \begin{equation} H(\mathbf x)+\sum_{i=1}^r\omega_ig_i(\mathbf x) \end{equation} with respect to $\mathbf x$ and $\pmb\omega=\{\omega_1,\ldots,\omega_r\}$. The corresponding gradient and Hessian for the problem are \begin{align} \nabla H(\mathbf x,\pmb\omega)=\partial H(\mathbf x)+\sum_{i=1}^r\omega_i\partial g_i(\mathbf x) \\ \operatorname{Hess}H(\mathbf x,\pmb\omega)=\partial\partial H(\mathbf x)+\sum_{i=1}^r\omega_i\partial\partial g_i(\mathbf x) \end{align} The number of stationary points in a landscape for a particular realization $H$ is found by integrating over the Kac--Rice measure \begin{equation} d\mu_H(\mathbf x,\pmb\omega)=d\mathbf x\,d\pmb\omega\,\delta\big(\nabla H(\mathbf x,\pmb\omega)\big)\,\delta\big(\mathbf g(\mathbf x)\big)\,\big|\det\operatorname{Hess}H(\mathbf x,\pmb\omega)\big| \end{equation} with a $\delta$-function of the gradient and the constraints ensuring that we count valid stationary points, and the Hessian entering in the determinant as the Jacobian of the argument to the $\delta$-function. It is usually more interesting to condition the count on interesting properties of the stationary points, like the energy and spectrum trace, \begin{equation} \begin{aligned} &d\mu_H(\mathbf x,\pmb\omega\mid E,\mu) \\ &\quad=d\mu_H(\mathbf x,\pmb\omega)\, \delta\big(NE-H(\mathbf x)\big) \,\delta\big(N\mu-\operatorname{Tr}\operatorname{Hess}H(\mathbf x,\pmb\omega)\big) \end{aligned} \end{equation} We further want to control the value of the minimum eigenvalue of the Hessian at the stationary points. Using the method introduced above, we can write the number of stationary points with energy $E$, Hessian trace $\mu$, and smallest eigenvalue $\lambda^*$ as \begin{widetext} \begin{equation} \begin{aligned} &\mathcal N_H(E,\mu,\lambda^*) =\int d\mu_H(\mathbf x,\pmb\omega\mid E,\mu)\,\delta\big(N\lambda^*-\lambda_\mathrm{min}(\operatorname{Hess}H(\mathbf x,\pmb\omega))\big) \\ &=\lim_{\beta\to\infty}\int d\mu_H(\mathbf x,\pmb\omega\mid E,\mu) \frac{d\mathbf s\,\delta(N-\mathbf s^T\mathbf s)\delta(\mathbf s^T\partial\mathbf g(\mathbf x))e^{-\beta\mathbf s^T\operatorname{Hess}H(\mathbf x,\pmb\omega)\mathbf s}} {\int d\mathbf s'\,\delta(N-\mathbf s'^T\mathbf s')\delta(\mathbf s'^T\partial\mathbf g(\mathbf x))e^{-\beta\mathbf s'^T\operatorname{Hess}H(\mathbf x,\pmb\omega)\mathbf s'}} \delta\big(N\lambda^*-\mathbf s^T\operatorname{Hess}H(\mathbf x,\pmb\omega)\mathbf s\big) \end{aligned} \end{equation} where the $\delta$-functions \begin{equation} \delta(\mathbf s^T\partial\mathbf g(\mathbf x)) =\prod_{s=1}^r\delta(\mathbf s^T\partial g_i(\mathbf x)) \end{equation} ensure that the integrals are constrained to the tangent space of the configuration manifold at the point $\mathbf x$. This likewise allows us to define the complexity of points with a specific energy, stability, and minimum eigenvalue as \begin{equation} \Sigma_{\lambda^*}(E,\mu) =\frac1N\overline{\log\mathcal N_H(E,\mu,\lambda^*)} \end{equation} In practice, this can be computed by introducing replicas to treat the logarithm ($\log x=\lim_{n\to0}\frac\partial{\partial n}x^n$) and replicating again to treat each of the normalizations in the numerator. This leads to the expression \begin{equation} \label{eq:min.complexity.expanded} \begin{aligned} \Sigma_{\lambda^*}(E,\mu) &=\lim_{\beta\to\infty}\lim_{n\to0}\frac1N\frac\partial{\partial n}\int\prod_{a=1}^n\Bigg[d\mu_H(\mathbf x_a,\pmb\omega_a\mid E,\mu)\,\delta\big(N\lambda^*-(\mathbf s_a^1)^T\operatorname{Hess}H(\mathbf x_a,\pmb\omega_a)\mathbf s_a^1\big)\\ &\hspace{12em}\times\lim_{m_a\to0} \left(\prod_{b=1}^{m_a} d\mathbf s_a^b \,\delta\big(N-(\mathbf s_a^b)^T\mathbf s_a^b\big) \,\delta\big((\mathbf s_a^b)^T\partial\mathbf g(\mathbf x_a)\big) \,e^{-\beta(\mathbf s_a^b)^T\operatorname{Hess}H(\mathbf x_a,\pmb\omega_a)\mathbf s_a^b}\right) \Bigg] \end{aligned} \end{equation} \end{widetext} for the complexity of stationary points of a given energy, trace, and smallest eigenvalue. Finally, the \emph{marginal} complexity is given by fixing $\mu=\mu_\text{m}$ so that the complexity is stationary with respect to changes in the value of the minimum eigenvalue, or \begin{equation} 0=\frac\partial{\partial\lambda^*}\Sigma_{\lambda^*}(E,\mu_\text{m}(E))\bigg|_{\lambda^*=0} \end{equation} Finally, the marginal complexity is defined by evaluating the complexity conditioned on $\lambda_{\text{min}}=0$ at $\mu=\mu_\text{m}(E)$, \begin{equation} \Sigma_\text{m}(E) =\Sigma_0(E,\mu_\text m(E)) \end{equation} \section{Examples} \subsection{Spherical spin glasses} The spherical spin glasses are a family of models that encompass every isotropic Gaussian field on the hypersphere $0=\mathbf x^T\mathbf x-N$ for $\mathbf x\in\mathbb R^N$. One can consider the models as defined by centered Gaussian functions $H$ such that the covariance between two points in the configuration space is \begin{equation} \overline{H(\mathbf x)H(\mathbf x')}=Nf\left(\frac{\mathbf x^T\mathbf x'}N\right) \end{equation} for some function $f$ with positive series coefficients. Such functions can be considered to be made up of all-to-all tensorial interactions, with \begin{equation} H(\mathbf x) =\sum_{p=0}^\infty\frac{\sqrt{f^{(p)}(0)}}{2N^{p-1}}J_{i_1\cdots i_p}x_{i_1}\cdots x_{i_p} \end{equation} and the elements of the tensors $J$ being independently distributed with the unit normal distribution. The marginal optima of these models can be studied without the methods described here, and have been in the past \cite{Folena_2020_Rethinking, Kent-Dobias_2023_How}. First, these models are Gaussian, so at large $N$ the Hessian is statistically independent of the gradient and energy \cite{Bray_2007_Statistics}. Therefore, conditioning the Hessian can be done mostly independently from the problem of counting stationary points. Second, in these models the Hessian at every point in the landscape belongs to the GOE class with the same width of the spectrum $\mu_\mathrm m=2\sqrt{f''(1)}$. Therefore, all marginal optima in these systems have the same constant shift $\mu=\pm\mu_\mathrm m$. Despite the fact the complexity of marginal optima is well known by simpler methods, it is instructive to carry through the calculation for this case, since we will something about its application in more nontrivial settings. The procedure to treat the complexity of the spherical models has been made in detail elsewhere \cite{Kent-Dobias_2023_How}. Here we will merely sketch the steps that are standard. We start by translating elements of the Kac--Rice measure into terms more familiar to physicists. This means writing \begin{align} \delta\big(\nabla H(\mathbf x_a,\pmb\omega_a)\big) &=\int\frac{d\hat{\mathbf x}_a}{(2\pi)^N}e^{i\hat{\mathbf x}_a^T\nabla H(\mathbf x_a,\pmb\omega_a)} \\ \delta\big(NE-H(\mathbf x_a)\big) &=\int\frac{d\hat\beta_a}{2\pi}e^{\hat\beta_a(NE-H(\mathbf x_a))} \\ \delta\big(N\lambda^*-\mathbf s^T\operatorname{Hess}H(\mathbf x_a,\pmb\omega)\mathbf s\big) &=\int\frac{d\hat\lambda_a}{2\pi}e^{\hat\lambda_a(N\lambda^*-\mathbf s^T\operatorname{Hess}H(\mathbf x_a,\pmb\omega)\mathbf s)} \end{align} for the Dirac $\delta$ functions. At this point we will also discuss an important step we will use repeatedly in this paper: to drop the absolute value signs around the determinant in the Kac--Rice measure. This can potentially lead to severe problems with the complexity. However, it is a justified step when the parameters of the problem, i.e., $E$, $\mu$, and $\lambda^*$ put us in a regime where the exponential majority of stationary points have the same index. This is true for maxima and minima, and for saddle points whose spectra have a strictly positive bulk with a fixed number of negative outliers. Dropping the absolute value sign allows us to write \begin{equation} \det\operatorname{Hess}H(\mathbf x_a, \pmb\omega_a) =\int d\pmb\eta_a\,d\bar{\pmb\eta}_a\,e^{\bar{\pmb\eta}_a^T\operatorname{Hess}H(\mathbf x_a,\pmb\omega)\pmb\eta_a} \end{equation} for $N$-dimensional Grassmann variables $\bar{\pmb\eta}_a$ and $\pmb\eta_a$. For the spherical models this step is unnecessary, since there are other ways to treat the determinant keeping the absolute value signs, as in previous works \cite{Folena_2020_Rethinking, Kent-Dobias_2023_How}. However, since other of our examples are for models where the same techniques are impossible, it is useful to see the fermionic method in action in this simple case. Once these substitutions have been made, the entire expression \eqref{eq:min.complexity.expanded} is an exponential integral whose argument is a linear functional of $H$. This allows for the average to be taken over the disorder. If we gather all the $H$-dependant pieces into the linear functional $\mathcal O$ then the average gives \begin{equation} \begin{aligned} \overline{ e^{\sum_a^n\mathcal O_aH(\mathbf x_a)} } &=e^{\frac12\sum_a^n\sum_b^n\mathcal O_a\mathcal O_b\overline{H(\mathbf x_a)H(\mathbf x_b)}} \\ &=e^{N\frac12\sum_a^n\sum_b^n\mathcal O_a\mathcal O_bf\big(\frac{\mathbf x_a^T\mathbf x_b}N\big)} \end{aligned} \end{equation} The result is an integral that only depends on the many vector variables we have introduced through their scalar products with each other. We therefore make a change of variables in the integration from those vectors to matrices that encode their possible scalar products. These matrices are \begin{align} C_{ab}=\frac1N\mathbf x_a\cdot\mathbf x_b && R_{ab}=-i\frac1N\mathbf x_a\cdot\hat{\mathbf x}_b \\ D_{ab}=\frac1N\hat{\mathbf x}_a\cdot\hat{\mathbf x}_b && F_{ab}=\frac1N\bar{\pmb\eta}_a^T\pmb\eta_b \\ A_{ab}^{cd}=\frac1N\mathbf s_a^c\cdot\mathbf s_b^d && X^c_{ab}=\frac1N\mathbf x_a\cdot\mathbf s_b^c \\ \hat X^c_{ab}=\frac1N\hat{\mathbf x}_a\cdot\mathbf s_b^c \end{align} Order parameters that mix the normal and Grassmann variables generically vanish in these settings \cite{Kurchan_1992_Supersymmetry}. After these steps, which follow identically to those more carefully outlined in the cited papers \cite{Folena_2020_Rethinking, Kent-Dobias_2023_How}, we arrive at a form of the integral as over an effective action \begin{equation} \begin{aligned} &\Sigma_{\lambda^*}(E,\mu) =\lim_{\beta\to\infty}\lim_{n\to0}\frac1N\frac\partial{\partial n} \int dC\,dR\,dD\,dF \\ &dA\,dX\,d\hat X\, d\hat\beta\,d\hat\lambda\,e^{N n\mathcal S_\mathrm{KR}(\hat\beta,\omega,C,R,D,F) +N\mathcal S_\beta(\omega,\hat\lambda,A,X,\hat X) +\frac12N\log\det J } \end{aligned} \end{equation} where the matrix $J$ is the Jacobian associated with the change of variables from the $\mathbf x$, $\hat{\mathbf x}$, and $\mathbf s$, and has the form \begin{equation} \label{eq:coordinate.jacobian} J=\begin{bmatrix} C&iR&X^1&\cdots&X^n \\ iR&D&i\hat X^1&\cdots&i\hat X^m\\ (X^1)^T&i(\hat X^1)^T&A^{11}&\cdots&A^{1n}\\ \vdots&\vdots&\vdots&\ddots&\vdots\\ (X^n)^T&i(\hat X^n)^T&A^{n1}&\cdots&A^{nn} \end{bmatrix} \end{equation} The structure of the integrand, with the effective action split between two terms which only share a dependence on the Lagrange multiplier $\omega$ that enforces the constraint, is generic to Gaussian problems. This is the appearance in practice of the fact mentioned before that conditions on the Hessian do not mostly effect the rest of the complexity problem. \begin{widetext} \begin{equation} \mathcal S_\mathrm{KR} =\frac12\sum_{ab}\left( \hat\beta_a\hat\beta_bf(C_{ab}) +\big(2\hat\beta_a(R_{ab}-F_{ab})-D_{ab}\big)f'(C_{ab}) +(R_{ab}^2-F_{ab}^2)f''(C_{ab}) \right) -\log\det F \end{equation} \begin{equation} \mathcal S_\beta =\sum_{ab}^n\left[\beta\omega A_{aa}^{bb}+\hat x\omega A_{aa}^{11}+\beta^2f''(1)\sum_{cd}^m(A_{ab}^{cd})^2+\hat x^2f''(1)(A_{ab}^{11})^2+\beta\hat xf''(1)\sum_c^m A_{ab}^{1c}\right] \end{equation} \end{widetext} There are some dramatic simplifications that emerge from the structure of this particular problem. First, notice that (outside of the `volume' term due to $J$) the dependence on the parameters $X$ and $\hat X$ are purely quadratic. Therefore, there will always be a saddle point condition where they are both zero. In this case, we except this solution to be correct. We can reason about why this is so: $X$, for instance, quantifies the correlation between the typical position of stationary points and the direction of their typical eigenvectors. In an isotropic landscape, where no direction is any more important than any other, we don't expect such correlations to be nonzero: where a state is location does not give any information as to the orientation of its soft directions. On the other hand, in the spiked case, or with an external field, the preferred direction can polarize both the direction of typical stationary points \emph{and} their soft eigenvectors. Therefore, in these instances one must account for solutions with nonzero $X$ and $\hat X$. When the $X$ and $\hat X$ order parameters are zero, as they are here, the term associated with the Jacobian separates into two terms, one dependent only on the order parameters of the traditional complexity problem $C$, $R$, and $D$, and one dependent only on the overlap of the minimum eigenvector, $A$. Now we see that, outside of the Lagrange multiplier $\omega$, the Kac--Rice complexity problem and the problem of fixing the smallest eigenvalue completely decouple. \begin{equation} \Sigma_{\lambda^*}(E,\mu) =\Sigma(E,\mu)+G_{\lambda^*}(\mu) \end{equation} where $G$ is precisely the function \eqref{eq:goe.large.dev} we found in the case of a GOE matrix added to an identity, with $\sigma=\sqrt{f''(1)}$. We find the marginal complexity by solving \begin{equation} 0 =\frac\partial{\partial\lambda^*}\Sigma_{\lambda^*}(E,\mu_\mathrm m(E))\bigg|_{\lambda^*=0} =\frac\partial{\partial\lambda^*}G_{\lambda^*}(\mu_\mathrm m(E))\bigg|_{\lambda^*=0} \end{equation} which gives $\mu_m(E)=2\sqrt{f''(1)}$ independent of $E$, as we presaged above. Since $G_0(\mu_\mathrm m)=0$, this gives finally \begin{equation} \Sigma_\mathrm m(E) =\Sigma_0(E,\mu_\mathrm m(E)) =\Sigma(E,\mu_\mathrm m) \end{equation} that the marginal complexity in these models is simply the ordinary complexity evaluated at a fixed trace of the Hessian. \subsection{Twin spherical spin glasses} $\Omega=S^{N-1}\times S^{N-1}$ \begin{equation} H(\pmb\sigma)=H_1(\pmb\sigma^{(1)})+H_2(\pmb\sigma^{(2)})+\epsilon\pmb\sigma^{(1)}\cdot\pmb\sigma^{(2)} \end{equation} \begin{equation} \overline{H_s(\pmb\sigma_1)H_s(\pmb\sigma_2)} =Nf_s\left(\frac{\pmb\sigma_1\cdot\pmb\sigma_2}N\right) \end{equation} \begin{widetext} \begin{equation} \mathcal S(C,R,D,W,\hat\beta,\omega) =\frac12\frac1n \sum_{ab}\left( \hat\beta^2f(C_{ab})+(2\hat\beta R_{ab}-D_{ab})f'(C_{ab})+(R_{ab}^2-W_{ab}^2)f''(C_{ab}) \right) \end{equation} \begin{equation} \begin{aligned} &\mathcal S(C^{11},R^{11},D^{11},W^{11},\hat\beta)+\mathcal S(C^{22},R^{22},D^{22},W^{22},\hat\beta) -\epsilon(r_{12}+r_{21})-\omega_1(r^{11}_d-w^{11}_d)-\omega_2(r^{22}_d-w^{22}_d)+\hat\beta E \\ &+\frac12\log\det\begin{bmatrix}C^{11}&iR^{11}\\iR^{11}&D^{11}\end{bmatrix} +\frac12\log\det\left( \begin{bmatrix}C^{22}-q_{12}^2C^{11}&iR^{22}\\iR^{22}&D^{22}\end{bmatrix} \right) -\log\det(W^{11}W^{22}+W^{12}W^{21}) \end{aligned} \end{equation} \begin{equation} \begin{aligned} &\sum_a^n\left[\hat q_a(Q^{11}_{aa}+Q^{22}_{aa}-1)-\beta(\omega_1Q^{11}_{aa}+\omega_2Q^{22}_{aa}+2\epsilon Q^{12}_{aa})\right] +\lambda(\omega_1Q^{11}_{11}+\omega_2Q^{22}_{11}+2\epsilon Q^{12}_{11}) \\ &+\sum_{i=1,2}f_i''(1)\left[\beta^2\sum_{ab}^n(Q^{ii}_{ab})^2-2\beta\lambda\sum_a^n(Q^{ii}_{1a})^2+\lambda^2(Q^{ii}_{11})^2\right] +\frac12\log\det\begin{bmatrix} Q^{11}&Q^{12}\\ Q^{12}&Q^{22} \end{bmatrix} \end{aligned} \end{equation} \end{widetext} \begin{equation} \log\det\begin{bmatrix} Q^{11}&Q^{12}\\ Q^{12}&Q^{22} \end{bmatrix} +\log\det(Q^{11}Q^{22}-Q^{12}Q^{12}) \end{equation} \subsection{Random nonlinear least squares} In this subsection we consider perhaps the simplest example of a non-Gaussian landscape: the problem of random nonlinear least squares optimization. Though, for reasons we will see it is easier to make predictions for random nonlinear \emph{most} squares, i.e., the problem of maximizing the sum of squared terms. We also take a spherical problem with $\mathbf x\in S^{N-1}$, and consider a set of $M$ random functions $V_k:\mathbf S^{N-1}\to\mathbb R$ that are centered Gaussians with covariance \begin{equation} \overline{V_i(\mathbf x)V_j(\mathbf x')}=\delta_{ij}f\left(\frac{\mathbf x^T\mathbf x'}N\right) \end{equation} The energy or cost function is the sum of squares of the $V_k$, or \begin{equation} H(\mathbf x)=\frac12\sum_{k=1}^MV_k(\mathbf x)^2 \end{equation} The landscape complexity and large deviations of the ground state for this problem were recently studied in a linear context, with $f(q)=\sigma^2+aq$ \cite{Fyodorov_2020_Counting, Fyodorov_2022_Optimization}. Some results on the ground state of the general nonlinear problem can also be found in \cite{Tublin_2022_A}. In particular, that work indicates that the low-lying minima of the problem tend to be either replica symmetric or full replica symmetry breaking. This is not good news for our analysis or marginal states, because in the former case the problem is typically easy to solve, and in the latter the analysis becomes much more technically challenging. \cite{Urbani_2023_A, Kamali_2023_Dynamical, Kamali_2023_Stochastic, Urbani_2024_Statistical} \cite{Montanari_2023_Solving, Montanari_2024_On} \cite{Subag_2020_Following} Fortunately, the \emph{maxima} of this problem have a more amenable structure for study, as they are typically described by 1-RSB like structure. There is a heuristic intuition for this: in the limit of $M\to1$, this problem is just the square of a spherical spin glass landscape. The distribution and properties of stationary points low and high in the spherical spin glass are not changed, except that their energies are stretched and minima are transformed into maxima. This is why the top of the landscape doesn't qualitatively change. The bottom, however, consists of the zero-energy level set in the spherical spin glass. This level set is well-connected, and so the ground states should also be well connected and flat. Focusing on the top of the landscape and therefore dealing with a 1-RSB like problem is good for our analysis. First, algorithms will tend to be stuck in the ways they are for hard optimization problems, and second we will be able to explicitly predict where. Therefore, we will study the most squares problem rather than the least squares one. We calculate the complexity of maxima under a replica symmetric ansatz (which covers 1-RSB like problems) for arbitrary covariance $f$, and then the marginal complexity. Applying the Lagrange multiplier method detailed above to enforce the spherical constraint, the gradient and Hessian are \begin{align} \nabla H(\mathbf x,\omega)=\sum_k^MV_k(\mathbf x)\partial V_k(\mathbf x)+\omega\mathbf x \\ \operatorname{Hess}H(\mathbf x,\omega)=\partial V_k(\mathbf x)\partial V_k(\mathbf x)+V_k(\mathbf x)\partial\partial V_k(\mathbf x)+\omega I \end{align} \begin{widetext} The number of stationary points in a circumstance where the determinants add constructively is \begin{equation} \begin{aligned} &\mathcal N(E,\mu)^n =\int\prod_{a=1}^nd\mathbf x_a\frac{d\hat{\mathbf x}_a}{(2\pi)^N}d\omega_a\,d\hat\beta_a\,\hat\mu_a\,d\bar\eta_a\,d\eta_a\,\exp\bigg\{ i\hat{\mathbf x}_a^T(V^k(\mathbf x_a)\partial V^k(\mathbf x_a)+\omega\mathbf x_a) +\hat\beta(NE-\frac12V^k(\mathbf x_a)V^k(\mathbf x_a)) \\ & +\bar\eta_a^T(\partial V^k(\mathbf x_a)\partial V^k(\mathbf x_a)^T+V^k(\mathbf x_a)\partial\partial V^k(\mathbf x_a)+\omega_a I)\eta_a +\hat\mu_a(N\mu-\partial V^k(\mathbf x_a)^T\partial V^k(\mathbf x_a)-V^k(\mathbf x_a)\operatorname{Tr}\partial\partial V^k(\mathbf x_a)-N\omega_a) \bigg\} \end{aligned} \end{equation} To linearize the argument of the exponential with respect to $V$, we define the following new fields: $w^k_a=V^k(\mathbf x_a)$ and $\mathbf v^k_a=\partial V^k(\mathbf x_1)$. Inserting these in $\delta$ functions, we have \begin{equation} \begin{aligned} &\mathcal N(E,\mu)^n =\int\prod_{a=1}^nd\mathbf x_a\frac{d\hat{\mathbf x}_a}{(2\pi)^N}d\omega_a\,d\hat\beta_a\,\hat\mu_a\,d\bar\eta_a\,d\eta_a\,\exp\bigg\{ i\hat{\mathbf x}_a^T(w^k_a\mathbf v^k_a+\omega\mathbf x_a) +\hat\beta(NE-\frac12w^k_aw^k_a) \\ & +\bar\eta_a^T(\mathbf v^k_a(\mathbf v^k_a)^T+w^k_a\partial\partial V^k(\mathbf x_a)+\omega_a I)\eta_a +\hat\mu_a(N\mu-(\mathbf v^k_a)^T\mathbf v^k_a-w^k_a\operatorname{Tr}\partial\partial V^k(\mathbf x_a)-N\omega_a) \\ & +i\hat w^k_a(w^k_a-V^k(\mathbf x_a)) +i(\hat{\mathbf v}^k_a)^T(\mathbf v^k_a-\partial V^k(\mathbf x_a)) \bigg\} \end{aligned} \end{equation} which is now linear in $V$. Averaging over $V$ yields, from only the terms that depend on it and to highest order in $N$, \begin{equation} -\frac12\left( f(C_{ab})\hat w^k_a\hat w^k_b +2f'(C_{ab})\hat w^k_a\frac{\mathbf x^T_a\hat{\mathbf v}^k_b}N +f'(C_{ab})\frac{(\hat{\mathbf v}^k_a)^T\hat{\mathbf v}^k_b}N +f''(C_{ab})\left(\frac{\mathbf x_a^T\hat{\mathbf v}^k_b}N\right)^2 +f''(C_{ab})w^k_aw^k_bG_{ab}^2 \right) \end{equation} The resulting integrand is Gaussian in the $w$, $\hat w$, $\mathbf y$, and $\hat{\mathbf y}$, with \begin{equation} \exp\left\{ -\frac12\sum_{k=1}^M\sum_{ab}^n\begin{bmatrix}w_a^k\\\mathbf v_a^k\\\hat w_a^k\\\hat{\mathbf v}_a^k\end{bmatrix}^T \begin{bmatrix} \hat\beta_a\delta_{ab}+G_{ab}^2f''(C_{ab}) & -i\hat{\mathbf x}_a^T\delta_{ab} & -i\delta_{ab} & 0 \\ -i\hat{\mathbf x}_a\delta_{ab} & 2(\hat\mu_a I-\bar\eta_a\eta_a^T)\delta_{ab} & 0 & -i\delta_{ab}I\\ -i\delta_{ab} & 0 & f(C_{ab}) & \frac1Nf'(C_{ab})\mathbf x_a^T \\ 0 & -i\delta_{ab}I & \frac1Nf'(C_{ab})\mathbf x_b & \frac1Nf'(C_{ab})I+\frac1{N^2}f''(C_{ab})\mathbf x_a\mathbf x_b^T \end{bmatrix} \begin{bmatrix}w_b^k\\\mathbf v_b^k\\\hat w_b^k\\\hat{\mathbf v}_b^k\end{bmatrix} \right\} \end{equation} which produces \begin{equation} \exp\left\{ \frac M2\log\det\left( I+\begin{bmatrix} \hat\beta_a\delta_{ac}+G_{ac}^2f''(C_{ac}) & -i\hat{\mathbf x}_a^T\delta_{ac} \\ -i\hat{\mathbf x}_a\delta_{ac} & 2(\hat\mu_a I-\bar\eta_a\eta_a^T)\delta_{ac} \end{bmatrix} \begin{bmatrix} f(C_{cb})&\frac1Nf'(C_{cb})\mathbf x_c^T \\ \frac1Nf'(C_{cb})\mathbf x_b & \frac1Nf'(C_{cb})I+\frac1{N^2}f''(C_{cb})\mathbf x_c\mathbf x_b^T \end{bmatrix} \right) \right\} \end{equation} \begin{equation} \begin{bmatrix} (\hat\beta_a\delta_{ac}+G_{ac}^2f''(C_{ac}))f(C_{cb}) + R_{ab}f'(C_{ab}) & \frac1N\left[(\hat\beta_a\delta_{ac}+G_{ac}^2f''(C_{ac}))f'(C_{cb})+R_{ab}f''(C_{ab})\right]\mathbf x_b^T-\frac1Nif'(C_{ab})\hat{\mathbf x}_a^T \\ -i\hat{\mathbf x}_af(C_{ab})+\frac1N\hat\mu f'(C_{ab})\mathbf x_b & -i\frac1Nf'(C_{ab})\hat{\mathbf x}_a\mathbf x_b^T +2\frac1N(\hat\mu_aI-\bar{\pmb\eta}_a\pmb\eta_a^T)f'(C_{ab}) +\frac2{N^2}\hat\mu_af''(C_{ab})\mathbf x_a\mathbf x_b^T \end{bmatrix} \end{equation} Here we already see that the terms dependent on $\hat\mu$ will be smaller by a factor of $N$ than those not. Therefore we can drop these terms safely at leading order in $N$. We treat this determinant by using block form, which gives two contributions \begin{equation} \begin{aligned} &\log\det\left[ \delta_{ab}+(\hat\beta_a\delta_{ac}+G_{ac}^2f''(C_{ac}))f(C_{cb}) + R_{ab}f'(C_{ab}) \right] \\ &\log\det\left( I\delta_{ab} -2\frac1N\bar{\pmb\eta}_a\pmb\eta_a^Tf'(C_{ab}) -\frac1Ni\hat{\mathbf x}_aB_{ab}\mathbf x_b^T-\frac1N\hat{\mathbf x}_af'(C_{ab})\hat{\mathbf x}_b^T \right) \end{aligned} \end{equation} \[ B=f'(C)+f(C)A^{-1} \left[(\hat\beta I+G\odot G\odot f''(C))f'(C)+R\odot f''(C)\right] \] \[ \det B_{ab}\det\begin{bmatrix} I&\frac1N\begin{bmatrix}\hat{\mathbf x}_a&\hat{\mathbf x}_a&\bar{\pmb\eta}_a\end{bmatrix} \\ \begin{bmatrix}i\mathbf x_b^T\\\hat{\mathbf x}_b^T\\\pmb\eta_b^T\end{bmatrix} & \begin{bmatrix} B_{ab} & 0 & 0\\ 0 & f'(C_{ab}) & 0 \\ 0 & 0 & f'(C_{ab}) \end{bmatrix}^{-1} \end{bmatrix} \] \[ \det\left( I- \frac1N\begin{bmatrix} B_{ab} & 0\\ 0 & f'(C_{ab}) \end{bmatrix} \begin{bmatrix}i\mathbf x_b^T\\\hat{\mathbf x}_b^T\end{bmatrix} \begin{bmatrix}\hat{\mathbf x}_a&\hat{\mathbf x}_a\end{bmatrix} \right) \det\left( I-\begin{bmatrix}0&f'(C_{ab})\\f'(C_{ab})&0\end{bmatrix}\begin{bmatrix}\bar{\pmb\eta}_a^T&\pmb\eta_a^T\end{bmatrix} \begin{bmatrix}\bar{\pmb\eta}_b\\\pmb\eta_b\end{bmatrix} \right)^{-1} \] \[ \det\left( I- \begin{bmatrix} B & 0\\ 0 & f'(C) \end{bmatrix} \begin{bmatrix} -R&-R\\D&D \end{bmatrix} \right) \det\left( I-\begin{bmatrix}0&-f'(C)\\f'(C)&0\end{bmatrix} \begin{bmatrix}0&-G\\G&0\end{bmatrix} \right)^{-1} =\det\left( \begin{bmatrix} 1+B\odot R&B\odot R\\-f'(C)\odot D&1-f'(C)\odot D \end{bmatrix} \right) \det\left( \begin{bmatrix}1+f'(C)\odot G&0\\0&1+f'(C)\odot G\end{bmatrix} \right)^{-1} \] \[ \det A\det\left[ I+B\odot R-f'(C)\odot D \right] =\det[ (I-f'(C)\odot D)A +A(f'(C)\odot R) +f(C) \left[(\hat\beta I+G\odot G\odot f''(C))f'(C)+R\odot f''(C)\right] ] \] \begin{equation} \begin{aligned} &\mathcal S =-\frac1n\frac\alpha2\left\{\log\det\left[ \hat\beta f(C)+\Big( f'(C)\odot D+(G\odot G-R\odot R)\odot f''(C) \Big)f(C) +(I+R\odot f'(C))^2 \right]-\log\det(I+G\odot f'(C))^2\right\} \\ &+\frac1n\frac12\Big(\log\det(CD+R^2)-\log\det G^2\Big) +\hat\beta E+(g_d-r_d)\mu \end{aligned} \end{equation} where $\odot$ gives the Hadamard or componentwise product between the matrices, while other products and powers are matrix products and powers. \begin{equation} \begin{aligned} &\hat\beta E+\mu(g_d-r_d)+\frac12\log\frac{d_d+r_d^2}{g_d^2} \\ &-\frac\alpha2\log\left[ 1+\hat\beta\big(f(1)-f(0)\big) \Big(d_d\big(f(1)-f(0)\big)+r_d\big(2+r_df'(1)\big)\Big)f'(1) +(g_d^2-r_d^2)\big(f(1)-f(0)\big)f''(1) \right] \\ &-\alpha f(0)\left( \big(f(1)-f(0)\big)+\frac{1+r_d\big(2+r_df'(1)\big)f'(1)}{\hat\beta+d_df'(1)+(g_d^2-r_d^2)f''(1)} \right)^{-1} \end{aligned} \end{equation} In the case where $\mu$ is not specified, in which the model is supersymmetric, $D=\hat\beta R$ and the effective action becomes particularly simple: \begin{equation} \hat\beta e -\frac12\frac{\alpha f(0)}{1+\hat\beta\big(f(1)-f(0)\big)+r_df'(1)} -\frac\alpha2\log\left(1+\frac{\hat\beta\big(f(1)-f(0)\big)}{1+r_df'(1)}\right) +\frac12\log\frac{\hat\beta+r_d}{r_d} \end{equation} \cite{DeWitt_1992_Supermanifolds} Consider supervectors in the $\mathbb R^{N|4}$ superspace of the form \begin{equation} \pmb\phi_{a\alpha}(1,2) =\mathbf x_a +\bar\theta_1\pmb\eta_a+\bar{\pmb\eta}_a\theta_1 +i\hat{\mathbf x}_a\bar\theta_1\theta_1 +\mathbf s_{a\alpha}(\bar\theta_1\theta_2+\bar\theta_2\theta_1) \end{equation} The Kac--Rice measure with the eigenvalue-fixing term included is \begin{equation} \begin{aligned} \mathcal N(E,\mu,\lambda^*)^n &=\int\prod_{a=1}^n\prod_{\alpha=1}^{m_a}d\pmb\phi_{a\alpha} \exp\left\{ \delta_{\alpha1}N(\hat\beta_aE+\hat\lambda_a\lambda^*) +\int d1\,d2\,B_{a\alpha}(1,2)\left[H(\pmb\phi_{a\alpha})+\frac12\mu(\|\pmb\phi_{a\alpha}\|^2-N)\right] \right\} \end{aligned} \end{equation} \begin{equation} B_{a\alpha}(1,2)=\delta_{\alpha1}\bar\theta_2\theta_2 (1-\hat\beta_a\bar\theta_1\theta_1) -\delta_{\alpha1}\hat\lambda_a-\beta \end{equation} \begin{align} d\pmb\phi_{a\alpha} =d\mathbf x_a\,\delta(\|\mathbf x_a\|^2-N)\,\frac{d\hat{\mathbf x}_a}{(2\pi)^N}\,d\pmb\eta_a\,d\bar{\pmb\eta}_a\, d\mathbf s_{a\alpha}\,\delta(\|\mathbf s_{a\alpha}\|^2-N)\, \delta(\mathbf x_a^T\mathbf s_{a\alpha}) \end{align} \begin{equation} i\int d1\,d2\,\hat v_{a\alpha}^k(1,2)(V^k(\pmb\phi_{a\alpha}(1,2))-v_{a\alpha}^k(1,2)) \end{equation} \begin{equation} -\sum_{ab}\sum_{\alpha\gamma}\sum_k\frac12\int d1\,d2\,d3\,d4\, \hat v_{a\alpha}^kf\big(\pmb\phi_{a\alpha}(1,2)^T\pmb\phi_{b\gamma}(3,4)\big)\hat v_{b\gamma}^k \end{equation} We're now quadratic in the $v$ and $\hat v$ with the kernel \begin{equation} \begin{bmatrix} B_{a\alpha}(1,2)\delta(1,3)\delta(2,4)\delta_{ab}\delta_{\alpha\gamma} & i\delta(1,3)\,\delta(2,4) \delta_{ab}\delta_{\alpha\gamma}\\ i\delta(1,3)\,\delta(2,4) \delta_{ab}\delta_{\alpha\gamma}& f\big(\pmb\phi_{a\alpha}(1,2)^T\pmb\phi_{b\gamma}(3,4)\big) \end{bmatrix} \end{equation} Upon integration, this results in a term in the effective action of the form \begin{equation} -\frac M2\log\operatorname{sdet}\left( \delta(1,3)\,\delta(2,4) \delta_{ab}\delta_{\alpha\gamma} +B_{a\alpha}(1,2)f\big(\pmb\phi_{a\alpha}(1,2)^T\pmb\phi_{b\gamma}(3,4)\big) \right) \end{equation} When expanded, this supermatrix is constructed of the scalar products of the real and Grassmann vectors that make up $\pmb\phi$. The change of variables to these order parameters again results in the Jacobian of \eqref{eq:coordinate.jacobian}, contributing \begin{equation} \frac N2\log\det J(C,R,D,G,Q,X,\hat X) \end{equation} Up to this point, the expressions above are general and independent of a given ansatz. However, we expect that the order parameters $X$ and $\hat X$ are zero, since this case is isotropic. Applying this ansatz here avoids a dramatically more complicated expression for the effective action found in the case with arbitrary $X$ and $\hat X$. We also will apply the ansatz that $Q_{a\alpha b\gamma}$ is zero for $a\neq b$, which is equivalent to assuming that the soft directions of typical pairs of stationary points are uncorrelated, and further that $Q_{\alpha\gamma}=Q_{a\alpha a\gamma}$ independently of the index $a$, implying that correlations in the tangent space of typical stationary points are the same. Given these simplifying forms of the ansatz, taking the superdeterminant yields \begin{equation} \begin{aligned} \log\det\left\{ \left[ f'(C)\odot D-\hat\beta I+\left(R^{\circ2}-G^{\circ2}+I\sum_{\alpha\gamma}2(\delta_{\alpha1}\hat\lambda+\beta)(\delta_{\gamma1}\hat\lambda+\beta)Q_{\alpha\gamma}^2\right)\odot f''(C) \right]f(C) +(I-R\odot f'(C))^2 \right\} \\ +n\log\det_{\alpha\gamma}(\delta_{\alpha\gamma}-2(\delta_{\alpha1}\hat\lambda+\beta)Q_{\alpha\gamma}) -2\log\det(I+G\odot f'(C)) \end{aligned} \end{equation} where once again $\odot$ is the Hadamard product and $A^{\circ n}$ gives the Hadamard power of $A$. We can already see one substantive difference between the structure of this problem and that of the spherical models: the effective action in this case mixes the order parameters $G$ due to the fermions with the ones $C$, $R$, and $D$ due to the other variables. This is the realization of the fact that the Hessian properties are no longer independent of the energy and gradient. Now we have reduced the problem to an extremal one over the order parameters $\hat\beta$, $\hat\lambda$, $C$, $R$, $D$, $G$, and $Q$, it is time to make an ansatz for the form of order we expect to find. We will focus on a regime where the structure of stationary points is replica symmetric, and further where typical pairs of stationary points have no overlap. This gives \begin{align} C=I && R=r_dI && D = d_dI && G = g_dI \end{align} We further take a planted replica symmetric structure for the matrix $Q$, identical to that in \eqref{eq:Q.structure}. \end{widetext} \bibliography{marginal} \end{document}