Lots of writing and figure tweaking.

1 files changed, 272 insertions, 152 deletions

diff --git a/when_annealed.tex b/when_annealed.tex
index b92e9a5..6e63ded 100644
--- a/when_annealed.tex
+++ b/when_annealed.tex
@@ -25,7 +25,7 @@
 \begin{document}
 
 \title{
-  When is the average number of saddles in a function typical?
+  When is the average number of saddle points typical?
 }
 
 \author{Jaron Kent-Dobias}
@@ -33,24 +33,28 @@
 
 \maketitle
 \begin{abstract}
-  A common measure of the complexity of a function is the count of its
-  stationary points. For complicated functions,
-  this count often grows exponentially with the volume and dimension of their
-  domain. In practice, the count is averaged over a class of such functions
-  (the annealed average), but the large numbers involved can result in averages
-  biased by extremely rare samples. Typical counts are reliably found by taking
-  the average of the logarithm (the quenched average), which is more costly and
-  not often done in practice. When most stationary points are uncorrelated with each other, quenched and anneals averages are equal. There are heuristics from equilibrium calculations that guarantee when most of the lowest minima will be uncorrelated. Here, we show that these equilibrium
-  heuristics cannot be used to draw conclusions about other minima and saddles. We produce examples among Gaussian-correlated functions on the
-  hypersphere where the count of certain saddles and minima has different quenched and
-  annealed averages, despite being guaranteed `safe' in the equilibrium
-  setting. We produce the necessary conditions for the emergence of nontrivial correlations between saddles. We discuss the implications for the geometry of those functions, and
-  in what out-of-equilibrium settings might show a signature of this.
+  A common measure of a function's complexity is the count of its stationary
+  points. For complicated functions, this count grows exponentially with the
+  volume and dimension of their domain. In practice, the count is averaged over
+  a class of such functions (the annealed average), but the large numbers
+  involved can result in averages biased by extremely rare samples. Typical
+  counts are reliably found by taking the average of the logarithm (the
+  quenched average), which is more difficult and not often done in practice.
+  When most stationary points are uncorrelated with each other, quenched and
+  anneals averages are equal. There are heuristics from equilibrium
+  calculations that guarantee when most of the lowest minima will be
+  uncorrelated. Here, we show that these equilibrium heuristics cannot be used
+  to draw conclusions about other minima and saddles. We produce examples among
+  Gaussian-correlated functions on the hypersphere where the count of certain
+  saddles and minima has different quenched and annealed averages, despite
+  being guaranteed `safe' in the equilibrium setting. We determine conditions
+  for the emergence of nontrivial correlations between saddles, and discuss the
+  implications for the geometry of those functions and what out-of-equilibrium
+  settings might be affected.
 \end{abstract}
 
-Random energies, cost functions, and interaction networks are important to many
-areas of modern science. The energy landscape of glasses, the likelihood
-landscape of machine learning and high-dimensional inference, and the
+Random high-dimensional energies, cost functions, and interaction networks are important in many fields. The energy landscape of glasses, the likelihood
+landscape of machine learning and inference, and the
 interactions between organisms in an ecosystem are just a few examples. A
 traditional tool for making sense of their behavior is to analyze the
 statistics of points where their dynamics are stationary. For energy or cost
@@ -65,51 +69,82 @@ Because their number is so large, it cannot be reliably averaged. The annealed
 approximation takes this average anyway, risking a systematic bias by rare and
 atypical samples. The annealed approximation is known to be exact for certain
 models and in certain circumstances, but it is used outside those circumstances
-without much reflection. In a few cases researches have made instead the
+without much reflection \cite{Wainrib_2013_Topological, Gershenzon_2023_On-Site}. In a few cases researches have made instead the
 better-controlled quenched average, which averages the logarithm of the number
-of stationary points \cite{Ros_2019_Complexity, Kent-Dobias_2023_How,
-Ros_2023_Quenched}.
-
-A heuristic line of reasoning for the correctness of the annealed
-approximation for the statistics of stationary points is sometimes made when
-the annealed approximation is correct for an equilibrium calculation on the same
-system. The argument goes like this: since the limit of zero temperature or
-noise in an equilibrium calculation concentrates the measure onto the lowest
-set of minima, the equilibrium free energy in the limit to zero temperature
-should be governed by the same statistics as the count of that lowest set of
-minima. This argument is valid, but only for the lowest set of minima, which at
-least in glassy problems are rarely relevant to dynamical behavior. What about
-the \emph{rest} of the stationary points?
+of stationary points, and find deviations from the annealed approximation with
+important implications for the system's behavior \cite{Muller_2006_Marginal, Ros_2019_Complexity,
+Kent-Dobias_2023_How, Ros_2023_Quenched}. Generically, the annealed
+approximation to the complexity is wrong when a nonvanishing fraction of pairs
+of stationary points have nontrivial correlations in their mutual position.
+
+A heuristic line of reasoning for the appropriateness of the annealed
+approximation is sometimes made when the approximation is correct for an
+equilibrium calculation on the same system. The argument goes like this: since
+the limit of zero temperature or noise in an equilibrium calculation
+concentrates the measure onto the lowest set of minima, the equilibrium free
+energy in the limit to zero temperature should be governed by the same
+statistics as the count of that lowest set of minima. This argument is valid,
+but only for the lowest set of minima, which at least in glassy problems are
+rarely relevant to dynamical behavior. What about the \emph{rest} of the
+stationary points?
 
 In this paper, we show that the behavior of the ground state, or \emph{any}
 equilibrium behavior, does not govern whether stationary points will have a
 correct annealed average. In a prototypical family of models of random
-functions, we calculate the conditions for when annealed averages should fail
+functions, we calculate a condition for determining when annealed averages should fail
 and stationary points should have nontrivial correlations in their mutual
 position. We produce examples of models whose equilibrium is guaranteed to
-never see correlations between states, but where a population of saddle points
-is correlated.
+never see such correlations between thermodynamic states, but where a
+population of saddle points is nevertheless correlated.
 
 We study the mixed spherical models, which are models of Gaussian-correlated
-random functions with isotropic statistics on the $(N-1)$-sphere. Each class of
-functions $H:S^{N-1}\to\mathbb R$ is defined by the average covariance between
-the function evaluated at two different points $\pmb\sigma_1,\pmb\sigma_2\in
-S^{N-1}$, which is a function of the scalar product or overlap between the two
-configurations:
+random functions with isotropic statistics on the $(N-1)$-sphere. Each model
+consists of a class of functions $H:S^{N-1}\to\mathbb R$ defined by the
+covariance between the function evaluated at two different points
+$\pmb\sigma_1,\pmb\sigma_2\in S^{N-1}$, which is a function of the scalar
+product (or overlap) between the two configurations:
 \begin{equation} \label{eq:covariance}
   \overline{H(\pmb\sigma_1)H(\pmb\sigma_2)}=\frac1Nf\bigg(\frac{\pmb\sigma_1\cdot\pmb\sigma_2}N\bigg)
 \end{equation}
-Specifying the covariance function $f$ uniquely specifies the class of functions. The
+Specifying the covariance function $f$ uniquely specifies the model. The
 series coefficients of $f$ need to be nonnnegative in order for $f$ to be a
 well-defined covariance. The case where $f$ is a homogeneous polynomial has
 been extensively studied, and corresponds to the pure spherical models of glass
 physics or the spiked tensor models of statistical inference. Here we will
-study cases where $f(q)=\frac12\big(\lambda q^p+(1-\lambda)q^s\big)$ for
-$\lambda\in(0,1)$. These are examples of \emph{mixed} spherical models.
+study cases where $f(q)=\frac12\big(\lambda q^3+(1-\lambda)q^s\big)$ for
+$\lambda\in(0,1)$, called $3+s$ models. These are examples of \emph{mixed} spherical models, which
+have been studied in the physics and statistics literature and host a zoo of
+complex orders and phase transitions \cite{Crisanti_2004_Spherical,
+Crisanti_2006_Spherical, Crisanti_2011_Statistical}.
 
-These classes of functions have been extensively studied in the physics and
-statistics literature, and host a zoo of complex orders and phase transitions
-\cite{Crisanti_2004_Spherical, Crisanti_2006_Spherical, Crisanti_2011_Statistical}.
+There are several well-established results on the equilibrium of this model.
+First, if the function $f$ is convex then it is not possible for the
+equilibrium solution to have nontrivial correlations between states at any
+temperature.\footnote{
+  More specifically, convex $f$ cannot have an equilibrium order with more than
+  {\oldstylenums1\textsc{rsb}} order among the configurations. In equilibrium,
+  {\oldstylenums1\textsc{rsb}} corresponds to trivial correlations between
+  thermodynamic states, but nontrivial correlations exist \emph{within} a state
+  at nonzero temperature. When temperature goes to zero,
+  {\oldstylenums1\textsc{rsb}} in equilibrium reduces to replica symmetry among
+  the lowest-lying states. Because in this paper we focus on symmetry breaking
+  between stationary points, we consider this form of \textsc{rsb} in
+  equilibrium trivial because it does not imply any nontrivial correlations
+  between states.
+}
+This is a very strong condition on the form of equilibrium order. Note that
+non-convex $f$ does not imply that you will see nontrivial correlations between
+states at some temperature. In the $3+s$ models we consider here, models with
+$s>8$ have non-convex $f$ and those with $s\geq8$ have convex $f$ independent
+of $\lambda$. Second, the characterization of the ground state has been made
+\cite{Crisanti_2004_Spherical, Crisanti_2006_Spherical,
+Crisanti_2011_Statistical, Auffinger_2022_The}. In the $3+s$ models we
+consider, for $s>12.430\ldots$ nontrivial ground state configurations appear in
+certain ranges of $\lambda$. These bounds on equilibrium order are shown in
+Fig.~\ref{fig:phases}, along with our result in this paper for where the
+complexity has nontrivial \textsc{rsb}. As evidenced in that figure,
+\textsc{rsb} among saddles is possible well outside the bounds from
+equilibrium.
 
 There are two important features which differentiate stationary points $\pmb\sigma^*$ in the
 spherical models: their \emph{energy density} $E=\frac1NH(\pmb\sigma^*)$ and
@@ -117,7 +152,7 @@ their \emph{stability}
 $\mu=\frac1N\operatorname{\mathrm{Tr}}\operatorname{\mathrm{Hess}}H(\pmb\sigma^*)$.
 The energy density should be familiar, as the `height' in the landscape. The
 stability is so-called because it governs the spectrum of the stationary point.
-In each spherical model, the spectrum of each stationary point is a Wigner
+In each spherical model, the spectrum of every stationary point is a Wigner
 semicircle of the same width $\mu_\mathrm m=\sqrt{4f''(1)}$, but shifted by
 constant. The stability $\mu$ sets this constant shift. When $\mu<\mu_\mathrm
 m$, the spectrum still has support over zero and we have saddles with an
@@ -132,10 +167,11 @@ $\mu=\mu_\mathrm m$, the spectrum has a pseudogap, and we have marginal minima.
     A phase diagram of the boundaries we discuss in this paper for the $3+s$
     model with $f=\frac12\big(\lambda q^3+(1-\lambda)q^s\big)$. The blue region
     shows where there exist some stationary points whose complexity is
-    {\oldstylenums1}\textsc{rsb}. The yellow region shows where $f$ is not
-    convex and therefore \textsc{rsb} solutions are possible in equilibrium.
-    The green region shows where \textsc{rsb} solutions are correct at
-    the ground state, adapted from \cite{Auffinger_2022_The}.
+    {\oldstylenums1}\textsc{rsb}, and is given by $G_f>0$ where $G_f$ is found
+    in \eqref{eq:condition}. The yellow region shows where $f$ is not convex
+    and therefore \textsc{rsb} solutions are possible in equilibrium. The green
+    region shows where \textsc{rsb} solutions are correct at the ground state,
+    adapted from \cite{Auffinger_2022_The}.
   } \label{fig:phases}
 \end{figure}
 
@@ -144,39 +180,51 @@ stability $\mu$ is exponential in $N$ for these models. Their complexity $\Sigma
 defined by the average of the logarithm of their number, or
 $\Sigma(E,\mu)=\frac1N\overline{\log\mathcal N(E,\mu)}$. More often the annealed
 complexity is calculated, where the average is taken before the logarithm:
-$\Sigma_\mathrm a(E,\mu)=\frac1N\log\overline{\mathcal N(E,\mu)}$.
+$\Sigma_\mathrm a(E,\mu)=\frac1N\log\overline{\mathcal N(E,\mu)}$. The annealed
+complexity has been computed for these models \cite{BenArous_2019_Geometry,
+Folena_2020_Rethinking}, and the quenched complexity has been computed for a
+couple examples which have correlations among ground-state minima
+\cite{Kent-Dobias_2023_How}.
+
+In these models, trivial correlations between stationary points correspond with
+zero overlap: almost all stationary points are orthogonal to each other. This
+corresponds with \emph{replica symmetric} (\textsc{rs}) order. The emergence of
+nontrivial correlations, and the invalidity of the anneal approximation, occurs
+when some non-vanishing fraction of stationary point pairs have a nonzero
+overlap. This corresponds to some kind of \emph{replica symmetry breaking}
+(\textsc{rsb}). Here we restrict ourselves to a {\oldstylenums1}\textsc{rsb}
+ansatz, which corresponds to two kinds of pairs of stationary point: a fraction
+$x$ of pairs have the trivial zero overlap, and the remaining fraction $1-x$
+have a nontrivial overlap $q_1$. In the annealed or replica-symmetric case,
+$x=1$ and all but a vanishing fraction of stationary points are uncorrelated
+with each other. Since other kinds of \textsc{rsb} order encompass {\oldstylenums1}\textsc{rsb}, we are guaranteed that
+$\Sigma\leq\Sigma_{\oldstylenums1\textsc{rsb}}\leq\Sigma_\mathrm a$. We
+will discuss later in what settings the {\oldstylenums1}\textsc{rsb} complexity
+is correct.
 
 When the complexity is calculated using the Kac--Rice formula and a physicists'
 tool set, the problem is reduced to the evaluation of an integral by the saddle
-point method for large $N$ \cite{Kent-Dobias_2023_How}. An ansatz for
-the complexity needs to be made. Here we restrict ourselves to a
-{\oldstylenums1}\textsc{rsb} ansatz, which is parameterized by two quantities:
-$q_1$ and $x$. They have a geometric interpretation: given a stationary point
-fixed with certain properties, $1-x$ corresponds to the proportion of other
-stationary points with the same properties that are correlated with it, 
-and $q_1$ gives the overlap that this correlated population has with it. In the annealed or replica-symmetric case, $x=1$ and all but a vanishing fraction of stationary points are uncorrelated with each other.
-We are guaranteed that
-$\Sigma\leq\Sigma_{\oldstylenums1\textsc{rsb}}\leq\Sigma_\mathrm a$, and we
-will discuss later in what settings the {\oldstylenums1}\textsc{rsb} complexity
-is correct. The complexity is given by extremizing an effective action,
+point method for large $N$ \cite{Kent-Dobias_2023_How}.
+The complexity is given by extremizing an effective action,
 \begin{equation}
-  \Sigma_{\oldstylenums1\textsc{rsb}}(E,\mu)=\lim_{n\to0}\int dq_1\,dx\,\mathcal S(q_1,x\mid E,\mu)e^{nN\mathcal S(q_1,x\mid E,\mu)}
-  =\mathop{\mathrm{extremum}}_{q_1,x}\mathcal S(q_1,x\mid E,\mu)
+  \Sigma_{\oldstylenums1\textsc{rsb}}(E,\mu)
+  =\lim_{n\to0}\int dq_1\,dx\,\mathcal S_{\oldstylenums1\textsc{rsb}}(q_1,x\mid E,\mu)e^{nN\mathcal S_{\oldstylenums1\textsc{rsb}}(q_1,x\mid E,\mu)}
+  =\mathop{\mathrm{extremum}}_{q_1,x}\mathcal S_{\oldstylenums1\textsc{rsb}}(q_1,x\mid E,\mu)
 \end{equation}
 for the action $\mathcal S$ given by
 \begin{equation}
   \begin{aligned}
-    \mathcal S&(q_1,x\mid E,\mu)
+    &\mathcal S_{\oldstylenums1\textsc{rsb}}(q_1,x\mid E,\mu)
     =\mathcal D(\mu)
     +\mathop{\textrm{extremum}}_{\hat\beta,r_\mathrm d,r_1,d_\mathrm d,d_1}
     \Bigg\{
       \hat\beta E-r_\mathrm d\mu\\
-      &+\frac12\bigg[
+      &\quad+\frac12\bigg[
         \hat\beta^2\big[f(1)-\Delta xf(q_1)\big]
         +(2\hat\beta r_\mathrm d-d_\mathrm d)f'(1)
         -\Delta x(2\hat\beta r_1-d_1)f'(q_1)
         +r_\mathrm d^2f''(1)-\Delta x\,r_1^2f''(q_1) \\
-        &+\frac1x\log\Big(
+        &\quad+\frac1x\log\Big(
           \big(r_\mathrm d-\Delta x\,r_1\big)^2+d_\mathrm d\big(1-\Delta x\,q_1\big)-\Delta x\,d_1\big(1-\Delta xq_1\big)
           \Big)
           -\frac{\Delta x}x\log\Big(
@@ -197,95 +245,130 @@ where $\Delta x=1-x$ and
     -\log\left(\left|\frac{\mu}{\mu_\text m}\right|-\sqrt{\big(\frac\mu{\mu_\text m}\big)^2-1}\right) & \mu^2>\mu_\text m^2
   \end{cases}
 \end{equation}
-The extremal problem in $\hat\beta$, $r_\mathrm d$, $r_1$,
-$d_\mathrm d$, and $d_1$ has a unique solution and can be found explicitly, but the
-resulting formula is much more complicated so we do not include it here. There
-can be multiple extrema at which to evaluate $\mathcal S$, in this case the one
-for which $\Sigma$ is \emph{smallest} gives the correct solution. There is
-always a solution for $x=1$ which is independent of $q_1$, which corresponds to
-the replica symmetric case and which is equal to the annealed calculation, so
-$\Sigma_\mathrm a(E,\mu)=\mathcal S(E,\mu\mid q_1,1)$. The crux of this paper
-will be to determine when this solution is not the global one.
+The extremal problem in $\hat\beta$, $r_\mathrm d$, $r_1$, $d_\mathrm d$, and
+$d_1$ has a unique solution and can be found explicitly, but the resulting
+formula is much more complicated so we do not include it here. There can be
+multiple extrema at which to evaluate $\mathcal
+S_{\oldstylenums1\textsc{rsb}}$, in this case the one for which $\Sigma$ is
+\emph{smallest} gives the correct solution. There is always a solution for
+$x=1$ which is independent of $q_1$, which corresponds to the replica symmetric
+case and which is equal to the annealed calculation, so $\Sigma_\mathrm
+a(E,\mu)=\mathcal S_{\oldstylenums1\textsc{rsb}}(E,\mu\mid q_1,1)$. The crux of
+this paper will be to determine when this solution is not the global one.
 
 It isn't accurate to say that a solution to the saddle point equations is
 `stable' or `unstable.' The problem of solving the complexity in this way is
 not a variational problem, so there is nothing to be maximized or minimized,
-and in general even global solutions have positive and negative eigenvalues of
-the Hessian. However, the eigenvalues of the Hessian can still tell us
-something about the emergence of new solutions: when another solution
-bifurcates smoothly from an existing one, the Hessian evaluated at that point
-will have a zero eigenvalue. Unfortunately this is a difficult procedure to
-apply in general, since one must know the parameters in the new solution, and
-some parameters, e.g., $q_1$, are unconstrained in the old solution.
-
-There is one place where one can consistently search for a bifurcating solution
+and in general even global solutions are not even local minima of the action.
+However, the stability of the action can still tell us something about the
+emergence of new solutions: when another solution bifurcates from an existing
+one, the action will have a flat direction. Unfortunately this is a difficult
+procedure to apply in general, since one must know the parameters of the new
+solution, and some parameters, e.g., $q_1$, are unconstrained and can take any
+value in the old solution.
+
+There is one place where we can consistently search for a bifurcating solution
 to the saddle point equations: along the zero complexity line
-$\Sigma(E,\mu)=0$. Going along this line in the replica symmetric solution, the
+$\Sigma_\mathrm a(E,\mu)=0$. Going along this line in the replica symmetric solution, the
 {\oldstylenums1}\textsc{rsb} complexity transitions at a critical point where
 $x=q_1=1$ \cite{Kent-Dobias_2023_How}. Since all the parameters in the
-bifurcating solution are known at this point, one can search for it by looking
-for a zero eigenvalue in the way described above. In the replica symmetric
-solution for points describing saddles, this line is
+bifurcating solution are known at this point, we can search for it by looking
+for a flat direction in the way described above. In the annealed
+solution for points describing saddles ($\mu<\mu_\mathrm m$), this line is
 \begin{equation} \label{eq:extremal.line}
   \mu_0=-\frac1{z_f}\left(2Ef'f''+\sqrt{2f''u_f\bigg(\log\frac{f''}{f'}z_f-E^2(f''-f')\bigg)}\right)
 \end{equation}
 where we define for brevity (here and elsewhere) the constants
 \begin{align}
-  u_f=f(f'+f'')-f'^2
+  u_f&=f(f'+f'')-f'^2
   &&
   v_f=f'(f''+f''')-f''^2 \\
-  w_f=2f''(f''-f')+f'f'''
+  w_f&=2f''(f''-f')+f'f'''
   &&
   y_f=f'(f'-f)+f''f
   &&
   z_f=f(f''-f')+f'^2
 \end{align}
-Note that for $f$ to define a sensible covariance, all of its series
-coefficients must be nonnegative. If $f$ has at least two nonzero coefficients
-at second order or higher, all of these constants are positive.
+If $f$ has at least two nonzero coefficients at second order or higher, all of
+these constants are positive. We also define $E_\textrm{min}$, the minimum
+energy at which saddle points with an extensive number of downward directions
+are found, as the energy for which $\mu_0(E_\mathrm{min})=\mu_\mathrm m$.
 
 Let $M$ be the matrix of double partial derivatives of $\mathcal S$ with
 respect to $q_1$ and $x$. We evaluate $M$ at the replica symmetric saddle point
 $x=1$ with the additional constraint that $q_1=1$ and along the extremal
 complexity line \eqref{eq:extremal.line}. We determine when a zero eigenvalue
-appears, indicated the presence of a bifurcating {\oldstylenums1}\textsc{rsb}
+appears, indicating the presence of a bifurcating {\oldstylenums1}\textsc{rsb}
 solution, by solving $0=\det M$. We find
 \begin{equation}
-  \det M=-\left(\frac{\partial^2\mathcal S}{\partial q_1\partial x}\bigg|_{\substack{x=1\\q_1=1}}\right)^2\propto(ay^2+bE^2+2cyE-d)^2
+  \det M=-\bigg(\frac{\partial^2\mathcal S}{\partial q_1\partial x}\bigg|_{\substack{x=1\\q_1=1}}\bigg)^2\propto(ay^2+bE^2+2cyE-d)^2
 \end{equation}
-where $y=-\frac12z_f\mu-f'f''E$ is proportional to the square-root term in \eqref{eq:extremal.line} and
-the constants $a$, $b$, $c$, and $d$ are defined below. Changing to $y$ is a
+where $y=-\frac12z_f\mu-f'f''E$ is proportional to the square-root term in
+\eqref{eq:extremal.line} and the constants $a$, $b$, $c$, and $d$ are defined
+by
+\begin{equation}
+  a=\frac{w_f\big(3y_f^2-4ff'f''(f'-f)\big)-6y_f^2(f''-f')f''}{(u_fz_ff'')^2f'}
+  \qquad
+  b=\frac{f'w_f}{z_f^2}
+  \qquad
+  c=\frac{w_f}{f''z_f^2}
+  \qquad
+  d=\frac{w_f}{f'f''}
+\end{equation}
+Changing variables to $y$ from $\mu$ is a
 convenient choice because the branch of \eqref{eq:extremal.line} is chosen
 by the sign of $y$ (the lower-energy branch we are interested in corresponds
 with $y>0$) and the relationship between $y$ and $E$ on the extremal line is
 $g=2hy^2+eE^2$, where the constants $e$, $g$, and $h$ are given by
 \begin{equation}
-  \begin{aligned}
-    a&=\frac{w_f\big(3y_f^2-4ff'f''(f'-f)\big)-6y_f^2(f''-f')f''}{(u_fz_ff'')^2f'}
-    \qquad
-    b=\frac{f'w_f}{z_f^2}
-      \\
-      c&=\frac{w_f}{f''z_f^2}
-      \qquad
-      d=\frac{w_f}{f'f''}
-      \qquad
-      e=f''-f'
-      \qquad
-      g=z_f\log\frac{f''}{f'}
-      \qquad
-      h=\frac1{f''u_f}
-  \end{aligned}
+  e=f''-f'
+  \qquad
+  g=z_f\log\frac{f''}{f'}
+  \qquad
+  h=\frac1{f''u_f}
 \end{equation}
-The solutions for $\det M=0$ correspond to energies that satisfy
-\begin{equation}
+
+\begin{figure}
+  \centering
+  \includegraphics{figs/complexity_35.pdf}
+
+  \caption{
+    Stationary point statistics as a function of energy density $E$ and
+    stability $\mu$ for a $3+5$ model with $\lambda=\frac12$. The dashed black
+    line shows the line of zero complexity, where stationary points vanish, and
+    enclosed inside they are found in exponential number. The red region (blown
+    up in the inset) shows where the annealed complexity gives the wrong count
+    and a {\oldstylenums1}\textsc{rsb} complexity in necessary. The red points
+    show where $\det M=0$. The left point, which is only an upper bound on the
+    transition, coincides with it in this case. The gray shaded region
+    highlights the minima, which are stationary points with $\mu>\mu_\mathrm
+    m$. $E_\textrm{min}$ is marked on the plot as the lowest energy at which
+    extensive saddles are found.
+  } \label{fig:complexity_35}
+\end{figure}
+
+The solutions for $\det M=0$ can be calculated explicitly and correspond to
+energies that satisfy
+\begin{equation} \label{eq:energies}
   E_{\oldstylenums1\textsc{rsb}}^\pm
   =\operatorname{sign}(bg-de)\frac{-cg\pm\sqrt{c^2g^2+(2dh-ag)(bg-de)}}
   {
     \sqrt{2c^2eg+(2bh-ae)(bg-de)\mp2ce\sqrt{c^2g^2+(2dh-ag)(bg-de)}}
   }
 \end{equation}
-The expression inside the inner square root is proportional to
-\begin{equation}
+This predicts two points where a {\oldstylenums1}\textsc{rsb} solution can
+bifurcate from the annealed one. The remainder of the transition line can be
+found by trying to solve the extremal problem for the action very close to one
+of these solutions, and then taking small steps in the parameters $E$ and $\mu$
+until it terminates. In many cases considered here, the line of transitions in
+the complexity that begins at $E_{\oldstylenums1\textsc{rsb}}^+$, the higher
+energy point, ends exactly at $E_{\oldstylenums1\textsc{rsb}}^-$, the lower
+energy point, so that these two points give the precise range of energies at
+which \textsc{rsb} saddles are found. An example that conforms with this
+picture for a $3+5$ mixed model is shown in Fig.~\ref{fig:complexity_35}.
+
+The expression inside the inner square root of \eqref{eq:energies} is proportional to
+\begin{equation} \label{eq:condition}
   G_f
   =
   f'\log\frac{f''}{f'}\big[
@@ -294,27 +377,12 @@ The expression inside the inner square root is proportional to
   -2(f''-f')u_fw_f
   -2\log^2\frac{f''}{f'}f'^2f''v_f
 \end{equation}
-If $G_f>0$, then there are two points along the extremal complexity line where
-a solution bifurcates, and a new line of {\oldstylenums1}\textsc{rsb} solutions
-between them. Therefore, $G_f>0$ is a necessary condition to see
-{\oldstylenums1}\textsc{rsb} in the complexity.
-
-\begin{figure}
-  \centering
-  \includegraphics{figs/complexity_35.pdf}
-
-  \caption{
-    Stationary point statistics as a function of energy density $E$ and
-    stability $\mu$ for a $3+5$ model with $\lambda=\frac12$. The dashed
-    black line shows the line of zero complexity, where stationary points
-    vanish, and enclosed inside they are found in exponential number. The red
-    region (blown up in the inset) shows where the annealed complexity gives
-    the wrong count and a {\oldstylenums1}\textsc{rsb} complexity in necessary.
-    The red points show where $\det M=0$. The left point, which is only an
-    upper bound on the transition, coincides with it in this case. The gray shaded region highlights the
-    minima, which are stationary points with $\mu>\mu_\mathrm m$.
-  } \label{fig:complexity_35}
-\end{figure}
+If $G_f>0$, then the bifurcating solutions exist, and there is someplace where
+the annealed solution is corrected by a {\oldstylenums1\textsc{rsb}} solution.
+Therefore, $G_f>0$ is a condition to see {\oldstylenums1}\textsc{rsb} in the
+complexity. If $G_f<0$, then there is nowhere along the extremal line where
+saddles can be described by such a complexity. The range of $3+s$ models where
+$G_f$ is positive is shown in Fig.~\ref{fig:phases}.
 
 \begin{figure}
   \centering
@@ -343,10 +411,28 @@ between them. Therefore, $G_f>0$ is a necessary condition to see
     bottom lines are given by $E_{\oldstylenums1\textsc{rsb}}$, but for $s=14$
     there is a portion where the low-energy boundary has $q_1<1$. In that plot,
     the continuation of the $E_{\oldstylenums1\textsc{rsb}}$ line is shown
-    dashed.
-  }
+    dashed. Also marked is the range of $\lambda$ for which the ground state
+    minima are characterized by nontrivial \textsc{rsb}.
+  } \label{fig:energy_ranges}
 \end{figure}
 
+Fig.~\ref{fig:energy_ranges} shows the range of energies at which nontrivial
+correlations are found between stationary points in several $3+s$ models as
+$\lambda$ is varied. For models with smaller $s$, such correlations are found
+only among saddles, with the boundary never dipping beneath the minimum energy
+of saddles $E_\mathrm{min}$. Also, these models have a transition boundary that
+smoothly connects $E_{\oldstylenums1\textsc{rsb}}^+$ and
+$E_{\oldstylenums1\textsc{rsb}}^-$, so $E_{\oldstylenums1\textsc{rsb}}^-$
+corresponds to the lower bound of \textsc{rsb} complexity. For large enough
+$s$, the range passes into minima, which is excepted as these models have
+nontrivial complexity of their ground states. This also seems to correspond
+with the decoupling of the \textsc{rsb} solutions connected to
+$E_{\oldstylenums1\textsc{rsb}}^+$ and $E_{\oldstylenums1\textsc{rsb}}^-$, with
+the two phase boundaries not corresponding, as in Fig.~\ref{fig:order}. In
+these cases, $E_{\oldstylenums1\textsc{rsb}}^-$ sometimes gives the lower
+bound, but sometimes it is given by the termination of the phase boundary
+extended from $E_{\oldstylenums1\textsc{rsb}}^+$.
+
 \begin{figure}
   \centering
   \includegraphics{figs/order_plot_1.pdf}\\
@@ -360,24 +446,21 @@ between them. Therefore, $G_f>0$ is a necessary condition to see
     shows $E_{\oldstylenums1\textsc{rsb}}^-$, while the solid red lines shows
     the transition boundary with the \textsc{rs} complexity. The dashed black
     line shows the \textsc{rs} zero complexity line, while the solid black line
-    shows the {\oldstylenums1}\textsc{rsb} zero complexity line. \textbf{Top:}
-    $\lambda=0.67$. Here the end of the transition line that begins at
-    $E_{\oldstylenums1\textsc{rsb}}^+$ does not match
+    shows the {\oldstylenums1}\textsc{rsb} zero complexity line. The dashed red
+    lines show where a nonphysical {\oldstylenums1\textsc{rsb}} phase appears
+    (the spinodal of that phase). The dotted red line shows an abrupt phase
+    transition between different {\oldstylenums1}\textsc{rsb} phases.
+    \textbf{Top:} $\lambda=0.67$. Here the end of the transition line that
+    begins at $E_{\oldstylenums1\textsc{rsb}}^+$ does not match
     $E_{\oldstylenums1\textsc{rsb}}^-$ but terminates at higher energies.
     $E_{\oldstylenums1\textsc{rsb}}^-$ still corresponds with the lower bound.
     \textbf{Bottom:} $\lambda=0.69$. Here the end of the transition line that
     begins at $E_{\oldstylenums1\textsc{rsb}}^+$ terminates at lower energies
     than $E_{\oldstylenums1\textsc{rsb}}^-$, and therefore its terminus defines
     the lower bound.
-  }
+  } \label{fig:order}
 \end{figure}
 
-\begin{equation}
-  \mu
-  =-\frac{(f_1'+f_0'')u_f}{(2f_1-f_1')f_1'f_0''^{1/2}}
-  -\frac{f_1''-f_1'}{f_1'-2f_1}E
-\end{equation}
-
 There are implications for the emergence of \textsc{rsb} in equilibrium. Consider a specific $H$ with
 \begin{equation}
   H(\pmb\sigma)
@@ -389,13 +472,50 @@ with $\overline{(J^{(p)})^2}=p!/2N^{p-1}$ and likewise for $J^{(s)}$. It is
 straightforward to confirm that $H$ defined this way has the covariance
 property \eqref{eq:covariance} with $f(q)=\frac12\big(\lambda
 q^p+(1-\lambda)q^s\big)$. With the $J$s drawn in this way and fixed for $p=3$
-and $s=16$, we can vary $\lambda$, and according to Fig.~\ref{fig:phases} we
+and $s=14$, we can vary $\lambda$, and according to Fig.~\ref{fig:phases} we
 should see a transition in the type of order at the ground state. What causes
 the change? Our analysis indicates that stationary points with the required
 order \emph{already exist in the landscape} as unstable saddles for small
 $\lambda$, then eventually stabilize into metastable minima and finally become
 the lowest lying states. This is different from the picture of existing
-uncorrelated low-lying states splitting apart into correlated clusters.
+uncorrelated low-lying states splitting apart into correlated clusters. Where
+existing stationary points do appear to split apart, when $\lambda$ is
+decreased from large values, is among saddles, not minima.
+
+Similar reasoning can be made for other mixed models, like the $2+s$, which
+should see complexities with other forms of \textsc{rsb}. For instance, in
+\cite{Kent-Dobias_2023_How} we show that the complexity transitions from
+\textsc{rs} to full \textsc{rsb} when
+\begin{equation}
+  \mu
+  =-\frac{(f'+f''(0))u_f}{(2f-f')f'f''(0)^{1/2}}
+  -\frac{f''-f'}{f'-2f}E
+\end{equation}
+which can only be realized when $f''(0)\neq0$, as in the $2+s$ models. For
+$s>2$, this transition line \emph{always} intersects the extremal line
+\eqref{eq:extremal.line}, and so \textsc{rsb} complexity should always be found
+among some population of stationary points. However, it is likely that for much
+of the parameter space the so-called one-full \textsc{rsb}
+({\oldstylenums1\textsc{frsb}}) is the correct solution, as it likely is for
+large $s$ in the $3+s$ model at hand. Further work to find the conditions for
+transitions of the complexity to these forms of order is necessary.
+
+What are the implications for dynamics? We find that nontrivial correlations
+tend to exist among saddle points with the maximum or minimum index possible at
+a given energy density, which are quite atypical in the landscape. However,
+these strangely correlated saddle points must descend to uncorrelated minima,
+which raises questions about whether structure on the boundary of a basin of
+attraction is influential to the dynamics that descends into that basin. These saddles might act as early-time separatrices for descent trajectories.  With
+large open problems in even the gradient decent dynamics on these models, it
+remains to be seen whether such structures could be influential
+\cite{Folena_2020_Rethinking, Folena_2023_On}.
+
+We have determined the conditions under which the complexity of the mixed $3+s$
+spherical models has different quenched and annealed averages, as the result of
+nontrivial correlations between stationary points. We saw that these conditions
+can arise among certain populations of saddle points even when the model is
+guaranteed to lack such correlations between equilibrium states, and exist for
+saddle points at a wide range of energies.
 
 \printbibliography