diff options
| -rw-r--r-- | when_annealed.tex | 121 |
1 files changed, 63 insertions, 58 deletions
diff --git a/when_annealed.tex b/when_annealed.tex index ae7e3b8..537381a 100644 --- a/when_annealed.tex +++ b/when_annealed.tex @@ -53,15 +53,15 @@ settings might be affected. \end{abstract} -Random high-dimensional energies, cost functions, and interaction networks are important in many fields. The energy landscape of glasses, the likelihood -landscape of machine learning and inference, and the -interactions between organisms in an ecosystem are just a few examples. A -traditional tool for making sense of their behavior is to analyze the -statistics of points where their dynamics are stationary. For energy or cost -landscapes, these correspond to the minima, maxima, and saddles, while for -ecosystems and other non-gradient dynamical systems these correspond to -equilibria of the dynamics. When many stationary points are present, the system -is considered complex. +Random high-dimensional energies, cost functions, and interaction networks are +important in many fields. The energy landscape of glasses, the likelihood +landscape of machine learning and inference, and the interactions between +organisms in an ecosystem are just a few examples. A traditional tool for +making sense of their behavior is to analyze the statistics of points where +their dynamics are stationary. For energy or cost landscapes, these correspond +to the minima, maxima, and saddles, while for ecosystems and other non-gradient +dynamical systems these correspond to equilibria of the dynamics. When many +stationary points are present, the system is considered complex. Despite the importance of stationary point statistics for understanding complex behavior, they are often calculated using an uncontrolled approximation. @@ -69,13 +69,15 @@ Because their number is so large, it cannot be reliably averaged. The annealed approximation takes this average anyway, risking a systematic bias by rare and atypical samples. The annealed approximation is known to be exact for certain models and in certain circumstances, but it is used outside those circumstances -without much reflection \cite{Wainrib_2013_Topological, Gershenzon_2023_On-Site}. In a few cases researches have made instead the +without much reflection \cite{Wainrib_2013_Topological, +Gershenzon_2023_On-Site}. In a few cases researches have made instead the better-controlled quenched average, which averages the logarithm of the number of stationary points, and find deviations from the annealed approximation with -important implications for the system's behavior \cite{Muller_2006_Marginal, Ros_2019_Complexity, -Kent-Dobias_2023_How, Ros_2023_Quenched}. Generically, the annealed -approximation to the complexity is wrong when a nonvanishing fraction of pairs -of stationary points have nontrivial correlations in their mutual position. +important implications for the system's behavior \cite{Muller_2006_Marginal, +Ros_2019_Complexity, Kent-Dobias_2023_How, Ros_2023_Quenched}. Generically, +the annealed approximation to the complexity is wrong when a nonvanishing +fraction of pairs of stationary points have nontrivial correlations in their +mutual position. A heuristic line of reasoning for the appropriateness of the annealed approximation is sometimes made when the approximation is correct for an @@ -91,10 +93,10 @@ stationary points? In this paper, we show that the behavior of the ground state, or \emph{any} equilibrium behavior, does not govern whether stationary points will have a correct annealed average. In a prototypical family of models of random -functions, we calculate a condition for determining when annealed averages should fail -and stationary points should have nontrivial correlations in their mutual -position. We produce examples of models whose equilibrium is guaranteed to -never see such correlations between thermodynamic states, but where a +functions, we calculate a condition for determining when annealed averages +should fail and stationary points should have nontrivial correlations in their +mutual position. We produce examples of models whose equilibrium is guaranteed +to never see such correlations between thermodynamic states, but where a population of saddle points is nevertheless correlated. We study the mixed spherical models, which are models of Gaussian-correlated @@ -106,16 +108,17 @@ product (or overlap) between the two configurations: \begin{equation} \label{eq:covariance} \overline{H(\pmb\sigma_1)H(\pmb\sigma_2)}=\frac1Nf\bigg(\frac{\pmb\sigma_1\cdot\pmb\sigma_2}N\bigg) \end{equation} -Specifying the covariance function $f$ uniquely specifies the model. The -series coefficients of $f$ need to be nonnnegative in order for $f$ to be a +Specifying the covariance function $f$ uniquely specifies the model. The series +coefficients of $f$ need to be nonnnegative in order for $f$ to be a well-defined covariance. The case where $f$ is a homogeneous polynomial has been extensively studied, and corresponds to the pure spherical models of glass physics or the spiked tensor models of statistical inference. Here we will study cases where $f(q)=\frac12\big(\lambda q^3+(1-\lambda)q^s\big)$ for -$\lambda\in(0,1)$, called $3+s$ models. These are examples of \emph{mixed} spherical models, which -have been studied in the physics and statistics literature and host a zoo of -complex orders and phase transitions \cite{Crisanti_2004_Spherical, -Crisanti_2006_Spherical, Crisanti_2011_Statistical}. +$\lambda\in(0,1)$, called $3+s$ models. These are examples of \emph{mixed} +spherical models, which have been studied in the physics and statistics +literature and host a zoo of complex orders and phase transitions +\cite{Crisanti_2004_Spherical, Crisanti_2006_Spherical, +Crisanti_2011_Statistical}. There are several well-established results on the equilibrium of this model. First, if the function $f$ is convex then it is not possible for the @@ -146,9 +149,9 @@ complexity has nontrivial \textsc{rsb}. As evidenced in that figure, \textsc{rsb} among saddles is possible well outside the bounds from equilibrium. -There are two important features which differentiate stationary points $\pmb\sigma^*$ in the -spherical models: their \emph{energy density} $E=\frac1NH(\pmb\sigma^*)$ and -their \emph{stability} +There are two important features which differentiate stationary points +$\pmb\sigma^*$ in the spherical models: their \emph{energy density} +$E=\frac1NH(\pmb\sigma^*)$ and their \emph{stability} $\mu=\frac1N\operatorname{\mathrm{Tr}}\operatorname{\mathrm{Hess}}H(\pmb\sigma^*)$. The energy density should be familiar, as the `height' in the landscape. The stability is so-called because it governs the spectrum of the stationary point. @@ -176,15 +179,15 @@ $\mu=\mu_\mathrm m$, the spectrum has a pseudogap, and we have marginal minima. \end{figure} The number $\mathcal N(E,\mu)$ of stationary points with energy density $E$ and -stability $\mu$ is exponential in $N$ for these models. Their complexity $\Sigma(E,\mu)$ is -defined by the average of the logarithm of their number, or -$\Sigma(E,\mu)=\frac1N\overline{\log\mathcal N(E,\mu)}$. More often the annealed -complexity is calculated, where the average is taken before the logarithm: -$\Sigma_\mathrm a(E,\mu)=\frac1N\log\overline{\mathcal N(E,\mu)}$. The annealed -complexity has been computed for these models \cite{BenArous_2019_Geometry, -Folena_2020_Rethinking}, and the quenched complexity has been computed for a -couple examples which have correlations among ground-state minima -\cite{Kent-Dobias_2023_How}. +stability $\mu$ is exponential in $N$ for these models. Their complexity +$\Sigma(E,\mu)$ is defined by the average of the logarithm of their number, or +$\Sigma(E,\mu)=\frac1N\overline{\log\mathcal N(E,\mu)}$. More often the +annealed complexity is calculated, where the average is taken before the +logarithm: $\Sigma_\mathrm a(E,\mu)=\frac1N\log\overline{\mathcal N(E,\mu)}$. +The annealed complexity has been computed for these models +\cite{BenArous_2019_Geometry, Folena_2020_Rethinking}, and the quenched +complexity has been computed for a couple examples which have correlations +among ground-state minima \cite{Kent-Dobias_2023_How}. In these models, trivial correlations between stationary points correspond with zero overlap: almost all stationary points are orthogonal to each other. This @@ -197,15 +200,16 @@ ansatz, which corresponds to two kinds of pairs of stationary point: a fraction $x$ of pairs have the trivial zero overlap, and the remaining fraction $1-x$ have a nontrivial overlap $q_1$. In the annealed or replica-symmetric case, $x=1$ and all but a vanishing fraction of stationary points are uncorrelated -with each other. Since other kinds of \textsc{rsb} order encompass {\oldstylenums1}\textsc{rsb}, we are guaranteed that -$\Sigma\leq\Sigma_{\oldstylenums1\textsc{rsb}}\leq\Sigma_\mathrm a$. We -will discuss later in what settings the {\oldstylenums1}\textsc{rsb} complexity -is correct. +with each other. Since other kinds of \textsc{rsb} order encompass +{\oldstylenums1}\textsc{rsb}, we are guaranteed that +$\Sigma\leq\Sigma_{\oldstylenums1\textsc{rsb}}\leq\Sigma_\mathrm a$. We will +discuss later in what settings the {\oldstylenums1}\textsc{rsb} complexity is +correct. When the complexity is calculated using the Kac--Rice formula and a physicists' tool set, the problem is reduced to the evaluation of an integral by the saddle -point method for large $N$ \cite{Kent-Dobias_2023_How}. -The complexity is given by extremizing an effective action, +point method for large $N$ \cite{Kent-Dobias_2023_How}. The complexity is given +by extremizing an effective action, \begin{equation} \Sigma_{\oldstylenums1\textsc{rsb}}(E,\mu) =\lim_{n\to0}\int dq_1\,dx\,\mathcal S_{\oldstylenums1\textsc{rsb}}(q_1,x\mid E,\mu)e^{nN\mathcal S_{\oldstylenums1\textsc{rsb}}(q_1,x\mid E,\mu)} @@ -268,13 +272,13 @@ solution, and some parameters, e.g., $q_1$, are unconstrained and can take any value in the old solution. There is one place where we can consistently search for a bifurcating solution -to the saddle point equations: along the zero complexity line -$\Sigma_\mathrm a(E,\mu)=0$. Going along this line in the replica symmetric solution, the +to the saddle point equations: along the zero complexity line $\Sigma_\mathrm +a(E,\mu)=0$. Going along this line in the replica symmetric solution, the {\oldstylenums1}\textsc{rsb} complexity transitions at a critical point where $x=q_1=1$ \cite{Kent-Dobias_2023_How}. Since all the parameters in the bifurcating solution are known at this point, we can search for it by looking -for a flat direction in the way described above. In the annealed -solution for points describing saddles ($\mu<\mu_\mathrm m$), this line is +for a flat direction in the way described above. In the annealed solution for +points describing saddles ($\mu<\mu_\mathrm m$), this line is \begin{equation} \label{eq:extremal.line} \mu_0=-\frac1{z_f}\left(2Ef'f''+\sqrt{2f''u_f\bigg(\log\frac{f''}{f'}z_f-E^2(f''-f')\bigg)}\right) \end{equation} @@ -319,11 +323,11 @@ by \qquad d=\frac{w_f}{f'f''} \end{equation} -Changing variables to $y$ from $\mu$ is a -convenient choice because the branch of \eqref{eq:extremal.line} is chosen -by the sign of $y$ (the lower-energy branch we are interested in corresponds -with $y>0$) and the relationship between $y$ and $E$ on the extremal line is -$g=2hy^2+eE^2$, where the constants $e$, $g$, and $h$ are given by +Changing variables to $y$ from $\mu$ is a convenient choice because the branch +of \eqref{eq:extremal.line} is chosen by the sign of $y$ (the lower-energy +branch we are interested in corresponds with $y>0$) and the relationship +between $y$ and $E$ on the extremal line is $g=2hy^2+eE^2$, where the constants +$e$, $g$, and $h$ are given by \begin{equation} e=f''-f' \qquad @@ -371,7 +375,8 @@ energy point, so that these two points give the precise range of energies at which \textsc{rsb} saddles are found. An example that conforms with this picture for a $3+5$ mixed model is shown in Fig.~\ref{fig:complexity_35}. -The expression inside the inner square root of \eqref{eq:energies} is proportional to +The expression inside the inner square root of \eqref{eq:energies} is +proportional to \begin{equation} \label{eq:condition} G_f = @@ -465,7 +470,8 @@ extended from $E_{\oldstylenums1\textsc{rsb}}^+$. } \label{fig:order} \end{figure} -There are implications for the emergence of \textsc{rsb} in equilibrium. Consider a specific $H$ with +There are implications for the emergence of \textsc{rsb} in equilibrium. +Consider a specific $H$ with \begin{equation} H(\pmb\sigma) =\frac{\sqrt\lambda}{p!}\sum_{i_1\cdots i_p}J^{(p)}_{i_1\cdots i_p}\sigma_{i_1}\cdots\sigma_{i_p} @@ -501,11 +507,10 @@ $s>2$, this transition line \emph{always} intersects the extremal line among some population of stationary points. However, it is likely that for much of the parameter space the so-called one-full \textsc{rsb} ({\oldstylenums1\textsc{frsb}}) is the correct solution, as it likely is for -large $s$ in the $3+s$ model at hand. Further work to find the -conditions for transitions of the complexity to these forms of order is -necessary. For values of $s$ where there is no -\textsc{rsb} of any kind in the ground state, we expect that the -{\oldstylenums1\textsc{rsb}} complexity is correct. +large $s$ in the $3+s$ model at hand. Further work to find the conditions for +transitions of the complexity to these forms of order is necessary. For values +of $s$ where there is no \textsc{rsb} of any kind in the ground state, we +expect that the {\oldstylenums1\textsc{rsb}} complexity is correct. What are the implications for dynamics? We find that nontrivial correlations tend to exist among saddle points with the maximum or minimum index possible at |