summaryrefslogtreecommitdiff
diff options
context:
space:
mode:
authorJaron Kent-Dobias <jaron@kent-dobias.com>2023-06-14 20:02:30 +0200
committerJaron Kent-Dobias <jaron@kent-dobias.com>2023-06-14 20:02:30 +0200
commit68949be308083087de56a2ba0b775880d23bc913 (patch)
treecd19ad421e8dffa112c94c10ae2628e37b1d3723
parent37bcaf67a5c37a803675ba2026ca292ff6ef2aba (diff)
downloadEPL_143_61003-68949be308083087de56a2ba0b775880d23bc913.tar.gz
EPL_143_61003-68949be308083087de56a2ba0b775880d23bc913.tar.bz2
EPL_143_61003-68949be308083087de56a2ba0b775880d23bc913.zip
Many changes.
-rw-r--r--figs/range_plot_1.pdfbin0 -> 10277 bytes
-rw-r--r--figs/range_plot_2.pdfbin0 -> 9991 bytes
-rw-r--r--figs/range_plot_3.pdfbin0 -> 10022 bytes
-rw-r--r--figs/range_plot_4.pdfbin0 -> 17355 bytes
-rw-r--r--figs/range_plot_log_1.pdfbin0 -> 10598 bytes
-rw-r--r--figs/range_plot_log_2.pdfbin0 -> 8486 bytes
-rw-r--r--figs/range_plot_log_3.pdfbin0 -> 8640 bytes
-rw-r--r--figs/range_plot_log_4.pdfbin0 -> 14509 bytes
-rw-r--r--when_annealed.tex171
9 files changed, 110 insertions, 61 deletions
diff --git a/figs/range_plot_1.pdf b/figs/range_plot_1.pdf
new file mode 100644
index 0000000..e74097c
--- /dev/null
+++ b/figs/range_plot_1.pdf
Binary files differ
diff --git a/figs/range_plot_2.pdf b/figs/range_plot_2.pdf
new file mode 100644
index 0000000..9fd1124
--- /dev/null
+++ b/figs/range_plot_2.pdf
Binary files differ
diff --git a/figs/range_plot_3.pdf b/figs/range_plot_3.pdf
new file mode 100644
index 0000000..8fb346a
--- /dev/null
+++ b/figs/range_plot_3.pdf
Binary files differ
diff --git a/figs/range_plot_4.pdf b/figs/range_plot_4.pdf
new file mode 100644
index 0000000..0b19e45
--- /dev/null
+++ b/figs/range_plot_4.pdf
Binary files differ
diff --git a/figs/range_plot_log_1.pdf b/figs/range_plot_log_1.pdf
new file mode 100644
index 0000000..4ef3fd3
--- /dev/null
+++ b/figs/range_plot_log_1.pdf
Binary files differ
diff --git a/figs/range_plot_log_2.pdf b/figs/range_plot_log_2.pdf
new file mode 100644
index 0000000..1c2f9d2
--- /dev/null
+++ b/figs/range_plot_log_2.pdf
Binary files differ
diff --git a/figs/range_plot_log_3.pdf b/figs/range_plot_log_3.pdf
new file mode 100644
index 0000000..ea24c21
--- /dev/null
+++ b/figs/range_plot_log_3.pdf
Binary files differ
diff --git a/figs/range_plot_log_4.pdf b/figs/range_plot_log_4.pdf
new file mode 100644
index 0000000..fada1e9
--- /dev/null
+++ b/figs/range_plot_log_4.pdf
Binary files differ
diff --git a/when_annealed.tex b/when_annealed.tex
index b5c9746..90dc067 100644
--- a/when_annealed.tex
+++ b/when_annealed.tex
@@ -49,36 +49,37 @@
\end{abstract}
Random energies, cost functions, and interaction networks are important to many
-areas of modern science. The energy landscape glasses, the likelihood landscape
-of machine learning and high-dimensional inference, and the interactions
-between organisms in an ecosystem are just a few examples. A traditional tool
-for making sense of the diverse behavior these systems exhibit is to analyze
-the statistics of points where their dynamics is stationary. For energy or cost
-landscapes, these correspond to the minima, maxima, and saddles of the
-function, while for ecosystems and other purely dynamical systems these
-correspond to the equilibria of the dynamics. When many stationary points are
-present, the system is considered complex.
+areas of modern science. The energy landscape of glasses, the likelihood
+landscape of machine learning and high-dimensional inference, and the
+interactions between organisms in an ecosystem are just a few examples. A
+traditional tool for making sense of their behavior is to analyze the
+statistics of points where their dynamics are stationary. For energy or cost
+landscapes, these correspond to the minima, maxima, and saddles, while for
+ecosystems and other non-gradient dynamical systems these correspond to
+equilibria of the dynamics. When many stationary points are present, the system
+is considered complex.
Despite the importance of stationary point statistics for understanding complex
behavior, they are often calculated using an uncontrolled approximation.
-Because their number is so large, it cannot reliably be averaged over disorder
-in the system. The annealed approximation takes this average anyway, risking a
-systematic bias by rare and atypical samples. The anneal approximation is known
-to be exact for certain models and in certain circumstances, but it is widely
-used outside those circumstances without much reflection. In a few cases, the
-controlled quenched average, which averages the logarithm of their number, has
-been considered \cite{Ros_2019_Complexity, Kent-Dobias_2023_How, Ros_2023_Quenched}.
+Because their number is so large, it cannot be reliably averaged. The annealed
+approximation takes this average anyway, risking a systematic bias by rare and
+atypical samples. The annealed approximation is known to be exact for certain
+models and in certain circumstances, but it is used outside those circumstances
+without much reflection. In a few cases researches have made instead the
+better-controlled quenched average, which averages the logarithm of the number
+of stationary points \cite{Ros_2019_Complexity, Kent-Dobias_2023_How,
+Ros_2023_Quenched}.
-One heuristic line of reasoning for the correctness of the annealed
+A heuristic line of reasoning for the correctness of the annealed
approximation for the statistics of stationary points is sometimes made when
the annealed approximation is correct for an equilibrium calculation on the same
system. The argument goes like this: since the limit of zero temperature or
noise in an equilibrium calculation concentrates the measure onto the lowest
-set of minima, the equilibrium average at very low temperature should be
-governed by the same statistics as the count of that lowest set of minima. This
-argument is valid, but only for the lowest set of minima, which at least in
-glassy problems are rarely relevant to dynamical behavior. What about the
-\emph{rest} of the stationary points?
+set of minima, the equilibrium free energy in the limit to zero temperature
+should be governed by the same statistics as the count of that lowest set of
+minima. This argument is valid, but only for the lowest set of minima, which at
+least in glassy problems are rarely relevant to dynamical behavior. What about
+the \emph{rest} of the stationary points?
In this paper, we show that the behavior of the ground state, or \emph{any}
equilibrium behavior, does not govern whether stationary points will have a
@@ -89,11 +90,12 @@ position. We produce examples of models whose equilibrium is guaranteed to
never see correlations between states, but where a population of saddle points
is correlated.
-We study classes of Gaussian-correlated random functions with isotropic
-statistics on the $(N-1)$-sphere. Each class of functions $H:S^{N-1}\to\mathbb
-R$ is defined by the average covariance between the function evaluated at two
-different points $\pmb\sigma_1,\pmb\sigma_2\in S^{N-1}$, which is a function of
-the scalar product or overlap between the two configurations:
+We study the mixed spherical models, which are models of Gaussian-correlated
+random functions with isotropic statistics on the $(N-1)$-sphere. Each class of
+functions $H:S^{N-1}\to\mathbb R$ is defined by the average covariance between
+the function evaluated at two different points $\pmb\sigma_1,\pmb\sigma_2\in
+S^{N-1}$, which is a function of the scalar product or overlap between the two
+configurations:
\begin{equation} \label{eq:covariance}
\overline{H(\pmb\sigma_1)H(\pmb\sigma_2)}=\frac1Nf\bigg(\frac{\pmb\sigma_1\cdot\pmb\sigma_2}N\bigg)
\end{equation}
@@ -109,7 +111,7 @@ These classes of functions have been extensively studied in the physics and
statistics literature, and host a zoo of complex orders and phase transitions
\cite{Crisanti_2004_Spherical, Crisanti_2006_Spherical, Crisanti_2011_Statistical}.
-There are two important features which differentiate stationary points in the
+There are two important features which differentiate stationary points $\pmb\sigma^*$ in the
spherical models: their \emph{energy density} $E=\frac1NH(\pmb\sigma^*)$ and
their \emph{stability}
$\mu=\frac1N\operatorname{\mathrm{Tr}}\operatorname{\mathrm{Hess}}H(\pmb\sigma^*)$.
@@ -138,17 +140,25 @@ $\mu=\mu_\mathrm m$, the spectrum has a pseudogap, and we have marginal minima.
\end{figure}
The number $\mathcal N(E,\mu)$ of stationary points with energy density $E$ and
-stability $\mu$ is exponential in $N$ for these models. The complexity is
-defined by the average of the logarithm of this, or
+stability $\mu$ is exponential in $N$ for these models. Their complexity $\Sigma(E,\mu)$ is
+defined by the average of the logarithm of their number, or
$\Sigma(E,\mu)=\frac1N\overline{\log\mathcal N(E,\mu)}$. More often the annealed
complexity is calculated, where the average is taken before the logarithm:
$\Sigma_\mathrm a(E,\mu)=\frac1N\log\overline{\mathcal N(E,\mu)}$.
When the complexity is calculated using the Kac--Rice formula and a physicists'
tool set, the problem is reduced to the evaluation of an integral by the saddle
-point method for large $N$ \cite{Kent-Dobias_2023_How}. The complexity is given
-by extremizing an effective action,
-$\Sigma(E,\mu)=\mathop{\mathrm{extremum}}_{q_1,x}\mathcal S(q_1,x\mid E,\mu)$
+point method for large $N$ \cite{Kent-Dobias_2023_How}. An ansatz for
+the complexity needs to be made. Here we restrict ourselves to a
+{\oldstylenums1}\textsc{rsb} ansatz. We are guaranteed that
+$\Sigma\leq\Sigma_{\oldstylenums1\textsc{rsb}}\leq\Sigma_\mathrm a$, and we
+will discuss later in what settings the {\oldstylenums1}\textsc{rsb}
+complexity is correct. The complexity is given by extremizing an effective
+action,
+\begin{equation}
+ \Sigma_{\oldstylenums1\textsc{rsb}}(E,\mu)=\lim_{n\to0}\int dq_1\,dx\,\mathcal S(q_1,x\mid E,\mu)e^{nN\mathcal S(q_1,x\mid E,\mu)}
+ =\mathop{\mathrm{extremum}}_{q_1,x}\mathcal S(q_1,x\mid E,\mu)
+\end{equation}
for the action $\mathcal S$ given by
\begin{equation}
\begin{aligned}
@@ -183,27 +193,16 @@ where $\Delta x=1-x$ and
-\log\left(\left|\frac{\mu}{\mu_\text m}\right|-\sqrt{\big(\frac\mu{\mu_\text m}\big)^2-1}\right) & \mu^2>\mu_\text m^2
\end{cases}
\end{equation}
-The extremal problem is quadratic in $\hat\beta$, $r_\mathrm d$, $r_1$,
-$d_\mathrm d$, and $d_1$ and therefore its solution is unique and can be found explicitly, but the
+The extremal problem in $\hat\beta$, $r_\mathrm d$, $r_1$,
+$d_\mathrm d$, and $d_1$ has a unique solution and can be found explicitly, but the
resulting formula is much more complicated so we do not include it here. There
can be multiple extrema at which to evaluate $\mathcal S$, in this case the one
for which $\Sigma$ is \emph{smallest} gives the correct solution. There is
always a solution for $x=1$ which is independent of $q_1$, which corresponds to
-the replica symmetric case and which is equal to the annealed calculation. The
-crux of this paper will be to determine when this solution is not the global
-one.
+the replica symmetric case and which is equal to the annealed calculation, so
+$\Sigma_\mathrm a(E,\mu)=\mathcal S(E,\mu\mid q_1,1)$. The crux of this paper
+will be to determine when this solution is not the global one.
-where we define for brevity (here and elsewhere) the constants
-\begin{align}
- u_f=f(f'+f'')-f'^2
- &&
- v_f=f'(f''+f''')-f''^2 \\
- w_f=2f''^2+f'(f'''-2f'')
- &&
- y_f=f'(f'-f)+f''f
- &&
- z_f=f(f''-f')+f'^2
-\end{align}
It isn't accurate to say that a solution to the saddle point equations is
`stable' or `unstable.' The problem of solving the complexity in this way is
not a variational problem, so there is nothing to be maximized or minimized,
@@ -212,7 +211,7 @@ the Hessian. However, the eigenvalues of the Hessian can still tell us
something about the emergence of new solutions: when another solution
bifurcates smoothly from an existing one, the Hessian evaluated at that point
will have a zero eigenvalue. Unfortunately this is a difficult procedure to
-apply in general, since on must know the parameters in the new solution, and
+apply in general, since one must know the parameters in the new solution, and
some parameters, e.g., $q_1$, are unconstrained in the old solution.
There is one place where one can consistently search for a bifurcating solution
@@ -226,6 +225,17 @@ solution for points describing saddles, this line is
\begin{equation} \label{eq:extremal.line}
\mu_0=-\frac1{z_f}\left(2Ef'f''+\sqrt{2f''u_f\bigg(\log\frac{f''}{f'}z_f-E^2(f''-f')\bigg)}\right)
\end{equation}
+where we define for brevity (here and elsewhere) the constants
+\begin{align}
+ u_f=f(f'+f'')-f'^2
+ &&
+ v_f=f'(f''+f''')-f''^2 \\
+ w_f=2f''^2+f'(f'''-2f'')
+ &&
+ y_f=f'(f'-f)+f''f
+ &&
+ z_f=f(f''-f')+f'^2
+\end{align}
Let $M$ be the matrix of double partial derivatives of $\mathcal S$ with
respect to $q_1$ and $x$. We evaluate $M$ at the replica symmetric saddle point
$x=1$ with the additional constraint that $q_1=1$ and along the extremal
@@ -233,30 +243,38 @@ complexity line \eqref{eq:extremal.line}. We determine when a zero eigenvalue
appears, indicated the presence of a bifurcating {\oldstylenums1}\textsc{rsb}
solution, by solving $0=\det M$. We find
\begin{equation}
- \det M=-\left(\frac{\partial^2\mathcal S}{\partial q_1\partial x}\bigg|_{\substack{x=1\\q_1=1}}\right)^2\propto(ay^2+bE^2+cyE+d)^2
+ \det M=-\left(\frac{\partial^2\mathcal S}{\partial q_1\partial x}\bigg|_{\substack{x=1\\q_1=1}}\right)^2\propto(ay^2+bE^2+2cyE+d)^2
\end{equation}
-where $y^2=eE^2+g$ and the constants $a$, $b$, $c$, $d$, $e$, and $g$ are defined by
+where $y=-\frac12z_f\mu-f'f''E$ is proportional to the square-root term in \eqref{eq:extremal.line} and
+the constants $a$, $b$, $c$, and $d$ are defined below. Changing to $y$ is a
+convenient choice because the branch of \eqref{eq:extremal.line} is chosen
+by the sign of $y$ (the lower-energy branch we are interested in corresponds
+with $y>0$) and the relationship between $y$ and $E$ on the extremal line is
+$g=2h^2y^2+eE^2$, where the constants $e$, $g$, and $h$ are given by
\begin{equation}
\begin{aligned}
- a&=\frac{f''}{u_f}\left[
- \big(3y_f^2-4ff'f''(f'-f)\big)f'''-8f(f'-f)f''^2(f''-f')
- \right]
+ a&=\frac{w_f\big(3y_f^2-4ff'f''(f'-f)\big)-6y_f^2(f''-f')f''}{(u_fz_ff'')^2f'}
\qquad
- b=2f'f''^2w_f
+ b=\frac{f'w_f}{z_f^2}
\\
- c&=2w_f\sqrt{2f''^3u_f}
+ c&=\frac{w_f}{f''z_f^2}
\qquad
- d=-\frac{2f''}{f'}z_f^2w_f
+ d=-\frac{w_f}{f'f''}
\qquad
- e=-(f''-f')
+ e=f''-f'
\qquad
g=z_f\log\frac{f''}{f'}
+ \qquad
+ h=\frac1{f''u_f}
\end{aligned}
\end{equation}
The solutions for $\det M=0$ correspond to energies that satisfy
\begin{equation}
- E_{\oldstylenums1\textsc{rsb}}
- =-\sqrt{\frac12\frac{c^2g-2(ade+a^2eg+b(d+ag))\pm|c|\sqrt{4d^2e-4d(b-ae)g+(c^2-4ab)g^2}}{b^2+2abe+e(a^2e-c^2)}}
+ E_{\oldstylenums1\textsc{rsb}}^\pm
+ =-\operatorname{sign}[c(de+bg)]\frac{|c|g\mp\sqrt{c^2g^2-(2dh+ag)(de+bg)}}
+ {
+ \sqrt{2c^2eg+(2bh-ae)(de+bg)\mp2|c|e\sqrt{c^2g^2-(2dh+ag)(de+bg)}}
+ }
\end{equation}
The expression inside the inner square root is proportional to
\begin{equation}
@@ -294,6 +312,37 @@ between them. Therefore, $G_f>0$ is a necessary condition to see
} \label{fig:complexity_35}
\end{figure}
+\begin{figure}
+ \centering
+ \includegraphics{figs/range_plot_1.pdf}
+ \hspace{-3em}
+ \includegraphics{figs/range_plot_2.pdf}
+ \hspace{-3em}
+ \includegraphics{figs/range_plot_3.pdf}
+ \hspace{-3em}
+ \includegraphics{figs/range_plot_4.pdf} \\
+ \vspace{-2em}
+ \includegraphics{figs/range_plot_log_1.pdf}
+ \hspace{-3em}
+ \includegraphics{figs/range_plot_log_2.pdf}
+ \hspace{-3em}
+ \includegraphics{figs/range_plot_log_3.pdf}
+ \hspace{-3em}
+ \includegraphics{figs/range_plot_log_4.pdf}
+
+ \caption{
+ The range of energies where \textsc{rsb} saddles are found for the $3+s$
+ model with varying $s$ and $\lambda$. In the top row the black line shows
+ $E_\textrm{min}$, the minimum energy where saddles are found, and in the
+ bottom row this energy is subtracted away to emphasize when the
+ \textsc{rsb} region crosses into minima. For most $s$, both the top and
+ bottom lines are given by $E_{\oldstylenums1\textsc{rsb}}$, but for $s=14$
+ there is a portion where the low-energy boundary has $q_1<1$. In that plot,
+ the continuation of the $E_{\oldstylenums1\textsc{rsb}}$ line is shown
+ dashed.
+ }
+\end{figure}
+
\begin{equation}
\mu
=-\frac{(f_1'+f_0'')u_f}{(2f_1-f_1')f_1'f_0''^{1/2}}