From 36947d8407c36560e991d4243001d08cddbee829 Mon Sep 17 00:00:00 2001 From: Jaron Kent-Dobias Date: Thu, 20 Oct 2022 13:59:49 +0200 Subject: Lots of writing. --- frsb_kac-rice_letter.tex | 76 ++++++++++++++++++++++++++++++++++-------------- 1 file changed, 54 insertions(+), 22 deletions(-) diff --git a/frsb_kac-rice_letter.tex b/frsb_kac-rice_letter.tex index d3369ac..801dc8e 100644 --- a/frsb_kac-rice_letter.tex +++ b/frsb_kac-rice_letter.tex @@ -45,13 +45,13 @@ complex landscapes also exist in very high dimensions: think of the dimensions of phase space for $N$ particles, or the number of parameters in a neural network. -The \emph{complexity} of a function is the logarithm of the average number of -its minima, maxima, and saddle points (collectively stationary points), under -conditions like the value of the energy or the index of the stationary point. -Since in complex landscapes this number grows exponentially with system size, -their complexity is an extensive quantity. Understanding the complexity offers -an understanding about the geometry and topology of the landscape, which can -provide insight into dynamical behavior. +The \emph{complexity} of a function is the average of the logarithm of the +number of its minima, maxima, and saddle points (collectively stationary +points), under conditions like the value of the energy or the index of the +stationary point. Since in complex landscapes this number grows exponentially +with system size, their complexity is an extensive quantity. Understanding the +complexity offers an understanding about the geometry and topology of the +landscape, which can provide insight into dynamical behavior. When complex systems are fully connected, i.e., each degree of freedom interacts directly with every other, they are often described by a hierarchical @@ -65,20 +65,20 @@ only been computed in RS cases. In this paper we share the first results for the complexity with nontrivial hierarchy. Using a general form for the solution, we detail the structure of -landscapes with a 1RSB complexity and a full RSB complexity. - -The Thouless--Anderson--Palmer (TAP) complexity is the complexity of a kind of -mean-field free energy. Because of some deep thermodynamic relationships -between the TAP complexity and the equilibrium free energy, the TAP complexity -can be computed with extensions of the equilibrium method. As a result, the TAP -complexity has been previously computed for nontrivial hierarchical structure. +landscapes with a 1RSB complexity and a full RSB complexity \footnote{The + Thouless--Anderson--Palmer (TAP) complexity is the complexity of a kind of + mean-field free energy. Because of some deep thermodynamic relationships + between the TAP complexity and the equilibrium free energy, the TAP + complexity can be computed with extensions of the equilibrium method. As a + result, the TAP complexity has been previously computed for nontrivial +hierarchical structure.}. We study the mixed $p$-spin spherical models, with Hamiltonian \begin{equation} \label{eq:hamiltonian} H(\mathbf s)=-\sum_p\frac1{p!}\sum_{i_1\cdots i_p}^NJ^{(p)}_{i_1\cdots i_p}s_{i_1}\cdots s_{i_p} \end{equation} -is defined for vectors $\mathbf s\in\mathbb R^N$ confined to the sphere -$\|\mathbf s\|^2=N$. The coupling coefficients $J$ are taken at random, with +is defined for vectors $\mathbf s\in\mathbb R^N$ confined to the $N-1$ sphere +$S^{N-1}=\{\mathbf s\mid\|\mathbf s\|^2=N\}$. The coupling coefficients $J$ are taken at random, with zero mean and variance $\overline{(J^{(p)})^2}=a_pp!/2N^{p-1}$ chosen so that the energy is typically extensive. The overbar will always denote an average over the coefficients $J$. The factors $a_p$ in the variances are freely chosen @@ -136,7 +136,7 @@ It's worth reviewing the complexity for the best-studied case of the pure model for $p\geq3$. Here, because the covariance is a homogeneous polynomial, $E$ and $\mu$ cannot be fixed separately, and one implies the other: $\mu=pE$. Therefore at each energy there is only one kind of stationary point. When the -energy reaches $E_\mathrm{th}=\mu_m/p$, the population of stationary points +energy reaches $E_\mathrm{th}=-\mu_m/p$, the population of stationary points suddenly shifts from all saddles to all minima, and there is an abrupt percolation transition in the topology of constant-energy slices of the landscape. This behavior of the complexity can be used to explain a rich @@ -148,11 +148,42 @@ dynamics, and to the algorithmic limit $E_\mathrm{alg}$. Things become much less clear in even the simplest mixed models. For instance, one mixed model known to have a replica symmetric complexity was shown to nonetheless not have a clear relationship between features of the complexity -and the asymptotic dynamics \cite{Folena_2020_Rethinking}. +and the asymptotic dynamics \cite{Folena_2020_Rethinking}. There is no longer a +sharp topological transition. To compute the complexity in the generic case, we use the replica method to treat the logarithm inside the average of \eqref{eq:complexity}, and the -$\delta$-functions are written in a Fourier basis. +$\delta$-functions are written in a Fourier basis. The average of the factor +including the determinant and the factors involving $\delta$-functions can be +averaged over the disorder separately \cite{Bray_2007_Statistics}. The result +can be written as a function of three matrices indexed by the replicas: one +which is a clear analogue of the usual overlap matrix of the equilibrium case, +and two which can be related to the response of stationary points to +perturbations of the potential. The general expression for the complexity as a +function of these matrices is also found in \cite{Folena_2020_Rethinking}. + +We make the \emph{ansatz} that all three matrices have a hierarchical +structure, and moreover that they share the same hierarchical structure. This +means that the size of the blocks of equal value of each is the same, though +the values inside these blocks will vary from matrix to matrix. This form can +be shown to exactly reproduce the ground state energy predicted by the +equilibrium solution, a key consistency check. + +Along one line in the energy--stability plane the solution takes a simple form: +the two hierarchical matrices corresponding to responses are diagonal, leaving +only the overlap matrix with nontrivial off-diagonal entries. This +simplification makes the solution along this line analytically tractable even +for FRSB. The simplification is related to the presence of an approximate +supersymmetry in the Kac--Rice formula, studied in the past in the context of +the TAP free energy. This line of `supersymmetric' solutions terminates at the +ground state, and describes the most numerous types of stable minima. + +In general, solving the saddle-point equations for the parameters of the three +replica matrices is challenging. Unlike the equilibrium case, the solution is +not extremal, and so minimization methods cannot be used. However, the line of +simple `supersymmetric' solutions offers a convenient foothold: starting from +one of these solutions, the parameters $E$ and $\mu$ can be slowly varied to +find the complexity everywhere. This is how the data in what follows was produced. \begin{figure} \centering @@ -176,9 +207,10 @@ $\delta$-functions are written in a Fourier basis. } \label{fig:2rsb.phases} \end{figure} -It is known that by choosing a covariance $f$ as the sum of polynomials with -well-separated powers, one develops 2RSB in equilibrium. This should correspond -to 1RSB in Kac--Rice. For this example, we take +For the first example, we study a model whose complexity has the simplest +replica symmetry breaking scheme, 1RSB. By choosing a covariance $f$ as the sum +of polynomials with well-separated powers, one develops 2RSB in equilibrium. +This should correspond to 1RSB in the complexity. For this example, we take \begin{equation} f(q)=\frac12\left(q^3+\frac1{16}q^{16}\right) \end{equation} -- cgit v1.2.3-70-g09d2