# A05 - Probing scales in equilibrated systems by optimal nonequilibrium forcing

### Project Details

**Status:**In Progress**Head(s):**Prof. Schütte, Prof. Hartmann, Dr. Weber**Project member(s):**Jannes Quer, Lara Neureither**Participating Institution(s):**FU Berlin, Zuse Institute Berlin (ZIB)**Area:**A: Efficient modelling of macro scales**Positions available?:**

### Project Summary

The dynamics of biomolecules show an inherent multiscale behaviour with cascades of time scales and strong interaction between them. Molecular dynamics (MD) simulations allow for analysis and, at least partly, understanding of this dynamical behaviour. However realistic simulations on timescales beyond milliseconds are still infeasible even on the most powerful computers, which renders the MD-based analysis of many important equilibrium processes – often processes that are related to biological function and require much longer simulation timescales – impossible. Driven by the recent progress in experimental techniques to manipulate single molecules, numerical nonequilibrium methods that attempt to bridge the timescale gap between the fastest random oscillations and the rare events that are related to the slowest function-related processes have gained enormous popularity. These methods are yet lacking both theoretical foundation and practicability, first and foremost due to the poor convergence of the corresponding numerical estimators.

This project aims at exploiting ideas from stochastic control, in order (1) to analyse the influence of nonequilibrium perturbation on the statistics of a system when it is driven out of thermodynamic equilibrium and (2) to devise novel efficient importance sampling strategies based on optimal controls that speed up the sampling of the relevant rare events while giving statistical estimators with small variance and good convergence properties, beyond the asymptotic regime of large deviations theory.

Fluctuation-dissipation and large deviations theorems provide exact expressions relating random nonequilibrium energy fluctuations (often in form of dissipated heat and work) to the respective equilibrium quantities (e.g., free energies or rates). These relations can be expressed in form of cumulant generating functions for the fluctuating quantities; they admit a dual representation in form of stochastic control problems. Based on this duality the first focus of the project is on the question of whether it is possible to compute equilibrium quantities with optimal efficiency (i.e., with zero variance) by optimally driving the molecular process into nonequilibrium. Moreover, the duality provides an abstract framework for both discrete- and continuous-state Markov processes, by which we can analyse which factors influence the variance of the nonequilibrium estimators and how to control it. The next question then is how the optimal controls can be computed in practice, especially for large multiscale molecular systems. Thererfore the second focus of the project will be on the analysis and development of numerical strategies for computing optimal controls. The high spatial dimension and different time scales in molecular systems require problem-adapted discretisation techniques like Markov State Models (MSM). The theory of MSM has been developed in the context of (reversible) equilibrium molecular systems, hence the goal of the project is to find MSM-based coarse-graining techniques for nonequilibrium optimal control problems and to analyse their approximation quality with regard to the relevant observables and the optimal controls that can be computed from them.

The theoretical investigations and the numerical analysis will be complemented by applications to small but realistic molecular systems. In the first phase we plan to apply the novel techniques to molecular systems showing multivalency starting with bivalent ammonium-pseudorotaxane-systems. The long-term goal of the project is the design of novel numerical methods that allow efficient sampling of rare events on very long timescales (like multivalent binding, folding, or aggregation) with low variance and minimal cost of the sampling step.

### Project Publications

Nüsken, N. and Richter, L. (2020) *Solving high-dimensional Hamilton-Jacobi-Bellman PDEs using neural networks: perspectives from the theory of controlled diffusions and measures on path space.* SFB 1114 Preprint in arXiv . pp. 1-40. (Submitted)

Hartmann, C. and Neureither, L. and Sharma, U. (2019) *Coarse-graining of non-reversible stochastic differential equations: quantitative results and connections to averaging.* SIAM J. Math. Anal. . pp. 1-39. (Submitted)

Kebiri, O. and Neureither, L. and Hartmann, C. (2019) *Adaptive importance sampling with forward-backward stochastic differential equations.* In: Stochastic Dynamics Out of Equilibrium. IHPStochDyn 2017. Springer Proceedings in Mathematics & Statistics, 282 . Springer, pp. 265-281. ISBN 978-3-030-15095-2

Neureither, L. and Hartmann, C. (2019) *Time scales and exponential trends to equilibrium: Gaussian model problems.* In: Stochastic Dynamics Out of Equilibrium. IHPStochDyn 2017. Springer Proceedings in Mathematics & Statistics, 282 . Springer, pp. 391-410. ISBN 978-3-030-15095-2

Hartmann, C. and Kebiri, O. and Neureither, L. and Richter, L. (2019) *Variational approach to rare event simulation using least-squares regression.* Chaos, 29 (6). 063107. ISSN 1054-1500 (print); 1089-7682 (online)

Hartmann, C. and Schütte, Ch. and Zhang, W. (2019) *Jarzysnki equality, fluctuation theorems and variance reduction: Mathematical analysis and numerical algorithms.* J. Stat. Phys., 175 (6). pp. 1214-1261. ISSN 0022-4715; ESSN: 1572-9613

Donati, L. and Heida, M. and Weber, M. and Keller, B. (2018) *Estimation of the infinitesimal generator by square-root approximation.* Journal of Physics: Condensed Matter, 30 (42). p. 425201. ISSN 0953-8984, ESSN: 1361-648X

Kebiri, O. and Neureither, L. and Hartmann, C. (2018) *Singularly perturbed forward-backward stochastic differential equations: application to the optimal control of bilinear systems.* Computation, 6 (3). p. 41. ISSN 2079-3197 (online)

Hartmann, C. and Schütte, Ch. and Weber, M. and Zhang, W. (2018) *Importance sampling in path space for diffusion processes with slow-fast variables.* Probab. Theory Rel. Fields, 170 (1-2). pp. 177-228. ISSN 0178-8051 (print) 1432-2064 (online)

Weber, M. (2018) *Implications of PCCA+ in Molecular Simulation.* Computation, 6 (1). ISSN 2079-3197 (online)

Hartmann, C. and Richter, L. and Schütte, Ch. and Zhang, W. (2017) *Variational characterization of free energy: theory and algorithms.* Entropy (Special Issue), 19 (11). pp. 1-27. ISSN 1099-4300

Quer, J. and Lie, H. (2017) *Some connections between importance sampling and enhanced sampling methods in molecular dynamics.* Journal of Chemical Physics . pp. 1-19. ISSN 0021-9606

Delle Site, L. and Ciccotti, G. and Hartmann, C. (2017) *Partitioning a macroscopic system into independent subsystems.* Journal of Statistical Mechanics: Theory and Experiment, 2017 . pp. 1-13.

Quer, J. and Donati, L. and Keller, B.G. and Weber, M. (2017) *An automatic adaptive importance sampling algorithm for molecular dynamics in reaction coordinates.* SIAM J. Sci. Comput. . pp. 1-19. ISSN 1064-8275 (print) (In Press)

Weber, M. and Fackeldey, K. and Schütte, Ch. (2017) *Set-free Markov state model building.* Journal of Chemical Physics, 146 (12). p. 124133.

Koltai, P. and Ciccotti, G. and Schütte, Ch. (2016) *On metastability and Markov state models for non-stationary molecular dynamics.* Journal of Chemical Physics, 145 (17). p. 174103.

Quer, J. and Weber, M. (2016) *Estimating exit rate for rare event dynamical systems by extrapolation.* ZIB-Report . pp. 1-19. ISSN 2192-7782 (online)

Banisch, Ralf and Hartmann, C. (2016) *A sparse Markov chain approximation of LQ-type stochastic control problems.* Math. Control Relat. F., 6 (3). pp. 363-389. ISSN 1064-8275

Hartmann, C. and Schütte, Ch. and Zhang, W. (2016) *Model reduction algorithms for optimal control and importance sampling of diffusions.* Nonlinearity, 29 (8). pp. 2298-2326. ISSN 0951-7715

Zhang, W. and Hartmann, C. and Schütte, Ch. (2016) *Effective dynamics along given reaction coordinates, and reaction rate theory.* Faraday discussions, 195 . pp. 365-394. ISSN 1359-6640

Bittracher, Andreas and Hartmann, C. and Junge, O. and Koltai, Péter (2015) *Pseudo generators for under-resolved molecular dynamics.* The European Physical Journal Special Topics, 224 (12). pp. 2463-2490. ISSN 1951-6355

Hartmann, C. and Latorre, J.C. and Pavliotis, G. A. and Zhang, W. (2014) *Optimal control of multiscale systems using reduced-order models.* J. Computational Dynamics, 1 (2). pp. 279-306. ISSN 2158-2505

Lie, Han Cheng and Schütte, Ch. and Hartmann, C. (2014) *Martingale-based gradient descent algorithm for estimating free energy values of diffusions.* SIAM J. Sci. Comput. . ISSN 1064-8275 (Submitted)