Trained recurrent neural networks (RNNs) are nonlinear systems that can be trained to do a variety of tasks (for example, finite state computation) in the presence of input. In this talk I will discuss some work on how excitable network attractors can be used to explain some aspects of function and malfunction in such networks.

Piecewise expanding interval maps, such as the doubling map, serve as the simplest examples of chaotic dynamical systems. A fertile approach to providing good statistical descriptions of these systems exhibiting highly irregular behaviour is to lift the dynamics to linear evolution operators acting on infinite dimensional Banach spaces. Spectral data of these operators, known as Koopman or transfer operators, yield insight into the long-term statistical behaviour of the underlying systems. In this talk I will first provide a brief overview of this approach, and will then focus on an algorithm known as Extended Dynamic Mode Decomposition (EDMD), which has been demonstrated to be empirically effective at identifying intrinsic modes of dynamical systems. The algorithm amounts to constructing a finite matrix by observing a dynamical system through a number of observables at a sequence of phase space points. While empirically successful, there are few rigorous results on the convergence this algorithm. In this context, I will report on recent joint work with Elliz Akindji, Wolfram Just and Julia Slipantschuk establishing convergence of EDMD for piecewise expanding analytic interval maps with a suitable choice of observables and phase space points. I will also briefly mention a choice of observables where EDMD may fail to produce dynamically relevant modes.

In this talk I will introduce tensor network kernel machines. These models are able to learn nonlinear patterns from data for both regression and classification tasks and are described by an exponential amount of model parameters. I will demonstrate that such models can be learned efficiently and at the same time achieve state-of-the art performance on validation data.

Computational audio analysis -also called machine listening- involves the development of algorithms capable of extracting information from audio signals such as speech, music, or environmental sounds, typically drawing knowledge from the fields of digital signal processing and machine learning. The talk will outline recent research carried out at the Centre for Digital Music of Queen Mary University of London that focuses on the investigation and use of machine learning methodologies for making sense of music and audio data. Topics covered will include the design of new architectures and learning objectives for music and audio analysis, the adaptation of machine listening systems to new domains, interpretable machine learning for audio and music, and machine listening models that learn from limited data.

There is a mystery at the heart of operator learning: how can one recover a non-self-adjoint operator from data without probing the adjoint? Current practical approaches suggest that one can accurately recover an operator while only using data generated by the forward action of the operator without access to the adjoint. However, naively, it seems essential to sample the action of the adjoint for learning time-dependent PDEs. In this talk, we will show that one can approximate a family of non-self-adjoint infinite-dimensional compact operators via projection onto a Fourier basis without querying the adjoint. We will apply the result to recover Green’s functions of elliptic partial differential operators and derive an adjoint-free sample complexity bound. While existing theory justifies low sample complexity in operator learning, ours is the first adjoint-free analysis that attempts to close the gap between theory and practice.

Koopman operators globally linearize nonlinear dynamical systems, and their spectral information serves as a powerful tool for analysing and decomposing nonlinear dynamics. However, Koopman operators are inherently infinite-dimensional, posing a significant challenge in computing their spectral information, especially in the presence of continuous spectra. We can often access this spectral information by considering a rigged-Hilbert space structure that allows us to diagonalise the operator. In the rigged Hilbert space setting, there is a smaller topological space which contains some structure corresponding to the operator, and the dual of this space where the generalised eigenvectors live. We have developed an algorithm capable of computing these generalized eigenfunctions for Koopman operators associated with general measure-preserving systems. This algorithm leverages the resolvent to compute smoothed approximations of generalized eigenfunctions. We demonstrate explicit high-order convergence theorems for our algorithm, termed RiggedDMD. These generalized eigenfunctions enable a rigorous approach to Koopman mode decomposition. We demonstrate the algorithm on several examples, include systems with Lebesgue spectrum, integrable systems, the Lorenz system and a turbulent boundary layer flow with Reynolds number $6\times 10^{4}$  and state-space dimension $>10^5$.

A major challenge in the study of dynamical systems is that of model discovery: turning data into reduced order models that are not just predictive, but provide insight into the nature of the underlying dynamical system that generated the data. We introduce a number of data-driven strategies for discovering nonlinear multiscale dynamical systems and their embeddings from data. We consider two canonical cases: (i) systems for which we have full measurements of the governing variables, and (ii) systems for which we have incomplete measurements. For systems with full state measurements, we show that the recent sparse identification of nonlinear dynamical systems (SINDy) method can discover governing equations with relatively little data and introduce a sampling method that allows SINDy to scale efficiently to problems with multiple time scales, noise and parametric dependencies. For systems with incomplete observations, we show that the Hankel alternative view of Koopman (HAVOK) method, based on time-delay embedding coordinates and the dynamic mode decomposition, can be used to obtain a linear models and Koopman invariant measurement systems that nearly perfectly captures the dynamics of nonlinear quasiperiodic systems. Neural networks are used in targeted ways to aid in the model reduction process. Together, these approaches provide a suite of mathematical strategies for reducing the data required to discover and model nonlinear multiscale systems.

Signature kernels are at the core of several machine learning algorithms for analysing multivariate time series. The kernels of bounded variation paths, such as piecewise linear interpolations of time series data, are typically computed by solving a linear hyperbolic second-order PDE. However, this approach becomes considerably less practical for highly oscillatory inputs, due to significant time and memory complexities.

To mitigate this issue, I will introduce a high-order method which involves replacing the original PDE, which has rapidly varying coefficients, with a system of coupled equations with piecewise constant coefficients. These coefficients are derived from the first few terms of the log-signatures of the input paths and can be computed efficiently using existing Python libraries.

We discuss a generative modelling strategy for the stochastic parametrisation of geophysical fluid dynamics models. Stochastic parametrisations are highly desired in fluid dynamics as they enable a principled ensemble generation for forecast and uncertainty quantification applications. Our contribution is a method to learn the stochastic parametrisation as a generative model. We demonstrate the parameterisation as a score-based generative model and discuss the numerical implementation for the rotating shallow-water equations and the simulation results.

Complex streams of evolving data are better understood by their effects on nonlinear systems that by their values at times. The question of which nonlinear systems would seem to be context dependent, but it is not. Core to rough path theory is a simple universal nonlinear system that captures all the information needed to predict any response to any nonlinear system. This idealised mathematical feature set is known as the signature of the stream. Its abstract simplicity opens the possibilities for understanding and working with streams in the same context free way that calculators work with numbers. Signature-based techniques offer simple to apply universal numerical methods that are robust to irregular data and efficient at representing the order of events and complex oscillatory data. Specific software can be developed and then applied across many contexts. Signatures underpin prize winning contributions in recognizing Chinese handwriting, in detecting sepsis, and in generating financial data, and most recently in the ability to score streams as outliers against a corpus of normal streams. This principled outlier technology has emerged as a powerful unifying technique; it identifies radio frequency interference in astronomical data and brain injury from MEG data. The underpinning theoretical contributions span a range from abstract algebra and non-commutative analysis to questions of organisation of efficient numerical calculation. See www.datasig.ac.uk/. New hyperbolic partial differential equations have been developed that compute the “signature kernel” trick without ever having to introduce signatures. Neural controlled differential equations can directly harness approaches such as the log ode method and consume the control as a rough path. The current step is the rough transformer. For this RoughPy needs to be on the GPU.

Analyzing climate scenarios is crucial for quantifying uncertainties, identifying trends, and validating models. Objective statistical methods provide decision support for policymakers, optimize resource allocation, and enhance our understanding of complex climate dynamics. These tools offer a systematic and quantitative framework for effective decision-making and policy formulation amid climate change, including accurate projections of extreme events—a fundamental requirement for Earth system modeling and actionable future predictions.

Identifying governing equations for a dynamical system is a topic of critical interest across an array of disciplines, from mathematics to engineering to biology. Machine learning – specifically deep learning – techniques have shown their capabilities in approximating dynamics from data, but a shortcoming of traditional deep learning is that there is little insight into the underlying mapping beyond its numerical output for a given input. This limits their utility in analysis beyond simple prediction. Simultaneously, a number of strategies exist which identify models based on a fixed dictionary of basis functions, but most either require some intuition or insight about the system, or are susceptible to overfitting or a lack of parsimony. Here, we present a novel approach that combines the flexibility and accuracy of deep learning approaches with the utility of symbolic solutions: a deep neural network that generates a symbolic expression for the governing equations. We first describe the architecture for our model and then show the accuracy of our algorithm across a range of classical dynamical systems. This talk is based on the paper “Symbolic Regression Via Neural Networks”, Chaos 33:083150, 2023.

I will present some scaling limit results for random recurrent and residual neural networks when width and depth tend to infinity. When the activation function is the identity, I will show that the limiting object is a Gaussian measure on some space of paths and its covariance agrees with the so-called signature kernel.