Advancements in Artificial Intelligence open the door to exciting possibilities, but how can these be captured to benefit the design of sustainable products? In order to machine-learn from a wealth of prior engineering achievement, design must be represented for computation. In this talk, methods for quantitatively representing various aspects of engineering design from function to form to fabrication will be presented. Applications of these methods to augment sustainable decision-making in circular manufacturing, policy review, and knowledge preservation will be illustrated, and the talk will conclude with a discussion on safety and bias in deploying such data-driven methods in society.

The application of DNNs to the denoising problem has resulted in highly performant denoisers for a wide range of applications from photographic image restoration to medical imaging. However, these images are typically subjected to relatively low degree of noise compared to applications such as cryogenic electron microscopy (cryo-EM), where noise power dominates the clean signal to the point where traditional denoising methods fail. In this talk, we will study the related problem of multireference alignment (MRA). Here, a clean image is randomly rotated and degraded by additive noise. The goal is to recover the original image from a set of such noisy rotated copies of the image. We show how a transformer architecture can be used to encode a signal prior that is used to aggregate the information from the entire set of observations, yielding superior denoising results compared to the single-image denoisers.

Machine learning for flow control has been increasingly used since the last decade. Starting with simple two-dimensional problems like the flow past a cylinder, a future goal has always been to move towards three-dimensional (3D) cases. In this presentation I will show the most recent problems where we have been able to effectively perform active flow control using a deep reinforcement learning (DRL) agent fully trained in a 3D environment: Rayleigh-Bénard convection, flow past a cylinder and a bubble of recirculation in a turbulent boundary layer. In all cases we leverage the well-know curse of dimensionality problem that arises in high dimensional cases by using the multi-agent reinforcement learning approach, which takes advantages of flow invariants to parallelize the DRL environment. Furthermore, we compare the results of the DRL-control strategies with the classical-control methods, obtaining a considerable improvement.

This seminar will address a standard Bayesian state estimation problem as an inverse problem, like Kalman Filter. The major new thing is that Kalman Filter to Particle Filter – allmost all of the standard state estimation methods - know the underline state space model or process model, but our new method DANSE does not. DANSE learns from noisy measurements without access to clean data and/or state space models. That means DANSE learns in an unsupervised manner, and fully model-free. It is an interesting combination of deep learning and Bayesian learning, and Bayesian estimation. Manuscript: https://arxiv.org/abs/2306.03897

In the last few years, we have seen a tremendous amount of scientific progress made as a result of the AI revolution, both in our much expanded ability to make use of the fundamental principles of nature, and our much expanded ability to make use of experimental data and the literature. In this talk, I will start with the origin of the AI for Science revolution, review some of the major progresses made so far, and discuss how it will impact the way we do research. I will also discuss some of the ongoing projects that we are working on, with the objective of constructing a new set of infrastructure for scientific research.

A large class of inverse problems for PDEs are only well-defined as mappings from operators to functions. Existing operator learning frameworks map functions to functions and need to be modified to learn inverse maps from data. We propose an architecture termed Neural Inverse Operators (NIOs) to solve these PDE inverse problems. Motivated by the underlying mathematical structure and PDE-constrained optimization techniques, NIO is based on a composition of DeepONets and Fourier Neural Operators to approximate mappings from operators to functions. Experiments will be presented to demonstrate the performance of the NIOs. They do very well compared to existing neural network baselines in solving PDE inverse problems robustly and accurately. The examples include the classical Calderon problem and optical and seismic imaging. PDE-constrained optimization methods currently can address more challenging problems, but the advantage of NIOs is that they are orders of magnitude faster.

Machine learning has increasingly provided powerful tools to tackle challenging inverse problems. This talk presents two works representing two extreme scenarios and discusses prospects for further development. In the first scenario, where the forward problem is a highly nonlinear scattering operator with small observation noise, machine learning models can warm start point estimation based on an optimization formulation. In the second scenario, where the forward operator is linear but the observation noise might be large, existing score-based diffusion models can provide realistic priors. I will discuss how these models can also assist in provable Bayesian posterior sampling using the tilted transport technique.

Recently, there has been a great interest in operator learning, where neural networks learn operators between function spaces from an essentially infinite-dimensional perspective. We present a generalized framework for neural operators, with layers including nonlinear integral operators and skip connections. We discuss and prove that injective neural operators are universal approximators and develop an algebra with bijective neural operators. Then, we give a more geometrical perspective based on diffeomorphisms in infinite dimensions, that is, for Hilbert manifolds. Using category theory, we give a no-go theorem that shows that diffeomorphisms between Hilbert spaces may not admit any continuous approximations by diffeomorphisms on finite-dimensional spaces, even if the underlying discretization is nonlinear. Strongly monotone diffeomorphisms do admit approximation by finite-dimensional strongly monotone diffeomophisms. We then introduce layerwise strongly monotone neural operators. Such layers are diffeomorphisms. We prove that all strongly monotone neural operator layers admit continuous approximations on finite-dimensional spaces. We provide different conditions under which a neural operator layer is strongly montone. Most notably, a bilipschitz neural operator layer can always be represented by a composition of strongly monotone neural operator layers and invertible linear maps and, hence, be discretized. Our framework may be used "out of the box" to prove quantitative approximation results for discretization of neural operators.

Joint research with T. Furuya, A. Kratsios, A. Lara, M. Lassas and M. Puthawala.

The new wave of artificial intelligence is impacting industry, public life, and the sciences in an unprecedented manner. It has by now already led to paradigm changes in several areas. However, one current major drawback is the lack of reliability. In this lecture we will first provide an introduction into this vibrant research area. We will then present some recent advances, in particular, concerning optimal combinations of traditional model-based methods with AI-based approaches in the sense of true hybrid algorithms, with a particular focus on limited-angle computed tomography and a novel approach coined "Deep Microlocal Reconstruction". Due to the importance of explainability for reliability, we will also touch upon this area by highlighting an approach which is itself reliable due to its mathematical foundation. Finally, we will discuss fundamental limitations of deep neural networks and related approaches in terms of computability, and how these can be circumvented in the future by next generation AI computing.

The success of deep learning approaches for inverse problems strongly depends on the chosen network architecture. In the first part of the talk Meira Iske will present some theoretical results concerning the regularization properties of iResNet architectures. Then, Janke Gödeke will discuss operator approximation properties of neural networks as needed for learning parameter-to-state operators. In the second part of the talk we review some recent results for comparing different network architectures for solving PDEs and related parameter identification problems. We close the talk with some industrial applications.

Random feature learning methods are attractive from the analysis point of view. A challenge in practice is to sample near optimally. I will present an elementary proof of the generalization error for random features and some attempts to sample random features efficiently, based on work with Xin Huang, Aku Kammonen, Jonas Kiessling, Petr Plechac, Mattias Sandberg and Raul Tempone.

The dimensional manifold hypothesis posits that the data found in many applications, such as those involving natural images, lie (approximately) on low dimensional manifolds embedded in a high dimensional Euclidean space. Since a typical neural network is constructed to be a function on the whole embedding space, one must consider the stability of an optimized network function when evaluating at points outside the training distribution. In this talk, we will discuss some consequences of the data manifold's curvatures and the arbitrariness of the high-dimensional ambient space. We will also discuss the regularization effects by introducing noise to the data. Finally, we discuss the multiscale properties of the empirical loss function induced by data distributions supported on a low dimensional submanifold.

The problem of identifying geometric structure in heterogeneous, high-dimensional data is a cornerstone of Representation Learning. In this talk, we study this problem from the perspective of Discrete Geometry. We start by reviewing discrete notions of curvature with a focus on Ricci curvature. Then we discuss how curvature characterizations of graphs can be used to improve the efficiency of Graph Neural Networks. Specifically, we propose curvature-based rewiring and encoding approaches and study their impact on the Graph Neural Network’s downstream performance through theoretical and computational analysis. We further discuss applications of discrete Ricci curvature in Manifold Learning, where discrete-to-continuum consistency results allow for characterizing the geometry of a suitable embedding space both locally and in the sense of global curvature bounds.

Machine learning has increasingly influenced the development of scientific computing. In this talk, I will share some recent experiences on how classical analysis can help machine learning. The first example is online learning, where ODEs and SDEs can help explain the optimal regret bounds concisely. In the second example, a perturbative analysis clarifies why sometimes line spectrum estimation algorithms exhibit a super-convergence phenomenon.