Optimal Control and Decision Making Under Uncertainty for Digital Twins

We present a data-driven modeling framework for scientific discovery, termed Flow Map Learning (FML). This framework enables the construction of accurate predictive models for complex systems that are not amenable to traditional modeling approaches. By leveraging data and the expressiveness of deep neural networks (DNNs), FML facilitates long-term system modeling and prediction even when governing equations are unavailable. FML is particularly powerful in the context of Digital Twins, an emerging concept in digital transformation. With sufficient offline learning, FML enables the construction of simulation models for key quantities of interest (QoIs) in complex Digital Twins, when direct mathematical modeling of the QoI is infeasible. During the online execution of a Digital Twin, the learned FML model can simulate and control the QoI without reverting to the computationally intensive Digital Twin itself. As a result, FML serves as an enabling methodology for real-time control and optimization of the physical twin, significantly enhancing the efficiency and practicality of Digital Twin applications.

Neural network approaches that parameterize the value function have shown success in high-dimensional optimal control problems when the maximization of the Hamiltonian yields a closed-form solution for the optimal control. However, many problems of practical importance including space shuttle reentry, aircraft landing, and constrained vehicle control involve Hamiltonians that do not admit such an explicit formulas (implicit Hamiltonians). We propose an end-to-end implicit deep learning approach based on Jacobian-Free Backpropagation that allows for efficient training of value functions for high-dimensional optimal problems when the Hamiltonian is implicitly defined.

In this talk,  we present Hybrid-Cooperative (HYCO) learning, a novel strategy for constructing mathematical models of physical systems that unites physics-based modeling with machine learning. Traditional approaches are often polarized: data-driven neural networks can capture patterns but struggle with physical consistency, while high-fidelity PDE-based models ensure accuracy but are computationally demanding. HYCO bridges these paradigms by embedding them in a cooperative, game-theoretic framework that exploits their complementary strengths. By merging empirical patterns extracted from data with the structural knowledge encoded in physical laws, HYCO delivers robust and interpretable models of complex phenomena. Numerical experiments show that HYCO outperforms both purely data-driven and purely physics-based methods, particularly in regimes with sparse, noisy, or localized data, highlighting its potential as a next-generation tool for scientific computing.

"Problem uncertainty typically limits how well decisions can be tailored to the problem at hand but often can not be avoided. The availability of large quantities of data in modern applications however presents an exciting opportunity to nevertheless make better informed decisions. Capitalizing on this opportunity requires developing novel tools on the intersection between operations research, stochastics as well as data science. In a modern setting the primitive describing uncertainty is often messy data rather than classical distributions. Simply quantifying the probability of an undesirable outcome becomes a challenging uncertainty quantification problem which I approach with a distributional optimization lens. Distributional robust optimization (DRO) has recently gained prominence as a paradigm for making data-driven decisions which are protected against adverse overfitting effects. We justify the effectiveness of this paradigm by pointing out that certain DRO formulations indeed enjoy optimal statistical properties. Furthermore, DRO formulations can also be tailored to efficiently protect decisions against overfitting even when working with messy corrupted data. Finally, as such formulations are often computationally tractable they provide a practical road to the development of tomorrow's trustworthy decision systems."

Bayesian optimal experimental design (OED) provides a principled framework for choosing experimental settings that maximize the informativeness of collected data. Traditional Bayesian OED has relied heavily on the expected information gain (EIG), based on Kullback–Leibler divergence, or variance-reduction criteria such as A-optimality. In this talk, we introduce a new design criterion grounded in optimal transport: the expected Wasserstein distance between prior and posterior distributions. This criterion inherits the fully Bayesian character of EIG while offering geometric and stability advantages. For Gaussian regression models, the Wasserstein-2 criterion yields explicit formulas akin to classical optimality measures, making it computationally tractable. A central contribution of our work is a stability analysis of the Wasserstein-1 criterion; this demonstrates robustness under perturbations of the prior and likelihood and enables rigorous error control when empirical priors are used.

Optimal feedback control and state-estimation of nonlinear systems
depend on the solution of high-dimensional Hamilton Jacobi Bellman equations. Soving such problems in practice represents represents
a significant challenge.
In the first part we survey three approaches, policy iteration, a data driven technique, and the averaged feedback learning scheme (AFLS), commenting on practical perfomance and analytical results which can be achieved. While these results are mainly justifiable for the case of sufficiently regular value functions, in the second part we concentrate
on value functions which are only semiconcave. For this case
again an algorithmic framework, based on the representation of semiconcave functions as minimum of $C^2$ regular functions, is provided.
This is joint work with B. Azmi,  S. Dolgov, D. Kalise, D. Vasquez, and D. Walter.

Optimization problems constrained by partial differential equations (PDEs) are ubiquitous in science and engineering, arising as optimal control, design and inverse problems. These problems are notoriously challenging to solve numerically. For example, simply evaluating the objective function requires solving a large-scale system of equations resulting from the discretized PDE. This exorbitant cost necessitates the use of rapidly converging optimization routines to reduce the number of evaluations of the objective function and its derivatives. Unfortunately, this expense is exacerbated when the objective function involves nonsmooth terms such sparsifying regularizers or risk measures for stochastic problems. Traditional nonsmooth optimization methods converge (sub)linearly, often requiring many iterations to achieve marginal accuracy. In this talk, we introduce a proximal trust-region Newton method for minimizing a specific class of structured nonsmooth objective functions in Hilbert space. Our method is unique in that it permits and systematically controls inexact objective function and derivative evaluations. We prove global convergence of our method and establish that it converges superlinearly, even quadratically, for specific objective function types. We demonstrate the efficiency of our algorithm on various risk-averse PDE-constrained optimization examples as well as the simulation of a viscoelastic flow.

Digital Twins (DTs) are adaptive, real-time virtual replicas of physical systems that integrate physics-based models, sensor data, and intelligent decision-making. At their core, DTs can be rigorously framed within PDE–constrained optimization (PDECO). This talk develops a unified PDECO framework for state estimation and control, leveraging adjoint-based methods in both deterministic and stochastic settings. To address the challenges of infinite-dimensional, large-scale optimization, novel function space based algorithms are discussed. Applications span a wide range of domains, including structural and biomedical systems—from bridges and dams to aneurysm modeling, optimal insulation, electromagnetic cloaking, light bending, and neuromorphic imaging. Together, these examples highlight a pathway toward predictive, adaptive, and trustworthy Digital Twins.

 In this talk, we consider optimal experimental design (OED) for
infinite-dimensional Bayesian inverse problems, with a focus on optimal sensor
placement. Traditional OED approaches seek sensor placements that minimize
uncertainty in the estimated model parameters. However, in some cases, parameter
estimation is an intermediate step towards a downstream goal of solving an
optimal control problem.  In this setting, a sensor placement that is optimal for parameter estimation may be suboptimal for the demands of the optimal control problem.  To address this, we propose a control-oriented OED (cOED) framework.  Namely, we seek sensor placements that minimize uncertainty in the state being controlled or in the control objective.  In the talk, I will discuss our proposed mathematical and computational framework for cOED for infinite-dimensional Bayesian linear inverse problems governed by PDEs. The focus is on optimal control problems with linear dependence on the control variable and the inversion parameter.  We will also consider some numerical results in the context of a model problem motivated by heat transfer problems. 

We present an efficient and scalable second-order computational framework for solving large-scale optimization problems with superquantile constraints. Unlike empirical risk models, superquantile models have non-separable constraints that make typical first-order algorithms difficult to scale. We address the challenge by adopting a hybrid of the second-order semismooth Newton method and the augmented Lagrangian method, which takes advantage of the structured sparsity brought by the risk sensitive measures. The key to make the proposed computational framework scalable in terms of the number of training data is that the matrix-vector multiplication in solving the resulting Newton system can be computed in a reduced space due to the aforementioned sparsity. The computational cost per iteration for the Newton method is similar or even smaller than that of the first-order method. Our developed solver is expected to help improve the reliability and accuracy of statistical estimation and prediction, as well as control the risk of adverse events for safety-critical applications.

Reduced-order models (ROMs) provide a viable avenue to enable (near) real-time decision-making and control with a digital twin, due to their rigorous mathematical derivation and fast simulation. However, not all ROM techniques are “fit-for-purpose” to serve as a digital twin, as decision-making and control require special properties of ROMs. In this talk, we focus on the system-theoretic aspects of controllability and observability, and describe ways to derive nonlinear ROMs that preserve those properties. In particular, we focus on nonlinear balanced truncation ROMs as initializations of the digital twin before the physical-to-digital data stream starts. To obtain the relevant balancing transforms, special forms of Hamilton-Jacobi-Bellman equations have to be solved, which we do so with Taylor-series-based techniques to produce scalable algorithms with 1,000s of state variables.

Stochastic optimization of engineering systems is often infeasible due to repeated evaluations of a computationally expensive, high-fidelity simulation. Multi-fidelity methods mitigate this challenge by leveraging cheaper, approximate models to accelerate convergence. Most existing multi-fidelity approaches, however, exploit either design-space curvature or random-space correlation, not both. We present STRUM, an unconstrained, stochastic, trust-region algorithm that unifies these strategies by combining multi-fidelity trust-region optimization with control variate gradient estimation. This approach exploits the curvature information of low-fidelity objective functions to find a local minima of the high-fidelity model while extracting correlations to reduce noise in gradient estimation.  We provide almost-sure convergence guarantees under certain regularity assumptions and ensure the best-case 1/n convergence rate for stochastic optimization. On benchmark problems and a 20-dimensional space shuttle reentry case, STRUM converges faster than existing methods and cuts computational expense by up to 100x.

We propose a general-purpose approach for improving the ability of large language models (LLMs) to intelligently and adaptively gather information from a user or other external source using the framework of sequential Bayesian experimental design (BED). This enables LLMs to act as effective multi-turn conversational agents and interactively interface with external environments. Our approach, which we call BED-LLM (Bayesian experimental design with large language models), is based on iteratively choosing questions or queries that maximize the expected information gain (EIG) about the task of interest given the responses gathered previously. We show how this EIG can be formulated (and then estimated) in a principled way using a probabilistic model derived from the LLM's predictive distributions and provide detailed insights into key decisions in its construction and updating procedure. We find that BED-LLM achieves substantial gains in performance across a wide range of tests based on the 20 Questions game and using the LLM to actively infer user preferences, compared to direct prompting of the LLM and other adaptive design strategies.

n this talk we show how graph theory provides a unifying framework to model, analyze, and solve complex optimization problems, as those arising in stochastic optimization (graph is a tree), optimal control and state estimation (graph is a line), PDE-constrained optimization (graph is a mesh), network optimization (graph is the network), and combinations (e.g., stochastic PDE-constrained problems). We show how the graph abstraction facilitates the derivation of theoretical properties of optimization problems. Specifically, we show that, under standard regularity conditions, nonconvex continuous optimization problems posses a property that we call exponential decay of sensitivity (EDS); this property indicates that the effect of parametric perturbations decays exponentially along the graph. We show that how EDS relates to controllability and observability properties of optimal control problems. In addition, we show that EDS can help derive graph coarsening and decomposition strategies to obtain approximate solutions of high quality. We also show how the graph abstraction facilitates the derivation of new algorithmic strategies to solve large-scale problems in distributed settings. Specifically, we introduce a new decomposition paradigm that we call overlapping Schwarz decomposition, which facilitates the implementation of distributions optimization settings with various degrees of communication. Finally, we discuss how graph theory can be used to model digital twins and to understand how data and decisions propagate through the structure of complex systems. This is work in collaboration with Sungho Shin (MIT), Sen Na (Georgia Tech), and Mihai Anitescu (Argonne)

In nonlinear Model Predictive Control (MPC), an optimal control problem is solved at each time instant to make an educated decision on the next control input. We propose a framework to ensure desired properties of the resulting MPC closed loop if data data-driven surrogate models are leveraged in the optimization step. Moreover, we show that the imposed assumptions can be rigorously verified using Koopman theory. To be more precise, we employ extended dynamic mode decomposition (EDMD) in the prediction step, provide an in-depth analysis of the approximation error, and propose novel proportional error bounds.

This session dives deep into ways to communicate actual mathematical and statistical content. Participants will take part in activities around the proper use of jargon, examine the important role definitions play in the mathematical sciences, and develop metaphors and analogies for their research. They will also delve into the important role the human aspect can play in connecting mathematics and statistics with lay audiences and why it is important to be careful with numbers. 

Modern research in control is undergoing a paradigm shift from finite-dimensional first-principles models to data-driven non-parametric representations, driven by the need for scalability and adaptability in complex, real-world systems. This transition poses both opportunities and challenges for optimal control theory, algorithm, and software development, particularly in balancing model fidelity with computational tractability. In this talk, I will present a critical assessment of this shift, arguing that operator-theoretic frameworks---rooted in reproducing kernel Hilbert spaces (RKHS), partial differential equation systems, and infinite-dimensional linear systems theory---offer a promising path forward for addressing data-driven optimal control problems. By leveraging linear algebraic structures and convex optimization, these models enable the treatment of nonlinear dynamics through the lens of stochastic diffusion processes, learned directly from data.

In detail, I will discuss recent advances in non-parametric learning of controlled Markov transitions via RKHS embeddings, capturing probabilistic uncertainties in system behavior without explicit parametric assumptions. Moreover, I will discuss recent advances in the field of operator-theoretic dynamic programming for infinite-dimensional systems, which enables the efficient solution of optimal control problems through low-rank regression based operator approximations while preserving theoretical guarantees. The talk will conclude with open challenges in probabilistic forecasting as well as robust and data-driven control.


REFERENCES:
----------------
[1] B. Houska. Convex operator-theoretic methods in stochastic control. Automatica, 2025.
[2] P. Bevanda, et.al. Kernel-based optimal control: an infinitesimal generator approach. In Proceedings of the 7th Annual Learning for Dynamics and Control Conference (L4DC), 2025.

Real-time decision-making is a crucial aspect of digital twins, facilitating the effective operation of physical assets. Specifically, optimal control solutions are employed to guide these assets in achieving their operational objectives. To enable timely decision-making that aligns with the dynamics of physical assets, reduced-order models are essential. However, despite their real-time capabilities, it is imperative to address modeling errors. To tackle this challenge, we introduce Hyper-Differential Sensitivity Analysis (HDSA), a computationally efficient technique designed to assess how optimization solutions are affected by modeling inaccuracies. By integrating HDSA with data from a limited number of high-fidelity forward evaluations, we can effectively update the reduced-order optimal control problem. We demonstrate the efficacy of our algorithmic strategy through several non-trivial examples.

We study solution sensitivity for nonlinear programs (NLPs) whose structures are induced by graphs, as is the case for control and spatially indexed problems. We show that for a given pair of nodes, the sensitivity of the primal-dual solution at one node against a data perturbation at the other node decays exponentially with respect to the distance between these two nodes on the graph. This result, which we refer to as the exponential decay of sensitivity, holds under the strong second-order sufficiency condition and the linear independence constraint qualification. We also present conditions under which the decay rate remains uniformly bounded; this allows us to characterize the sensitivity behavior of NLPs defined over subgraphs of infinite graphs. Moreover, we demonstrate that slightly stronger second-order sufficiency and linear independence constraint qualification holding over subgraphs yield this property over the entire graph. This property may help ensure that control problems with multicomponent systems, such as digital twins of complex systems, possess exponential decay of sensitivity by providing sufficient quality of the component problems alone. This makes the property not only easier to verify, but it also implies that decomposition-based control policies, which are computationally faster, will be exponentially close to optimal. If possible, I would pre