Data Sciences for Mesoscale and Macroscale Materials Models

Physics-informed neural networks and operator networks have shown promise for effectively solving equations modeling physical systems. However, these networks can be difficult or impossible to train accurately for some systems of equations. One way to improve training is through the use of a small amount of data, however, such data is expensive to produce. We will introduce our novel multi-fidelity framework for stacking physics-informed neural networks and operator networks that facilitates training by progressively reducing the errors in our predictions for when no data is available. In stacking networks, we successively build a chain of networks, where the output at one step can act as a low-fidelity input for training the next step, gradually increasing the expressivity of the learned model.

Materials simulations based on direct numerical solvers are accurate but computationally expensive for predicting materials evolution across length- and timescales, due to the complexity of the underlying evolution equations, the nature of multiscale spatiotemporal interactions, and the need to reach long-time integration. We develop a method that blends direct numerical solvers with neural operators to accelerate such simulations. This methodology is based on the integration of a community numerical solver with a U-Net neural operator, enhanced by a temporal-conditioning mechanism to enable accurate extrapolation and efficient time-to-solution predictions of the dynamics. We demonstrate the effectiveness of this hybrid framework on simulations of microstructure evolution via the phase-field method. Such simulations exhibit high spatial gradients and the co-evolution of different material phases with simultaneous slow and fast materials dynamics. We establish accurate extrapolation of the coupled solver with large speed-up compared to DNS depending on the hybrid strategy utilized. This methodology is generalizable to a broad range of materials simulations, from solid mechanics to fluid dynamics, geophysics, climate, and more.

Inverse Reinforcement Learning (IRL) is a compelling technique for revealing the rationale underlying the behavior of autonomous agents. IRL seeks to estimate the unknown reward function of a Markov decision process (MDP) from observed agent trajectories. However, IRL needs a transition function, and most algorithms assume it is known or can be estimated in advance from data. It therefore becomes even more challenging when such transition dynamics is not known a-priori, since it enters the estimation of the policy in addition to determining the system’s evolution. When the dynamics of these agents in the state-action space is described by stochastic differential equations (SDE) in It^{o} calculus, these transitions can be inferred from the mean-field theory described by the Fokker-Planck (FP) equation. We conjecture there exists an isomorphism between the time-discrete FP and MDP that extends beyond the minimization of free energy (in FP) and maximization of the reward (in MDP). We identify specific manifestations of this isomorphism and use them to create a novel physics-aware IRL algorithm, FP-IRL, which can simultaneously infer the transition and reward functions using only observed trajectories. We employ variational system identification to infer the potential function in FP, which consequently allows the evaluation of reward, transition, and policy by leveraging the conjecture. We demonstrate the effectiveness of FP-IRL by applying it to a synthetic benchmark and a biological problem of cancer cell dynamics, where the transition function is inaccessible.

This is joint work with Changyang Huang, Siddhartha Srivastava, Kenneth Ho, Kathryn Luker, Gary Luker and Xun Huan.

Dislocation mobility, which dictates the response of dislocations to an applied stress, is a fundamental property of crystalline materials that governs the evolution of plastic deformation. Traditional approaches for deriving mobility laws rely on phenomenological models of the underlying physics, whose free parameters are in turn fitted to a small number of intuition-driven atomic scale simulations under varying conditions of temperature and stress. This tedious and time-consuming approach becomes particularly cumbersome for materials with complex dependencies on stress, temperature, and local environment, such as body-centered cubic crystals (BCC) metals and alloys. In this talk, I will present a novel, uncertainty quantification-driven exascale active learning paradigm for learning on-the-fly materials evolution laws from automated high-throughput large-scale molecular dynamics simulations, using Graph Neural Networks (GNN) with a physics-informed architecture. I will demonstrate that such automated on-the-fly multiscale modeling framework captures the underlying physics more accurately compared to existing phenomenological mobility laws in BCC metals.

Computational models have greatly improved how we understand complex biological systems. Yet, the variety of these systems prohibits a one-size-fits-all solution. Hence, to effectively tackle the specific challenges posed by varying contexts within computational medicine, we must tailor our computational strategies whether they be data-driven, knowledge-driven, or a hybrid approach integrating the two.

In this talk, I will dissect the unique strengths and situational superiority of each modeling paradigm in computational medicine. First, I will show how to provide accurate predictions and distill novel biological knowledge using data-driven models. Next, I will demonstrate how to validate observed disease-mediated changes in blood rheology via knowledge-driven models. Finally, I will also discuss patient-specific decision-making enabled by a hybrid model. 

We develop a machine learning approach to identifying parameters with steady-state solutions, locating such solutions, and determining their linear stability for systems of ordinary differential equations and dynamical systems with parameters. Our approach begins with the construction of target functions that can be used to identify parameters with steady-state solution and the linear stability of such solutions. We design a parameter-solution neural network (PSNN) that couples a parameter neural network and a solution neural network to approximate the target function, and develop efficient algorithms to train the PSNN and to locate steady-state solutions. We also present a theory of approximation of the target function by our PSNN based on the neural network kernel decomposition. Numerical results are reported to show that our approach is robust in identifying the phase boundaries separating different regions in the parameter space corresponding to no solution or different numbers of solutions and in classifying the stability of solutions. These numerical results also validate our analysis. Although the primary focus in this study centers on steady states of parameterized dynamical systems, our approach is applicable generally to finding solutions for parameterized nonlinear systems of algebraic equations. Some potential improvements and future work are discussed.

Since the publication of the Mission Innovation Materials Acceleration Platform, AI is increasingly responsible for driving automated experimental and computational tools. There have been multiple case studies for which autonomy was demonstrated to successfully drive materials optimization or discovery problem and the world of scientific robots has moved from science fiction to reality. However, within the broader AI community it is well known that AI’s carry with them their creators’ biases and this has serious implications on model development and deployment. Using several case studies, I will illustrate how biases can arise in materials science and specific steps that can be taken to remove them. Specifically, I will discuss some of our recent work in (1) reducing human bias in label generation by applying robust statistics to spectroscopic data analysis, (2) identifying and mitigating search space bias through model disagreement, and (3) circumventing the big data bias loop by illustrating how to the presence of information redundancy in large computational datasets and (4) how construct an optimally informative dataset for model training.

The trajectory of scientific research worldwide is guided by long-term goals, spanning the spectrum from curiosity and fundamental discoveries in physics to the applied challenges of enhancing materials and devices for a wide array of applications. However, the execution and assessment of daily research efforts typically hinges on multiobjective reward functions, which can be evaluated either during or at the conclusion of an experimental campaign. Although this concept is tacitly acknowledged within the scientific community, the implementation of autonomous experimental workflows in automated laboratories necessitates the formulation of robust reward functions and their seamless integration across various domains. Should these reward functions be universally established, the entirety of experimental efforts could be conceptualized as optimization problems. Here, we present our latest advancements in the development of autonomous research systems based on electron and scanning probe microscopy, as well as for automated materials synthesis. We identify several categories of reward functions that are discernible during the experimental process, encompassing fundamental physical discoveries, the elucidation of correlative structure-property relationships, and the optimization of microstructures. The operationalization of these rewards function on autonomous microscopes is demonstrated, as well as strategies for human in the loop intervention. Utilizing these classifications, we construct a framework that facilitates the integration of multiple optimization workflows, demonstrated through the synchronous orchestration of diverse characterization tools across a shared chemical space, and the concurrent navigation of costly experiments and models that adjust for epistemic uncertainties between them. Our findings lay the groundwork for the integration of multiple discovery cycles, ranging from rapid, laboratory-level exploration within relatively low-dimensional spaces and strong basic physics priors to more gradual, manufacturing-level optimization in highly complex parameter spaces underpinned by poorly-known and phenomenological physical models.

1. Y. Liu, M. Ziatdinov, R. Vasudevan, and S.V. Kalinin, Explainability and Human Interventions in Autonomous Scanning Probe Microscopy, Patterns 4, (2023), arXiv:2302.06577
2. S.V. Kalinin, Y. Liu, A. Biswas, G. Duscher, U. Pratiush, K. Roccapriore, M. Ziatdinov, R. Vasudevan, Human-in-the-loop: The future of Machine Learning in Automated Electron Microscopy, arXiv:2310.05018
3. Y. Liu, A. Morozovska, E. Eliseev, K.P. Kelley, R. Vasudevan, M. Ziatdinov, and S.V. Kalinin, Autonomous scanning probe microscopy with hypothesis learning: Exploring the physics of domain switching in ferroelectric materials, Patterns 4 (2023) arXiv:2202.01089
4. Y. Liu, K.P. Kelley, R.K. Vasudevan, H. Funakubo, M.A. Ziatdinov, and S.V. Kalinin, Experimental discovery of structure-property relationships in ferroelectric materials via active learning, Nat. Mach. Int. 4, 341 (2022), arXiv:2108.06037

Coarse-graining or model reduction is a term describing a range of approaches used to extend the timescale of molecular simulations by reducing the number of degrees of freedom. In the context of molecular simulation, standard coarse-graining approaches approximate the potential of mean force and use this to drive an effective Markovian model. To gain insight into this process, we consider the simple case of a quadratic energy is studied in an overdamped setting. A hierarchy of reduced models is derived and analyzed, and the merits of these different coarse-graining approaches are discussed. In particular, while standard recipes for model reduction accurately capture static equilibrium statistics, it is shown that dynamical statistics such as the mean-squared displacement display systematic errors, even when a system exhibits large timescale separation. In the linear setting studied, it is demonstrated both analytically and numerically that such models can be augmented in a simple way to better capture dynamical statistics.

Kernels are efficient in representing nonlocal dependence and are widely used to design operators between function spaces or high-dimensional data. Thus, learning kernels in operators from data is an inverse problem of general interest. Due to the nonlocal dependence, the inverse problem is often severely ill-posed with a data-dependent operator that is nearly singular. Therefore, regularization is necessary. However, little information is available to select a proper regularization norm in many applications. We overcome the challenge by introducing a new regularization norm based on an RKHS that is determined by the nonlocal operator and data. It leads to convergent estimators that are robust to noise, outperforming the widely used L2 or l2 regularizers. We will discuss both direct and iterative methods for implementation.