机器学习：数学理论与科学应用

Weinan E Joint work with: Jiequn Han, Arnulf Jentzen, Chao Ma, Zheng Ma,Han Wang, Qingcan Wang, Lei Wu, Linfeng Zhang, Yajun Zhou Roberto Car, Wissam A. Saidi PDEs and fundamental laws of physics Period 1: Solving differential equations numerically Period 2: Multiscale, multi-physics modeling Period 3: Integrating machine learning with physical modelingMolecular modeling Kinetic model for gas dynamicsEconomics and FinanceLinguistics Mathematical theory of machine learningExample 1: Random feature model Example 2: Two-layer neural networksExample 3. Deep residual networks PDEs and fundamental laws of physics Period 1: Solving differential equations numerically Period 2: Multiscale, multi-physics modeling Period 3: Integrating machine learning with physical modelingMolecular modeling Kinetic model for gas dynamicsEconomics and FinanceLinguistics Mathematical theory of machine learningExample 1: Random feature model Example 2: Two-layer neural networksExample 3. Deep residual networks Navier-Stokes equationsBoltzmann equationSchr¨odinger equation...... ”The underlying physical laws necessary for the mathematical theory of a large part ofphysics and the whole of chemistry are thus completely known, and the difficulty is only thatthe exact application of these laws leads to equations much too complicated to be soluble. ” For most practical applications, the difficulty is not in the fundamental laws of physics, but inthe mathematics. PDEs and fundamental laws of physics Period 1: Solving differential equations numerically Period 2: Multiscale, multi-physics modeling Period 3: Integrating machine learning with physical modelingMolecular modeling Kinetic model for gas dynamicsEconomics and FinanceLinguistics Mathematical theory of machine learningExample 1: Random feature model Example 2: Two-layer neural networksExample 3. Deep residual networks finite differencefinite elementspectral methods...... These have completely changed the way we do science, and to an even greater extend,engineering. gas dynamicsstructural analysisradar, sonar, opticscontrol of flight vehicles, satellites...... If the finite difference method was invented today, the shock wave that it will generate wouldbe just as strong as the one generated by deep learning. many-body problems (classical and quantum, in molecular science)quantum controlfirst principle-based drug and materials designprotein foldingturbulence, weather forecastingtransitional flows in gas dynamicspolymeric fluidsplasticitycontrol problems in high dimensions...... Common feature of these problems: Dependence on many variables. Curse of dimensionality: As the dimension grows, the complexity (or computational cost)grows exponentially. Outline PDEs and fundamental laws of physics Period 1: Solving differential equations numerically Period 2: Multiscale, multi-physics modeling Period 3: Integrating machine learning with physical modelingMolecular modeling Kinetic model for gas dynamicsEconomics and FinanceLinguistics Mathematical theory of machine learningExample 1: Random feature model Example 2: Two-layer neural networksExample 3. Deep residual networks works well when the macro- and micro-scales are very well separatednot very effective when there are no separation of scales (e.g. turbulence problem) Solved: low dimensional problems (few dependent variables) Unsolved: high dimensional problems (many dependent variables) Machine learning, particularly deep learning, seems to be a powerful tool for high dimensionalproblems. Outline PDEs and fundamental laws of physics Period 1: Solving differential equations numerically Period 2: Multiscale, multi-physics modeling Period 3: Integrating machine learning with physical modelingMolecular modeling Kinetic model for gas dynamicsEconomics and FinanceLinguistics Mathematical theory of machine learningExample 1: Random feature model Example 2: Two-layer neural networksExample 3. Deep residual networks Example 1: Molecular dynamics Traditional dilemma: accuracyvscost. Two ways to calculateEandF: Computing the inter-atomic forces on the fly using QM, e.g. the Car-Parrinello MD.Accurate but expensive:∑ Empirical potentials: efficient but unreliable. The Lennard-Jones potential: New paradigm: quantum mechanics model – data generatormachine learning – parametrize (represent) the modelmolecular dynamics – simulator Issues (different from usual AI applications): preserving physical symmetries (translation, rotation, permutation)getting the “optimal data set” Deep potential The whole sub-network consists of an encoding netDi(Ri)and a fitting netEi(Di). (Rotation:˜Ri( ˜Ri)T, permutation:(Gi1)T˜Riand( ˜Ri)TGi2.) DeepPot-SE (arxiv: 1805.09003, NIPS 2018), see also Behler and Parrinello, PRL 2007. Indicator:= maxi√〈‖fi−¯fi‖2〉,¯fi=〈fi〉”Active Learning of Uniformly Accurate Inter-atomic Potentialsfor Materials Simulation.” arXiv:1810.11890 (2018). ∼0.005% configurations explored by