大语言模型时代的科学人工智能

Xuan WangAssistant ProfessorDepartment of Computer ScienceSanghaniCenter for AI and Data AnalyticsVirginia Tech About the Tutor Xuan Wang, AssistantProfessor,Department ofComputer Science,Virginia Tech Research Interests:naturallanguage processing, data mining, AIfor sciences, and AI for healthcare. Website:https://xuanwang91.github.io/ Tutorial Outline •8:30 am–9:00 am:Introduction•9:00am–10:00 am:Part I: Scientific Text•10:00 am–10:30 am:Part II: Brain Signals •10:30 am–11:00 am: Break •11:00 am–12:00 am:Part III: Biological Sequences•12:00 pm–12:30 pm:Summary and Q&A AI for Sciences Zhanget al., “Artificial Intelligence for Science in Quantum,Atomistic, and Continuum Systems”,arXiv, 2023 Large Language Models (LLMs) BERTKentonet al., 2019 GPTBrownet al., 2019 T5Raffelet al., 2020 Machine Translation Dialog Systems, Chatbots, Digital Assistants Natural Language Generation Resemblance Between Scientific Data and Language •Sequences! •Scientific Textual Data: Scientific Literature, Electronic Health Record •Sensor Data: Brain Electroencephalogram (EEG) Signals•Biological Sequences: DNA, RNA, protein This Tutorial: Can we harnessthe powerof theserecent LLMsto drivescientific progress? Scientific Large Language Models:Challenges and Opportunities Xuan WangAssistant ProfessorDepartment of Computer ScienceSanghaniCenter for AI and Data AnalyticsVirginia Tech Tutorial Outline •8:30 am–9:00 am:Introduction•9:00 am–10:00 am:Part I: Scientific Text•10:00 am–10:30 am:Part II: Brain Signals •10:30 am–11:00 am: Break •11:00 am–12:00 am:Part III: Biological Sequences•12:00 pm–12:30 pm:Summary and Q&A Outline •Scientific Large Language Models •Future Directions: •Complex Reasoning and Planning•Multi-modal Learning•Trustworthiness of LLMs LargeLanguageModels(LLMs) Yang, J., Jin, H., Tang, R.,Han, X., Feng, Q., Jiang, H., ...& Hu, X. (2024). Harnessingthe Power of LLMs in Practice:A Survey onChatGPTandBeyond.ACM Transactions onKnowledge Discovery fromData,18(6), 1-32. Jian Ma, “Large LanguageModels in ComputationalBiology–A Primer (2024Update)”, 2024 A Comprehensive Survey of Scientific Large LanguageModels and Their Applications in Scientific Discovery(Zhanget al.,EMNLP 2024) •Survey over260 scientific LLMs •Across fields: 1) general science, 2) mathematics, 3) physics, 4)chemistry and material science, 5) biology and medicine, and 6)geography, geology, and environmental science •Across modalities: 1) text, 2) graph, 3) vision, and 4) time series •Website:https://github.com/yuzhimanhua/Awesome-Scientific-Language-Models A Comprehensive Survey of ScientificLarge Language Models and TheirApplications in Scientific Discovery(Zhang et al., EMNLP 2024) Towards Expert-Level Medical Question Answering withLarge Language Models (Med-Palm-2, Google, 2024) 18Singhal, K., Tu, T.,Gottweis, J.,Sayres, R.,Wulczyn, E., Hou, L., ... & Natarajan, V. (2023). Towards expert-level medical questionanswering with large language models.arXivpreprint arXiv:2305.09617. OpenAI o1 Surpasses Human Performance on PhD-Level Science Questions (OpenAI, 2024) OpenAI o1 Surpasses Human Performance on PhD-Level Science Questions (OpenAI, 2024) Outline •Scientific Large Language Models •Future Directions:•Complex Reasoning and Planning•Multi-modal Learning•Trustworthiness of LLMs Evaluation and Mitigation of the Limitations of LargeLanguage Models in Clinical Decision-Making(Hageret al.,Nature Medicine 2024) Evaluation and Mitigation of the Limitations of LargeLanguage Models in Clinical Decision-Making(Hageret al.,Nature Medicine 2024) LLMs Diagnose Significantly Worse than Clinicians Diagnostic Accuracy of LLMs Decreased in anAutonomous Clinical Decision-Making Scenario LLMs Do Not Consistently Recommend Essential andPatient-Specific Treatment LLMs Are Sensitive to the Quantity of InformationProvided LLMs Are Sensitive to the Order of Information TriageAgent:Towards Better Multi-Agent Collaboration forLarge Language Model-Based Clinical Triage(Luet al.,EMNLP 2024) TriageAgent:Towards Better Multi-Agent Collaboration forLarge Language Model-Based Clinical Triage(Luet al.,EMNLP 2024) Outline •Scientific Large Language Models •Future Directions: •Complex Reasoning and Planning•Multi-Modal Learning•Trustworthiness of LLMs Vision–Language Foundation Model for EchocardiogramInterpretation (Christensenet al.,Nature Medicine 2024) EchoCLIPWorkflow Vision–Language Foundation Model for EchocardiogramInterpretation (Christensenet al.,Nature Medicine 2024) Transparent Medical Image AI via an Image–TextFoundation Model Grounded in Medical Literature(Kimet al.,Nature Medicine 2024) Transparent Medical Image AI via an Image–TextFoundation Model Grounded in Medical Literature(Kimet al.,Nature Medicine 2024) Transparent Medical Image AI via an Image–TextFoundation Model Grounded in Medical Literature(Kimet al.,Nature Medicine 2024) Transparent Medical Image AI via an Image–TextFoundation Model Grounded i