AIGC与大模型赋能机器人智能控制

穆尧-香港大学-在读博士 DataFunSummit#2023 目录CONTENT 01IntroductionofAIGC此部分内容作为文字排版占位显示（建议使用主题字体） IntroductionofEmbodiedAI 03 此部分内容作为文字排版占位显示（建议使用主题字体） Adaptdiffuser此部分内容作为文字排版占位显示（建议使用主题字体） EmbodiedGPT 02 04 此部分内容作为文字排版占位显示（建议使用主题字体） 01AIGC简介 DataFunSummit#2023 DiffusionModelsarePowerfulGenerativeModel1.1 FromVAEtoDiffusionModel1.2 MultilayersVAE FromVAEtoDiffusionModel1.2 Normalizingflow:reversible functionwith limited expressive power GAN:Needs to learn a discriminator, trainingprocess unstable RLApplication:Flow-based Recurrent Belief StateLearning for POMDPs(ICML2022) IntroductionofDiffusionModel1.3 HKUMMLAB,TheUniversityofHongKong,HongKong ForwardDiffusionProcess1.3 ReverseDenoisingProcess`1.3 02Adaptdiffuser DataFunSummit#2023 ConvertRLproblemstoTrajectoryGenerationwithGenerativeModel2.1 DiffusionModelforTrajectoryGeneration2.2 DiffusionModelforTrajectoryGeneration2.3 TransferUnconditionalTrajectoryModelto aConditionalPolicy2.4 Guidance functions transforms an unconditional trajectorymodel into a conditional policy for diverse tasks. Goal-conditionedGuidence2.4 ChallengesofDiffusionPlanning2.5 LimitedbythedistributionofTrainingofflinedatasetandischallengingtoadaptnewtasks/goalsandnewenvironments Solutions: FrameworkofAdaptDiffuser3.1 DiversetaskgenerationwithChatgpt3.1 HKUMMLAB,TheUniversityofHongKong,HongKong ExperimentalResults3.1 03具身智能简介 DataFunSummit#2023 ChallengesofEmbodiedAI 1.Embodied Cognitive Systems from a First-Person Perspective2.Achieving Highly Autonomous Decision-Planning Capabilities3.Achieving Goal-conditionedPhysical Interaction with the World IntroductionofEmbodiedAI Embodied AI refers to the integration of artificial intelligence systems with physical bodies or roboticplatforms, allowing them to perceive and interact with the physical world in a manner similar tohumans. EmbodiedGPT •1）Building a Human Manipulation Video-Text DatasetEgoCOTwith a Multimodal CognitiveChain, Associating Visual Information with Sub-Goals in Manipulation Tasks. Openthedrawer task : open a drawerplans :graspthehandlewiththegripperandpull thehandleactions :1.grasp(handle,gripper)1. pull(handle) Pickupthecup task : pick up a cup on the tableplans : grasp the handle of the cupwiththegripperand lift it upactions :1. grasp(handle of the cup,gripper)2.lift up(cup) EmbodiedGPT 2)Introducing a Visual-Language Pretraining Method based on a Multimodal Cognitive Chain. EmbodiedGPT 3)Extracting Highly Relevant Features of Current Visual Observations and Planning's Specific Sub-Goals using Self-Attention Mechanism, Enabling the Model to Learn Low-Level Control withMinimal Demonstration Data. 感谢观看