开放科学、元科学和人工智能的机会

ExecutiveSummary OnMarch26,2025,theCenterforSecurityandEmergingTechnology(CSET)andtheOpenResearchCommunityAccelerator(ORCA),withsupportfromtheNationalScienceFoundation,hostedaworkshopinWashington,DC.Webroughttogether35experts,researchers,andfunderstoexploreopportunitiesandchallengesinopenscience,metascience,andAI.Thisreportsummarizestheconversationthattookplaceovertwopanelsessions,twoguidedgroupdiscussions,andaconcludingsynthesissession. Thisreportalsoaimstofillagapidentifiedbyparticipants:theneedforaclearagendatoinformandguideeffortsattheintersectionofmetascienceresearch,opensciencemonitoringandimpactassessment,andAIforresearchandscience.Tothatend,wehighlightpriorityresearchquestionsandfeasibleopportunitiesforcollaborativework. WethanktheNationalScienceFoundation—andspecifically,theCyberinfrastructureforPublicAccessandOpenScienceprogram(CIPAOS)withintheOfficeofAdvancedCyberinfrastructure—forthesupportinplanningandhostingthisproductiveworkshop. TableofContents ExecutiveSummary1 KeyTakeaways3 Panel:MeasuringOpenScienceOutcomesandAssessingImpact5 Discussion:MeasuringOpenScienceOutcomesandAssessingImpact7 Panel:AIforMetascience8 Discussion:AIforMetascience10 APathForward11 AppendixA:ListofAttendees13 AppendixB:ProjectsSharedbyParticipants15 KeyTakeaways Duringtheworkshop,participantsengagedwithpracticalandconceptualquestionsattheintersectionofopenscience,metascience,andAI. Somepracticalquestionsincluded: ●Whatareexistinginitiativesformonitoringandassessingopenscience? ●Howcanwemakereliabledataandmethodsmoreaccessible? ●Howcanweavoidduplicatingefforts? ●Howcanwefacilitatecollaborationandsustainedengagementbetweenopenscience,metascience,andAI? Someconceptualquestionsthegroupdiscussedincluded: ●HowshouldweevaluatetheappropriatenessofAIforagivenresearchtask? ●Howcanweadaptmetascienceinsightstospecificdisciplinesandcontexts? ●Shouldwebethinkingaboutdataandscientificoutputsaspublicgoods? ●Howdowedoresearchintoday’srapidlyevolvingscientificandtechnological landscapeandensureresponsibledeploymentofAIinthatresearch? Theworkshopsurfacedopportunitiesforleveragingexistingdata,tools,andknowledgetoadvanceopenscienceandmetascience.AIpresentsnumerousopportunitiestoimproveopenresearchoutputs(e.g.,papers,software,data,protocols,patents),including: ●Transformingopenresearchoutputs(e.g.,intomachine-readableformats) ●Extractinginformationfromopenresearchoutputs(e.g.,scientificclaims) ●Surfacinghiddenconnectionsacrossopenresearchoutputs ●Improvinginformationretrievalandsearchoveropenoutputs ●Enhancingmetadatacuration(e.g.,viapersistentidentifiers) ●Confirmingandenablingthereusabilityofopenoutputs(e.g.,codereview) ●Expandingsciencecommunicationbymakingitmoreaccessible(e.g.,translation)anddiversifyingoutputformats(e.g.,blogs,repositories) However,participantsemphasizedthatincentives,andtheecosystemsthatrespondtothem,shapehowopenscienceandmetasciencearedone.Participantsalsoemphasizedthathumanreviewandtraditionalresearchmethodsremaincriticaltosoundscience.Inthenearterm,somehigh-levelprioritiessurfacedbyparticipantswere: ●EvaluatingAImodelswithinscientificresearchcontexts ●Fosteringgreaterinformation-sharing ●Developingsharedtoolkits ●Collaboratingonafeasiblepathforward Towardthatend,thereportsummarizesthetwopanelsandgroupdiscussionsfromtheworkshopandconcludeswithalistofpotentialprojectsandprioritiesareastheopenscience,metascience,andAIcommunitiescanacton. Panel:MeasuringOpenScienceOutcomesandAssessingImpact OurfirstpanelincludedDr.TimErrington(CenterforOpenScience),Dr.CassidySugimoto(SchoolofPublicPolicy,GeorgiaTech),andDr.DanielS.Katz(UniversityofIllinoisUrbana-Champaign)andwasmoderatedbyDr.CatherineAiken(CSET). Thepanelistswereaskedtosharecurrentworkrelatedtomeasuringopenscienceandassessingitsimpact,aswellastheirideasforwhereresearchersandtheopensciencecommunitycouldfocuseffortstomakeprogressonopenquestionsandknownneeds. PanelistQuestions:MeasuringOpenScienceOutcomesandAssessingImpact 1.Pleaseshareacurrenteffortyouareinvolvedinthataimstomeasuretheoutcomesand/orassesstheimpactofopensciencepolicies.Whataresomesuccesses?Whatchallengesdoestheeffortface? 2.Wherecanwemakeprogress?Whataresomemeasuresormethodswecouldbuildonortry?Whatdata,tools,orresourcesdoweneed?HowmightAIhelp? 3.Whatactionscanthecommunitytake?Whatshoulddifferentactorsfocusonandprioritize?Howmightweparceloutnextstepstofocusonspecificoutcomesorknowngaps? Existingmeasurementandimpactassessmentprojectsmentionedincludedincorporationofpre-registrationandpeerreviewintomoreresearchstages(e.g.,RegisteredReports),developmentofopenscienceindicators(e.g.,PLOSopenscienceindicators),implementationofFAIRprinciples(Findable,Accessible,Interoperable,Reproducible)formachinelearning(e.g.,FAIR4ML),creationofdomain-specificAIreadinesschecklists(e.g.,FARR),researchingtheimpactofrequiringopenaccessforsomeU.S.federallyfundedresearch,andtrackingtheimpactandadoptionofopensourcesoftware(e.g.,OpenSourceSoftwareforScience). Manychallengestosucheffortswerediscussed,thoughtheytendedtofallintotw