人类与人工智能协作的科学与艺术

by Matthew Groh M.A., Massachusetts Institute of Technology (2019)B.A. Middlebury College (2010) Submitted to the Program in Media Arts and Sciences,School of Architecture and Planning,in partial fulfillment of the requirements for the degree of Doctor of Philosophy in Media Arts and Sciences at the MASSACHUSETTS INSTITUTE OF TECHNOLOGY June 2023 ©Matthew Groh, MMXXIII. All rights reserved. The author hereby grants to MIT a nonexclusive, worldwide, irrevocable,royalty-free license to exercise any and all rights under copyright, including toreproduce, preserve, distribute and publicly display copies of the thesis, or releasethe thesis under an open-access license. Program in Media Arts and SciencesMay 8, 2023 Rosalind PicardProfessor of Media Arts and SciencesProgram in Media Arts and SciencesThesis Supervisor Todd MachoverAcademic HeadProgram in Media Arts and Sciences The Science and Art of Human and Artificial Intelligence Collaboration by Matthew Groh Submitted to the Program in Media Arts and Sciences,School of Architecture and Planning,on May 8, 2023, in partial fulfillment of therequirements for the degree ofDoctor of Philosophy in Media Arts and Sciences Abstract While artificial intelligence (AI) appears to be surpassing the performance of human expertson a wide variety of games and real-world tasks, these algorithms are prone to systematic andsurprising failures when deployed. In contrast to today’s state-of-the-art algorithms, humansare highly capable of adapting to new contexts. The different strengths and weaknesses ofhumans and AI motivate a guiding research question for the emerging field of human-AIcollaboration: When, where, why, and how does the combination of human problem solvingand AI systems lead to a hybrid system that surpasses (or fails to surpass) the performanceof either humans or the machine alone? This dissertation addresses various dimensions ofthis guiding question by conducting large-scale, digital experiments across three distincttasks and domains:deepfake detection, dermatology diagnosis, and Wordle.First, theexperiments in deepfake detection examine the similarities and differences between humanand machine vision in identifying visual manipulations of people’s faces in videos and identifyimportant performance trade-offs between hybrid systems and human or AI only systems fordeepfake detection. Second, the experiments in dermatology diagnosis reveal that non-visualinformation is often essential for diagnosing skin disease, diagnostic accuracy disparitiesacross skin color exist in image-only store-and-forward teledermatology, and clinical decisionsupport based on a fair deep learning system can significantly increase physicians’ diagnosticaccuracy in this experimental setting. Third, the experiment on Wordle demonstrates thatdigitally mediated expressions of empathy can counteract the negative effect of anger onhuman creative problem solving. In addition to these digital experiments, this dissertationpresents two algorithmic audits on clinical dermatology images to reveal where systematicerrors arise in state-of-the-art algorithms, examines how context influences automated affectrecognition, and proposes methods for more effectively incorporating context in appliedmachine learning. Together, these contributions provide empirical evidence for why human-AI collaborations succeed and fail across a variety of tasks and domains, insights into howto design human-AI collaborations more effectively, and a framework for when and wherehybrid systems should rely on human problem solving. Thesis Supervisor: Rosalind PicardTitle: Professor of Media Arts and Sciences, Program in Media Arts and Sciences The Science and Art of Human and Artificial Intelligence Collaboration by Matthew Groh The following people served as readers for this thesis: Rosalind PicardProfessor of Media Arts and SciencesMassachusetts Institute of Technology David RandErwin H. Schell Professor of Management Science and Brain and Cognitive SciencesMassachusetts Institute of Technology Inspiration “By this art you may contemplate the variations of the 23 letters...”– Robert Burton andJorge Luis Borges “The important thing is not to stop questioning. Curiosity has its own reason for existing.One cannot help but be in awe when contemplating the mysteries of eternity, of life, of themarvelous structure of reality.” – Albert Einstein “Although there is a sense in which the camera does indeed capture reality, not just interpretit, photographs are as much an interpretation of the work as paintings and drawings are.”– Susan Sontag “Understanding a theory means, I suggest, understanding itas an attempt to solve a certainproblem.” – Karl Popper “The hybrid or the meeting of two media is a moment of truth and revelation from whichnew form is born.” – Marshall McLuhan Acknowledgments Doing a PhD felt like a Type II fun adventure through the world of ideas. I am absolutelygrateful for the gift of th