GenAICHI 2024

A conference seminar involving with an AI that does not simply categorize data and interpret text as determined by models, but instead creates something new—e.g., in images, molecules, or designs. This work moves the potential of AI systems from problem solving to problem finding, and it tends to change the “role” of the AI from decision-maker to human-supporter. The session is focussed on various aspects of generative AI and its interactions with humans, including new sociotechnical opportunities for work and recreation that are afforded by powerful new interfaces.
Created with Adobe Firefly

Outline

Please join us for the third Generative AI and HCI workshop - this year at CHI 2024. We previously ran workshops in 2022 and 2023.

In the past year, we have seen or made powerful tools that can create images from textual descriptions or conduct reasonably coherent conversations, make writing suggestions for creative writers, and write code as a pair programmer. We have also seen claims of what an historical person “really looked like,” and of a “completed” version of a musical compositions left unfinished by their composer’s untimely death. What all of these examples have in common is that the AI does not simply categorize data and interpret text as determined by models, but instead creates something new—e.g., in images, molecules, or designs. This work moves the potential of AI systems from problem solving to problem finding, and it tends to change the “role” of the AI from decision-maker to human-supporter. Following a successful CHI workshop in 2022, we focus on various aspects of generative AI and its interactions with humans, including

  • new sociotechnical opportunities for work and recreation that are afforded by powerful new interactive capabili- ties
  • novel design challenges of systems that produce a different outcome after each invocation
  • ethical issues related to their design and use; and
  • useful patterns for collaboration between humans and generative AI in different domains

Generative AI can be defined as an AI system that uses existing media to create new, plausible media. This scope is broad, and the generative potential of AI systems varies greatly. Over the last decade, we have seen a shift in methodology moving from expert systems based on patterns and heavy human curating towards stochastic and generative models such as Generative Adversarial Networks (GANs) that use big data to produce convincingly human-like results in various domains, and Large Language Models (LLMs) that can generate text, source code, and images from simple instructions (“prompts”).

Past Workshop Proceedings

The Generative AI and HCI workshop has been running since 2022. Here you can find links to past workshop proceedings including calls for papers, programs, and accepted workshop paper.

Program

(in progress) All times are in HST local time

Papers

09:00-09:20 Welcome + Introductions

09:20-09:50 Paper session 1: Prompting

  • 09:20 Inkspire: Supporting Designers to Prototype Product Designs through Sketching. David Chuan-En Lin (Carnegie Mellon University); Hyeonsu Kang (Carnegie Mellon University); Nikolas Martelaro (Carnegie Mellon University); Aniket Kittur (Carnegie Mellon University); Yan-Ying Chen (Toyota Research Institute); Matthew K Hong (Toyota Research Institute).
  • 09:25 Equivalence: An analysis of artists’ roles with Image Generative AI from Conceptual Art perspective through an interactive installation design practice. Yixuan Li (Georgia Institute of Technology); Dan Baciu (Delft University of Technology); Marcos Novak (Media Arts & Technology UCSB); george legrady (media arts & technology UCSB).
  • 09:30 Augmenting the Author: Exploring the Potential of AI Collaboration in Academic Writing. Joseph Tu (University of Waterloo); Hilda Hadan (University of Waterloo); Derrick M Wang (University of Waterloo); Sabrina A Sgandurra (University of Waterloo); Reza Hadi Mogavi (University of Waterloo); Lennart E Nacke (University of Waterloo).
  • 09:35 Leveraging AI to Generate Audio for User-generated Content in Video Games. Thomas Marrinan (University of St. Thomas); Pakeeza Akram (Univeristy of St. Thomas); Oli Gurmessa (University of St. Thomas); Anthony Shishkin (University of St. Thomas).
  • 09:40 Discussion

09:50-10:20 Paper session 2: Creativity: Media

  • 09:50 Human-AI Collaboration Insights from Music Composition. Eric Tron Gianet (University of Torino); Luigi Di Caro (University of Turin); Amon Rapp (University of Torino).
  • 09:55 Interaction Design for Human-AI Choreography Co-creation Yimeng Liu (UC Santa Barbara).
  • 10:00 Seizing the Means of Production: Exploring the Landscape of Crafting, Adapting and Navigating Generative AI Models. Ahmed M. Abuzuraiq (Simon Fraser University); Philippe Pasquier (Simon Fraser University).
  • 10:05 Designing Live Human-AI Collaboration for Musical Improvisation. Nic Becker (Stanford University); Ryan Louie (Stanford University); John Thickstun (University of Washington).
  • 10:10 Discussion

*10:20-11:00 Break

11:00-11:20 Paper session 3: Creativity: Media: Trade-offs

  • 11:00 Shaping Realities: Enhancing 3D Generative AI with Fabrication Constraints. Faraz Faruqi (MIT CSAIL); Yingtao Tian (Google Research, Brain Team); Vrushank Phadnis (Google Research); Varun Jampani (Stability AI); Stefanie Mueller (MIT CSAIL).
  • 11:05 Case Study of GAI for Generating Novel Images for Real-World Embroidery. Kate Glazko (University of Washington); Anika Arugunta (University of Washington); Janelle Chan (University of Washington); Nancy Jimenez-Garcia (University of Washington); Tashfia Sharmin (University of Washington); Jennifer C Mankoff (University of Washington).
  • 11:10 Discussion

11:20-11:45 Paper session 4: Creativity: Text

  • 11:20 Workplace Everyday-Creativity through a Highly-Conversational UI to Large Language Models Michael Muller (IBM); Jessica He (IBM Research); Justin Weisz (IBM Research AI).
  • 11:25 Working with Large Language Models in the Rapid Ideation Process. Gionnieve Lim (Singapore University of Technology and Design); Simon Perrault (Singapore University of Technology and Design).
  • 11:30 Productively Hallucinating Desirable Futures with Generative AI. Peter Kun (IT University of Copenhagen); Matthias Freiberger (Copenhagen University); Sebastian Risi (IT University of Copenhagen); Anders S Løvlie (IT University of Copenhagen).
  • 11:35 Discussion.

11:45-12:20 Posters 1

12:20-14:00 Lunch

14:00-14:25 Posters 2

14:25-14:50 Paper session 5: Values: Harms

  • 14:25 From Melting Pots to Misrepresentations: Exploring Harms in Generative AI. Sanjana Gautam (Pennsylvania State University); Pranav N Venkit (Pennsylvania State University ); Sourojit Ghosh (University of Washington).
  • 14:30 The Need for Flexible Interfaces for Text-to-Image Auditing: A Case Study of DALL·E 2 and DALL·E 3. Clare Provenzano (Simon Fraser University); Parsa Rajabi (Simon Fraser University); Diana Cukierman (Simon Fraser University); Nicholas Vincent (Simon Fraser University).
  • 14:35 How an AI Generated Experience Impacts Negative Perceptions of AI.* MJ Johns (University of California Santa Cruz)*; Tyler Coleman (University of California).
  • 14:40 Discussion

14:50-15:20 Paper session 6: Values: Process and Media

  • 14:50 Compliance Rating Scheme: Introducing Data Provenance for Dataset Use in Generative AI Applications. Matyas Bohacek (Stanford University); Ignacio Vilanova Echavarri (Imperial College London).
  • 14:55 A Value-Oriented Investigation of Photoshop’s Generative Fill. Ian Swift (University of Illinois at Chicago); Debaleena Chattopadhyay (University of Illinois at Chicago).
  • 15:00 Echo Chamber: Generative Music AI within a Participatory Museum Sound Installation. Monica M Lim (University of Melbourne); Jarrod Knibbe (University of Queensland); Bingqing Chen (University of Melbourne); Ying Sima (University of Melbourne).
  • 15:05 Posthumanist AI: Rethinking ‘the human’ as a model for generative AI. Matt V Ratto (University of Toronto); Sarah Gram (University of Toronto); Olivia Doggett (University of Toronto); Peter Selby (Center for Mental Health and Addiction); Nadia Minian (Center for Mental Health and Addiction); Marta Maslej (Center for Mental Health and Addiction); Osnat Melamed (Center for Mental Health and Addiction); Jonathan Rose (University of Toronto).
  • 15:10 Discussion.

15:20-16:00 Break

16:00-16:20 Paper session 7: Analysis

  • 16:00 Can Nuanced Language Lead to More Actionable Insights? Exploring the Role of Generative AI in Analytical Narrative Structure. Vidya Setlur (Tableau Research); Larry Birnbaum (Salesforce/Northwestern University).
  • 16:05 Unlocking the User Experience of Generative AI Applications: Design Patternsand Principles. Vinita Tibdewal (Google USA).
  • 16:10 Discussion

16:20-16:50 Posters 2

16:50-17:20 Closing

Posters

All poster sessions run concurrently during all three Poster periods.

Poster session 1: Tools or Partners

  • Loremaster: Towards Better Mixed-Initiative Content Co-Creation in the Creative Industries. Oliver H Wood (Those Beyond).
  • Generating Summary Videos from User Questions to Support Video-Based Learning. Kazuki Kawamura (The University of Tokyo); Jun Rekimoto (The Univertsity of Tokyo).
  • Exploring Human-AI Collaboration Continuum in Augmented Reality Applications. Shokoufeh Bozorgmehrian (Virginia Tech); Joseph Gabbard (Virginia Tech).
  • How GenAI Can Affect Design Work. Samangi Wadinambiarachchi (The University of Melbourne)*; Jenny Waycott (The University of Melbourne); Ryan Kelly (RMIT University); Eduardo Velloso (The University of Sydney); Greg Wadley (University of Melbourne, AUS).

Poster session 2: Prompting

  • Generative AI in Color-Changing Systems: Re-Programmable 3D Object Textures with Material and Design Constraints. Yunyi Zhu (MIT CSAIL); Faraz Faruqi (MIT CSAIL); Stefanie Mueller (MIT CSAIL).
  • On Abduction and Reflective Prompt Engineering. Gabriel D J Barbosa (Pontifical Catholic University of Rio de Janeiro); Clarisse Sieckenius de Souza (Pontifical Catholic University of Rio de Janeiro).

Poster session 3: Creativity

  • Uncovering Challenges and Changes in Artists’ Practices as a Consequence of AI. Petra Jääskeläinen (KTH); Anna-Kaisa Kaila (KTH Royal Institute of Technology, Stockholm); Andre Holzapfel (KTH Royal Institute of Technology in Stockholm).
  • Food Development through Co-creation with AI: bread with a “taste of love”. Takuya Sera (NEC Corporation); Izumi Kuwata (NEC Corporation); Yuki Taya (NEC Corporation); Noritaka Shimura (NEC Corporation); Yosuke Motohashi (NEC Corporation).
  • Towards a Hierarchy of Trust in Human-AI Music-Making. Adrian Hazzard (University of Nottingham); Craig Vear (University of Nottingham); Solomiya Moroz (University of Nottingham).
  • Generative AI for Musicians: Small-Data Prototyping to Design Intelligent Musical Instruments. Charles Patrick Martin (Australian National University).
  • Unstuck: Charting the Design Space of Generative AI-based Creativity Interventions. Matthew K Hong (Toyota Research Institute); Pablo Paredes (Toyota Research Institute); Shabnam Hakimi (Toyota Research Institute); Monica Van (Toyota Research Institute); Matthew Klenk (Toyota Research Institute).

Poster session 4: Harms: Privacy

  • Privacy Risks and Legal Challenges in the Emergent Use of Generative AI in Research with Participant Data. Wendy Moncur (University of Strathclyde); Ryan Gibson (University of Strathclyde); Ali Farooq (University of Strathclyde); Burkhard Schafer (University of Edinburg).
  • Exploring ChatGPT’s ability to detect privacy violations in photo sharing. Arun Balaji Buduru (IIIT Delhi); Apu Kapadia (Indiana University)*; Christine Chen (Indiana University); Chris Page (Indiana University); Kendrick Mernitz (Indiana University); Ben Malone (Indiana University).
  • Closing the Loop: Embedding Observability in the GenAI Product Lifecycle for Systematic Bias Mitigation. Freyam Mehta (International Institute of Information technology Hyderabad); Nimmi Rangaswamy (International Institute of Information technology Hyderabad).
  • On CLIP’s Ability of Analyzing Fake Images at a Large Scale: Why are they fake? Jinbin Huang (Arizona State University); Chen Chen (University of Maryland, College Park); Aditi Mishra (Arizona State University); Bum Chul Kwon (IBM Research); Zhicheng Liu (University of Maryland, College Park); Chris Bryan (Arizona State University).
  • Detecting Generative AI Usage in Application Essays. Neil Natarajan (University of Oxford).
  • Holding the Line: A Study of Writers’ Attitudes on Co-creativity with AI. Morteza Behrooz (Meta).

Poster session 5: Values: Synthetic

  • How can LLMs support UX Practitioners with image-related tasks? Ruican Zhong (University of Washington); Gary Hsieh (University of Washington); David McDonald (University of Washington).
  • Exploring the role of Generative AI models for research in the public sector. Kelly McConvey (University of Toronto); Erina Moon (University of Toronto); Shion Guha (University of Toronto).
  • Machine Conversations as Design Conversations. Alice Cai (Harvard University); Celine Offerman (Delft University of Technology); Shiyan Zhang (Stevens Institute of Technology); Lydia B Chilton (Columbia University); Kevin Crowston (Syracuse University); Jeffrey V Nickerson (Stevens Institute of Technology).

Poster session 6: Emotion.

  • Crafting Emotional TTS Conversation Responses Based on User Preferences. Laya Iyer (Stanford University).
  • Physiological Signals as Implicit Multimodal Input in Human-AI Interactions During Creative Tasks. Teodora Mitrevska (LMU).

Topics and Themes

Our workshop is open to diverse interpretations of interactive generative AI, characterized by the AI systems’ abilities to make new things, learn new things and foster serendipity and emergence. We are interested in several dimensions of generative AI, including mixed initiative, human–computer collaboration, or human–computer competition, with the main focus on interaction between humans and generative AI agents. We welcome researchers from various disciplines, inviting researchers from different creative domains including, but not limited to art, images, music, text, style transfer, text-to-image, programming, architecture, design, fashion and movement. Because of the far-reaching implications of Generative AI, we propose the following list of non-exhaustive, thematic questions to guide our discussions at the workshop:

  • What is generative AI and how can we leverage diverse definitions of it? Does generative AI go beyond intelligent interaction? What distinguishes generative AI?
  • How do you design in this characteristically uncertain space? What design patterns do we need to think about? How does uncertainty play into this and how to we help people set expectations to designing with AI?
  • Do generative AI algorithms contribute needed serendipity to the design process—or simply randomness—or worse, chaos?
  • Is presenting AI as a desirable and “objective” method appropriate for generative AI?

We encourage people to write and answer their own questions as well. We hope that the workshop leads to new ways-of-thinking.

These themes can be addressed within the following topics:

  • The emerging capabilities of generative AI.
  • Generative AI existence in different domains including (but not limited to) images, music, text, design, and motion.
  • The role of generative AI in assisting, replacing, and regimenting human work.
  • Human-AI collaboration and co-creative systems.
  • Ethical issues including misuses and abuses, provenance, copyright, bias, and diversity.
  • The uncanny valley in Human-AI interactions.
  • Speculative futures of generative AI and its implications for human-AI utopias and dystopias.

As above, we encourage people to add new topics and domains.

Contributing Your Work

Submissions may be up to 4 pages long (references may appear on additional pages), following the CHI 2024 instructions for papers.

The deadline for submissions and submission website is found at the top of the page.

Please send any comments or questions to Michael Muller, michael_muller@us.ibm.com.

Accepted papers will be presented in the workshop and authors can choose to publish their paper here on the workshop website under a Creative Commons Attribution 4.0 International License (CC BY 4.0).

Program Committee

  • Alexa Steinbrück, Hochschule für Gestaltung Schwäbisch-Gmünd
  • Andre Holzapfel, KTH Royal Institute of Technology in Stockholm
  • Anna Kantosalo, University of Helsinki
  • Anna-Kaisa Kaila, KTH Royal Institute of Technology, Stockholm
  • Anthony Jameson, Contaction AG
  • Benedikte Wallace, University of Oslo
  • Briane Paul Samson, De La Salle University
  • Charles Martin, Australian National University
  • Cory Newman, University of Baltimore
  • Deepak Giri, Indiana University
  • Dmitry Zubarev, IBM Research-Almaden
  • Erinma Ochu, UWE Bristol
  • Gaole He, Delft University of Technology
  • Garrett Allen, Delft University of Technology
  • Greg Walsh, University of Baltimore
  • Han Li, National University of Singapore
  • Hernisa Kacorri, University of Maryland, College Park
  • Hyo Jin Do, IBM Research
  • Imke Grabe, IT University of Copenhagen
  • Jacquelyn Martino, IBM Research
  • Jessica He, IBM Research
  • Jordan Aiko Deja, De La Salle University
  • Jussi Holopainen, City University of Hong Kong
  • Justin Weisz, IBM Research AI
  • Lorenzo Corti, Delft University of Technology
  • Manish Nagireddy, IBM Research
  • Mary Lou Maher, Computer Research Association
  • Matthew Klenk, Toyota Research Institute
  • Matthew Hong, Toyota Research Institute
  • Michael Hind, IBM Research
  • Michael Muller, IBM Research
  • Minsik Choi, The Australian National University
  • Nabila Chowdhury, University of Toronto
  • Paolo Grigis, Unibz
  • Rgee Wharlo Gallega, Future University Hakodate
  • Richard Lance Parayno, University of Salzburg
  • Sachita Nishal, Northwestern University
  • Shabnam Hakimi, Toyota Research Institute
  • Tae Soo Kim, KAIST
  • Viktoria Pammer-Schindler, Graz University of Technology
  • Werner Geyer, IBM Research
  • Yash Tadimalla, University of North Carolina Charlotte
  • Yichen Wang, Australian National University
  • Zijian Ding, The University of Maryland

Organizers

Anna Kantosalo is a Postdoctoral Researcher at the University of Helsinki. The focus of her research is Human–Computer Co-Creativity and she is defining models and methods for building and describing systems in which humans and autonomous creative agents can work together. She has chaired the Future of Co-Creative Systems workshop adjoined with the International Conference on Computational Creativity twice.

Mary Lou Maher is a Professor in the Software and Information Systems Department at the University of North Carolina at Charlotte. Her early research in AI-based generative design has lead to a human centered approach to computational creativity and co-creative systems. She has Chaired the Creativity and Cognition Conference (2019) and the International Conference on Computational Creativity (2019) as well as organized several workshops on AI-based design and creativity.

Charles Patrick Martin is a Lecturer in Computer Science at the Australian National University. Charles works at the intersection of music, AI/ML and HCI. He studies how humans can interact creatively with intelligent computing systems and how such systems might fit in the real world. Charles has organised multiple generative-AI-focused workshops at the New Interfaces for Musical Expression conference.

Michael Muller works as a Senior Research Scientist at IBM Research in Cambridge MA USA. With colleagues, he has analyzed how domain experts make use of generative AI outcomes, and how humans intervene between “the data” and “the model” as aspects of responsible and accountable data science work. His research occurs in a hybrid space of Human-Centered AI (HCAI), Human-Computer Interaction (HCI), design, and social justice.

Greg Walsh is an associate professor at the University of Baltimore where he teaches courses in Design. He is an interaction design researcher who focuses on user-centered, inclusive design for children and adults. His work seeks to include more voices in the design process and has been a recipient of a prestigious Google Faculty Research Award. His work has included participatory design sessions in Baltimore City libraries and is now exploring the use of generative AI as a co-design partner.