research

My research focuses on three complementary themes: useful AI, safe AI, and personal AI.

AI systems don’t need to be perfect to be useful (Weisz et al., 2021). But to be useful, they:

Need to fit into people’s existing workflows: the tools they already use and the context of the work they’re doing.
Need to minimize their potential for harm, both through explicit mitigations and by helping people critically evaluate their outputs.
Need to meet people where they’re at. People differ in how fluently they use AI and in how they prefer it to interact with them. People and AI both need new mechanisms for understanding each other.

These three points correspond to the three themes of my research: useful AI, safe AI, and personal AI. These themes overlap, and some of my projects sit in that overlap.

Jump to: Useful AI · Safe AI · Personal AI · Intersections

Useful AI

Useful AI fits into the way people already work and draws on their working context. My work in this space has focused on professionals at the heart of building technology, such as software engineers, designers, and data scientists.

AI for software engineering

Software engineering was one of the first domains where generative AI became genuinely useful. I’ve studied it with my colleagues since 2019, before large language models became mainstream.

We began by asking whether software engineers would find value in a generative model whose output was often wrong, because state-of-the-art model performance was much poorer than it is today. In a design probe of AI-supported code translation, we found that software engineers were receptive to AI despite its imperfections because they simply needed help (Weisz et al., 2021). In a follow-up experiment with 32 software engineers, we examined the utility of AI-produced code translations (Weisz et al., 2022). We found that some, but not all, software engineers were able to work productively with imperfect translations by finding and fixing errors, and overall, participants produced code with fewer errors when aided by AI than when working alone. We also observed cases of a scaffolding effect, in which AI translations enabled developers less familiar with the programming language to complete a translation task that would otherwise have been too difficult for them. Along the way, we identified the kinds of explanations software engineers need from generative code models (Sun et al., 2022), developed a method for estimating the quality of AI-produced code translations (Agarwal et al., 2020), and created a technique for building the datasets needed to train code translation models (Agarwal et al., 2021).

In 2022, interactions with code-fluent language models primarily took the form of “push button, get results.” We built the Programmer’s Assistant, a tool that embedded a conversational LLM inside a code editor, to explore how software engineers felt about having conversational interactions around their code. In our evaluation, we found that developers valued the ability to ask both general programming questions and questions specific to their own code (Ross et al., 2023; Ross et al., 2023; Ross et al., 2023). While our paper was under review, ChatGPT was released in November 2022, demonstrating that our hypothesis was correct: LLMs reached a point where they could drive natural conversational interactions, and our work demonstrated that those interactions could be situated in a co-creative context: what we termed artifact-grounded conversation. Our work helped pave the way for the first generation of conversational AI coding assistants, including IBM’s watsonx Code Assistant, and our demo of the Programmer’s Assistant received a Best Demo Honorable Mention award at IUI 2023.

In parallel with the development of watsonx Code Assistant (WCA), we built the Agentic Code Explorer (ACE). ACE combined retrieval-augmented generation against a code repository with user modeling to produce personalized code explanations tailored to an individual programmer (Silva Moran et al., 2025).

Left: the Programmer's Assistant embeds a conversational assistant alongside a code editor (Ross et al., 2023). Right: the Agentic Code Explorer keeps a profile of each developer's roles and expertise, which it uses to tailor its explanations (Silva Moran et al., 2025).

During an internal deployment of an early version of WCA, we examined its use and impact on developer productivity. Counter to our expectations, we found that its most common use was not for generating code, but rather for explaining it (Weisz et al., 2025). Feedback from WCA’s users also told us that its explanations were often verbose and not well-matched to their expertise. Together, these observations reinforced the ideas behind ACE and motivated new research into personalized code explanations.

Finally, we used speculative design methods to think about the future of AI-assisted software engineering. We wrote multiple design fictions, including one that probes the value tensions of pair programming with an AI (Muller et al., 2022; Muller et al., 2024) and another that examines the role of the software engineer when they lose the ability to comprehend the code produced by AI (Weisz et al., 2025). This fiction was produced with Story Arena, an environment I developed in which AI agents adopting various personas come together to collectively write design fictions.

AI for designers

Generative AI opened new possibilities for creative and design work. In this space, we developed frameworks for mixed-initiative generative AI interfaces and studied opportunities for generative AI in design and UX work (Muller et al., 2020; Houde et al., 2020; Houde et al., 2022; Muller et al., 2023; Muller et al., 2025).

One example stems from our observation that designers spent a lot of time running group ideation sessions. Sometimes, these sessions would stall when people were quiet for uncomfortably long periods of time. Other times, people would make comments or suggestions that would have benefitted from having someone else in the room with a particular expertise or viewpoint. In response, we built two prototypes to explore how an LLM-based agent might participate in the group ideation process:

Koala is a Slack bot that participates in group brainstorming discussions. It contributes ideas either reactively, when its name is mentioned, or proactively, when it judges that it has something valuable to add to the conversation. In an evaluation study, we found that participants preferred having Koala in their groups, and most of the top ideas that were generated in the brainstorm were shaped by both the participants and Koala (Muller et al., 2024).
Collaborative Canvas is a shared virtual canvas where people place digital stickies together, enhanced with the ability to generate & rewrite stickies using an LLM (like Mural, before they added AI features). Across multiple studies, design practitioners saw value in embedding AI into the canvas to fill in expertise gaps and break through lulls (He et al., 2024; Gonzalez et al., 2024).

Left: Koala contributes ideas to a brainstorming session in Slack (Muller et al., 2024). Right: the Collaborative Canvas, where people and an LLM add sticky notes to a shared board (He et al., 2024).

These prototypes helped us develop a framework around how to design an AI agent’s participatory behaviors in group conversations: when it should contribute, what and where, and who has control to change the agent’s behavior (Houde et al., 2025).

AI for data scientists

In 2018, automated machine learning (AutoML) tools were beginning to automate several portions of the data science lifecycle: feature engineering, model selection, and hyperparameter tuning. We were very keen to understand how data scientists would feel about these technologies: were they viewed as helpful or were they not to be trusted? We interviewed 20 data scientists about their perceptions of automated AI (Wang et al., 2019). Reactions were mixed: they worried about automation encroaching on their jobs and felt it was inevitable, yet they remained optimistic, because they saw the future of data science as a collaboration in which both automation and human expertise are indispensable. We then examined what information data scientists need to trust the models AutoML produces, which we measured as their willingness to deploy them. Adding transparency features to an AutoML tool increased both trust and understanding, and model performance metrics and visualizations mattered most (Drozdal et al., 2020).

These findings shaped IBM AutoAI, a product that automates the end-to-end AI lifecycle while keeping humans in the loop (Wang et al., 2020; Wang et al., 2020). We also developed AutoAIViz to visualize the pipelines that AutoAI generates (Weidele et al., 2020), Model LineUpper to help data scientists interactively compare the models it produced (Narkar et al., 2021), and AutoDS to explore human-centered automation of the broader data science workflow (Wang et al., 2021). We also built an AI system that generates documentation for data science code in computational notebooks and found that it helped data scientists document their code more quickly without sacrificing the quality of that documentation (Wang et al., 2022; Muller et al., 2021). Finally, we reflected on how AI-assisted data science can cultivate human expertise rather than replace it (Weisz & Muller, 2020; Andres et al., 2020).

Safe AI

Developing AI that is safe requires identifying its risks and potential harms and helping both developers and users recognize and mitigate them.

Mapping the risks of AI

AI systems are probabilistic, so they make mistakes, and those mistakes can cause real harm when AI is used in high-stakes settings such as hiring, lending, and healthcare. A biased model can deny someone a loan; an inaccurate one can miss a diagnosis. Generative AI widened this risk landscape: alongside familiar risks such as bias, it introduced new ones, such as hallucination, harmful content, and the leakage of copyrighted or personal data. As generative AI entered the mainstream in 2023, I helped document its risks, their potential harms, and strategies for mitigating them. This material was incorporated into the IBM AI Risk Atlas, which helps IBM’s customers understand the AI risk landscape and make informed decisions about developing and deploying AI in their organizations.

Factuality and transparency

Hallucination is one of the biggest risks of generative AI: LLMs can produce output that sounds confident but is wrong. Many groups, including IBM, developed quantitative methods for scoring the factuality of an LLM’s output. But scores alone may be difficult to interpret; if a sentence scores “0.67 out of 1” for factuality, what does that really mean? We explored how to present factuality scores to people in ways that were easy to understand and interpret. Across two experiments, we found that people preferred and trusted a design that color-coded every phrase within an LLM’s response by its factuality score, and they found this highlighting scheme the easiest to validate (Do et al., 2025; Do et al., 2024). Our results informed the design of evaluations in watsonx.governance.

Designs for displaying factuality scores

Designs we tested for showing the factuality of an LLM's response, at the level of individual terms or whole phrases. Red marks content that is likely inaccurate and green marks content that is likely accurate (Do et al., 2025).

Privacy

New technologies can erode people’s privacy in ways they don’t expect. When consumer drones took off in the mid-2010s, we conducted a large-scale survey of people’s privacy expectations and fears regarding drones, along with how they felt about different privacy-preserving mitigations (Uchidiuno et al., 2018). Now, the rising popularity of AI agents is raising new privacy concerns, especially when people overshare sensitive information that isn’t required to complete a task. With my colleagues at IBM, we examined ways to help people maintain contextual privacy: sharing sensitive information only within the contexts that require it, such as sharing medical information with your doctor but not your accountant. We developed methods that protect people from oversharing with conversational agents and a way to evaluate contextual privacy across agentic workflows (Ngong et al., 2025; Ngong et al., 2026).

Trust

Why should someone trust an AI system to make a decision for them? Well before the popularity of generative AI, we set out to understand what made an AI-infused decision-making process trustworthy (Ashoori & Weisz, 2019). Trust is a complicated construct, and it can be examined behaviorally, by observing whether people act on an AI’s recommendations, or attitudinally, by asking how willing people are to rely on it. We took the attitudinal view in our work and ran a crowdsourced study to examine people’s willingness to trust AI across a variety of scenarios. People rated the trustworthiness of decision-making processes that ranged from a lower-stakes decision (planning meals) to a higher-stakes one (recommending a prison sentence). Across the scenarios, we varied factors such as the stakes of the decision, who had final authority over it, who trained the model, whether the model was interpretable, and how confident the model was in its recommendation. We found that people accepted AI more readily for lower-stakes decisions and for interpretable models. Information about how a model was trained and tested also increased their trust.

Personal AI

People construct mental models of AI systems that help them understand how to work with them effectively. Why can’t AI systems do the same? Personal AI is about much more than personalization: it’s about helping human-AI teams form a mutual understanding of each other to enable the team to accomplish goals that neither party could have accomplished alone.

Mutual theory of mind

Teams of people work well together when their members understand each other: they share common ground about what each person knows, and they can anticipate how their teammates will act. Yet, many human-AI teaming studies show that AI doesn’t always deliver the performance gains we might expect. We hypothesized that human-AI teams would perform better if the human and the AI held a mutual theory of mind: when each party holds a model of the other’s goals, knowledge, and reasoning, and uses it to predict how the other will behave. We used design fictions to envision both the promise and the pitfalls of an operationalized mutual theory of mind (Weisz et al., 2024). We also identified a difference between literal theory of mind — the ability to predict what another agent will do — and functional theory of mind — the ability to adapt one’s own behavior to that prediction — and argued that many theory of mind LLM benchmarks were limited because they only focused on evaluating the former and not the latter (Riemer et al., 2025).

To study mutual theory of mind in practice, we built Wavelength.AI, a version of the cooperative party game Wavelength in which people play alongside an LLM-based agent. Wavelength’s defining characteristic is that players must understand each other well to play well. We tested two interventions designed to help human-AI teams form a mutual theory of mind: introductions before play to establish common ground, and retrospectives in which players saw the reasoning behind each other’s clues and guesses. In a preliminary study with 24 human-AI teams, teams that had both interventions tended to score higher than teams that did not, although the difference was not statistically significant. Players said that the introductions helped them give better clues and that the retrospectives helped them adjust their strategies (Morrison et al., 2026; Morrison et al., 2025).

Wavelength.AI: a person and an LLM-based agent take turns giving clues and guessing where a target falls on a spectrum (Morrison et al., 2026).

Intersections

The three themes of my research reinforce each other. AI is rarely useful if people can’t trust it, and it is rarely trustworthy if it ignores who is using it. Some of my projects address these themes in tandem.

Design principles for generative AI

Useful AI · Safe AI · Personal AI

Generative AI applications pose new design challenges stemming from their non-deterministic nature: their outputs can vary even given the same prompt, they can be confidently wrong, and when people overrely on incorrect outputs, the results can be harmful. I led a cross-IBM effort to distill research on human-AI collaboration and co-creation into a set of design principles, strategies, training materials, and exercises for designing generative AI applications (Weisz et al., 2024; Weisz et al., 2023; Weisz et al., 2025). Our materials were used to train over 3,000 IBM design practitioners to work with AI as a design material.

Design principles for generative AI applications

Six principles for designing generative AI applications. Three offer new interpretations of known issues in AI, and three address characteristics unique to generative AI (Weisz et al., 2024).

Human-centered explainable AI

Useful AI · Safe AI

Explainable AI (XAI) research overwhelmingly treats AI as a black box to be opened: if only one could explain how the model operates (i.e. global explanations) and why the model made a specific prediction (i.e. local explanations), then explainability has been achieved and people can properly calibrate their trust in the model. But, models are increasingly becoming inscrutable (e.g. deep learning models, transformer-based LLMs), and explainability approaches such as training surrogate models (e.g. LIME) can only provide approximations of the model’s actual behavior. Instead, we recognized that a wealth of information about a model’s performance was present outside of the black box, in the social and organizational contexts in which the model was being used. We introduced social transparency as a form of explainability in which information about whether a model’s recommendations were followed or not, along with the reasons why, were captured and presented to other users to help them calibrate their trust (Ehsan et al., 2021). This work helped launch the field of human-centered explainable AI (HCXAI), which we have grown through a workshop series at CHI (Ehsan et al., 2023; Ehsan et al., 2025; Ehsan et al., 2026).

Social transparency in an AI-mediated sales decision

Social transparency in an AI pricing tool: alongside the AI's recommendation and its justification, salespeople see what their colleagues did with past recommendations, along with who acted, why, and when (Ehsan et al., 2021).

AI attribution

Useful AI · Safe AI

When AI helps write source code or an essay, or creates an image, who should get credit for the work? Our norms for crediting human collaborators don’t map neatly onto AI. In a scenario-based study, we found that people felt that AI does deserve credit for its contributions, although the amount of credit it deserves varies based on its contribution (e.g. synthesizing information or contributing new ideas deserves more credit than making grammatical changes). Thus, we learned that how AI was used matters, not just whether it was used. Yet, at the time, many organizational AI-use policies only required disclosing the fact of whether AI was used (He et al., 2025). In response, we built the AI Attribution Toolkit to give content creators a more precise language for describing how AI contributed to their work. We then examined the social stigmas surrounding AI use and disclosure. Through a survey of workers who used AI, we found that disclosures stating one’s compliance with AI policies, disclosures of how AI contributed, and disclosures that a human reviewed AI-generated content made people feel more comfortable disclosing their AI use. However, these disclosures were no more effective than a simple acknowledgment of AI use at changing whether people chose to disclose; shifting that decision may also require sociocultural enablers, such as messaging from leadership and the disclosure behaviors of peers (He et al., 2026).

Anatomy of an attribution statement from the AI Attribution Toolkit, which describes how much AI contributed, what it contributed, who initiated the work, whether a person reviewed the AI's content, and which AI application was used.

Personalized code explanations

Useful AI · Personal AI

People differ in the ways they process information, how they learn, and how confident they feel with technology. Returning to the domain of software engineering, these observations imply that one-size-fits-all explanations of code won’t be equally beneficial. We asked software engineers what kinds of information they wanted in their code explanations (Brachman et al., 2025) and found that there wasn’t a single consistent answer: they wanted to configure the structure and level of detail of their explanations, and they wanted explanations that would personalize themselves automatically to their experience and goals. We then prompted an LLM to adapt its code explanations to five different problem-solving styles and found that explanations that matched a software engineer’s style were more often beneficial, although not every matched adaptation helped (Anderson et al., 2025). Across 1,072 explanations from six LLMs, we uncovered 13 distinct ways in which the models adapted their language, forming a taxonomy of ways AI coding assistants could personalize their explanations to users as individuals (Anderson et al., 2026).

Code explanations adapted to information processing style

Explanations of the same COBOL program from an LLM, both unadapted and prompted for selective or comprehensive information processing styles (Anderson et al., 2025).

References

2026

arXiv

AgentSCOPE: Evaluating Contextual Privacy Across Agentic Workflows

Ivoline C. Ngong, Keerthiram Murugesan, Swanand Kadhe, Justin D. Weisz, Amit Dhurandhar, and Karthikeyan Natesan Ramamurthy

CoRR, 2026

arXiv Bib PDF

@article{abs-2603-04902,
  author = {Ngong, Ivoline C. and Murugesan, Keerthiram and Kadhe, Swanand and Weisz, Justin D. and Dhurandhar, Amit and Ramamurthy, Karthikeyan Natesan},
  title = {AgentSCOPE: Evaluating Contextual Privacy Across Agentic Workflows},
  journal = {CoRR},
  volume = {abs/2603.04902},
  year = {2026},
  url = {https://doi.org/10.48550/arXiv.2603.04902},
}

IJCAI

Wavelength.AI: Extending the Collaborative Game Wavelength as a Testbed for Studying Shared Understanding in Human-Agent Collaboration

Katelyn Morrison, Gabriel Enrique Gonzalez, Zahra Ashktorab, Matthew Riemer, Andrew Anderson, Djallel Bouneffouf, and 1 more author

In Proceedings of the Thirty-Fifth International Joint Conference on Artificial Intelligence and the 29th European Conference on Artificial Intelligence, IJCAI/ECAI 2026, Bremen, Germany, 15-21 August 2026, 2026

DOI Bib PDF

@inproceedings{MorrisonGARABW26,
  author = {Morrison, Katelyn and Gonzalez, Gabriel Enrique and Ashktorab, Zahra and Riemer, Matthew and Anderson, Andrew and Bouneffouf, Djallel and Weisz, Justin D.},
  title = {Wavelength.AI: Extending the Collaborative Game Wavelength as a Testbed for Studying Shared Understanding in Human-Agent Collaboration},
  booktitle = {Proceedings of the Thirty-Fifth International Joint Conference on Artificial Intelligence and the 29th European Conference on Artificial Intelligence, {IJCAI/ECAI} 2026, Bremen, Germany, 15-21 August 2026},
  pages = {8466--8470},
  publisher = {ijcai.org},
  year = {2026},
  url = {https://doi.org/10.24963/ijcai.2026/981},
  doi = {10.24963/IJCAI.2026/981},
}

CHI EA

Human-Centered Explainable AI (HCXAI): Re-examining XAI in the Era of Agentic AI

Upol Ehsan, Amal Alabdulkarim, Kenneth Holstein, Min Kyung Lee, Andreas Riener, and Justin D. Weisz

In Proceedings of the Extended Abstracts of the 2026 CHI Conference on Human Factors in Computing Systems, CHI EA 2026, Barcelona, Spain, April 13-17, 2026, 2026

DOI Bib PDF

@inproceedings{EhsanAHLRW26,
  author = {Ehsan, Upol and Alabdulkarim, Amal and Holstein, Kenneth and Lee, Min Kyung and Riener, Andreas and Weisz, Justin D.},
  title = {Human-Centered Explainable {AI} {(HCXAI):} Re-examining {XAI} in the Era of Agentic {AI}},
  booktitle = {Proceedings of the Extended Abstracts of the 2026 {CHI} Conference on Human Factors in Computing Systems, {CHI} {EA} 2026, Barcelona, Spain, April 13-17, 2026},
  pages = {955:1--955:6},
  publisher = {{ACM}},
  year = {2026},
  url = {https://doi.org/10.1145/3772363.3778728},
  doi = {10.1145/3772363.3778728},
}

FAccT

Overcoming Barriers to AI Disclosure: The Role of Granularity and Sociocultural Enablers in Reducing Perceived Stigmas

Jessica He, Avinash Akella, Hyo Jin Do, and Justin D. Weisz

In Proceedings of the 2026 ACM Conference on Fairness, Accountability, and Transparency, FAccT 2026, Montreal, QC, Canada, June 25-28, 2026, 2026

DOI Bib PDF

@inproceedings{HeADW26,
  author = {He, Jessica and Akella, Avinash and Do, Hyo Jin and Weisz, Justin D.},
  title = {Overcoming Barriers to {AI} Disclosure: The Role of Granularity and Sociocultural Enablers in Reducing Perceived Stigmas},
  booktitle = {Proceedings of the 2026 {ACM} Conference on Fairness, Accountability, and Transparency, FAccT 2026, Montreal, QC, Canada, June 25-28, 2026},
  pages = {2117--2143},
  publisher = {{ACM}},
  year = {2026},
  url = {https://doi.org/10.1145/3805689.3806737},
  doi = {10.1145/3805689.3806737},
}

AIES

Can LLM Code Explanations Adapt to Diverse Problem-Solvers’ Needs?

Andrew Anderson, David Piorkowski, Justin D. Weisz, Margaret Burnett, and Kush R. Varshney

In Proceedings of the Ninth AAAI/ACM Conference on AI, Ethics, and Society, AIES 2026, 2026

DOI arXiv Bib PDF

@inproceedings{AndersonPWBV26,
  author = {Anderson, Andrew and Piorkowski, David and Weisz, Justin D. and Burnett, Margaret and Varshney, Kush R.},
  title = {Can {LLM} Code Explanations Adapt to Diverse Problem-Solvers' Needs?},
  booktitle = {Proceedings of the Ninth {AAAI/ACM} Conference on AI, Ethics, and Society, {AIES} 2026},
  volume = {9},
  pages = {192--205},
  publisher = {{AAAI} Press},
  year = {2026},
  url = {https://doi.org/10.1609/aies.v9i1.43133},
  doi = {10.1609/aies.v9i1.43133},
}

2025

IUI WS

ACE: Moving toward Co-Investigation with the Agentic Code Explorer

Darío Andrés Silva Moran, Kristina Brimijoin, Gabriel Enrique Gonzalez, Stephanie Houde, Michael J. Muller, Michelle Brachman, and 1 more author

In Joint Proceedings of the ACM IUI 2025 Workshops co-located with the 30th Annual ACM Conference on Intelligent User Interfaces (IUI 2025), Cagliari, Italy, March 24th, 2025, 2025

Bib PDF

@inproceedings{MoranBGHMBW25,
  author = {Silva Moran, Darío Andrés and Brimijoin, Kristina and Gonzalez, Gabriel Enrique and Houde, Stephanie and Muller, Michael J. and Brachman, Michelle and Weisz, Justin D.},
  title = {{ACE:} Moving toward Co-Investigation with the Agentic Code Explorer},
  booktitle = {Joint Proceedings of the {ACM} {IUI} 2025 Workshops co-located with the 30th Annual {ACM} Conference on Intelligent User Interfaces {(IUI} 2025), Cagliari, Italy, March 24th, 2025},
  series = {{CEUR} Workshop Proceedings},
  volume = {3957},
  pages = {211--224},
  publisher = {CEUR-WS.org},
  year = {2025},
  url = {https://ceur-ws.org/Vol-3957/HAI-GEN-paper03.pdf},
}

CHI EA

Examining the Use and Impact of an AI Code Assistant on Developer Productivity and Experience in the Enterprise

Justin D. Weisz, Shraddha Vijay Kumar, Michael J. Muller, Karen-Ellen Browne, Arielle Goldberg, Katrin Ellice Heintze, and 1 more author

In Proceedings of the Extended Abstracts of the CHI Conference on Human Factors in Computing Systems, CHI EA 2025, Yokohama, Japan, 26 April 2025- 1 May 2025, 2025

DOI arXiv Bib PDF

@inproceedings{WeiszKMBGHB25,
  author = {Weisz, Justin D. and Kumar, Shraddha Vijay and Muller, Michael J. and Browne, Karen{-}Ellen and Goldberg, Arielle and Heintze, Katrin Ellice and Bajpai, Shagun},
  title = {Examining the Use and Impact of an {AI} Code Assistant on Developer Productivity and Experience in the Enterprise},
  booktitle = {Proceedings of the Extended Abstracts of the {CHI} Conference on Human Factors in Computing Systems, {CHI} {EA} 2025, Yokohama, Japan, 26 April 2025- 1 May 2025},
  pages = {673:1--673:13},
  publisher = {{ACM}},
  year = {2025},
  url = {https://doi.org/10.1145/3706599.3706670},
  doi = {10.1145/3706599.3706670},
}

arXiv

Story Arena: A Multi-Agent Environment for Envisioning the Future of Software Engineering

Justin D. Weisz, Michael J. Muller, and Kush R. Varshney

CoRR, 2025

arXiv Bib PDF

@article{abs-2511-05410,
  author = {Weisz, Justin D. and Muller, Michael J. and Varshney, Kush R.},
  title = {Story Arena: {A} Multi-Agent Environment for Envisioning the Future of Software Engineering},
  journal = {CoRR},
  volume = {abs/2511.05410},
  year = {2025},
  url = {https://doi.org/10.48550/arXiv.2511.05410},
}

CHIWORK

Exploration and Optimization of Generative Variability in Future Work: A Mixed-Initiative Analysis

Michael J. Muller, Jessica He, and Justin D. Weisz

In Proceedings of the 4th Annual Symposium on Human-Computer Interaction for Work, CHIWORK 2025, Amsterdam, Netherlands, June 23-25, 2025, 2025

DOI Bib PDF

@inproceedings{MullerHW25,
  author = {Muller, Michael J. and He, Jessica and Weisz, Justin D.},
  title = {Exploration and Optimization of Generative Variability in Future Work: {A} Mixed-Initiative Analysis},
  booktitle = {Proceedings of the 4th Annual Symposium on Human-Computer Interaction for Work, {CHIWORK} 2025, Amsterdam, Netherlands, June 23-25, 2025},
  pages = {26:1--26:13},
  publisher = {{ACM}},
  year = {2025},
  url = {https://doi.org/10.1145/3729176.3729195},
  doi = {10.1145/3729176.3729195},
}

IUI

Controlling AI Agent Participation in Group Conversations: A Human-Centered Approach

Stephanie Houde, Kristina Brimijoin, Michael J. Muller, Steven I. Ross, Darío Andrés Silva Moran, Gabriel Enrique Gonzalez, and 3 more authors

In Proceedings of the 30th International Conference on Intelligent User Interfaces, IUI 2025, Cagliari, Italy, March 24-27, 2025, 2025

DOI arXiv Bib PDF

@inproceedings{HoudeBMRMGKFW25,
  author = {Houde, Stephanie and Brimijoin, Kristina and Muller, Michael J. and Ross, Steven I. and Silva Moran, Darío Andrés and Gonzalez, Gabriel Enrique and Kunde, Siya and Foreman, Morgan A. and Weisz, Justin D.},
  title = {Controlling {AI} Agent Participation in Group Conversations: {A} Human-Centered Approach},
  booktitle = {Proceedings of the 30th International Conference on Intelligent User Interfaces, {IUI} 2025, Cagliari, Italy, March 24-27, 2025},
  pages = {390--408},
  publisher = {{ACM}},
  year = {2025},
  url = {https://doi.org/10.1145/3708359.3712089},
  doi = {10.1145/3708359.3712089},
}

AIES

Highlight All the Phrases: Enhancing LLM Transparency Through Visual Factuality Indicators

Hyo Jin Do, Rachel Ostrand, Werner Geyer, Keerthiram Murugesan, Dennis Wei, and Justin D. Weisz

In Proceedings of the Eighth AAAI/ACM Conference on AI, Ethics, and Society, AIES 2025, Madrid, Spain, October 20-22, 2025, Main Track I, 2025

DOI arXiv Bib PDF

@inproceedings{DoOGMWW25,
  author = {Do, Hyo Jin and Ostrand, Rachel and Geyer, Werner and Murugesan, Keerthiram and Wei, Dennis and Weisz, Justin D.},
  title = {Highlight All the Phrases: Enhancing {LLM} Transparency Through Visual Factuality Indicators},
  booktitle = {Proceedings of the Eighth {AAAI/ACM} Conference on AI, Ethics, and Society, {AIES} 2025, Madrid, Spain, October 20-22, 2025, Main Track {I}},
  pages = {772--784},
  publisher = {{AAAI} Press},
  year = {2025},
  url = {https://doi.org/10.1609/aies.v8i1.36588},
  doi = {10.1609/AIES.V8I1.36588},
}

ACL

Protecting Users From Themselves: Safeguarding Contextual Privacy in Interactions with Conversational Agents

Ivoline C. Ngong, Swanand Ravindra Kadhe, Hao Wang, Keerthiram Murugesan, Justin D. Weisz, Amit Dhurandhar, and 1 more author

In Findings of the Association for Computational Linguistics, ACL 2025, Vienna, Austria, July 27 - August 1, 2025, 2025

DOI arXiv Bib PDF

@inproceedings{NgongK0MWDR25,
  author = {Ngong, Ivoline C. and Kadhe, Swanand Ravindra and Wang, Hao and Murugesan, Keerthiram and Weisz, Justin D. and Dhurandhar, Amit and Ramamurthy, Karthikeyan Natesan},
  title = {Protecting Users From Themselves: Safeguarding Contextual Privacy in Interactions with Conversational Agents},
  booktitle = {Findings of the Association for Computational Linguistics, {ACL} 2025, Vienna, Austria, July 27 - August 1, 2025},
  series = {Findings of {ACL}},
  volume = {{ACL} 2025},
  pages = {26196--26220},
  publisher = {Association for Computational Linguistics},
  year = {2025},
  url = {https://doi.org/10.18653/v1/2025.findings-acl.1343},
  doi = {10.18653/V1/2025.FINDINGS-ACL.1343},
}

ICML

Position: Theory of Mind Benchmarks are Broken for Large Language Models

Matthew Riemer, Zahra Ashktorab, Djallel Bouneffouf, Payel Das, Miao Liu, Justin D. Weisz, and 1 more author

In Forty-second International Conference on Machine Learning, ICML 2025, Vancouver, BC, Canada, July 13-19, 2025 - Position Paper Track, 2025

arXiv Bib PDF

@inproceedings{RiemerA0D0WC25,
  author = {Riemer, Matthew and Ashktorab, Zahra and Bouneffouf, Djallel and Das, Payel and Liu, Miao and Weisz, Justin D. and Campbell, Murray},
  title = {Position: Theory of Mind Benchmarks are Broken for Large Language Models},
  booktitle = {Forty-second International Conference on Machine Learning, {ICML} 2025, Vancouver, BC, Canada, July 13-19, 2025 - Position Paper Track},
  publisher = {OpenReview.net},
  year = {2025},
  url = {https://proceedings.mlr.press/v267/riemer25a.html},
}

AAAI WS

Establishing the Cooperative Game Wavelength as a Testbed to Explore Mutual Theory of Mind

Katelyn Morrison, Zahra Ashktorab, Djallel Bouneffouf, Gabriel Enrique Gonzalez, and Justin D. Weisz

In Proceedings of the International Workshop on Advancing AI Through Theory of Mind (ToM4AI), held in conjunction with AAAI 2025, Philadelphia, PA, USA, 2025

Bib PDF

@inproceedings{MorrisonABGW25tom4ai,
  author = {Morrison, Katelyn and Ashktorab, Zahra and Bouneffouf, Djallel and Gonzalez, Gabriel Enrique and Weisz, Justin D.},
  title = {Establishing the Cooperative Game Wavelength as a Testbed to Explore Mutual Theory of Mind},
  booktitle = {Proceedings of the International Workshop on Advancing AI Through Theory of Mind (ToM4AI), held in conjunction with AAAI 2025, Philadelphia, PA, USA},
  year = {2025},
}

Book

Human-Centered AI Design Principles for Generative AI Applications

Justin D. Weisz, Jessica He, and Michael Muller

In Handbook of Human-Centered Artificial Intelligence, 2025

DOI Bib

@incollection{WeiszHM25handbook,
  author = {Weisz, Justin D. and He, Jessica and Muller, Michael},
  title = {Human-Centered {AI} Design Principles for Generative {AI} Applications},
  booktitle = {Handbook of Human-Centered Artificial Intelligence},
  editor = {Xu, Wei},
  publisher = {Springer, Singapore},
  year = {2025},
  doi = {10.1007/978-981-97-8440-0_74-1},
}

CHI EA

New Frontiers of Human-centered Explainable AI (HCXAI): Participatory Civic AI, Benchmarking LLMs, XAI Hallucinations, and Responsible AI Audits

Upol Ehsan, Elizabeth Anne Watkins, Philipp Wintersberger, Carina Manger, Nina C. Hubig, Saiph Savage, and 2 more authors

In Proceedings of the Extended Abstracts of the CHI Conference on Human Factors in Computing Systems, CHI EA 2025, Yokohama, Japan, 26 April 2025- 1 May 2025, 2025

DOI Bib PDF

@inproceedings{EhsanWWMHSWR25,
  author = {Ehsan, Upol and Watkins, Elizabeth Anne and Wintersberger, Philipp and Manger, Carina and Hubig, Nina C. and Savage, Saiph and Weisz, Justin D. and Riener, Andreas},
  title = {New Frontiers of Human-centered Explainable {AI} {(HCXAI):} Participatory Civic AI, Benchmarking LLMs, {XAI} Hallucinations, and Responsible {AI} Audits},
  booktitle = {Proceedings of the Extended Abstracts of the {CHI} Conference on Human Factors in Computing Systems, {CHI} {EA} 2025, Yokohama, Japan, 26 April 2025- 1 May 2025},
  pages = {793:1--793:6},
  publisher = {{ACM}},
  year = {2025},
  url = {https://doi.org/10.1145/3706599.3706713},
  doi = {10.1145/3706599.3706713},
}

CHI

Which Contributions Deserve Credit? Perceptions of Attribution in Human-AI Co-Creation

Jessica He, Stephanie Houde, and Justin D. Weisz

In Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems, CHI 2025, YokohamaJapan, 26 April 2025- 1 May 2025, 2025

DOI arXiv Bib PDF

@inproceedings{HeHW25,
  author = {He, Jessica and Houde, Stephanie and Weisz, Justin D.},
  title = {Which Contributions Deserve Credit? Perceptions of Attribution in Human-AI Co-Creation},
  booktitle = {Proceedings of the 2025 {CHI} Conference on Human Factors in Computing Systems, {CHI} 2025, YokohamaJapan, 26 April 2025- 1 May 2025},
  pages = {540:1--540:18},
  publisher = {{ACM}},
  year = {2025},
  url = {https://doi.org/10.1145/3706598.3713522},
  doi = {10.1145/3706598.3713522},
}

UMAP

Towards Personalized and Contextualized Code Explanations

Michelle Brachman, Arielle Goldberg, Andrew Anderson, Stephanie Houde, Michael J. Muller, and Justin D. Weisz

In Adjunct Proceedings of the 33rd ACM Conference on User Modeling, Adaptation and Personalization, UMAP Adjunct 2025, New York City, NY, USA, June 16-19, 2025, 2025

DOI Bib PDF

@inproceedings{BrachmanG0HMW25,
  author = {Brachman, Michelle and Goldberg, Arielle and Anderson, Andrew and Houde, Stephanie and Muller, Michael J. and Weisz, Justin D.},
  title = {Towards Personalized and Contextualized Code Explanations},
  booktitle = {Adjunct Proceedings of the 33rd {ACM} Conference on User Modeling, Adaptation and Personalization, {UMAP} Adjunct 2025, New York City, NY, USA, June 16-19, 2025},
  pages = {120--125},
  publisher = {{ACM}},
  year = {2025},
  url = {https://doi.org/10.1145/3708319.3733681},
  doi = {10.1145/3708319.3733681},
}

arXiv

An LLM’s Attempts to Adapt to Diverse Software Engineers’ Problem-Solving Styles: More Inclusive & Equitable?

Andrew Anderson, David Piorkowski, Margaret Burnett, and Justin D. Weisz

CoRR, 2025

arXiv Bib PDF

@article{abs-2503-11018,
  author = {Anderson, Andrew and Piorkowski, David and Burnett, Margaret and Weisz, Justin D.},
  title = {An LLM's Attempts to Adapt to Diverse Software Engineers' Problem-Solving Styles: More Inclusive {\&} Equitable?},
  journal = {CoRR},
  volume = {abs/2503.11018},
  year = {2025},
  url = {https://doi.org/10.48550/arXiv.2503.11018},
}

2024

CHIWORK

Drinking Chai with Your (AI) Programming Partner: Value Tensions in the Tokenization of Future Human-AI Collaborative Work

Michael J. Muller, Justin D. Weisz, Stephanie Houde, and Steven I. Ross

In Proceedings of the 3rd Annual Meeting of the Symposium on Human-Computer Interaction for Work, CHIWORK 2024, Newcastle upon Tyne, United Kingdom, June 25-27, 2024, 2024

DOI Bib PDF

@inproceedings{MullerWHR24,
  author = {Muller, Michael J. and Weisz, Justin D. and Houde, Stephanie and Ross, Steven I.},
  title = {Drinking Chai with Your {(AI)} Programming Partner: Value Tensions in the Tokenization of Future Human-AI Collaborative Work},
  booktitle = {Proceedings of the 3rd Annual Meeting of the Symposium on Human-Computer Interaction for Work, {CHIWORK} 2024, Newcastle upon Tyne, United Kingdom, June 25-27, 2024},
  pages = {7:1--7:15},
  publisher = {{ACM}},
  year = {2024},
  url = {https://doi.org/10.1145/3663384.3663390},
  doi = {10.1145/3663384.3663390},
}

ICCC

Group Brainstorming with an AI Agent: Creating and Selecting Ideas

Michael J. Muller, Stephanie Houde, Gabriel Enrique Gonzalez, Kristina Brimijoin, Steven I. Ross, Darío Andrés Silva Moran, and 1 more author

In Proceedings of the 15th International Conference on Computational Creativity, ICCC 2024, Jönköping, Sweden, June 17-21, 2024, 2024

Bib PDF

@inproceedings{MullerHGBRMW24,
  author = {Muller, Michael J. and Houde, Stephanie and Gonzalez, Gabriel Enrique and Brimijoin, Kristina and Ross, Steven I. and Silva Moran, Darío Andrés and Weisz, Justin D.},
  title = {Group Brainstorming with an {AI} Agent: Creating and Selecting Ideas},
  booktitle = {Proceedings of the 15th International Conference on Computational Creativity, {ICCC} 2024, J{\"{o}}nk{\"{o}}ping, Sweden, June 17-21, 2024},
  pages = {156--165},
  publisher = {Association for Computational Creativity {(ACC)}},
  year = {2024},
  url = {https://computationalcreativity.net/iccc24/papers/ICCC24\_paper\_18.pdf},
}

CHIWORK

AI and the Future of Collaborative Work: Group Ideation with an LLM in a Virtual Canvas

Jessica He, Stephanie Houde, Gabriel Enrique Gonzalez, Darío Andrés Silva Moran, Steven I. Ross, Michael J. Muller, and 1 more author

In Proceedings of the 3rd Annual Meeting of the Symposium on Human-Computer Interaction for Work, CHIWORK 2024, Newcastle upon Tyne, United Kingdom, June 25-27, 2024, 2024

Honorable Mention DOI Bib PDF

Best Paper Honorable Mention, CHIWORK 2024.

@inproceedings{HeHGMRMW24,
  author = {He, Jessica and Houde, Stephanie and Gonzalez, Gabriel Enrique and Silva Moran, Darío Andrés and Ross, Steven I. and Muller, Michael J. and Weisz, Justin D.},
  title = {{AI} and the Future of Collaborative Work: Group Ideation with an {LLM} in a Virtual Canvas},
  booktitle = {Proceedings of the 3rd Annual Meeting of the Symposium on Human-Computer Interaction for Work, {CHIWORK} 2024, Newcastle upon Tyne, United Kingdom, June 25-27, 2024},
  pages = {9:1--9:14},
  publisher = {{ACM}},
  year = {2024},
  url = {https://doi.org/10.1145/3663384.3663398},
  doi = {10.1145/3663384.3663398},
}

IUI WS

Collaborative Canvas: A Tool for Exploring LLM Use in Group Ideation Tasks

Gabriel Enrique Gonzalez, Darío Andrés Silva Moran, Stephanie Houde, Jessica He, Steven I. Ross, Michael J. Muller, and 2 more authors

In Joint Proceedings of the ACM IUI 2024 Workshops co-located with the 29th Annual ACM Conference on Intelligent User Interfaces (IUI 2024), Greenville, South Carolina, USA, March 18, 2024, 2024

Bib PDF

@inproceedings{GonzalezMHHRMKW24,
  author = {Gonzalez, Gabriel Enrique and Silva Moran, Darío Andrés and Houde, Stephanie and He, Jessica and Ross, Steven I. and Muller, Michael J. and Kunde, Siya and Weisz, Justin D.},
  title = {Collaborative Canvas: {A} Tool for Exploring {LLM} Use in Group Ideation Tasks},
  booktitle = {Joint Proceedings of the {ACM} {IUI} 2024 Workshops co-located with the 29th Annual {ACM} Conference on Intelligent User Interfaces {(IUI} 2024), Greenville, South Carolina, USA, March 18, 2024},
  series = {{CEUR} Workshop Proceedings},
  volume = {3660},
  publisher = {CEUR-WS.org},
  year = {2024},
  url = {https://ceur-ws.org/Vol-3660/paper4.pdf},
}

arXiv

Facilitating Human-LLM Collaboration through Factuality Scores and Source Attributions

Hyo Jin Do, Rachel Ostrand, Justin D. Weisz, Casey Dugan, Prasanna Sattigeri, Dennis Wei, and 2 more authors

CoRR, 2024

arXiv Bib PDF

@article{abs-2405-20434,
  author = {Do, Hyo Jin and Ostrand, Rachel and Weisz, Justin D. and Dugan, Casey and Sattigeri, Prasanna and Wei, Dennis and Murugesan, Keerthiram and Geyer, Werner},
  title = {Facilitating Human-LLM Collaboration through Factuality Scores and Source Attributions},
  journal = {CoRR},
  volume = {abs/2405.20434},
  year = {2024},
  url = {https://doi.org/10.48550/arXiv.2405.20434},
}

arXiv

Expedient Assistance and Consequential Misunderstanding: Envisioning an Operationalized Mutual Theory of Mind

Justin D. Weisz, Michael J. Muller, Arielle Goldberg, and Darío Andrés Silva Moran

CoRR, 2024

arXiv Bib PDF

@article{abs-2406-11946,
  author = {Weisz, Justin D. and Muller, Michael J. and Goldberg, Arielle and Silva Moran, Darío Andrés},
  title = {Expedient Assistance and Consequential Misunderstanding: Envisioning an Operationalized Mutual Theory of Mind},
  journal = {CoRR},
  volume = {abs/2406.11946},
  year = {2024},
  url = {https://doi.org/10.48550/arXiv.2406.11946},
}

CHI

Design Principles for Generative AI Applications

Justin D. Weisz, Jessica He, Michael J. Muller, Gabriela Hoefer, Rachel Miles, and Werner Geyer

In Proceedings of the CHI Conference on Human Factors in Computing Systems, CHI 2024, Honolulu, HI, USA, May 11-16, 2024, 2024

DOI arXiv Bib PDF

@inproceedings{WeiszHMHMG24,
  author = {Weisz, Justin D. and He, Jessica and Muller, Michael J. and Hoefer, Gabriela and Miles, Rachel and Geyer, Werner},
  title = {Design Principles for Generative {AI} Applications},
  booktitle = {Proceedings of the {CHI} Conference on Human Factors in Computing Systems, {CHI} 2024, Honolulu, HI, USA, May 11-16, 2024},
  pages = {378:1--378:22},
  publisher = {{ACM}},
  year = {2024},
  url = {https://doi.org/10.1145/3613904.3642466},
  doi = {10.1145/3613904.3642466},
}

2023

IUI

The Programmer’s Assistant: Conversational Interaction with a Large Language Model for Software Development

Steven I. Ross, Fernando Martinez, Stephanie Houde, Michael J. Muller, and Justin D. Weisz

In Proceedings of the 28th International Conference on Intelligent User Interfaces, IUI 2023, Sydney, NSW, Australia, March 27-31, 2023, 2023

DOI arXiv Bib PDF

@inproceedings{RossMHMW23,
  author = {Ross, Steven I. and Martinez, Fernando and Houde, Stephanie and Muller, Michael J. and Weisz, Justin D.},
  title = {The Programmer's Assistant: Conversational Interaction with a Large Language Model for Software Development},
  booktitle = {Proceedings of the 28th International Conference on Intelligent User Interfaces, {IUI} 2023, Sydney, NSW, Australia, March 27-31, 2023},
  pages = {491--514},
  publisher = {{ACM}},
  year = {2023},
  url = {https://doi.org/10.1145/3581641.3584037},
  doi = {10.1145/3581641.3584037},
}

IUI Comp.

The Programmer’s Assistant User Experience

Steven I. Ross, Stephanie Houde, Fernando Martinez, Michael J. Muller, and Justin D. Weisz

In Companion Proceedings of the 28th International Conference on Intelligent User Interfaces, IUI 2023, Sydney, NSW, Australia, March 27-31, 2023, 2023

Best Demo HM DOI Bib PDF

Best Demo Honorable Mention, IUI 2023.

@inproceedings{RossHMMW23,
  author = {Ross, Steven I. and Houde, Stephanie and Martinez, Fernando and Muller, Michael J. and Weisz, Justin D.},
  title = {The Programmer's Assistant User Experience},
  booktitle = {Companion Proceedings of the 28th International Conference on Intelligent User Interfaces, {IUI} 2023, Sydney, NSW, Australia, March 27-31, 2023},
  pages = {102--104},
  publisher = {{ACM}},
  year = {2023},
  url = {https://doi.org/10.1145/3581754.3584143},
  doi = {10.1145/3581754.3584143},
}

IUI WS

A Case Study in Engineering a Conversational Programming Assistant’s Persona

Steven I. Ross, Michael J. Muller, Fernando Martinez, Stephanie Houde, and Justin D. Weisz

In Joint Proceedings of the IUI 2023 Workshops: HAI-GEN, ITAH, MILC, SHAI, SketchRec, SOCIALIZE co-located with the ACM International Conference on Intelligent User Interfaces (IUI 2023), Sydney, Australia, March 27-31, 2023, 2023

arXiv Bib PDF

@inproceedings{RossMMHW23,
  author = {Ross, Steven I. and Muller, Michael J. and Martinez, Fernando and Houde, Stephanie and Weisz, Justin D.},
  title = {A Case Study in Engineering a Conversational Programming Assistant's Persona},
  booktitle = {Joint Proceedings of the {IUI} 2023 Workshops: HAI-GEN, ITAH, MILC, SHAI, SketchRec, {SOCIALIZE} co-located with the {ACM} International Conference on Intelligent User Interfaces {(IUI} 2023), Sydney, Australia, March 27-31, 2023},
  series = {{CEUR} Workshop Proceedings},
  volume = {3359},
  pages = {121--129},
  publisher = {CEUR-WS.org},
  year = {2023},
  url = {https://ceur-ws.org/Vol-3359/paper13.pdf},
}

ICCC

Interactional Co-Creativity of Human and AI in Analogy-Based Design

Michael J. Muller, Heloisa Candello, and Justin D. Weisz

In Proceedings of the 14th International Conference on Computational Creativity, ICCC 2023, Ontario, Canada, June 19-23, 2023, 2023

Bib PDF

@inproceedings{MullerCW23,
  author = {Muller, Michael J. and Candello, Heloisa and Weisz, Justin D.},
  title = {Interactional Co-Creativity of Human and {AI} in Analogy-Based Design},
  booktitle = {Proceedings of the 14th International Conference on Computational Creativity, {ICCC} 2023, Ontario, Canada, June 19-23, 2023},
  pages = {112--116},
  publisher = {Association for Computational Creativity {(ACC)}},
  year = {2023},
  url = {https://computationalcreativity.net/iccc23/papers/ICCC-2023\_paper\_106.pdf},
}

IUI WS

Toward General Design Principles for Generative AI Applications

Justin D. Weisz, Michael J. Muller, Jessica He, and Stephanie Houde

arXiv Bib PDF

@inproceedings{WeiszMHH23,
  author = {Weisz, Justin D. and Muller, Michael J. and He, Jessica and Houde, Stephanie},
  title = {Toward General Design Principles for Generative {AI} Applications},
  booktitle = {Joint Proceedings of the {IUI} 2023 Workshops: HAI-GEN, ITAH, MILC, SHAI, SketchRec, {SOCIALIZE} co-located with the {ACM} International Conference on Intelligent User Interfaces {(IUI} 2023), Sydney, Australia, March 27-31, 2023},
  series = {{CEUR} Workshop Proceedings},
  volume = {3359},
  pages = {130--144},
  publisher = {CEUR-WS.org},
  year = {2023},
  url = {https://ceur-ws.org/Vol-3359/paper14.pdf},
}

CHI EA

Human-Centered Explainable AI (HCXAI): Coming of Age

Upol Ehsan, Philipp Wintersberger, Elizabeth Anne Watkins, Carina Manger, Gonzalo A. Ramos, Justin D. Weisz, and 3 more authors

In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems, CHI EA 2023, Hamburg, Germany, April 23-28, 2023, 2023

DOI Bib PDF

@inproceedings{EhsanWWMRWDRR23,
  author = {Ehsan, Upol and Wintersberger, Philipp and Watkins, Elizabeth Anne and Manger, Carina and Ramos, Gonzalo A. and Weisz, Justin D. and III, Hal Daum{\'{e}} and Riener, Andreas and Riedl, Mark O.},
  title = {Human-Centered Explainable {AI} {(HCXAI):} Coming of Age},
  booktitle = {Extended Abstracts of the 2023 {CHI} Conference on Human Factors in Computing Systems, {CHI} {EA} 2023, Hamburg, Germany, April 23-28, 2023},
  pages = {353:1--353:7},
  publisher = {{ACM}},
  year = {2023},
  url = {https://doi.org/10.1145/3544549.3573832},
  doi = {10.1145/3544549.3573832},
}

2022

IUI

Better Together? An Evaluation of AI-Supported Code Translation

Justin D. Weisz, Michael J. Muller, Steven I. Ross, Fernando Martinez, Stephanie Houde, Mayank Agarwal, and 2 more authors

In IUI 2022: 27th International Conference on Intelligent User Interfaces, Helsinki, Finland, March 22 - 25, 2022, 2022

Honorable Mention DOI arXiv Bib PDF

IBM Pat Goldberg Memorial Best Paper Honorable Mention, 2022.

@inproceedings{WeiszMRMHATR22,
  author = {Weisz, Justin D. and Muller, Michael J. and Ross, Steven I. and Martinez, Fernando and Houde, Stephanie and Agarwal, Mayank and Talamadupula, Kartik and Richards, John T.},
  title = {Better Together? An Evaluation of AI-Supported Code Translation},
  booktitle = {{IUI} 2022: 27th International Conference on Intelligent User Interfaces, Helsinki, Finland, March 22 - 25, 2022},
  pages = {369--391},
  publisher = {{ACM}},
  year = {2022},
  url = {https://doi.org/10.1145/3490099.3511157},
  doi = {10.1145/3490099.3511157},
}

IUI

Investigating Explainability of Generative AI for Code through Scenario-based Design

Jiao Sun, Q. Vera Liao, Michael J. Muller, Mayank Agarwal, Stephanie Houde, Kartik Talamadupula, and 1 more author

In IUI 2022: 27th International Conference on Intelligent User Interfaces, Helsinki, Finland, March 22 - 25, 2022, 2022

DOI arXiv Bib PDF

@inproceedings{SunLMAHTW22,
  author = {Sun, Jiao and Liao, Q. Vera and Muller, Michael J. and Agarwal, Mayank and Houde, Stephanie and Talamadupula, Kartik and Weisz, Justin D.},
  title = {Investigating Explainability of Generative {AI} for Code through Scenario-based Design},
  booktitle = {{IUI} 2022: 27th International Conference on Intelligent User Interfaces, Helsinki, Finland, March 22 - 25, 2022},
  pages = {212--228},
  publisher = {{ACM}},
  year = {2022},
  url = {https://doi.org/10.1145/3490099.3511119},
  doi = {10.1145/3490099.3511119},
}

IUI WS

Drinking Chai with Your (AI) Programming Partner: A Design Fiction about Generative AI for Software Engineering

Michael J. Muller, Steven I. Ross, Stephanie Houde, Mayank Agarwal, Fernando Martinez, John T. Richards, and 2 more authors

In Joint Proceedings of the IUI 2022 Workshops: APEx-UI, HAI-GEN, HEALTHI, HUMANIZE, TExSS, SOCIALIZE co-located with the ACM International Conference on Intelligent User Interfaces (IUI 2022), Virtual Event, Helsinki, Finland, March 21-22, 2022, 2022

Bib PDF

@inproceedings{MullerRHAMRTW22,
  author = {Muller, Michael J. and Ross, Steven I. and Houde, Stephanie and Agarwal, Mayank and Martinez, Fernando and Richards, John T. and Talamadupula, Kartik and Weisz, Justin D.},
  title = {Drinking Chai with Your {(AI)} Programming Partner: {A} Design Fiction about Generative {AI} for Software Engineering},
  booktitle = {Joint Proceedings of the {IUI} 2022 Workshops: APEx-UI, HAI-GEN, HEALTHI, HUMANIZE, TExSS, {SOCIALIZE} co-located with the {ACM} International Conference on Intelligent User Interfaces {(IUI} 2022), Virtual Event, Helsinki, Finland, March 21-22, 2022},
  series = {{CEUR} Workshop Proceedings},
  volume = {3124},
  pages = {107--122},
  publisher = {CEUR-WS.org},
  year = {2022},
  url = {https://ceur-ws.org/Vol-3124/paper11.pdf},
}

IUI WS

Opportunities for Generative AI in UX Modernization

Stephanie Houde, Steven I. Ross, Michael J. Muller, Mayank Agarwal, Fernando Martinez, John T. Richards, and 2 more authors

Bib PDF

@inproceedings{HoudeRMAMRTW22,
  author = {Houde, Stephanie and Ross, Steven I. and Muller, Michael J. and Agarwal, Mayank and Martinez, Fernando and Richards, John T. and Talamadupula, Kartik and Weisz, Justin D.},
  title = {Opportunities for Generative {AI} in {UX} Modernization},
  booktitle = {Joint Proceedings of the {IUI} 2022 Workshops: APEx-UI, HAI-GEN, HEALTHI, HUMANIZE, TExSS, {SOCIALIZE} co-located with the {ACM} International Conference on Intelligent User Interfaces {(IUI} 2022), Virtual Event, Helsinki, Finland, March 21-22, 2022},
  series = {{CEUR} Workshop Proceedings},
  volume = {3124},
  pages = {81--91},
  publisher = {CEUR-WS.org},
  year = {2022},
  url = {https://ceur-ws.org/Vol-3124/paper8.pdf},
}

TOCHI

Documentation Matters: Human-Centered AI System to Assist Data Science Code Documentation in Computational Notebooks

April Yi Wang, Dakuo Wang, Jaimie Drozdal, Michael J. Muller, Soya Park, Justin D. Weisz, and 3 more authors

ACM Trans. Comput. Hum. Interact., 2022

DOI arXiv Bib PDF

@article{WangWDMPWLWD22,
  author = {Wang, April Yi and Wang, Dakuo and Drozdal, Jaimie and Muller, Michael J. and Park, Soya and Weisz, Justin D. and Liu, Xuye and Wu, Lingfei and Dugan, Casey},
  title = {Documentation Matters: Human-Centered {AI} System to Assist Data Science Code Documentation in Computational Notebooks},
  journal = {{ACM} Trans. Comput. Hum. Interact.},
  volume = {29},
  number = {2},
  pages = {17:1--17:33},
  year = {2022},
  url = {https://doi.org/10.1145/3489465},
  doi = {10.1145/3489465},
}

2021

IUI

Perfection Not Required? Human-AI Partnerships in Code Translation

Justin D. Weisz, Michael J. Muller, Stephanie Houde, John T. Richards, Steven I. Ross, Fernando Martinez, and 2 more authors

In IUI ’21: 26th International Conference on Intelligent User Interfaces, College Station, TX, USA, April 13-17, 2021, 2021

DOI arXiv Bib PDF

@inproceedings{WeiszMHRRMAT21,
  author = {Weisz, Justin D. and Muller, Michael J. and Houde, Stephanie and Richards, John T. and Ross, Steven I. and Martinez, Fernando and Agarwal, Mayank and Talamadupula, Kartik},
  title = {Perfection Not Required? Human-AI Partnerships in Code Translation},
  booktitle = {{IUI} '21: 26th International Conference on Intelligent User Interfaces, College Station, TX, USA, April 13-17, 2021},
  pages = {402--412},
  publisher = {{ACM}},
  year = {2021},
  url = {https://doi.org/10.1145/3397481.3450656},
  doi = {10.1145/3397481.3450656},
}

arXiv

Using Document Similarity Methods to create Parallel Datasets for Code Translation

Mayank Agarwal, Kartik Talamadupula, Fernando Martinez, Stephanie Houde, Michael J. Muller, John T. Richards, and 2 more authors

CoRR, 2021

arXiv Bib PDF

@article{abs-2110-05423,
  author = {Agarwal, Mayank and Talamadupula, Kartik and Martinez, Fernando and Houde, Stephanie and Muller, Michael J. and Richards, John T. and Ross, Steven I. and Weisz, Justin D.},
  title = {Using Document Similarity Methods to create Parallel Datasets for Code Translation},
  journal = {CoRR},
  volume = {abs/2110.05423},
  year = {2021},
  url = {https://doi.org/10.48550/arXiv.2110.05423},
}

IUI

Model LineUpper: Supporting Interactive Model Comparison at Multiple Levels for AutoML

Shweta Narkar, Yunfeng Zhang, Q. Vera Liao, Dakuo Wang, and Justin D. Weisz

In IUI ’21: 26th International Conference on Intelligent User Interfaces, College Station, TX, USA, April 13-17, 2021, 2021

DOI arXiv Bib PDF

@inproceedings{NarkarZLWW21,
  author = {Narkar, Shweta and Zhang, Yunfeng and Liao, Q. Vera and Wang, Dakuo and Weisz, Justin D.},
  title = {Model LineUpper: Supporting Interactive Model Comparison at Multiple Levels for AutoML},
  booktitle = {{IUI} '21: 26th International Conference on Intelligent User Interfaces, College Station, TX, USA, April 13-17, 2021},
  pages = {170--174},
  publisher = {{ACM}},
  year = {2021},
  url = {https://doi.org/10.1145/3397481.3450658},
  doi = {10.1145/3397481.3450658},
}

CHI

AutoDS: Towards Human-Centered Automation of Data Science

Dakuo Wang, Josh Andres, Justin D. Weisz, Erick Oduor, and Casey Dugan

In CHI ’21: CHI Conference on Human Factors in Computing Systems, Virtual Event / Yokohama, Japan, May 8-13, 2021, 2021

DOI arXiv Bib PDF

@inproceedings{WangAWOD21,
  author = {Wang, Dakuo and Andres, Josh and Weisz, Justin D. and Oduor, Erick and Dugan, Casey},
  title = {AutoDS: Towards Human-Centered Automation of Data Science},
  booktitle = {{CHI} '21: {CHI} Conference on Human Factors in Computing Systems, Virtual Event / Yokohama, Japan, May 8-13, 2021},
  pages = {79:1--79:12},
  publisher = {{ACM}},
  year = {2021},
  url = {https://doi.org/10.1145/3411764.3445526},
  doi = {10.1145/3411764.3445526},
}

IUI WS

How Data Scientists Improve Generated Code Documentation in Jupyter Notebooks

Michael J. Muller, April Yi Wang, Steven I. Ross, Justin D. Weisz, Mayank Agarwal, Kartik Talamadupula, and 7 more authors

In Joint Proceedings of the ACM IUI 2021 Workshops co-located with 26th ACM Conference on Intelligent User Interfaces (ACM IUI 2021), College Station, United States, April 13-17, 2021, 2021

Bib PDF

@inproceedings{MullerWRWATHMRD21,
  author = {Muller, Michael J. and Wang, April Yi and Ross, Steven I. and Weisz, Justin D. and Agarwal, Mayank and Talamadupula, Kartik and Houde, Stephanie and Martinez, Fernando and Richards, John T. and Drozdal, Jaimie and Liu, Xuye and Piorkowski, David and Wang, Dakuo},
  title = {How Data Scientists Improve Generated Code Documentation in Jupyter Notebooks},
  booktitle = {Joint Proceedings of the {ACM} {IUI} 2021 Workshops co-located with 26th {ACM} Conference on Intelligent User Interfaces {(ACM} {IUI} 2021), College Station, United States, April 13-17, 2021},
  series = {{CEUR} Workshop Proceedings},
  volume = {2903},
  publisher = {CEUR-WS.org},
  year = {2021},
  url = {https://ceur-ws.org/Vol-2903/IUI21WS-HAIGEN-10.pdf},
}

CHI

Expanding Explainability: Towards Social Transparency in AI systems

Upol Ehsan, Q. Vera Liao, Michael J. Muller, Mark O. Riedl, and Justin D. Weisz

In CHI ’21: CHI Conference on Human Factors in Computing Systems, Virtual Event / Yokohama, Japan, May 8-13, 2021, 2021

Honorable Mention DOI arXiv Bib PDF

Best Paper Honorable Mention, CHI 2021.

@inproceedings{EhsanLMRW21,
  author = {Ehsan, Upol and Liao, Q. Vera and Muller, Michael J. and Riedl, Mark O. and Weisz, Justin D.},
  title = {Expanding Explainability: Towards Social Transparency in {AI} systems},
  booktitle = {{CHI} '21: {CHI} Conference on Human Factors in Computing Systems, Virtual Event / Yokohama, Japan, May 8-13, 2021},
  pages = {82:1--82:19},
  publisher = {{ACM}},
  year = {2021},
  url = {https://doi.org/10.1145/3411764.3445188},
  doi = {10.1145/3411764.3445188},
}

2020

arXiv

Quality Estimation & Interpretability for Code Translation

Mayank Agarwal, Kartik Talamadupula, Stephanie Houde, Fernando Martinez, Michael J. Muller, John T. Richards, and 2 more authors

CoRR, 2020

arXiv Bib PDF

@article{abs-2012-07581,
  author = {Agarwal, Mayank and Talamadupula, Kartik and Houde, Stephanie and Martinez, Fernando and Muller, Michael J. and Richards, John T. and Ross, Steven I. and Weisz, Justin D.},
  title = {Quality Estimation {\&} Interpretability for Code Translation},
  journal = {CoRR},
  volume = {abs/2012.07581},
  year = {2020},
  url = {https://doi.org/10.48550/arXiv.2012.07581},
}

ICCC WS

Mixed Initiative Generative AI Interfaces: An Analytic Framework for Generative AI Applications

Michael Muller, Justin D. Weisz, and Werner Geyer

In Proceedings of the Workshop The Future of Co-Creative Systems: A Workshop on Human-Computer Co-Creativity of the 11th International Conference on Computational Creativity (ICCC 2020), 2020

Bib PDF

@inproceedings{MullerWG20iccc,
  author = {Muller, Michael and Weisz, Justin D. and Geyer, Werner},
  title = {Mixed Initiative Generative {AI} Interfaces: An Analytic Framework for Generative {AI} Applications},
  booktitle = {Proceedings of the Workshop The Future of Co-Creative Systems: A Workshop on Human-Computer Co-Creativity of the 11th International Conference on Computational Creativity (ICCC 2020)},
  year = {2020},
}

IUI WS

Business (mis)Use Cases of Generative AI

Stephanie Houde, Vera Liao, Jacquelyn Martino, Michael J. Muller, David Piorkowski, John T. Richards, and 2 more authors

In Joint Proceedings of the Workshops on Human-AI Co-Creation with Generative Models and User-Aware Conversational Agents co-located with 25th International Conference on Intelligent User Interfaces (IUI 2020), Cagliari, Italy, March 17, 2020, 2020

arXiv Bib PDF

@inproceedings{HoudeLMMPRWZ20,
  author = {Houde, Stephanie and Liao, Vera and Martino, Jacquelyn and Muller, Michael J. and Piorkowski, David and Richards, John T. and Weisz, Justin D. and Zhang, Yunfeng},
  title = {Business (mis)Use Cases of Generative {AI}},
  booktitle = {Joint Proceedings of the Workshops on Human-AI Co-Creation with Generative Models and User-Aware Conversational Agents co-located with 25th International Conference on Intelligent User Interfaces {(IUI} 2020), Cagliari, Italy, March 17, 2020},
  series = {{CEUR} Workshop Proceedings},
  volume = {2848},
  publisher = {CEUR-WS.org},
  year = {2020},
  url = {https://ceur-ws.org/Vol-2848/HAI-GEN-Paper-5.pdf},
}

IUI

Trust in AutoML: exploring information needs for establishing trust in automated machine learning systems

Jaimie Drozdal, Justin D. Weisz, Dakuo Wang, Gaurav Dass, Bingsheng Yao, Changruo Zhao, and 3 more authors

In IUI ’20: 25th International Conference on Intelligent User Interfaces, Cagliari, Italy, March 17-20, 2020, 2020

DOI arXiv Bib PDF

@inproceedings{DrozdalWWDYZMJS20,
  author = {Drozdal, Jaimie and Weisz, Justin D. and Wang, Dakuo and Dass, Gaurav and Yao, Bingsheng and Zhao, Changruo and Muller, Michael J. and Ju, Lin and Su, Hui},
  title = {Trust in AutoML: exploring information needs for establishing trust in automated machine learning systems},
  booktitle = {{IUI} '20: 25th International Conference on Intelligent User Interfaces, Cagliari, Italy, March 17-20, 2020},
  pages = {297--307},
  publisher = {{ACM}},
  year = {2020},
  url = {https://doi.org/10.1145/3377325.3377501},
  doi = {10.1145/3377325.3377501},
}

IUI Comp.

AutoAI: Automating the End-to-End AI Lifecycle with Humans-in-the-Loop

Dakuo Wang, Parikshit Ram, Daniel Karl I. Weidele, Sijia Liu, Michael J. Muller, Justin D. Weisz, and 5 more authors

In IUI ’20: 25th International Conference on Intelligent User Interfaces, Cagliari, Italy, March 17-20, 2020, Companion, 2020

DOI Bib PDF

@inproceedings{WangRW0MWVCTSA20,
  author = {Wang, Dakuo and Ram, Parikshit and Weidele, Daniel Karl I. and Liu, Sijia and Muller, Michael J. and Weisz, Justin D. and Valente, Abel N. and Chaudhary, Arunima and Torres, Dustin Ramsey and Samulowitz, Horst and Amini, Lisa},
  title = {AutoAI: Automating the End-to-End {AI} Lifecycle with Humans-in-the-Loop},
  booktitle = {{IUI} '20: 25th International Conference on Intelligent User Interfaces, Cagliari, Italy, March 17-20, 2020, Companion},
  pages = {77--78},
  publisher = {{ACM}},
  year = {2020},
  url = {https://doi.org/10.1145/3379336.3381474},
  doi = {10.1145/3379336.3381474},
}

KDD WS

IBM AutoAI: Human-in-the-Loop Automated Machine Learning Supports Data Scientists to Build Better Models

Dakuo Wang, Josh Andres, Justin D. Weisz, Erick Oduor, Udayan Khurana, Horst Samulowitz, and 4 more authors

In 1st Workshop on Data Science with Human in the Loop, DaSH@KDD 2020, San Diego, California, USA, August 24, 2020, 2020

Bib PDF

@inproceedings{WangAWOKSCVTD20,
  author = {Wang, Dakuo and Andres, Josh and Weisz, Justin D. and Oduor, Erick and Khurana, Udayan and Samulowitz, Horst and Chaudhary, Arunima and Valente, Abel N. and Torres, Dustin Ramsey and Dugan, Casey},
  title = {{IBM} AutoAI: Human-in-the-Loop Automated Machine Learning Supports Data Scientists to Build Better Models},
  booktitle = {1st Workshop on Data Science with Human in the Loop, DaSH@KDD 2020, San Diego, California, USA, August 24, 2020},
  year = {2020},
  url = {https://drive.google.com/file/d/1RmuGOdwcalfV0YaJh8nTdL0zutF1\_hx2/view},
}

IUI

AutoAIViz: opening the blackbox of automated artificial intelligence with conditional parallel coordinates

Daniel Karl I. Weidele, Justin D. Weisz, Erick Oduor, Michael J. Muller, Josh Andres, Alexander G. Gray, and 1 more author

In IUI ’20: 25th International Conference on Intelligent User Interfaces, Cagliari, Italy, March 17-20, 2020, 2020

DOI arXiv Bib PDF

@inproceedings{WeideleWOMAGW20,
  author = {Weidele, Daniel Karl I. and Weisz, Justin D. and Oduor, Erick and Muller, Michael J. and Andres, Josh and Gray, Alexander G. and Wang, Dakuo},
  title = {AutoAIViz: opening the blackbox of automated artificial intelligence with conditional parallel coordinates},
  booktitle = {{IUI} '20: 25th International Conference on Intelligent User Interfaces, Cagliari, Italy, March 17-20, 2020},
  pages = {308--312},
  publisher = {{ACM}},
  year = {2020},
  url = {https://doi.org/10.1145/3377325.3377538},
  doi = {10.1145/3377325.3377538},
}

KDD WS

The Next Decade of Data Science

Justin D. Weisz and Michael J. Muller

In 1st Workshop on Data Science with Human in the Loop, DaSH@KDD 2020, San Diego, California, USA, August 24, 2020, 2020

Bib PDF

@inproceedings{WeiszM20,
  author = {Weisz, Justin D. and Muller, Michael J.},
  title = {The Next Decade of Data Science},
  booktitle = {1st Workshop on Data Science with Human in the Loop, DaSH@KDD 2020, San Diego, California, USA, August 24, 2020},
  year = {2020},
  url = {https://drive.google.com/file/d/1jS4g4SYCDDj6pUXAVK8kNBI82seEhv7\_/view},
}

KDD WS

Cultivating Human Expertise Through AI-Assisted Data Science

Josh Andres, Christine T. Wolf, Michael J. Muller, Justin D. Weisz, Narendra Nath Joshi, Aabhas Sharma, and 6 more authors

In 1st Workshop on Data Science with Human in the Loop, DaSH@KDD 2020, San Diego, California, USA, August 24, 2020, 2020

Bib PDF

@inproceedings{AndresWMWJSBDAP20,
  author = {Andres, Josh and Wolf, Christine T. and Muller, Michael J. and Weisz, Justin D. and Joshi, Narendra Nath and Sharma, Aabhas and Brimijoin, Krissy and Desmond, Michael and Ashktorab, Zahra and Pan, Qian and Duesterwald, Evelyn and Dugan, Casey},
  title = {Cultivating Human Expertise Through AI-Assisted Data Science},
  booktitle = {1st Workshop on Data Science with Human in the Loop, DaSH@KDD 2020, San Diego, California, USA, August 24, 2020},
  year = {2020},
  url = {https://drive.google.com/file/d/1c2X0RybLpXdqh2XMCN2luUPdDUhaQMRc/view},
}

2019

CSCW

Human-AI Collaboration in Data Science: Exploring Data Scientists’ Perceptions of Automated AI

Dakuo Wang, Justin D. Weisz, Michael J. Muller, Parikshit Ram, Werner Geyer, Casey Dugan, and 3 more authors

Proc. ACM Hum. Comput. Interact., 2019

DOI arXiv Bib PDF

@article{WangWMRGDTSG19,
  author = {Wang, Dakuo and Weisz, Justin D. and Muller, Michael J. and Ram, Parikshit and Geyer, Werner and Dugan, Casey and Tausczik, Yla R. and Samulowitz, Horst and Gray, Alexander G.},
  title = {Human-AI Collaboration in Data Science: Exploring Data Scientists' Perceptions of Automated {AI}},
  journal = {Proc. {ACM} Hum. Comput. Interact.},
  volume = {3},
  number = {{CSCW}},
  pages = {211:1--211:24},
  year = {2019},
  url = {https://doi.org/10.1145/3359313},
  doi = {10.1145/3359313},
}

arXiv

In AI We Trust? Factors That Influence Trustworthiness of AI-infused Decision-Making Processes

Maryam Ashoori and Justin D. Weisz

CoRR, 2019

arXiv Bib PDF

@article{abs-1912-02675,
  author = {Ashoori, Maryam and Weisz, Justin D.},
  title = {In {AI} We Trust? Factors That Influence Trustworthiness of AI-infused Decision-Making Processes},
  journal = {CoRR},
  volume = {abs/1912.02675},
  year = {2019},
  url = {https://doi.org/10.48550/arXiv.1912.02675},
}

2018

CHI EA

Privacy and Fear in the Drone Era: Preserving Privacy Expectations Through Technology

Judith Odili Uchidiuno, Justin Manweiler, and Justin D. Weisz

In Extended Abstracts of the 2018 CHI Conference on Human Factors in Computing Systems, CHI 2018, Montreal, QC, Canada, April 21-26, 2018, 2018

DOI Bib PDF

@inproceedings{UchidiunoMW18,
  author = {Uchidiuno, Judith Odili and Manweiler, Justin and Weisz, Justin D.},
  title = {Privacy and Fear in the Drone Era: Preserving Privacy Expectations Through Technology},
  booktitle = {Extended Abstracts of the 2018 {CHI} Conference on Human Factors in Computing Systems, {CHI} 2018, Montreal, QC, Canada, April 21-26, 2018},
  publisher = {{ACM}},
  year = {2018},
  url = {https://doi.org/10.1145/3170427.3188457},
  doi = {10.1145/3170427.3188457},
}