Publications

No Two Developers Think Alike: How Problem-Solving Styles and Experience Shape Needs in Conversational Interaction with Copilot

Jonan Richards, Bruno Alves de Oliveira, Iury Oliveira, Igor Wiese, Mairieli Wessel

2026 International Conference on Software Maintenance and Evolution (ICSME)

Abstract

Developers highlight the benefits of conversational LLM-based “programming assistants”. However, recent studies demonstrate the variety in individual developers’ needs regarding programming assistants, and challenges encountered by only specific groups of developers. In this study, we explore the role of cognitive diversity in shaping interactions with GitHub Copilot chat. Through a mixed-methods think aloud study with 27 professional developers and students, we characterize 5 distinct “interaction modes” and 10 underlying needs in developers’ interactions, forming a conceptual model. We characterize links between these modes, needs, and developers’ problem-solving styles and experience profiles, showing how cognitive diversity may shape developers’ interactions. We provide insights and recommendations for researchers and practitioners on how to design programming assistants to better account for diverse developer needs.

Personalizing LLM-Based Conversational Programming Assistants

Jonan Richards

2026 Doctoral and Early Career Symposium (DECS) track of the International Conference on Cooperative and Human Aspects of Software Engineering (CHASE)

Preprint PDF

Abstract

Large Language Models (LLMs) have shown much promise in powering a variety of software engineering (SE) tools. Offering natural language as an intuitive interaction mechanism, LLMs have recently been employed as conversational “programming assistants” capable of supporting several SE activities simultaneously. As with any SE tool, it is crucial that these assistants effectively meet developers’ needs. Recent studies have shown addressing this challenge is complicated by the variety in developers’ needs, and the ambiguous and unbounded nature of conversational interaction. This paper discusses our current and future work towards characterizing how diversity in cognition and organizational context impacts developers’ needs, and exploring personalization as a means of improving the inclusivity of LLM-based conversational programming assistants.

Ask, Then Think: Enhancing LLM Performance with Socratic Reasoning

Antonio Della Porta, Jonan Richards, Lucageneroso Cammarota, Stefano Lambiase, Fabio Palomba, Mairieli Wessel

2026 Registered Report (RR) track of the International Conference on Mining Software Repositories (MSR)

Preprint

Abstract

Large Language Models (LLMs) are widely used for code generation, repair, and explanation, yet they still make mistakes when instructions are vague or when their initial assumptions are wrong. Current prompting methods help models show their reasoning but do not encourage them to check whether that reasoning is sound. This confirmatory study introduces MIND, a Socratic-style prompting framework that guides models through self-questions aimed at clarifying the task, identifying missing information, and checking intermediate conclusions. We will evaluate the framework across four benchmarks covering code generation, repair, explanation, and efficiency, comparing it with methods such as Chain-of-Thought, Self-Ask, Self-Refine, and ReAct. We will also run an ablation study to examine which stages of the questioning process contribute most to performance.

Bridging HCI and AI Research for the Evaluation of Conversational SE Assistants

Jonan Richards, Mairieli Wessel

2025 IEEE/ACM International Workshop on Bots in Software Engineering (BotSE)

Paper Preprint PDF

Abstract

As Large Language Models (LLMs) are increasingly adopted in software engineering, recently in the form of conversational assistants, ensuring these technologies align with developers’ needs is essential. The limitations of traditional human-centered methods for evaluating LLM-based tools at scale raise the need for automatic evaluation. In this paper, we advocate combining insights from human-computer interaction (HCI) and artificial intelligence (AI) research to enable human-centered automatic evaluation of LLM-based conversational SE assistants. We identify requirements for such evaluation and challenges down the road, working towards a framework that ensures these assistants are designed and deployed in line with user needs.

What You Need is What You Get: Theory of Mind for an LLM-Based Code Understanding Assistant

Jonan Richards, Mairieli Wessel

2024 IEEE International Conference on Software Maintenance and Evolution (ICSME)

Paper Preprint PDF Code

Abstract

A growing number of tools have used Large Language Models (LLMs) to support developers’ code understanding. However, developers still face several barriers to using such tools, including challenges in describing their intent in natural language, interpreting the tool outcome, and refining an effective prompt to obtain useful information. In this study, we designed an LLM-based conversational assistant that provides a personalized interaction based on inferred user mental state (e.g., background knowledge and experience). We evaluate the approach in a within-subject study with fourteen novices to capture their perceptions and preferences. Our results provide insights for researchers and tool builders who want to create or improve LLM-based conversational assistants to support novices in code understanding.