HCOMP '26: Proceedings of the 2026 ACM Conference on Human-AI Complementarity and Alignment

Full Citation in the ACM Digital Library

Qualification by Calibration: A Readable Benchmark for Admitting Language Models to Human-Computation Tasks

When several language models are wired together into a human-computation pipeline, the system routes, arbitrates, and escalates work according to how confident each model says it is, so a team builder needs a way to screen candidate models the way crowdsourcing has long screened human contributors: with a small, inspectable qualification test. We present such an instrument, a compact battery of yes-or-no questions across everyday domains on which a model reports an answer and a confidence, with every gold label independently audited. Our central contribution, however, is the audited harness and the interface properties it measures, not accuracy discrimination: qualification verdicts for model workers are only as valid as the harness that administers them. Evaluating a diverse panel of contemporary models under three administration protocols, we show that a naive harness, with a fixed token budget and an unaudited parser, manufactures failing workers out of competent ones, misreading a model that answers essentially every item correctly as badly inaccurate, answer-biased, and overconfident. Properly administered, every model in the panel proves admissible on accuracy for these common-knowledge items, and the differentiators that remain, and that transfer to disjoint downstream tasks, are interface properties: calibration of stated confidence, format discipline, and availability under budget. A controlled manipulation further shows the calibration axis is dissociable from accuracy. We release the items, the audited protocol, all per-trial responses under every protocol, and the scoring code, so the check, and the audit of the check, are each a single command.

One AI Signal, Many Human Judgments: A Bayesian Cascade Analysis of AI-based Credibility Indicators in Online Information Spread

Social media platforms increasingly use AI-based credibility indicators to help users judge misinformation. Unlike individual human-AI decision-making, these indicators are embedded in information spread: users see both an AI prediction and earlier judgments shaped by the same AI, and their own judgments may then enter the public history. Yet how to analytically characterize this process remains under-explored. We therefore introduce a social-learning lens for this setting by extending the classical Bayesian cascade model with the AI indicator as a shared public signal. The resulting Gateway condition compares the evidence from the AI prediction with users’ private impressions. Through this view, we show that AI changes what public history means. Crowd agreement may reflect accumulated independent human evidence, or repeated dependence on the same AI prediction. This creates a preservation–correction trade-off: stronger reliance on AI can preserve correct predictions, but can also lock in incorrect ones by blocking corrective private impressions. We calibrate the model using human-subject data on news veracity judgments. Although the AI outperforms human users, the average user weights it below her own impression but above several peer judgments, while individual users vary from discounting the AI to relying on it enough to cascade. Simulations show that over-reliance on a weak AI is especially harmful, and that diversifying AI signals across users can better keep the crowd informative. We conclude with implications for understanding human-AI interaction in information spread and designing misinformation interventions.

DoubleAgents: Human-Agent Alignment in a Socially Embedded Workflow

Aligning agentic AI with user intent is critical for delegating complex, socially embedded tasks, yet user preferences are often implicit, evolving, and difficult to specify upfront. We present DoubleAgents, a system for human–agent alignment in coordination tasks, grounded in distributed cognition. DoubleAgents integrates three components: (1) a coordination agent that maintains state and proposes plans and actions, (2) a dashboard visualization that makes the agent’s reasoning legible for user evaluation, and (3) a policy module that transforms user edits into reusable alignment artifacts, including coordination policies, email templates, and stop hooks, which improve system behavior over time. We evaluate DoubleAgents through a two-day in-lab interactive simulation study (n=10), three real-world deployments, and a technical evaluation. Participants’ comfort in offloading tasks and reliance on DoubleAgents both increased over time, correlating with the three distributed cognition components. Participants still required control at points of uncertainty — edge-case flagging and context-dependent actions. We contribute a distributed cognition approach to human-agent alignment in socially embedded tasks. We further introduce interactive simulation as a methodological testbed for rapid iteration and alignment testing of agentic systems.

Adaptive Selection of Effective AI Assistance in AI-assisted Decision Making Using Multi-Armed Bandits

In AI-assisted decision making, the way that AI assistance is presented significantly influences human cognition, leading to varied reliance strategies. Current human-AI interaction designs often constrain decision makers to interact with specified static forms of AI assistance, rather than providing personalized, intelligent support that adapts to individual behavior and decision contexts. Inspired by the nudge theory in behavioral economics, in this paper, we conceptualize various forms of AI assistance as “choice architectures”. We then introduce a human-adaptive AI interaction framework in AI-assisted decision making, which is based on contextualized multi-armed bandits, to adaptively identify the most effective choice architecture for each individual that can nudge them towards relying on AI more appropriately and making more accurate decisions under different decision contexts. Results from between-subjects experiments across two types of decision making tasks show that our framework promoted more appropriate human reliance on AI compared to other default forms of AI assistance. We conclude by discussing the limitations and design implications of our findings for AI-assisted decision making.

Easy to Read, Easy to Trust: How Processing Fluency in LLM Explanations Drives Over-Reliance in Hate Speech Moderation

This study examines the challenges human moderators face in AI-assisted detection of online hate speech, particularly when discriminatory intent is conveyed through implicit or nuanced language. Following the Judge-Advisor System (JAS) paradigm, we evaluate how different types of Large Language Model (LLM)-generated explanations support human moderators in hate speech detection. We hypothesized that natural language explanations would offer more interpretable rationales, helping moderators better assess the accuracy of AI recommendations and make informed decisions that reflect appropriate reliance on AI. Contrary to this expectation, our results show that explanation form does not affect decision accuracy but significantly reshapes reliance: fluent natural language explanations heighten moderators’ over-reliance on the AI—leading them to follow it even when it errs—whereas feature-based explanations promote scrutiny and self-reliance. We interpret this phenomenon through the lens of processing fluency within the meta-reasoning framework. Drawing on these cognitive science insights and the observed variation in how moderators process different explanation types, we provide practical design suggestions for AI-assisted moderation tools and call for greater cross-disciplinary engagement in future HCI research on human-AI decision-making.

The Belief Update Gate: Separating Inertia from Learning in Human–AI Interaction

Repeated human–AI interaction is often analyzed through pooled belief-updating slopes: users observe AI successes and failures, revise reported beliefs in the feedback-consistent direction, but appear conservative on average. We show that such averages can obscure an important distinction between whether an elicited belief report changes at all and how it changes conditional on movement. We refer to this measurement-aware decomposition as the belief update gate. Reanalyzing a multi-task human–AI decision-making dataset with 240 participants, 7,200 trials, and three task domains, we find substantial non-movement in reported beliefs: 67.3% of trial-level belief changes are exactly zero, and 76.4% are smaller than five percentage points. Separating non-moving from moving reports changes the descriptive interpretation of pooled conservatism: the within-trajectory slope rises from 0.494 overall to 0.949 among rows with nonzero movement. Since this latter estimate conditions on observed movement, we interpret it as a descriptive decomposition rather than as evidence of a near-Bayesian latent learning process. Complementary hurdle style analyses (i.e., modeling zero vs. non-zero changes before predicting update magnitude) show that the absolute discrepancy between feedback and entering belief predicts whether a report changes, while the signed feedback discrepancy predicts the direction and magnitude of change among reports that move. Importantly, observed non-movement does not distinguish genuine latent belief inertia from small unexpressed updates, rounding, or other reporting processes. These findings show that calibration analyses of repeated human–AI interaction should distinguish visible non-movement in elicited belief reports from updating conditional on movement rather than treating reported beliefs as a single continuous updating process.

How AI Assistance Affects Human Skill Development: A Study of Learning with Logic Puzzles

While AI assistance can improve human task performance in the short term, it may also undermine the development of skills in the longer term. We examine this tension in a controlled logic-puzzle experiment involving on-demand AI assistance, where participants complete tasks before, during, and after AI is available. By experimentally varying AI request costs, we find that lower-cost assistance induces more frequent AI use. We also find that participants who request AI assistance during the AI-access phase perform worse at the task after assistance is removed, and their subsequent unassisted performance is overestimated when predicted from earlier AI-assisted performance. We use a Bayesian latent ability model to separate initial ability, post-AI ability, and participant-specific skill change, while estimating how independent reasoning during the AI-access phase relates to skill development. The results show that greater independent problem-solving effort is associated with larger gains in latent ability, consistent with the interpretation that skill development is weaker when AI assistance substitutes for independent reasoning.

Schemex: Discovering Structural Abstractions from Examples

Creative and communicative work is often underpinned by implicit structures, such as the Hero’s Journey in storytelling, design patterns in software, or chord progressions in music. People often learn these structures from examples, a process known as schema induction. However, because schemas are abstract and implicit, they are difficult to discover: shared structural patterns are obscured by surface-level variation, and balancing generality with specificity is challenging. We present Schemex, an interactive AI workflow that systematically supports schema induction by decomposing it into three stages: clustering examples, abstracting candidate schemas, and contrastively refining them by generating new instances and comparing against originals. Studies show that Schemex produces more actionable schemas than a frontier baseline without sacrificing generalizability, with participants uncovering deep and nuanced structural patterns. We also discuss design implications for the cognitive role of interactive process in structure discovery.

FIBB: Capturing False Information Beliefs and Behaviors Across Physical and Online Environments

Humans assess the veracity of newly presented false information using multiple sources: from observations in the online environment (e.g., social media content or responses from AI-mediated systems) to observations in the physical world (e.g., the current state of the world or other people’s behaviors). Belief in false information can lead to adverse societal and physical consequences, including impacts on health and emergency response decisions, increased strain on public resources, and decreased public trust. Currently, false information datasets are limited to examining either online or physical behaviors, making it difficult to evaluate the impact of false claims holistically across environments. Moreover, retrieving beliefs and behaviors across topics at scale through traditional methods, such as user surveys, can be time- and resource-intensive. To address these gaps, we propose a False Information Dataset Containing Beliefs and Behaviors Across Physical and Online Environments (FIBB). FIBB comprises 1,500 LLM-generated scenarios grounded in 300 online false claims in natural disasters (e.g., hurricanes, wildfires, earthquakes) and health (e.g., COVID-19, mental health, chronic diseases). Leveraging cognitive science principles, each FIBB scenario cycles through a person’s veracity assessment and decision-making by depicting their mental states, observations, actions, and the changes in the world resulting from their actions. By bridging the gap between online and physical environments using the use cases of natural disaster and health domains, FIBB aims to inform future downstream goals, such as facilitating the development of world models that can represent people’s likely beliefs and behavioral consequences of false information in tandem.

Measuring Proof Burden in Public Bounty Listings: A RentAHuman Case Study

Online bounty markets let requesters advertise paid tasks to workers. A worker may be asked not only to complete a task but also to prove that it was completed, and proving can mean exposure: revealing identity or location, using a personal account, posting publicly, acting in the physical world, or supplying evidence again during later checks—none disclosed by the posted price. We call these advertised requirements proof burden and measure them on RentAHuman, a 2026 market publicized as a place where AI agents could hire humans. We study what listings request, not what workers submit or experience.

We manually audited a nonrandom May 31, 2026 snapshot: every listing our searches returned from RentAHuman and Human Pages, another such market—981 listings, all but one from RentAHuman. Two independent coders recorded 13 features—11 kinds of evidence, recurring monitoring (repeated checks), and physical-world action—and our 0–5 Proof Burden Score; a blinded third coder resolved every disagreement. A planned content screen leaves 779 bounty/task listings as the primary population; among them, physical-world action appears in 48.5%, photo proof in 37.1%, and identity proof in 30.9%, and 438 listings (56.2%) score 4 or 5. Those listings span 154 distinct feature combinations: a checklist, not a single score, tells workers what a listing entails.

Platform metadata labels some requester accounts as agents or bots. In exploratory comparisons, physical-world action, location proof, or recurring monitoring appeared in 75.0% of agent-or-bot-labeled versus 55.3% of human-labeled listings, while their score-4-or-5 shares did not clearly differ. The labels are self-reported or platform-assigned, the agent-or-bot-labeled listings come from only 20 displayed names, and the comparison was chosen after seeing the data: a hypothesis, not a confirmed difference. Our main contributions are the 13-requirement vocabulary, the adjudicated manual audit, and the descriptive case study of this market; the score is a secondary screening summary, and the requester-label comparison exploratory. The study offers no worker-validated measure or automated detector yet; it is groundwork for both.

Guiding Worker Self-Selection in Crowdsourcing Contests: An LLM-Augmented Algorithmic Approach

Crowdsourcing platforms coordinate large pools of online workers who compete in contests to produce solutions for clients, from logo design to machine-learning challenges. A defining feature of these platforms is that workers self-select: Each chooses which contests to enter and how much effort to invest, and these choices are strategic rather than aligned with what the platform needs. As a result, the platform may end up with too few participants, or too little effort, on the contests where high-quality solutions matter most—while individual workers may end up dissatisfied, regretting choices that left them worse off than an alternative they could have taken. To reconcile these competing interests, platforms increasingly offer workers recommendations on where to compete. We study how to generate such recommendations as the game of self-selection in Tullock contests (SSTC), a two-stage model in which workers first choose contests and then compete within them. We introduce GRAF, a greedy algorithmic framework that produces self-selection outcomes in polynomial time by ordering workers according to a score vector, with provable guarantees—zero worker regret and platform optimality—for special cases of SSTC. Because a good ordering is difficult to design by hand once workers are heterogeneous, we propose LLMScore, an LLM-driven evolutionary framework that automatically designs the scoring algorithm GRAF uses. LLMScore addresses two obstacles to applying existing LLM-based methods here: jointly optimizing solution quality and worker satisfaction, and the intractability of evaluating worker regret. It is trained only on small instances of a single setting yet transfers to larger and structurally different settings, and—because its output is human-readable code—a platform operator can inspect and adjust the recommendation logic it learns. Across 1,000 synthetic instances spanning four settings, GRAF augmented by LLMScore consistently yields high-quality or even near-optimal outcomes with low worker regret, benefiting platform and workers alike.

Beyond Truth Discovery: A Two-Stage Framework to Assess the Severity of False Claim during Disasters

False information spreads rapidly on social media during disasters and can undermine emergency response efforts, public trust, and crisis communication. Existing research primarily focuses on determining whether social media posts contain false information, but provides limited insight into the specific false claims embedded within posts and the severity of individual false claims. To address the limitations, we propose a two-stage framework to assess the severity of false claims during disasters. In the first stage, we develop a false claim extraction agent that identifies false claims from multimodal social media posts containing text, images, videos, and links. A subsequent verification step validates extracted claims with supporting evidence. In the second stage, we define false claim severity as the combination of two complementary dimensions: believability, which determines the likelihood that a claim will be believed, and harmfulness, which captures the potential consequences if it is believed. Human annotators assess both dimensions to construct a claim-level severity benchmark using false claims extracted from Reddit posts related to hurricanes and wildfires. Building upon this benchmark, we investigate false claim severity assessment as a human-AI alignment problem, evaluating whether models can reproduce human judgments under a shared evaluation rubric rather than merely predicting severity labels. Experiments on the benchmark show that traditional supervised models exhibit limited alignment with human judgments, whereas Large Language Models (LLMs) achieve substantially stronger performance. Among the evaluated strategies, in-context learning consistently achieves the strongest alignment with human judgments, highlighting the importance of human examples and shared decision criteria for severity assessment.

Partner-Adaptive Delegation in Ad Hoc Teamwork

When agents collaborate with unfamiliar teammates, they must decide not only how to act, but also when to rely on others. This is challenging because a teammate may be strong on some subtasks but less effective on others, making delegation both useful and risky. Delegating control can improve performance when the teammate has the right skill, but it can also waste time or reduce performance when the teammate is less competent. Delegation also provides useful evidence about the teammate’s competence through their success or failure on the current subtask. We study this tradeoff in ad hoc teamwork, where an agent must learn when to act on its own and when to delegate to an unseen teammate whose strengths vary across subtasks. We propose CoDeT, a competence-belief delegation method that uses factored Bayesian beliefs to estimate teammate competence across subtasks and decide when to delegate. We evaluate CoDeT using a strict complementarity metric, which measures whether teamwork creates value beyond what either agent could achieve alone. Across sequential decision making tasks including Level-Based Foraging and Overcooked, CoDeT achieves positive teamwork value, avoids harmful delegation, and outperforms baselines that do not learn when to delegate. In settings where teammate contributions are consistently useful, CoDeT also recovers delegation strategies that are competitive with strong static engagement policies. These results suggest that factored competence beliefs can support more calibrated delegation, with implications for human-AI and multiagent teamwork.

Looks Can be Deceiving: Annotator and Reviewer Performance Across Imagery Scales In Aerial Damage Assessment

This paper presents the first known empirical investigation of annotator and reviewer performance across multi-source remotely sensed imagery, evaluating human labeling across drone, crewed aviation, and satellite views. Because existing aerial imagery datasets rely predominantly on single-source imagery, there is no currently established state of practice for efficiently allocating human labor to curate large-scale, multi-source aerial datasets. This work addresses this limitation by analyzing annotator and reviewer performance within a post-disaster building damage assessment dataset of 9 disasters, where 20,041 buildings in drone, 20,695 buildings in crewed aviation, and 33,392 buildings in satellite imagery were labeled. These labels, provided by 187 annotators, were then refined through two successive quality-control stages: a single-reviewer pass followed by a consensus-committee review. Our analysis reveals two findings that raise questions for standard crowd-sourcing practices. First, initial annotations were revised by the final committee at rates that rise steeply from higher- to lower-resolution sources (25.27% for crewed aviation and 36.95% for satellite), with the same ordering at every observed workflow stage. Second, a single individual review reduced but did not resolve this disagreement: after review, the committee still revised 6.85% of drone, 14.05% of crewed, and 20.86% of satellite labels. These observations suggest that, in workflows like this one, uniform review allocation leaves the most residual disagreement in lower-resolution imagery. Based on this evidence, and consistent with prior work on adaptive task assignment and budget-aware quality control, this paper offers three recommendations for multi-source dataset curation: (1) tailor labeling schemas to each imagery source, (2) prioritize consensus-based adjudication over individual review alone, and (3) preferentially target quality-control effort toward lower-resolution sources.

Information Satisfaction: A Reader-Centered Axis for Summarization Evaluation

The majority of work on summarization evaluation focuses on general summary quality (e.g., ROUGE, BERTScore) or specific desired properties (e.g., readability, factuality). However, these metrics fail to measure the utility of a summary to an individual user. For example, a biomedical researcher learning about the latest vaccine research will have different informational needs from a family doctor. Query-focused summarization captures part of this need, but in practice, users rarely state everything relevant in a query: a single short query is likely inadequate to distinguish the needs of a researcher from those of a physician. By contrast, a reader’s background or persona—their role and expertise—is comparatively stable across queries and recovers much of this missing context, which makes it a practical signal for assessing whether a summary satisfies that reader’s needs. In this work, we assess how sensitive popular summarization metrics are to both informational and persona differences, and find that many popular metrics, including strong LLM-as-judge metrics, fail basic perturbation tests of informational content. We additionally conduct an expert human evaluation, measuring summary preferences based on information satisfaction given a specific person’s background and use case. We find that both traditional and LLM-based metrics are insufficient measures of information satisfaction and agree poorly with human judgment.

Exploring the Design Space of AI Simulation-Based Communication Training: A Qualitative Study of Online Peer Counselor Preferences

A growing body of work uses AI-based simulation to teach communication skills, but current simulation-based communication training (SBCT) systems make inconsistent choices on design dimensions that prior theory identifies as critical: how skills are modeled, when feedback is given, and how self-reflection is scaffolded. These choices are especially consequential in domains like peer counseling, where trainees navigate emotional ambiguity and must anticipate how their words will land. Through prototype-driven and semi-structured interviews with 16 online peer counselors, we explore this design space of SBCT systems by eliciting reactions to 10 prototypes varying across the three dimensions. We find that participants wanted support throughout the process of composing responses (interpreting support-seekers’ messages, formulating goals, and anticipating consequences), not solely evaluation of finished messages. Participants also saw feedback timing as instilling mindsets about real-world consequences in sensitive communication, such as signaling proactive prevention of harm through feedback before sending messages. Participants described and anticipated different needs for novice and experienced trainees, suggesting a progression of training environments that adjusts simulation realism and scaffolding to experience level. We conclude with design implications for AI SBCT and peer counseling training, including a tension between users’ desire for validation and the need for ambiguity tolerance and authentic communication.

Towards Actionable Surgical Team Dynamics: from Teamwork to Counterfactual Annotations

Modeling team interactions in high-stakes environments such as surgical operating rooms is critical for understanding how coordination, communication, and individual behaviors jointly shape team performance and safety outcomes. However, existing multimodal datasets in this domain are often fragmented across modalities, annotation schemes, and formats, limiting their ability to support integrated analyses of real-world collaborative processes.

We address this limitation by introducing a curated and extended multimodal dataset for surgical team interaction analysis, built from real operating room recordings. Starting from an existing corpus, we construct a harmonized and analysis-ready version of the data by providing temporally consistent speaker diarization, bilingual (German and English) transcripts, and aligned multi-level annotations capturing team performance, interaction processes, and individual characteristics. Leveraging expert human annotations, team performance is assessed using a standardized surgical teamwork evaluation protocol, while interaction quality and individual attributes are annotated through structured rating schemes covering collaboration, group dynamics, and non-technical skills.

To further support the study of coordination breakdowns and performance variability, we introduce counterfactual annotations that describe plausible alternative team outcomes in the presence of observed interaction failures, enabling analysis of how specific behavioral patterns may relate to different trajectories of team performance. In addition, we provide structured temporal and relational representations designed to support computational modeling of teamwork processes and the design of AI-assisted collaborative systems.

Overall, the dataset is designed to support the study of how individual actions, interaction patterns, and team-level processes jointly contribute to team outcomes in surgical settings. It provides a unified resource for analyzing collaborative behavior in complex, real-world environments and enables both predictive modeling and behavioral analysis of team coordination in high-stakes domains.

AgentDynEx: Nudging the Mechanics and Dynamics of Multi-Agent Simulations

Multi-agent large language model simulations have the potential to model complex human behaviors and interactions. If the mechanics are set up properly, unanticipated and valuable social dynamics can surface. However, it is challenging to consistently enforce simulation mechanics while still allowing for rich and emergent dynamics. We present AgentDynEx, an AI system that helps set up, track, and repair simulations. Specifically, AgentDynEx introduces milestones that act as checkpoints and failure conditions that act as guardrails to ensure dynamics are relevant and mechanics are respected as the simulation progresses. It also introduces a method called nudging, where the system dynamically reflects on simulation progress and gently intervenes if it begins to deviate from intended outcomes. A technical evaluation found that nudging enables simulations to progress further without reducing the presence of notable dynamics compared to simulations without nudging. A case study with AgentDynEx documented instances where real users were able to simulate lived experiences. We discuss the importance of nudging as a technique for guiding agents towards desirable behaviors while preserving their freedom of choice.

Adapting Socio-Technical Congruence to Human-Agent Collectives

Socio-technical congruence is the fit between the coordination required by a project’s technical dependencies and the coordination its participants actually perform. The construct was developed for human software teams, where assignments are stable, timescales are comparable, channels are observable, and dependencies are code-level. Once AI agents join such collectives, these conditions break down. Existing taxonomies map the design space of human-agent collectives, but do not adapt the dependency-coordination construct itself. We contribute a typology of eight incongruence modes as a vocabulary for the structural breakdowns that human-agent collectives produce. We derive the typology by naming four scope conditions the original construct implicitly relied on, redefining the coordination-requirements matrix over a typed and time-varying actor and dependency graph, and enumerating the asymmetric ways the four conditions fail. To give each mode a testable form, we specify a generative model of human-agent collectives and use Monte Carlo simulation to parameterize it, yielding one incongruence rate curve per mode as a starting point for empirical work.

When the Interviewer Is a Bot: Behavior, Breakdowns, and Trust in MLLM-Led Interviews

Semi-structured interviews are a cornerstone of qualitative research but remain labor-intensive. We report an empirical study of what actually happens when the interviewer is an off-the-shelf real-time multimodal LLM (MLLM). We built InterviewBot, a voice-based interviewing system that wraps a real-time MLLM with a researcher-authored outline, and deployed it not as a novel architecture but as a research instrument for observing default MLLM interviewing behavior. In a practice study (N = 15), participants completed a bot-led semi-structured interview and then a human-led reflection session about that experience. We contribute (i) a turn-level behavioral analysis of an MLLM interviewer (Nturns = 428) showing that it is acknowledgment-heavy but probe-light (deepening probes account for 4.9% of all turns), and that 28.7% of question-bearing turns pack multiple questions into one turn despite an explicit one-question-at-a-time instruction; (ii) an inductive catalogue of four data-collection breakdowns (information loss, premature termination, latency, and interruption) observed in a deployed rather than simulated system; and (iii) three social dynamics from participants’ reflections: disclosure calibration, where reduced social pressure coincided with shallower elaboration; institutional legitimacy, where trust tracked perceived stakes and what delegation to AI signaled about the organizer rather than conversational competence; and conversational grounding, where content-grounded paraphrase, not generic social filler, was what participants read as listening. We conclude with design implications for depth control, transparent handoffs, and non-templated listening mechanisms in human-centered interview automation.

When AI Says "I have been in similar situations": Synthetic Lived Experience in Peer-like Caregiver Support

Caregivers often turn to online communities for informational and emotional support. In these spaces, peer supporters frequently draw on personal narratives to respond to emotionally complex caregiving situations. As large language models (LLMs) are increasingly designed as peer-like sources of support, they introduce a critical tension: AI can provide immediate, private, and nonjudgmental support, but it cannot authentically possess the lived experiences that make human peer support meaningful. Yet, when prompted to sound peer-like, LLMs may generate language that implies lived experience, such as “I’ve been in similar situations.” This creates a synthetic lived experience paradox: the same experiential language that may make AI support feel warm, relatable, and peer-like can also falsely position the system as someone with lived experience. We examine this paradox in the context of family caregivers of people living with Alzheimer’s Disease and Related Dementias (ADRD). Drawing on caregiver support exchanges from online communities and prompted peer-like responses from three LLMs—LLaMA, GPT-4o-mini, and MedGemma—we analyze how human peers use personal narratives and how AI incorporates similar narrative forms. Our psycholinguistic analysis shows that online peer responses used significantly more first-person and past-focused language than prompted peer-like AI responses. Qualitatively, we identify seven types of personal narratives in human peer support and show that AI often captures their emotional work, but can generalize, flatten, or fabricate their experiential grounding. These findings reveal a narrative authenticity gap: peer-like AI can generate synthetic lived experience without the real experience that makes peer support meaningful. We argue that caregiver-support AI systems need mechanisms to distinguish supportive peer-like framing from fabricated lived experience, ensuring that models can offer warmth and validation without falsely positioning themselves as experiential peers.

From Answers to Audit Opinions: Cost-Aware Expert Verification of Financial Artifacts from Tool-Using LLM Agents

Tool-using LLM agents increasingly produce financial artifacts such as ratio tables, evidence-backed memos, and spreadsheet-style analyses. Existing evaluations often reduce such outputs to binary correctness or faithfulness labels, overlooking whether the artifact has sufficient evidence, whether errors are material or pervasive, and whether evidence gaps should prevent acceptance. We adapt audit-opinion logic to agentic artifact verification as a research-label framework with four labels: Clean, Qualified, Adverse, and Disclaimer. The proposed audit certificate turns each artifact into a checkable workpaper by linking claims to provenance, formulas, verification outcomes, scope limits, and Critical Artifact Matters. Our evaluation package combines SEC-backed cases, oracle-backed injected failures, expert adjudication, blind re-audit, hard-case stress diagnostics, and an expanded 400-case oracle-backed checker-field diagnostic. In the expanded diagnostic, full-certificate checker fields remove false-clean labels that remain common under artifact-only and partial-certificate views. The completed Phase 3 expert batch shows that oracle-backed perturbation labels and expert audit opinions diverge in materiality-sensitive cases, while blind re-audit shows substantial within-expert stability.