Advisor: Ray Friedman

We investigated two connected research questions: can language models make transcript coding more scalable without hiding uncertainty, and can controlled agents expand the range of negotiation scenarios that researchers are able to study?

Coding negotiation transcripts

The first research track turns unstructured negotiation transcripts into structured observations while keeping the coding scheme and uncertain cases visible to researchers.

  • What it codes: It assigns predefined labels, such as "Substantiation" or "Providing Information," to transcript segments according to a researcher's coding scheme.
  • How we assessed consistency: It repeats the coding pass five times, measures agreement among the outputs, and reports a consistency score so researchers can inspect uncertain cases rather than accepting every label at face value.
  • What the evaluation found: In the reported study setting, the estimated cost of coding one transcript fell from more than $5,000 to $3, while measured human–AI agreement increased from 30% to 80%.

Developing the system required careful preprocessing of a curated transcript dataset. We compared multiple GPT-4 and Claude 2/3 variants, then evaluated how prompt design, in-context examples, and scoring choices affected coding quality and consistency. This framing matters because repeated model agreement is a diagnostic signal, not proof that a label is correct; human review remains important when the coding scheme is ambiguous or the runs disagree.

The work was presented at the 2025 AI Negotiation Summit at Harvard and MIT.

Simulating negotiation scenarios

The second research track explores LLM agents that negotiate under assigned experimental conditions. The objective is to generate varied transcripts while preserving researcher control over the factors being studied. If validated against human negotiation data, this approach could reduce data-collection cost and make a wider range of scenarios practical to test; synthetic realism alone would not establish external validity.

We conducted this work with Professor Ray Friedman and the AI Negotiation Lab research team.