Multi-Agent Council

Overview

Multi-Agent Council is a local experiment environment for studying how AI agents deliberate. A FastAPI workspace provides the controls and saved records for each run. It lets you cast 2 to 5 debaters plus a judge, assign each seat its own provider, model, persona, stance, and intensity, then watch the session unfold in one shared Chamber thread.

The experiment centers on the transcript and the decisions that produced it. It keeps provider CLIs, auth, transcripts, and session state local while making every turn, pause, fallback, trace event, and judge decision visible enough to replay later. That makes it a practical testbed for seeing when multiple model voices improve reasoning and when they mostly add latency, theater, or noise.

Enlarge image ↗

Why I Built It

Partly inspired by Clawbook, Karpathy’s llm-council, and Gnosia, I wanted to make my own debate hall for LLMs. I liked the idea of giving each agent a clearer role and letting them discuss the same topic in a setup that felt a little more like a game than a normal chat workflow.

The bigger motivation was to inspect multi-agent behavior more closely. I wanted to see when multiple agents actually helped with reasoning, when they only added theater, and how things like persona design, turn structure, and transcript visibility changed the quality of the conversation.

What I’m Testing

• Do different roles and personas bring distinct arguments into a discussion?
• How do turn order, reply pacing, and moderator notes change the debate?
• Does the judge’s decision follow the evidence in the transcript?
• When does another round add useful information, and what latency does it introduce?

The environment lets me vary these settings and inspect the resulting runs. It does not establish that multi-agent debates outperform a single model.

How It Works

• Starts from the Chamber, where a user defines the question, debate mode, topic metadata, debater lineup, judge, personas, and provider models
• Checks local model readiness through provider presets, including Codex/OpenAI and Claude/Anthropic CLI paths, with manual command overrides available for less-verified backends
• Resolves auto-selected personas when needed, then runs opening statements and reply cycles through the FastAPI debate engine
• Streams saved thread entries and trace events over WebSockets so the live Chamber, classic arena, and Pixel Stage stay in sync with the persisted transcript
• Pauses between debate cycles so the user can continue, add a moderator note, end the session, ask the judge to decide, or manually pick a winner
• Persists sessions, messages, thread entries, rounds, trace events, judge scores, human votes, persona files, and provider-session state in local SQLite and local JSON files

What I Inspect

The saved transcript shows opening arguments, replies, and the judge’s decision. Trace events expose provider choices, latency, fallback status, and timing so I can examine how a run unfolded alongside what the agents said.

Session JSON and trace JSON make the run available for later review. Judge scores and human votes are recorded as part of the session; they are signals to inspect, not proof that a conclusion is correct.

Environment Components

• Chamber-based orchestration: setup, live turns, transcript controls, session history, judging, and exports live in one local web app instead of separate scripts or chat panes
• Persona system: built-in and custom personas can be assigned manually, generated from a description, auto-selected before the opening round, and tuned with per-seat intensity controls
• Multiple debate modes: serious analysis, theatrical style, and shorter conversational back-and-forth all use the same transcript-first engine with different reply pacing
• Real-provider execution: CLI adapters run against local Codex/OpenAI and Claude/Anthropic setups, with persistent provider threads, recovery fallbacks, reset controls, and a mock backend for tests
• Inspectable local state: SQLite-backed sessions include messages, rounds, thread entries, provider-session state, judge scores, human votes, and metadata for replay or later comparison
• Observability and export: the Chamber exposes latency, provider/model choice, fallback status, token estimates, trace timelines, session JSON, and trace JSON so debate behavior can be audited after the run