feat: inject the human conversation into every agent prompt #22

Merged

ellen merged 1 commit from feat/comment-context-injection into main 2026-08-01 21:14:59 +00:00

Conversation 0 Commits 1 Files changed 10 +353 -13

cloudcli commented 2026-08-01 20:30:17 +00:00

Member

Copy link

Closes #21.

The bug

No agent route read a single human-authored comment.

Route	What it fed the model
plan	issue .title + .body — nothing else (plan.yaml:61-75)
dev	title + body, its own last plan comment, its own last findings comment (dev.yaml:94-134)
test	last plan comment + git diff --stat (test.yaml:73-102)
coordinate	epic .title + .body (coordinate.yaml:60-75)

So re-triggering an agent re-ran a byte-identical prompt. The comment written to redirect it was never in context, and it produced the same answer again. On valssi-ai/development-container#15 that gave three near-identical plans and made course correction impossible.

The fix

New lib/thread composite action renders the human side of an issue or PR thread to a file the route pastes into its prompt:

• human comments in full, attributed and timestamped
• agent artifacts (plan / findings / result) elided to one-line markers — the routes that need them fetch them directly; in the thread they only need to mark where they happened so the model can order the conversation
• other bot comments reduced to their first line (preflight notices, acks)
• <!-- agent-state --> / <!-- agent-digest --> dropped entirely
• paginated — the API's 50-comment default was silently truncating long threads
• windowed to the most recent 30 comments, truncated at 4000 chars each

Wired into all four model-using routes, with the thread last in the prompt and an explicit instruction that later comments override earlier ones, the approved plan, and the test findings — a human asking for something different outranks all of them.

Two extras that fall out of the same problem:

• plan now also receives its own previous plan, so a re-trigger is a revision — it opens with a **What changed:** line — rather than a fresh derivation. This is what stops the duplicate-plan pile-up on #15.
• dev and test read the PR thread too, since that is where review feedback actually lands.

Safety

Fails safe throughout. An API error, an empty thread, or a bot-only thread all yield an empty file and has-human=false; the route omits the block and runs exactly as before. The action never fails the job.

Bot logins are matched exactly (claude-plan, claude-dev) — an early draft used jq's inside, which does substring matching and would have misclassified a human whose login is a substring of a bot's as an agent.

No new triggers, no new permissions, no cost change: the same label re-trigger, now with the conversation in context.

Testing

Ran against a fixture reproducing the real #15 thread (3 plan comments, 1 state comment, 2 human comments, 1 bot notice, plus a human login that is a substring of a bot login):

• rendering — plans elided, state dropped, human comments in full and in order, substring login correctly kept as human
• bash -n clean across all 16 run: blocks; all touched YAML parses
• edge cases — empty thread / bot-only thread / total API failure / missing issue number all degrade to has_human=false without crashing
• pagination — 120 comments over 3 pages fetched, windowed to the most recent 30
• truncation — over-long comments clipped with a …[truncated] marker

Not yet exercised against the live Forgejo API — the first real run will be the planner on #15 or on this repo.

Follow-ups (not in this PR)

• Comment router (#21's other half): make an @mention re-fire a route directly, so steering does not need a label round-trip. The plumbing is already half-built — caller-agent.yaml:15 forwards issue_comment events today and dispatch.yaml has no job that matches them.
• Superseding plan artifacts rather than appending them, and wiring the defined-but-unused needs/input label.

🤖 Generated with Claude Code

Closes #21. ## The bug No agent route read a single human-authored comment. | Route | What it fed the model | | --- | --- | | plan | issue `.title` + `.body` — nothing else (`plan.yaml:61-75`) | | dev | title + body, its own last plan comment, its own last findings comment (`dev.yaml:94-134`) | | test | last plan comment + `git diff --stat` (`test.yaml:73-102`) | | coordinate | epic `.title` + `.body` (`coordinate.yaml:60-75`) | So re-triggering an agent re-ran a **byte-identical prompt**. The comment written to redirect it was never in context, and it produced the same answer again. On valssi-ai/development-container#15 that gave three near-identical plans and made course correction impossible. ## The fix New `lib/thread` composite action renders the *human* side of an issue or PR thread to a file the route pastes into its prompt: - **human comments** in full, attributed and timestamped - **agent artifacts** (plan / findings / result) elided to one-line markers — the routes that need them fetch them directly; in the thread they only need to mark *where* they happened so the model can order the conversation - **other bot comments** reduced to their first line (preflight notices, acks) - `<!-- agent-state -->` / `<!-- agent-digest -->` dropped entirely - **paginated** — the API's 50-comment default was silently truncating long threads - windowed to the most recent 30 comments, truncated at 4000 chars each Wired into all four model-using routes, with the thread **last** in the prompt and an explicit instruction that later comments override earlier ones, the approved plan, and the test findings — a human asking for something different outranks all of them. Two extras that fall out of the same problem: - **plan** now also receives its own previous plan, so a re-trigger is a *revision* — it opens with a `**What changed:**` line — rather than a fresh derivation. This is what stops the duplicate-plan pile-up on #15. - **dev** and **test** read the **PR** thread too, since that is where review feedback actually lands. ## Safety Fails safe throughout. An API error, an empty thread, or a bot-only thread all yield an empty file and `has-human=false`; the route omits the block and runs exactly as before. The action never fails the job. Bot logins are matched **exactly** (`claude-plan`, `claude-dev`) — an early draft used jq's `inside`, which does substring matching and would have misclassified a human whose login is a substring of a bot's as an agent. No new triggers, no new permissions, no cost change: the same label re-trigger, now with the conversation in context. ## Testing Ran against a fixture reproducing the real #15 thread (3 plan comments, 1 state comment, 2 human comments, 1 bot notice, plus a human login that is a substring of a bot login): - rendering — plans elided, state dropped, human comments in full and in order, substring login correctly kept as human - `bash -n` clean across all 16 `run:` blocks; all touched YAML parses - edge cases — empty thread / bot-only thread / total API failure / missing issue number all degrade to `has_human=false` without crashing - pagination — 120 comments over 3 pages fetched, windowed to the most recent 30 - truncation — over-long comments clipped with a `…[truncated]` marker Not yet exercised against the live Forgejo API — the first real run will be the planner on #15 or on this repo. ## Follow-ups (not in this PR) - **Comment router** (#21's other half): make an `@mention` re-fire a route directly, so steering does not need a label round-trip. The plumbing is already half-built — `caller-agent.yaml:15` forwards `issue_comment` events today and `dispatch.yaml` has no job that matches them. - Superseding plan artifacts rather than appending them, and wiring the defined-but-unused `needs/input` label. 🤖 Generated with [Claude Code](https://claude.com/claude-code)

cloudcli added 1 commit 2026-08-01 20:30:17 +00:00

feat: inject the human conversation into every agent prompt ... ff73ed8c5c

Re-triggering an agent re-ran a byte-identical prompt. The routes built their
prompts from the issue title+body (plan, coordinate), or title+body plus the
agent's own plan/findings comments (dev, test) — no route read a single
human-authored comment. So a comment written to redirect an agent was never in
context, and the agent produced the same answer again. On
valssi-ai/development-container#15 that produced three near-identical plans and
made course correction impossible (agent-workflows#21).

Add lib/thread, a composite action that renders the human side of an issue or PR
thread to a file the route pastes into its prompt:

- human comments in full, attributed and timestamped
- agent artifacts (plan / findings / result) elided to one-line markers — the
  routes that need them fetch them directly, and in the thread they only need to
  mark where they happened so the model can order the conversation
- other bot comments reduced to their first line
- <!-- agent-state --> / <!-- agent-digest --> dropped
- paginated (the API's 50-comment default silently truncated long threads),
  windowed to the most recent 30, truncated per comment

Wire it into all four model-using routes, with the thread last in the prompt and
an explicit instruction that later comments override earlier ones, the approved
plan, and the test findings — a human asking for something different outranks
all of them. plan additionally receives its own previous plan, so a re-trigger
is a revision that opens by saying what changed, rather than a fresh start. dev
and test also read the PR thread, since that is where review feedback lands.

Fails safe throughout: an API error, an empty thread or a bot-only thread all
yield an empty file and has-human=false, and the route omits the block.

Refs: valssi-public/agent-workflows#21

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

ellen merged commit b7b86f4599 into main 2026-08-01 21:14:59 +00:00

ellen deleted branch feat/comment-context-injection 2026-08-01 21:14:59 +00:00

ellen referenced this pull request from a commit 2026-08-01 21:15:01 +00:00

Merge pull request 'feat: inject the human conversation into every agent prompt' (#22) from feat/comment-context-injection into main

Sign in to join this conversation.

Reviewers

No reviewers

Labels

Clear labels

  

agent

coordinate

Summon coordinator on an epic (one-shot; agent removes on pickup)

  

agent

dev

Summon developer (one-shot; agent removes on pickup)

  

agent

plan

Summon planner (one-shot; agent removes on pickup)

  

agent

reflect

Summon reflection agent (one-shot; agent removes on pickup)

  

agent

test

Summon tester on a PR (one-shot; agent removes on pickup)

  

agent

triage

Summon triage (one-shot; agent removes on pickup)

  

ai/off

Agents ignore this issue entirely

  

blocked

night

Queued: waiting for night window

  

blocked

usage

Queued: usage window exhausted (resume time in state comment)

  

loop

1

Dev↔test bounce 1 of 3 at current tier

  

loop

2

Dev↔test bounce 2 of 3 at current tier

  

loop

3

Dev↔test bounce 3 of 3 — cap; next is escalate or needs/human

  

needs/approval

Agent → you: plan awaits approval

  

needs/human

Agent → you: stuck / escalated out (see summary comment)

  

needs/input

Agent → you: blocked on a question only you can answer

  

needs/review

Agent → you: PR passed tests, awaits human review/merge

  

plan/approved

Human approval gate — honored only when applied by APPROVER

  

run/night

Start only inside NIGHT_WINDOW (default 00:00–06:00 Europe/Helsinki)

  

state

blocked

Machine state: blocked (see needs/* or blocked/*)

  

state

building

Machine state (workflow-managed)

  

state

passed

Machine state: tests passed, awaiting merge

  

state

planning

Machine state (workflow-managed)

  

state

testing

Machine state (workflow-managed)

  

tier

fable

Model rung: Fable 5 — HUMAN-APPLIED ONLY

  

tier

haiku

Model rung: Haiku 4.5

  

tier

local

Model rung: local vLLM (no quota)

  

tier

opus

Model rung: Opus 4.8 (hard dev, default plan)

  

tier

sonnet

Model rung: Sonnet 5 (default dev/test)

No labels

agent

coordinate

agent

dev

agent

plan

agent

reflect

agent

test

agent

triage

ai/off

blocked

night

blocked

usage

loop

1

loop

2

loop

3

needs/approval

needs/human

needs/input

needs/review

plan/approved

run/night

state

blocked

state

building

state

passed

state

planning

state

testing

tier

fable

tier

haiku

tier

local

tier

opus

tier

sonnet

Milestone

Clear milestone

No items

No milestone

Projects

Clear projects

No items

No project

Assignees

Clear assignees

No assignees

1 participant

Notifications

Subscribe

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

valssi-public/agent-workflows!22

No description provided.