Skip to content

Address PR Comments (#4) #8

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
ableinc wants to merge 1 commit into
base: main
Choose a base branch
from
Draft

Address PR Comments (#4) #8

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
80 changes: 69 additions & 11 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,16 +5,16 @@ isolated git worktree, and opens a **draft** pull request for you to review. It
anything.

```
gh search issues --label agent-ready
│
▼
claim (SQLite lease, one issue per repo at a time)
│
▼
gh search issues --label agent-ready gh search prs --author <bot>
│ │
▼ ▼
claim (SQLite lease, one issue per repo at a time, shared with the PR path)
│ │
▼ ▼
git worktree ──▶ claude -p --output-format stream-json
│
▼
run the repo's tests ──▶ commit ──▶ push ──▶ gh pr create --draft
│ │
▼ ▼
run the repo's tests ──▶ commit ──▶ push ──▶ gh pr create --draft react 👀 ──▶ push ──▶ react 👍
```

## Table of contents
Expand All @@ -26,6 +26,7 @@ gh search issues --label agent-ready
- [CLI flags](#cli-flags)
- [How work is selected](#how-work-is-selected)
- [Lifecycle of one issue](#lifecycle-of-one-issue)
- [Responding to PR comments](#responding-to-pr-comments)
- [Configuration reference](#configuration-reference)
- [config.json](#configjson)
- [models.json](#modelsjson)
Expand Down Expand Up @@ -215,6 +216,39 @@ issue is worked at all: remove `agent-ready` to stop the retries.
A usage limit hit mid-run is recorded as `deferred` — neither an attempt nor a failure, so it
neither extends the back-off nor drops the issue down the ladder.

## Responding to PR comments

Once a draft PR is open, a reviewer can hand feedback back to the agent without re-labelling
anything: comment on the PR mentioning the handle in `github.pr_comments.mention` (`@coding-agent`
by default), and the daemon picks it up on its next poll.

1. **Discover** — `gh search prs --author <the daemon's own login>` scoped to `github.owners`, run
before issue discovery in every tick and drawing from the same per-repo concurrency budget: a
reviewer waiting on a reply outranks starting a new issue. Only PRs the daemon itself opened, on a
branch starting with `workspace.branch_prefix`, are ever considered — a hard rule, not a config
knob, so the daemon only ever pushes to branches it created itself.
2. **Match** — every conversation comment and inline review comment is checked for the mention.
Quoted lines (`>`) and fenced code blocks don't count, so a comment that merely quotes or shows a
previous mention can't re-trigger the agent, and the daemon skips its own comments and marker-
tagged replies. Only commenters whose `author_association` is `OWNER`, `MEMBER`, or `COLLABORATOR`
(or who appear in `github.pr_comments.allowed_authors`) can trigger a run — review summary bodies
are shown to the model as context but can never trigger one, since GitHub's REST API has no
reactions endpoint for a review as a whole.
3. **Acknowledge** — every matching comment gets `github.pr_comments.ack_reaction` (👀 by default)
immediately, before any cloning: that's the visible promise that it was seen.
4. **Address** — the PR's own branch is checked out as-is (not reset against the default branch), and
Claude is given the PR and its triggering comments, with the same no-git/no-GitHub-mutation
contract as the issue flow, reframed around a branch and PR that already exist. If the feedback is
a question rather than a change request, the agent answers it in its final summary instead of
editing code — that still counts as addressed.
5. **Deliver** — if there's a code change, it's committed, verified, and pushed to the PR's branch;
either way a reply is posted summarising what was done, and each comment gets
`github.pr_comments.done_reaction` (👍 by default).
6. **Retry** — a failed attempt leaves the 👀 in place (the comment was seen) and retries after the
same exponential back-off as a failed issue, tracked per comment so one stuck comment doesn't hold
up others on the same PR. A daemon restart between the 👀 and the reply is not stranded: the task
is recorded as soon as the reaction goes out, and a subsequent pass retries it.

## Configuration reference

### config.json
Expand All @@ -237,7 +271,17 @@ This repository's own `config.json` is also **compiled into the binary** at buil
"exclude_repos": [],
"search_limit": 50,
"poll_interval": "5m",
"binary": "gh"
"binary": "gh",
"pr_comments": {
"enabled": true,
"mention": "@coding-agent",
"search_limit": 30,
"max_age": "168h",
"ack_reaction": "eyes",
"done_reaction": "+1",
"allowed_authors": [],
"allowed_associations": ["OWNER", "MEMBER", "COLLABORATOR"]
}
},
"workspace": {
"root": "~/.agent-loop/work",
Expand Down Expand Up @@ -291,6 +335,13 @@ This repository's own `config.json` is also **compiled into the binary** at buil
| `github.exclude_repos` | `owner/name` repos to never touch, even if labelled |
| `github.search_limit` | max issues fetched per discovery pass |
| `github.poll_interval` | how often discovery runs |
| `github.pr_comments.enabled` | watch the daemon's own open PRs for `@`-mentions and act on them (see [Responding to PR comments](#responding-to-pr-comments)) |
| `github.pr_comments.mention` | handle a comment must contain to trigger a response; **must start with `@`** |
| `github.pr_comments.search_limit` | max of the daemon's own open PRs checked per pass |
| `github.pr_comments.max_age` | ignore comments older than this; `0` disables the limit |
| `github.pr_comments.ack_reaction` / `done_reaction` | GitHub reaction content applied on pickup / once addressed; one of `+1 -1 laugh confused heart hooray rocket eyes` |
| `github.pr_comments.allowed_authors` | explicit login allowlist for who may trigger the agent; empty falls back to `allowed_associations` |
| `github.pr_comments.allowed_associations` | `author_association` values permitted to trigger the agent (`OWNER`, `MEMBER`, `COLLABORATOR`, ...) when `allowed_authors` is empty |
| `workspace.root` | where per-issue worktrees live |
| `workspace.repos_root` | where the one-per-repo checkout-less clones live |
| `workspace.logs_root` | where JSONL run transcripts are written |
Expand Down Expand Up @@ -421,7 +472,7 @@ Loopback-only by default. It can pause and cancel work, so do not expose it.
| ---------------------------- | ------------------------------------------------------------------- |
| `GET /healthz` | liveness |
| `GET /status` | gate state, in-flight runs, claims, model cooldowns, usage snapshot |
| `GET /runs?limit=&repo=` | recent runs with outcome, model, cost, PR link, created/started/ended timestamps |
| `GET /runs?limit=&repo=` | recent runs with outcome, model, cost, PR link, created/started/ended timestamps; `kind` distinguishes an issue run from a PR-comment run |
| `GET /runs/{id}` | one run plus its event timeline |
| `GET /runs/{id}/log` | the raw JSONL transcript of the Claude run |
| `GET /sessions?repo=&issue=&limit=` | Claude session IDs recorded per repo/issue, newest first |
Expand Down Expand Up @@ -494,6 +545,13 @@ user can. What constrains it:
- **Nothing is ever merged**, and every PR is a draft.
- Child processes run in their own process group and are killed as a group on timeout, so a
runaway grandchild (a stray build/test process) can't outlive the run.
- **PR comments only ever act on PRs the daemon itself opened, on a branch under
`workspace.branch_prefix`.** A mention on any other pull request is ignored outright.
- **Only permitted commenters can trigger a run from a PR comment** — `author_association` in
`OWNER`/`MEMBER`/`COLLABORATOR`, or an explicit `github.pr_comments.allowed_authors` entry —
otherwise an arbitrary commenter on a public repo could drive a `bypassPermissions` Claude run.
- `--dry-run` suppresses PR-comment reactions and replies exactly like it does labels, comments, and
pushes on the issue path.

`--install` below scopes the systemd unit's filesystem access to `/opt/coding-agent-loop` and its
own `~/.agent-loop` regardless of which account it runs as (see
Expand Down
12 changes: 11 additions & 1 deletion config.example.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,17 @@
"exclude_repos": [],
"search_limit": 50,
"poll_interval": "5m",
"binary": "gh"
"binary": "gh",
"pr_comments": {
"enabled": true,
"mention": "@coding-agent",
"search_limit": 30,
"max_age": "168h",
"ack_reaction": "eyes",
"done_reaction": "+1",
"allowed_authors": [],
"allowed_associations": ["OWNER", "MEMBER", "COLLABORATOR"]
}
},
"workspace": {
"root": "~/.agent-loop/work",
Expand Down
60 changes: 60 additions & 0 deletions internal/config/config.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -74,6 +74,34 @@ type GitHubConfig struct {
PollInterval Duration `json:"poll_interval"`
// Binary is the gh executable; overridable for tests.
Binary string `json:"binary"`
// PRComments controls responding to @-mentions on the daemon's own pull requests.
PRComments PRCommentsConfig `json:"pr_comments"`
}

// PRCommentsConfig controls watching the daemon's own pull requests for
// review comments that @-mention it, and acting on them.
type PRCommentsConfig struct {
Enabled bool `json:"enabled"`
// Mention is the handle a comment must contain to trigger a response,
// e.g. "@coding-agent".
Mention string `json:"mention"`
// SearchLimit caps how many of the daemon's own open PRs one pass checks.
SearchLimit int `json:"search_limit"`
// MaxAge bounds how old a comment may be and still trigger a response. 0
// means no limit.
MaxAge Duration `json:"max_age"`
// AckReaction/DoneReaction are GitHub reaction content values (one of
// "+1 -1 laugh confused heart hooray rocket eyes") applied to a triggering
// comment when it is picked up and when it has been addressed.
AckReaction string `json:"ack_reaction"`
DoneReaction string `json:"done_reaction"`
// AllowedAuthors is an explicit login allowlist of commenters who may
// trigger the agent. Empty falls back to AllowedAssociations.
AllowedAuthors []string `json:"allowed_authors"`
// AllowedAssociations lists the author_association values (OWNER, MEMBER,
// COLLABORATOR, ...) permitted to trigger the agent when AllowedAuthors is
// empty.
AllowedAssociations []string `json:"allowed_associations"`
}

type WorkspaceConfig struct {
Expand Down Expand Up @@ -165,6 +193,15 @@ func Default() Config {
SearchLimit: 50,
PollInterval: Duration(5 * time.Minute),
Binary: "gh",
PRComments: PRCommentsConfig{
Enabled: true,
Mention: "@coding-agent",
SearchLimit: 30,
MaxAge: Duration(168 * time.Hour),
AckReaction: "eyes",
DoneReaction: "+1",
AllowedAssociations: []string{"OWNER", "MEMBER", "COLLABORATOR"},
},
},
Workspace: WorkspaceConfig{
Root: "~/.agent-loop/work",
Expand Down Expand Up @@ -289,9 +326,32 @@ func (c *Config) Validate() error {
if c.Discord.Enabled && c.Discord.WebhookURL == "" {
return fmt.Errorf("discord.webhook_url must be set when discord.enabled is true")
}
if c.GitHub.PRComments.Enabled {
pc := c.GitHub.PRComments
if !strings.HasPrefix(pc.Mention, "@") {
return fmt.Errorf("github.pr_comments.mention must start with '@', got %q", pc.Mention)
}
if pc.SearchLimit < 1 {
return fmt.Errorf("github.pr_comments.search_limit must be >= 1, got %d", pc.SearchLimit)
}
if !validReaction(pc.AckReaction) {
return fmt.Errorf("github.pr_comments.ack_reaction %q is not a valid GitHub reaction", pc.AckReaction)
}
if !validReaction(pc.DoneReaction) {
return fmt.Errorf("github.pr_comments.done_reaction %q is not a valid GitHub reaction", pc.DoneReaction)
}
}
return nil
}

// validReactions are the only content values GitHub's reactions API accepts.
var validReactions = map[string]bool{
"+1": true, "-1": true, "laugh": true, "confused": true,
"heart": true, "hooray": true, "rocket": true, "eyes": true,
}

func validReaction(r string) bool { return validReactions[r] }

// validOwners returns the entries of owners that are non-blank after trimming.
func validOwners(owners []string) []string {
out := make([]string, 0, len(owners))
Expand Down
46 changes: 46 additions & 0 deletions internal/config/config_test.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -174,3 +174,49 @@ func TestRetryBackoffMaxMustNotBeBelowTheBase(t *testing.T) {
t.Fatalf("want a backoff validation error, got %v", err)
}
}

func TestPRCommentsDefaults(t *testing.T) {
cfg, err := Load(writeConfig(t, `{"github":{"owners":["acme"]}}`), false)
if err != nil {
t.Fatal(err)
}
pc := cfg.GitHub.PRComments
if !pc.Enabled {
t.Fatal("pr_comments should be enabled by default")
}
if pc.Mention != "@coding-agent" {
t.Fatalf("unexpected default mention %q", pc.Mention)
}
if pc.AckReaction != "eyes" || pc.DoneReaction != "+1" {
t.Fatalf("unexpected default reactions: ack=%q done=%q", pc.AckReaction, pc.DoneReaction)
}
if len(pc.AllowedAssociations) == 0 {
t.Fatal("default allowed associations should not be empty")
}
}

// A config file written before this feature existed has no pr_comments block
// at all; it must still load using the defaults.
func TestConfigWithoutPRCommentsBlockStillLoads(t *testing.T) {
cfg, err := Load(writeConfig(t, `{"github":{"owners":["acme"],"label":"agent-ready"}}`), false)
if err != nil {
t.Fatalf("a config predating pr_comments must still load: %v", err)
}
if !cfg.GitHub.PRComments.Enabled {
t.Fatal("defaults should still populate pr_comments")
}
}

func TestPRCommentsRejectsInvalidReaction(t *testing.T) {
_, err := Load(writeConfig(t, `{"github":{"owners":["acme"],"pr_comments":{"enabled":true,"mention":"@coding-agent","search_limit":30,"ack_reaction":"nope","done_reaction":"+1"}}}`), false)
if err == nil || !strings.Contains(err.Error(), "ack_reaction") {
t.Fatalf("want an invalid reaction error, got %v", err)
}
}

func TestPRCommentsRejectsMentionWithoutAt(t *testing.T) {
_, err := Load(writeConfig(t, `{"github":{"owners":["acme"],"pr_comments":{"enabled":true,"mention":"coding-agent","search_limit":30,"ack_reaction":"eyes","done_reaction":"+1"}}}`), false)
if err == nil || !strings.Contains(err.Error(), "mention") {
t.Fatalf("want a mention validation error, got %v", err)
}
}
22 changes: 22 additions & 0 deletions internal/discord/notifier.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -290,6 +290,28 @@ func (n *Notifier) PlanPosted(r RunRef, res *claude.Result, elapsed time.Duratio
})
}

// PRCommentsAddressed reports that review feedback on a pull request was
// acted on: code pushed (or not, when the feedback needed only a reply) and
// verified.
func (n *Notifier) PRCommentsAddressed(r RunRef, handled int, res *claude.Result, v verify.Result, elapsed time.Duration) {
model, cost := "", 0.0
if res != nil {
model, cost = res.PrimaryModel(), res.TotalCostUSD
}
n.post(embed{
Title: r.title("PR comments addressed"),
Description: r.description(),
Color: colorGreen,
Fields: append(r.fields(),
embedField{Name: "Comments handled", Value: fmt.Sprintf("%d", handled), Inline: true},
embedField{Name: "Model", Value: orNone(model), Inline: true},
embedField{Name: "Cost", Value: money(cost), Inline: true},
embedField{Name: "Verification", Value: orNone(v.Status), Inline: true},
embedField{Name: "Duration", Value: humanDuration(elapsed), Inline: true},
),
})
}

// RunFailed reports a failed run and when it will be tried again. Retries are
// unbounded, so "when" is the useful number, not "how many are left".
func (n *Notifier) RunFailed(r RunRef, cause string, nextAttempt time.Time) {
Expand Down
Loading
Loading