Home

Appearance

Copilot Tools — PR Summary Workflow ​

This guide walks through setting up and using the automated PR summary feature powered by the Copilot SDK.

💰 Cost-friendly: Uses an efficient model by default — fast, inexpensive model ideal for summaries.

What is the PR Summary Workflow? ​

Automatic AI-generated summaries for every pull request using GitHub Copilot.

When you open or update a PR:

  1. GitHub Actions triggers automatically
  2. Copilot SDK reads the PR diff
  3. Generates a concise, non-technical summary (example: using gpt-5-mini)
  4. Posts the summary as a PR comment
  5. Updates on re-runs (old comment is deleted)

Example output:

🤖 Copilot PR Summary

This PR adds a persistent multi-turn chat feature to the Copilot CLI, allowing users to:

  • Start an interactive terminal chat with Copilot
  • Ask follow-up questions within the same session
  • View conversation history and request summaries
  • Switch between different Copilot sessions

The implementation uses the Copilot SDK's streaming event API to display responses word-by-word.

Generated by Copilot Tools · Model: gpt-5-mini (example)

Setup ​

Step 1: Create a GitHub PAT ​

Requirement: Your personal account must have GitHub Copilot active (Individual or Business plan).

  1. Go to github.com → your avatar → Settings → Developer settings → Tokens (classic)
  2. Click Generate new token (classic)
  3. Configure:
    • Name: copilot-ci (or any name)
    • Expiration: 90 days (recommended) or no expiration
    • Scopes: Check ✅ repo (no other scopes needed)
  4. Click Generate token
  5. Copy the token (it won't show again) — starts with ghp_

⚠️ The built-in GITHUB_TOKEN in Actions lacks Copilot permissions. You must create a Classic PAT.

Step 2: Add Secret to Your Repo ​

  1. Go to your repo → Settings → Secrets and variables → Actions
  2. Click New repository secret
    • Name: COPILOT_TOKEN
    • Value: Paste the ghp_ token from Step 1
  3. Click Add secret

Step 3: Verify Workflow Files Are Present ​

Check that these files exist in your repo:

.github/
├── actions/
│   └── copilot/
│       └── action.yml          ✓ Copilot action definition
└── workflows/
    └── copilot-pr-summary.yml  ✓ PR summary workflow

If missing, copy them from this repository or create them (see Configuration below).

Step 4: Test on a PR ​

Open a new PR against main (or your target branch):

  1. Push a branch with a change
  2. Open a PR
  3. Watch GitHub Actions
    • Go to PR → Actions tab
    • Find Copilot PR Summary workflow
    • Wait for Summary PR job to complete
  4. Check the PR comment
    • Scroll to comments section
    • You should see the Copilot summary

Configuration ​

Default Workflow ​

The model is defined once as a job-level env var and reused in both the action call and the footer, avoiding duplication:

yaml
jobs:
  summary:
    env:
      MODEL: gpt-5-mini   # change here to update everywhere

    steps:
      - uses: ./.github/actions/copilot
        with:
          model: ${{ env.MODEL }}
          options: '{"type":"summary","format":"markdown"}'

Customize the Summary Type ​

Edit .github/workflows/copilot-pr-summary.yml and change the options input:

Generate a changelog entry:

yaml
options: '{"type":"changelog","format":"markdown"}'
model: gpt-5-mini

Generate documentation:

yaml
options: '{"type":"docs","format":"markdown"}'
model: gpt-5-mini

Generate test suggestions:

yaml
options: '{"type":"tests","format":"typescript"}'
model: gpt-5-mini

Add Custom Instructions ​

Extend the prompt with domain-specific guidance:

yaml
options: |
  {
    "type": "summary",
    "instructions": "Focus on breaking changes, new features, and deprecations. Highlight any security implications."
  }
model: gpt-5-mini

Use a Different Model ​

Change the model (see the centralized default in src/shared/constants.ts):

yaml
# Example models:
model: gpt-5.1      # Newer, larger model
model: gpt-4.1      # Older, smaller model
# Default is defined in src/shared/constants.ts

How It Works ​

Workflow Stages ​

1. Checkout                    → Fetch repo code
2. Generate PR summary         → Copilot SDK reads diff + generates
3. Write summary to file       → Capture output
4. Post summary comment        → Add/update PR comment
5. Write job summary           → Make visible in Actions tab
6. Report failure (if needed)  → Exit cleanly on errors

Token Flow ​

COPILOT_TOKEN (repo secret)
  │
  ├─ Passed to Copilot Action
  │
  ├─ Set as COPILOT_GITHUB_TOKEN env var (SDK auto-detects)
  │
  └─ Copilot API validates subscription
       ↓
       Model from src/shared/constants.ts (default) or workflow override
       ↓
       Generated summary

PR Comment Management ​

  • First run: New comment is posted
  • Subsequent runs: Old comment is deleted, new one posted
    • Keeps the PR clean
    • Always shows the latest generated summary
    • Based on the ## 🤖 Copilot PR Summary header

Troubleshooting ​

Workflow doesn't appear in Actions ​

❌ Workflow files may not be in the default branch yet.

✅ Fix:

bash
git add .github/workflows/copilot-pr-summary.yml
git commit -m "feat: add Copilot PR summary workflow"
git push

"Token not found" or "Invalid token" ​

❌ The secret isn't set, or the name is wrong.

✅ Fix:

  1. Verify COPILOT_TOKEN secret is set in repo settings
  2. Confirm it's added to the correct repository (not organization-level)
  3. Workflow references ${{ secrets.COPILOT_TOKEN }}

"Copilot subscription not found" ​

❌ The token's account doesn't have GitHub Copilot active.

✅ Fix:

  1. Verify your personal account has Copilot Individual or Copilot Business
  2. Re-create the PAT from the same account
  3. Verify PAT has repo scope

Comment not posting to PR ​

❌ Permission or comment content issue.

✅ Debug:

  1. Check Actions logs for errors
  2. Verify GITHUB_TOKEN has pull-requests:write permission
  3. Verify the generated summary isn't empty

Workflow runs but produces no summary ​

❌ Copilot API error or diff parsing failure.

✅ Debug:

  1. Open the Actions tab → Copilot PR Summary run
  2. Click Summary job → Generate PR summary step
  3. Review the logs for error messages
  4. Check that PR has actual code changes (empty PRs may fail)

Advanced Usage ​

Use in a CI/CD Pipeline ​

Combine with other tools:

yaml
jobs:
  quality-checks:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - name: Lint
        run: npm run lint

      - name: Type check
        run: npm run typecheck

      - name: Generate summary (example: gpt-5-mini)
        uses: ./.github/actions/copilot
        with:
          tool: generate
          token: ${{ secrets.COPILOT_TOKEN }}
          model: gpt-5-mini
          options: '{"type":"summary"}'

Use the Resume Tool for Multi-Step Reviews ​

yaml
- name: First review pass
  uses: ./.github/actions/copilot
  with:
    tool: review
    token: ${{ secrets.COPILOT_TOKEN }}
    model: gpt-5-mini

- name: Security-focused review
  uses: ./.github/actions/copilot
  with:
    tool: resume
    token: ${{ secrets.COPILOT_TOKEN }}
    model: gpt-5-mini
    options: '{"prompt":"Now check for security issues","focus":"security"}'

Export Summary to Artifact ​

yaml
- name: Save summary
  if: always()
  run: |
    echo "${{ steps.copilot.outputs.generated }}" > /tmp/summary.md

- name: Upload artifact
  uses: actions/upload-artifact@v4
  with:
    name: pr-summary
    path: /tmp/summary.md

Performance & Cost ​

AspectNotes
SpeedUsually completes in 5–15 seconds
CostUses the default model defined in src/shared/constants.ts
QuotaUses your GitHub Copilot quota
ConcurrencyOne summary per PR number (cancels older runs)
LogsCheck Actions UI for full Copilot API responses

Testing Locally ​

Test the generate tool without pushing ​

bash
# Uses the default model from src/shared/constants.ts
GITHUB_TOKEN=$(gh auth token) bun run summary \
  --owner your-org \
  --repo your-repo \
  --pr 42 \
  --type summary \
  --model gpt-5-mini

Run the workflow in a test environment ​

Use GitHub's workflow dispatcher:

yaml
# Add to copilot-pr-summary.yml
on:
  workflow_dispatch:  # Manual trigger for testing
    inputs:
      pr-number:
        description: 'PR number to test'
        required: true

Then trigger manually from Actions → Copilot PR Summary → Run workflow.

See Also ​

Setup ​

Step 1: Create a GitHub PAT ​

Requirement: Your personal account must have GitHub Copilot active (Individual or Business plan).

  1. Go to github.com → your avatar → Settings → Developer settings → Tokens (classic)
  2. Click Generate new token (classic)
  3. Configure:
    • Name: copilot-ci (or any name)
    • Expiration: 90 days (recommended) or no expiration
    • Scopes: Check ✅ repo (no other scopes needed)
  4. Click Generate token
  5. Copy the token (it won't show again) — starts with ghp_

⚠️ The built-in GITHUB_TOKEN in Actions lacks Copilot permissions. You must create a Classic PAT.

Step 2: Add Secret to Your Repo ​

  1. Go to your repo → Settings → Secrets and variables → Actions
  2. Click New repository secret
    • Name: COPILOT_TOKEN
    • Value: Paste the ghp_ token from Step 1
  3. Click Add secret

Step 3: Verify Workflow Files Are Present ​

Check that these files exist in your repo:

.github/
├── actions/
│   └── copilot/
│       └── action.yml          ✓ Copilot action definition
└── workflows/
    └── copilot-pr-summary.yml  ✓ PR summary workflow

If missing, copy them from this repository or create them (see Configuration below).

Step 4: Test on a PR ​

Open a new PR against main (or your target branch):

  1. Push a branch with a change
  2. Open a PR
  3. Watch GitHub Actions
    • Go to PR → Actions tab
    • Find Copilot PR Summary workflow
    • Wait for Summary PR job to complete
  4. Check the PR comment
    • Scroll to comments section
    • You should see the Copilot summary

Configuration ​

Default Workflow ​

The default workflow (copilot-pr-summary.yml) generates a summary of the PR changes:

yaml
options: '{"type":"summary","format":"markdown"}'
model: gpt-4.1

Customize the Summary Type ​

Edit .github/workflows/copilot-pr-summary.yml and change the options input:

Generate a changelog entry:

yaml
options: '{"type":"changelog","format":"markdown"}'

Generate documentation:

yaml
options: '{"type":"docs","format":"markdown"}'

Generate test suggestions:

yaml
options: '{"type":"tests","format":"typescript"}'

Add Custom Instructions ​

Extend the prompt with domain-specific guidance:

yaml
options: |
  {
    "type": "summary",
    "instructions": "Focus on breaking changes, new features, and deprecations. Highlight any security implications."
  }

Use a Different Model ​

Change the model (requires Copilot access to that model):

yaml
model: gpt-5.1

How It Works ​

Workflow Stages ​

1. Checkout                    → Fetch repo code
2. Generate PR summary         → Copilot SDK reads diff + generates
3. Write summary to file       → Capture output
4. Post summary comment        → Add/update PR comment
5. Write job summary           → Make visible in Actions tab
6. Report failure (if needed)  → Exit cleanly on errors

Token Flow ​

COPILOT_TOKEN (repo secret)
  │
  ├─ Passed to Copilot Action
  │
  ├─ Set as COPILOT_GITHUB_TOKEN env var (SDK auto-detects)
  │
  └─ Copilot API validates subscription
       ↓
       gpt-4.1 / gpt-5.1 / etc.
       ↓
       Generated summary

PR Comment Management ​

  • First run: New comment is posted
  • Subsequent runs: Old comment is deleted, new one posted
    • Keeps the PR clean
    • Always shows the latest generated summary
    • Based on the ## 🤖 Copilot PR Summary header

Troubleshooting ​

Workflow doesn't appear in Actions ​

❌ Workflow files may not be in the default branch yet.

✅ Fix:

bash
git add .github/workflows/copilot-pr-summary.yml
git commit -m "feat: add Copilot PR summary workflow"
git push

"Token not found" or "Invalid token" ​

❌ The secret isn't set, or the name is wrong.

✅ Fix:

  1. Verify COPILOT_TOKEN secret is set in repo settings
  2. Confirm it's added to the correct repository (not organization-level)
  3. Workflow references ${{ secrets.COPILOT_TOKEN }}

"Copilot subscription not found" ​

❌ The token's account doesn't have GitHub Copilot active.

✅ Fix:

  1. Verify your personal account has Copilot Individual or Copilot Business
  2. Re-create the PAT from the same account
  3. Verify PAT has repo scope

Comment not posting to PR ​

❌ Permission or comment content issue.

✅ Debug:

  1. Check Actions logs for errors
  2. Verify GITHUB_TOKEN has pull-requests:write permission
  3. Verify the generated summary isn't empty

Workflow runs but produces no summary ​

❌ Copilot API error or diff parsing failure.

✅ Debug:

  1. Open the Actions tab → Copilot PR Summary run
  2. Click Summary job → Generate PR summary step
  3. Review the logs for error messages
  4. Check that PR has actual code changes (empty PRs may fail)

Advanced Usage ​

Use in a CI/CD Pipeline ​

Combine with other tools:

yaml
jobs:
  quality-checks:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - name: Lint
        run: npm run lint

      - name: Type check
        run: npm run typecheck

      - name: Generate summary
        uses: ./.github/actions/copilot
        with:
          tool: generate
          token: ${{ secrets.COPILOT_TOKEN }}
          options: '{"type":"summary"}'

Use the Resume Tool for Multi-Step Reviews ​

yaml
- name: First review pass
  uses: ./.github/actions/copilot
  with:
    tool: review
    token: ${{ secrets.COPILOT_TOKEN }}

- name: Security-focused review
  uses: ./.github/actions/copilot
  with:
    tool: resume
    token: ${{ secrets.COPILOT_TOKEN }}
    options: '{"prompt":"Now check for security issues","focus":"security"}'

Export Summary to Artifact ​

yaml
- name: Save summary
  if: always()
  run: |
    echo "${{ steps.copilot.outputs.generated }}" > /tmp/summary.md

- name: Upload artifact
  uses: actions/upload-artifact@v4
  with:
    name: pr-summary
    path: /tmp/summary.md

Testing Locally ​

Test the generate tool without pushing ​

bash
# Requires GITHUB_TOKEN and a real PR
GITHUB_TOKEN=$(gh auth token) bun run summary \
  --owner your-org \
  --repo your-repo \
  --pr 42 \
  --type summary

Run the workflow in a test environment ​

Use GitHub's workflow dispatcher:

yaml
# Add to copilot-pr-summary.yml
on:
  workflow_dispatch:  # Manual trigger for testing
    inputs:
      pr-number:
        description: 'PR number to test'
        required: true

Then trigger manually from Actions → Copilot PR Summary → Run workflow.

Performance & Limits ​

AspectNotes
SpeedUsually completes in 10–30 seconds
CostUses your GitHub Copilot quota
ConcurrencyOne summary per PR number (cancels older runs)
LogsCheck Actions UI for full Copilot API responses

See Also ​