Debugging Ralph

Learn how to check status, troubleshoot issues, and understand what Ralph is doing

Debugging Ralph

Ralph runs autonomously, but you need to monitor progress and debug issues when they occur.

Quick Status Check

See which stories are complete and which are pending:

cat scripts/ralph/prd.json | jq '.userStories[] | {id, title, passes}'

Output:

{
  "id": "US-001",
  "title": "Add status field to tasks table",
  "passes": true
}
{
  "id": "US-002",
  "title": "Display status badge on task cards",
  "passes": true
}
{
  "id": "US-003",
  "title": "Add status toggle to task list rows",
  "passes": false
}

passes: true means the story is complete. Ralph picks the highest priority story where passes: false.

Check Recent Progress

See what Ralph learned in recent iterations:

# View full progress log
cat scripts/ralph/progress.txt

# View just the last 50 lines
tail -50 scripts/ralph/progress.txt

# Search for a specific story
cat scripts/ralph/progress.txt | grep -A10 "US-003"

Check Git History

See what Ralph committed:

# Last 10 commits
git log --oneline -10

# Commits from Ralph feature branch
git log main..ralph/task-status --oneline

# See what changed in the last commit
git show HEAD

Common Issues

Ralph keeps failing the same story

Symptoms: Same story has passes: false after multiple iterations

Debugging steps:

1. Check progress.txt for errors

   cat scripts/ralph/progress.txt | grep -A20 "US-003"
   

   Look for repeated error messages or failed quality checks.

2. Check if story is too large

   The story might be too big for one context window. Check if Ralph is getting close but running out of context.

   Solution: Split the story into smaller pieces in prd.json.

3. Check acceptance criteria

   Are the criteria verifiable? Vague criteria cause confusion.

   ❌ Bad: “Make it work correctly”

   ✅ Good: “Status dropdown shows three options: Pending, In Progress, Done”

4. Check quality commands

   Verify your project’s quality checks actually work:

   npm run typecheck
   npm run lint
   npm test
   

   If they fail, Ralph can’t pass them either.

Ralph is not finding the prd.json file

Symptoms: Error about missing prd.json

Solution:

# Check if prd.json exists
ls scripts/ralph/prd.json

# If missing, create one from your PRD:
# Load the ralph skill and convert tasks/prd-feature.md to prd.json

# Or manually create in the correct location
cat scripts/ralph/prd.json

The prd.json file must be in the same directory as ralph.sh and prompt.md/CLAUDE.md.

Quality checks are passing but Ralph says they failed

Symptoms: You can run npm run typecheck successfully, but Ralph reports failures

Debugging:

1. Check exit codes

   Ralph checks exit codes. Make sure your commands exit with code 0 on success:

   npm run typecheck
   echo $?  # Should be 0
   
2. Check command output

   Some commands output warnings that Ralph might interpret as failures.

   Add --quiet flags or redirect stderr if needed:

   npm run typecheck 2>/dev/null
   
3. Test commands in isolation

   Run each quality check command manually to see what Ralph sees:

   cd your-project
   npm run typecheck
   npm run lint
   npm test -- --passWithNoTests
   

Ralph committed broken code

Symptoms: CI is failing, or you manually found bugs

This is serious. Broken code compounds across iterations.

Solution:

1. Stop Ralph immediately

   Press Ctrl+C to stop the loop

2. Fix the broken commit manually

   # Check out the branch
   git checkout ralph/your-feature
   
   # Fix the issue
   # ...
   
   # Commit the fix
   git add .
   git commit -m "fix: correct issue from US-003"
   
3. Update progress.txt

   Add a manual entry explaining what was fixed:

   ## 2026-03-11 15:00 - Manual Fix
   - Fixed bug in US-003 where status dropdown was not updating
   - Issue: Missing revalidatePath() call
   - **Learning**: Always call revalidatePath() after mutations
   ---
   
4. Improve quality checks

   Update your prompt.md/CLAUDE.md to catch this type of error:

   ## Common Gotchas
   - After mutations, always call revalidatePath("/path")
   
5. Resume Ralph

   ./scripts/ralph/ralph.sh
   

Prevention: Make sure your quality checks actually catch errors. Add tests, linting rules, and type checks.

Ralph is not updating AGENTS.md files

Symptoms: progress.txt shows learnings but AGENTS.md files are not being updated

Debugging:

1. Check if AGENTS.md files exist

   find . -name "AGENTS.md" -type f
   

   Ralph only updates existing files. Create them if needed:

   echo "# Agent Instructions" > src/app/AGENTS.md
   
2. Check your prompt template

   Ensure prompt.md (for Amp) or CLAUDE.md (for Claude Code) includes instructions to update these files.

   See Customizing Prompts for details.

3. Check if learnings are generic enough

   Ralph should only add reusable patterns, not story-specific details.

   ✅ Good: “Use revalidatePath() after mutations”

   ❌ Too specific: “Added a blue badge to the task card”

Browser verification is failing

Symptoms: Stories with “Verify in browser” are not completing

Solution: See Browser Verification for detailed debugging steps.

Quick checks:

# Is dev server running?
lsof -i :3000

# Can you access it?
curl http://localhost:3000

# Is dev-browser skill installed?
ls ~/.config/amp/skills/ | grep browser

Reading Ralph’s Output

When Ralph runs, you’ll see output from both the bash loop and the AI tool:

===============================================================
  Ralph Iteration 3 of 10 (amp)
===============================================================

[Amp output begins here...]

Reading prd.json...
Picking story US-003: Add status toggle to task list rows
Implementing changes...
Running typecheck... ✓
Committing changes...
Updating prd.json...
Appending to progress.txt...

[Amp output ends]

Iteration 3 complete. Continuing...

If you see Ralph repeating the same story across multiple iterations, check progress.txt for what’s blocking it.

Using jq for Advanced Queries

Count completed vs remaining stories

# Total stories
cat scripts/ralph/prd.json | jq '.userStories | length'

# Completed stories
cat scripts/ralph/prd.json | jq '[.userStories[] | select(.passes == true)] | length'

# Remaining stories
cat scripts/ralph/prd.json | jq '[.userStories[] | select(.passes == false)] | length'

Show only incomplete stories

cat scripts/ralph/prd.json | jq '.userStories[] | select(.passes == false) | {id, title, priority}'

Show stories with notes

cat scripts/ralph/prd.json | jq '.userStories[] | select(.notes != "") | {id, title, notes}'

Get next story Ralph will work on

cat scripts/ralph/prd.json | jq '[.userStories[] | select(.passes == false)] | sort_by(.priority) | .[0] | {id, title}'

Monitoring Long-Running Ralph

If Ralph is running for many iterations:

# In another terminal, watch progress in real-time
watch -n 5 'cat scripts/ralph/prd.json | jq "[.userStories[] | select(.passes == true)] | length"'

# Or tail the progress file
tail -f scripts/ralph/progress.txt

When to Intervene

1. Same story fails 3+ times

   Stop Ralph and investigate. The story is likely too big or has unclear criteria.

2. CI breaks

   Stop immediately. Fix the commit manually before continuing.

3. Ralph completes a story incorrectly

   Stop, revert the commit, update acceptance criteria to be more specific, and resume.

4. Context seems wrong

   Check progress.txt Codebase Patterns section. Ralph may have learned incorrect patterns from previous iterations.

Never let Ralph run overnight until you’ve verified it works correctly on your project. Start with small PRDs and supervise the first few iterations.

Debugging Checklist

When things go wrong:

• Check which story Ralph is working on: jq '.userStories[] | {id, passes}' prd.json
• Read recent progress: tail -50 progress.txt
• Check git history: git log --oneline -5
• Verify quality checks work manually: npm run typecheck && npm test
• Check if story is too large (see Right-Sizing Stories)
• Check if acceptance criteria are specific enough
• Look for repeated errors in progress.txt
• Verify dev server is running (for UI stories)

Next Steps

• Right-Sizing Stories - Prevent story size issues
• Customizing Prompts - Add better error handling
• Archiving - Learn how Ralph manages previous runs