Coverage Agent: LLM-Assisted Test Generation for Django

Writing tests for existing Django code is tedious: you run coverage, squint at the report, figure out which gaps actually matter, write boilerplate, run the suite, fix import errors, repeat. Coverage Agent automates the tedious parts while keeping you in control of every decision.

It runs as a three-step CLI pipeline. Each step pauses for your input before moving forward. Session state lives in a local SQLite database, so you can close the terminal between steps and resume later.

How it works#

suggest  →  generate  →  apply

1. suggest — runs coverage run against your Django project, analyses the gaps, and presents up to 5 ranked test suggestions with rationale and complexity estimate.
2. generate — you pick the suggestions you want by ID; the LLM writes complete test files and prints them for review before anything is written to disk.
3. apply — test files are written to your project and manage.py test is run. On success, coverage run --append computes an accurate before/after delta. On failure, you can retry (the LLM sees the full traceback) or discard.

Pipeline flowchart#

flowchart TD
    A([suggest]) --> B[collect_coverage]
    B --> C[suggest_tests]
    C --> D[/PAUSE: pick suggestions/]
    D --> E([generate])
    E --> F[generate_tests]
    F --> G[/PAUSE: review generated files/]
    G --> H([apply])
    H --> I[insert_and_validate]
    I --> J{Tests pass?}
    J -->|Yes| K[coverage --append]
    K --> L([END: delta reported])
    J -->|No| M{Retries left?}
    M -->|Yes| N[/PAUSE: retry or discard?/]
    N -->|retry| F
    N -->|discard| P[discard_tests]
    M -->|No| P[discard_tests]
    P --> L

    style D fill:#f0f4ff,stroke:#6b7aff
    style G fill:#f0f4ff,stroke:#6b7aff
    style N fill:#f0f4ff,stroke:#6b7aff

Every [PAUSE] node is a session checkpoint: the process exits, state is saved, and the next CLI call resumes from exactly that point.

Installation#

Requirements: Python 3.12+, pipenv, an Anthropic API key.

git clone <repo-url>
cd unit-tester
pipenv install
pipenv shell

Configuration#

Create .env next to cli.py:

# Required
ANTHROPIC_API_KEY=sk-ant-...
DJANGO_PROJECT_ROOT=/path/to/your/django/project

# Python inside your Django project's virtualenv
# Find it with: cd /path/to/project && pipenv --venv
# then append /bin/python
DJANGO_PYTHON=/Users/you/.local/share/virtualenvs/myproject-HASH/bin/python

# Optional — sensible defaults
COVERAGE_AGENT_MODEL=claude-opus-4-7
COVERAGE_OMIT=*/migrations/*,*/tests/*,manage.py
STATE_DB_PATH=.coverage_agent_state.db
MAX_RETRIES=2

A typical session#

# Step 1: collect coverage and see suggestions
# Always use --debug on the first run to verify coverage is actually running
python cli.py suggest --debug

# Step 2: generate test code for suggestions 1, 2, and 4
python cli.py generate --picks 1,2,4

# Review the printed test code, then apply
# --retry 2 will auto-regenerate up to 2 times if tests fail
python cli.py apply --retry 2 --debug

# Check session state at any time
python cli.py status

Commands#

suggest#

Runs coverage run, parses the results, and asks the LLM to suggest up to 5 high-value test cases. Streams Django’s test output live to your terminal. Pre-existing test failures are summarised after the run — they don’t block coverage collection.

python cli.py suggest
python cli.py suggest --explain          # detailed paragraph per suggestion
python cli.py suggest --focus orders     # scope coverage to a single Django app
python cli.py suggest --reset            # discard session and start fresh
python cli.py suggest --thread feature  # named session for parallel work

generate#

Writes test code for your selected suggestions. Prints the generated files before writing anything to disk.

python cli.py generate --picks 1,3,5

Pass --samples to point the LLM at existing test files so the generated code follows your project’s conventions — imports, base classes, fixture patterns:

# Expand a directory (up to 5 largest test files by default)
python cli.py generate --picks 1,3 --samples tests/

# Specific files
python cli.py generate --picks 1,3 --samples tests/test_orders.py,tests/test_users.py

# Raise the cap
python cli.py generate --picks 1,3 --samples tests/ --samples-limit 10

Set --samples at suggest time to store the file list in the session — you won’t need to repeat it on each generate call.

apply#

Writes test files to the Django project and runs manage.py test. On success, merges coverage with --append and reports the delta. On failure, shows the full traceback and waits for retry or discard.

python cli.py apply
python cli.py apply --retry 3           # auto-retry generation up to 3 times
python cli.py apply --keepdb            # reuse existing test database (faster)

retry and discard#

After a failed apply:

• retry — asks the LLM to regenerate using the full traceback. Run apply again after.
• discard — deletes the written test files and resets to the generate step.

python cli.py retry
python cli.py apply --debug

# or, start the generation over
python cli.py discard
python cli.py generate --picks 1,2

retest#

Re-runs the last batch of test files without touching the agent or regenerating anything. Useful after manually editing a generated file.

python cli.py retest --debug

status#

Shows the current session phase, coverage summary, suggestion list, last test result, and coverage delta.

python cli.py status
python cli.py status --explain

Flags reference#

Debug mode#

Run with --debug on the first session against any new Django project. The most common issues are invisible without it:

• Coverage finishes in under 5 seconds — almost always means coverage run failed silently. --debug reveals the actual error.
• Tests fail with a cryptic error — --debug shows the full manage.py test output including tracebacks and fixture errors.

python cli.py suggest --debug
python cli.py apply --retry 2 --debug

Multiple sessions#

Use --thread to run independent sessions against different branches or targets simultaneously:

python cli.py suggest --thread payments-module
python cli.py suggest --thread auth-module
python cli.py status --thread payments-module

# Reset a named session
python cli.py suggest --thread payments-module --reset

Each thread has its own state in the SQLite database.