Testing Regimen

Pro Only

The Testing Regimen setting on the AI Agent tab in Project Properties controls how much the agent writes and runs unit tests as it works. The chosen regimen is written into the guides in the .wing directory, which every agent session started from Wing reads.

The regimens trade thoroughness for speed. Several of them run only stale tests, which are the tests that recorded code-coverage data shows depend on code you changed. Coverage records only what past test runs executed, so stale-test detection can miss tests that have not run recently and code paths the tracer could not follow, such as threads and subprocesses. None of the regimens replace running the full test suite before a release, a merge to a long-lived branch, or any other point where thoroughness matters more than turnaround time.

None -- The agent never writes or runs tests on its own. Use this if you run tests as part of a separate QA or release process.
Minimal -- The agent writes tests when adding new code and runs the new tests at task end, but does not run tests before review or commit. This is the default.
Balanced -- Same as Minimal at task end. Before review, and before committing without review, the agent also runs stale tests in the directories you changed. These are the tests in the same directory as a file you edited or in its sibling test/ directory.
Thorough -- At task end, runs the new tests plus stale tests in the directories you changed. Before review, and before committing without review, runs all stale tests anywhere in the project.
Obsessive -- At task end, runs every test in the directories you changed, stale or not. Before review, and before committing without review, runs the project's full test suite.
Custom -- Sets each stage separately, with the four settings below, which are ignored for the other regimens.

Custom: Write Tests

Whether the agent writes unit tests on its own: Only when the user asks or Always when adding new code.

Custom: Run at End of Task

Which tests the agent runs when finishing a task: Don't run tests, Run new tests only, Run stale tests in changed directories, Run new & stale tests in changed directories, Run all stale tests, Run new and all stale tests, Run all tests in directories with changed files, or Run all tests in the project.

The four stale options differ in scope and in whether they also run the new tests the agent wrote:

Run stale tests in changed directories runs only the stale tests in the directories you changed. It does not run new tests, so use it at a stage where they have already run, such as before review when the task-end stage ran them.
Run new & stale tests in changed directories adds the new tests, so it works as a self-contained check.
Run all stale tests runs stale tests anywhere in the project, but not the new tests. In a large project even a small edit can make many tests stale.
Run new and all stale tests adds the new tests to that, and is the most thorough of the stale options.

Custom: Run Before Review

Which tests the agent runs before handing changes off for review, with the same choices.

Custom: Run Before Commit (without review)

Which tests the agent runs before committing without a review, with the same choices.

Max Stale Tests to Run (seconds)

The longest estimated run time of stale tests that an agent may run without asking you. This applies to every regimen. By default, it is 120. Set it to 0 to have the agent always ask before running stale tests.

Stale-Test Safeguard

A small edit to a widely imported module can make hundreds of tests stale. When the stale tests would take longer than Max Stale Tests to Run, Wing runs only as many as fit, taking those in the directories you changed first and the cheapest first within each group. It tells the agent what it skipped, and the agent asks you before running the rest. Run new & stale tests in changed directories is limited to the directories you changed in any case.

Code Coverage During Agent Test Runs

Finding stale tests needs code-coverage data, which Wing collects when agents run tests. With Python 3.12 or later and coverage.py 7.4 or later, coverage costs little, so Wing always collects it, and stale tests can be found as soon as you switch to a regimen that uses them. With older Python versions, coverage roughly doubles the time tests take, so Wing collects it only while the regimen runs stale tests at some stage.

The Testing tool's own coverage display in editors is controlled by a separate Testing preference, and the regimen does not affect it.