Regression testing: automate it without losing a sprint

A regression is a feature that worked and stops working after a change. Regression tests are there to catch it before your customers do. The problem in many young teams: the codebase grew faster than the tests, and catching up seems to need a whole sprint nobody has.

Here is a way to make progress in small deliveries.

1. Start with the paths that cost the most

List the five paths whose failure would cost you most: sign-up, log-in, the product's main action, data exports and notifications. One end-to-end test per path already protects what matters.

2. Pin current behaviour before you change it

On older code, write characterisation tests first: they describe what the code does today, not what it should do. You can then change the code and know at once if a behaviour moved.

3. Keep the suite fast

A slow suite ends up switched off. Aim for a few minutes in continuous integration:

• unit tests for business logic;
• a few integration tests for the database and API calls;
• few end-to-end tests, kept for the critical paths.

4. Run it on every pull request

A test that doesn't run automatically protects nothing. Wire the suite into CI and block the merge when it fails.

5. Deal with flaky tests straight away

A test that fails at random teaches the team to ignore failures. Quarantine it, fix the cause (waits, execution order, shared data), then put it back.

Where does AI fit?

AI is useful for drafting tests from existing code and tickets: edge cases, test data, characterisation tests. It doesn't replace review: a wrong test that passes is worse than no test. The rule is the same as for human code: every test arrives as a pull request, and your team reviews it before merging.

Key takeaways

• Five critical paths covered beat a high coverage figure with no value.
• Fast tests, run on every pull request.
• Zero tolerance for flaky tests.

We deliver tests and migrations as pull requests, reviewed and merged by your team: see what we do for tech teams.