/// Resources / Cost
How to avoid per-run test execution fees in CI
A test suite that costs nothing to run is the only kind you run on every commit. Most commercial test tools are priced the other way round: the recorder is cheap or free, and the bill arrives when the tests execute - per parallel runner, per device minute, per CI node, per "test run". This article names the four models, shows what they do to a suite that grows, and lays out the setup that makes execution free: generated open code, your own agents, a Jenkins matrix for parallelism.
The four ways a tool charges you to run tests
Vendors rarely say "execution fee" on the pricing page. It shows up as one of these:
- Runtime engine licences. The tests only execute through the vendor's runtime, licensed per CI node or per concurrent run. Katalon's TestCloud and Runtime Engine are the well-known example (see katalon.com/pricing, late 2026).
- Parallel-session pricing. Cloud grids such as BrowserStack and Sauce Labs charge per parallel browser or device session per month; more parallelism costs more every month whether you use it or not.
- Metered minutes. Real-device clouds (Kobiton and others) meter device time; a suite that doubles in length doubles the bill.
- Per-run or per-test counts. Some AI-driven tools price by "test runs" or "test cases per month", which turns a healthy habit - run everything on every pull request - into an overage.
What growth does to each model
Take a suite that starts at 200 scenarios and grows to 1,000 over two years, run on 40 pull requests a day with four-way parallelism:
| Model | Year 1 | Year 2 (5x the tests) |
|---|---|---|
| Runtime engine, 4 nodes | 4 node licences | Need 8-12 nodes to keep the wall-clock time - 2-3x |
| 4 parallel cloud sessions | ~$800+/mo | Longer runs queue behind the 4-session cap, or you buy 12 |
| Metered device minutes | Baseline | Roughly 5x the minutes |
| Open code on your own agents | $0 per run | $0 per run; add an agent you already own |
Only the last row is flat. The agents it needs are the same Jenkins or GitHub Actions runners that already build the application.
The setup that makes execution free
1. Tests must be ordinary code
If the tests are a proprietary artifact, they need the vendor's runtime to execute, and that runtime is where the meter lives. Insist on tests that are plain Selenium, REST Assured or Appium in a normal Maven project - something mvn test runs on any machine with a JDK. That is what L3git generates from a recording; the project runs with or without the tool.
2. Parallelism from the CI server, not a hub
A Selenium Grid is free software but not free to run: a hub to patch, nodes to version-match, a single point of failure. A declarative Jenkins matrix gives you the same fan-out using agents you label, with no hub at all. The parallel Selenium in Jenkins article has the full pipeline; the short version is an axis of shards, an axis of browsers (or devices), and a deterministic selector that gives each cell its own slice of scenarios.
axes {
axis { name 'SHARD'; values '1', '2', '3', '4' }
axis { name 'BROWSER'; values 'edge', 'chrome' }
}
3. Browsers and phones you already have
For web, the IT-installed Edge or Chrome on each agent is the browser; air-gapped agents only need the matching WebDriver on disk. For Android, phones on a USB hub or emulators on the agent, with Appium bundled so nothing is installed per machine. A cloud grid stays an option for the one job that genuinely needs forty browser versions - the same project can target it through a single property - but it stops being the thing every run passes through.
4. Pay for people, not executions
The remaining cost is the seat for whoever records and maintains the tests. Per-seat pricing scales with the team, which is a number you control; per-run pricing scales with your test discipline, which is a number you want to grow.
A worked comparison
A five-person QA team, web + API coverage, four-way parallel runs all day:
- Recorder + runtime-engine vendor: 5 seats plus 4 runtime nodes; adding a fifth node to keep runs fast is another licence.
- Open code + cloud grid: 4 parallel sessions at roughly $199 each per month (late-2026 list price) is about $9,500 a year before devices.
- L3git UI + API: 5 seats at $169 per user per month (annual) is $10,140 a year total, and the runs - four, eight or twelve wide - cost nothing extra on agents you own.
The numbers move with your team size and vendor discounts; the shape does not. Two of the three models get more expensive the more you test.
Checklist before you sign
- Can I run the generated tests with
mvn teston a laptop with no vendor software installed? - Is there any charge tied to parallel runs, nodes, minutes or run counts?
- Does the suite keep working if the subscription lapses?
- Can the tests run on an agent with no internet access?
Four yeses and the execution line on your invoice is zero. L3git is built to answer yes to all four; the Selenium Grid alternative and cloud device grid alternative pages go deeper on the parallelism and device sides.