Testing AI models without losing sight of workflow costs

Retrospective · week of 14 September 2026

A prototype that answers three messages correctly cannot tell you what a month of operation will cost. For a small business, the useful question is not simply “which model is cheapest?” It is “what do we spend to handle a request to the required standard?” That distinction changes how an experiment should be designed and assessed.

What the announcement makes easier

On 17 September, n8n introduced Gateway credits. From version 2.36, selected Cloud plans support compatible models and services without a separate provider account. Availability depends on the plan and service. See the official announcement.

An easier start should not replace financial planning. Before testing, define what “handled” means: an answer sent, a complete record, or a request handed over and accepted by a salesperson? The last outcome is often closer to the business value you want than the number of successful workflow executions.

Build a calculation the team can understand

We suggest separating five categories in a tracking sheet:

• The subscription share allocated to the process, using a consistent method.
• Consumption of models and any other services involved.
• Any execution charges, without counting costs already included elsewhere twice.
• Human time spent reviewing, correcting and handling exceptions.
• Recurring maintenance, kept separate from the initial build cost.

Divide the full cost for the period by the number of requests handled correctly. This produces a useful operating measure. Keep the initial investment separate too, and state explicitly whether you choose to spread it over time. Without that convention, two estimates may look inconsistent when they simply measure different things.

Illustrative example: two prototypes, one required outcome

Imagine two versions of a qualification assistant evaluated against the same fictional requests. The first consumes fewer credits but leaves more fields needing correction. The second consumes more but more often produces a usable record. Neither finding is enough on its own to identify the better option.

For each request, record consumption, retries, review time and the final result. An incomplete record does not become a success because the model answered quickly. Conversely, correctly routing a request to a person can count as success if that handover was part of the intended process. This example illustrates an evaluation method, not a SWM customer result.

Run a fair comparison

Use a small but representative set of examples: short requests, confusing messages, missing information and cases outside scope. Define acceptance criteria before comparing models. Keep reference documents and business rules constant, or you may end up measuring several changes at once.

Set both a spending cap and a retry limit for each request. A persistent failure should enter a review queue rather than trigger an endless loop. Document what should happen when the test budget is reached: suspend new processing, notify the owner and preserve pending requests. That behaviour is part of the test, not an administrative detail to resolve later.

Decide before scaling up

The assessment should combine cost, quality and supervision workload. If the team cannot absorb exceptions, a low technical cost does not make the process viable. Where demand varies substantially, model several volume scenarios instead of blindly multiplying a single average. Keep the assumptions visible so that someone else can challenge them.

This is part of AI consulting and project scoping, before turning the prototype into a dependable no-code solution. It gives your team a budget to discuss, with explicit assumptions, rather than an unsupported savings promise.

Let's estimate the operating cost of your automation before increasing its volume.