AI pipeline: from 24 hours to under 3 hours

The context

An AI pipeline was used to automatically transcribe and classify a full day of calls.

Its execution time initially exceeded 24 hours, limiting processing frequency and keeping compute resources active for long periods.

The objective was twofold: drastically reduce processing time and make execution more economically efficient without degrading classification quality.

Rethinking the end-to-end processing

Optimisation was not simply a matter of adding more compute.

I revisited the project architecture to identify the main performance levers:

• use of an LLM better sized for the classification need;
• evolution of Speech-to-Text processing;
• processing both audio channels of a call;
• intelligent reconstruction of the conversation to preserve context;
• optimisation of packaging and the Docker image;
• fixing execution issues that unnecessarily kept processes alive;
• rationalising the execution cycle so resources matched the actual need.

Balancing quality, performance and cost

The main challenge was finding the right balance between processing quality, execution time and compute cost.

The choice of a lighter model had to remain compatible with classification requirements. Likewise, the new transcription approach had to improve the representation of the conversation while remaining compatible with performance goals.

Reconstructing both channels also makes it possible to preserve the overall context of the exchange instead of processing audio streams independently.

Results

>24 h → <3 h

A full day of calls is now processed in under three hours of compute, compared with more than 24 hours previously.

>8× faster

Processing time was reduced by more than eight times.

On-demand compute

The shorter execution time limits compute-resource usage to the actual processing window.

Containerised applications can therefore return to zero once processing is complete, instead of keeping resources active continuously.

Performance thus also becomes a lever for controlling infrastructure costs.

What I take from it

Optimising an AI system is not limited to model selection.

The biggest gains can come from combining several levers: model, data representation, context processing, software architecture, packaging and resource lifecycle.

Here, the reduction in processing time is the result of a global AI Engineering approach combining model optimisation and execution redesign.

The result is not just a faster pipeline: it is a system whose resources are better aligned with the actual need.