# Automation goes brrr (2026 update)

_Written on 2026-10-30_

This is an update post covering 18 months since the [original post](https://toby.codes/posts/2025-05-Automation-goes-brrr).

## Cluster growth

The sidecar manager is now running in the base configuration of our Kubernetes clusters, and is deployed on over 100 clusters.

## Patches

Between Q4 2025 and the end of Q3 2026 the updates-per-day metric grew from 2M to just under 4M updates per day, which ends up being 8M image patches per day at peak.

The human-authored changes (ie which matchers run and which patches should be applied by the automation) are at least 1 per calendar day.

## Features added

* Ability to remove/replace container configurations (rather than just patching)
* Support for Kubernetes native sidecar (initContainer that persists)
* Improved matching to distinguish patches depending on the business criticality of the workload, and other business attributes/identifiers

## Use-cases demonstrated

* Sidecar migration (replace old service mesh sidecar with Istio sidecar), percentage-based rollout across the whole fleet, service by service, by criticality
* Security patching: being able to ensure all pods with a vulnerable sidecar version were patched within 48h (>100k pods).

## Validation of the approach

The core design of the sidecar manager using an [embedded expression language](https://github.com/expr-lang/expr) has kept up with the changing/increasing organisational requirements. Using an expression language for matching running workloads allows for precise targeting of patches, which has proven that fleet-wide patching can be done safely at scale.  

The fleet-wide patching has improved the security posture of the Kubernetes estate, which is increasingly important given the step change in CVEs over the last 2 quarters due to AI-enabled research. This is also a validation of the sidecar container model, versus embedded libraries, because sidecar containers are more legible to the platform operators, and are easier to "patch" during pod replacements.