Small models on device change the privacy calculus
Compact models now handle classification, extraction, and summarisation well enough to run locally — which removes entire categories of data-residency objection.
Models in the low-billions parameter range have crossed a practical threshold. For classification, entity extraction, and short-form summarisation, their outputs are competitive with the cloud models of two years ago.
That matters less for quality than for placement. Work that runs on the client never crosses a jurisdictional boundary, never enters a vendor's logs, and never requires a data processing agreement.
For customers in regulated sectors, this converts a procurement blocker into a configuration choice. We are staging a hybrid path where sensitive pre-processing runs locally and only de-identified structure reaches the platform.
The engineering cost is a dual code path and a quantisation pipeline. The commercial return is access to buyers who were previously unreachable.
Investor takeaway
On-device inference turns data residency from a blocker into a deployment option.