Keeping models private: isolation, encryption, and deletion
A fine-tuned model often encodes information derived from proprietary data: contract language, pricing logic, or internal terminology. Treating the resulting weights as a sensitive asset, on par with the training data itself, is a requirement many teams overlook until an audit or customer questionnaire forces the issue. This post covers the three controls that matter most in practice.
Tenant isolation at the compute layer
Shared inference endpoints that multiplex several customers' fine-tuned models on the same process introduce risk if isolation is only enforced in application code. A misconfigured routing rule or a bug in request handling can leak one tenant's model outputs or, worse, its weights, to another.
Dedicated GPU instances per tenant, or per sensitivity tier, remove an entire class of these failures. On Nevastack this typically means a customer's fine-tuned model runs on infrastructure not shared with other customers, with network policies that block any path between tenant workloads.
Encryption at rest and in transit
Model weights should be encrypted on disk using keys managed separately from the storage system, so that a compromised storage volume does not expose usable weights. Key rotation policies should be documented and testable, not just described in a policy PDF.
In transit, this covers the obvious case of the inference API, but also the less obvious path of weights moving from a training job to storage and from storage to a serving node. Each hop should use TLS with certificate validation, not an internal network assumed to be trusted by default.
- Encrypt training checkpoints, not only the final deployed weights
- Separate key management from the team operating the storage layer
- Log every access to raw weight files, including automated jobs
Verifiable deletion
When a customer contract ends or a data subject exercises a deletion right, deleting a database row is not sufficient if the same information persists inside model weights or cached embeddings. Fine-tuned models built on data that must be deleted need an explicit retraining or discard plan.
We recommend maintaining a mapping between training datasets, resulting checkpoints, and any downstream artifacts such as distilled or merged models. Deletion requests then have a clear scope, and completion can be verified against that mapping rather than relying on memory of what was built from what.
Access control for weights, not just for the API
It is common to secure the inference API carefully while leaving the underlying weight files accessible to a broad engineering team through shared storage credentials. Weight files deserve the same access discipline as production database credentials.
Apply least-privilege access, require justification for direct download of weights outside the serving pipeline, and alert on unusual access patterns such as bulk downloads outside deployment windows.
Documenting the model lifecycle for audits
Customers and auditors increasingly ask for a clear account of where a model's weights live, who can access them, and how they are destroyed at end of contract. Building this documentation as part of the deployment process, rather than reconstructing it under audit pressure, saves significant time.
A short lifecycle document per model, covering creation, storage location, access list, and deletion procedure, is usually sufficient and can be produced automatically if the pipeline records these facts at each stage.
Want this applied to your data?
We scope private AI projects in one call and start with a pilot you can evaluate.