Downloading a pretrained model from a public hub is, from a security standpoint, similar to pulling an unvetted open-source dependency: you inherit whatever was baked into its weights, and in some formats, you inherit executable code. Security researchers have repeatedly found malicious models uploaded to public model-sharing hubs using older pickle-based serialization formats that execute arbitrary code the moment the file is loaded — an attacker doesn't need to compromise your infra if they can get you to load a poisoned checkpoint yourself, and several such cases have been publicly disclosed as the major hubs added automated scanning in response.
Supply chain hygiene here looks like software supply chain hygiene elsewhere: prefer safe serialization formats (like `safetensors`, designed specifically to exclude executable code), verify checksums and provenance, scan for known-malicious model repos, and pin specific versions rather than pulling 'latest' from an untrusted source.