It would be nice to support SafeTensors. It seems to gaining some momentum and helps avoid a lot of the issues with malicious models: https://kaitchup.substack.com/p/safe-fast-and-memory-efficient-loading
Hi Justin,
I got pulled onto other AI projects unrelated to running models at the edge so I had to back burner this for the moment. During that time a perfect example of what we're trying to protect against came out:
there's also things like this happening all over now: https://github.com/microsoft/aici
Last updated: Dec 23 2024 at 12:05 UTC