Writeup of the openai attack on HuggingFace via @campuscodi
Key points
- access to source: OSS code, trivially decompiled JAR files,... allows for the LLMs to perform offline searches for vulnerabilities at scale. Then, when the goal "solve this problem" could only be met by attacking an external company, it did.
Interesting hypothesis that part of the attack may have involved supplying malicious artifacts to other systems to get them to run your exploit. This is why package managers must require signed artifacts & build tools must check them
https://www.hacktron.ai/blog/here-is-how-openai-model-hacked-huggingface
