Ml

Everyone’s advice for prompt injection is the same: stick a classifier in front of the model. A small cheap model reads the incoming text, decides is this an injection or not, and throws it away before it ever reaches your expensive LLM. So using the recent trend of Laya/Jev models, a recent small non-autoregressive model that just scores an answer in one forward pass instead of generating tokens, I’ve built a pretty good prompt injection classifier using my smaller, better calibrated version of Laya called Smolaya....

Everyone has run this line: model = AutoModelForCausalLM.from_pretrained("some-user/some-model") It looks like a download. It is closer to running someone else’s installer. Some model formats execute code the moment you load them, some configs pull Python straight out of the repository, and the weights themselves can carry behaviour nobody mentioned in the model card. It has also been shown that its EXTREMELY easy to poison a big-ass LLM. With a near constant number of samples regardless of the size of the model 1....

I already explained DP ML in another post 1, so this blog post covers the question, how can we design a service that lets customers finetune Large Language Models in a privacy preserving way. With the rise of data privacy laws like GDPR, DSGVO and CCPA, companies face increased scrutiny on data handling practices. The demand for privacy-preserving AI models is growing, especially in highly regulated industries. Despite this demand, many businesses lack the in-house expertise to implement their own model fine-tuning....

In this post, I want to briefly introduce Differential Privacy to you, which, in my honest opinion, needs to get more attention in the software developer community. During my Master thesis, I evaluated the use of Differential Privacy for Federated Learning (I might explain Federated Learning in another post). The Theory Differential Privacy, originally $\epsilon$-Differential Privacy (DP)1, is a way to secure the privacy of individuals in a statistical database. A statistical database is a database, where only aggregation functions like “sum”, “average”, “count”, et cetera… can be executed....