The win from CRAM is that it is not swap, and operates using mostly normal RAM semantics. If it works as advertised you should be able to allocate most (all?!) of your RAM as CRAM.

It’s a little better than your read. The ~400x speedup is in comparison to using ZRAM, which, in very brief terms, compresses memory by creating an in-memory compressed block device and assigning that as your swap space. So your “swap” is actually a compressed chunk of RAM, not on disk.

This was significantly slower than normal memory access because page faulting when looking up something in memory then fetching from swap was, itself, expensive. Regardless of how fast that swap was. When your swap is on a storage device that overhead is comparatively tiny, but when it’s just another chunk of memory suddenly it’s what you’re spending most of your time on.

Not everyone views it though that lens (though maybe they should). There are lots of people out there for whom LLMs are just another tool that they have become used to using and, in many cases, formed a bond with. However silly or groundless that may be when looked at rationally.

It’s not really strange at all that people would speak out against “hurting” one. Though it should be regarded in exactly the same way as if they were making the same argument about a set of living room furniture.

I guess what I’m saying is that while I don’t agree with them I empathize with the people who are in turn empathizing with the LLMs. It’s still a ridiculous and indefensible position, but I get it.

I’m not disagreeing with you, but the human capacity for empathy has extended to subjects not capable of emotion or pain since long before LLMs came around.

Lots of people don’t like “evil” paths in games because they feel bad about mistreating the NPCs. People regret throwing or damaging stuffed toys and may even apologize to them. There are people loyal to their toasters because they have reliably kept working for so long.

It’s no surprise at all that people empathize with LLMs. It would be more surprising if no one did.