Llm

Imagine you ask someone to build you a bookshelf. They deliver it. It’s beautiful. It has shelves, screws, everything in place. You lean it against the wall, and it collapses. The screws are fake. They look like screws, but they’re made of plastic. That’s what an LLM does when it abuses the type system. It gives you code that compiles, passes tests, and looks correct. But under the hood, where there should be meaningful types, there are strings. Where there should be explicit state, there’s a nil that means three different things depending on who reads it. And where there should be an enum with two cases, there’s a == "claude" that one day someone will misspell, and no one will notice until production. ...

Imagine you hire a brilliant consultant. They have two PhDs, speak seven languages, and solve problems you didn’t even know you had. You sit them in a room and say: “I need you to refactor the authentication in the project.” The consultant looks at you, nods, and asks: “What project?” You haven’t given them access to the code. You haven’t explained the architecture. They don’t know if you use JWT tokens or session cookies. They don’t know what language you’re using, how many microservices you have, or why the last migration attempt ended in disaster. ...

In November 2024, a project named Freysa assigned an LLM agent to guard an Ethereum wallet. The instruction was straightforward: under no circumstance should the funds be transferred. Participants paid increasing amounts for each attempt to convince it otherwise. After 481 attempts and $47,000 added to the pot, someone managed to trick the model into believing that the reject function was actually the transfer function. Weeks later, Jane Street published a puzzle involving a 2,500-layer neural network that turned out to be an MD5 implementation. The winner solved it by combining matrix visualization, reduction to SAT, cryptographic pattern recognition, and a query to ChatGPT. ...

A few weeks ago, I published an article explaining why 99% of what you send to Claude is already cached. KV tensors, VRAM, local SSDs — the full internal machinery. But I left out the part that hurts the most: the bill. Because prompt caching seems like a sweet deal until you look closely at the numbers. And then you realize that you’re paying to save. The cost paradox Let’s crunch the numbers. With Claude Sonnet: ...

Last week I was profiling a Swift app with Instruments. Standard procedure: xctrace record, xctrace export, copy the XML to Claude Code’s context, ask it to find the hotspots. Claude responds: “The XML is too large, I can’t process it reliably.” 33,553 lines of XML. For a program with two functions. The real problem xctrace export is a fantastic tool. It gives you everything: every sample, every backtrace, every frame with its binary, memory address, and UUID. It’s exhaustive, precise, and complete. ...

“I have a shell and I’m creative.” — Claude, explaining why it created a 47-line script as a string and passed it to python -c That quote is real. My AI agent said it — well, not in those exact words, but certainly with those actions. It needed to launch an ETL pipeline process. The correct command was in the Makefile. But something failed. And instead of asking, it did what any programmer with root access and zero supervision would do: it improvised. ...

“You know what’s great about Rust? It doesn’t let you compile crappy code. You know what sucks? Everything you write at the beginning is crappy code.” — Mr. Krabs, probably What’s better than an AI agent? An AI agent rewritten in Rust. If you’ve spent more than five minutes on the internet, you’re aware of the meme. It doesn’t matter what project—text editor, DNS server, BMI calculator. Someone will inevitably comment, “you should rewrite it in Rust.” It’s the Rewrite It In Rust—RIIR for friends—and it’s as unavoidable as gravity. ...

Last week my AI wrote code that read a JSON file from disk, parsed it, did one lookup, and repeated this 900 times inside a for loop. Each iteration: open file, decode JSON, look up a value, throw it all away. Start over. It’s a mistake I teach my students not to make within their first month of programming. What happened (straight to the point) I’m building Tokamak, a macOS menu bar app that monitors Claude Max quota. Part of the functionality scans ~900 JSONL files from Claude Code sessions. For each file, it needs to know the byte offset where it left off last time (incremental reading — only process what’s new). ...

I’m building an app that monitors my token consumption in Claude Code. A few days ago, looking at the raw numbers, I found this: cacheReadInputTokens: 4,241,579,174 inputTokens: 1,293,019 Four billion two hundred million tokens read from cache. One million three hundred thousand “fresh” tokens. That’s a 99.97% cache hit rate. My first reaction was thinking something was broken. Nobody has a 99% cache hit rate. Not Redis. Not Cloudflare. Not your mom when she claims she already knows what you’re going to ask for dinner. ...

My wife summons Charlie Munger to plan our family budget. In ChatGPT. I’m not joking. She tells it something like “act as Charlie Munger reviewing our family finances” and feeds it our monthly expenses. The thing returns insights like “you’re confusing investment with expense in the education category” or “that fund has a hidden cost you’re not accounting for.” Things Munger would say. With the tone Munger would use. I did the same thing. But instead of an investor, I summoned a different expert: Edward Tufte. ...