Foundation-Models

I’ve spent weeks stress-testing Apple’s on-device model — the ~3B parameter one that runs on the Neural Engine of any Apple Silicon Mac. To test it thoroughly, I built Think Local, a macOS app that exercises every capability of the model: chat, image generation, structured output, tool calling, and parameter comparison. My conclusion: As a chatbot, the model is terrible. As a structured output and tool calling engine, it’s surprisingly good. ...

Yesterday I wrote a commit message with Claude Code. The diff was a one-line change: a typo in a comment. Claude Opus read the diff, thought for two seconds, and generated fix: correct typo in auth comment. That consumed about 800 input tokens and 30 output tokens, at $15 and $75 per million respectively. Cost: a fraction of a cent. But multiply that by 40 commits per day, 250 days per year, across a company with 200 developers using coding agents, and the fraction of a cent becomes thousands of dollars spent on the intellectual equivalent of applying band-aids. ...

You’re paying anywhere from $20 to $200 a month for access to LLMs. Claude, GPT, Gemini, whatever. And most of the calls you’re making from your scripts and dev tools boil down to something like this: “Classify this bug into one of these five categories” “Name this variable for me” “Tell me if this commit is a fix, feat, or refactor” “Summarize this block of text in two sentences” Meanwhile, your Apple Silicon Mac has a 3-billion-parameter language model baked right in, integrated with the operating system. No cost, no internet needed, no API key, no network latency. And you’re probably not using it at all. ...