TL;DR: An LLM is like Don Quijote—you can’t cure his madness, it’s stochastic by nature. The solution isn’t to fix the madman but to assign him a deterministic Sancho Panza as a sidekick. MDD consists of two layers: first, you study the errors it makes to design tools that absorb those mistakes, and then you let it loose with those tools to verify you’ve closed any gaps. Design for madness, not against it. ...
TL;DR: Your AI will invent API fields that sound perfect but don’t exist. The solution isn’t hoping it gets it right: download the real schema before writing code, capture real responses as fixtures, and separate fetch from processing so you can test without the network. Adversarial programming: code assuming your copilot lies. Have you ever written code against an API where everything compiled, tests passed, the logic made sense… and when you connected to the real API, nothing worked? ...
I’m building a menu bar app for macOS. It works perfectly on my Mac. Now I need to know if it works on a clean macOS: without my settings, without my permissions, without my data. A user installing it from scratch. How do you test that? You need a virtual machine. “Easy,” I thought. “I have UTM installed. I’ll open the wizard, create a macOS VM, and we’re good to go.” ...
Last week I wrote about how my AI invented a complete JSON structure and wrapped it in DTOs, fixtures, and passing tests. 90 green tests. All fiction. That post was the diagnosis. This is the treatment. After discovering the disaster, I did what any engineer with wounded pride does: obsessively research for days to prevent it from happening again. I read papers, tried tools, analyzed real data from my APIs, and built a defense system for my app. ...
Yesterday I discovered that half of a module in my app was based on fabricated data. Not by a confused junior developer. By my AI. The worst part isn’t that it made things up. The worst part is that everything compiled and all 90 tests passed. The coherent fiction I’m building BFClaude-9000, a macOS menu bar app that monitors Claude Max quota. Part of the functionality requires distinguishing whether a Claude account is paid or free by calling the claude.ai API. ...