The model could only click
We let a model write store configs in 2023. It covered the long tail and never touched the stores that made money — and the reason wasn't trust
Applying a coupon at checkout is deterministic. A state machine walks the codes, a per-store configuration tells it where the fields and totals are, and the run either reduces the total or does not. No model is involved in any of that, on purpose.
The models were involved somewhere else: writing the configurations.
Why the configuration is the whole problem
Coverage is a data problem, not an engineering one. That was the design decision that made the engine work — a store that redesigns its checkout is a config fix, not a release. But it means the work never ends and never gets more interesting. Every store is a small act of reverse engineering, and there are more stores than there are hours.
So in 2023 we tried to have a model do it.
That date matters. In 2023 this was hard, and being hard was the reason to try. Today you could hand a free model a page and Puppeteer and get something working in an afternoon, without a configuration at all.
The pipeline
The extension stripped the checkout page down before anything left the browser: raw HTML, minimal classes, nothing that was not structurally necessary. Partly that was signal-to-noise. Mostly it was that a checkout page is full of things that must not be sent anywhere — cart contents, addresses, session state. What went up was skeletal by requirement, not by preference.
The model returned a finished configuration. Not a draft for a human to approve — a config that went straight into use. That sounds reckless and was not, because the apply module already has a validation step: a bad config fails during a run rather than doing something wrong. The floor was low enough that generation could be unattended.
The interesting part was reading the markup. Framework-generated class names are obvious once you have seen a few hundred — they change per build, sometimes per runtime, and any selector that leans on them is dead on the next deploy. So the job was not “find the coupon field”, it was lower the specificity until the selector survives a redesign. Structural anchors, pseudo-classes, positional relationships. We ended up declaring our own pseudo-class cases for patterns that came up often enough.
That is a real skill and the model was mediocre at it. But mediocre was fine, because of where we let it work.
What the model could not see
Here is the actual limit, and it is not about intelligence.
The model looked at a page and produced a configuration built from clicks. Click the field, type, click apply, read the total. That is the only interaction it could infer from markup, and it is the slowest, most fragile way to drive a checkout: strictly serial, one code at a time, waiting on a page that reloads itself whenever it feels like it.
The stores that made real money were not configured that way. Those were written by engineers who opened the network tab, worked out what the checkout actually posts, and wrote a config that talks to the endpoint directly. Requests can be issued in parallel. They do not care about layout. They break only when something specific changes server-side — a major update, not a redesign.
So the split was never a question of trusting the machine. It was that the machine’s output was confined to the weaker of the two available strategies, and the weaker strategy was only good enough where the stakes were low.
Top stores: hand-written, request-driven, engineer-owned. The long tail: generated. A shopper lands on a store nobody has configured, a config gets made, and from that moment real users are the test suite.
Users as the test suite
Every run emitted events. Click rate, bounce, and the number that mattered — percentage of successful applications per store.
Those fed a health view that considered more than the failure rate: the tier of the store, how many times its config had been edited, whether we actually held live coupons for it, who fixed it last, how many runs it had seen. It rendered as a grid of squares, like a contribution graph, and you could tell at a glance which stores were rotting.
When a store’s numbers dropped it entered an escalation pipeline. Fix, mark fixed, escalation tiers reset to zero, watch the events come in again. If it fell a second time the tier went up and a human went to look.
That reset is the part I would defend to anyone. It means the system does not accumulate opinions about a store from history — it re-earns its assessment every time from live traffic. A config that was terrible last month and has been fixed is just a config now.
What this actually was
Semi-automation, in the plain engineering sense. A conveyor that runs on its own, with a person standing at the end holding a stick, ready to redo anything the machine got wrong. The model was never the system. It was a way of making the first draft free so that the human time went to the stores that deserved it.
Which is, more or less, the honest shape of most useful automation, then and now.
The part that has not changed
The obvious ending would be: this was hard in 2023 and it is trivial now, so the problem went away.
It did not. What got easier was generation, and generation was never the expensive part. Tokens and subscriptions are real money, per run, forever. A person who can open devtools, read a DOM tree, look at the styles and debug a network request is still cheaper than that — and still more reliable, which matters more.
That was true when we built this. As far as we can tell it is still true. We do not know when it stops being true, and we would not bet on the date.
The product side of this — who paid for it, and what happened to the whole category — is a separate story.