63 bets, 3 worked: why we publish what didn't
Since December 2025 we have shipped 211 changes and logged 63 bets. Three clearly worked. Eleven clearly didn't. We publish all of it.
Most AI companies show you demos. We would rather show you our scorecard. Every change we ship to Space AI goes into a public changelog with three things attached: the result we expected, how we would know, and what actually happened once the numbers came in.
The scorecard
| Result | Bets |
|---|---|
| Worked | 3 |
| Didn't work | 11 |
| Couldn't tell yet | 37 |
| Still measuring | 12 |
That is not a typo. Most of what we try either fails or can't be measured yet. That is what real product work looks like, and anyone who tells you otherwise is showing you the demo, not the dashboard.
What didn't work, and what we learnt
Product videos made entirely by AI. We built showcase videos from a brand's own product photos. None were published. Most never got past the idea stage. We paused new video formats until we understand why brands pass on them.
Text generated inside ad images. Models still misspell, and a typo on an ad costs trust. Our automatic checks didn't fix it either: posts edited by hand rose from about 2% to about 5%. So we changed the approach. AI now draws only the scene, and the exact headline, copy and logo are placed on top.
Self-serve upgrades. About 2% of free users upgraded on their own, against our bar of one in twenty. Paying customers mostly came through a conversation. We now treat sales-assisted onboarding as the main path.
Publishing to more platforms. We added Facebook, X, YouTube and Reddit. None of the brands published to them through us. We stopped widening coverage and focused on the channels people actually use.
What did work
Reliable publishing. About 97% of Instagram publish attempts succeeded, above our 95% bar. Scheduling became the main way brands publish: about six in ten posts went out at a pre-set time.
Unglamorous, yes. But the things that work are usually the boring foundations everything else depends on.
Why this matters to you
Every AI vendor will tell you their product works. Ask them three questions:
- What did you expect it to do, in a number?
- How would you know if it didn't?
- What have you killed because it failed?
If they can't answer the third, they aren't testing. They are selling. We test on our own brands first, publish the results, and only bring what survives into client work.
That is also why our Lab exists: a short, dated list of what we are adopting and trialling right now.