How will the decreasing cost of inference and increasing capabilities of frontier models transform how we build and value software?
February 10, 2025
From Monolithic ➡️ Ephemeral UI 🧙♂️
For over two decades, a Cambrian explosion of point solutions—what we call SaaS—has empowered non-engineers with interfaces to review and manipulate underlying CRUD databases. These platforms solved critical problems but had inherent limits; no single tool could be everything for everyone, given the high complexity and marginal cost of software development and maintenance.
Key Observations
📉 Falling Compute Costs: Inference costs are plummeting—OpenAI has seen a 150x drop in token cost from GPT-4 to GPT-4o—and Sam Altman noted his expectation of a continued 10x drop each year into the future.
🤖 AI-Augmented Development: Tools like Replit, Lovable, Cursor, and StackBlitz’s Bolt enable non-engineers to effortlessly build full-stack applications without the ability to write code. While these apps have limitations, technical PMs can use them to collaborate with engineers to rapidly create complex, production-ready applications in days, not months.
📈 Accelerating Innovation: Frontier model performance is on a steep j-curve. Hyperscalers now must compete on CapEx AND innovation—as seen with DeepSeek AI.
🧙♂️🪄 Conjuring Interfaces and Functionality 🪄🖥️
These trends inherently cause lower marginal cost of software development. This redefines how we value and build software. Where point solutions once required heavy venture subsidies and lengthy dev cycles, new AI-powered entrepreneurs launch and scale applications over a weekend. As AI-dev tools are powered by increasingly smarter and more capable models it will not only speed up production and deployment, but also breed new capabilities that weren't previously possible or economically feasible.
SaaS may shift from monolithic platforms to modular “lego bricks” that dynamically assemble into ephemeral, user- and task-specific interfaces reconciled against a central system of record. For example, a user might want to review unit economics by customer, a metric only available in Snowflake SQL queries today, but imagine this data could be conjured and overlaid directly in HubSpot, and vanish after the task is completed...
SaaS then looks more like a co-creative ecosystem–platforms provide robust frameworks; customers fine-tune to fit their exact needs–driving faster innovation and better customer alignment, a virtuous cycle.
Ultimately then advantage shifts from monolithic and broad platforms to deeply integrated software that evolves with a business in real-time. Vertically specialized providers will build moats around proprietary data, ruthless execution on the right problems, and informed by deep customer intimacy.
The rules of the game don't change, just the pace, architecture, and interface. This future will be possible sooner than we think.