Software companies, in particular B2B SaaS companies, have historically been valued on 7x-10x revenue multiples, in large part due to their gross margin profile. The median gross margin across the public cloud software universe is 76%. The math is simple. With >75% gross margins, software companies are able to take on new revenue without increasing their costs to serve that revenue (i.e. their COGS). That's the magic of SaaS.
But as we transition from legacy SaaS to AI-first SaaS, gross margin profiles are fundamentally changing. Typically, cloud costs (i.e. AWS), data costs (i.e. Snowflake), and customer support were the leading contributors to SaaS COGS. Now, token costs have emerged as by far the greatest contributor to SaaS COGS and have led investors and operators to question whether we'll see 10x+ NTM revenue multiples in the future.
I'm not here to relitigate SaaS economics in the age of AI. I'm here because this math forces a daily tradeoff inside companies like ours: provide as much intelligence (and thereby value) to our customers as possible, or control token costs to maintain healthy software gross margins.
To understand this tradeoff, it helps to understand what drives token costs in the first place. To clarify, we're referring to the token costs that we incur specifically to serve our customers, not the token costs that we incur to operate our business (i.e. coding agents for our engineering team). For Gigi specifically, there are three contributing factors:
The frequency that our customers leverage Gigi for tasks. Every audit, optimization, and report Gigi runs consumes tokens, so our COGS scales directly with how much our customers use us.
The model choice to execute that task. A frontier reasoning model can cost 10x more per task than a smaller model, so which brain we assign to which job is a direct margin decision.
The tooling (i.e. harness) that we build around task execution. For example, collapsing ten LLM round trips into a single efficient tool call, or having the agent write a script once and rerun it deterministically instead of reasoning through the same task from scratch every time.
Let's break down how we reconcile each of these factors.
If we want our customers to have the best possible experience using our product and extract the most value out of Gigi, then we want to optimize for as much task execution as possible in bucket 1. We want this number as high as possible. More usage means more value delivered. This bucket is the whole point of the product, and it's why buckets 2 and 3 have to absorb all of the optimization.
Historically, model choice has been simple: we almost always choose the best possible frontier model for the task at hand. But something has changed over the past couple of months. The models have gotten too good. We're increasingly finding the highest reasoning frontier models, like Fable, are unnecessary to execute certain tasks in some instances, and our previous choice to "always use the best models" no longer holds. In fact, many folks building AI are exploring moving off frontier models entirely in favor of a new batch of open source models that are able to execute many tasks at ~80% of the cost of a frontier model. The rise of open source LLMs is fascinating, but more specifically, the combination of the rise of open source and the exponential intelligence increases in frontier models has made model selection and model routing a critical piece of every AI agent's harness.
The third bucket is where we've allocated most of our resources over the past year. We've focused here because we've historically believed two things: 1) the customer experience should always be prioritized, and 2) our customers should spend limited mental bandwidth on token rationalization. They should just get the best possible Gigi experience.
Quick disclaimer: we pass back token costs as part of our agreements with customers on a monthly basis. We charge a platform fee tied to LLM token usage that scales in either $500 or $1,000 increments each month. We do this to protect our gross margin, but in reality this isn't matched 1-1. We eat a portion of the token costs of most of our customers. We do this because we don't want our customers to rationalize token costs against their experience with Gigi, and we know there are always harness optimizations we can make on an ongoing basis to limit token costs.
Over the past year, we've spent close to 5%-10% of our engineering resources on token cost optimization. Investing that much of our engineering into something without a clear customer benefit (vs. building a feature our customers request) could on the surface be tough to rationalize. Whenever we do it, we ensure we're investing in token optimization when there are other customer benefits beyond just costs, such as latency and stability improvements.
But here's what we've come to realize: token optimization is a customer benefit. It's what allows our customers to obtain the highest use of Gigi (frequency) with the best possible intelligence (highest reasoning) without having to worry (too much) about the cost associated with all of this. These operational optimizations aren't like traditional DevOps improvements that a customer seldom sees. They're fundamental to the customer's experience, they lead to Gigi doing more and better work for our customers, making it core to the product and business that we're building.
So yes, we monitor gross margins. We have a gross margin floor for mature customers that is admittedly lower than the gross margin of those top public SaaS companies. We created this floor to ensure that we're building a sustainable software business not reliant on outside capital to fund tokens. And in the same vein, we're less concerned about how prospective investors perceive our margin profile. There will always be future token optimizations to make if we choose to prioritize them. But we believe that if we create as much customer value as possible while maintaining our gross margin floor, we build a category-defining business.
Cherry Picked is a monthly newsletter from Adam Epstein, co-founder and CEO at Gigi, covering the AI and commerce media insights you just gotta know.
