Notorious: Why the Future of AI Developer Platforms is Product-led Sales (Predibase)
The best founders, startups, strategies, metrics and community. Every week.
Current subscribers: 9,388, +38 since last week
Share the love: please forward to colleagues and friends! 🙏
Notorious Startup of the Week: Predibase
One of the most exciting capabilities that Large Language Models (LLMs) offer is personalization. LLMs can help personalize our experiences as they get to know our preferences. For example, in our email client, LLMs can classify the tone we are writing in (e.g. formal, friendly, assertive, etc.) and provide responses that match our tone and substance.
However, personalization is difficult because it often requires customizing through fine-tuning or building individual models for each user or subset of users. This presents a difficult infrastructure problem that Predibase has taken head on. Predibase is building the fastest and easiest way to fine-tune and serve any open-source large language model on state-of-the-art-infrastructure hosted in their cloud or within your private cloud.
Predibase has raised $28M from top VCs including Greylock and Felicis and is working with customers like WWF, Sekure Payments, Whatfix and large US Banks. For this edition of Notorious, Predibase’s CEO Devvret Rishi and Director of Product Marketing Will Van Eaton share Predibase’s approach to building a Product-led Sales GTM strategy:
Why the Future of AI Developer Platforms is Product-led Sales
“Generative AI and Large Language Models (LLMs) have been in the limelight since the introduction of generative services like Midjourney and ChatGPT in late 2022 and teams across every industry have been racing to develop and deploy GenAI products. Since OpenAI and other commercial LLM providers offer intuitive APIs and simple guides for getting started, developers face far fewer barriers to entry.
That being said, the high cost, inconsistent performance, limited throughput, and concerns about privacy and ownership associated with commercial AI APIs have caused many teams to turn to open-source LLMs and managed infrastructure.
Predibase offers a fully-managed platform for teams to fine-tune and serve open-source LLMs for production use cases. We make using open-source LLMs in production easy by orchestrating the compute resources and infrastructure for fine-tuning and serving LLMs and building emerging best practices into an easy to use UI and SDK.
Most teams are still exploring how to incorporate LLMs into their organization. In fact, based on a recent survey of AI practitioners we conducted, only 30% of respondents have an LLM in production today, with another 40% either still experimenting with LLMs or planning to use LLMs in the next 12 months. Since fine-tuning and serving LLMs is still in such an early stage, we knew we needed to make it easy for teams to experiment with our platform before committing to a larger contract.
Meet customers where they are with product-led growth
Given how many teams are still in the early stages of planning or experimenting, it was crucial for us to provide an easy on ramp for them to start using Predibase without committing to a larger contract. To do this, we first launched a free trial. This trial was initially 14-days and included unlimited usage, but we later extended it to 30 days and included $25 of free credits for fine-tuning and serving because we found 14 days was too brief for teams to experiment with how they might use LLMs in their organizations. That being said, given LLM fine-tuning and serving is powered by high-end GPUs, a $25 cap ensured teams wouldn’t potentially rack up significant compute costs, while still being enough to fine-tune several small models (and achieve good results with the product).
For those teams that are still in experimentation mode and aren’t ready to jump to production levels, we also released a pay-as-you-go (PAYGO) Developer Tier with no contract minimum. Our hypothesis was that this would let teams use the platform for experimentation–which often shows up as lumpy or inconsistent usage–and only pay for the compute resources they use plus a small margin. We released this tier to help remove roadblocks and, unlike other PAYGO tiers in the space, don’t charge minimum fees for fine-tuning jobs (we found that for small training jobs these flat fees could easily be larger than the actual compute fees, and didn’t want to cause a customer to think twice before firing off a training job). Since launching at the beginning of the year, we’ve had over 100 customers sign up for the Developer Tier, putting to rest many of our initial concerns about offering a self-serve tier.
How we enabled a PLG motion through product innovation
Since building GenAI and LLMs relies on GPUs–which are much more expensive than general cloud computing resources, often costing $1-$3 per hour of usage or more–cost is a massive concern for customers. Incidentally, GPU costs are necessarily a concern for providers when designing a free trial or free tier that relies on GPUs because, if it’s a successful offering, it could get very expensive to maintain very quickly.
To make fine-tuning and serving LLMs more cost-effective for our customers we developed LoRA eXchange (LoRAX), an open-source framework for serving hundreds of fine-tuned LLMs on a single GPU. LoRAX is a first-of-its-kind innovation that unlocks previously unheard of cost-efficiencies because rather than loading each fine-tuned LLM into its own dedicated GPU–and paying for each GPU–teams can now load hundreds of fine-tuned LLMs into a single GPU.
This innovation is also what makes offering our free trial and PAYGO Developer Tier (cost-effectively) possible. When free trial customers fine-tune an LLM or serve fine-tuned LLMs, they are securely leveraging GPU resources that are shared by other users. LoRAX also enables our serverless fine-tuned endpoints, which essentially let teams pay per token (usage-based pricing) rather than pay per GPU-hr, a much more cost-effective proposition for a team that’s not at production volumes.
Drive free trial leads with hands-on content
Since GenAI is such an emerging space, a large share of practitioners are still learning how to get started with LLMs and there’s ample discussion in the community debating the pros and cons of emerging techniques. We’ve found that generating a steady stream of relevant, hands-on content (like tutorials with accompanying Colab notebooks, examples of real-world use cases, and webinars designed to help teams get started building AI) drives the most engagement resulting in free trial signups. In fact, it’s not uncommon for leads to tell us they’ve already watched a handful of webinars and completed tutorials before taking a meeting with our sales team, making it possible to jump right into a more advanced conversation.
In addition to tutorials, blogs, and webinars, we recently tested a more product-driven approach to demonstrating emerging best practices and our latest innovations. In late February, we launched LoRA Land, a collection of 25+ fine-tuned LLMs that outperformed GPT-4 on task-specific use cases (you can try it yourself or read more about it). Since many teams are still very new to fine-tuning, we wanted to share real-world examples of how, compared to GPT-4, fine-tuned open-source LLMs can be:
More accurate for task-specific use cases (4-15% better in this case)
Much cheaper to operate (~97% less per month)
Incredibly cost-effective to train (less than $8 per model)
Those visiting LoRA Land can prompt each of the 25+ adapters, see their responses compared to the responses of the original base model, and learn more about how each was trained. For those interested in working with the fine-tuned models themselves, we made them all available on Hugging Face.
Enterprise contracts rely on a more hands-on sales motion
While the aforementioned PLG motion helps AI practitioners get their hands on the platform and start experimenting cost-effectively, leadership teams and the C-suite often want to be very involved in AI strategy because it is such an emerging opportunity. This is especially true as teams are making massive organizational decisions like whether to use commercial LLM providers (e.g. OpenAI) or open-source LLMs.
GenAI is an exciting frontier of emerging technologies and best practices, and many teams need support, education, and a consultative sales process to thoughtfully evaluate the best path forward for their organization. Since many building GenAI products are doing so for the first time, we found it important to offer strong support throughout the sales process. Sales Engineers with deep experience in ML are an essential part of helping prospects recognize value and onboard more quickly in a free trial or paid proof of concept period.
The best of both worlds
For an emerging category like developer platforms for GenAI, we’ve found product-led sales to offer the right mix of organic bottom-up product-led growth and high-touch, consultative sales-led growth. Using our free trial and Developer Tier to fill parts of our funnel also pressures us to make sure teams can get started on their own as easily as possible. We’re always thinking of ways to improve the onboarding process or provide hands-on tutorials showcasing different workflows and use cases, and we appreciate how much feedback we get from free trial and Developer Tier users.
If you’re interested in trying Predibase or learning more, check out our free trial or jump into our Discord.”
Thanks for reading. By way of background, I am an early-stage investor at Wing and a former founder. Please reach out to me on X @zacharydewitt or at zach@wing.vc. Some of the early-stage PLG + AI companies that I have the privilege to work with and learn from are: AirOps, Copy.ai, Deepgram, Hireguide, Slang.ai, Tango and Tome.
Operating Benchmarks (from PLG Startups):
I will continue to update these metrics and add new metrics. Let me know what metrics you want me to add (zach@wing.vc)
Organic Traffic (as % of all website traffic):
Great: 70%
Good: 50%
Conversion rate (website → free user):
Great: 10%
Good: 5%
Activation rate (free user → activated user):
Great: 50%
Good: 30%
Paid conversion rate (free user → paid user):
Great: 10%
Good: 5%
Enterprise conversion rate (free user → enterprise plan):
Great: 4%
Good: 2%
3-month user retention (% of all users still using product after 3 months):
Great: 30%
Good: 15%
Conversion from waitlist to free user:
<1 month on waitlist: ~50%
>3 months on waitlist: 20%
For more detail on acqusition rates by channel (Organic, SEM, Social etc), please refer to this prior Notorious episode.
Financial Benchmarks (from PLG Public Companies):
Financial data as of previous business day market close.
Best-in-Class Benchmarking:
15 Highest EV/ NTM Revenue Multiples:
15 Biggest Stock Gainers (1 month):
Complete Dataset (click to zoom):
Note: TTM = Trailing Twelve Months; NTM = Next Twelve Months. Rule of 40 = TTM Revenue Growth % + FCF Margin %. GM-Adjusted CAC Payback = Change in Quarterly Revenue / (Gross Margin % * Prior Quarter Sales & Marketing Expense) * 12. Recent IPOs will have temporary “N/A”s as Wall Street Research has to wait to initiate converge.
Recent PLG + AI Financings:
Seed:
Haiper, a developer of perceptual foundation models aimed to empower creative expression, has raised $13.8M. The round was led by Octopus Ventures.
Tollbit, a content monetization software intended to help monitor the scraped data of websites intended to protect and monetize publisher's content, has raised $7M. The round was led by Sunflower Capital Partners, with participation from Operator Collective, Lerer Hippeau, AIX Ventures, and Liquid 2 Ventures.
Early Stage:
Ema, an AI assistant that allows enterprises to build generative AI personas and applications, has emerged from stealth with $25M. Accel, Section 32 and Prosus Ventures led the round, with Wipro Ventures, Maum Group, Firebolt Ventures, AME Cloud Ventures, Frontier Ventures and Venture Highway joining.
Series A:
Metaplane, a data observability platform designed to catch data quality issues, has raised $13.8M. The round was led by Felicis Ventures, with participation from Khosla Ventures, Y Combinator, Flybridge Capital Partners, Stage 2 Capital and B37 Ventures.
Zama, an open source cryptography startup, has raised $73M at a valuation approaching $400 million. The round was led by Multicoin Capital, with participation from Protocol Labs, Stake Capital, Blockchange Ventures, Metaplanet Holdings and VSquared Ventures.
Series B:
Baseten, a developer of machine learning infrastructure to deploy models performantly, scalably, and cost-effectively, has raised $40M at a $200M valuation. The round was led by IVP and Spark Capital, with participation from South Park Commons, Greylock and Base Case Capital.
Perplexity, an SF-based developer of AI-powered search, has raised $56M at a $1B valuation. The round was led by Daniel Gross.
Series C:
Argyle, a payroll connectivity platform designed for offering modern financial services, has raised $30M at a $195M valuation. The round was led by Rockefeller Capital Management, with participation from Checkr, SignalFire and Bain Capital Ventures.