Skip to content
unnudged
Get early access
brandCooling

Baseten on X

Approval of how fast Baseten serves open models and how quickly people get running leads the talk, with a free window on its mammoth-1 and elephant-1 decision models drawing recent posts. Under that runs a debate over whether the business holds up as inference prices keep falling.

First written 27 Oct · what changed · from 86 posts by 76 people

Tracking posts that say “baseten”

Track this for your market
Posts, last 24h
39
−36%vs. previous 24h
Posts, last 7 days
366
147 in the last 72h
Velocity
52/day
7-day average
Momentum
−25%
Last 24h vs. a normal day this week

Posts per day, last 30 days

as of Nov 3, 11:01 UTC
100500Oct 20: 34 postsOct 20Oct 21: 0 postsOct 22: 53 postsOct 23: 11 postsOct 24: 79 postsOct 25: 18 postsOct 26: 36 postsOct 26Oct 27: 49 postsOct 28: 51 postsOct 29: 52 postsOct 30: 43 postsOct 31: 64 postsNov 1: 58 postsNov 2: 60 postsNov 2

What changed

as of 27 Oct · checked once a month

First written 27 Oct. When what people say changes, it shows here.

Tone· 66 posts from 59 people· written 27 Oct

How do people feel about Baseten?

Plain approval leads: 22 of the section's 59 people say outright they like Baseten, naming five-minute setup, free windows on its decision models, and fast serving. The biggest counter-current, 11 people, doubts the business holds up as inference prices keep falling.

Approval is the largest theme by people and its reasons are practical: someone getting colleagues "up and running from scratch in 5mins", a free window of the decision models mammoth-1 and elephant-1 that posts put at October 27, and the original checkpoint served without extra quantization.

Speed stands as its own theme, 10 people, with large claims: 10 posts credit Baseten with the highest tokens per second for redteaming, 1000 t/s at a 1M context, and reports of one comparison where it came first to first token.

Views on the business split. 8 people are bullish, citing $1.5B raised at a $13B valuation, ARR going from $200M to $600M in three months, and the decision models topping a ranking. 11 people are skeptical: the market as a "permanent reverse auction" of undercutting, a question about how margins above 50% survive cheap entrants, whether open weights are taking share at all, and Nebius's spot GPU pricing called highly disruptive.

8 people praise the content instead — the inference engineering book, the YouTube videos, and a whiteboard session on Kimi K3.

  • Loves it24
  • Bullish on the business12
  • Skeptical of the business11
  • Fastest inference10
  • Praises content and case studies9
See 5 of 66 posts
Jaya Gupta@JayaGup10

Inference is the second way to raise a massive round right now. After the labs, you can still justify massive rounds by pointing at data centers and compute, which means more inference companies keep getting funded even as the underlying product stays extremely easy to switch away from. On the flip side, many neolabs are also renting out their own compute, adding yet another source of supply into the same market. Companies that tried to verticalize around batch workloads and post-training are now pivoting back to "pure inference" and joining the same undercutting cycle. The result is a permanent reverse auction: providers undercut each other over and over: half off Fireworks, half off Baseten, 10% cheaper GLM or Kimi—and every new entrant or neolab offering capacity just accelerates the cheaper pricing At the same time, app companies have so much choice right now that they can bake off and compete prices down extremely hard. They move millions of run-rate spend overnight whenever a meaningfully cheaper option appears, because the technical switching cost is essentially "one line of code". That combination: easy capital, extra compute from neolabs, low stickiness, and continuous price pressure makes inference one of the less defensible layers in the stack even as it continues to attract large valuations. However, if you are a smart team of infra people, it is the easiest way to raise 100M seed after a neolab Researchers: neolab <> engineers: inference

John Iosifov ✨💥 Ender Turing | AiCMO@johniosifov

Baseten raised $1.5B at $13B. They don't build AI models. They make running open-source models profitable at scale. ARR: $200M in January 2026 → $600M by March 2026. Three months. 3x. Revenue grew 20x year-over-year. They process 1 billion inference calls per day across 87 clusters. The foundation model layer is commoditizing faster than anyone predicted. GPT-4 class inference: $60/million tokens in 2023, $0.40 today. 150x collapse in 3 years. What Baseten understood early: when the model is free (or close to it), the bottleneck becomes latency, GPU scheduling, and cost-per-token at scale. Infrastructure wins when the commodity layer is commoditizing. This is the picks-and-shovels moment for AI. Not the models. Not the applications. The pipes that carry the inference. Altimeter, Conviction, Spark Capital all saw the same thing. The $13B bet is on infrastructure becoming the durable margin in an ecosystem where everything else is racing to zero. One implication most people are missing: as inference costs keep dropping, the minimum viable AI deployment gets cheaper every quarter. That $7M average enterprise AI budget (up from $1.2M in 2024)? It buys 5-7x more compute per dollar today than it did 18 months ago. The ROI problem has replaced the cost problem. That is a completely different business challenge.

David Kwon@dkwon98

Not sure who at @baseten spearheads their YouTube, but it’s just plain awesome 1) increases aura 2) highlights the technical depth of different team members 3) unique yet fun formats like HORSE and whiteboarding

5 of the 66 posts behind this answer, one per person. The other 61, and every new one as it’s posted, come with tracking. Get early access

Pain points· 4 posts· written 27 Oct

What do people complain about with Baseten?

Two complaints, level at 2 people each: rate limits that throw 429s even on paid usage, and head-to-head tests where Baseten came out more expensive or less capable than a rival.

On rate limits, one user says they get a stream of 429s even as a paying customer and lumps Baseten in with Groq and Cerebras; another calls the service "amazing" but the limits "extremely low" and asks where to find the same speed with limits they can use.

On cost and accuracy, both reports come from side-by-side tests. One comparison of the same open model across hosting providers put a single Kimi K3 task at $12.65 on Baseten and as little as $5.43 elsewhere, a gap of up to 2.3x, though overall accuracy stayed within the margin of error of each model maker's own API. Another tested the mammoth-1 and elephant-1 decision models against Typesafe AI's Jev on 35 compliance questions: the text answers were close, but the tester says the Baseten models could not read images, Jev routed 16 of 16 photos, and Jev was about five times faster. That tester notes the set was small and labelled by one person.

  • Rate limits2
  • Costlier and less accurate than rivals2
See the 4 posts
Brandon OPPA@eltc9584

Same open model, same tasks. Depending on which provider served it, the bill came out more than twice as high. 3 models · 3 hosting providers plus each maker's own API · up to 2.3x cost gap for the same model. One Kimi K3 task cost $12.65 on Baseten and as little as $5.43 elsewhere. Key points: ① Vals AI ran three open-weight models (Kimi K3, DeepSeek V4.1 Flash, MiniMax M3) on Together AI, Fireworks, Baseten and each model maker's own API with identical work: multi-step agent jobs spanning finance, coding and law, totaling more than 650 hours of model runtime ② On overall accuracy, every provider landed within the margin of error of the maker's API. Broken out by field, that changes. Kimi K3 scored 75.8% on legal tasks via Moonshot's API but 60.6% via Baseten ③ Price and speed don't line up either. DeepSeek V4.1 Flash is a third cheaper on Fireworks than on DeepSeek's API, yet it generates 103 tokens per second there versus 286. For all three models, the maker's own API came out neither cheapest nor fastest So why do bills split when per-token prices look alike? Caching. Baseten handled 72% of input from cache, while Together AI and Fireworks handled 94%, so a different share of each prompt got billed at full price. Vals AI released it on October 23 (local time) as the Inference Provider Index, and plans to rerun it under the same conditions every two weeks to catch providers quietly changing how they serve a model behind the same name. One caveat: every figure is Vals AI's own measurement, and it covers just three models and three providers. The ranking could change in the next run. Source↓ link

Alternatives· 9 posts· written 27 Oct

What are the alternatives to Baseten?

Fireworks is the name most often put next to Baseten, 3 people, mainly as a like-for-like option for serving the same open models; 2 people name Modal the same way. The rest are one-person comparisons.

The Fireworks thread is about positioning rather than benchmarks: one post argues that Baseten and Fireworks have both spent heavily on infrastructure for coding agents, that specializing by workload builds stickiness, and that switching provider is "like switching your database", while another predicts each will lead the workloads it positions for — and calls the inference market one with "zero switching cost and lower moats". Elsewhere the two are simply listed together as places that offer the same thing.

Modal appears as an interchangeable place to run open weights, including serving a model to a customer from a private container.

Four more comparisons have one person each. Nebius's spot GPU pricing and bring-your-own-GPU network is called "highly disruptive" for Fireworks and Baseten. A tester weighs the mammoth-1 and elephant-1 decision models against Typesafe AI's Jev. One post corrects first-token figures against DeepSeek's own API, putting Baseten Standard ahead at 627ms versus 673ms. Another argues Baseten, being built on TensorRT-LLM and its own kernels, would not switch engines for Pearl mining.

  • Fireworks3
  • Modal2
  • Nebius1
  • Jev decision models1
  • The model maker's own API1
  • Pearl mining1
See 5 of 9 posts
Arko C@arkobuilds

fair point: all roads lead to inference if you're engineers building in infra. however, with inference being such a large market with so many options, specializing towards a workload and offering a stack that improves with more telemetry, allows to build long term stickiness with an amazing nps within customers. the entire stack gets tuned to your workload patterns (input, output, cache hits, etc) and over a period of time, re-doing the cycle with another provider is like switching your database. specialize: codings agents and voice agents need completely different inference infrastructure, not just kernels. baseten and fireworks have spent so much capital building infra for cursor, that they have made everyone believe that the only way to tune performance is custom kernels.

Matheus@matheusgab

That’s a very good point. Thanks! Even so I think Baseten and fireworks will both be the leaders on the kind of workload they position themselves to serve. The inference market has zero switching cost and lower moats but fire attention and Baseten’s solution are already generating value to the application layer

Daave “3 and 30” Tepper@institLP

2:30am wake-up insight Nebius now has spot GPU pricing for available GPUs Nebius also has a software overlay for “bring your own GPU” for bare metal owners to contribute GPU capacity to the Nebius distribution network What GPU owner wouldn’t - in the most liquid way possible - contribute GPUs to the Nebius network. It’s free money as a bridge between use or even more indefinite arbitrage between cost and market. I have to believe this becomes a big business for Nebius. Highly disruptive for Fireworks and Baseten. Am I wrong? $nbis

5 of the 9 posts behind this answer, one per person. The other 4, and every new one as it’s posted, come with tracking. Get early access

Questions· 9 posts· written 27 Oct

What do people ask about Baseten?

What people ask most is how the safety layer behaves, 3 people, starting with what the default blocks and whether a customer can turn it off. Voice and streaming latency and whether more models are coming follow, 2 and 2 people.

The safety questions are about control and edge cases: what "safety as a default" actually blocks, whether it can be disabled, how the system handles the race between a safety signal and a tool call that is already streaming, and whether monitoring catches changes to fine-tuned models — one asker points at edited open-model weights carrying backdoors.

Voice and streaming questions come from one thread about speech: the first-chunk latency on a cold cloned voice, and how streaming handles text arriving in short chunks before a sentence is finished without losing natural prosody across 48 languages.

Two people ask about the model line itself: whether more releases are coming, and whether a base model will be open-weighted.

Singles: one asks whether the endpoint serves the official model or a quantized copy; one asks whether Baseten is more than a reseller of GPU time.

  • How does the safety layer work?3
  • Voice and streaming latency2
  • Will more models be released or open-weighted?2
  • Is this the official, unquantized model?1
  • Does it just resell GPU time?1
See 5 of 9 posts
A Taylor@AlexTaylor12112

Will you be able to catch changes in fine-tuned open source models? For example: link "How abliterated models can get you pwned (Prince Chaddha, 10/6/2026). Any open model whose weights have been edited can carry a backdoor, whether it's a task-specific... "

飞叔@RyanJ_KOL

For streaming speech, how does it handle text arriving in short chunks before the sentence is complete? Keeping prosody natural without waiting for all the punctuation is an interesting latency-quality trade-off, especially across 48 languages.

5 of the 9 posts behind this answer, one per person. The other 4, and every new one as it’s posted, come with tracking. Get early access

Wishes· 2 posts from 1 people· written 27 Oct

What do people want from Baseten?

Wishes on the page come from one person, 1 in all, and both are about evaluation and citation rather than new features: keep page structure on parsed chunks, and test the two decision model sizes at one error budget.

The first wish is about Parse and retrieval tested end to end: if page coordinates and table structure stayed attached to each chunk, an agent could cite the actual cell or figure instead of only the extracted text.

The second, addressed to Baseten and Base Labs, argues that for routing use cases confidence calibration and abstention matter as much as top-line accuracy. The suggestion is to compare the two decision model sizes at the same error budget — how many requests each can decide cheaply before escalating uncertain cases to a larger model.

Both are phrased as tests worth running rather than complaints about what exists, and each is a single post.

  • Keep document structure in chunks1
  • Compare model sizes at one error budget1
See the 2 posts
飞叔@RyanJ_KOL

The handoff between Parse and retrieval is worth testing end to end. Keeping page coordinates and table structure attached to chunks would let an agent cite the actual cell or figure, rather than only the extracted text.

飞叔@RyanJ_KOL

For routing use cases, confidence calibration and abstention are as useful as top-line accuracy. I’d compare the two sizes at the same error budget: how many requests can each decide cheaply before escalating uncertain cases to a larger model?

In their words

How do people talk about Baseten?

The phrases people use most, counted once per person.

  1. 1“open weights”4
  2. 2“switching cost”2
  3. 3“per token”2
  4. 4“inference providers”2
  5. 5“open models”2
  6. 6“inference infrastructure”2
  7. 7“bare metal”2
  8. 8“open source”2

Posts are shown the way X shows them. One of them yours? Here’s why it’s shown, and how to take it off.

Also in Inference and model APIs

Get every post about Baseten, filtered and labelled

This page is the summary. Tracking gives you the stream behind it for any brand, keyword or @mention: each new pain point, request and question as it’s posted, with the spam and promotions already taken out.