同意楼上，这种主页benchmark我从来不看。冷启动那块是硬伤，他们不解决我是不会上量的

diego did you test against their eu endpoint or us. asking because mine was way worse on one of them

i actually think diegos skepticism is healthy and i agree with the core of it but i want to add some nuance. ive been running ninja in a small side project for about three weeks now and the experience really does split into two worlds. warm path is genuinely good, like good enough that i stopped thinking about it. cold path is where it falls apart and thats a real architectural thing not a tuning thing. so both camps are kinda right

主页数字别信。等冷启动数据再说

good thread to start diego. one genuine question for you since youve poked at it most, when you saw the spikes were they correlated with autoscale events or did they happen on a stable node too? trying to figure out if its purely cold start or something deeper in their scheduler

agree with diego, the homepage benchmark is meaningless without methodology. what hardware, what concurrency, what region. until they publish that its just a poster

diego coming in hot day one lol. let the man cherry pick his counter benchmark first

the page numbers are marketing. ignore them. wait for third party runs

请问下你们说的预热端点是哪个啊，文档里我咋没找到，新手撸毛中求带

genuine q 🤔 are the spikes on first request only or do they come back after idle? makes a huge difference for how id deploy this 🙏

我觉得Diego说得有道理，主页那个数字明显是理想环境跑出来的。真要看就看连续高并发的曲线，光看峰值没意义，别被截图忽悠了

冷启动这么差还吹快，过了

diego right to be careful but warm ninja is legit fast. tested it

别吹了！我这边跑出来峰值延迟翻倍，所谓的快就是挑时间段截图，真上量直接崩，韭菜别冲！

same spikes on my end. demo is staged. not impressed

lol i was hyping this thing nonstop last month and then i actually paid for sustained throughput and YEAH the bill scaling is rough. still like it but im dialing the hype back a notch ngl

warm path good. cold path bad. thread over. somehow it wont be

yeah no i had the exact same spikes diego did and i didnt do anything weird. the demo lies a little, sorry not sorry

diego skeptical of ninja is my roman empire at this point. every thread brother 😤 LOVE the energy never change

ninja so fast it outran my expectations and my wallet

yeah idk man the demo felt good but the second i wired it into something real the cold starts killed my whole flow. not buying the hype yet

jokes aside tho i had the same spike problem and never figured out if it was them or my client. leaning towards it being real

yeah im siding with diego for now. ran it twice and got two totally different latency profiles which is its own red flag tbh

我同意热路径的说法，自己跑了几天确实稳。想补充一点，ninja这个项目最大的问题不是性能而是文档，预热这种关键操作居然藏着，新手根本摸不到门。性能我给好评，文档真心得改

nah ninja is THE play rn. ran the demo myself and it FLEW. you were just unlucky with the window diego, try again off peak and watch it cook

at this point ive spent more time benchmarking ninja than actually building the thing i wanted to build with ninja. theres a metaphor in there somewhere about all of us i think

别画饼了好吗，ninja的demo我也跑过，峰值直接起飞然后掉下来摔死。截图都是挑出来的，真实负载根本撑不住，别信！

话说你们预热具体怎么配的，是定时打个空请求还是有专门的接口，求个具体步骤

ah yes the famous warmup endpoint, documented exclusively in the dreams of the dev team

ok im annoyed to report that after mocking it i actually tried the keepalive and the latency did flatten out. ill stop being a hater. mildly.

the two runs being different is just cold vs warm me probably lol. i played myself

ninja fast until you look at it

love that the killer feature of ninja is a ping you write yourself. truly cutting edge ⚡

fair callout. for what its worth the ninja team did post a longer run in their discord last week and the p99 held up better than i expected. not perfect but not the trainwreck people feared either. worth a look if you havent seen it.

i hear everyone saying warmup solves it but i think were collectively normalizing something we shouldnt. a serving layer that needs a babysitter keepalive to be usable is an unfinished serving layer. the warm numbers are real and good, im not denying that, but the bar should be that it just works cold too. i dont want to give them a pass on the part that actually matters for real deployments

按你们说的预热弄了一下，稳态确实改善很多，之前是我冤枉它了。收回一部分意见

ninja的用户体验就是先让你冷启动哭一次，再让你预热爽一次，主打一个先苦后甜哈哈

i want to second the warm path praise but in a measured way. ive had ninja serving a small internal tool for two weeks with a keepalive and the steady state is honestly some of the best ive seen at this price. the cold start critique is valid and i hope they ship a real fix, but for a persistent workload its already a strong pick. depends entirely on your traffic shape

行吧我收回一半。老周给我看了他的预热配置，重新跑了一遍稳态确实稳，是我之前没预热好。冷启动还是拉胯但热起来是真香

started this thread agreeing with the haters and im flipping. set up the keepalive, watched it for a day, its flat and fast. cold start aside im actually impressed now

whats the cold start time actually. nobody posts it

wait people are saying theres a fix in the discord, can someone just paste it here instead of making me dig. asking nicely

im with diego here honestly. one good run off peak doesnt mean much when the whole pitch is handling load. the spikes are the actual question and nobody answered it yet

wait what region are you all testing from? half the latency complaints i see are just people routing through the wrong endpoint. genuinely curious if thats whats going on here

补一句，之前喷得太狠了。预热到位之后那个吞吐确实能打，是我没耐心。冷启动这块他们再优化优化就真能梭哈了

update, laozhus warmup tip actually fixed most of my pain. still annoyed i had to find it on a forum and not in the docs but credit where due, its way better now

wait so is the WARMUP ENDPOINT documented anywhere official or is it just tribal knowledge in the discord?? 🔥 need to know before i commit a project to this

i was firmly anti ninja last week but reading this whole thread changed my read. warm path good, cold path needs work, fix is known. thats just a normal early product not a scam. softening my stance

so for the warmup approach, are people scripting a keepalive ping or is there a real persistent mode? trying to understand what im signing up for

for real tho once warmed it absolutely RIPS. team just needs to fix the front door experience and its game over for the competition 🚀

honest question for the people who got it working well, are you keeping a node permanently warm to dodge cold start? because if so id love to know the cost of that, since that changes the whole value math for me. trying to figure out if its actually cheap or just cheap on paper

ninja cold start be like: hello? 🥶 ... oh youre back 🥵 ... wait where did you go 🥶 emotional rollercoaster fr

saved my whole grafana dashboard just for ninja cold starts. my therapist asked me to describe a hobby and i showed her a p99 chart. she did not laugh

ninja演示飞起，上真实负载就拉胯，老剧本了。截图一时爽，p99火葬场

everyone celebrating the keepalive fix like its not just paying to keep a gpu warm 24/7. thats not a fix thats a bill. push back from me

i keep seeing the warm path defense and i dont think it holds up under real conditions. most production traffic is spiky by nature, you scale up and down, and every scale event is a cold start. so if cold start is bad your effective experience is bad no matter how pretty the steady state graph looks. i wanted to believe but the architecture has to actually solve this part

不过我还是坚持一点，需要手动预热才能用的东西，对生产环境就是不友好。热路径再快也得先解决这个，不然我只能给一半分

every ninja thread is the same arc. someone shills, someone benchmarks, someone discovers cold start, and then we all go home. like clockwork honestly

anyone got actual cold start ms numbers

that said the warm numbers are stupid good, ill give them that. credit where its due

and is that with or without the warmup endpoint

ninja docs missing the one endpoint everyone needs. classic

ok did the keepalive ping thing and yeah it works, latency is flat now. agree the warm experience is genuinely good

i want to push back on the people saying warm performance settles it. for anything user facing the cold path IS your worst case and your users feel the worst case more than the median. so no, a clean warm number does not close the question for me, it kind of dodges it 🐸

ok i came in agreeing with the skeptics but i ran my own batch last night with proper concurrency and... it actually held. the spikes everyone screenshots are cold start, once its warm its steady. walking back my earlier take.

and yeah ive been running the keepalive for two days now, latency is flat as a board. fine. it works. moving on

follow up, does anyone know if the keepalive counts against your rate limits? dont want to find out the hard way

ninja onboarding flow: step one cry, step two find a random discord message, step three ascend. seamless experience truly

warm path checks out for me too, ran it most of yesterday and it was flat. the keepalive trick is the move

ninja: fast in the demo, mysterious in prod. truly the schrodingers subnet

warm path is the real story here. agreed

fine. i was the loudest skeptic in this thread and im eating it. ran the warmed config, its good. the cold start gripe stands but i was wrong about the whole thing being smoke

想问下大家最后那个keepalive ping的频率定多少合适，太频繁怕撞rate limit，太松又预热不住，有经验的说下呗

does warmup even persist across their autoscale or does it reset every node

ok got my answer warm path checks out for me too

warm path is solid. ran it. agree with larica

ninja really said FAST in prod but only if you keep poking it like a tamagotchi 🥲 honestly kind of love it

预热接口到底官方文档有没有写。求个链接

im going to be the wet blanket and say if your whole defense of a serving product is a community discovered keepalive hack, the product isnt ready for the people who arent on the discord. thats most people

cold start aside its still the best warm latency ive measured on a subnet this cheap

warm steady at what concurrency tho. give a number

real q tho 🤨 does the keepalive ping have to hit the same node or does their LB spread it around uselessly? 🤷

yeah the off peak thing is the tell. anyone can look fast when nothings hitting them. id want to see it under contention before calling it anything

came here ready to dunk on it after reading the early replies but the warmed runs people posted changed my mind. its good, the cold start just needs a fix. fair is fair

ok after a week of this i agree, warm ninja is the real product and its good. just keep it warm and youre fine

still no. needing a keepalive to be fast is a workaround not a feature. judging it on the cold path

warm being fine doesnt fix cold start for bursty traffic. thats most real workloads

also 👀 anyone tested if cold start got better in the latest version or are we all still on the old one?

ok the warm path consensus matches what i got too 🔥 once its rolling its smooth, thanks all 🙌