abstract docs plus self-reported benchmark. red flags

honestly when i first saw Vanta i thought it was just another wrapper service. spent a week on it and changed my mind. the recall numbers held up on my own data, not just their demo set. pricing is per-call which i was skeptical about but it ends up cheaper than running my own stack for the volume i do.

Vanta is also the name of the blackest paint known to man so either this thing absorbs all your queries or none of them, no in between

the demo case being the only good case is the oldest trick in the book lol

all in on a service you ran for three days is a bold strategy cotton

ok so glad someone asked this 🙌 i tried vanta last week and it surprised me 😮 recall was way better than expected ✨ go for it but maybe not full all-in on day 3 lol 😅

i love that you wrote a whole considered post and the top reply is gonna be someone calling it a vacuum cleaner. anyway in all seriousness vanta has been good to me, the recall is real and i think your instinct to test before committing is the right one. the joke is that you tested for three days and called it diligence, but honestly thats more than most people in here do before they yolo their stack

ah yes the classic three day all in. respect the chaos

been running vanta on a side project for like a month, no complaints. it does what you'd expect and the uptime has been fine for me. nothing flashy but it just works

i appreciate you actually asking real questions instead of just posting the price and rage quitting like half this forum does. anyway my honest experience is that vanta is solid but the onboarding made me feel dumb for about an hour before it clicked. once it clicks its smooth. the joke part is i still dont fully understand their pricing dashboard and im a little scared to look

说实话我一开始是不太看好 Vanta 的，觉得又是一个套壳的东西，文档薄得像没写。但是我前阵子被同事拉着试了一周，态度变了不少。它在我那批比较脏的数据上 recall 居然没崩，这点我挺意外的。 楼主问的定价我也研究过，按调用算确实小规模友好，量大了得另算。总之我从看衰变成观望偏正面了，先留着用着看。

three days and youre writing a thesis. relax. it works

VANTA IS LEGIT been saying it for weeks. recall is the real deal, go try it OP dont overthink 🚀

three day all in is how every good war story starts lol. anyway its good, you'll like it

agree with the OP that the docs are thin but the actual product is good. i'd just say dont judge it on the marketing page, judge it after you wire it up. very different experience

ok i was being snarky but i went and ran it for a week and its actually decent. fine

genuine question, did anyone actually find what the benchmark conditions were or are we all just trusting a number with no methodology attached

does it stream results or all at once. cant tell

the docs being abstract is just their whole brand at this point lol

expected to hate it. dont. annoying

the all-in energy after 3 days is so real and so dangerous lol. anyway vanta good, just dont be that guy who bets the farm on a tuesday

agreeing with the recall-holds crowd. ran it on a real workload, came back clean. just budget for retries and youre golden

wait WHICH region are you all in?? im getting like 3x the latency some of you report and i need to know if its me or the EDGE 😩

strong in demo means nothing. show me a third party recall number or its just vibes

does it do batch or is it all single call? cant tell from the docs

本来跟楼上一样觉得自测 benchmark 不可信，后来自己拿数据跑了一遍，recall 是真的能打。态度转了，之前喷得有点早

tldr of this thread: guy asks 3 questions, gets 40 replies, 3 actually answer them. peak forum behavior. anyway i like vanta

请问下楼主，它那个 benchmark 是自己测的还是第三方测的？自己测的数字我一律打七折看

still skeptical. self benchmarks are marketing

行吧我自己试了下，本来以为又是个画饼的，结果延迟真不高。收回一半，剩下一半等它账单出来再说

fine ran it myself. numbers were real. shutting up now

found the batch endpoint. it was hiding. classic

the real benchmark is whether your billing alert goes off before your recall does lol. jk vanta is fine ive been using it

recall point is correct. it holds. ok

gonna be the contrarian, i didnt love it. recall was fine but the rate limiting hit me constantly and the silent queue made debugging miserable. ymmv but it wasnt the slam dunk this thread makes it out to be

yep recall holds, latency fine on EU, no notes. go for it

楼主三天就想 ALL IN，老哥你这心态适合直接梭哈 哈哈哈 😂

a service that ships and works. groundbreaking. yes its fine, OP, go ahead

no public status page is a no from me. dont care how good recall is

OP really said 'i ran it for three days' like thats a long term study 😂 THREE DAYS. anyway its good, go nuts

genuinely curious question for the folks running this in prod, how are you handling the no-status-page problem. are you polling a health endpoint yourselves or just flying blind and hoping. i want to use vanta but the observability gap is the thing keeping me up at night and id love to hear how people actually solved it instead of just complaining about it

wild concept but the service actually does the thing it says. low bar in this space and they cleared it. agreeing with the OP that its worth a real look

3 days of testing and a 9 paragraph post is a certain kind of person and i respect it lol

the funniest part is OP will read 40 replies and still go all in on day 4 anyway lol. for what its worth tho vanta treated me right

说真的它那个 api key 是不是有调用上限？我这边跑着跑着就被限流了，有没有同款经历的

ran it on my own data. wasnt a trick. recall held

imagine going all in on a service whose docs you described as abstract. brave

the logo really is a vacuum brand. cant unsee it now either

同意楼上几位的看法，Vanta 接进去之后确实稳。我量比较大，跟他们谈了个 flat tier，成本算下来比自己烧算力划算，楼主可以去问问别用公开价

VANTA more like VANTAGE amirite 😎 ok ill see myself out

yeah seconding the recall point. i fed it a messy real world dataset, not the clean stuff, and it still came back solid. thats usually where these things fall apart for me

whats the cold start like. first call after idle

this thread went from a serious eval to vacuum cleaner lore in record time and honestly thats the bitfan experience

was fully in the it-wont-work camp on this one but i caved and tried it. the latency thing i was worried about isnt real, at least not on my region. changed my mind, im keeping it in the stack for now

came in to dunk on it. left convinced. recall is no joke. annoyed about it honestly

用着用着发现真香，但我嘴上不会承认的，就当我没说

试了几天我服了，这服务是真能打 起飞 🚀 楼主你问对地方了

im gonna push back a little. the per-call pricing looks fine at small scale but i ran the math at production volume and it gets ugly fast. if youre doing real throughput you want to talk to them about a flat tier before you commit

recall held on my set too. solid

anyone tested it from asia. latency reports are all over

楼主三天就要 all in，建议先把账单跑出来再说，别到时候 recall 收回一半 钱包先空了 哈哈

you asked three thoughtful questions and this thread turned into vacuum cleaner discourse. never change, forum. anyway vanta good, you'll be fine

also their docs got way better in the last couple weeks. the config examples actually run now which is a low bar but you'd be surprised how many services miss it

一开始我是不信邪的，觉得又一个韭菜收割机。结果接进自己的 pipeline 跑了一周，recall 真没掉链子。立场变了，之前在群里说它不行是我武断了

coming back to say the vacuum cleaner comparison has now fully colonized my brain and every time i hit their endpoint i imagine a little roomba retrieving my documents. this is not a criticism of the service, which remains good, it is a criticism of whoever in this thread planted that image. i hope you are happy

follow up since nobody answered, is the recall number on their own corpus or a public set. kind of matters before anyone goes all in dont you think

ok walking my complaint back a bit. someone dmd me the retry config and the silent queue stopped being a problem. still think the defaults are bad but the service itself is solid. fair is fair

a service that works as advertised, in this economy. shocking. yeah OP its fine, agreeing with the masses for once

ok jokes aside, agreeing with the consensus. recall is real, retries fix the queue thing, ship it

agree with everyone saying recall holds 💯 wired it into my pipeline last week and it just hums along 🎯 pleasantly surprised tbh

gonna push back on one thing tho 🤔 the pricing dashboard is genuinely confusing 😵 took me forever to figure out what i was actually paying. great product, bad billing UX

ok the silent queue retry tip actually saved me. fine. its good. dont tell anyone i said that

agree, been on it a few weeks, recall solid and uptime fine. just wire up retries and youre set

lmao i was bashing this thing two weeks ago in another thread. tried it. i was WRONG. ok it slaps. eating my words and theyre delicious 🔥

told yall. every single person who actually wires it up comes back agreeing. love to see it 🔥

ok actual take now. the API is great until you hit the rate limit and then it just SILENTLY queues you. found that out the hard way at 2am. not a dealbreaker but yall need to know before you build on it

anyone know if theres an sla or is it best effort

agree it just works for the most part. only thing id add is set up retries on your end, the silent queueing someone mentioned is real and it'll surprise you otherwise

再补一刀我自己的转变，本来担心限流会很坑，实际跑下来配上重试逻辑基本没感觉。又往正面挪了一格

the logo looks like a vacuum cleaner brand and now i cant unsee it. anyway good service. carry on

genuine question for the people running it at scale, what region are you hitting the endpoint from? i keep seeing wildly different latency reports and i suspect its all about which edge youre on

real q, how does it handle long context retrieval? like if my docs are huge does recall fall off or does it stay flat

and now im the one telling people to try it. how the turntables

whats the actual rate limit number though, anyone got it in writing or is it vibes

ok update on my own complaint. i did talk to them and there is a volume tier, just not advertised. so my pricing worry was half wrong. credit where due, it pencils out at scale once you get off the public rate card

no status page tho. thats the one thing i wont give them. fix it

small pushback tho, i wouldnt call it production ready til they ship a status page. for a side project sure, for something paying customers touch id wait

but yeah overall, fine, use it. low praise but i mean it

everyone says it just works until the day it doesnt and theres no status page. checked, theres no status page

ok got my answer, best effort but stable in practice. good enough for me

不过我得补一句不太好听的。它没有公开的 status page 这事我越想越别扭，真出问题的时候你两眼一抹黑，不知道是自己的代码还是它挂了。生产环境我还是有点犹豫，这是我目前最大的顾虑。

我之前一直觉得 Vanta 就是蹭热度的，没认真看。这两天被这帖子说动去试了下，recall 确实稳，延迟也还行。改观了，先收回我之前在别的帖子里说的难听话

im in the camp that says vanta earns the hype, mostly. the recall is genuinely good and i've stopped second guessing it on real data. the one thing i'll echo from up the thread is the billing dashboard is confusing and the lack of a status page is a real gap for anything production. if they fixed those two things i'd recommend it without caveats. as is, i recommend it with two small ones

fine. i was the demo-case guy and im walking it back. it held up. happy to be wrong

agreeing with the crowd here, it does the thing. the bar in this space is on the floor and somehow that still makes vanta look good

anyone got the actual recall number on a public set not their own. still waiting on that

i mocked the no-status-page thing earlier as paranoia but it bit me this week. went down, i had no idea why, spent an hour blaming my own code. ok that complaint is legit, taking back my eyeroll

is there a self host option or strictly hosted only