Been using it as the web-fetch layer for a coding agent setup instead of maintaining my own scraper/proxy stack. The markdown-cleanup output is genuinely good for feeding pages into an LLM without a bunch of nav/footer junk eating tokens, and the brand/company enrichment endpoints saved me from writing a bunch of one-off logo/favicon scraping code. Typed SDK meant it was a 10 minute integration like they claim, not a "10 minutes then 2 hours of debugging" situation.
Context.dev
@yahia_bakour3 Hey Yahia!
Awesome product, we plan to integrate it with the product we're building. Hopefully it improves our agents web capabilities :)
Cheers!
Context.dev
@ankushkun Amazing! Happy to help with the integration, my email is yahia@context.dev if you have any questions.
Content Writing Jobs
@yahia_bakour3 let's goooo Yahia!
Context.dev
@tomaslau LETS GOOOO
@yahia_bakour3 This product looks really cool. How does it get around user permission restrictions? Often, we don't want to grant access to overly sensitive
Context.dev
@auula_ hey! what type of permission restrictions?
@yahia_bakour3 user data.
Migma AI
@yahia_bakour3 bro, just changed our logo and you got the new one already! This is almost realtime.
Context.dev
@adam_lab hahaha we work fast.
Co.Lab Learning
@yahia_bakour3 This looks great!!
Context.dev
@herenhuang thank you!!
Right, I wasn't doubting the render fidelity, I meant determinism across fetches. Same URL scraped today vs next week: if the live DOM reorders a section, the markdown shape moves with it and an agent that indexed against the first shape drifts. Do you expose a content hash or a diff between fetches, so a pipeline can tell 'page actually changed' from 'page just reordered'? That's the bit that decides whether I wire it into an agent loop or keep it a one-off pull.
Context.dev
@dipankar_sarkar good point and a great idea, i will build this shortly :)
WUPHF by Nex.ai
@dipankar_sarkar @yahia_bakour3 I'd be into this.
Context.dev
@dipankar_sarkar @frandias now i have to build it twice as fast
Context.dev
@dipankar_sarkar 100000% this is an excellent point, we're working on a custom "hash" that takes into account whether a page materially changed rather than shipped a new design or animation. Excellent question!
Congrats on launch #2 @yahia_bakour3 — "extract structured data into your own schema" caught my eye. I spend a lot of my time building exactly this kind of pipeline for e-commerce catalog data, so two questions from that trench:
How do you handle sites behind serious bot protection (Akamai/Kasada-tier)? Is the escalation abstracted away, or do those URLs fail with an error I can act on?
And is JS rendering on by default, or a per-request flag with its own latency + pricing cost? Recently when I looked for a solution related to it, came across this nice model from https://huggingface.co/jinaai/ReaderLM-v2 Jina AI, its really good.
Context.dev
@akbar_b Excellent question
For context (pun): I'm the founder, i also wrote 90% of the code myself, probably half of that by hand.
I built the API i always wish i had, which means
per minute rate limit, no concurrency bs
relentless focus on quality + cost efficiency
permanent backwards compatibility
Every single request is JS rendered.
1 credit = 1 successful scrape, even if we had to go to the moon to get the data.
Stealth is built in and automatic.
@yahia_bakour3 thats wonderful hear every request is JS rendered, will definetly give a strong try :)
Context.dev
@akbar_b please do! reach out to me at yahia@context.dev if you have any questions!
Putting crawl, Markdown cleanup, brand data, and schema extraction behind one API is a useful shape for agent builders! The place I would care about most is freshness, because stale web context can quietly poison a workflow. Do responses include enough source and timestamp detail for an app to decide when to reuse context and when to fetch again?
Context.dev
@monolithdread We have a maxAgeMs parameter so you can control the cache down to the ms, we're also surfacing our own "hash" that takes into account the web is a mess and that content shifts on pages without being meaningful changes.
Great question.
Voquill
Interesting. Congrats on the launch. How does it handle sites with aggressive bot protection or frequent layout changes?
Context.dev
@henry_habib we handle both. you never have to worry about a thing :)
Context.dev
@henry_habib Covered by default at no additional cost, i recommend you check it out on our free tier!
Pricing tied to successful scrapes only looks useful! Paying for failed fetches could be painful, I know this from own experience. Curious on the stealth layer as plenty of sites serve completely different content by visitor country (price, availability, language)... Can I pin the exit region per request or does the geo just fall out of whatever proxy the pool grabs that day?
Context.dev
@artstavenka1 yep the country parameter is in there!!
the brittle scraping problem is real. every ai agent i've built ends up with a graveyard of playwright scripts held together with hope. one clean api for "the web as markdown" is where we should have started.
genuine question: how do you handle sites that fingerprint requests? does the api rotate infrastructure or is that not really an issue at your scale yet?
Context.dev
@thenameisarian everything is handled for you, we do upwards of 40M requests per month right now and climbing super fast.
We're set up for scale.