Resolve melonstube's top-20 destination hosts into direct formats

Browse Source

melonstube is a meta-search aggregator that redirects out to ~120
third-party hosts. A live crawl of /new, /popular, and /rating
(~3,350 out-links) found the previous "assume yt-dlp can resolve it"
fallback was hiding real failures on the highest-traffic hosts:

- eporner.com is ~34% of all melonstube traffic by itself, and like
  the eporner channel (fixed earlier), serves a restricted preview
  clip without its hash/xhr dance.
- fapcat.com/momvids.com 403 a plain fetch (Cloudflare).
- Several others just needed real extraction instead of a guess.

Per-card enrichment now dispatches by destination host to:
- Eporner: reuses eporner's own hash/xhr + Referer resolution, moved
  to src/util/eporner_resolve.rs so both providers' single-provider
  builds still compile (HOT_TUB_PROVIDER=<id> only compiles the one
  selected provider module; src/util/* always compiles).
- Kvs (pornfreenow, analdin, babepump, w1mp, fetishshrine, pervclips,
  xozilla, ok.xxx, ebonypeek, xgroovy): a shared get_file extractor,
  the same KVS backend several other Hot Tub providers already scrape.
- KvsCfGated (fapcat, momvids): same extractor via the Safari18
  bypass client melonstube.com itself already needs.
- XhamsterMirror (xh.partners): a genuine xHamster mirror instance,
  Referer built from the actual resolved host.
- SomePorn: HLS playlist URL built from the page's video id directly.
- Pornone: direct <source> tag extraction.

flirt4free.com (a live-cam page, not a VOD) moved into the dropped-
host bucket alongside the existing junk/join.* hosts.

Enrichment is bounded (concurrency, a 10s per-card timeout, and
capped to the requested perPage rather than the full ~240-card page)
so a slow or rate-limited destination can't stall the response.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

...

This commit is contained in:

Simon

2026-09-22 08:28:05 +00:00

parent a9da585f45

commit c244461d0b

5 changed files with 746 additions and 203 deletions

Show all changes Ignore whitespace when comparing lines Ignore changes in amount of whitespace Ignore changes in whitespace at EOL

Download Patch File Download Diff File Expand all files Collapse all files

4

docs/provider-catalog.md

File diff suppressed because one or more lines are too long