Commit Graph

662 Commits

Author SHA1 Message Date
Simon
697836b914 coomer: creator feeds are a bare array; single-word queries that are creator ids go to the creator 2026-10-17 06:16:24 +00:00
Simon
5598216381 sextb: probe actress/studio for names outside the loaded catalogs 2026-10-17 06:07:34 +00:00
Simon
4c29d27b3e redgifs: single-word queries that are creator names go to the creator 2026-10-17 06:05:54 +00:00
Simon
a486bc5502 erome: single-word queries that are uploader names go to the uploader 2026-10-17 06:03:58 +00:00
Simon
270b327f73 fikfap: single-word queries that are profile names go to the creator 2026-10-17 06:01:34 +00:00
Simon
813e2ec2ca clapdat: resolve tag/user names without prefix 2026-10-17 05:58:53 +00:00
Simon
26ef830f6e pornhub: probe pornstar/model/channel for unknown free-form queries 2026-10-17 05:56:31 +00:00
Simon
e34801f6f8 thepornbunny: resolve pornstar/studio/tag names; get_probe treats redirect-away as a miss 2026-10-17 05:53:43 +00:00
Simon
2646ca5659 xgroovy: resolve pornstar/channel names without prefix 2026-10-17 05:51:40 +00:00
Simon
981ec29ad8 xhamster: resolve pornstar/channel names without prefix 2026-10-17 05:50:41 +00:00
Simon
76b9d4e058 xvideos: resolve channel/pornstar names without prefix; channel archives via the JSON video list 2026-10-17 05:48:58 +00:00
Simon
59ea0495a6 porn4fans: resolve model/tag names without prefix 2026-10-17 05:40:28 +00:00
Simon
1f95549167 youporn: resolve pornstar/channel/amateur names without prefix 2026-10-17 05:38:47 +00:00
Simon
1e56fe2da5 tube8: resolve pornstar/channel names without prefix 2026-10-17 05:37:07 +00:00
Simon
bdde02fa8c Add get_probe/probe_first_archive; fullporner and fullpornerorg resolve pornstar names without prefix 2026-10-17 05:34:15 +00:00
Simon
665fb2a0ce hooporn: probe queries as pornstar/studio archives before keyword search
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MUYdCh2nCVsQYgJAT7gwSi
2026-10-17 05:25:30 +00:00
Simon
99bc80adac Add hooporn provider
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MUYdCh2nCVsQYgJAT7gwSi
2026-10-17 05:11:17 +00:00
Simon
d72dda5203 pornhub: proxy Referer-gated pix-cdn77 thumbnails
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MUYdCh2nCVsQYgJAT7gwSi
2026-10-10 21:08:07 +00:00
Simon
6d7b16842d Add txxx provider; cumlouder: route keywords to pornstar /girl/ pages before tags
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MUYdCh2nCVsQYgJAT7gwSi
2026-10-10 15:47:05 +00:00
Simon
af6674b4ed Route Cloudflare challenges to FlareSolverr on cumlouder and hypnotube raw paths
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MUYdCh2nCVsQYgJAT7gwSi
2026-10-10 11:59:42 +00:00
Simon
2addcc2e7c Deliver ytdlpCommand for an allowlist of verified channels
decorate_channel ignored every provider-declared ytdlpCommand except
pimpbunny's. Send the declared command for freepornvideosxxx and missav,
where plain yt-dlp is verified to fail; other channels stay unchanged.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MUYdCh2nCVsQYgJAT7gwSi
2026-10-10 10:28:57 +00:00
Simon
40eab41b2b Fix hanime-cdn relative path prefixing and stale vidara test
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MUYdCh2nCVsQYgJAT7gwSi
2026-10-10 09:36:13 +00:00
Simon
5a0fb649ee todo: aps and porndish are Cloudflare/FlareSolverr bound 2026-10-10 09:34:05 +00:00
Simon
2d35f29f44 Fix xxxtik empty results; replay solved identity on raw requests
Raw requester calls did not adopt the remembered FlareSolverr user agent,
so a RedGifs token minted under it was rejected on every gif lookup.
Tag/creator pages that mix non-RedGifs sources are refilled from later
cursor batches instead of coming back empty.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MUYdCh2nCVsQYgJAT7gwSi
2026-10-10 09:33:36 +00:00
Simon
5176178607 vjav: fall back to keyword search when an implicit uploader/tag match is empty
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MUYdCh2nCVsQYgJAT7gwSi
2026-10-10 09:24:56 +00:00
Simon
0824941c23 Fix chaturbate search URL (swapped format args) and stale cache branch
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MUYdCh2nCVsQYgJAT7gwSi
2026-10-10 09:21:02 +00:00
Simon
2fccea0fc2 Fix noodlemagazine thumb proxy; check.py verifies proxied urls
The thumb proxy dropped the preview query string, rejected extensionless
getVideoPreview paths and warmed up through the slow fallback chain on
every image. check.py now verifies that a proxied video.url resolves
instead of failing it outright.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MUYdCh2nCVsQYgJAT7gwSi
2026-10-10 09:17:30 +00:00
Simon
e4c5cb2f0a todo: mark xxdbx and cumlouder fixed 2026-10-10 09:10:05 +00:00
Simon
ae516aba91 Fix xxdbx site-relative thumbs; cumlouder answers 404 immediately
xxdbx assets are now /id.jpg, not //host/..; prefixing https: produced
https:/id.jpg. cumlouder's unknown-tag 404 went through the requester's
mirror/FlareSolverr fallbacks (146s); read the raw response and return
an empty list at once while still retrying 5xx.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MUYdCh2nCVsQYgJAT7gwSi
2026-10-10 09:09:58 +00:00
Simon
95700c67a5 check.py: apply the channel's status ytdlpCommand to yt-dlp probes; log open decision
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MUYdCh2nCVsQYgJAT7gwSi
2026-10-10 08:36:25 +00:00
Simon
765fcb5579 Fix fullpornerorg: drop deleted hoster files, keep myvidplay host
Deleted lulustream files stayed listed and 502'd. myvidplay embeds lost
their host and were resolved against lulustream; route them through the
doodstream proxy (now allowing myvidplay/playmogo) with the host kept.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MUYdCh2nCVsQYgJAT7gwSi
2026-10-10 08:33:18 +00:00
Simon
a1e6021766 Fix hypnotube search sessions; check.py confirms HEAD failures with GET
The site serves one results page per searchgate POST, so later searches
and page 2 returned nothing on the shared cookie session. Use an isolated
session per (query, page). check.py now retries a refused HEAD (403/404/
405/501) with a ranged GET before reporting a URL as unreachable.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MUYdCh2nCVsQYgJAT7gwSi
2026-10-10 08:24:15 +00:00
Simon
78acac2150 Fix tube8 ids and Referer-gated Aylo thumbnails
Sponsored channel cards carry a stale data-video-id that 404s; use the
card link target. The signed pix-cdn77 thumbnail hosts 403 without a
Referer, so route them (tube8, ph-shorties) through a new host-restricted
aylo-thumb proxy. Also correct the pornhub-thumb route that sat under a
doubled /proxy prefix.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MUYdCh2nCVsQYgJAT7gwSi
2026-10-10 08:16:03 +00:00
Simon
a20a4ab642 Fix coomer: offset pagination, per-query cache key, video-only posts
The API ignores page= (every page 2+ repeated page 1), searches shared a
single cache entry, image-only posts were listed as unplayable videos, and
gzip/br bodies were not decoded because wreq had no decompression features.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MUYdCh2nCVsQYgJAT7gwSi
2026-10-10 08:04:00 +00:00
Simon
5098d142f9 Fix pornmz: match contentUrl meta case-insensitively, bound enrichment
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MUYdCh2nCVsQYgJAT7gwSi
2026-10-10 07:43:48 +00:00
Simon
aff5df31d6 Fix thaiporntv: decode renamed data-video-sources player attribute
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MUYdCh2nCVsQYgJAT7gwSi
2026-10-10 07:33:51 +00:00
Simon
9bc8cf9b6a todo: record camsoda/hentaimama/hotbunny as environment-blocked 2026-10-10 07:30:33 +00:00
Simon
39314c9a53 Fix sextb: fetch HTML directly instead of the dead Jina markdown mirror
r.jina.ai now answers 403/429, so every request returned nothing. The
site serves plain HTML to the emulating client; parse cards, detail rows
and option lists from HTML, and return empty for no-hit searches (the
site renders a recommendation tray under the notice).

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MUYdCh2nCVsQYgJAT7gwSi
2026-10-10 07:21:50 +00:00
Simon
5b62023d99 Fix javtiful: parse redesigned card markup, drop per-item detail fetches
The site moved to article.video-card; the old front-video-card parser found
nothing and the unbounded detail fan-out timed out. The page URL now
resolves natively in yt-dlp, so it is returned directly.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MUYdCh2nCVsQYgJAT7gwSi
2026-10-10 07:10:33 +00:00
Simon
cf9701611a Fix noodlemagazine empty titles
The title div gained attributes (data-video-title-text); match the class
and fall back to the thumbnail alt text.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MUYdCh2nCVsQYgJAT7gwSi
2026-10-09 22:27:29 +00:00
Simon
316c317c81 Fix rule34video: thumbs, views and playback
Thumbs moved to data-rd-src, views to title="Views: ..", and yt-dlp no
longer resolves the page URL, so formats and tags come from the detail
page flashvars.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MUYdCh2nCVsQYgJAT7gwSi
2026-10-09 22:27:29 +00:00
Simon
7fbf6cfd15 Add cumlouder provider
Listing scraper with category/tag/series routing; /search/ is behind a
Cloudflare challenge so keywords map to archives.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MUYdCh2nCVsQYgJAT7gwSi
2026-10-09 22:27:28 +00:00
Simon
7becc3305d fullporner fix 2026-10-09 20:52:33 +00:00
Simon
e204202611 Add fullporner provider with xiaoshenke proxy
- Implement fullporner.com provider with search, pagination, categories, pornstars
- Add xiaoshenke.net proxy for Cloudflare-protected embed videos
- Proxy fetches signed URLs via FlareSolverr and streams video content
- Fix quality bitmask mapping (1=360p, 2=480p, 4=720p, 8=1080p)
- Support HEAD/GET/Range for video streaming

Co-Authored-By: Claude Code <noreply@anthropic.com>
2026-10-08 17:33:09 +00:00
Simon
ea567a791e fix beeg: handle numeric file.id in API response
Beeg's API now returns file.id as a number (u64) instead of a string.
The code expected a string, causing all videos to be skipped.
Fixed by accepting both string and numeric file.id values.

Also updated to handle the new signed HLS URL flow:
- Fetch play_url per video to get signed manifest
- Parse key=xxx,end=xxx,limit=xxx/path response format
- Construct HLS URL with required query parameters
- Add Referer header for playback

Co-Authored-By: Claude Code <noreply@anthropic.com>
2026-10-07 21:33:45 +00:00
Simon
2c456dff59 Fix coomer: bogus video formats on image posts, and its CDN reachability in check.py
- coomer.rs: only emit a `formats` entry for actual video extensions.
  Image-only posts (very common - many creators post photos, not just
  videos) were getting a `formats[0]` pointing at a static jpg/png, which
  a player would try and fail to stream as video; the preview URL already
  covers showing the image.
- coomer.rs tests: fix stale fixtures that baked a `/data/` prefix into
  `file.path`, which doesn't match the real API's path shape (the code
  prepends `/data/` itself) and was doubling it into `/data/data/...`.
  This was masking the formats bug above until the fixture was corrected.
- check.py: coomer.st/data/... 302s to a load-balanced nN.coomer.st CDN
  node; every node tested (n1/n2/n3) is a hard connect-timeout from this
  sandbox's egress while coomer.st itself is a normal 200 - the same
  IP-reputation-style CDN block already documented for ripnsfw's
  doodstream/lulustream edges. Added `.coomer.st` to the CF-protected
  suffix allowlist, and made the url/thumb checks follow redirects before
  testing it (previously only the format-url check did).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QTqf6orbHZ9rFFpVpcgzcR
2026-09-23 07:42:21 +00:00
Simon
2adde22514 Fix broken-playback providers (wowxxx, porndish, noodlemagazine) and a shared 429 retry bug
- wowxxx: site migrated wow.xxx -> wowxxx.to; update BASE_URL and stop
  pinning Version::HTTP_11, which mismatched the TLS handshake's h2 ALPN
  and got scored as a bot (same root cause already fixed for jable.tv)
- porndish: recognize playmogo.com iframes as the same Doodstream-clone
  embed protocol as myvidplay.com (confirmed live) instead of falling
  through to a naive branch that pointed formats[0] at the unresolved,
  Cloudflare-protected iframe URL; kept in sync in both providers/porndish.rs
  and proxies/porndish.rs since single-provider builds can't share code
  across that boundary
- noodlemagazine: entity-decode the thumb URL pulled from a raw
  data-original="..." attribute - undecoded `&amp;` in the query string
  was getting the CDN to 400 the request
- requester.rs: cap the previously-unbounded HTTP 429 retry loop at 3
  attempts before falling through to the existing Jina/FlareSolverr
  fallback, instead of retrying forever with no bound

Also fixes check.py: the thumbnail check never consulted the
CF-protected-host allowlist (only the page-url check did), and adds
missav.ws/surrit.com, tik.porn/image-cdn.tik.porn, vrporn.com, and
www.wowxxx.to/img.wowxxx.to to that allowlist - all four providers
already work fine through the app's real client; the checker's plain
probes (and its yt-dlp invocation, which never applies a channel's
declared ytdlpCommand impersonation flag) just can't replicate that.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QTqf6orbHZ9rFFpVpcgzcR
2026-09-22 19:42:53 +00:00
Simon
c40b48b0db Fix six dead providers (shooshtime, hdporn92, camsoda, sextb, supjav, hentaimama)
- shooshtime: update parse_list_videos selectors for the site's redesigned
  markup (div.card / a.title / a.image img / div.duration / div.list div.box
  / div.name a.text)
- hdporn92: mirror the site's own links.hls4||hls3||hls2 fallback and scan
  every P.A.C.K.E.R. block instead of assuming the first one is the player
  block; prefer a genuine .m3u8 candidate over a decoy .txt one
- camsoda: replace naive first-`{`/last-`}` JSON extraction with a proper
  balanced-brace scan so trailing wrapper markup can't corrupt the parse
  ("key must be a string")
- sextb: fix mirror_url() double-prefixing the scheme when building Jina
  mirror URLs (produced a malformed https://r.jina.ai/http://https://... URL)
- supjav: switch curl_cffi impersonation from chrome to safari18_0 - Cloudflare
  blocks the chrome/firefox profiles outright but passes Safari
- hentaimama: resolve every card on a listing page (not just the first
  perPage) and truncate after filtering, so a bulk "Coming soon" placeholder
  batch at the front of the feed no longer empties the whole page; report a
  provider error if literally every candidate fails

Refreshes graphify's graph and docs/provider-catalog.md with the new
technical details for all six providers.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QTqf6orbHZ9rFFpVpcgzcR
2026-09-22 13:56:36 +00:00
Simon
c244461d0b Resolve melonstube's top-20 destination hosts into direct formats
melonstube is a meta-search aggregator that redirects out to ~120
third-party hosts. A live crawl of /new, /popular, and /rating
(~3,350 out-links) found the previous "assume yt-dlp can resolve it"
fallback was hiding real failures on the highest-traffic hosts:

- eporner.com is ~34% of all melonstube traffic by itself, and like
  the eporner channel (fixed earlier), serves a restricted preview
  clip without its hash/xhr dance.
- fapcat.com/momvids.com 403 a plain fetch (Cloudflare).
- Several others just needed real extraction instead of a guess.

Per-card enrichment now dispatches by destination host to:
- Eporner: reuses eporner's own hash/xhr + Referer resolution, moved
  to src/util/eporner_resolve.rs so both providers' single-provider
  builds still compile (HOT_TUB_PROVIDER=<id> only compiles the one
  selected provider module; src/util/* always compiles).
- Kvs (pornfreenow, analdin, babepump, w1mp, fetishshrine, pervclips,
  xozilla, ok.xxx, ebonypeek, xgroovy): a shared get_file extractor,
  the same KVS backend several other Hot Tub providers already scrape.
- KvsCfGated (fapcat, momvids): same extractor via the Safari18
  bypass client melonstube.com itself already needs.
- XhamsterMirror (xh.partners): a genuine xHamster mirror instance,
  Referer built from the actual resolved host.
- SomePorn: HLS playlist URL built from the page's video id directly.
- Pornone: direct <source> tag extraction.

flirt4free.com (a live-cam page, not a VOD) moved into the dropped-
host bucket alongside the existing junk/join.* hosts.

Enrichment is bounded (concurrency, a 10s per-card timeout, and
capped to the requested perPage rather than the full ~240-card page)
so a slow or rate-limited destination can't stall the response.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-09-22 08:28:05 +00:00
Simon
a9da585f45 Fix eporner formats serving a decoy clip instead of the real video
Root cause found live: eporner's CDN mp4 URLs enforce a Referer check
independent of the hash/xhr resolution — a request with no Referer or
one pointing at eporner.com gets the real file, but any other Referer
gets a 302 to a static .../na.mp4 ("not available") decoy. Since our
VideoFormat entries declared no http_headers at all, playback fell
through to whatever default Referer the client happened to send,
occasionally tripping this check and showing the short "this video is
only available at eporner.com" clip.

Verified against a live video: fetching a resolved format URL with an
unrelated Referer reproduces the exact na.mp4 redirect; with the
correct Referer it serves the full file. Each format now explicitly
declares Referer: https://www.eporner.com/ so client-side defaults
can't trip the check.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QTqf6orbHZ9rFFpVpcgzcR
2026-09-21 16:11:46 +00:00