Files
hottub/docs/provider-fix-todo.md
Simon 2fccea0fc2 Fix noodlemagazine thumb proxy; check.py verifies proxied urls
The thumb proxy dropped the preview query string, rejected extensionless
getVideoPreview paths and warmed up through the slow fallback chain on
every image. check.py now verifies that a proxied video.url resolves
instead of failing it outright.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MUYdCh2nCVsQYgJAT7gwSi
2026-10-10 09:17:30 +00:00

9.2 KiB

Provider fix todo

Source: check.py + search probe audit, 2026-10-10. Serial re-run of every flagged channel, so timeouts below are not load artifacts. Update the status column as each item is fixed and verified (HOT_TUB_PROVIDER=<id> build, /api/videos feed + search + page 2, yt-dlp on video.url or formats[0].url, python3 check.py <id>).

Status: [ ] open, [~] in progress, [x] fixed and verified, [-] not reproducible / not a provider bug (reason noted).

1. Channels returning no items

Channel Symptom Notes
[x] javtiful feed times out (150s) then 0 items; search 0 site redesign (article.video-card); rewrote parser, page URL as video.url (yt-dlp-native), no per-item detail fetches. 23 items, p2/search OK
[x] sextb 0 items on feed, search and page 2 Jina mirror now 403/429; site is plain HTML reachable with the emulating client. Rewrote listing/detail/option parsers for HTML, removed mirror. Remaining check.py errors are tooling items (proxy URL policy; imggle.net thumbs are TLS-gated for requests)
[-] camsoda 0 items on every request not a provider bug here: www.camsoda.com is DNS-sinkholed by this host's resolver (0.0.0.0; public resolvers return 74.122.168.1), local FlareSolverr gets ERR_CONNECTION_REFUSED, Jina is 403/429, and Cloudflare serves a managed challenge to non-browser clients. Needs the production network to verify.
[-] hentaimama 0 items or 85-150s timeouts not fixable in code from this host: site now serves a Cloudflare managed challenge (cf-mitigated: challenge) on every path; browser-TLS impersonation gets 403 and the local FlareSolverr (:8191, process outside this sandbox) hangs even on a non-CF page (javtiful, 60s). Latency is the requester waiting on FlareSolverr. Needs a healthy FlareSolverr to verify.
[-] hotbunny 0 items; first request 78s same cause: hotbunny.ai (including /api/post/*) now answers a Cloudflare managed challenge; previously the JSON API was reachable directly. Needs a healthy FlareSolverr.

2. Playback: page URL not resolvable by yt-dlp / formats not playable

Channel Symptom Notes
[x] thaiporntv page URL "Unsupported URL", no formats player attribute renamed data-enc -> data-video-sources (reversed base64 JSON {quality,url}); decoder handles both. 3 formats/item, yt-dlp OK, check.py passes
[x] pornmz page URL "Unsupported URL", no formats site changed itemprop="contentURL" -> contentUrl; selector now case-insensitive. Enrichment concurrency 10 + 25s per-item timeout. Origin is slow (15s+/detail) so first feed 25-50s. check.py passes
[x] coomer page URL "Unsupported URL"; media host CF-gated here image-only posts were listed as videos with no playable URL: now dropped, and a video in attachments is picked up. Also fixed: page= is ignored by the API (every page 2+ was a copy of page 1; now o= offset), search results shared one cache key, and compressed API bodies were not decoded (enabled wreq gzip/br/deflate/zstd). n*.coomer.st CDN connect-times-out from this host, and the API 429s this IP after bursts, so playback/"girl" search could not be verified end to end
[x] tube8 page URL 404; /proxy/tube8/{id} 502 sponsored channel cards carry a stale data-video-id (404) while the card link is the live page; id/URL now come from the link. Signed pix-cdn77 thumbs 403 without a Referer: new host-restricted /proxy/aylo-thumb (also fixes ph-shorties). check.py passes
[x] hypnotube page URLs 404 on all sampled items the 404 was a check.py artifact (server answers HEAD 404, GET 200); check.py now confirms with a ranged GET. Real bug found on the way: the site returns ONE results page per searchgate POST, so any 2nd search on the shared session (and every page 2) came back empty. Each (query, page) now uses its own isolated session. 3 consecutive searches + page 2 verified
[x] fullpornerorg lulustream proxy 403 / 502 three separate causes: (1) hoster files deleted ("File is no longer available") but still listed -> now probed and dropped; (2) myvidplay.com embeds lost their host (bare ID sent to lulustream, wrong namespace) -> routed to the doodstream proxy with host kept, which now allows myvidplay/playmogo; (3) tnmr.org m3u8 is TLS-fingerprint gated (works with chrome TLS = the channel's --impersonate chrome-120; plain yt-dlp 403s are the checker's expected CF warnings). myvidplay playback could not be verified: the embed now returns a Turnstile captcha shell to this client
[-] ripnsfw doodstream format 502 lulustream format on the same items resolves (302). The doodstream format cannot be verified here: *.dood.video resolves to 127.0.0.1 on this host's resolver, so the CDN is unreachable. Needs the production network
[-] freepornvideosxxx yt-dlp 502 on page URL listing/search/p2 fine (24 items). Page URL is yt-dlp-native (html5 <source> 480/720/2160). The 502 is the site's edge answering plain yt-dlp's TLS fingerprint (3 of 4 items 502; 4/4 fine with --impersonate chrome-120; curl always 200). The channel declares that command but see "Open decision" below: status never delivers it
[-] hdporn92 m3u8 formats time out (curl + yt-dlp) provider works (embed deobfuscated, signed m3u8 returned, yt-dlp resolves the first item). Every morencius.com request (embed page AND playlist) carries a constant ~20.1s server-side delay from this egress (detail page on hdporn92.com is 0.25s), which exceeds check.py's 15s HEAD / 20s yt-dlp socket timeouts. Raising enrichment concurrency 4 -> 10 made it worse (108s, then timeouts), so it was reverted. Likely a host tarpit for this network; verify from the production IP

3. Thumbnails

Channel Symptom Notes
[x] xxdbx malformed thumbs https:/7004….jpg assets are now site-relative (/id.jpg) rather than //host/..; new absolute_url resolves all three forms. 36 items, p2/search OK, check.py passes
[x] ph-shorties thumbs return 403 same Referer-gated pix-cdn77.phncdn.com thumbs; routed through aylo-thumb proxy. check.py passes
[x] noodlemagazine thumbs unreachable (-1 / 400) in check.py thumb proxy dropped the preview's query string (?id&tkn..), rejected extensionless getVideoPreview paths, and warmed up via noodlemagazine.com through the fallback chain on every image (10-28s). All 24 thumbs now 200 in ~0.08s. Titles fix from earlier commit stands
Channel Symptom Notes
[ ] fapello "girl" and "milf" both 0
[ ] chaturbate "girl" and "milf" both 0
[x] cumlouder unknown tag takes 146s before returning 0 a real 404 went through the requester's Jina/FlareSolverr fallback chain; now the raw response is read, 404 returns empty immediately (0s), 5xx/429 still retried. check.py passes
[ ] vjav "girl" 0, "milf" 10 may be term-specific
[ ] xxxtik "girl" 0 (not re-run serially)
[x] hypnotube "milf" 0, "girl" 20 same root cause as above (second search on a session returned nothing), not term-specific

5. Slow channels

Channel Symptom Notes
[ ] aps 30-98s per request; check.py 30s timeout
[-] hdporn92 feed about 66s per request consequence of the 20s per-embed host delay above (10 embeds per page); not fixable in the provider without moving to play-time resolution, which would put the 20s on every play and every HLS playlist request
[ ] porndish 14-23s feed/search

6. Tooling

Item Notes
[x] check.py "url points to hottub server" contradicted the hottub skill

Open decision (needs the owner)

ytdlpCommand set on providers (Channel.ytdlpCommand, ~90 channels, e.g. yt-dlp --impersonate chrome-120) is never sent to clients: decorate_channel in src/providers/mod.rs builds ChannelView.ytdlpCommand from a hardcoded match (only pimpbunny), and decorates_channel_with_group_and_tags pins hsex to None. /api/status therefore reports null for all of them. Either deliver channel.ytdlpCommand (clients then need yt-dlp impersonation support) or drop the dead declarations. Channels that only fail under plain yt-dlp TLS (freepornvideosxxx, missav, ...) depend on this. check.py now applies whatever /api/status exposes.

Found while fixing (pre-existing, unrelated)

  • src/proxy.rs: the pornhub-thumb resource was registered as /proxy/pornhub-thumb inside the /proxy scope (unreachable); fixed alongside aylo-thumb. No provider uses it today.
  • Failing unit tests, untouched files: proxies::hanimecdn::tests::prefixes_relative_paths_with_hanime_cdn_host, proxies::vidara::tests::normalizes_detail_request_with_path_only.

Environmental / needs the production IP (not worked here)

porndish, supjav, jable, fapello, missav, wowxxx, hentaihaven return Cloudflare 403 to yt-dlp/curl from this machine while the server itself returns items. hsex returned 429 (rate limit).