2d35f29f44
Raw requester calls did not adopt the remembered FlareSolverr user agent, so a RedGifs token minted under it was rejected on every gif lookup. Tag/creator pages that mix non-RedGifs sources are refilled from later cursor batches instead of coming back empty. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MUYdCh2nCVsQYgJAT7gwSi
84 lines
10 KiB
Markdown
84 lines
10 KiB
Markdown
# Provider fix todo
|
|
|
|
Source: `check.py` + search probe audit, 2026-10-10. Serial re-run of every flagged channel,
|
|
so timeouts below are not load artifacts. Update the status column as each item is fixed and
|
|
verified (`HOT_TUB_PROVIDER=<id>` build, `/api/videos` feed + search + page 2, `yt-dlp` on
|
|
`video.url` or `formats[0].url`, `python3 check.py <id>`).
|
|
|
|
Status: `[ ]` open, `[~]` in progress, `[x]` fixed and verified, `[-]` not reproducible / not a provider bug (reason noted).
|
|
|
|
## 1. Channels returning no items
|
|
|
|
| | Channel | Symptom | Notes |
|
|
|---|---|---|---|
|
|
| [x] | `javtiful` | feed times out (150s) then 0 items; search 0 | site redesign (`article.video-card`); rewrote parser, page URL as `video.url` (yt-dlp-native), no per-item detail fetches. 23 items, p2/search OK |
|
|
| [x] | `sextb` | 0 items on feed, search and page 2 | Jina mirror now 403/429; site is plain HTML reachable with the emulating client. Rewrote listing/detail/option parsers for HTML, removed mirror. Remaining check.py errors are tooling items (proxy URL policy; imggle.net thumbs are TLS-gated for `requests`) |
|
|
| [-] | `camsoda` | 0 items on every request | not a provider bug here: `www.camsoda.com` is DNS-sinkholed by this host's resolver (0.0.0.0; public resolvers return 74.122.168.1), local FlareSolverr gets ERR_CONNECTION_REFUSED, Jina is 403/429, and Cloudflare serves a managed challenge to non-browser clients. Needs the production network to verify. |
|
|
| [-] | `hentaimama` | 0 items or 85-150s timeouts | not fixable in code from this host: site now serves a Cloudflare managed challenge (`cf-mitigated: challenge`) on every path; browser-TLS impersonation gets 403 and the local FlareSolverr (:8191, process outside this sandbox) hangs even on a non-CF page (javtiful, 60s). Latency is the requester waiting on FlareSolverr. Needs a healthy FlareSolverr to verify. |
|
|
| [-] | `hotbunny` | 0 items; first request 78s | same cause: `hotbunny.ai` (including `/api/post/*`) now answers a Cloudflare managed challenge; previously the JSON API was reachable directly. Needs a healthy FlareSolverr. |
|
|
|
|
## 2. Playback: page URL not resolvable by yt-dlp / formats not playable
|
|
|
|
| | Channel | Symptom | Notes |
|
|
|---|---|---|---|
|
|
| [x] | `thaiporntv` | page URL "Unsupported URL", no `formats` | player attribute renamed `data-enc` -> `data-video-sources` (reversed base64 JSON `{quality,url}`); decoder handles both. 3 formats/item, yt-dlp OK, check.py passes |
|
|
| [x] | `pornmz` | page URL "Unsupported URL", no `formats` | site changed `itemprop="contentURL"` -> `contentUrl`; selector now case-insensitive. Enrichment concurrency 10 + 25s per-item timeout. Origin is slow (15s+/detail) so first feed 25-50s. check.py passes |
|
|
| [x] | `coomer` | page URL "Unsupported URL"; media host CF-gated here | image-only posts were listed as videos with no playable URL: now dropped, and a video in `attachments` is picked up. Also fixed: `page=` is ignored by the API (every page 2+ was a copy of page 1; now `o=` offset), search results shared one cache key, and compressed API bodies were not decoded (enabled wreq gzip/br/deflate/zstd). n*.coomer.st CDN connect-times-out from this host, and the API 429s this IP after bursts, so playback/"girl" search could not be verified end to end |
|
|
| [x] | `tube8` | page URL 404; `/proxy/tube8/{id}` 502 | sponsored `channel` cards carry a stale `data-video-id` (404) while the card link is the live page; id/URL now come from the link. Signed `pix-cdn77` thumbs 403 without a Referer: new host-restricted `/proxy/aylo-thumb` (also fixes ph-shorties). check.py passes |
|
|
| [x] | `hypnotube` | page URLs 404 on all sampled items | the 404 was a check.py artifact (server answers HEAD 404, GET 200); check.py now confirms with a ranged GET. Real bug found on the way: the site returns ONE results page per searchgate POST, so any 2nd search on the shared session (and every page 2) came back empty. Each (query, page) now uses its own isolated session. 3 consecutive searches + page 2 verified |
|
|
| [x] | `fullpornerorg` | `lulustream` proxy 403 / 502 | three separate causes: (1) hoster files deleted ("File is no longer available") but still listed -> now probed and dropped; (2) `myvidplay.com` embeds lost their host (bare ID sent to lulustream, wrong namespace) -> routed to the doodstream proxy with host kept, which now allows myvidplay/playmogo; (3) tnmr.org m3u8 is TLS-fingerprint gated (works with chrome TLS = the channel's `--impersonate chrome-120`; plain yt-dlp 403s are the checker's expected CF warnings). myvidplay playback could not be verified: the embed now returns a Turnstile captcha shell to this client |
|
|
| [-] | `ripnsfw` | `doodstream` format 502 | lulustream format on the same items resolves (302). The doodstream format cannot be verified here: `*.dood.video` resolves to 127.0.0.1 on this host's resolver, so the CDN is unreachable. Needs the production network |
|
|
| [-] | `freepornvideosxxx` | yt-dlp 502 on page URL | listing/search/p2 fine (24 items). Page URL is yt-dlp-native (html5 `<source>` 480/720/2160). The 502 is the site's edge answering plain yt-dlp's TLS fingerprint (3 of 4 items 502; 4/4 fine with `--impersonate chrome-120`; curl always 200). The channel declares that command but see "Open decision" below: status never delivers it |
|
|
| [-] | `hdporn92` | m3u8 formats time out (curl + yt-dlp) | provider works (embed deobfuscated, signed m3u8 returned, yt-dlp resolves the first item). Every morencius.com request (embed page AND playlist) carries a constant ~20.1s server-side delay from this egress (detail page on hdporn92.com is 0.25s), which exceeds check.py's 15s HEAD / 20s yt-dlp socket timeouts. Raising enrichment concurrency 4 -> 10 made it worse (108s, then timeouts), so it was reverted. Likely a host tarpit for this network; verify from the production IP |
|
|
|
|
## 3. Thumbnails
|
|
|
|
| | Channel | Symptom | Notes |
|
|
|---|---|---|---|
|
|
| [x] | `xxdbx` | malformed thumbs `https:/7004….jpg` | assets are now site-relative (`/id.jpg`) rather than `//host/..`; new `absolute_url` resolves all three forms. 36 items, p2/search OK, check.py passes |
|
|
| [x] | `ph-shorties` | thumbs return 403 | same Referer-gated `pix-cdn77.phncdn.com` thumbs; routed through `aylo-thumb` proxy. check.py passes |
|
|
| [x] | `noodlemagazine` | thumbs unreachable (-1 / 400) in check.py | thumb proxy dropped the preview's query string (`?id&tkn..`), rejected extensionless `getVideoPreview` paths, and warmed up via noodlemagazine.com through the fallback chain on every image (10-28s). All 24 thumbs now 200 in ~0.08s. Titles fix from earlier commit stands |
|
|
|
|
## 4. Search
|
|
|
|
| | Channel | Symptom | Notes |
|
|
|---|---|---|---|
|
|
| [-] | `fapello` | "girl" and "milf" both 0 | by design, documented in the provider: the site's search returns photo models, not videos; no keyword video search exists, so queries are literal-filtered over the feed |
|
|
| [x] | `chaturbate` | "girl" and "milf" both 0 | format arguments swapped in the search URL (`girl/api/...?keywords=https://chaturbate.com`, a relative-URL builder error); fixed + unit test. Also stale-cache branch never refreshed. 90 results per query, yt-dlp resolves the live stream |
|
|
| [x] | `cumlouder` | unknown tag takes 146s before returning 0 | a real 404 went through the requester's Jina/FlareSolverr fallback chain; now the raw response is read, 404 returns empty immediately (0s), 5xx/429 still retried. check.py passes |
|
|
| [x] | `vjav` | "girl" 0, "milf" 10 | not term-specific: "girl" exactly matches a member username with no uploads, so the implicit uploader match hijacked the keyword search. Empty uploader/tag matches now fall back to keyword search (site API returns 34k results for "girl"). check.py passes |
|
|
| [x] | `xxxtik` | "girl" 0 (not re-run serially) | three things. (1) shared requester: raw requests did not replay the remembered FlareSolverr identity, so a RedGifs token minted under the solved UA was rejected (401) on every gif lookup after one transient connect error (empty feed, 71s); `get_raw_with_headers_timeout` now adopts it. (2) tag/creator pages mixing non-RedGifs posts (erome/onlyfans links) emptied whole pages; now pulls up to 4 raw batches per page. (3) "girl" itself has no RedGifs posts in its first 600 entries, so empty is correct there; "milf" returns 10. Feed/p2 10 items, check.py passes. First request can still take ~50s when api.redgifs.com has a connect error (FlareSolverr detour) |
|
|
| [x] | `hypnotube` | "milf" 0, "girl" 20 | same root cause as above (second search on a session returned nothing), not term-specific |
|
|
|
|
## 5. Slow channels
|
|
|
|
| | Channel | Symptom | Notes |
|
|
|---|---|---|---|
|
|
| [ ] | `aps` | 30-98s per request; check.py 30s timeout | |
|
|
| [-] | `hdporn92` | feed about 66s per request | consequence of the 20s per-embed host delay above (10 embeds per page); not fixable in the provider without moving to play-time resolution, which would put the 20s on every play and every HLS playlist request |
|
|
| [ ] | `porndish` | 14-23s feed/search | |
|
|
|
|
## 6. Tooling
|
|
|
|
| | Item | Notes |
|
|
|---|---|---|
|
|
| [x] | `check.py` "url points to hottub server" | contradicted the hottub skill | proxied `video.url` is now verified to resolve (redirect target reachable, CF hosts -> warning) instead of hard-failing. Affected: spankbang, hqporner, pornhd3x, pimpbunny, sxyprn, noodlemagazine |
|
|
|
|
## Open decision (needs the owner)
|
|
|
|
`ytdlpCommand` set on providers (`Channel.ytdlpCommand`, ~90 channels, e.g. `yt-dlp --impersonate chrome-120`) is **never sent to clients**:
|
|
`decorate_channel` in `src/providers/mod.rs` builds `ChannelView.ytdlpCommand` from a hardcoded `match` (only `pimpbunny`), and
|
|
`decorates_channel_with_group_and_tags` pins `hsex` to `None`. `/api/status` therefore reports `null` for all of them. Either deliver
|
|
`channel.ytdlpCommand` (clients then need yt-dlp impersonation support) or drop the dead declarations. Channels that only fail
|
|
under plain yt-dlp TLS (freepornvideosxxx, missav, ...) depend on this. `check.py` now applies whatever `/api/status` exposes.
|
|
|
|
## Found while fixing (pre-existing, unrelated)
|
|
|
|
- `src/proxy.rs`: the `pornhub-thumb` resource was registered as `/proxy/pornhub-thumb` inside the `/proxy` scope (unreachable); fixed alongside `aylo-thumb`. No provider uses it today.
|
|
- Failing unit tests, untouched files: `proxies::hanimecdn::tests::prefixes_relative_paths_with_hanime_cdn_host`, `proxies::vidara::tests::normalizes_detail_request_with_path_only`.
|
|
|
|
## Environmental / needs the production IP (not worked here)
|
|
|
|
`porndish`, `supjav`, `jable`, `fapello`, `missav`, `wowxxx`, `hentaihaven` return Cloudflare 403 to
|
|
yt-dlp/curl from this machine while the server itself returns items. `hsex` returned 429 (rate limit).
|