hooporn: probe queries as pornstar/studio archives before keyword search
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MUYdCh2nCVsQYgJAT7gwSi
This commit is contained in:
@@ -93,7 +93,7 @@ This is the current implementation inventory as of this snapshot of the repo. Us
|
||||
| `xlecx` | `hentai-animation` | no | no | DLE (DataLife Engine)-style HTML scraper for xlecx.one, following the `thaiporntv` template. Latest feed `/f/cat={id}/sort={mode}/order=desc/` (page N≥2 appends `page/{N}/`); `sort` options new/popular/commented/rated map to DLE's `date`/`news_read`/`comm_num`/`rating` sort params; `category` option is the 2 hardcoded top-level archives (`videos`, `hentai-video`). Search via `/index.php?do=search&subaction=search&story={query}` (page N adds `search_start={N}`). Tag shortcut (`#slug` query) routes to `/tags/{slug}/` (paginated via `/tags/{slug}/page/{N}/`); the ~1,044-entry tag catalog is background-loaded once at startup from `/tags/` (`.tags-list1 a[href*='/tags/']`, absolute-URL anchors) into an `Arc<RwLock<Vec<FilterOption>>>` and exposed as the `filter`/"Tags" `ChannelOption` — note this option (and any `categories`) is deliberately stripped from the live `/api/status` response by `should_hide_status_option`/`decorate_channel` in `providers/mod.rs` (any `id=="filter"` option titled "Tags" is hidden site-wide), so its absence there is expected, not a bug; it is still honored by `/api/videos`. Cards parsed from `h3.thumb__title` + `img` inside standard DLE thumb blocks; id extracted from the `/{id}-{slug}.html` URL suffix. Thumbnails are direct `xlecx.one/uploads/posts/...` webp (no proxy/Referer needed). `video.url` is the detail page URL, which yt-dlp resolves natively via the generic HTML5 embed extractor; no `formats`/proxy needed. No `/api/uploaders` (no stable uploader identity on the site). |
|
||||
| `xxxtik` | `tiktok` | yes | no | JSON-API short-form aggregator for xxxtik.com — every post is a moderated repost of a RedGifs clip, so the real media backend is the public **RedGifs v2 API**, not xxxtik itself (new pattern; no other provider currently resolves through a third-party media API). Listing API (`xxxtik-api-iw98m.ondigitalocean.app`, found by grepping the Angular `main-es2015.js` bundle): `GET /post/new`, `/post/top/{week,month,year,all}`, `/post/tag/{name}`, `/post/creator/{username}` — all **cursor**-paginated (`?cursor={lastItemId}&limit=N`; the `cursor` is the numeric `id` of the last item from the previous batch, not an offset/count — reaching page N walks N sequential requests, mirroring `fikfap`'s `fetch_cursor_page`). `GET /search?query=Q` returns tag/profile autocomplete suggestions only (no posts), so free-text search is routed through it to resolve a `Tag`/`Creator` target before a second listing call; `tag:`/`category:`/`cat:` and `user:`/`uploader:`/`creator:` query prefixes skip that lookup. Each post's `source` field is a `redgifs.com/watch/{id}` URL; the provider fetches a short-lived anonymous bearer token from `POST api.redgifs.com/v2/auth/temporary` (cached, refreshed once on a 401) and resolves `GET api.redgifs.com/v2/gifs/{id}` (bounded to 8-way concurrency via `buffer_unordered`) for the real `media.redgifs.com/*.mp4` + poster, both fetchable with zero auth/Referer. `video.url` is the xxxtik page (`https://xxxtik.com/feed/{uuid}`, not yt-dlp-resolvable — Angular SPA, generic extractor fails), with `formats` populated from the resolved redgifs mp4; tags merge xxxtik's own tags with RedGifs' tags. 20 curated tags exposed via `categories` (xxxtik has ~62k tags total, too many to background-load). `/api/uploaders` works via `GET /user/by-username/{name}` + `/post/creator/{name}`, but xxxtik's "creator" accounts are inconsistent: some (e.g. `dlhoodninja`, `besttits`) return real posts matching their profile `_count.posts`; others with a nonzero `_count.posts` (e.g. `bigboobsgw`, count 472) return an empty `/post/creator/` list — likely synthetic curation accounts (`name@default` emails) rather than real uploaders. The provider degrades gracefully (returns the profile with `videos: []`, no error) rather than guessing which accounts are "real". No proxy needed — all media/thumb URLs are publicly fetchable with no Referer or auth. |
|
||||
| `txxx` | `mainstream-tube` | no | no | JSON-API scraper for txxx.me (mirror of txxx.com; Vue SPA, same KVS-based API family as the hclips/upornia/hdzog network). Listing: `GET /api/json/videos2/86400/str/{sort}/{count}/{section}.{object}.{page}.all...jsond` with sort `latest-updates`/`most-popular`/`top-rated`/`longest`; **the `.json` variant ignores the page number, `jsond` does not**. Category archive = section `categories`, object = category dir slug (numeric ids return `category_not_found`); ~80 top categories hardcoded as the `categories` option (225 exist at `/api/json/categories/86400/str.all.en.jsond`). Search: `GET /api/videos2.php?params=86400/str/relevance/{count}/search..{page}.all..&s={query}` (relevance only; sort is ignored; page works although `params.page` echoes 1). `count` follows `perPage` (max 60). The page URL `/videos/{id}/{dir}/` is **not** yt-dlp-resolvable, so every card is enriched (bounded `buffered(8)`) via `GET /api/videofile.php?video_id={id}&lifetime=8640000`, whose `video_url` is base64 of `/get_file/...` with Cyrillic look-alike letters (А Е М С …) substituted and `~` for `=`; the decoded `https://txxx.me/get_file/...` 302s to a per-client signed `*.ahcdn.com` URL (IP-pinned, so it must NOT be resolved server-side; no Referer needed). Items without formats are dropped. Heights come from the listing `file_formats` (`||_hq.mp4|1278x720|…||_tr.mp4|…`; `_tr.mp4` is the trailer and becomes `preview` via `pv`). Rating 0..5 → ×20; `likes`/`dislikes` ignored; category titles become tags; uploader = content source else member `display_name` (`txxx:{user_id}`; no profile endpoint). Thumbs `tn.txxx.tube` direct. Queries matching a category name route to its archive; `cat:`/`category:` prefix. No models/channels archives (sections `models`/`channels` are rejected by the API). No proxy needed. |
|
||||
| `hooporn` | `studio-network` | no | no | HTML scraper for hooporn.com (studio/pornstar tube; Cloudflare-fronted but serves plain 200 to the shared requester). Cards `div.video-card` with lazy thumb in `img[data-src]` (`//imgs.hooporn.com/thumb/...jpg?v2`, direct, no Referer), title in `div.video-title a`, `.time` duration, `span.create` unix timestamp → `uploadedAt`; no views/rating. Feed `/home/{N}`, category `/category/{slug}/{N}` (63 hardcoded from `/categories`), pornstar `/pornstar/{slug}/{N}`, studio `/studio/{slug}/{N}` (studio slugs are concatenated, e.g. `familystrokes`), search `/search?q=…&p={N}` (24 per page; `?search=`/`/search/q/N` do not work). Prefix queries `cat:`/`category:`, `pornstar:`/`star:`/`model:`, `studio:`/`site:`; a bare keyword matching a category name routes to the archive, anything else is a search. The watch page `/watch/{24hex}` is **not** yt-dlp-resolvable (player is an iframe), and the mp4 is signed (`videos.trendyporn.com/...mp4?md5=…&expires=…`, ~1 day, 403 without the signature, no Referer/IP binding), so every card is enriched (`buffered(8)`): watch page → `iframe[src*="/video/"]` (`//vid.hooporn.com/video/{n}`) → `var video = "…"` into one `formats[]` entry (quality unknown, labelled `auto`); the watch page also yields category tags, pornstar names (as tags) and the studio (uploader `hooporn:{studio-slug}`). Items whose enrichment fails are dropped. No proxy needed. |
|
||||
| `hooporn` | `studio-network` | no | no | HTML scraper for hooporn.com (studio/pornstar tube; Cloudflare-fronted but serves plain 200 to the shared requester). Cards `div.video-card` with lazy thumb in `img[data-src]` (`//imgs.hooporn.com/thumb/...jpg?v2`, direct, no Referer), title in `div.video-title a`, `.time` duration, `span.create` unix timestamp → `uploadedAt`; no views/rating. Feed `/home/{N}`, category `/category/{slug}/{N}` (63 hardcoded from `/categories`), pornstar `/pornstar/{slug}/{N}`, studio `/studio/{slug}/{N}` (studio slugs are concatenated, e.g. `familystrokes`), search `/search?q=…&p={N}` (24 per page; `?search=`/`/search/q/N` do not work). Prefix queries `cat:`/`category:`, `pornstar:`/`star:`/`model:`, `studio:`/`site:`; a bare keyword matching a category name routes to the archive; any other query is first probed on the site (`/pornstar/{slug}/1`, then `/studio/{slug-without-dashes}/1`; unknown archives are an instant 404 read through the raw response, Cloudflare challenges are handed to the requester/FlareSolverr chain) and used as that archive if it exists, otherwise it is a keyword search. The probe runs on every page, so page N of a pornstar/studio query stays in that archive. The watch page `/watch/{24hex}` is **not** yt-dlp-resolvable (player is an iframe), and the mp4 is signed (`videos.trendyporn.com/...mp4?md5=…&expires=…`, ~1 day, 403 without the signature, no Referer/IP binding), so every card is enriched (`buffered(8)`): watch page → `iframe[src*="/video/"]` (`//vid.hooporn.com/video/{n}`) → `var video = "…"` into one `formats[]` entry (quality unknown, labelled `auto`); the watch page also yields category tags, pornstar names (as tags) and the studio (uploader `hooporn:{studio-slug}`). Items whose enrichment fails are dropped. No proxy needed. |
|
||||
|
||||
## Proxy Routes
|
||||
|
||||
|
||||
Reference in New Issue
Block a user