rule34video: artist, category and tag archives; resolve bare queries by probing; credit artists as uploader
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MUYdCh2nCVsQYgJAT7gwSi
This commit is contained in:
@@ -59,7 +59,7 @@ This is the current implementation inventory as of this snapshot of the repo. Us
|
||||
| `redtube` | `mainstream-tube` | no | no | Mainstream archive. |
|
||||
| `ripnsfw` | `onlyfans` | no | yes | Leaked-creator aggregator for ripnsfw.com — the entire catalogue (~2.5k rows) is a single client-side-rendered page whose data source is a published Google Sheet exported as CSV (`GET /data.csv`, no auth/CF gate); the provider fetches and parses that CSV directly instead of scraping any HTML, replicating the site's own JS `parseCSV`/`csvSplit` column layout (`THUMB,NEW,NOME,DOODSTREAM,LULUSTREAM,<link3>,DOOD,<guests>,SITE(bunkr links),TITLE,ID,#(tags),POST,,DATE,SIZE`). Feed/search/pagination/sort are all done in-memory over the fetched rows (no native site pagination exists — `perPage`-sized pages are sliced locally), with a 180s in-process cache to avoid re-fetching the CSV on every request. `tag:`/`category:`/`cat:` and `model:`/`uploader:` query prefixes filter by the parsed tag list / model name; bare keywords substring-match model name, guests, and tags. `video.url` is the site's client-rendered `https://ripnsfw.com/model/{slug}/{id}` route — the host has no server-side SPA fallback, so this 404s to any plain HTTP client (its custom 404 page stashes the path and JS-redirects to `/`, which restores the route — works in a real browser/webview only); `check.py`'s `_CF_PROTECTED_HOSTS` allowlist includes `ripnsfw.com` for this reason. Media is never hosted on ripnsfw.com itself: each row carries a Doodstream and/or Lulustream embed link, resolved into `formats[]` (Lulustream first, matching the site's own default-player order) via the **existing, previously-unused** `src/proxies/doodstream.rs` / `src/proxies/lulustream.rs` redirect proxies (`build_proxy_url(&options, "doodstream"/"lulustream", &strip_url_scheme(link))`) — no new proxy code was written. Both proxies resolve correctly to real signed CDN URLs (verified against live data), but neither CDN's final edge is reachable from this repo's dev sandbox: Lulustream's `*.tnmr.org` edge 403s datacenter/proxy IPs regardless of UA or TLS impersonation, and Doodstream's final `*.dood.video` edge resolves to a loopback/refused address from non-residential egress — both are IP-reputation-style CDN blocks, not code defects (same class of sandbox limitation already documented for `camsoda`/`hentaihaven`/`animeidhentai`), so `check.py` allowlists both by suffix (`_CF_PROTECTED_SUFFIXES`). No thumbnail proxy needed (`cdn.ripnsfw.com/thumbs/...` is directly hotlinkable, no CF/referer gate). No `/api/uploaders` (model pages exist but have no dedicated profile/stats schema confirmed). |
|
||||
| `rule34gen` | `ai` | no | no | AI group example. |
|
||||
| `rule34video` | `hentai-animation` | no | no | Hentai group example. |
|
||||
| `rule34video` | `hentai-animation` | no | no | Hentai group example. **Archives:** artists are `/models/{slug}/`, franchises/genres `/categories/{slug}/`, characters and everything else `/tags/{slug}/`; each is paged and sorted through `?mode=async&function=get_block&block_id=custom_list_videos_common_videos&sort_by={sort}&from={page}` (unknown slugs 404). Prefix queries `artist:`/`model:`/`creator:`, `category:`/`franchise:`/`series:`, `tag:`/`character:`; a bare query is probed as artist, then category, then tag, then resolved through `/search_ajax.php?tag=` (characters are tagged `name (franchise)`; an unambiguous exact base-name match wins), else keyword search. Cards are enriched from the detail page, which also yields the credited artist as `uploader` (`rule34video:{slug}`) and artist names as tags. |
|
||||
| `sextb` | `jav` | no | no | JAV family provider. sextb.net is Cloudflare-gated, so listing/pagination markdown is fetched through a hand-built Jina mirror URL (`mirror_url()`); the target is already an absolute `https://sextb.net/...` URL, so the scheme must be stripped before prepending the `https://r.jina.ai/http://` prefix — concatenating them directly (the pre-fix bug) produces a malformed `.../http://https://...` URL that Jina rejects outright. |
|
||||
| `shooshtime` | `onlyfans` | no | yes | Redirect proxy plus dedicated media route. Listing cards are `div.card` (card→`a.title` for the link/title, `a.image img` for thumb/preview, `div.duration`, `div.list div.box` for views+date, `div.name a.text` for uploader) — the site redesigned from an older `div.item.thumb`/`span.description` layout, so re-verify these selectors against `{BASE_URL}/videos/` (the real default-sort URL, not the bare homepage) if the feed ever goes empty again. |
|
||||
| `spankbang` | `mainstream-tube` | no | yes | Best template for redirect proxy plus anti-bot fetches. |
|
||||
|
||||
Reference in New Issue
Block a user