Add hooporn provider

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MUYdCh2nCVsQYgJAT7gwSi
This commit is contained in:
Simon
2026-10-17 05:11:17 +00:00
parent d72dda5203
commit 99bc80adac
3 changed files with 571 additions and 0 deletions

View File

@@ -366,6 +366,11 @@ const PROVIDERS: &[ProviderDef] = &[
module: "txxx", module: "txxx",
ty: "TxxxProvider", ty: "TxxxProvider",
}, },
ProviderDef {
id: "hooporn",
module: "hooporn",
ty: "HoopornProvider",
},
ProviderDef { ProviderDef {
id: "fullporner", id: "fullporner",
module: "fullporner", module: "fullporner",

View File

@@ -93,6 +93,7 @@ This is the current implementation inventory as of this snapshot of the repo. Us
| `xlecx` | `hentai-animation` | no | no | DLE (DataLife Engine)-style HTML scraper for xlecx.one, following the `thaiporntv` template. Latest feed `/f/cat={id}/sort={mode}/order=desc/` (page N≥2 appends `page/{N}/`); `sort` options new/popular/commented/rated map to DLE's `date`/`news_read`/`comm_num`/`rating` sort params; `category` option is the 2 hardcoded top-level archives (`videos`, `hentai-video`). Search via `/index.php?do=search&subaction=search&story={query}` (page N adds `search_start={N}`). Tag shortcut (`#slug` query) routes to `/tags/{slug}/` (paginated via `/tags/{slug}/page/{N}/`); the ~1,044-entry tag catalog is background-loaded once at startup from `/tags/` (`.tags-list1 a[href*='/tags/']`, absolute-URL anchors) into an `Arc<RwLock<Vec<FilterOption>>>` and exposed as the `filter`/"Tags" `ChannelOption` — note this option (and any `categories`) is deliberately stripped from the live `/api/status` response by `should_hide_status_option`/`decorate_channel` in `providers/mod.rs` (any `id=="filter"` option titled "Tags" is hidden site-wide), so its absence there is expected, not a bug; it is still honored by `/api/videos`. Cards parsed from `h3.thumb__title` + `img` inside standard DLE thumb blocks; id extracted from the `/{id}-{slug}.html` URL suffix. Thumbnails are direct `xlecx.one/uploads/posts/...` webp (no proxy/Referer needed). `video.url` is the detail page URL, which yt-dlp resolves natively via the generic HTML5 embed extractor; no `formats`/proxy needed. No `/api/uploaders` (no stable uploader identity on the site). | | `xlecx` | `hentai-animation` | no | no | DLE (DataLife Engine)-style HTML scraper for xlecx.one, following the `thaiporntv` template. Latest feed `/f/cat={id}/sort={mode}/order=desc/` (page N≥2 appends `page/{N}/`); `sort` options new/popular/commented/rated map to DLE's `date`/`news_read`/`comm_num`/`rating` sort params; `category` option is the 2 hardcoded top-level archives (`videos`, `hentai-video`). Search via `/index.php?do=search&subaction=search&story={query}` (page N adds `search_start={N}`). Tag shortcut (`#slug` query) routes to `/tags/{slug}/` (paginated via `/tags/{slug}/page/{N}/`); the ~1,044-entry tag catalog is background-loaded once at startup from `/tags/` (`.tags-list1 a[href*='/tags/']`, absolute-URL anchors) into an `Arc<RwLock<Vec<FilterOption>>>` and exposed as the `filter`/"Tags" `ChannelOption` — note this option (and any `categories`) is deliberately stripped from the live `/api/status` response by `should_hide_status_option`/`decorate_channel` in `providers/mod.rs` (any `id=="filter"` option titled "Tags" is hidden site-wide), so its absence there is expected, not a bug; it is still honored by `/api/videos`. Cards parsed from `h3.thumb__title` + `img` inside standard DLE thumb blocks; id extracted from the `/{id}-{slug}.html` URL suffix. Thumbnails are direct `xlecx.one/uploads/posts/...` webp (no proxy/Referer needed). `video.url` is the detail page URL, which yt-dlp resolves natively via the generic HTML5 embed extractor; no `formats`/proxy needed. No `/api/uploaders` (no stable uploader identity on the site). |
| `xxxtik` | `tiktok` | yes | no | JSON-API short-form aggregator for xxxtik.com — every post is a moderated repost of a RedGifs clip, so the real media backend is the public **RedGifs v2 API**, not xxxtik itself (new pattern; no other provider currently resolves through a third-party media API). Listing API (`xxxtik-api-iw98m.ondigitalocean.app`, found by grepping the Angular `main-es2015.js` bundle): `GET /post/new`, `/post/top/{week,month,year,all}`, `/post/tag/{name}`, `/post/creator/{username}` — all **cursor**-paginated (`?cursor={lastItemId}&limit=N`; the `cursor` is the numeric `id` of the last item from the previous batch, not an offset/count — reaching page N walks N sequential requests, mirroring `fikfap`'s `fetch_cursor_page`). `GET /search?query=Q` returns tag/profile autocomplete suggestions only (no posts), so free-text search is routed through it to resolve a `Tag`/`Creator` target before a second listing call; `tag:`/`category:`/`cat:` and `user:`/`uploader:`/`creator:` query prefixes skip that lookup. Each post's `source` field is a `redgifs.com/watch/{id}` URL; the provider fetches a short-lived anonymous bearer token from `POST api.redgifs.com/v2/auth/temporary` (cached, refreshed once on a 401) and resolves `GET api.redgifs.com/v2/gifs/{id}` (bounded to 8-way concurrency via `buffer_unordered`) for the real `media.redgifs.com/*.mp4` + poster, both fetchable with zero auth/Referer. `video.url` is the xxxtik page (`https://xxxtik.com/feed/{uuid}`, not yt-dlp-resolvable — Angular SPA, generic extractor fails), with `formats` populated from the resolved redgifs mp4; tags merge xxxtik's own tags with RedGifs' tags. 20 curated tags exposed via `categories` (xxxtik has ~62k tags total, too many to background-load). `/api/uploaders` works via `GET /user/by-username/{name}` + `/post/creator/{name}`, but xxxtik's "creator" accounts are inconsistent: some (e.g. `dlhoodninja`, `besttits`) return real posts matching their profile `_count.posts`; others with a nonzero `_count.posts` (e.g. `bigboobsgw`, count 472) return an empty `/post/creator/` list — likely synthetic curation accounts (`name@default` emails) rather than real uploaders. The provider degrades gracefully (returns the profile with `videos: []`, no error) rather than guessing which accounts are "real". No proxy needed — all media/thumb URLs are publicly fetchable with no Referer or auth. | | `xxxtik` | `tiktok` | yes | no | JSON-API short-form aggregator for xxxtik.com — every post is a moderated repost of a RedGifs clip, so the real media backend is the public **RedGifs v2 API**, not xxxtik itself (new pattern; no other provider currently resolves through a third-party media API). Listing API (`xxxtik-api-iw98m.ondigitalocean.app`, found by grepping the Angular `main-es2015.js` bundle): `GET /post/new`, `/post/top/{week,month,year,all}`, `/post/tag/{name}`, `/post/creator/{username}` — all **cursor**-paginated (`?cursor={lastItemId}&limit=N`; the `cursor` is the numeric `id` of the last item from the previous batch, not an offset/count — reaching page N walks N sequential requests, mirroring `fikfap`'s `fetch_cursor_page`). `GET /search?query=Q` returns tag/profile autocomplete suggestions only (no posts), so free-text search is routed through it to resolve a `Tag`/`Creator` target before a second listing call; `tag:`/`category:`/`cat:` and `user:`/`uploader:`/`creator:` query prefixes skip that lookup. Each post's `source` field is a `redgifs.com/watch/{id}` URL; the provider fetches a short-lived anonymous bearer token from `POST api.redgifs.com/v2/auth/temporary` (cached, refreshed once on a 401) and resolves `GET api.redgifs.com/v2/gifs/{id}` (bounded to 8-way concurrency via `buffer_unordered`) for the real `media.redgifs.com/*.mp4` + poster, both fetchable with zero auth/Referer. `video.url` is the xxxtik page (`https://xxxtik.com/feed/{uuid}`, not yt-dlp-resolvable — Angular SPA, generic extractor fails), with `formats` populated from the resolved redgifs mp4; tags merge xxxtik's own tags with RedGifs' tags. 20 curated tags exposed via `categories` (xxxtik has ~62k tags total, too many to background-load). `/api/uploaders` works via `GET /user/by-username/{name}` + `/post/creator/{name}`, but xxxtik's "creator" accounts are inconsistent: some (e.g. `dlhoodninja`, `besttits`) return real posts matching their profile `_count.posts`; others with a nonzero `_count.posts` (e.g. `bigboobsgw`, count 472) return an empty `/post/creator/` list — likely synthetic curation accounts (`name@default` emails) rather than real uploaders. The provider degrades gracefully (returns the profile with `videos: []`, no error) rather than guessing which accounts are "real". No proxy needed — all media/thumb URLs are publicly fetchable with no Referer or auth. |
| `txxx` | `mainstream-tube` | no | no | JSON-API scraper for txxx.me (mirror of txxx.com; Vue SPA, same KVS-based API family as the hclips/upornia/hdzog network). Listing: `GET /api/json/videos2/86400/str/{sort}/{count}/{section}.{object}.{page}.all...jsond` with sort `latest-updates`/`most-popular`/`top-rated`/`longest`; **the `.json` variant ignores the page number, `jsond` does not**. Category archive = section `categories`, object = category dir slug (numeric ids return `category_not_found`); ~80 top categories hardcoded as the `categories` option (225 exist at `/api/json/categories/86400/str.all.en.jsond`). Search: `GET /api/videos2.php?params=86400/str/relevance/{count}/search..{page}.all..&s={query}` (relevance only; sort is ignored; page works although `params.page` echoes 1). `count` follows `perPage` (max 60). The page URL `/videos/{id}/{dir}/` is **not** yt-dlp-resolvable, so every card is enriched (bounded `buffered(8)`) via `GET /api/videofile.php?video_id={id}&lifetime=8640000`, whose `video_url` is base64 of `/get_file/...` with Cyrillic look-alike letters (А Е М С …) substituted and `~` for `=`; the decoded `https://txxx.me/get_file/...` 302s to a per-client signed `*.ahcdn.com` URL (IP-pinned, so it must NOT be resolved server-side; no Referer needed). Items without formats are dropped. Heights come from the listing `file_formats` (`||_hq.mp4|1278x720|…||_tr.mp4|…`; `_tr.mp4` is the trailer and becomes `preview` via `pv`). Rating 0..5 → ×20; `likes`/`dislikes` ignored; category titles become tags; uploader = content source else member `display_name` (`txxx:{user_id}`; no profile endpoint). Thumbs `tn.txxx.tube` direct. Queries matching a category name route to its archive; `cat:`/`category:` prefix. No models/channels archives (sections `models`/`channels` are rejected by the API). No proxy needed. | | `txxx` | `mainstream-tube` | no | no | JSON-API scraper for txxx.me (mirror of txxx.com; Vue SPA, same KVS-based API family as the hclips/upornia/hdzog network). Listing: `GET /api/json/videos2/86400/str/{sort}/{count}/{section}.{object}.{page}.all...jsond` with sort `latest-updates`/`most-popular`/`top-rated`/`longest`; **the `.json` variant ignores the page number, `jsond` does not**. Category archive = section `categories`, object = category dir slug (numeric ids return `category_not_found`); ~80 top categories hardcoded as the `categories` option (225 exist at `/api/json/categories/86400/str.all.en.jsond`). Search: `GET /api/videos2.php?params=86400/str/relevance/{count}/search..{page}.all..&s={query}` (relevance only; sort is ignored; page works although `params.page` echoes 1). `count` follows `perPage` (max 60). The page URL `/videos/{id}/{dir}/` is **not** yt-dlp-resolvable, so every card is enriched (bounded `buffered(8)`) via `GET /api/videofile.php?video_id={id}&lifetime=8640000`, whose `video_url` is base64 of `/get_file/...` with Cyrillic look-alike letters (А Е М С …) substituted and `~` for `=`; the decoded `https://txxx.me/get_file/...` 302s to a per-client signed `*.ahcdn.com` URL (IP-pinned, so it must NOT be resolved server-side; no Referer needed). Items without formats are dropped. Heights come from the listing `file_formats` (`||_hq.mp4|1278x720|…||_tr.mp4|…`; `_tr.mp4` is the trailer and becomes `preview` via `pv`). Rating 0..5 → ×20; `likes`/`dislikes` ignored; category titles become tags; uploader = content source else member `display_name` (`txxx:{user_id}`; no profile endpoint). Thumbs `tn.txxx.tube` direct. Queries matching a category name route to its archive; `cat:`/`category:` prefix. No models/channels archives (sections `models`/`channels` are rejected by the API). No proxy needed. |
| `hooporn` | `studio-network` | no | no | HTML scraper for hooporn.com (studio/pornstar tube; Cloudflare-fronted but serves plain 200 to the shared requester). Cards `div.video-card` with lazy thumb in `img[data-src]` (`//imgs.hooporn.com/thumb/...jpg?v2`, direct, no Referer), title in `div.video-title a`, `.time` duration, `span.create` unix timestamp → `uploadedAt`; no views/rating. Feed `/home/{N}`, category `/category/{slug}/{N}` (63 hardcoded from `/categories`), pornstar `/pornstar/{slug}/{N}`, studio `/studio/{slug}/{N}` (studio slugs are concatenated, e.g. `familystrokes`), search `/search?q=…&p={N}` (24 per page; `?search=`/`/search/q/N` do not work). Prefix queries `cat:`/`category:`, `pornstar:`/`star:`/`model:`, `studio:`/`site:`; a bare keyword matching a category name routes to the archive, anything else is a search. The watch page `/watch/{24hex}` is **not** yt-dlp-resolvable (player is an iframe), and the mp4 is signed (`videos.trendyporn.com/...mp4?md5=…&expires=…`, ~1 day, 403 without the signature, no Referer/IP binding), so every card is enriched (`buffered(8)`): watch page → `iframe[src*="/video/"]` (`//vid.hooporn.com/video/{n}`) → `var video = "…"` into one `formats[]` entry (quality unknown, labelled `auto`); the watch page also yields category tags, pornstar names (as tags) and the studio (uploader `hooporn:{studio-slug}`). Items whose enrichment fails are dropped. No proxy needed. |
## Proxy Routes ## Proxy Routes

565
src/providers/hooporn.rs Normal file
View File

@@ -0,0 +1,565 @@
use crate::DbPool;
use crate::api::ClientVersion;
use crate::providers::{Provider, report_provider_error, requester_or_default};
use crate::status::*;
use crate::util::cache::VideoCache;
use crate::util::requester::Requester;
use crate::util::time::parse_time_to_seconds;
use crate::videos::{ServerOptions, VideoFormat, VideoItem};
use async_trait::async_trait;
use error_chain::error_chain;
use futures::stream::{self, StreamExt};
use percent_encoding::{NON_ALPHANUMERIC, utf8_percent_encode};
use scraper::{Html, Selector};
pub const CHANNEL_METADATA: crate::providers::ProviderChannelMetadata =
crate::providers::ProviderChannelMetadata {
group_id: "studio-network",
tags: &["studio", "hd", "pornstars", "free"],
};
const BASE_URL: &str = "https://hooporn.com";
const CHANNEL_ID: &str = "hooporn";
const ENRICH_CONCURRENCY: usize = 8;
// Scraped from /categories (`div.channels-card a[href^="/category/"]`).
const CATEGORIES: &[(&str, &str)] = &[
("anal", "anal"),
("ass-licking", "ass licking"),
("atm", "ass to mouth"),
("babe", "babe"),
("bbw", "BBW"),
("big-ass", "big ass"),
("big-tits", "big tits"),
("black", "black"),
("blonde", "blonde"),
("blowjob", "blowjob"),
("bondage", "bondage"),
("brunette", "brunette"),
("busty", "busty"),
("casting", "casting"),
("college", "college"),
("compilation", "compilation"),
("creampie", "creampie"),
("cuckold", "cuckold"),
("czech", "czech"),
("deepthroat", "deepthroat"),
("doggystyle", "doggystyle"),
("dp", "double penetration"),
("ebony", "ebony"),
("fantasy", "fantasy"),
("fetish", "fetish"),
("fingering", "fingering"),
("footjob", "footjob"),
("foursome", "foursome"),
("gangbang", "gangbang"),
("gangbang-creampie", "gangbang creampie"),
("gaping", "gaping"),
("german", "german"),
("gloryhole", "gloryhole"),
("hairy", "hairy"),
("handjob", "handjob"),
("hardcore", "hardcore"),
("homemade", "homemade"),
("interracial", "interracial"),
("latina", "latina"),
("lesbian", "lesbian"),
("lingerie", "lingerie"),
("massage", "massage"),
("masturbation", "masturbation"),
("mature", "mature"),
("milf", "MILF"),
("orgy", "orgy"),
("outdoor", "outdoor"),
("petite", "petite"),
("pov", "POV"),
("public", "public"),
("pussy-licking", "pussy licking"),
("red-head", "red head"),
("riding", "riding"),
("russian", "russian"),
("skinny", "skinny"),
("small-tits", "small tits"),
("solo", "solo"),
("squirt", "squirt"),
("swallow", "swallow"),
("teen", "teen"),
("threesome", "threesome"),
("toys", "toys"),
("wife", "wife"),
];
error_chain! {
foreign_links {
Io(std::io::Error);
}
errors {
Parse(msg: String) {
description("parse error")
display("parse error: {}", msg)
}
}
}
#[derive(Debug, Clone)]
pub struct HoopornProvider;
#[derive(Debug, Clone, PartialEq, Eq)]
enum Target {
Latest,
Category(String),
Pornstar(String),
Studio(String),
Search(String),
}
/// Detail-page data: the player iframe plus taxonomy links.
#[derive(Debug, Default, PartialEq, Eq)]
struct Detail {
iframe: Option<String>,
tags: Vec<String>,
studio: Option<(String, String)>,
}
impl HoopornProvider {
pub fn new() -> Self {
Self
}
fn build_channel(&self, _clientversion: ClientVersion) -> Channel {
let mut category_options = vec![FilterOption {
id: "all".to_string(),
title: "All".to_string(),
}];
category_options.extend(CATEGORIES.iter().map(|(slug, title)| FilterOption {
id: (*slug).to_string(),
title: Self::title_case(title),
}));
Channel {
id: CHANNEL_ID.to_string(),
name: "HooPorn".to_string(),
description: "Free HD studio videos from HooPorn with category, pornstar and studio archives.".to_string(),
premium: false,
favicon: "https://www.google.com/s2/favicons?sz=64&domain=hooporn.com".to_string(),
status: "active".to_string(),
categories: vec![],
options: vec![
ChannelOption {
id: "sort".to_string(),
title: "Sort".to_string(),
description: "Browse HooPorn by newest videos.".to_string(),
systemImage: "arrow.up.arrow.down".to_string(),
colorName: "blue".to_string(),
options: vec![FilterOption {
id: "new".to_string(),
title: "Latest".to_string(),
}],
multiSelect: false,
},
ChannelOption {
id: "categories".to_string(),
title: "Categories".to_string(),
description: "Browse a HooPorn category archive.".to_string(),
systemImage: "square.grid.2x2".to_string(),
colorName: "orange".to_string(),
options: category_options,
multiSelect: false,
},
],
nsfw: true,
cacheDuration: Some(1800),
ytdlpCommand: None,
}
}
fn title_case(value: &str) -> String {
if value.chars().any(|c| c.is_uppercase()) {
return value.to_string();
}
value
.split(' ')
.map(|word| {
let mut chars = word.chars();
match chars.next() {
Some(first) => first.to_uppercase().collect::<String>() + chars.as_str(),
None => String::new(),
}
})
.collect::<Vec<_>>()
.join(" ")
}
fn slugify(value: &str) -> String {
let mut out = String::new();
let mut last_dash = true;
for c in value.trim().chars() {
if c.is_alphanumeric() {
out.extend(c.to_lowercase());
last_dash = false;
} else if !last_dash {
out.push('-');
last_dash = true;
}
}
out.trim_matches('-').to_string()
}
fn category_slug(value: &str) -> Option<&'static str> {
let slug = Self::slugify(value);
if slug.is_empty() {
return None;
}
CATEGORIES
.iter()
.find_map(|(s, title)| (*s == slug || Self::slugify(title) == slug).then_some(*s))
}
fn pick_target(query: Option<&str>, options: &ServerOptions) -> Target {
if let Some(q) = query.map(str::trim).filter(|q| !q.is_empty()) {
let lower = q.to_ascii_lowercase();
for prefix in ["cat:", "category:"] {
if let Some(rest) = lower.strip_prefix(prefix) {
let slug = Self::category_slug(rest)
.map(str::to_string)
.unwrap_or_else(|| Self::slugify(rest));
return Target::Category(slug);
}
}
for prefix in ["pornstar:", "star:", "model:"] {
if let Some(rest) = lower.strip_prefix(prefix) {
return Target::Pornstar(Self::slugify(rest));
}
}
for prefix in ["studio:", "site:"] {
if let Some(rest) = lower.strip_prefix(prefix) {
return Target::Studio(Self::slugify(rest).replace('-', ""));
}
}
if let Some(slug) = Self::category_slug(q) {
return Target::Category(slug.to_string());
}
return Target::Search(q.to_string());
}
if let Some(cat) = options.categories.as_deref() {
let cat = cat.trim();
if !cat.is_empty() && !cat.eq_ignore_ascii_case("all") {
let slug = Self::category_slug(cat)
.map(str::to_string)
.unwrap_or_else(|| Self::slugify(cat));
return Target::Category(slug);
}
}
Target::Latest
}
fn build_listing_url(target: &Target, page: u32) -> String {
match target {
Target::Latest => format!("{BASE_URL}/home/{page}"),
Target::Category(slug) => format!("{BASE_URL}/category/{slug}/{page}"),
Target::Pornstar(slug) => format!("{BASE_URL}/pornstar/{slug}/{page}"),
Target::Studio(slug) => format!("{BASE_URL}/studio/{slug}/{page}"),
Target::Search(query) => format!(
"{BASE_URL}/search?q={}&p={page}",
utf8_percent_encode(query, NON_ALPHANUMERIC)
),
}
}
fn selector(value: &str) -> Result<Selector> {
Selector::parse(value).map_err(|e| Error::from(format!("selector `{value}` failed: {e}")))
}
fn absolute(url: &str) -> String {
if let Some(rest) = url.strip_prefix("//") {
format!("https://{rest}")
} else if url.starts_with('/') {
format!("{BASE_URL}{url}")
} else {
url.to_string()
}
}
fn parse_listing(html: &str) -> Result<Vec<VideoItem>> {
let document = Html::parse_document(html);
let card_sel = Self::selector("div.video-card")?;
let link_sel = Self::selector("div.video-title a[href^=\"/watch/\"]")?;
let img_sel = Self::selector("img")?;
let time_sel = Self::selector(".time")?;
let created_sel = Self::selector("span.create")?;
let mut items = Vec::new();
let mut seen = std::collections::HashSet::new();
for card in document.select(&card_sel) {
let Some(link) = card.select(&link_sel).next() else {
continue;
};
let href = link.value().attr("href").unwrap_or_default();
let id = href.trim_start_matches("/watch/").trim_matches('/').to_string();
if id.is_empty() || !seen.insert(id.clone()) {
continue;
}
let title = link.text().collect::<String>().trim().to_string();
if title.is_empty() {
continue;
}
let thumb = card
.select(&img_sel)
.next()
.and_then(|i| i.value().attr("data-src"))
.map(Self::absolute)
.unwrap_or_default();
let duration = card
.select(&time_sel)
.next()
.map(|t| t.text().collect::<String>())
.and_then(|t| parse_time_to_seconds(t.trim()))
.and_then(|s| u32::try_from(s).ok())
.unwrap_or(0);
let mut item = VideoItem::new(
id.clone(),
title,
format!("{BASE_URL}/watch/{id}"),
CHANNEL_ID.to_string(),
thumb,
duration,
);
item.uploadedAt = card
.select(&created_sel)
.next()
.and_then(|c| c.text().collect::<String>().trim().parse::<u64>().ok());
item.aspectRatio = Some(16.0 / 9.0);
items.push(item);
}
Ok(items)
}
fn parse_detail(html: &str) -> Result<Detail> {
let document = Html::parse_document(html);
let iframe_sel = Self::selector("iframe[src*=\"/video/\"]")?;
let tag_sel = Self::selector("p.tag-link a[href^=\"/category/\"]")?;
let star_sel = Self::selector("a[href^=\"/pornstar/\"]")?;
let studio_sel = Self::selector("a[href^=\"/studio/\"]")?;
let mut detail = Detail::default();
detail.iframe = document
.select(&iframe_sel)
.next()
.and_then(|f| f.value().attr("src"))
.map(Self::absolute);
let mut push = |tag: String| {
let tag = tag.trim().trim_start_matches('#').trim().to_string();
if !tag.is_empty() && !detail.tags.iter().any(|t| t.eq_ignore_ascii_case(&tag)) {
detail.tags.push(tag);
}
};
for a in document.select(&tag_sel) {
push(a.text().collect::<String>());
}
for a in document.select(&star_sel) {
push(a.text().collect::<String>());
}
if let Some(a) = document.select(&studio_sel).next() {
let name = a.text().collect::<String>().trim().to_string();
let slug = a
.value()
.attr("href")
.unwrap_or_default()
.trim_start_matches("/studio/")
.trim_matches('/')
.to_string();
if !name.is_empty() && !slug.is_empty() {
detail.studio = Some((name, slug));
}
}
Ok(detail)
}
/// The player frame carries `var video = "https://videos…mp4?md5=…&expires=…"`.
fn parse_player_video(html: &str) -> Option<String> {
let marker = "var video = \"";
let start = html.find(marker)? + marker.len();
let end = html[start..].find('"')? + start;
let url = html[start..end].replace("&amp;", "&");
url.starts_with("http").then_some(url)
}
async fn enrich(items: Vec<VideoItem>, requester: Requester) -> Vec<VideoItem> {
let referer = format!("{BASE_URL}/");
stream::iter(items.into_iter().map(|mut item| {
let mut requester = requester.clone();
let referer = referer.clone();
async move {
let headers = vec![("Referer".to_string(), referer.clone())];
let page = requester
.get_with_headers(&item.url, headers.clone(), None)
.await
.ok()?;
let detail = Self::parse_detail(&page).ok()?;
let frame = detail.iframe?;
let player = requester
.get_with_headers(&frame, headers, None)
.await
.ok()?;
let video_url = Self::parse_player_video(&player)?;
item.formats = Some(vec![
VideoFormat::new(video_url, "auto".to_string(), "mp4".to_string())
.format_id("mp4".to_string()),
]);
if !detail.tags.is_empty() {
item.tags = Some(detail.tags);
}
if let Some((name, slug)) = detail.studio {
item.uploaderUrl = Some(format!("{BASE_URL}/studio/{slug}"));
item.uploaderId = Some(format!("{CHANNEL_ID}:{slug}"));
item.uploader = Some(name);
}
Some(item)
}
}))
.buffered(ENRICH_CONCURRENCY)
.filter_map(|item| async move { item })
.collect::<Vec<_>>()
.await
}
async fn fetch(target: &Target, page: u32, options: &ServerOptions) -> Result<Vec<VideoItem>> {
let url = Self::build_listing_url(target, page);
let mut requester = requester_or_default(options, CHANNEL_ID, "fetch");
let html = requester
.get_with_headers(
&url,
vec![("Referer".to_string(), format!("{BASE_URL}/"))],
None,
)
.await
.map_err(|e| Error::from(format!("fetch failed for {url}: {e}")))?;
let items = Self::parse_listing(&html)?;
Ok(Self::enrich(items, requester).await)
}
}
#[async_trait]
impl Provider for HoopornProvider {
async fn get_videos(
&self,
_cache: VideoCache,
_pool: DbPool,
_sort: String,
query: Option<String>,
page: String,
_per_page: String,
options: ServerOptions,
) -> Vec<VideoItem> {
let page = page.parse::<u32>().unwrap_or(1).max(1);
let target = Self::pick_target(query.as_deref(), &options);
match Self::fetch(&target, page, &options).await {
Ok(items) => items,
Err(e) => {
report_provider_error(CHANNEL_ID, "get_videos", &e.to_string()).await;
vec![]
}
}
}
fn get_channel(&self, clientversion: ClientVersion) -> Option<Channel> {
Some(self.build_channel(clientversion))
}
}
#[cfg(test)]
mod tests {
use super::*;
fn opts() -> ServerOptions {
ServerOptions {
featured: None,
category: None,
sites: None,
filter: None,
language: None,
public_url_base: None,
requester: None,
network: None,
stars: None,
categories: None,
duration: None,
sort: None,
sexuality: None,
client_version: None,
}
}
#[test]
fn builds_urls() {
assert_eq!(HoopornProvider::build_listing_url(&Target::Latest, 2), "https://hooporn.com/home/2");
assert_eq!(
HoopornProvider::build_listing_url(&Target::Category("milf".into()), 3),
"https://hooporn.com/category/milf/3"
);
assert_eq!(
HoopornProvider::build_listing_url(&Target::Search("big tits".into()), 2),
"https://hooporn.com/search?q=big%20tits&p=2"
);
assert_eq!(
HoopornProvider::build_listing_url(&Target::Studio("mylf".into()), 1),
"https://hooporn.com/studio/mylf/1"
);
}
#[test]
fn routes_queries() {
let o = opts();
assert_eq!(
HoopornProvider::pick_target(Some("Double Penetration"), &o),
Target::Category("dp".into())
);
assert_eq!(
HoopornProvider::pick_target(Some("pornstar:Gal Ritchie"), &o),
Target::Pornstar("gal-ritchie".into())
);
assert_eq!(
HoopornProvider::pick_target(Some("studio:Family Strokes"), &o),
Target::Studio("familystrokes".into())
);
assert_eq!(
HoopornProvider::pick_target(Some("golf"), &o),
Target::Search("golf".into())
);
assert_eq!(HoopornProvider::pick_target(None, &o), Target::Latest);
}
#[test]
fn parses_cards_detail_and_player() {
let list = r#"<div class="video-card"><div class="video-card-image">
<a class="popout" href="/watch/abc123"><img src="//cdn.hooporn.com/img/blank.gif" data-src="//imgs.hooporn.com/t.jpg?v2" alt="x"></a>
<div class="time">37:16</div></div><div class="video-card-body"><div class="video-title">
<a class="popout" href="/watch/abc123">A &amp; B - Title</a></div>
<div class="video-view"><span class="create">1792152610</span></div></div></div>"#;
let items = HoopornProvider::parse_listing(list).unwrap();
assert_eq!(items.len(), 1);
assert_eq!(items[0].title, "A & B - Title");
assert_eq!(items[0].thumb, "https://imgs.hooporn.com/t.jpg?v2");
assert_eq!(items[0].duration, 37 * 60 + 16);
assert_eq!(items[0].uploadedAt, Some(1792152610));
let detail = r##"<iframe src="//vid.hooporn.com/video/182550"></iframe>
<p class="tag-link"><span><a class="popout" href="/category/milf">#milf</a></span></p>
<a href="/pornstar/gal-ritchie">gal ritchie</a><a href="/studio/familystrokes">FAMILY STROKES</a>"##;
let d = HoopornProvider::parse_detail(detail).unwrap();
assert_eq!(d.iframe.as_deref(), Some("https://vid.hooporn.com/video/182550"));
assert_eq!(d.tags, vec!["milf".to_string(), "gal ritchie".to_string()]);
assert_eq!(d.studio, Some(("FAMILY STROKES".to_string(), "familystrokes".to_string())));
let player = r#"var video = "https://videos.trendyporn.com/v/a.mp4?md5=X&amp;expires=1";"#;
assert_eq!(
HoopornProvider::parse_player_video(player).as_deref(),
Some("https://videos.trendyporn.com/v/a.mp4?md5=X&expires=1")
);
}
}