simon/hottub
0
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

docs and prompt

Browse Source
This commit is contained in:
Simon
2026-04-10 22:57:27 +00:00
parent 57eb2d7063
commit f85a725883
6 changed files with 923 additions and 61 deletions
Download Patch File Download Diff File Expand all files Collapse all files

40
docs/README.md Normal file
View File

@@ -0,0 +1,40 @@
# Hottub Docs
This folder is the fastest handoff path for anyone adding or repairing a channel.
Start here:
1. Read `architecture.md` for the server flow, request lifecycle, and core types.
2. Read `provider-playbook.md` for the exact process to add a new provider or proxy.
3. Use `provider-catalog.md` to find the closest existing implementation to copy.
4. Use `docs/hottubapp/*.html` when you need the client-facing API contract for status, videos, or uploaders.
5. Only then touch `prompts/new-channel.md`; it assumes the docs above exist.
Recommended local workflow:
```bash
cargo check -q
HOT_TUB_PROVIDER=<channel_id> cargo check -q
HOT_TUB_PROVIDER=<channel_id> cargo run --features debug
```
Useful runtime baseline:
```dotenv
DATABASE_URL=hottub.db
RUST_LOG=info
PROXY=0
BURP_URL=http://127.0.0.1:8081
FLARE_URL=http://127.0.0.1:8191/v1
DOMAIN=127.0.0.1:18080
DISCORD_WEBHOOK=
```
Key facts:
- Hottub is a Rust `ntex` server with providers under `src/providers/`.
- `build.rs` controls compile-time provider registration.
- `/api/videos` is the main provider execution path.
- `/proxy/...` exists for sites whose direct media or thumbnails need a redirect/proxy layer.
- Only three providers currently implement `/api/uploaders`: `hsex`, `omgxxx`, and `vjav`.
- Uploader IDs should be namespaced like `<channel>:<site-local-id>` so `/api/uploaders` can route directly.

313
docs/architecture.md Normal file
View File

@@ -0,0 +1,313 @@
# Hottub Architecture
## Purpose
Hottub is a Rust server that exposes Hot Tub compatible endpoints for channel discovery, video search, uploader lookups, and site-specific proxying. Most work in this repo is adding or repairing a provider module under `src/providers/`.
## Top-Level Structure
- `src/main.rs`: server bootstrap, env loading, database pool, shared requester/cache, route mounting.
- `src/api.rs`: `/api/status`, `/api/videos`, `/api/uploaders`, `/api/test`, `/api/proxies`.
- `src/providers/mod.rs`: provider trait, provider registry, build-time provider selection, status decoration, runtime validation, panic/error guards.
- `src/providers/*.rs`: one module per channel/provider.
- `src/proxy.rs`: route table for `/proxy/...`.
- `src/proxies/*.rs`: redirect/media/thumb proxy implementations.
- `src/videos.rs`: request/response payloads, `VideoItem`, `VideoFormat`, `ServerOptions`.
- `src/status.rs`: status/channel/group payloads.
- `src/uploaders.rs`: uploader request/profile payloads.
- `src/util/requester.rs`: outbound HTTP with cookies, optional Burp proxying, Jina fallback, and FlareSolverr fallback.
- `build.rs`: compile-time provider registry generation and single-provider build support.
## Startup Flow
1. `main` loads `.env` and ensures `RUST_LOG` is set.
2. It creates the Diesel SQLite pool from `DATABASE_URL`.
3. It creates a shared `Requester`, enables Burp proxying when `PROXY != 0`, and builds the LRU video cache.
4. It configures provider runtime validation in `providers::configure_runtime_validation`.
5. It spawns a background thread that forces provider initialization via `providers::init_providers_now()`.
6. It starts an `ntex` HTTP server on `0.0.0.0:18080`.
## Runtime Environment
Important environment variables:
- `DATABASE_URL`: required SQLite path.
- `RUST_LOG`: defaults to `warn` if unset.
- `PROXY`: enables Burp proxying when not equal to `0`.
- `BURP_URL`: outbound proxy URL used when `PROXY` is enabled.
- `FLARE_URL`: FlareSolverr endpoint used as the last HTML-fetch fallback.
- `DOMAIN`: used by the `/` redirect target.
- `DISCORD_WEBHOOK`: enables `/api/test` and provider error reporting.
Bundled reference material:
- `docs/hottubapp/📡 Status - Hot Tub Docs.html`
- `docs/hottubapp/🎬 Videos - Hot Tub Docs.html`
- `docs/hottubapp/👤 Uploaders - Hot Tub Docs.html`
Those HTML files are useful when a provider author needs to confirm the expected client payload shape without reading Rust structs first.
## Build-Time Provider Selection
`build.rs` reads `HOT_TUB_PROVIDER` or `HOTTUB_PROVIDER`.
- If unset, every provider in `build.rs` is compiled and registered.
- If set, only that provider is compiled into the binary.
- In a single-provider build, `/api/videos` remaps `"channel": "all"` to the compiled provider.
Generated files in `OUT_DIR` are included by `src/providers/mod.rs`:
- `provider_modules.rs`
- `provider_registry.rs`
- `provider_metadata_fn.rs`
- `provider_selection.rs`
This means adding a new provider always requires updating `build.rs`.
## HTTP Surface
### `/`
Returns a `302` redirect to `hottub://source?url=<DOMAIN-or-request-host>`.
### `/api/status`
Builds the channel list by iterating `ALL_PROVIDERS` and calling `Provider::get_channel`.
Important behavior:
- The `User-Agent` is parsed into `ClientVersion`.
- A provider can hide itself by returning `None`.
- `providers::build_status_response` decorates channels with `groupKey`, top tags, runtime status, and sort order.
- Some heavy status filters are intentionally removed from the client-facing response. The server still accepts them in `/api/videos`.
### `/api/videos`
This is the main provider execution path.
Flow:
1. Parse `VideosRequest`.
2. Normalize `channel`, `sort`, `query`, `page`, and `perPage`.
3. Build `ServerOptions`.
4. If `query` is a full `http://` or `https://` URL, try the `yt-dlp -J` fast path first.
5. Otherwise call `provider.get_videos(...)` through `run_provider_guarded`.
6. For quoted queries like `"teacher"`, apply a literal substring filter after provider fetch.
7. Spawn a background prefetch for the next page.
8. For short videos (`duration <= 120`), populate `preview` from the main URL or first format.
Important behavior:
- Leading `#` is stripped from queries before provider dispatch.
- `"all"` uses `AllProvider` in a normal build, but resolves to the single compiled provider in a single-provider build.
- Older `Hot Tub/38` clients are patched by replacing `video.url` with the last format URL when formats exist.
### `/api/uploaders`
Uploader lookup is optional and provider-specific.
Important behavior:
- At least one of `uploaderId` or `uploaderName` is required.
- If `uploaderId` looks like `channel:id`, the server directly targets that provider.
- Otherwise it scans all providers and returns the best exact-name match.
- Only `hsex`, `omgxxx`, and `vjav` currently implement `get_uploader`.
- In practice, provider-owned uploader IDs should be namespaced, for example `vjav:12345` or `hsex:author_slug`.
### `/api/test`
Sends a Discord error test if `DISCORD_WEBHOOK` is configured.
### `/api/proxies`
Returns the background-fetched outbound proxy snapshot from `src/util/proxy.rs`.
## Core Data Structures
### `VideosRequest`
Defined in `src/videos.rs`. Common fields used by providers:
- `channel`
- `sort`
- `query`
- `page`
- `perPage`
- `featured`
- `category`
- `sites`
- `all_provider_sites`
- `filter`
- `language`
- `networks`
- `stars`
- `categories`
- `duration`
- `sexuality`
### `ServerOptions`
The server’s normalized option bag. Providers should read from this instead of reparsing the raw API request.
Important fields:
- `public_url_base`: needed when generating `/proxy/...` URLs.
- `requester`: the shared request client with cookies/debug trace/proxy state.
- `sort`, `sites`, `filter`, `category`, `language`, `network`, `stars`, `categories`, `duration`, `sexuality`.
### `VideoItem`
Minimum useful fields for a provider:
- `id`
- `title`
- `url`
- `channel`
- `thumb`
- `duration`
High-value optional fields:
- `views`
- `rating`
- `uploader`
- `uploaderUrl`
- `uploaderId`
- `tags`
- `uploadedAt`
- `formats`
- `preview`
- `aspectRatio`
Avoid setting `embed` for new providers unless the site truly needs it.
### `VideoFormat`
Use `formats` when:
- the site returns a better direct media URL than the page URL
- HLS or multiple qualities exist
- extra HTTP headers such as `Referer` are required
Use `http_header` or `add_http_header` when the player endpoint needs request headers.
### `Channel` and `ChannelOption`
Each provider’s `get_channel` returns the status metadata exposed by `/api/status`.
Typical option IDs used across the repo:
- `sort`
- `filter`
- `sites`
- `category`
- `language`
- `networks`
- `stars`
- `categories`
Use the same IDs when possible so the server and client behavior stay consistent.
### `UploaderProfile`
If a provider supports `/api/uploaders`, keep the ID routable:
- preferred format: `<channel>:<site-local-id>`
- examples in the repo: `vjav:<user_id>`, `hsex:<author>`, `omgxxx:<kind>:<id>`
This lets `src/api.rs` derive the owning provider immediately.
## Provider Contract
Defined in `src/providers/mod.rs`:
- `async fn get_videos(...) -> Vec<VideoItem>`
- `fn get_channel(clientversion: ClientVersion) -> Option<Channel>`
- `async fn get_uploader(...) -> Result<Option<UploaderProfile>, String>` optional
The server wraps provider execution in:
- `run_provider_guarded` for video paths
- `run_uploader_provider_guarded` for uploader paths
Panics and reported errors trigger runtime validation and optional Discord reporting.
## Runtime Validation and Error Handling
`src/providers/mod.rs` includes a validation subsystem that:
- runs a small sample request against a provider after failures
- checks that enough video items exist
- tries media URLs or format URLs with a `Range` header
- marks repeated failures over time
This means a provider that returns page URLs but no real media/formats may pass visually but still fail operationally.
## Requester Behavior
`src/util/requester.rs` is the standard outbound HTTP layer.
Capabilities:
- shared cookie jar across clones
- optional Burp proxying via `PROXY` and `BURP_URL`
- direct request retries for `429`
- Jina mirror fallback for blocked HTML fetches
- FlareSolverr fallback via `FLARE_URL`
- raw response helpers for media validation and custom headers
Use the shared requester from `ServerOptions` through `requester_or_default`. Do not instantiate a brand-new requester in normal provider fetch paths unless you have a very specific reason.
FlareSolverr note:
- `src/util/flaresolverr.rs` keeps a reusable session pool pattern by rotating a ready session per solve.
- If a provider only works after anti-bot negotiation, the shared requester is the path that benefits from that solved session and cookie state.
## Proxy Subsystem
There are two proxy styles.
### Redirect proxies
These take a provider-specific endpoint and return `302 Location: <resolved-media-url>`.
Examples:
- `/proxy/spankbang/...`
- `/proxy/sxyprn/...`
- `/proxy/pornhd3x/...`
- `/proxy/vjav/...`
### Media or image proxies
These actively fetch media or thumbnails and stream or rewrite the response.
Examples:
- `/proxy/noodlemagazine/...`
- `/proxy/noodlemagazine-thumb/...`
- `/proxy/shooshtime-media/...`
- `/proxy/hanime-cdn/...`
If a site only needs a referer-preserving redirect, use a redirect proxy. If manifests, relative playlist entries, cookies, or binary thumbs need rewriting, use a media/image proxy.
## Best Existing Templates
Use the closest existing provider instead of inventing a new style.
- `src/providers/vjav.rs`: rich API-backed provider with tags, uploader support, and detail enrichment.
- `src/providers/hsex.rs`: HTML scraping with background-loaded filters, uploader support, and direct HLS formats.
- `src/providers/omgxxx.rs`: large filter catalogs and uploader lookup by site/network identity.
- `src/providers/noodlemagazine.rs`: proxied media/thumbs, Jina fallback, and mirrored listing parsing.
- `src/providers/pornhd3x.rs`: complex filter catalogs, detail enrichment, and proxy-generated playback URLs.
- `src/providers/spankbang.rs`: anti-bot handling and a redirect-proxy-based media strategy.
## Important Gotchas
- New providers must export `CHANNEL_METADATA`.
- New providers must be listed in `build.rs` or they will never compile into the registry.
- If a provider returns proxied URLs, it usually also needs `options.public_url_base`.
- Keep filter IDs stable. The `title` is for display; the `id` is what the provider matches on.
- `categories` in `Channel` are not the same as `ChannelOption { id: "categories" }`.
- `/api/status` sanitizes some options away from the client-facing payload. That does not mean the provider option is useless in `/api/videos`.
- If a site needs per-request cookies or a solved user agent, rely on the shared requester.

95
docs/provider-catalog.md Normal file
View File

@@ -0,0 +1,95 @@
# Provider And Proxy Catalog
This is the current implementation inventory as of this snapshot of the repo. Use it to find the nearest existing pattern before adding a new channel.
## Providers
| Provider | Group | `/api/uploaders` | Uses local `/proxy` | Notes |
| --- | --- | --- | --- | --- |
| `all` | `meta-search` | no | no | Aggregates all compiled providers. |
| `beeg` | `mainstream-tube` | no | no | Basic mainstream tube pattern. |
| `chaturbate` | `live-cams` | no | no | Live cam channel. |
| `freepornvideosxxx` | `studio-network` | no | no | Studio-style scraper. |
| `freeuseporn` | `fetish-kink` | no | no | Fetish archive pattern. |
| `hanime` | `hentai-animation` | no | yes | Uses proxied CDN/thumb handling. |
| `heavyfetish` | `fetish-kink` | no | no | Direct media handling. |
| `hentaihaven` | `hentai-animation` | no | no | HLS format builder pattern. |
| `homoxxx` | `gay-male` | no | no | Gay category grouping example. |
| `hqporner` | `studio-network` | no | yes | Uses thumb and redirect proxy helpers. |
| `hsex` | `chinese` | yes | no | Strong template for tags, uploaders, and direct HLS formats. |
| `hypnotube` | `fetish-kink` | no | no | Fetish/tube hybrid. |
| `javtiful` | `jav` | no | no | JAV channel family. |
| `missav` | `jav` | no | no | HLS format pattern. |
| `noodlemagazine` | `mainstream-tube` | no | yes | Best template for media and thumbnail proxying. |
| `okporn` | `mainstream-tube` | no | no | Simple mainstream archive. |
| `okxxx` | `mainstream-tube` | no | no | Mainstream search/archive pattern. |
| `omgxxx` | `studio-network` | yes | no | Best template for sites/networks/stars filter catalogs. |
| `paradisehill` | `mainstream-tube` | no | no | Simple page scraper. |
| `perfectgirls` | `studio-network` | no | no | Studio archive. |
| `perverzija` | `studio-network` | no | no | Multi-format/HLS examples. |
| `pimpbunny` | `onlyfans` | no | yes | Proxy-backed playback and thumbnail handling. |
| `pmvhaven` | `pmv-compilation` | no | no | PMV grouping example. |
| `porn00` | `mainstream-tube` | no | no | Lightweight scraper. |
| `porn4fans` | `onlyfans` | no | no | OnlyFans-like grouping example. |
| `porndish` | `studio-network` | no | yes | Redirect proxy plus thumb proxy usage. |
| `pornhat` | `mainstream-tube` | no | no | Basic tube provider. |
| `pornhd3x` | `studio-network` | no | yes | Best template for complex catalogs and redirect proxy generation. |
| `pornhub` | `mainstream-tube` | no | no | Rich metadata and format examples. |
| `pornmz` | `mainstream-tube` | no | no | Mainstream archive. |
| `pornzog` | `mainstream-tube` | no | no | Basic list/detail scraper. |
| `redtube` | `mainstream-tube` | no | no | Mainstream archive. |
| `rule34gen` | `ai` | no | no | AI group example. |
| `rule34video` | `hentai-animation` | no | no | Hentai group example. |
| `sextb` | `jav` | no | no | JAV family provider. |
| `shooshtime` | `onlyfans` | no | yes | Redirect proxy plus dedicated media route. |
| `spankbang` | `mainstream-tube` | no | yes | Best template for redirect proxy plus anti-bot fetches. |
| `supjav` | `jav` | no | no | JAV/HLS and uploader-id examples. |
| `sxyprn` | `mainstream-tube` | no | yes | Redirect proxy helper usage. |
| `tnaflix` | `mainstream-tube` | no | no | Mainstream tube provider. |
| `tokyomotion` | `jav` | no | no | JAV/tube hybrid. |
| `viralxxxporn` | `mainstream-tube` | no | no | Basic parser with format extraction. |
| `vjav` | `jav` | yes | no | Best API-style template with uploaders and tag-id lookup maps. |
| `vrporn` | `studio-network` | no | no | Multi-format direct playback. |
| `xfree` | `tiktok` | no | no | Short-form grouping example. |
| `xxdbx` | `onlyfans` | no | no | OnlyFans-like grouping example. |
| `xxthots` | `onlyfans` | no | no | OnlyFans-like metadata example. |
| `yesporn` | `mainstream-tube` | no | no | Preview format examples. |
| `youjizz` | `mainstream-tube` | no | no | Mainstream tube provider. |
## Proxy Routes
### Redirect proxies
These resolve a provider-specific input into a `302 Location`.
- `/proxy/doodstream/{endpoint}*`
- `/proxy/sxyprn/{endpoint}*`
- `/proxy/javtiful/{endpoint}*`
- `/proxy/spankbang/{endpoint}*`
- `/proxy/porndish/{endpoint}*`
- `/proxy/hqporner/{endpoint}*`
- `/proxy/heavyfetish/{endpoint}*`
- `/proxy/vjav/{endpoint}*`
- `/proxy/pornhd3x/{endpoint}*`
- `/proxy/shooshtime/{endpoint}*`
- `/proxy/pimpbunny/{endpoint}*`
### Media/image proxies
These return binary media or images, sometimes rewriting manifests or forwarding cookies/referers.
- `/proxy/shooshtime-media/{endpoint}*`
- `/proxy/noodlemagazine/{endpoint}*`
- `/proxy/noodlemagazine-thumb/{endpoint}*`
- `/proxy/hanime-cdn/{endpoint}*`
- `/proxy/hqporner-thumb/{endpoint}*`
- `/proxy/porndish-thumb/{endpoint}*`
- `/proxy/pornhub-thumb/{endpoint}*`
## Best Copy Sources By Problem
- Need uploader support: copy `hsex`, `omgxxx`, or `vjav`.
- Need proxied media: copy `noodlemagazine`.
- Need proxied redirect-only playback: copy `spankbang` or `pornhd3x`.
- Need big background-loaded filter catalogs: copy `pornhd3x` or `omgxxx`.
- Need tag title to site-ID lookup maps: copy `vjav` or `hsex`.

349
docs/provider-playbook.md Normal file
View File

@@ -0,0 +1,349 @@
# New Provider Playbook
This is the implementation checklist for adding a working channel with the least guessing.
## Definition Of Done
A provider is not done when it compiles. It is done when:
1. `/api/status` shows the channel with sensible options and grouping.
2. `/api/videos` returns real items for the default feed.
3. Search works.
4. Pagination works.
5. Thumbnails load.
6. `video.url` or at least one `formats[*].url` resolves to playable media.
7. If the site needs proxying, the `/proxy/...` route works.
8. `HOT_TUB_PROVIDER=<id> cargo check -q` passes.
## Files To Touch
Always:
- `build.rs`
- `src/providers/<channel_id>.rs`
Sometimes:
- `src/proxy.rs`
- `src/proxies/<channel_id>.rs`
- `src/proxies/<channel_id>thumb.rs`
- `prompts/new-channel.md` if you are improving the handoff prompt
- `docs/provider-catalog.md` if you add a new provider or proxy
## Step 1: Pick The Closest Template
Do not start from an empty file.
Choose the nearest match:
- API-first site with tags/uploader metadata: copy `vjav.rs`
- HTML site with background-loaded tags/uploaders: copy `hsex.rs`
- Site with multiple large catalogs like sites/networks/stars: copy `omgxxx.rs`
- Site whose media or thumbs need local proxying: copy `noodlemagazine.rs`, `pornhd3x.rs`, `spankbang.rs`, or `porndish.rs`
- Very simple archive/search site: copy a small provider from `mainstream-tube`
Before writing code, confirm the site shape:
1. home or latest feed URL
2. search URL and page 2 URL
3. detail page URL shape
4. player request or manifest request
5. thumbnail host and whether it needs referer/cookies
6. tag/category/uploader/studio routes if they exist
7. whether the site exposes JSON endpoints that are easier than HTML scraping
Use browser/network tooling for this if needed. Do not guess URL patterns from one page.
## Step 2: Register The Provider
Add the provider to `build.rs`:
- `id`: channel id used by `/api/videos`
- `module`: Rust file name
- `ty`: provider struct name
If this is missing, the server will not discover the provider.
## Step 3: Define Channel Metadata
Every provider should export:
```rust
pub const CHANNEL_METADATA: crate::providers::ProviderChannelMetadata =
crate::providers::ProviderChannelMetadata {
group_id: "...",
tags: &["...", "...", "..."],
};
```
Pick `group_id` from the existing set in `src/providers/mod.rs`:
- `meta-search`
- `mainstream-tube`
- `tiktok`
- `studio-network`
- `amateur-homemade`
- `onlyfans`
- `chinese`
- `jav`
- `fetish-kink`
- `hentai-animation`
- `ai`
- `gay-male`
- `live-cams`
- `pmv-compilation`
## Step 4: Build The Channel Surface
Implement `build_channel` or equivalent and return it from `get_channel`.
Required:
- `id`
- `name`
- `description`
- `favicon`
- `status`
- `nsfw`
Recommended:
- `cacheDuration: Some(1800)` unless the site is unusually stable
- use standard option IDs like `sort`, `filter`, `sites`, `category`, `stars`, `categories`
- keep options minimal at first; only expose filters that actually work in `get_videos`
The option `id` values matter more than the display `title`.
## Step 5: Model Provider Routing Explicitly
Create a local enum like:
```rust
enum Target {
Latest,
Search { query: String },
Tag { slug: String },
Uploader { id: String },
}
```
Then write one function that resolves `sort`, `query`, `filter`, `sites`, and related options into a `Target`.
This is easier to debug than scattering URL decisions across the provider.
## Step 6: Load Filter Catalogs In The Background If Needed
If the site exposes tags, uploaders, studios, networks, or stars:
- store them in `Arc<RwLock<Vec<FilterOption>>>`
- initialize them with an `All` option
- spawn a background thread in `new()`
- create a tiny Tokio runtime inside that thread
- fill the lists without blocking server startup
Patterns:
- `hsex.rs`
- `omgxxx.rs`
- `pornhd3x.rs`
- `vjav.rs`
If tags or uploaders need stable IDs, keep a lookup map such as:
- `HashMap<String, String>` from title to site ID
- `HashMap<String, String>` from site ID to URL target
Normalize lookup keys to lowercase trimmed strings.
## Step 7: Fetch Pages Through The Shared Requester
In `get_videos`, start with:
```rust
let mut requester = requester_or_default(&options, CHANNEL_ID, "get_videos");
```
Use it for HTML, JSON, and raw media requests.
Why:
- it preserves cookies
- it carries debug trace IDs
- it respects Burp proxying
- it can fall back to Jina or FlareSolverr
## Step 8: Parse Listing Cards First, Then Enrich Only If Needed
Preferred flow:
1. Fetch the archive or search page.
2. Parse a lightweight list of stubs.
3. Return list data directly if enough metadata is already present.
4. Fetch detail pages or JSON endpoints only for fields the card does not expose.
Use bounded concurrency for detail enrichment. Existing providers usually use `futures::stream` with `buffer_unordered`.
## Step 9: Build High-Quality `VideoItem`s
Always fill:
- `id`
- `title`
- `url`
- `channel`
- `thumb`
- `duration`
Fill when available:
- `views`
- `rating`
- `uploader`
- `uploaderUrl`
- `uploaderId`
- `tags`
- `uploadedAt`
- `preview`
- `aspectRatio`
- `formats`
Rules:
- Keep `tags` as a list of displayable titles.
- Keep uploader data as structured fields, not mashed into the title.
- If you support uploader profiles, set `uploaderId` to a namespaced value like `<channel>:<site-local-id>`.
- Do not include `embed` unless the provider truly needs it.
- If direct media exists, prefer `formats` and keep `url` stable.
## Step 10: Decide Whether A Proxy Is Required
Use no proxy when:
- page URLs are enough and the client can resolve media itself
- or direct media URLs already work cleanly
Use a redirect proxy when:
- the provider must turn a detail URL into a resolved media URL
- headers/cookies do not need full response rewriting
Use a media/image proxy when:
- the site requires a referer for every fetch
- thumbnails need cookie-backed access
- manifests contain relative URIs that must be rewritten
- the server must stream binary content itself
If a proxy is needed:
1. add `src/proxies/<id>.rs`
2. wire the route in `src/proxy.rs`
3. generate provider URLs with `build_proxy_url(&options, "<id>", target)`
## Step 11: Implement Search Correctly
Check for three search modes:
1. native site search endpoint
2. tag/uploader shortcut search from preloaded filter catalogs
3. literal client-side substring search after fetch, triggered by quoted queries
Important server behavior:
- `#tag` becomes `tag`
- `"teacher"` becomes a literal post-fetch filter
- raw URL queries may bypass the provider through the `yt-dlp` fast path
Provider guidance:
- if the query matches a known tag/uploader shortcut, prefer the site’s direct archive URL instead of generic search
- otherwise fall back to the site’s keyword search
## Step 12: Support Pagination Explicitly
Do not assume pagination is `?page=N`.
Confirm:
- archive page 2 URL shape
- search page 2 URL shape
- tag page 2 URL shape
- uploader page 2 URL shape
If the site uses infinite scroll or an XHR endpoint, document that in code comments and hit the underlying endpoint directly.
## Step 13: Only Add `/api/uploaders` When The Site Has Real Uploader Identity
Uploader support is optional. Only implement it when the site exposes stable uploader pages or IDs.
Use `hsex.rs`, `omgxxx.rs`, or `vjav.rs` as the template.
Minimum expectations for `UploaderProfile`:
- stable `id`
- `name`
- `channel`
- `videoCount`
- `totalViews`
Nice to have:
- `avatar`
- `description`
- `videos`
- `layout`
- per-channel stats
## Validation Checklist
Run all of these:
```bash
cargo check -q
HOT_TUB_PROVIDER=<channel_id> cargo check -q
HOT_TUB_PROVIDER=<channel_id> cargo run --features debug
```
Then hit:
```bash
curl -s http://127.0.0.1:18080/api/status \
-H 'User-Agent: Hot%20Tub/22c CFNetwork/1494.0.7 Darwin/23.4.0' | jq
```
```bash
curl -s http://127.0.0.1:18080/api/videos \
-H 'Content-Type: application/json' \
-d '{"channel":"<channel_id>","sort":"new","page":1,"perPage":10}' | jq
```
Also verify:
- search query works
- page 2 works
- tag shortcut works if implemented
- uploader shortcut works if implemented
- `yt-dlp '<video.url or first format url>'` resolves media
- thumbnail URL returns an image
- proxy route returns a `302` or working media body, whichever is expected
- if uploaders are implemented, `/api/uploaders` works with both `uploaderId` and `uploaderName`
## Common Failure Modes
- Forgot `build.rs` entry.
- Returned page URLs but no playable media/formats.
- Used a local requester instead of the shared one and lost cookies.
- Built `/proxy/...` URLs without `public_url_base`.
- Put human-readable titles into filter IDs, making routing brittle.
- Added huge option lists to the status response without background loading.
- Implemented search but not search pagination.
- Implemented proxies but forgot to test them independently with `curl -I`.
## Best Reference Matrix
- Rich uploader support: `vjav.rs`, `hsex.rs`, `omgxxx.rs`
- Tag and uploader lookup maps: `vjav.rs`, `hsex.rs`
- Background catalog loading: `hsex.rs`, `omgxxx.rs`, `pornhd3x.rs`
- Redirect proxy: `spankbang.rs` plus `src/proxies/spankbang.rs`
- Manifest or image proxy: `noodlemagazine.rs` plus `src/proxies/noodlemagazine.rs`
- Complex detail enrichment: `pornhd3x.rs`

60
docs/uploaders-endpoint-plan.md
View File

@@ -1,60 +0,0 @@
# Uploaders Endpoint Plan
## Summary
Implement `POST /api/uploaders` using the Hot Tub uploader profile contract and ship it framework-first. The server will expose shared uploader request/response types, a provider hook for uploader lookup, endpoint routing in `src/api.rs`, and a first real provider implementation in `hsex`.
## Implementation
- Add dedicated uploader API types in `src/uploaders.rs`:
- `UploadersRequest`
- `UploaderProfile`
- `UploaderChannelStat`
- `UploaderVideoRef`
- `UploaderLayoutRow`
- Keep camelCase as the canonical serialized shape.
- Accept documented decode aliases:
- `uploader_id`
- `uploader_name`
- `profile_content`
- `profile_picture_url`
- `video_ids`
- `horizontal_videos`
- Add `POST /api/uploaders` in `src/api.rs`.
- Validate that at least one of `uploaderId` or `uploaderName` is present.
- Return:
- `400` for invalid request
- `404` for no match
- `500` for provider execution failure
- Add `Provider::get_uploader(...)` with a default `Ok(None)` implementation.
- Add a guarded uploader execution helper in `src/providers/mod.rs`.
- Use canonical uploader IDs in the format `<channel>:<provider-local-id>`.
- Implement the first provider-backed uploader profile in `src/providers/hsex.rs`.
## Hsex Strategy
- Resolve uploader lookup by canonical uploader ID or exact uploader name.
- Reuse existing uploader archive discovery and archive page fetching.
- Build uploader profile metadata from uploader archive pages.
- Populate `videos` with `UploaderVideoRef` values derived from existing `VideoItem`s.
- Always return `layout`.
- When `profileContent == true`, return:
- `videos`
- `tapes: []`
- `playlists: []`
- a `"For You"` horizontal row plus the default videos row
- When `profileContent == false`, return metadata and layout only.
## Tests
- Request alias decoding for uploader request fields.
- Response alias decoding for avatar and layout row compatibility fields.
- Endpoint helper tests for request validation and provider routing.
- Hsex uploader ID generation and uploader page parsing coverage.
## Assumptions
- The first ship focuses on the endpoint framework and one real provider implementation.
- Providers without explicit uploader support remain unsupported by `/api/uploaders`.
- Name-based resolution uses exact display-name matching.
- `videoCount` and `totalViews` are best-effort when the upstream site does not expose authoritative profile totals.