simon/jacuzzi
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Load a split step's videos together, and three steps ahead

Browse Source 
Preloading was counted in steps and then shared out between the panels,
so with four panels only the very next step was ready and each panel did
its resolving and buffering at the moment it was swiped to -- four
panels, four starts in a row. It is now counted per panel: every panel
keeps its next three videos ready (N * 3 for a step of N), the rendered
window reaches far enough forward to hold them, and the step right
behind the one being watched buffers for real rather than taking its
metadata alone.

Their formats are resolved up front and all at once, rather than one
panel at a time as each is swiped to.

The listing is fetched several pages at a time to keep up: a page number
is claimed before its request goes out, which is what lets pages
overlap, and finished pages wait their turn so the reader still gets
them in order. A page whose request fails puts its number back rather
than leaving a hole. How far ahead to run scales with the split, since a
step of N panels gets through a page N times as fast.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MPZFnNdHbPGDTqQUNiE4ZN
This commit is contained in:
Simon
2026-09-22 09:11:52 +00:00
parent 1b38f9d9e7
commit 8163693150
3 changed files with 536 additions and 84 deletions
Download Patch File Download Diff File Expand all files Collapse all files

98
frontend/js/feed.js
View File

@@ -22,7 +22,14 @@ App.feed = App.feed || {};
// Separately we prefetch PREFETCH_PAGES worth of video JSON ahead of the
// active slide so the data buffer is always full before we need to render
// a slide from it.
const PRELOAD_COUNT = 2; // slides ahead kept with a live <video> playing/preloaded
// Preloading is counted in steps, so it scales with the split rather than
// being shared out between the panels: every panel keeps its own next three
// videos ready, which is N * 3 videos loading ahead for a step of N panels.
// The step right behind the one being watched buffers for real; the two
// behind that hold their metadata, which is what makes their first frame
// immediate without four more streams downloading in full.
const PRELOAD_STEPS = 3; // steps ahead kept loaded, per panel
const BUFFER_STEPS = 1; // of those, how many buffer rather than just resolve
const RENDER_AHEAD = 5; // slides ahead kept materialized in the DOM
const HISTORY_COUNT = 5; // slides behind kept materialized in the DOM
const PREFETCH_PAGES = 2; // pages of JSON to keep buffered ahead of the active slide
@@ -453,7 +460,9 @@ App.feed = App.feed || {};
// `opts.silent` plays without sound whatever the panel's own switch says --
// for a panel that is only half on screen (see peek), whose audio would
// otherwise talk over the one still being watched.
// otherwise talk over the one still being watched. `opts.buffer` asks for a
// real buffer rather than metadata alone, for a step close enough that it
// is about to be swiped to.
const loadSlideSource = function(slide, videoData, autoplay, opts) {
const video = slide.querySelector('.feed-video');
if (!video) return;
@@ -506,11 +515,11 @@ App.feed = App.feed || {};
const isHls = App.videos.classifySource(resolved).isHls;
video.muted = silent || slide._muted !== false;
// The step being watched buffers properly; the one preloaded behind it
// only needs enough to start instantly on the swipe. With four panes
// that is the difference between four extra streams downloading and
// four holding a few seconds each.
video.preload = autoplay ? 'auto' : 'metadata';
// The step being watched and the one right behind it buffer properly,
// so a swipe lands on video that is already there. Steps further ahead
// take their metadata only: enough to start without delay, without four
// more streams downloading in full behind them.
video.preload = (autoplay || (opts && opts.buffer)) ? 'auto' : 'metadata';
video._tearingDown = false;
applyResume(video, videoData && videoData.id, resolved.isLive);
@@ -534,11 +543,13 @@ App.feed = App.feed || {};
video._hlsPlayer = hls;
hls.loadSource(streamUrl);
hls.attachMedia(video);
if (!autoplay && split) {
if (!autoplay && split && !(opts && opts.buffer)) {
// Buffered enough to start instantly, then stopped: four
// panels' worth of hls.js all fetching and demuxing ahead is
// work for a step the reader has not swiped to yet. loadSlide-
// Source runs again with autoplay when they do.
// work for a step the reader has not swiped to yet. The step
// right behind the active one is exempt -- it is the one about
// to be swiped to, and maxBufferLength stops it on its own.
// loadSlideSource runs again with autoplay when they do.
hls.on(HlsLib.Events.FRAG_BUFFERED, function once() {
hls.off(HlsLib.Events.FRAG_BUFFERED, once);
if (video._hlsPlayer === hls) hls.stopLoad();
@@ -955,10 +966,12 @@ App.feed = App.feed || {};
// that fell outside [active - HISTORY_COUNT, active + RENDER_AHEAD], builds
// any missing ones inside it, and sizes the top spacer to stand in for the
// slides above the window.
// Which steps are materialised. Steps ahead/behind are counted in steps,
// but each one now costs a whole step's worth of <video> elements, so
// splitting four ways should not put four times as many in the document:
// the window narrows as panes are added.
// Which steps are materialised. Behind, the window narrows as panes are
// added -- history is the cheap thing to give up. Ahead it does not: what
// is preloaded has to be in the document to be preloaded at all, so the
// window always reaches at least PRELOAD_STEPS forward however many panels
// there are. A four-way split therefore holds four times as many elements
// as a single panel, which is the price of four panels never waiting.
//
// The top spacer stands in for the steps below `start`, so anything that
// sizes the spacer has to agree with this exactly -- disagreeing by a step
@@ -972,7 +985,7 @@ App.feed = App.feed || {};
return {
start: Math.max(0, activeIndex - Math.max(1, Math.round(HISTORY_COUNT / per))),
end: Math.min(stepCount() - 1,
activeIndex + Math.max(1, Math.round(RENDER_AHEAD / per)))
activeIndex + Math.max(PRELOAD_STEPS, Math.round(RENDER_AHEAD / per)))
};
};
@@ -1002,13 +1015,38 @@ App.feed = App.feed || {};
if (spacer) spacer.style.height = `${start * slideHeight()}px`;
};
// Once the active slide gets within PREFETCH_PAGES of the end of the loaded
// JSON, pull the next page so the buffer stays ahead of the rendered window.
// The videos the next few swipes will want, resolved now and all at once.
// Resolving is a round trip per video (yt-dlp, on the backend), and leaving
// it until load time put that round trip between a panel and its first
// frame -- with four panels, four of them, arriving one after another. They
// are deduped per video, so asking early costs nothing but the request, and
// asking for all of them at once is the point: they run in parallel.
const warmFormats = function(activeIndex) {
if (!App.videos || typeof App.videos.ensureFormats !== 'function') return;
const videos = state.loadedVideos || [];
const per = paneCount();
const from = Math.max(0, activeIndex * per);
const to = Math.min(videos.length, from + per * (PRELOAD_STEPS + 1));
for (let i = from; i < to; i++) {
if (videos[i]) App.videos.ensureFormats(videos[i]);
}
};
// Keeps the JSON buffer ahead of what the feed is about to eat. A step of N
// panels consumes N videos per swipe, so the same page of listing goes N
// times as fast -- the buffer it needs, and the number of pages worth having
// on the wire at once, both scale with the split.
const prefetchIfNeeded = function(activeIndex) {
const per = paneCount();
const perPage = state.perPage || 12;
const total = (state.loadedVideos || []).length;
const bufferAhead = total - 1 - ((activeIndex + 1) * paneCount() - 1);
if (bufferAhead < PREFETCH_PAGES * (state.perPage || 12)
&& state.hasNextPage && !state.isLoading) {
const bufferAhead = total - 1 - ((activeIndex + 1) * per - 1);
// Everything the rendered window will reach into, plus room to grow.
const needed = Math.max(PREFETCH_PAGES * perPage, per * (PRELOAD_STEPS + 2));
if (App.videos.setPrefetchDepth) {
App.videos.setPrefetchDepth(Math.ceil(needed / perPage));
}
if (bufferAhead < needed && state.hasNextPage && !state.isLoading) {
// The feed is its own reader: the grid's scroll position says
// nothing about whether it needs the next page.
App.videos.loadVideos({ force: true });
@@ -1016,7 +1054,7 @@ App.feed = App.feed || {};
};
// Promotes the slide at `index` to active: syncs the window, updates the
// active styling, plays it, preloads the next PRELOAD_COUNT, and tears down
// active styling, plays it, preloads the next PRELOAD_STEPS, and tears down
// playback for everything else in the window.
const setActive = function(index) {
const clamped = clampIndex(index);
@@ -1044,14 +1082,14 @@ App.feed = App.feed || {};
slidesByIndex.forEach((slide, i) => {
if (i === clamped) return;
// Same floor, same reason: at least the next step is preloaded, so
// every panel has its next video ready before the swipe. It is
// preloaded for *all* of that step's panes, which is what makes the
// guarantee hold per panel rather than only for the first.
const preloadAhead = Math.max(1, Math.round(PRELOAD_COUNT / paneCount()));
const ahead = i - clamped;
panesOf(slide).forEach((pane) => {
if (i > clamped && i <= clamped + preloadAhead) {
loadSlideSource(pane, pane._videoData, false);
if (ahead > 0 && ahead <= PRELOAD_STEPS) {
// Every panel of the step, started together: what used to
// make them start one after another is that only the first
// step ahead was loaded, so each panel did its resolving
// and buffering at the moment it was swiped to.
loadSlideSource(pane, pane._videoData, false, { buffer: ahead <= BUFFER_STEPS });
} else if (i >= clamped - KEEP_BEHIND && i < clamped) {
// Recently-watched panes stay loaded but paused so scrolling
// back resumes seamlessly from where it was paused.
@@ -1063,6 +1101,7 @@ App.feed = App.feed || {};
});
updateAutoPiPTarget();
warmFormats(clamped);
prefetchIfNeeded(clamped);
};
@@ -1729,6 +1768,9 @@ App.feed = App.feed || {};
container.setAttribute('aria-hidden', 'true');
document.body.classList.remove('feed-mode-open');
document.body.style.overflow = 'auto';
// The grid reveals a page at a time and is back in charge of the
// buffer: the feed's deeper one was for swipes eating N videos each.
if (App.videos.setPrefetchDepth) App.videos.setPrefetchDepth(1);
App.feed.updateToggleButton();
};

190
frontend/js/videos.js
View File

@@ -598,6 +598,31 @@ App.videos = App.videos || {};
// dominating a single interleaved batch.
const GROUP_CHANNEL_PAGE_SIZE = 4;
// A page number is claimed before its request goes out, not after it comes
// back, which is what lets several pages be on the wire at once. A request
// that fails puts its number back in the hat rather than leaving a hole in
// the listing. `cursor` is the group's per-channel cursor, or the session-
// wide one below for a plain channel.
const claimPage = function(cursor) {
if (cursor.retries && cursor.retries.length) return cursor.retries.shift();
return cursor.page++;
};
const returnPage = function(cursor, page) {
cursor.retries = cursor.retries || [];
cursor.retries.push(page);
cursor.retries.sort((a, b) => a - b);
};
// The plain-channel cursor. state.currentPage is the number itself, so
// resetGrid keeps working by setting it back to 1; the retry list is
// cleared alongside it in resetPrefetch.
const channelCursor = {
get page() { return state.currentPage; },
set page(next) { state.currentPage = next; },
retries: []
};
// Fetches one page from every channel in a group and zips the results
// together round-robin so the feed alternates between sources instead of
// running through one channel's videos before moving to the next.
@@ -631,23 +656,28 @@ App.videos = App.videos || {};
}
const results = await Promise.all(active.map(async (cursor) => {
const response = await fetch('/api/videos', {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({
channel: cursor.id,
query: query || "",
page: cursor.page,
perPage: GROUP_CHANNEL_PAGE_SIZE,
server: session.server
}),
signal: signal
});
const data = await response.json();
const items = data && Array.isArray(data.items) ? data.items : [];
cursor.page++;
cursor.hasNextPage = items.length > 0 && (data && data.pageInfo ? data.pageInfo.hasNextPage !== false : true);
return items;
const page = claimPage(cursor);
try {
const response = await fetch('/api/videos', {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({
channel: cursor.id,
query: query || "",
page: page,
perPage: GROUP_CHANNEL_PAGE_SIZE,
server: session.server
}),
signal: signal
});
const data = await response.json();
const items = data && Array.isArray(data.items) ? data.items : [];
cursor.hasNextPage = items.length > 0 && (data && data.pageInfo ? data.pageInfo.hasNextPage !== false : true);
return items;
} catch (err) {
returnPage(cursor, page);
throw err;
}
}));
const interleaved = [];
@@ -668,10 +698,11 @@ App.videos = App.videos || {};
const searchInput = document.getElementById('search-input');
const query = searchInput ? searchInput.value : "";
const page = claimPage(channelCursor);
const body = {
channel: session.channel.id,
query: query || "",
page: state.currentPage,
page: page,
perPage: state.perPage,
server: session.server
};
@@ -684,14 +715,19 @@ App.videos = App.videos || {};
}
});
const response = await fetch('/api/videos', {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify(body),
signal: signal
});
const videos = await response.json();
state.currentPage++;
let videos;
try {
const response = await fetch('/api/videos', {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify(body),
signal: signal
});
videos = await response.json();
} catch (err) {
returnPage(channelCursor, page);
throw err;
}
return {
items: videos && Array.isArray(videos.items) ? videos.items : [],
hasNextPage: videos && videos.pageInfo ? videos.pageInfo.hasNextPage !== false : true
@@ -712,11 +748,28 @@ App.videos = App.videos || {};
// reader reaches the second-to-last row (see the virtualizer's update()).
// So the cards that come into view are complete on arrival instead of
// shimmering their way in while the request is still on the wire.
//
// How far ahead is `prefetch.depth`: one page for the grid, more when the
// reels feed is running a split and gets through a page in two swipes.
// Pages are fetched at once and given to the reader in page order.
// ---------------------------------------------------------------------
const prefetch = {
batch: null, // fetched + warmed, waiting for the reader
inFlight: null, // promise of the fetch currently running
controller: null
ready: [], // fetched pages waiting for the reader, in page order
inFlight: [], // pages on the wire, in the order their numbers were claimed
depth: 1, // how many pages to keep fetched or in flight
exhausted: false // a page came back saying it was the last one
};
// How deep to run ahead. One page is right for the grid, which reveals a
// page at a time. The reels feed asks for more: a step of N panels eats N
// videos per swipe, so a page is a couple of swipes and fetching them one
// after another leaves the feed waiting on a round trip it could have made
// earlier. See prefetchIfNeeded in feed.js.
App.videos.setPrefetchDepth = function(depth) {
const next = Math.max(1, Math.round(depth) || 1);
if (next === prefetch.depth) return;
prefetch.depth = next;
if (next > 1) App.videos.prefetchNextBatch();
};
// One slow thumbnail must not hold a whole page back.
@@ -768,29 +821,54 @@ App.videos = App.videos || {};
App.videos.prefetchNextBatch(); // stay one page ahead again
};
App.videos.prefetchNextBatch = function() {
if (prefetch.batch || prefetch.inFlight) return Promise.resolve();
if (!state.hasNextPage) return Promise.resolve();
const session = App.storage.getSession();
if (!session || !session.channel) return Promise.resolve();
// Pages finish out of order -- page 3 can beat page 2 back -- so a finished
// page waits its turn here. Only a run of finished pages from the front
// moves across, which keeps what the reader is given in page order.
const drainFinished = function() {
while (prefetch.inFlight.length && prefetch.inFlight[0].done) {
const entry = prefetch.inFlight.shift();
if (entry.batch) prefetch.ready.push(entry.batch);
}
};
prefetch.controller = new AbortController();
const signal = prefetch.controller.signal;
prefetch.inFlight = (async () => {
const startPrefetch = function(session) {
const controller = new AbortController();
const entry = { controller: controller, batch: null, done: false };
prefetch.inFlight.push(entry);
entry.promise = (async () => {
try {
const batch = await fetchNextBatch(session, signal);
await warmThumbnails(batch.items);
if (!signal.aborted) prefetch.batch = batch;
const batch = await fetchNextBatch(session, controller.signal);
if (!controller.signal.aborted) entry.batch = batch;
// Running several pages ahead means learning where the end is
// from a page the reader hasn't been given yet; without this,
// depth more requests would be claimed past it.
if (batch && batch.hasNextPage === false) prefetch.exhausted = true;
// Warming decodes the thumbnails so cards are complete when
// they come into view. It runs alongside the pages behind it
// rather than in front of them: a page held back for a slow
// image is a page the feed is waiting on.
warmThumbnails(batch.items);
} catch (err) {
if (err.name !== 'AbortError') {
console.error("Failed to prefetch videos:", err);
}
} finally {
prefetch.inFlight = null;
prefetch.controller = null;
entry.done = true;
drainFinished();
}
})();
return prefetch.inFlight;
return entry.promise;
};
App.videos.prefetchNextBatch = function() {
if (!state.hasNextPage || prefetch.exhausted) return Promise.resolve();
const session = App.storage.getSession();
if (!session || !session.channel) return Promise.resolve();
while (prefetch.ready.length + prefetch.inFlight.length < prefetch.depth) {
startPrefetch(session);
}
return prefetch.inFlight.length ? prefetch.inFlight[0].promise : Promise.resolve();
};
// Throws away a video's resolved formats and asks the server again. Media
@@ -808,20 +886,20 @@ App.videos = App.videos || {};
// Called by the virtualizer once the reader is within two rows of the end.
App.videos.releasePrefetched = function() {
if (!prefetch.batch || state.isLoading) return false;
const batch = prefetch.batch;
prefetch.batch = null;
commitBatch(batch);
if (!prefetch.ready.length || state.isLoading) return false;
commitBatch(prefetch.ready.shift());
return true;
};
// Drops anything in flight or held, for when the result set changes out
// from under it (new search, channel, or filters).
App.videos.resetPrefetch = function() {
if (prefetch.controller) prefetch.controller.abort();
prefetch.batch = null;
prefetch.inFlight = null;
prefetch.controller = null;
prefetch.inFlight.forEach((entry) => entry.controller.abort());
prefetch.inFlight = [];
prefetch.ready = [];
prefetch.exhausted = false;
// Page numbers claimed against the old result set mean nothing now.
channelCursor.retries = [];
};
// Is the reader actually out of content? True when nothing is rendered yet,
@@ -863,16 +941,16 @@ App.videos = App.videos || {};
loadRunning = true;
try {
if (prefetch.batch || prefetch.inFlight) {
if (prefetch.ready.length || prefetch.inFlight.length) {
// A page is already here or on its way; revealing it before the
// reader gets near the end would defeat the point of holding it.
if (!force && !needsContentNow()) return;
if (App.videos.releasePrefetched()) return;
if (prefetch.inFlight) {
// Wait for it rather than asking for the same page twice --
// the fetch advances the page cursor, so a second request
if (prefetch.inFlight.length) {
// Wait for the oldest rather than asking for the same page
// twice -- its number is already claimed, so a request from
// here would skip a page entirely.
await prefetch.inFlight;
await prefetch.inFlight[0].promise;
if (App.videos.releasePrefetched()) return;
}
}