Manual no-ff merge of origin/pr/68 into main.
Includes follow-up fixes for vendored bcrypt/blowfish sources, admin/user management, permanent IP bans, proxy IP handling, mobile UI polish, persisted likes, and hot-video ordering.
Remove the legacy video category feature from the frontend, public API, admin API, scanner, crawler import flow, and catalog schema. Add startup migration handling that drops the old videos.category column, including a rebuild fallback for legacy databases with category-dependent indexes, and extend regression tests for existing deployments.
Keep tag-based workflows as the only classification surface by updating listing, promo, tag management, and video edit UI. Replace the old categories data module with promos and prevent category data from leaking through list or admin responses.
Improve mobile and desktop video management UX: align drive/search/refresh controls, keep bulk actions from shifting video cards, add blacklist drive filtering, center mobile modals, tighten drive picker cards, and fix edit-video tag options so short labels stay on one line.
Refine public video cards and detail navigation: truncate long author metadata, avoid empty author separators, and return users to their originating page after deleting a video instead of always navigating to /list.
Verified with go test ./..., npm run lint, npm test, npm run build, and git diff --check.
Remove the legacy spider91-specific storage, route, migration, and admin upload-target handling so crawler imports are treated as generic scriptcrawler drives.
Replace the spider91 migrator with crawlerupload and update the nightly pipeline to run generic crawler crawling, crawler uploads, and full-library duplicate video maintenance.
Add exact duplicate removal by size_bytes plus sampled_sha256 and near-duplicate removal by title similarity, duration, and thumbnail SSIM, keeping the larger source and deleting duplicate catalog rows with tombstones.
Mark automatically deduped tombstones with reason=duplicate and show a compact 重复文件 pill in the admin blacklist table while leaving manual blacklist entries unmarked.
Add media similarity helpers, scriptcrawler near-duplicate checks, file_name-backed public search, crawler upload UI updates, and tests for the new behavior.
Remove the old /p/spider91 playback route and frontend special casing after the dedicated spider91 drive implementation was removed.
Verified with: go test ./... -count=1; npm test; npm run build.
Add a manual crawler upload action in the admin UI and backend so users can retry uploads when automatic migration leaves local crawler videos behind.
Keep the button always clickable and return clear refusal messages when there are no local videos, no upload target, unfinished fingerprints/previews, failed generated assets, or active crawler work.
Simplify crawler cards by removing pipeline/status capsules, dropping the ready pill, and aligning the preview toggle with the existing action button style.
Avoid the small-/tmp upload bug by reusing seekable local files for PikPak GCID calculation/uploads and by routing fallback upload temp files for PikPak, 115, 123Pan, and WoPan into the application data upload-tmp directory.
Add regression coverage for crawler manual upload handling, frontend form expectations, configured upload temp dirs, and PikPak seekable-reader uploads.
Verification: npm run lint; npm test; npm run build; go test ./... -count=1.
Expose per-crawler teaser settings on crawler cards and persist them through the admin API.\n\nWhen preview generation is disabled, crawler imports still create thumbnails and fingerprints while marking previews disabled and allowing migration without waiting for teaser files.\n\nPreserve the latest teaser setting after crawler runs so stale crawl state cannot overwrite a user toggle.
Implement a new GuangYaPan cloud drive integration across the backend, admin UI, playback proxy, and Spider91 migration flow.
Backend changes:\n- Add a GuangYaPan drive driver with token refresh, QR/device login support, directory listing, stream link resolution, directory creation, rename/delete operations, OSS multipart upload, and upload task polling.\n- Register GuangYaPan as a supported storage kind in configuration, catalog normalization, admin APIs, public drive labels, and 302 playback redirects.\n- Allow Spider91 crawler uploads to target GuangYaPan through a dedicated migration adapter.\n- Add scan, thumbnail, preview, and fingerprint cooldown handling for GuangYaPan based on explicit HTTP status codes, Retry-After values, and structured provider codes instead of natural-language message matching.\n- Tighten existing provider cooldown detectors so OneDrive, Google Drive, 115, PikPak, 123pan, Wopan, and media workers avoid treating arbitrary response text as a rate-limit signal.\n- Keep large videos eligible for preview generation unless the user disables preview generation.
Admin and tooling changes:\n- Add GuangYaPan as a selectable drive type with QR login UI and token/root-path credential fields.\n- Add crawler upload target support for GuangYaPan in the admin UI.\n- Add drive branding, labels, metadata display, and docs/config examples for GuangYaPan.\n- Include a standalone GuangYaPan QR login helper script for manual credential acquisition.
Tests:\n- Add GuangYaPan driver, QR login, proxy, admin API, crawler upload target, fingerprint, cooldown, and form coverage.\n- Update rate-limit tests to assert that message-only throttling text no longer starts cooldowns.\n- Cover explicit HTTP status parsing through shared drive helper tests.
Add the sky theme across the frontend and backend theme APIs, including starfield assets and icon-only branding.
Refresh themed grid backgrounds, admin/login/sidebar styling, and theme-specific video/listing polish.
Local .strm files that pointed to a path outside the configured storage
root previously failed cover/preview/fingerprint generation and playback
with "strm target escapes root", breaking the common layout where the
strm library and the real media files (e.g. an rclone mount) live in
separate directories (issue #22 follow-up).
- localstorage driver gains STRMAllowOutsideRoot; when on, strm targets
outside the root are allowed (still resolves symlinks and still rejects
nested strm, so no new escape vector). Default off preserves the
existing security boundary
- Toggle persisted as the strm_allow_outside_root credential; editing a
localstorage drive now merges credentials per-key so leaving the path
blank keeps the old value while flipping the toggle
- Saving a localstorage drive with the toggle on auto-re-enqueues
previously-failed thumbnails/previews/fingerprints, so enabling it
recovers without manually clicking the three retry buttons
- Drives API exposes strmAllowOutsideRoot for form echo-back; admin
drive form adds a "允许指向目录外" select with a security warning
- Tests cover allow-outside-root on/off and that nested strm stays
rejected even when the toggle is on
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Add a transcode control to each storage in the admin drives page,
modeled after the cover/preview generation controls:
- Manual start/stop button per storage; transcoding is off by default
and never runs automatically (not triggered by scans or the nightly
pipeline)
- New transcode worker probes candidates (non mp4/webm extensions)
with ffprobe: already-compatible files are marked skipped; AVI with
H.264 is remuxed losslessly; incompatible codecs (MPEG-4 Part 2,
WMV, RMVB, HEVC...) are transcoded to H.264/AAC MP4 with +faststart
- Transcoded output is uploaded back to the same storage under a
"91转码" directory which is auto-added to the drive's scan skip list
so the scanner never re-imports the artifacts
- Playback source automatically prefers the transcoded file once
ready, keeping the 302 direct-link mode for cloud drives
- videos table gains transcode_status/error/file_id/size columns via
startup migration; counts and live task status surface in the
admin drives API and generation panel UI
- Stop semantics: per-drive stop button, drive-level "stop all tasks"
and global stop all include the transcode task; interrupted videos
keep their candidate status and resume on next start
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Add Unicom cloud drive support for source-file deletion and crawler uploads.
- Implement source-file removal for Unicom cloud drive so deleting videos can also remove the original cloud-drive file when requested.
- Resolve Unicom cloud drive source identifiers across file FID, object ID, directory ID, rename, and delete flows.
- Add upload support for Spider91 crawler imports targeting Unicom cloud drive storage.
- Add Unicom cloud drive QR login backend APIs, frontend form support, and tests.
- Extend drive capability metadata, scanner behavior, proxy handling, preview handling, and migration coverage for cloud-drive source operations.
- Rename Chinese display labels from 联通沃盘 to 联通网盘 and from 123 云盘 to 123网盘 while keeping the root README aligned with origin/main.
- Add referrer-policy coverage for 302 video playback and update related frontend playback tests.
- Count crawler assets by crawler source ID prefix after cloud migration
- Add crawler API totals for cumulative, local, and migrated videos
- Let crawler thumbnail and preview readiness inherit equivalent canonical videos
- Show cumulative crawl data in crawler management cards
- Remove low-value expanded crawler metadata fields from the card body
- Move return-to-site into the main admin navigation with grouped sections
- Rename the content admin group to management and adjust footer icon sizing
- Update backend and frontend tests for crawler/admin behavior
Redesign crawler management around imported Python scripts instead of built-in crawler storage. Crawler scripts now declare CRAWLER_NAME, imports validate metadata, crawler IDs are generated internally, and deleted crawler scripts are detached without deleting already imported videos.
Add backend support for file and URL script imports, dry-run testing, metadata parsing, safer job paths, original filename preservation, and crawler listing that ignores detached script records. Remove the legacy built-in Spider91 script path flow and hidden Python/config JSON fields from the crawler API.
Rework the admin crawler page into an independent crawler console with script import, dry-run testing, status metrics, spider iconography, and simplified controls. Update docs, examples, installer checks, Docker/release packaging, and tests for the new protocol.
- add generic scriptcrawler backend runner using the crawler.v1 JSONL protocol
- support crawler script upload and HTTP(S) URL import from the admin crawler page
- simplify the user-facing crawler contract to title, media_url, optional thumbnail_url and optional source_id
- convert Spider91 into a normal script crawler and reject new Spider91 storage-drive configs
- keep legacy Spider91 storage rows visible only for cleanup/deletion
- add crawler protocol docs, example script, admin UI, tests and migration coverage
Add per-drive and global admin controls to stop scan, preview, thumbnail, and fingerprint work.
Keep stopped pending generation resumable, wire cancellation through workers and nightly runs, and refine mobile drive-management UI/history behavior.
Restore the previous fixed-tag classification flow, including startup backfill for existing videos and the 91porn spider tag.
Also commit the current drive scanning, preview scheduling, and admin drive-control updates present in the workspace.
nianzhibai