21 Commits

Author SHA1 Message Date
nianzhibai 00aaeed736 feat: add recent watch sorting 2026-06-20 15:08:47 +08:00
nianzhibai 5efbceb205 fix: add safe-area home screen icons 2026-06-20 14:36:24 +08:00
nianzhibai 0faeaf408f fix(crawlers): stabilize manual upload workflow
Add a manual crawler upload action in the admin UI and backend so users can retry uploads when automatic migration leaves local crawler videos behind.

Keep the button always clickable and return clear refusal messages when there are no local videos, no upload target, unfinished fingerprints/previews, failed generated assets, or active crawler work.

Simplify crawler cards by removing pipeline/status capsules, dropping the ready pill, and aligning the preview toggle with the existing action button style.

Avoid the small-/tmp upload bug by reusing seekable local files for PikPak GCID calculation/uploads and by routing fallback upload temp files for PikPak, 115, 123Pan, and WoPan into the application data upload-tmp directory.

Add regression coverage for crawler manual upload handling, frontend form expectations, configured upload temp dirs, and PikPak seekable-reader uploads.

Verification: npm run lint; npm test; npm run build; go test ./... -count=1.
2026-06-20 00:14:37 +08:00
nianzhibai 1b5eda92b0 Merge pull request #64 from iBenzene/feat/google-drive-openlist-api-url 2026-06-19 10:57:15 +08:00
nianzhibai 840a858dbd chore: remove bundled 91VideoSpider script 2026-06-19 10:42:35 +08:00
nianzhibai 1ee5ee35be Merge pull request #63 from iBenzene/fix/rescan-sync-renamed-videos
fix(scanner): sync renamed video metadata on rescan
2026-06-19 10:32:20 +08:00
iBenzene 12b737b6fe feat: add google drive openlist api url field 2026-06-19 10:23:08 +08:00
iBenzene bd33d26a1f fix(scanner): sync renamed video metadata on rescan 2026-06-19 08:40:38 +08:00
nianzhibai 36fe32cb84 feat: optimize toast feedback and bundle splitting
- lazy-load route pages and use the HLS light build to remove large chunk warnings

- copy toast messages to the clipboard with success/failure feedback

- make scriptcrawler dry-run stderr capture deterministic
2026-06-18 23:29:16 +08:00
nianzhibai 194d98895a fix: allow guangyapan crawler uploads and improve admin toasts 2026-06-17 17:33:58 +08:00
nianzhibai 2437fbd779 发布v0.1.9 版本
1. 修复Apple设备短视频页面无法播放问题
2. 优化短视频播放页面交互
2026-06-15 20:23:08 +08:00
nianzhibai 4dd66b8120 fix: stabilize shorts playback and iPhone fullscreen controls 2026-06-15 20:19:28 +08:00
nianzhibai 30b736cf36 Merge pull request #54 2026-06-15 17:38:32 +08:00
Long Chen 57391e0e98 fix: render short videos on iOS by dropping fixed positioning
Short-video mode showed a black screen on iOS Safari/WebKit while working
on desktop. iOS does not composite an inline <video> nested inside a
`position: fixed` ancestor: the video decodes and plays but its layer
never paints. `.shorts-page` used `position: fixed; inset: 0`, trapping
every shorts video.

Make `.shorts-page` a normal-flow full-viewport block
(`position: relative; height: 100svh` with dvh/vh fallbacks) instead.
The immersive scroll lock is already provided by the component setting
html/body `overflow: hidden` on mount, so the look and behavior are
unchanged. Fixes the iOS black screen; desktop is unaffected.
2026-06-15 17:20:35 +08:00
nianzhibai 052e142520 chore: bump version to 0.1.8 2026-06-14 18:25:19 +08:00
nianzhibai f9351324c6 fix: show active preview generation status 2026-06-14 18:22:04 +08:00
nianzhibai bb83277d62 feat: add crawler preview generation toggle
Expose per-crawler teaser settings on crawler cards and persist them through the admin API.\n\nWhen preview generation is disabled, crawler imports still create thumbnails and fingerprints while marking previews disabled and allowing migration without waiting for teaser files.\n\nPreserve the latest teaser setting after crawler runs so stale crawl state cannot overwrite a user toggle.
2026-06-14 17:52:29 +08:00
nianzhibai aa856db1f6 fix: refine video UI loading and playback behavior 2026-06-14 16:59:41 +08:00
nianzhibai 7e5e67697e feat: add GuangYaPan drive support
Implement a new GuangYaPan cloud drive integration across the backend, admin UI, playback proxy, and Spider91 migration flow.

Backend changes:\n- Add a GuangYaPan drive driver with token refresh, QR/device login support, directory listing, stream link resolution, directory creation, rename/delete operations, OSS multipart upload, and upload task polling.\n- Register GuangYaPan as a supported storage kind in configuration, catalog normalization, admin APIs, public drive labels, and 302 playback redirects.\n- Allow Spider91 crawler uploads to target GuangYaPan through a dedicated migration adapter.\n- Add scan, thumbnail, preview, and fingerprint cooldown handling for GuangYaPan based on explicit HTTP status codes, Retry-After values, and structured provider codes instead of natural-language message matching.\n- Tighten existing provider cooldown detectors so OneDrive, Google Drive, 115, PikPak, 123pan, Wopan, and media workers avoid treating arbitrary response text as a rate-limit signal.\n- Keep large videos eligible for preview generation unless the user disables preview generation.

Admin and tooling changes:\n- Add GuangYaPan as a selectable drive type with QR login UI and token/root-path credential fields.\n- Add crawler upload target support for GuangYaPan in the admin UI.\n- Add drive branding, labels, metadata display, and docs/config examples for GuangYaPan.\n- Include a standalone GuangYaPan QR login helper script for manual credential acquisition.

Tests:\n- Add GuangYaPan driver, QR login, proxy, admin API, crawler upload target, fingerprint, cooldown, and form coverage.\n- Update rate-limit tests to assert that message-only throttling text no longer starts cooldowns.\n- Cover explicit HTTP status parsing through shared drive helper tests.
2026-06-14 15:44:50 +08:00
nianzhibai 9cc8e02bec feat: add sky theme and refresh themed UI
Add the sky theme across the frontend and backend theme APIs, including starfield assets and icon-only branding.

Refresh themed grid backgrounds, admin/login/sidebar styling, and theme-specific video/listing polish.
2026-06-14 11:53:07 +08:00
nianzhibai 139e63eef2 Update README to remove outdated features
Removed compatibility transcoding and video management features from the README.
2026-06-13 16:45:03 +08:00
104 changed files with 6603 additions and 2183 deletions
-3
View File
@@ -30,9 +30,6 @@ tmp/
# 91 爬虫脚本独立运行时的默认输出文件(backend 跑时会显式 --output 到 backend/data/spider91/,所以不会落在这里)
91porn_videos.json
91VideoSpider/91porn_videos.json
91VideoSpider/data/
91VideoSpider/__pycache__/
__pycache__/
*.pyc
-988
View File
@@ -1,988 +0,0 @@
#!/usr/bin/env python3
# -*- coding: utf-8 -*-
"""
91porn 视频爬虫脚本
===================
爬取 https://www.91porn.com/v.php?category=top&viewtype=basic 下的所有视频信息:
- 视频名称
- 封面图直链
- 视频直链 (MP4)
依赖安装:
pip install requests beautifulsoup4 lxml PySocks
使用方法:
# 作为 video-site-91 通用爬虫脚本运行(后台会自动这样调用)
python spider_91porn.py --job /path/to/job.json
# 全量爬取(默认行为,从 page=1 一直爬到末尾,写到 OUTPUT_FILE
python spider_91porn.py
# 只爬指定页(单页模式,手动调试用)
python spider_91porn.py --page 1 --output /tmp/spider91_page1.json
# 凑够 N 个新视频模式(backend 凌晨任务用)
python spider_91porn.py --target-new 15 --seen-viewkeys-file /tmp/seen.txt --output /tmp/new.json
CLI 参数:
--job FILE crawler.v1 job JSON 路径;后台爬虫管理会使用此模式
--page N 只爬第 N 页,配合 --output 用于手动调试
--target-new N 从 page 1 起翻页直到凑够 N 个新视频(不在 seen 列表里的)
--seen-viewkeys-file FILE 每行一个已知 viewkey 或 mp4 源 ID,命中即跳过;与 --target-new 配合使用
--output FILE 输出 JSON 路径,覆盖默认的 OUTPUT_FILE
--no-resume 禁用断点续爬(单页/target-new 模式下自动禁用)
--quiet 压缩日志,每条视频只输出一行
-h / --help 帮助
配置说明 (编辑脚本内 "配置区域"):
- MIN_PAGE_DELAY / MAX_PAGE_DELAY : 列表页请求间隔 (默认 3-6 秒)
- MIN_DETAIL_DELAY / MAX_DETAIL_DELAY : 详情页请求间隔 (默认 2-5 秒)
- MAX_PAGES : 限制最大爬取页数 (None=不限, 如 5=只爬前5页)
- OUTPUT_FILE : 输出文件名
输出格式 (JSON):
--job 模式下 stdout 输出 crawler.v1 JSON Lines,日志全部写到 stderr。
手动运行模式仍会写传统 JSON 文件:
{
"videos": [
{
"title": "视频标题",
"thumb_url": "https://...thumb/xxxx.jpg",
"video_url": "https://...mp43/xxxx.mp4?st=...",
"viewkey": "abc123...",
"source_id": "xxxx",
"detail_url": "https://...view_video.php?viewkey=..."
},
...
]
}
注意:
1. 视频直链包含时效性token (e参数为过期时间戳),会过期,需定期重新爬取
2. 脚本已内置随机延时,请勿移除,避免对服务器造成压力
3. 网站有Cloudflare保护,如遇到403/5xx错误,可能需要使用带cookie的session
4. 本脚本仅供学习交流,请遵守当地法律法规
作者: OpenCode
日期: 2026-05-22
"""
import argparse
import requests
import re
import time
import random
import json
import os
import socket
import sys
import html
from urllib.parse import urljoin, unquote, urlparse
from datetime import datetime
try:
from bs4 import BeautifulSoup
except ImportError:
print("错误: 缺少依赖库 beautifulsoup4", file=sys.stderr)
print("请运行: pip install beautifulsoup4 lxml", file=sys.stderr)
sys.exit(1)
def prefer_ipv4_for_plain_socks5_proxy():
"""PySocks may pick IPv6 first for socks5://; some SOCKS5 servers only accept IPv4."""
proxy_envs = (
os.environ.get("HTTPS_PROXY", ""),
os.environ.get("HTTP_PROXY", ""),
os.environ.get("https_proxy", ""),
os.environ.get("http_proxy", ""),
)
uses_plain_socks5 = any(v.strip().lower().startswith("socks5://") for v in proxy_envs)
if not uses_plain_socks5 or getattr(socket, "_spider91_ipv4_first", False):
return
original_getaddrinfo = socket.getaddrinfo
def getaddrinfo_ipv4_first(*args, **kwargs):
infos = original_getaddrinfo(*args, **kwargs)
return sorted(infos, key=lambda info: 0 if info[0] == socket.AF_INET else 1)
socket.getaddrinfo = getaddrinfo_ipv4_first
socket._spider91_ipv4_first = True
# ===================== 配置区域 =====================
BASE_URL = "https://www.91porn.com/v.php"
LIST_PARAMS = {
"category": "top",
"viewtype": "basic"
}
# 请求头 (模拟真实浏览器)
HEADERS = {
"User-Agent": (
"Mozilla/5.0 (Windows NT 10.0; Win64; x64) "
"AppleWebKit/537.36 (KHTML, like Gecko) "
"Chrome/125.0.0.0 Safari/537.36"
),
"Accept": (
"text/html,application/xhtml+xml,application/xml;"
"q=0.9,image/avif,image/webp,image/apng,*/*;"
"q=0.8,application/signed-exchange;v=b3;q=0.7"
),
"Accept-Language": "zh-CN,zh;q=0.9",
# 注意: 不要包含 "br" (brotli),除非安装了 brotli 库
# "Accept-Encoding": "gzip, deflate, br",
"Connection": "keep-alive",
"Upgrade-Insecure-Requests": "1",
"Sec-Fetch-Dest": "document",
"Sec-Fetch-Mode": "navigate",
"Sec-Fetch-Site": "none",
"Sec-Fetch-User": "?1",
}
# 延时配置 (秒) - 控制爬取频率,避免被封
MIN_PAGE_DELAY = 3.0 # 列表页之间最小延时
MAX_PAGE_DELAY = 6.0 # 列表页之间最大延时
MIN_DETAIL_DELAY = 2.0 # 详情页之间最小延时
MAX_DETAIL_DELAY = 5.0 # 详情页之间最大延时
# 重试配置
MAX_RETRIES = 3
RETRY_DELAY = 5.0
# 输出配置
OUTPUT_FILE = "91porn_videos.json"
MAX_PAGES = None # 设置为 None 爬取所有页,或设置整数如 5 只爬前5页
RESUME = True # 是否跳过输出文件中已存在的 viewkey (断点续爬)
MAX_EMPTY_PAGES = 2 # 连续空页数达到此值时停止爬取
CRAWLER_NAME = "91Porn"
CRAWLER_PROTOCOL = "crawler.v1"
# ===================================================
def crawler_source_id(raw: str) -> str:
"""Return a backend-safe source_id, preserving existing numeric 91 IDs."""
value = str(raw or "").strip()
if not value:
return ""
safe = re.sub(r"[^A-Za-z0-9_.-]+", "_", value).strip("._-")
return safe[:160]
def write_jsonl(event: dict):
print(json.dumps(event, ensure_ascii=False), flush=True)
class Porn91Spider:
def __init__(
self,
output_file: str = None,
start_page: int = 1,
max_pages: int = None,
resume: bool = None,
max_empty_pages: int = None,
quiet: bool = False,
target_new: int = None,
seen_viewkeys: list = None,
stream_output: bool = False,
stream_protocol: str = "legacy",
):
"""
构造函数。所有参数都有默认值,等同于使用脚本顶部的全局配置。
backend 调用时会传 output_file/seen_viewkeys/target_new,等价于:
"从第 1 页开始爬,跳过 seen_viewkeys 里的视频,凑够 target_new 个新视频后停止"
stream_output=True 时(backend 流水线用):
- 每凑齐一个 video 直链就把该 entry 作为一行 JSON 写到 stdout 并 flush
便于上层(Go crawler)边读边下载,不再等所有详情页处理完。
- 所有日志改走 stderr,避免与 stdout JSONL 流混合。
- --output 仍生效,作为离线归档用(脚本退出时一次性写完整 JSON)。
"""
self.session = requests.Session()
self.session.headers.update(HEADERS)
# 91porn 没有固定 mode cookie 时,详情页首次请求可能返回与列表卡片
# 不一致的视频源;固定桌面模式让列表页和详情页解析保持一致。
self.session.cookies.set("mode", "d")
# 解析后的实际配置;优先使用构造参数,回退到模块级配置
self.output_file = output_file if output_file is not None else OUTPUT_FILE
self.start_page = max(1, int(start_page or 1))
# max_pages=None 表示不限制;max_pages=N 表示从 start_page 起爬 N 页
self.max_pages = max_pages if max_pages is None or max_pages > 0 else None
# resume 默认跟模块配置;单页模式下调用方应该显式传 False
self.resume = RESUME if resume is None else bool(resume)
self.max_empty_pages = (
MAX_EMPTY_PAGES if max_empty_pages is None else int(max_empty_pages)
)
# target_new 是 backend 触发时的核心模式:累计处理这么多新源视频后退出。
self.target_new = target_new if target_new and target_new > 0 else None
self.quiet = bool(quiet)
# stream_output:每解析出一个 video 直链立即输出一行 JSON 到 stdout
# (配合 backend Go 端 bufio.Scanner 实时消费,下载一个就开始下一个)。
# 开启后所有 log 都走 stderr。
self.stream_output = bool(stream_output)
self.stream_protocol = stream_protocol or "legacy"
# 添加重试适配器
try:
from requests.adapters import HTTPAdapter
from urllib3.util.retry import Retry
retry_strategy = Retry(
total=MAX_RETRIES,
backoff_factor=1,
status_forcelist=[429, 500, 502, 503, 504],
)
adapter = HTTPAdapter(max_retries=retry_strategy)
self.session.mount("https://", adapter)
self.session.mount("http://", adapter)
except ImportError:
pass # urllib3 版本可能较低
self.results = []
self.pages_crawled = 0
self.processed_videos = 0
self.skipped_videos = 0
self.failed_videos = 0
self.skip_viewkeys = set()
# backend 通过 --seen-viewkeys-file 传进来一批已入库的历史 ID。
# 兼容旧名:文件里可能是 viewkey,也可能是新逻辑使用的 mp4 源 ID。
if seen_viewkeys:
for vk in seen_viewkeys:
if not vk:
continue
vk = vk.strip()
if vk:
self.skip_viewkeys.add(vk)
# 断点续爬:加载已有结果,跳过已处理的 viewkey
if self.resume and os.path.exists(self.output_file):
try:
with open(self.output_file, 'r', encoding='utf-8') as f:
existing_data = json.load(f)
existing_videos = existing_data.get('videos', [])
self.results = existing_videos
for v in existing_videos:
vk = v.get('viewkey', '')
if vk:
self.skip_viewkeys.add(vk)
self.processed_videos = existing_data.get('successful', 0)
self.failed_videos = existing_data.get('failed', 0)
self.log(f"加载已有数据: {len(self.results)} 个视频, 将跳过已处理项")
except Exception:
pass
def log(self, message: str):
"""带时间戳的日志输出。stream_output 模式下走 stderr,避免污染 stdout JSONL。"""
timestamp = datetime.now().strftime("%Y-%m-%d %H:%M:%S")
line = f"[{timestamp}] {message}"
if self.stream_output:
print(line, file=sys.stderr, flush=True)
else:
print(line)
def emit_stream_video(self, video: dict):
"""stream_output 模式下把单条 video entry 作为一行 JSON 写到 stdout 并立即刷盘。
Go 端 bufio.Scanner 按行读取,每收到一行就立即下载视频和封面。"""
if not self.stream_output:
return
try:
if self.stream_protocol == "crawler.v1":
source_id = crawler_source_id(video.get("source_id") or video.get("viewkey") or "")
item = {
"title": video.get("title") or "",
"detail_url": video.get("detail_url") or "",
"author": "91porn",
"tags": ["91porn"],
"media_url": video.get("video_url") or "",
"thumbnail_url": video.get("thumb_url") or "",
"headers": {
"Referer": video.get("detail_url") or BASE_URL,
},
}
if source_id:
item["source_id"] = source_id
event = {
"type": "item",
"item": item,
}
write_jsonl(event)
else:
print(json.dumps(video, ensure_ascii=False), flush=True)
except Exception as e:
# stdout 异常基本只在管道断开时发生(消费方进程死了);
# 写到 stderr 让 backend 看到,然后让 crawl 循环自己 break。
print(f"[stream] emit failed: {e}", file=sys.stderr, flush=True)
def random_sleep(self, min_sec: float, max_sec: float):
"""随机延时,模拟人类行为"""
delay = random.uniform(min_sec, max_sec)
if not self.quiet:
self.log(f" 随机延时 {delay:.2f} 秒...")
time.sleep(delay)
def fetch_page(self, url: str, description: str = "", referer: str = "") -> str:
"""
获取页面HTML内容,带错误处理和重试
"""
headers_extra = {}
if referer:
headers_extra["Referer"] = referer
for attempt in range(1, MAX_RETRIES + 1):
try:
self.log(f"正在请求: {description or url} (尝试 {attempt}/{MAX_RETRIES})")
response = self.session.get(url, timeout=30, headers=headers_extra)
# 检查是否被Cloudflare拦截 (需在 raise_for_status 之前)
if response.status_code == 403:
self.log("警告: 收到 403 Forbidden,可能被拦截")
if attempt < MAX_RETRIES:
self.random_sleep(RETRY_DELAY, RETRY_DELAY + 3)
continue
return ""
response.raise_for_status()
# 优先使用 content.decode('utf-8'),避免 requests 编码检测问题
try:
html_content = response.content.decode('utf-8', errors='replace')
except Exception:
html_content = response.text
# Cloudflare 挑战检测:如果页面主要内容只有挑战页面,而非正常内容
# 注意:网站本身会加载 challenge-platform 脚本,所以不能仅凭此判断
is_cf_challenge = (
"Just a moment" in html_content and
len(html_content) < 8000
)
if is_cf_challenge:
self.log("警告: 页面被Cloudflare挑战拦截,需要浏览器环境或正确cookie")
if attempt < MAX_RETRIES:
self.random_sleep(RETRY_DELAY, RETRY_DELAY + 5)
continue
return ""
return html_content
except requests.exceptions.HTTPError as e:
self.log(f"HTTP错误: {e}")
if attempt < MAX_RETRIES:
self.random_sleep(RETRY_DELAY, RETRY_DELAY + 3)
else:
return ""
except requests.exceptions.RequestException as e:
self.log(f"请求失败: {e}")
if attempt < MAX_RETRIES:
self.random_sleep(RETRY_DELAY, RETRY_DELAY + 3)
else:
self.log(f"达到最大重试次数,放弃: {url}")
return ""
return ""
def parse_list_page(self, html: str) -> list:
"""
解析列表页,提取视频基本信息
返回: [{title, detail_url, thumb_url, viewkey}, ...]
"""
videos = []
soup = BeautifulSoup(html, 'lxml')
# 只解析正常视频卡片。页面中还混有 col-lg-8 的异常大卡片,里面的标题、
# thumb、detail URL 会串到其它视频,不能作为入库来源。
video_cards = soup.select('div.col-xs-12.col-sm-4.col-md-3.col-lg-3')
seen_cards = set()
for card in video_cards:
link = card.find('a', href=re.compile(r'view_video\.php\?viewkey='))
if not link:
continue
href = link.get('href', '')
if not href:
continue
# 提取 viewkey
match = re.search(r'viewkey=([^&]+)', href)
if not match:
continue
viewkey = match.group(1)
detail_url = urljoin(BASE_URL, href)
# 提取标题
title = self._extract_title(link)
# 提取列表卡片来源 ID 和封面图 URL
thumb_url = ""
source_id = ""
overlay = link.find(id=re.compile(r'^playvthumb_\d+$'))
if overlay:
source_id = overlay.get('id', '').rsplit('_', 1)[-1]
img = link.find('img', class_=re.compile(r'img-responsive'))
if img:
thumb_url = img.get('src', '') or img.get('data-original', '')
if thumb_url:
thumb_url = urljoin(BASE_URL, thumb_url)
if not source_id and thumb_url:
source_id = self._extract_thumb_source_id(thumb_url)
card_key = source_id or detail_url
if card_key in seen_cards:
continue
seen_cards.add(card_key)
videos.append({
"title": title,
"detail_url": detail_url,
"thumb_url": thumb_url,
"viewkey": viewkey,
"source_id": source_id
})
return videos
def _extract_title(self, link) -> str:
"""
从视频链接标签中提取并清理标题
"""
# 优先从 span.video-title 获取 (已渲染的干净标题)
title_el = link.find('span', class_=re.compile(r'video-title'))
if title_el:
title = title_el.get_text(strip=True)
if title:
return html.unescape(title)
# 备用: 从 link 的 title 属性提取
title = link.get('title', '').strip()
if title:
return html.unescape(title)
# 最后手段: 从链接文本提取并清理前缀
text = link.get_text(separator=' ', strip=True)
# 去掉前缀: "HD" / "91" / 时间戳 "HH:MM:SS"
text = re.sub(r'^(HD\s+|91\s+)?\d{2}:\d{2}:\d{2}\s*', '', text)
text = re.sub(r'\s+', ' ', text).strip()
return html.unescape(text)[:120]
def parse_detail_page(self, html: str) -> dict:
"""
解析详情页,提取视频直链
返回: {"video_url": "...", "source_id": "...", "title": "..."} 或空字典
"""
result = {}
if not html:
return result
title = self._extract_detail_title(html)
if title:
result["title"] = title
# 方法1: 解码 strencode2 (主要方式, 页面通过 document.write 动态写入 video 标签)
# 格式: document.write(strencode2("%3c%73%6f..."));
strencode_match = re.search(r'strencode2\(["\']([^"\']+)["\']\)', html)
if strencode_match:
encoded = strencode_match.group(1)
try:
# strencode2 在JS中等价于 unescape / decodeURIComponent
decoded = unquote(encoded)
# 从解码后的 HTML 片段中提取 src
src_match = re.search(r"src=['\"]([^'\"]+)['\"]", decoded)
if src_match:
video_url = src_match.group(1)
# 规范化双斜杠 (如 https://host//path -> https://host/path)
video_url = re.sub(r'(https?://[^/]+)//+', r'\1/', video_url)
result["video_url"] = video_url
result["source_id"] = self._extract_source_id(video_url)
return result
except Exception as e:
self.log(f" 解码 strencode2 失败: {e}")
# 方法2: 通用正则匹配页面中的 mp4 链接 (备用, 过滤广告)
mp4_match = re.search(
r'https?://[^\s"\'<>]+\.mp4[^\s"\'<>]*',
html
)
if mp4_match:
url = mp4_match.group(0)
if 'kwai' not in url and 'ad-' not in url.lower():
result["video_url"] = url
result["source_id"] = self._extract_source_id(url)
return result
return result
def _extract_detail_title(self, html_text: str) -> str:
soup = BeautifulSoup(html_text, 'lxml')
title_el = soup.find('title')
if not title_el:
return ""
title = title_el.get_text(" ", strip=True)
title = re.sub(r'\s*-\s*91porn.*$', '', title, flags=re.IGNORECASE).strip()
return html.unescape(title)[:160]
def _extract_source_id(self, video_url: str) -> str:
path = urlparse(video_url or "").path
name = os.path.basename(path)
stem, ext = os.path.splitext(name)
if ext.lower() not in {".mp4", ".m4v", ".mov", ".webm", ".mkv", ".avi"}:
return ""
source_id = re.sub(r'[^0-9]+', '', stem)
if not source_id or source_id != stem:
return ""
return source_id
def _extract_thumb_source_id(self, thumb_url: str) -> str:
path = urlparse(thumb_url or "").path
match = re.search(r'/thumb/(\d+)\.[A-Za-z0-9]+$', path)
return match.group(1) if match else ""
def _thumb_url_for_source(self, thumb_url: str, source_id: str) -> str:
if not thumb_url or not source_id:
return thumb_url
parsed = urlparse(thumb_url)
match = re.search(r'/thumb/([^/?#]+)\.[A-Za-z0-9]+$', parsed.path)
if not match:
return thumb_url
current = match.group(1)
if current == source_id:
return thumb_url
path = re.sub(
r'/thumb/[^/?#]+\.[A-Za-z0-9]+$',
f'/thumb/{source_id}.jpg',
parsed.path,
)
return parsed._replace(path=path, query="", fragment="").geturl()
def crawl(self):
"""
主爬取流程。停止条件(任一满足即停):
- 达到 max_pages 配置
- 连续 max_empty_pages 页都没有视频
- target_new 模式下,已经累计处理 target_new 个新视频
"""
self.log("=" * 60)
self.log("91porn 视频爬虫启动")
self.log("=" * 60)
self.log(f"配置: 列表页延时 {MIN_PAGE_DELAY}-{MAX_PAGE_DELAY}s, 详情页延时 {MIN_DETAIL_DELAY}-{MAX_DETAIL_DELAY}s")
self.log(f"配置: 最大重试 {MAX_RETRIES} 次, 连续空页上限 {self.max_empty_pages}")
self.log(f"配置: 起始页 {self.start_page}, 最大爬取页数 {self.max_pages if self.max_pages else '不限'}")
if self.target_new:
self.log(f"配置: 目标新增视频数 {self.target_new}")
self.log(f"配置: 输出文件 {os.path.abspath(self.output_file)}")
if self.skip_viewkeys:
self.log(f"配置: 已跳过 {len(self.skip_viewkeys)} 个已知 viewkey")
self.log("")
page_num = self.start_page
consecutive_empty = 0
crawled_in_session = 0
while True:
if self.max_pages is not None and crawled_in_session >= self.max_pages:
self.log(f"达到配置的页数上限 {self.max_pages},停止")
break
if consecutive_empty >= self.max_empty_pages:
self.log(f"连续 {self.max_empty_pages} 页无结果,已达到末尾")
break
if self.target_new is not None and self.processed_videos >= self.target_new:
self.log(f"已累计 {self.processed_videos} 个新视频,达到目标 {self.target_new},停止")
break
if page_num == 1:
page_url = f"{BASE_URL}?category=top&viewtype=basic"
else:
page_url = f"{BASE_URL}?category=top&viewtype=basic&page={page_num}"
if crawled_in_session > 0:
self.log("")
self.random_sleep(MIN_PAGE_DELAY, MAX_PAGE_DELAY)
self.log(f"[页 {page_num}] 请求: {page_url}")
page_html = self.fetch_page(page_url, f"列表页 第{page_num}")
if not page_html:
self.log(f"[页 {page_num}] 获取失败,跳过")
consecutive_empty += 1
page_num += 1
crawled_in_session += 1
continue
page_videos = self.parse_list_page(page_html)
# 判断页面是否真的没有视频(而非全部已处理)
if not page_videos:
self.log(f"[页 {page_num}] 页面无视频,可能已到末尾")
consecutive_empty += 1
page_num += 1
crawled_in_session += 1
continue
consecutive_empty = 0
# 过滤已处理的 viewkey,只保留新视频
new_videos = [v for v in page_videos if v['viewkey'] not in self.skip_viewkeys]
skipped_on_page = len(page_videos) - len(new_videos)
if skipped_on_page > 0:
self.log(f"[页 {page_num}] 发现 {len(page_videos)} 个链接, 其中 {skipped_on_page} 个已处理, {len(new_videos)} 个新视频")
else:
self.log(f"[页 {page_num}] 发现 {len(new_videos)} 个视频")
if new_videos:
self._process_video_list(new_videos, referer=page_url)
self.pages_crawled += 1
page_num += 1
crawled_in_session += 1
self._save_results()
self._print_summary()
def _process_video_list(self, videos: list, referer: str = ""):
"""
处理一批视频列表,逐个获取详情页
"""
for idx, video in enumerate(videos, 1):
# target_new 模式下,凑够后立即停止,不再请求详情页
if self.target_new is not None and self.processed_videos >= self.target_new:
return
# 跳过已处理的 viewkey (断点续爬)
if video['viewkey'] in self.skip_viewkeys:
self.log(f" [SKIP] 已处理过: {video['viewkey']}")
self.skipped_videos += 1
continue
self.log(f" 处理视频 {idx}/{len(videos)}: {video['title'][:40]}...")
# 延时控制 (同一批次内第一个视频不延时)
if idx > 1:
self.random_sleep(MIN_DETAIL_DELAY, MAX_DETAIL_DELAY)
# 获取详情页
detail_html = self.fetch_page(video['detail_url'], f"详情页 viewkey={video['viewkey']}", referer=referer)
if not detail_html:
self.log(f" [FAIL] 详情页获取失败: {video['viewkey']}")
video["video_url"] = ""
self.results.append(video)
self.skip_viewkeys.add(video['viewkey'])
self.failed_videos += 1
continue
# 解析视频直链
detail_info = self.parse_detail_page(detail_html)
if detail_info.get("video_url"):
video["video_url"] = detail_info["video_url"]
if detail_info.get("title"):
video["title"] = detail_info["title"]
list_source_id = video.get("source_id", "")
detail_source_id = detail_info.get("source_id", "")
if list_source_id and detail_source_id and list_source_id != detail_source_id:
self.log(
f" [FAIL] 详情页视频源不匹配: list_source_id={list_source_id} "
f"detail_source_id={detail_source_id} viewkey={video['viewkey']}"
)
self.failed_videos += 1
self.skip_viewkeys.add(video['viewkey'])
continue
if not list_source_id and detail_source_id:
video["source_id"] = detail_source_id
if video.get("source_id"):
video["thumb_url"] = self._thumb_url_for_source(
video.get("thumb_url", ""),
video["source_id"],
)
if video["source_id"] in self.skip_viewkeys:
self.log(f" [SKIP] 已处理过 source_id: {video['source_id']}")
self.skipped_videos += 1
continue
self.results.append(video)
self.skip_viewkeys.add(video['viewkey'])
if video.get("source_id"):
self.skip_viewkeys.add(video["source_id"])
self.processed_videos += 1
self.log(f" [OK] 成功提取视频直链")
# 流式:立刻把这条 entry 交给 Go 端开始下载,不等本批余下视频
self.emit_stream_video(video)
else:
self.log(f" [FAIL] 未找到视频直链: {video['viewkey']}")
video["video_url"] = ""
self.results.append(video)
self.skip_viewkeys.add(video['viewkey'])
self.failed_videos += 1
def _save_results(self):
"""
保存结果到JSON文件
"""
output_data = {
"crawl_time": datetime.now().isoformat(),
"source_url": BASE_URL,
"pages_crawled": self.pages_crawled,
"total_videos": len(self.results),
"successful": self.processed_videos,
"skipped": self.skipped_videos,
"failed": self.failed_videos,
"videos": self.results
}
try:
# 保证父目录存在;写入临时文件后原子 rename,避免读到半截 JSON
out_path = self.output_file
parent = os.path.dirname(os.path.abspath(out_path))
if parent:
os.makedirs(parent, exist_ok=True)
tmp_path = out_path + ".part"
with open(tmp_path, 'w', encoding='utf-8') as f:
json.dump(output_data, f, ensure_ascii=False, indent=2)
os.replace(tmp_path, out_path)
self.log(f"结果已保存到: {os.path.abspath(out_path)}")
except Exception as e:
self.log(f"保存文件失败: {e}")
# 尝试输出到控制台作为备份
backup_out = sys.stderr if self.stream_output else sys.stdout
print("\n--- 备份输出 ---", file=backup_out, flush=True)
print(json.dumps(output_data, ensure_ascii=False, indent=2), file=backup_out, flush=True)
def _print_summary(self):
"""
打印爬取摘要
"""
self.log("")
self.log("=" * 60)
self.log("爬取完成!")
self.log("=" * 60)
self.log(f"爬取页数: {self.pages_crawled}")
self.log(f"总视频数: {len(self.results)}")
self.log(f"成功提取直链: {self.processed_videos}")
self.log(f"跳过(已处理): {self.skipped_videos}")
self.log(f"失败/缺失直链: {self.failed_videos}")
self.log(f"输出文件: {os.path.abspath(self.output_file)}")
self.log("=" * 60)
def print_help():
print("""
================================================
91porn 视频爬虫 v1.0
================================================
本脚本将爬取 91porn "本月最热" 分类下的所有视频信息:
- 视频名称
- 封面图直链
- 视频直链 (MP4)
依赖安装:
pip install requests beautifulsoup4 lxml PySocks
使用方法:
python spider_91porn.py
配置说明 (编辑脚本内 "配置区域"):
MIN_PAGE_DELAY / MAX_PAGE_DELAY : 列表页请求间隔 (默认 3-6 秒)
MIN_DETAIL_DELAY / MAX_DETAIL_DELAY : 详情页请求间隔 (默认 2-5 秒)
MAX_PAGES : 限制最大爬取页数 (None=不限, 如 5=只爬前5页)
OUTPUT_FILE : 输出文件名 (默认 91porn_videos.json)
按 Ctrl+C 可随时中断并保存已爬取的数据
注意:
1. 视频直链包含时效性token,会过期,需定期重新爬取
2. 脚本已内置随机延时,请勿移除,避免对服务器造成压力
3. 如遇到Cloudflare拦截,需要先通过浏览器获取Cookie
4. 本脚本仅供学习交流,请遵守当地法律法规
================================================
""")
def run_job(job_path: str):
"""Run as a crawler.v1 script plugin.
The Go host passes a job JSON file and expects stdout JSONL events. Logs go
to stderr so stdout stays machine-readable.
"""
with open(job_path, "r", encoding="utf-8") as f:
job = json.load(f)
if job.get("protocol") != CRAWLER_PROTOCOL:
raise ValueError(f"unsupported crawler protocol: {job.get('protocol')!r}")
if job.get("mode") not in ("", None, "crawl"):
raise ValueError(f"unsupported crawler mode: {job.get('mode')!r}")
try:
target_new = int(job.get("target_new") or 15)
except (TypeError, ValueError):
target_new = 15
if target_new <= 0:
target_new = 15
seen_file = job.get("seen_source_ids_file") or ""
output_dir = job.get("output_dir") or os.getcwd()
run_id = job.get("run_id") or datetime.utcnow().strftime("%Y%m%dT%H%M%SZ")
os.makedirs(output_dir, exist_ok=True)
output_file = os.path.join(output_dir, f"spider91-{run_id}.json")
network = job.get("network") if isinstance(job.get("network"), dict) else {}
proxy_url = str(network.get("proxy_url") or "").strip()
if proxy_url:
os.environ["HTTP_PROXY"] = proxy_url
os.environ["HTTPS_PROXY"] = proxy_url
os.environ["http_proxy"] = proxy_url
os.environ["https_proxy"] = proxy_url
os.environ["NO_PROXY"] = ""
os.environ["no_proxy"] = ""
seen_viewkeys = []
if seen_file:
try:
with open(seen_file, "r", encoding="utf-8") as f:
for line in f:
line = line.strip()
if line:
seen_viewkeys.append(line)
except FileNotFoundError:
print(f"警告: seen_source_ids_file 不存在: {seen_file}", file=sys.stderr, flush=True)
except Exception as e:
print(f"警告: 读取 seen_source_ids_file 失败: {e}", file=sys.stderr, flush=True)
prefer_ipv4_for_plain_socks5_proxy()
spider = Porn91Spider(
output_file=output_file,
start_page=1,
max_pages=None,
resume=False,
quiet=True,
target_new=target_new,
seen_viewkeys=seen_viewkeys,
stream_output=True,
stream_protocol="crawler.v1",
)
try:
spider.crawl()
done = {
"type": "done",
"stats": {
"emitted": spider.processed_videos,
"failed": spider.failed_videos,
"skipped": spider.skipped_videos,
},
}
write_jsonl(done)
except KeyboardInterrupt:
spider.log("\n用户中断,正在保存已爬取的数据...")
spider._save_results()
raise
def main():
if len(sys.argv) > 1 and sys.argv[1] in ('-h', '--help', 'help'):
print_help()
return
parser = argparse.ArgumentParser(
prog="spider_91porn.py",
description="91porn 视频元数据爬虫",
add_help=False, # 让 -h/--help 走 print_help() 中文版本
)
parser.add_argument("--page", type=int, default=None,
help="只爬指定页(单页模式,配合 --output 用于定时任务)")
parser.add_argument("--output", type=str, default=None,
help="输出 JSON 路径,覆盖默认 OUTPUT_FILE")
parser.add_argument("--max-pages", type=int, default=None,
help="单页模式下,从 --page 起最多再爬几页(默认 1)")
parser.add_argument("--no-resume", action="store_true",
help="禁用断点续爬(单页模式默认禁用)")
parser.add_argument("--quiet", action="store_true",
help="压缩日志,每条视频只输出关键事件")
parser.add_argument("--target-new", type=int, default=None,
help="目标新增模式:从 page 1 起翻页直到累计处理这么多新源视频后停止(backend 凌晨任务用)")
parser.add_argument("--seen-viewkeys-file", type=str, default=None,
help="文件路径,每行一个已处理过的 viewkey 或 mp4 源 ID;脚本会跳过这些视频")
parser.add_argument("--stream-output", action="store_true",
help="流式模式:每解析一条视频直链就立即把它作为一行 JSON 写到 stdout 并 flush"
"日志改走 stderr。配合 backend 边读边下载使用。")
parser.add_argument("--job", type=str, default=None,
help="crawler.v1 job JSON 路径;作为通用脚本爬虫运行。")
args, _ = parser.parse_known_args()
if args.job:
run_job(args.job)
return
cli_out = sys.stderr if args.stream_output else sys.stdout
prefer_ipv4_for_plain_socks5_proxy()
print("""
================================================
91porn 视频爬虫启动中...
================================================
按 Ctrl+C 可随时中断并保存进度
""", file=cli_out)
# 加载已知 ID(来自 backend 的 catalog 已入库列表;兼容旧参数名)
seen_viewkeys = []
if args.seen_viewkeys_file:
try:
with open(args.seen_viewkeys_file, 'r', encoding='utf-8') as f:
for line in f:
line = line.strip()
if line:
seen_viewkeys.append(line)
except FileNotFoundError:
print(f"警告: --seen-viewkeys-file 不存在: {args.seen_viewkeys_file}", file=cli_out)
except Exception as e:
print(f"警告: 读取 --seen-viewkeys-file 失败: {e}", file=cli_out)
# 决定运行模式
if args.target_new is not None:
# 凑够 N 个新视频模式:从 page 1 起翻页,直到累计 target_new 个新视频
spider = Porn91Spider(
output_file=args.output,
start_page=1,
max_pages=None,
resume=False, # 凑够 N 模式靠 seen_viewkeys 去重,不读 OUTPUT_FILE
quiet=args.quiet,
target_new=args.target_new,
seen_viewkeys=seen_viewkeys,
stream_output=args.stream_output,
)
elif args.page is not None:
# 单页模式(保留作手动调试用):start_page=N, max_pages=1
start_page = max(1, args.page)
max_pages = args.max_pages if args.max_pages and args.max_pages > 0 else 1
spider = Porn91Spider(
output_file=args.output,
start_page=start_page,
max_pages=max_pages,
resume=False,
quiet=args.quiet,
seen_viewkeys=seen_viewkeys,
stream_output=args.stream_output,
)
else:
# 全量模式(向后兼容):从 page 1 起爬到末尾
spider = Porn91Spider(
output_file=args.output,
resume=False if args.no_resume else None,
quiet=args.quiet,
seen_viewkeys=seen_viewkeys,
stream_output=args.stream_output,
)
try:
spider.crawl()
except KeyboardInterrupt:
spider.log("\n用户中断,正在保存已爬取的数据...")
spider._save_results()
spider._print_summary()
sys.exit(0)
except Exception as e:
spider.log(f"发生未预料的错误: {e}")
import traceback
traceback.print_exc()
spider._save_results()
raise
if __name__ == "__main__":
main()
+2 -5
View File
@@ -20,14 +20,11 @@
## 功能特性
- **多后端支持** — 兼容 115 云盘、PikPak 云盘、123网盘、联通网盘、OneDrive、Google Drive 和本地存储
- **低带宽播放** — 115 云盘、PikPak 云盘、123网盘、联通网盘、OneDrive 支持302模式,在线播放视频时,不占用服务器带宽,播放体验不受服务器带宽影响;Google Drive 不支持302模式,走服务器中转,观看体验会受服务器带宽影响
- **多后端支持** — 兼容 115 云盘、PikPak 云盘、123网盘、联通网盘、光鸭网盘、OneDrive、Google Drive 和本地存储
- **低带宽播放** — 115 云盘、PikPak 云盘、123网盘、联通网盘、光鸭网盘、OneDrive 支持302模式,在线播放视频时,不占用服务器带宽,播放体验不受服务器带宽影响;Google Drive 不支持302模式,走服务器中转,观看体验会受服务器带宽影响
- **封面 & 预览片段** — 自动为每个视频生成封面图和预览片段,首页快速选片
- **爬虫脚本** — 项目支持导入自定义脚本,但是有一些规范,具体可以参考 [SpiderFor91](https://github.com/Just-Spider/SpiderFor91),项目不再内置任何爬虫脚本
- **短视频模式** — 一键切换抖音风格,沉浸刷片
- **兼容性转码** — 对 AVI、WMV、RMVB 等浏览器无法直接播放的视频,可在后台手动转码为 H.264 MP4
- **视频管理** — 删除或隐藏的视频进入黑名单,支持在后台移出黑名单后重新扫盘入库
---
## 预览图
+3 -1
View File
@@ -2,7 +2,7 @@
视频聚合站的 Go 后端。提供三件事:
1. 多家网盘统一抽象(夸克 / 115 / PikPak / 联通网盘 / OneDrive / Google Drive / 本地存储)
1. 多家网盘统一抽象(夸克 / 115 / PikPak / 联通网盘 / 光鸭网盘 / OneDrive / Google Drive / 本地存储)
2. 视频元数据目录(SQLite)+ 扫描 + 预览视频预生成
3. REST API(前台)+ 管理后台 + 直链代理
4. 标签池、视频隐藏、按网盘统计和详情页来源网盘类型展示能力
@@ -20,6 +20,7 @@ internal/
p115/ 115(壳子 + SheltonZhu/115driver
pikpak/ PikPak(自己实现,参考 OpenList pikpak
wopan/ 联通网盘(壳子 + OpenListTeam/wopan-sdk-go
guangyapan/ 光鸭网盘(参考 AList GuangYaPan
onedrive/ OneDriveOpenList 在线续期 + Microsoft Graph 文件接口)
googledrive/ Google DriveOpenList 在线续期 + Google Drive API;播放走后端代理)
localstorage/ 本地目录扫描(服务器已有视频目录)
@@ -108,6 +109,7 @@ go run ./cmd/server 后端 9192
| p115 | `cookie`(形如 `UID=...; CID=...; SEID=...; KID=...` |
| pikpak | `username`、`password`(token、验证码和设备 ID 由服务端自动处理并保存) |
| wopan | `access_token`、`refresh_token`,可选 `family_id` |
| guangyapan | 推荐后台扫码登录自动写入 `access_token`、`refresh_token`;也可手工填写 token;可选 `root_path` |
| onedrive | `refresh_token` |
| googledrive | 默认只需 `refresh_token`;自建 OAuth 客户端模式还需 `use_online_api=false`、`client_id`、`client_secret` |
| localstorage | `path`(服务器上的已有视频目录,如 `/mnt/videos` |
+306 -41
View File
@@ -26,6 +26,7 @@ import (
"github.com/video-site/backend/internal/config"
"github.com/video-site/backend/internal/drives"
"github.com/video-site/backend/internal/drives/googledrive"
"github.com/video-site/backend/internal/drives/guangyapan"
"github.com/video-site/backend/internal/drives/localstorage"
"github.com/video-site/backend/internal/drives/localupload"
"github.com/video-site/backend/internal/drives/onedrive"
@@ -213,6 +214,9 @@ func main() {
}
return app.scheduleScan(ctx, driveID)
},
OnCrawlerUploadRequested: func(driveID string) (bool, string) {
return app.scheduleManualCrawlerUploadMigration(ctx, driveID)
},
OnStopDriveTasks: func(driveID string) bool {
return app.stopDriveTasks(ctx, driveID)
},
@@ -246,6 +250,9 @@ func main() {
GetDriveGenerationStatuses: func() map[string]api.DriveGenerationStatuses {
return app.driveGenerationStatuses()
},
GetPreviewGenerationVideoIDs: func() map[string]bool {
return app.previewGenerationVideoIDs()
},
OnTeaserEnabledChanged: func(driveID string, enabled bool) {
// 从关到开时立刻补扫该盘 pending 预览视频,行为对齐旧的"全局开关从关到开"。
// 关闭分支不需要做事 —— 入队前会重新查 catalog,新的 enqueue 自然停。
@@ -352,13 +359,13 @@ type App struct {
// 串行化可以避免启动后台挂载和手动扫盘按需挂载同一个 drive 时重复创建 worker。
driveAttachMu sync.Mutex
// 全站主题("dark" | "pink"),从 DB 读
// 全站主题("dark" | "pink" | "sky"),从 DB 读
theme string
// 显式指定的 spider91 上传目标 drive ID。
// 空字符串表示本地保存不上传,不再自动挑选 pikpak/p115/p123/onedrive/wopan drive。
// 空字符串表示本地保存不上传,不再自动挑选 pikpak/p115/p123/onedrive/wopan/guangyapan drive。
spider91UploadDriveID string
// spider91Migrator 把 spider91 视频上传到目标 drivePikPak、115、123、OneDrive、Google Drive 或联通网盘)。
// spider91Migrator 把 spider91 视频上传到目标 drivePikPak、115、123、OneDrive、Google Drive、联通网盘或光鸭网盘)。
spider91Migrator spider91MigrationRunner
// nightlyRunner 是凌晨流水线调度器:每天 cron_hour 串行跑扫盘 → 91 爬虫 → 迁移。
@@ -401,8 +408,9 @@ type App struct {
}
type driveScanProgress struct {
Scanned int
Added int
Scanned int
Added int
CooldownUntil time.Time
}
type driveUploadProgress struct {
@@ -451,7 +459,7 @@ func (a *App) Theme() string {
// SetTheme 切换并持久化主题;未知值会返回错误。
func (a *App) SetTheme(ctx context.Context, theme string) error {
if theme != "dark" && theme != "pink" {
if theme != "dark" && theme != "pink" && theme != "sky" {
return fmt.Errorf("unsupported theme %q", theme)
}
a.mu.Lock()
@@ -470,7 +478,7 @@ func (a *App) loadTheme(ctx context.Context) {
a.mu.Unlock()
return
}
if v != "pink" && v != "dark" {
if v != "pink" && v != "dark" && v != "sky" {
v = "dark"
}
a.mu.Lock()
@@ -479,7 +487,7 @@ func (a *App) loadTheme(ctx context.Context) {
}
// Spider91UploadDriveID 返回当前配置的 spider91 上传目标 drive ID。
// 空字符串表示本地保存不上传;只有管理员显式选择 pikpak/p115/p123/onedrive/googledrive/wopan drive 时才迁移上传。
// 空字符串表示本地保存不上传;只有管理员显式选择 pikpak/p115/p123/onedrive/googledrive/wopan/guangyapan drive 时才迁移上传。
func (a *App) Spider91UploadDriveID() string {
a.mu.Lock()
explicit := a.spider91UploadDriveID
@@ -496,7 +504,7 @@ func (a *App) Spider91UploadDriveID() string {
// SetSpider91UploadDriveID 设置 spider91 上传目标 drive ID 并持久化。
// 接受空字符串(本地保存不上传)。
// 设置一个不存在或 kind 不是 pikpak / p115 / p123 / onedrive / googledrive / wopan 的 drive 会返回错误。
// 设置一个不存在或 kind 不是 pikpak / p115 / p123 / onedrive / googledrive / wopan / guangyapan 的 drive 会返回错误。
func (a *App) SetSpider91UploadDriveID(ctx context.Context, driveID string) error {
driveID = strings.TrimSpace(driveID)
if driveID != "" {
@@ -505,7 +513,7 @@ func (a *App) SetSpider91UploadDriveID(ctx context.Context, driveID string) erro
return fmt.Errorf("drive %q not found", driveID)
}
if !isSpider91UploadKind(d.Kind()) {
return fmt.Errorf("drive %q kind=%s, only pikpak, p115, p123, onedrive, googledrive or wopan can be spider91 upload target", driveID, d.Kind())
return fmt.Errorf("drive %q kind=%s, only pikpak, p115, p123, onedrive, googledrive, wopan or guangyapan can be spider91 upload target", driveID, d.Kind())
}
}
a.mu.Lock()
@@ -538,7 +546,7 @@ func formatOptionalRFC3339(t time.Time) string {
// isSpider91UploadKind 是 spider91 迁移目标盘的 allowlist。
// 与 spider91migrate.adaptUploadTarget 的支持范围保持一致。
func isSpider91UploadKind(kind string) bool {
return kind == "pikpak" || kind == "p115" || kind == "p123" || kind == "onedrive" || kind == "googledrive" || kind == "wopan"
return kind == "pikpak" || kind == "p115" || kind == "p123" || kind == "onedrive" || kind == "googledrive" || kind == "wopan" || kind == guangyapan.Kind
}
// loadSpider91UploadDriveID 从 DB 读上传目标 drive ID 设置;不存在时使用空串。
@@ -595,17 +603,25 @@ func (a *App) driveGenerationStatuses() map[string]api.DriveGenerationStatuses {
a.transcodeMu.Unlock()
out := make(map[string]api.DriveGenerationStatuses, len(scanningDrives)+len(previewWorkers)+len(thumbWorkers)+len(fingerprintWorkers)+len(uploadProgresses)+len(transcodeWorkers))
now := time.Now()
for id, running := range scanningDrives {
if !running {
continue
}
progress := scanProgresses[id]
state := "scanning"
if progress.CooldownUntil.After(now) {
state = "cooling"
}
status := out[id]
status.Scan = api.GenerationStatus{
State: "scanning",
State: state,
ScannedCount: progress.Scanned,
AddedCount: progress.Added,
}
if !progress.CooldownUntil.IsZero() {
status.Scan.CooldownUntil = progress.CooldownUntil.Format(time.RFC3339)
}
out[id] = status
}
for id, worker := range previewWorkers {
@@ -646,6 +662,23 @@ func (a *App) driveGenerationStatuses() map[string]api.DriveGenerationStatuses {
return out
}
func (a *App) previewGenerationVideoIDs() map[string]bool {
a.mu.Lock()
previewWorkers := make([]*preview.Worker, 0, len(a.workers))
for _, worker := range a.workers {
previewWorkers = append(previewWorkers, worker)
}
a.mu.Unlock()
out := make(map[string]bool)
for _, worker := range previewWorkers {
for _, id := range worker.ActiveVideoIDs() {
out[id] = true
}
}
return out
}
func (a *App) updateCrawlerUploadProgress(progress spider91migrate.UploadProgress) {
driveID := strings.TrimSpace(progress.DriveID)
if driveID == "" {
@@ -908,18 +941,20 @@ func (a *App) attachDriveUnlocked(ctx context.Context, d *catalog.Drive) error {
})
case "p115":
drv = p115.New(p115.Config{
ID: d.ID,
Cookie: d.Credentials["cookie"],
RootID: d.RootID,
ID: d.ID,
Cookie: d.Credentials["cookie"],
RootID: d.RootID,
UploadTempDir: a.uploadWorkDir("p115"),
})
case p123.Kind:
drv = p123.New(p123.Config{
ID: d.ID,
Username: d.Credentials["username"],
Password: d.Credentials["password"],
AccessToken: d.Credentials["access_token"],
Platform: d.Credentials["platform"],
RootID: d.RootID,
ID: d.ID,
Username: d.Credentials["username"],
Password: d.Credentials["password"],
AccessToken: d.Credentials["access_token"],
Platform: d.Credentials["platform"],
RootID: d.RootID,
UploadTempDir: a.uploadWorkDir(p123.Kind),
OnTokenUpdate: func(access string) {
if d.Credentials == nil {
d.Credentials = make(map[string]string)
@@ -940,6 +975,7 @@ func (a *App) attachDriveUnlocked(ctx context.Context, d *catalog.Drive) error {
DeviceID: d.Credentials["device_id"],
RootID: d.RootID,
DisableMediaLink: pikpak.ParseBoolDefault(d.Credentials["disable_media_link"], true),
UploadTempDir: a.uploadWorkDir("pikpak"),
OnTokenUpdate: func(access, refresh, captcha, deviceID string) {
d.Credentials["access_token"] = access
d.Credentials["refresh_token"] = refresh
@@ -950,17 +986,45 @@ func (a *App) attachDriveUnlocked(ctx context.Context, d *catalog.Drive) error {
})
case "wopan":
drv = wopan.New(wopan.Config{
ID: d.ID,
AccessToken: d.Credentials["access_token"],
RefreshToken: d.Credentials["refresh_token"],
FamilyID: d.Credentials["family_id"],
RootID: d.RootID,
ID: d.ID,
AccessToken: d.Credentials["access_token"],
RefreshToken: d.Credentials["refresh_token"],
FamilyID: d.Credentials["family_id"],
RootID: d.RootID,
UploadTempDir: a.uploadWorkDir("wopan"),
OnTokenUpdate: func(access, refresh string) {
d.Credentials["access_token"] = access
d.Credentials["refresh_token"] = refresh
_ = a.cat.UpsertDrive(ctx, d)
},
})
case guangyapan.Kind:
drv = guangyapan.New(guangyapan.Config{
ID: d.ID,
RootID: d.RootID,
RootPath: d.Credentials["root_path"],
PhoneNumber: d.Credentials["phone_number"],
CaptchaToken: d.Credentials["captcha_token"],
SendCode: parseBoolDefault(strings.TrimSpace(d.Credentials["send_code"]), false),
VerifyCode: d.Credentials["verify_code"],
VerificationID: d.Credentials["verification_id"],
AccessToken: d.Credentials["access_token"],
RefreshToken: d.Credentials["refresh_token"],
ClientID: d.Credentials["client_id"],
DeviceID: d.Credentials["device_id"],
PageSize: parseIntDefault(strings.TrimSpace(d.Credentials["page_size"]), 100),
OrderBy: parseIntDefault(strings.TrimSpace(d.Credentials["order_by"]), 3),
SortType: parseIntDefault(strings.TrimSpace(d.Credentials["sort_type"]), 1),
OnCredentialsUpdate: func(updated map[string]string) {
if d.Credentials == nil {
d.Credentials = make(map[string]string)
}
for k, v := range updated {
d.Credentials[k] = v
}
_ = a.cat.UpsertDrive(ctx, d)
},
})
case "onedrive":
drv = onedrive.New(onedrive.Config{
ID: d.ID,
@@ -1081,7 +1145,7 @@ func generationCooldownForDrive(drv drives.Drive) time.Duration {
return 0
}
switch strings.ToLower(drv.Kind()) {
case "wopan":
case "wopan", "guangyapan":
return 10 * time.Minute
}
return 0
@@ -1101,13 +1165,24 @@ func (a *App) localUploadDir() string {
return filepath.Join(filepath.Dir(a.cfg.Storage.LocalPreviewDir), "uploads")
}
func (a *App) uploadWorkDir(kind string) string {
if a == nil || a.cfg == nil || strings.TrimSpace(a.cfg.Storage.LocalPreviewDir) == "" {
return ""
}
kind = strings.Trim(strings.ToLower(strings.TrimSpace(kind)), string(filepath.Separator))
if kind == "" {
kind = "generic"
}
return filepath.Join(filepath.Dir(a.cfg.Storage.LocalPreviewDir), "upload-tmp", kind)
}
func fingerprintConfigForDrive(drv drives.Drive) fingerprint.Config {
cfg := fingerprint.Config{RateLimitCooldown: 5 * time.Minute}
if drv == nil {
return cfg
}
switch strings.ToLower(drv.Kind()) {
case "p115", "p123", "onedrive", "wopan":
case "p115", "p123", "onedrive", "wopan", "guangyapan":
cfg.RateLimitCooldown = 10 * time.Minute
case "pikpak":
cfg.RateLimitCooldown = 5 * time.Minute
@@ -1179,6 +1254,7 @@ func (a *App) attachScriptCrawler(d *catalog.Drive, drv *scriptcrawler.Driver) {
CommonThumbDir: a.commonThumbsDir(),
ProxyURL: proxyURL,
ConfigJSON: configJSON,
DisablePreview: !d.TeaserEnabled,
OnProgress: func(progress scriptcrawler.CrawlProgress) {
scanned := progress.Checked
if scanned < progress.TotalEntries {
@@ -1439,11 +1515,77 @@ func (a *App) updateDriveScanProgress(driveID string, scanned, added int) {
if a.scanProgress == nil {
a.scanProgress = make(map[string]driveScanProgress)
}
a.scanProgress[driveID] = driveScanProgress{Scanned: scanned, Added: added}
progress := a.scanProgress[driveID]
progress.Scanned = scanned
progress.Added = added
a.scanProgress[driveID] = progress
}
a.scanQueueMu.Unlock()
}
func (a *App) updateDriveScanCooldown(driveID string, until time.Time) {
driveID = strings.TrimSpace(driveID)
if driveID == "" {
return
}
a.scanQueueMu.Lock()
if a.scanQueued[driveID] {
if a.scanProgress == nil {
a.scanProgress = make(map[string]driveScanProgress)
}
progress := a.scanProgress[driveID]
progress.CooldownUntil = until
a.scanProgress[driveID] = progress
}
a.scanQueueMu.Unlock()
}
func (a *App) pauseDriveScanForRateLimit(ctx context.Context, driveID string, drv drives.Drive, err error) bool {
wait, ok := drives.RateLimitRetryAfter(err)
if !ok {
return false
}
if wait <= 0 {
wait = scanCooldownForDrive(drv)
}
if wait <= 0 {
wait = 5 * time.Minute
}
until := time.Now().Add(wait)
a.updateDriveScanCooldown(driveID, until)
log.Printf("[scan] drive=%s rate limited; cooling until=%s wait=%s: %v", driveID, until.Format(time.RFC3339), wait, err)
if !sleepDriveScanCooldown(ctx, wait) {
log.Printf("[scan] drive=%s cooldown canceled: %v", driveID, ctx.Err())
}
return true
}
func scanCooldownForDrive(drv drives.Drive) time.Duration {
if drv == nil {
return 5 * time.Minute
}
switch strings.ToLower(drv.Kind()) {
case "guangyapan":
return 10 * time.Minute
default:
return 5 * time.Minute
}
}
func sleepDriveScanCooldown(ctx context.Context, d time.Duration) bool {
if d <= 0 {
return true
}
timer := time.NewTimer(d)
defer timer.Stop()
select {
case <-ctx.Done():
return false
case <-timer.C:
return true
}
}
func (a *App) driveHasActiveWork(driveID string) bool {
driveID = strings.TrimSpace(driveID)
if driveID == "" {
@@ -1908,6 +2050,8 @@ func (a *App) runScanWithTaskContext(ctx context.Context, driveID string) {
if err != nil {
if errors.Is(err, context.Canceled) || errors.Is(err, context.DeadlineExceeded) {
log.Printf("[scan] drive=%s canceled: %v", driveID, err)
} else if a.pauseDriveScanForRateLimit(ctx, driveID, drv, err) {
return
} else {
log.Printf("[scan] drive=%s error: %v", driveID, err)
}
@@ -3038,18 +3182,7 @@ func (a *App) runScriptCrawlerCrawlWithTaskContext(ctx context.Context, driveID
driveID, res.TargetNew, res.CandidateBudget, res.TotalEntries, res.NewVideos, res.Skipped, res.Failed, res.SeenSnapshot)
}
if d.Credentials == nil {
d.Credentials = make(map[string]string)
}
d.Credentials["last_crawl_at"] = strconv.FormatInt(time.Now().Unix(), 10)
if runErr != nil {
d.Status = "error"
d.LastError = runErr.Error()
} else {
d.Status = "ok"
d.LastError = ""
}
if err := a.cat.UpsertDrive(ctx, d); err != nil {
if err := a.updateScriptCrawlerRunState(ctx, driveID, runErr); err != nil {
log.Printf("[scriptcrawler] drive=%s update last_crawl_at: %v", driveID, err)
}
if err := ctx.Err(); err != nil {
@@ -3067,6 +3200,25 @@ func (a *App) runScriptCrawlerCrawlWithTaskContext(ctx context.Context, driveID
return runErr == nil
}
func (a *App) updateScriptCrawlerRunState(ctx context.Context, driveID string, runErr error) error {
d, err := a.cat.GetDrive(ctx, driveID)
if err != nil {
return err
}
if d.Credentials == nil {
d.Credentials = make(map[string]string)
}
d.Credentials["last_crawl_at"] = strconv.FormatInt(time.Now().Unix(), 10)
if runErr != nil {
d.Status = "error"
d.LastError = runErr.Error()
} else {
d.Status = "ok"
d.LastError = ""
}
return a.cat.UpsertDrive(ctx, d)
}
func (a *App) runSpider91MigrationAfterManualCrawl(ctx context.Context, driveID string) {
a.runCrawlerMigrationAfterManualCrawl(ctx, driveID)
}
@@ -3151,6 +3303,108 @@ func (a *App) runCrawlerUploadMigrationAfterSave(ctx context.Context, driveID st
}
}
func (a *App) scheduleManualCrawlerUploadMigration(ctx context.Context, driveID string) (bool, string) {
driveID = strings.TrimSpace(driveID)
if driveID == "" || a == nil || a.cat == nil {
return false, "爬虫不存在"
}
if a.spider91Migrator == nil {
return false, "上传迁移器未初始化"
}
if a.driveHasActiveWork(driveID) {
return false, "当前爬虫有正在进行的任务,请稍后重试"
}
d, err := a.cat.GetDrive(ctx, driveID)
if err != nil || d == nil || d.Kind != scriptcrawler.Kind {
return false, "爬虫不存在"
}
targetDriveID := strings.TrimSpace(d.Credentials["upload_drive_id"])
if targetDriveID == "" {
return false, "请先配置上传网盘"
}
assets, err := a.cat.CountCrawlerAssets(ctx, driveID, crawlerCatalogVideoIDPrefixes(d))
if err != nil {
log.Printf("[scriptcrawler] drive=%s manual upload count assets: %v", driveID, err)
return false, "读取待上传视频失败"
}
if reason := crawlerUploadAssetBlockReason(d, assets); reason != "" {
return false, reason
}
if err := a.ensureDriveAttached(ctx, driveID); err != nil {
log.Printf("[scriptcrawler] drive=%s manual upload source attach: %v", driveID, err)
return false, "爬虫本地存储不可用"
}
if err := a.ensureDriveAttached(ctx, targetDriveID); err != nil {
log.Printf("[scriptcrawler] drive=%s manual upload target=%s attach: %v", driveID, targetDriveID, err)
return false, "上传网盘不可用:" + err.Error()
}
a.crawlerUploadMu.Lock()
if a.crawlerUploadRunning == nil {
a.crawlerUploadRunning = make(map[string]bool)
}
if a.crawlerUploadRunning[driveID] {
a.crawlerUploadMu.Unlock()
return false, "当前爬虫已有上传任务正在运行"
}
a.crawlerUploadRunning[driveID] = true
a.crawlerUploadMu.Unlock()
taskCtx, done := a.registerDriveTaskContext(ctx, driveID)
go func() {
defer func() {
done()
a.crawlerUploadMu.Lock()
delete(a.crawlerUploadRunning, driveID)
a.crawlerUploadMu.Unlock()
}()
a.runManualCrawlerUploadMigration(taskCtx, driveID, targetDriveID)
}()
return true, ""
}
func crawlerUploadAssetBlockReason(d *catalog.Drive, assets catalog.CrawlerAssetCounts) string {
if assets.Local <= 0 {
return "没有待上传的本地视频"
}
if assets.Fingerprint.Pending > 0 {
return "还有待生成的视频指纹"
}
if assets.Fingerprint.Failed > 0 {
return "存在指纹生成失败的视频,请先重试或处理失败项"
}
if d != nil && d.TeaserEnabled {
if assets.Teaser.Pending > 0 {
return "还有待生成的预览视频"
}
if assets.Teaser.Failed > 0 {
return "存在预览视频生成失败的视频,请先重试或处理失败项"
}
}
return ""
}
func crawlerCatalogVideoIDPrefixes(d *catalog.Drive) []string {
if d == nil {
return nil
}
return []string{
scriptcrawler.Kind + "-" + d.ID + "-",
spider91.Kind + "-" + d.ID + "-",
}
}
func (a *App) runManualCrawlerUploadMigration(ctx context.Context, driveID, targetDriveID string) {
if err := ctx.Err(); err != nil {
log.Printf("[scriptcrawler] drive=%s skip manual upload migration: %v", driveID, err)
return
}
log.Printf("[scriptcrawler] drive=%s running manual upload migration target=%s", driveID, targetDriveID)
if err := a.spider91Migrator.RunOnce(ctx); err != nil {
log.Printf("[scriptcrawler] drive=%s manual upload migration: %v", driveID, err)
}
}
func (a *App) runCrawlerMigrationAfterManualCrawl(ctx context.Context, driveID string) {
if err := ctx.Err(); err != nil {
log.Printf("[scriptcrawler] drive=%s skip post-crawl migration: %v", driveID, err)
@@ -3365,3 +3619,14 @@ func parseBoolDefault(raw string, def bool) bool {
}
return v
}
func parseIntDefault(raw string, def int) int {
if raw == "" {
return def
}
v, err := strconv.Atoi(raw)
if err != nil {
return def
}
return v
}
+6
View File
@@ -41,6 +41,7 @@ func TestSpider91UploadDriveIDDoesNotAutoSelectTarget(t *testing.T) {
reg.Set("p123-one", &spider91UploadTargetFakeDrive{id: "p123-one", kind: "p123"})
reg.Set("onedrive-one", &spider91UploadTargetFakeDrive{id: "onedrive-one", kind: "onedrive"})
reg.Set("wopan-one", &spider91UploadTargetFakeDrive{id: "wopan-one", kind: "wopan"})
reg.Set("guangyapan-one", &spider91UploadTargetFakeDrive{id: "guangyapan-one", kind: "guangyapan"})
app := &App{registry: reg}
if got := app.Spider91UploadDriveID(); got != "" {
@@ -67,6 +68,11 @@ func TestSpider91UploadDriveIDDoesNotAutoSelectTarget(t *testing.T) {
t.Fatalf("explicit wopan upload target = %q, want wopan-one", got)
}
app.spider91UploadDriveID = "guangyapan-one"
if got := app.Spider91UploadDriveID(); got != "guangyapan-one" {
t.Fatalf("explicit guangyapan upload target = %q, want guangyapan-one", got)
}
app.spider91UploadDriveID = "missing"
if got := app.Spider91UploadDriveID(); got != "" {
t.Fatalf("missing upload target = %q, want empty", got)
+200
View File
@@ -227,6 +227,53 @@ func TestRegisterPreviewWorkersBackfillsHistoricalFingerprints(t *testing.T) {
t.Fatalf("fingerprint status=%q sampled=%q, want ready with hash", got.FingerprintStatus, got.SampledSHA256)
}
func TestUpdateScriptCrawlerRunStatePreservesCurrentTeaserSwitch(t *testing.T) {
ctx := context.Background()
cat, err := catalog.Open(t.TempDir() + "/catalog.db")
if err != nil {
t.Fatalf("open catalog: %v", err)
}
t.Cleanup(func() {
if err := cat.Close(); err != nil {
t.Fatalf("close catalog: %v", err)
}
})
if err := cat.UpsertDrive(ctx, &catalog.Drive{
ID: "crawler-id",
Kind: scriptcrawler.Kind,
Name: "Crawler",
RootID: "/",
Credentials: map[string]string{
"script_path": "/tmp/crawler.py",
"target_new": "10",
},
TeaserEnabled: false,
}); err != nil {
t.Fatalf("seed crawler drive: %v", err)
}
if err := cat.SetDriveTeaserEnabled(ctx, "crawler-id", true); err != nil {
t.Fatalf("toggle teaser: %v", err)
}
app := &App{cat: cat}
if err := app.updateScriptCrawlerRunState(ctx, "crawler-id", nil); err != nil {
t.Fatalf("update run state: %v", err)
}
got, err := cat.GetDrive(ctx, "crawler-id")
if err != nil {
t.Fatalf("get crawler drive: %v", err)
}
if !got.TeaserEnabled {
t.Fatal("teaserEnabled = false after run state update, want preserved true")
}
if got.Status != "ok" || got.LastError != "" {
t.Fatalf("status=%q lastError=%q, want ok with no error", got.Status, got.LastError)
}
if got.Credentials["last_crawl_at"] == "" || got.Credentials["target_new"] != "10" {
t.Fatalf("credentials after run state update = %#v", got.Credentials)
}
}
func TestStopDriveTasksCancelsQueuedTasksAndReplacesWorkers(t *testing.T) {
ctx, cancel := context.WithCancel(context.Background())
defer cancel()
@@ -391,6 +438,37 @@ func TestDriveGenerationStatusIncludesScanState(t *testing.T) {
}
}
func TestDriveGenerationStatusIncludesScanCooldown(t *testing.T) {
until := time.Now().Add(time.Hour).Round(time.Second)
app := &App{
scanQueued: map[string]bool{"drive-id": true},
scanProgress: map[string]driveScanProgress{
"drive-id": {Scanned: 12, Added: 3, CooldownUntil: until},
},
}
status := app.driveGenerationStatuses()["drive-id"].Scan
if status.State != "cooling" {
t.Fatalf("scan status = %#v, want cooling", status)
}
if status.CooldownUntil != until.Format(time.RFC3339) {
t.Fatalf("cooldown until = %q, want %q", status.CooldownUntil, until.Format(time.RFC3339))
}
}
func TestGuangYaPanGenerationCooldowns(t *testing.T) {
drv := &serverFakeKindDrive{id: "gy", kind: "guangyapan"}
if got := generationCooldownForDrive(drv); got != 10*time.Minute {
t.Fatalf("generation cooldown = %s, want 10m", got)
}
if got := fingerprintConfigForDrive(drv).RateLimitCooldown; got != 10*time.Minute {
t.Fatalf("fingerprint cooldown = %s, want 10m", got)
}
if got := scanCooldownForDrive(drv); got != 10*time.Minute {
t.Fatalf("scan cooldown = %s, want 10m", got)
}
}
func TestRunSpider91MigrationAfterManualCrawlRequiresConfiguredUploadTarget(t *testing.T) {
ctx := context.Background()
registry := proxy.NewRegistry()
@@ -500,6 +578,128 @@ func TestScheduleCrawlerUploadMigrationSkipsWithoutUploadTarget(t *testing.T) {
}
}
func TestScheduleManualCrawlerUploadMigrationRunsWhenAssetsReady(t *testing.T) {
ctx := context.Background()
cat, err := catalog.Open(t.TempDir() + "/catalog.db")
if err != nil {
t.Fatalf("open catalog: %v", err)
}
t.Cleanup(func() {
if err := cat.Close(); err != nil {
t.Fatalf("close catalog: %v", err)
}
})
if err := cat.UpsertDrive(ctx, &catalog.Drive{
ID: "crawler-ready",
Kind: scriptcrawler.Kind,
Name: "Ready Crawler",
RootID: "/",
TeaserEnabled: true,
Credentials: map[string]string{
"script_path": "/tmp/ready.py",
"upload_drive_id": "pikpak-target",
},
}); err != nil {
t.Fatalf("seed crawler: %v", err)
}
if err := cat.UpsertVideo(ctx, &catalog.Video{
ID: scriptcrawler.BuildVideoID("crawler-ready", "source-1"),
DriveID: "crawler-ready",
FileID: "source-1.mp4",
FileName: "source-1.mp4",
Title: "Source 1",
Size: 123,
Ext: "mp4",
SampledSHA256: "sampled-source-1",
FingerprintStatus: "ready",
PreviewStatus: "ready",
PublishedAt: time.Now(),
CreatedAt: time.Now(),
UpdatedAt: time.Now(),
}); err != nil {
t.Fatalf("seed video: %v", err)
}
registry := proxy.NewRegistry()
registry.Set("crawler-ready", &serverFakeKindDrive{id: "crawler-ready", kind: scriptcrawler.Kind})
registry.Set("pikpak-target", &serverFakeKindDrive{id: "pikpak-target", kind: "pikpak"})
migrator := &serverFakeSpider91MigrationRunner{}
app := &App{
cat: cat,
registry: registry,
spider91Migrator: migrator,
workers: map[string]*preview.Worker{},
thumbWorkers: map[string]*preview.ThumbWorker{},
fingerprintWorkers: map[string]*fingerprint.Worker{},
}
accepted, message := app.scheduleManualCrawlerUploadMigration(ctx, "crawler-ready")
if !accepted {
t.Fatalf("accepted = false, message = %q", message)
}
deadline := time.After(time.Second)
for migrator.called == 0 {
select {
case <-deadline:
t.Fatalf("migration calls = %d, want 1", migrator.called)
case <-time.After(10 * time.Millisecond):
}
}
}
func TestScheduleManualCrawlerUploadMigrationRejectsPendingFingerprint(t *testing.T) {
ctx := context.Background()
cat, err := catalog.Open(t.TempDir() + "/catalog.db")
if err != nil {
t.Fatalf("open catalog: %v", err)
}
t.Cleanup(func() {
if err := cat.Close(); err != nil {
t.Fatalf("close catalog: %v", err)
}
})
if err := cat.UpsertDrive(ctx, &catalog.Drive{
ID: "crawler-pending",
Kind: scriptcrawler.Kind,
Name: "Pending Crawler",
RootID: "/",
TeaserEnabled: true,
Credentials: map[string]string{
"script_path": "/tmp/pending.py",
"upload_drive_id": "pikpak-target",
},
}); err != nil {
t.Fatalf("seed crawler: %v", err)
}
if err := cat.UpsertVideo(ctx, &catalog.Video{
ID: scriptcrawler.BuildVideoID("crawler-pending", "source-1"),
DriveID: "crawler-pending",
FileID: "source-1.mp4",
FileName: "source-1.mp4",
Title: "Source 1",
Size: 123,
Ext: "mp4",
PreviewStatus: "ready",
PublishedAt: time.Now(),
CreatedAt: time.Now(),
UpdatedAt: time.Now(),
}); err != nil {
t.Fatalf("seed video: %v", err)
}
migrator := &serverFakeSpider91MigrationRunner{}
app := &App{cat: cat, registry: proxy.NewRegistry(), spider91Migrator: migrator}
accepted, message := app.scheduleManualCrawlerUploadMigration(ctx, "crawler-pending")
if accepted {
t.Fatal("accepted = true, want false")
}
if !strings.Contains(message, "指纹") {
t.Fatalf("message = %q, want fingerprint reason", message)
}
if migrator.called != 0 {
t.Fatalf("migration calls = %d, want 0", migrator.called)
}
}
func TestDriveGenerationStatusUsesWorkerQueueNotPendingCatalogRows(t *testing.T) {
ctx := context.Background()
cat, err := catalog.Open(t.TempDir() + "/catalog.db")
+12 -1
View File
@@ -56,7 +56,7 @@ preview:
width: 480
# 盘列表。上线后请通过管理后台添加,本文件可留空。
# kind 支持 quark / p115 / p123 / pikpak / wopan / onedrive / googledrive / localstorage。
# kind 支持 quark / p115 / p123 / pikpak / wopan / guangyapan / onedrive / googledrive / localstorage。
# OneDrive 示例:
# - id: "my-onedrive"
# kind: "onedrive"
@@ -76,6 +76,17 @@ preview:
# # use_online_api: "false"
# # client_id: "..."
# # client_secret: "..."
# 光鸭网盘示例:
# - id: "my-guangyapan"
# kind: "guangyapan"
# name: "我的光鸭网盘"
# # 留空表示光鸭网盘根目录;也可以填写光鸭目录 fileId
# root_id: ""
# params:
# # 推荐在后台使用扫码登录自动写入 access_token / refresh_token。
# refresh_token: "..."
# # 可选:按路径解析扫描根目录,优先于 root_id
# # root_path: "影视/电影"
# 本地存储示例:
# - id: "local-media"
# kind: "localstorage"
+209 -32
View File
@@ -21,6 +21,7 @@ import (
"github.com/video-site/backend/internal/auth"
"github.com/video-site/backend/internal/catalog"
"github.com/video-site/backend/internal/drives/guangyapan"
"github.com/video-site/backend/internal/drives/p123"
"github.com/video-site/backend/internal/drives/scriptcrawler"
"github.com/video-site/backend/internal/drives/spider91"
@@ -48,33 +49,35 @@ type AdminServer struct {
// LocalPreviewDir is the local directory that stores generated preview videos and thumbs.
LocalPreviewDir string
// Hooks:外层注入实际执行者
OnDriveSaved func(driveID string) error
OnDriveDeleteCleanup func(ctx context.Context, driveID string) (int, error)
OnDriveRemoved func(driveID string)
OnScanRequested func(driveID string) bool
OnStopDriveTasks func(driveID string) bool
OnStopAllTasks func() int
OnRegenPreview func(videoID string)
OnRegenAllPreviews func()
OnRegenFailedPreviews func(driveID string)
OnRegenFailedThumbnails func(driveID string)
OnRegenFailedFingerprints func(driveID string)
OnDriveSaved func(driveID string) error
OnDriveDeleteCleanup func(ctx context.Context, driveID string) (int, error)
OnDriveRemoved func(driveID string)
OnScanRequested func(driveID string) bool
OnCrawlerUploadRequested func(driveID string) (bool, string)
OnStopDriveTasks func(driveID string) bool
OnStopAllTasks func() int
OnRegenPreview func(videoID string)
OnRegenAllPreviews func()
OnRegenFailedPreviews func(driveID string)
OnRegenFailedThumbnails func(driveID string)
OnRegenFailedFingerprints func(driveID string)
// OnStartDriveTranscode 手动开启某盘的浏览器兼容性转码任务。
// 返回 (是否接受, 拒绝原因)。转码从不自动运行,只能在这里手动触发;
// 处理完候选列表后任务自然结束。
OnStartDriveTranscode func(driveID string) (bool, string)
// OnStopDriveTranscode 手动停止某盘正在进行的转码任务。返回是否有任务被停。
OnStopDriveTranscode func(driveID string) bool
OnDeleteVideo func(ctx context.Context, videoID string, deleteSource bool) (DeleteVideoResult, error)
GetDriveGenerationStatuses func() map[string]DriveGenerationStatuses
OnStopDriveTranscode func(driveID string) bool
OnDeleteVideo func(ctx context.Context, videoID string, deleteSource bool) (DeleteVideoResult, error)
GetDriveGenerationStatuses func() map[string]DriveGenerationStatuses
GetPreviewGenerationVideoIDs func() map[string]bool
// OnTeaserEnabledChanged 在 per-drive 预览视频开关被切换后调用。
// enabled=true 时上层应该重新把 pending 预览视频入队(类似旧的全局开关从关到开);
// enabled=false 时通常不用做事 —— worker 入队前会再次查 catalog,自然停止。
OnTeaserEnabledChanged func(driveID string, enabled bool)
// Theme 读写("dark" | "pink"
// Theme 读写("dark" | "pink" | "sky"
GetTheme func() string
SetTheme func(theme string) error
// Spider91 → 115/123/PikPak/OneDrive/Google Drive/联通网盘 上传目标 drive ID 读写
// Spider91 → 115/123/PikPak/OneDrive/Google Drive/联通网盘/光鸭网盘 上传目标 drive ID 读写
GetSpider91UploadDriveID func() string
SetSpider91UploadDriveID func(driveID string) error
// OnRunNightlyJob 触发一次完整的凌晨流水线(Phase1 扫盘 + Phase2 91 爬虫 +
@@ -94,6 +97,9 @@ type AdminServer struct {
// 联通网盘扫码登录接口测试注入;生产留空走官方 panservice.mail.wo.cn。
WopanQRAPIBaseURL string
WopanQRHTTPClient *http.Client
// 光鸭网盘扫码登录接口测试注入;生产留空走官方 account.guangyapan.com。
GuangYaPanAccountBaseURL string
GuangYaPanHTTPClient *http.Client
}
const (
@@ -167,6 +173,8 @@ func (a *AdminServer) Register(r chi.Router) {
r.Get("/drives/p123/qr/{uniID}", a.handleP123QRStatus)
r.Post("/drives/wopan/qr", a.handleWopanQRStart)
r.Get("/drives/wopan/qr/{uuid}", a.handleWopanQRStatus)
r.Post("/drives/guangyapan/qr", a.handleGuangYaPanQRStart)
r.Get("/drives/guangyapan/qr/status", a.handleGuangYaPanQRStatus)
r.Delete("/drives/{id}", a.handleDeleteDrive)
r.Post("/drives/{id}/rescan", a.handleRescan)
r.Post("/drives/{id}/tasks/stop", a.handleStopDriveTasks)
@@ -187,6 +195,7 @@ func (a *AdminServer) Register(r chi.Router) {
r.Post("/crawlers/test-script", a.handleTestCrawlerScript)
r.Delete("/crawlers/{id}", a.handleDeleteCrawler)
r.Post("/crawlers/{id}/run", a.handleRunCrawler)
r.Post("/crawlers/{id}/upload", a.handleUploadCrawlerVideos)
r.Post("/crawlers/{id}/tasks/stop", a.handleStopCrawlerTasks)
// 视频
@@ -463,7 +472,8 @@ func (a *AdminServer) handleListDrives(w http.ResponseWriter, r *http.Request) {
Status string `json:"status"`
LastError string `json:"lastError,omitempty"`
HasCredential bool `json:"hasCredential"`
// TeaserEnabled 控制是否给本盘生成预览视频/封面。前端用它在网盘列表/编辑表单展示开关状态
// TeaserEnabled 控制是否给本盘生成预览视频封面生成不受影响
// 前端用它在网盘列表/编辑表单展示开关状态。
TeaserEnabled bool `json:"teaserEnabled"`
// SkipDirIDs 是用户在 admin 配置的"扫描跳过目录"集合(drive 侧目录 fileID)。
// 前端用它在"设置跳过目录"弹窗里回显已选项;JSON 字段名 camelCase 与
@@ -471,11 +481,12 @@ func (a *AdminServer) handleListDrives(w http.ResponseWriter, r *http.Request) {
SkipDirIDs []string `json:"skipDirIds"`
// LastCrawlAt 是 spider91 上次成功爬取的 unix 秒(来自 credentials.last_crawl_at)。
// 其它 kind 留 0;前端用它显示"上次抓取: N 小时前"。
Spider91Proxy string `json:"spider91Proxy,omitempty"`
LastCrawlAt int64 `json:"lastCrawlAt,omitempty"`
GoogleDriveUseOnlineAPI *bool `json:"googleDriveUseOnlineAPI,omitempty"`
Spider91Proxy string `json:"spider91Proxy,omitempty"`
LastCrawlAt int64 `json:"lastCrawlAt,omitempty"`
GoogleDriveUseOnlineAPI *bool `json:"googleDriveUseOnlineAPI,omitempty"`
GoogleDriveOpenListAPIURL string `json:"googleDriveOpenListApiUrl,omitempty"`
// STRMAllowOutsideRoot 是 localstorage 的 .strm 越root开关;其它 kind 省略。
STRMAllowOutsideRoot *bool `json:"strmAllowOutsideRoot,omitempty"`
STRMAllowOutsideRoot *bool `json:"strmAllowOutsideRoot,omitempty"`
ScanGenerationStatus GenerationStatus `json:"scanGenerationStatus"`
ThumbnailGenerationStatus GenerationStatus `json:"thumbnailGenerationStatus"`
PreviewGenerationStatus GenerationStatus `json:"previewGenerationStatus"`
@@ -552,6 +563,7 @@ func (a *AdminServer) handleListDrives(w http.ResponseWriter, r *http.Request) {
Spider91Proxy: spider91ProxyForDrive(d),
LastCrawlAt: lastCrawlAt,
GoogleDriveUseOnlineAPI: googleDriveUseOnlineAPIForDrive(d),
GoogleDriveOpenListAPIURL: googleDriveOpenListAPIURLForDrive(d),
STRMAllowOutsideRoot: strmAllowOutsideRootForDrive(d),
ScanGenerationStatus: generation.Scan,
ThumbnailGenerationStatus: generation.Thumbnail,
@@ -585,7 +597,7 @@ type upsertDriveReq struct {
// Deprecated: 扫描起点已固定为 rootId;保留字段只为兼容旧客户端请求体。
ScanRootID string `json:"scanRootId"`
Credentials map[string]string `json:"credentials"`
// TeaserEnabled 是 per-drive 预览视频/封面生成开关。
// TeaserEnabled 是 per-drive 预览视频生成开关;封面生成不受影响
// 用 *bool 区分 "未传" / "传了 false":未传时表示客户端不打算改这个字段,
// 沿用 catalog 现有值;新建时未传一律默认开启(true)。
TeaserEnabled *bool `json:"teaserEnabled,omitempty"`
@@ -620,9 +632,11 @@ func (a *AdminServer) handleUpsertDrive(w http.ResponseWriter, r *http.Request)
return
}
body.Credentials = credentials
} else if body.Kind == "googledrive" || body.Kind == "localstorage" {
// 按键合并、空值沿用旧值:localstorage 编辑表单里 path 留空表示不改,
// 但 strm_allow_outside_root 开关每次都会带值,必须逐键合并而不是整体替换。
} else if body.Kind == "googledrive" {
body.Credentials = mergeGoogleDriveCredentials(existing, body.Credentials)
} else if body.Kind == "localstorage" || body.Kind == "guangyapan" {
// 按键合并、空值沿用旧值:这些网盘的编辑表单允许只改某几个字段,
// 其它 token / 路径 / 开关字段应保留旧值。
body.Credentials = mergeNonEmptyCredentials(existing, body.Credentials)
} else if len(body.Credentials) == 0 && existing != nil && len(existing.Credentials) > 0 {
body.Credentials = existing.Credentials
@@ -684,6 +698,7 @@ type crawlerDTO struct {
Proxy string `json:"proxy,omitempty"`
TargetNew string `json:"targetNew,omitempty"`
UploadDriveID string `json:"uploadDriveId,omitempty"`
TeaserEnabled bool `json:"teaserEnabled"`
LastCrawlAt int64 `json:"lastCrawlAt,omitempty"`
ScanGenerationStatus GenerationStatus `json:"scanGenerationStatus"`
ThumbnailGenerationStatus GenerationStatus `json:"thumbnailGenerationStatus"`
@@ -711,6 +726,7 @@ type upsertCrawlerReq struct {
Proxy string `json:"proxy"`
TargetNew string `json:"targetNew"`
UploadDriveID string `json:"uploadDriveId"`
TeaserEnabled *bool `json:"teaserEnabled,omitempty"`
}
func (a *AdminServer) handleListCrawlers(w http.ResponseWriter, r *http.Request) {
@@ -772,6 +788,7 @@ func (a *AdminServer) crawlerDTOForDrive(d *catalog.Drive, assets catalog.Crawle
Proxy: strings.TrimSpace(d.Credentials["proxy"]),
TargetNew: strings.TrimSpace(d.Credentials["target_new"]),
UploadDriveID: strings.TrimSpace(d.Credentials["upload_drive_id"]),
TeaserEnabled: d.TeaserEnabled,
LastCrawlAt: lastCrawlAt,
ScanGenerationStatus: generation.Scan,
ThumbnailGenerationStatus: generation.Thumbnail,
@@ -858,6 +875,13 @@ func (a *AdminServer) handleUpsertCrawler(w http.ResponseWriter, r *http.Request
return
}
name := meta.Name
teaserEnabled := true
if existing != nil {
teaserEnabled = existing.TeaserEnabled
}
if body.TeaserEnabled != nil {
teaserEnabled = *body.TeaserEnabled
}
if id == "" {
generatedID, err := a.generateCrawlerID(r.Context(), name)
if err != nil {
@@ -873,15 +897,15 @@ func (a *AdminServer) handleUpsertCrawler(w http.ResponseWriter, r *http.Request
RootID: "/",
Credentials: merged,
Status: "disconnected",
TeaserEnabled: true,
}
if existing != nil {
d.TeaserEnabled = existing.TeaserEnabled
TeaserEnabled: teaserEnabled,
}
if err := a.Catalog.UpsertDrive(r.Context(), d); err != nil {
writeErr(w, http.StatusInternalServerError, err)
return
}
if existing != nil && existing.TeaserEnabled != teaserEnabled && a.OnTeaserEnabledChanged != nil {
a.OnTeaserEnabledChanged(id, teaserEnabled)
}
if a.OnDriveSaved != nil {
if err := a.OnDriveSaved(id); err != nil {
writeJSON(w, http.StatusOK, map[string]any{"ok": true, "id": id, "warning": err.Error()})
@@ -931,14 +955,14 @@ func (a *AdminServer) validateCrawlerUploadDrive(ctx context.Context, driveID st
return fmt.Errorf("上传目标网盘 %q 不存在", driveID)
}
if !isCrawlerUploadTargetKind(d.Kind) {
return fmt.Errorf("上传目标网盘 %q 类型为 %s,仅支持 115网盘、PikPak、123网盘、Google Drive、OneDrive、联通网盘", driveID, d.Kind)
return fmt.Errorf("上传目标网盘 %q 类型为 %s,仅支持 115网盘、PikPak、123网盘、Google Drive、OneDrive、联通网盘、光鸭网盘", driveID, d.Kind)
}
return nil
}
func isCrawlerUploadTargetKind(kind string) bool {
switch strings.TrimSpace(kind) {
case "p115", "pikpak", "p123", "googledrive", "onedrive", "wopan":
case "p115", "pikpak", "p123", "googledrive", "onedrive", "wopan", "guangyapan":
return true
default:
return false
@@ -1267,6 +1291,104 @@ func (a *AdminServer) handleRunCrawler(w http.ResponseWriter, r *http.Request) {
writeJSON(w, http.StatusAccepted, resp)
}
func (a *AdminServer) handleUploadCrawlerVideos(w http.ResponseWriter, r *http.Request) {
id := chi.URLParam(r, "id")
d, err := a.Catalog.GetDrive(r.Context(), id)
if err != nil || d == nil || !isConfiguredCrawlerDrive(d) {
http.Error(w, "crawler not found", http.StatusNotFound)
return
}
status := a.nightlyJobStatus()
if status.Running || status.Queued {
writeJSON(w, http.StatusAccepted, map[string]any{
"ok": true,
"accepted": false,
"message": fullScanBusyMessage,
"status": status,
})
return
}
assets, err := a.Catalog.CountCrawlerAssets(r.Context(), d.ID, crawlerVideoIDPrefixes(d))
if err != nil {
writeErr(w, http.StatusInternalServerError, err)
return
}
generation := DriveGenerationStatuses{}
if a.GetDriveGenerationStatuses != nil {
generation = a.GetDriveGenerationStatuses()[d.ID]
}
if reason := crawlerUploadBlockedReason(d, assets, generation); reason != "" {
writeJSON(w, http.StatusAccepted, map[string]any{
"ok": true,
"accepted": false,
"message": reason,
})
return
}
accepted := true
message := ""
if a.OnCrawlerUploadRequested != nil {
accepted, message = a.OnCrawlerUploadRequested(id)
}
resp := map[string]any{"ok": true, "accepted": accepted}
if !accepted {
if strings.TrimSpace(message) == "" {
message = driveTaskBusyMessage
}
resp["message"] = message
}
writeJSON(w, http.StatusAccepted, resp)
}
func crawlerUploadBlockedReason(d *catalog.Drive, assets catalog.CrawlerAssetCounts, generation DriveGenerationStatuses) string {
if d == nil || !isConfiguredCrawlerDrive(d) {
return "爬虫不存在"
}
if strings.TrimSpace(d.Credentials["upload_drive_id"]) == "" {
return "请先配置上传网盘"
}
if assets.Local <= 0 {
return "没有待上传的本地视频"
}
if crawlerGenerationBusy(generation) {
return "当前爬虫有正在进行的任务,请稍后重试"
}
if assets.Fingerprint.Pending > 0 {
return "还有待生成的视频指纹"
}
if assets.Fingerprint.Failed > 0 {
return "存在指纹生成失败的视频,请先重试或处理失败项"
}
if d.TeaserEnabled {
if assets.Teaser.Pending > 0 {
return "还有待生成的预览视频"
}
if assets.Teaser.Failed > 0 {
return "存在预览视频生成失败的视频,请先重试或处理失败项"
}
}
return ""
}
func crawlerGenerationBusy(g DriveGenerationStatuses) bool {
return generationBusy(g.Scan) ||
generationBusy(g.Thumbnail) ||
generationBusy(g.Preview) ||
generationBusy(g.Fingerprint) ||
generationBusy(g.Upload)
}
func generationBusy(g GenerationStatus) bool {
switch strings.TrimSpace(g.State) {
case "", "idle":
return false
default:
return true
}
}
func (a *AdminServer) handleStopCrawlerTasks(w http.ResponseWriter, r *http.Request) {
a.handleStopDriveTasks(w, r)
}
@@ -1394,8 +1516,23 @@ func googleDriveUseOnlineAPIForDrive(d *catalog.Drive) *bool {
return &result
}
func googleDriveOpenListAPIURLForDrive(d *catalog.Drive) string {
if d == nil || d.Kind != "googledrive" || d.Credentials == nil {
return ""
}
return strings.TrimSpace(d.Credentials["api_url_address"])
}
func mergeGoogleDriveCredentials(existing *catalog.Drive, incoming map[string]string) map[string]string {
merged := mergeNonEmptyCredentials(existing, incoming)
if _, ok := incoming["api_url_address"]; ok && strings.TrimSpace(incoming["api_url_address"]) == "" {
delete(merged, "api_url_address")
}
return merged
}
// mergeNonEmptyCredentials 逐键合并凭证:incoming 里非空的键覆盖旧值,
// 空值/缺失的键沿用旧值。googledrivelocalstorage 的编辑表单都依赖
// 空值/缺失的键沿用旧值。googledrivelocalstorage 和 guangyapan 的编辑表单都依赖
// 这个语义(留空 = 不修改)。
func mergeNonEmptyCredentials(existing *catalog.Drive, incoming map[string]string) map[string]string {
merged := map[string]string{}
@@ -1696,6 +1833,38 @@ func (a *AdminServer) handleWopanQRStatus(w http.ResponseWriter, r *http.Request
writeJSON(w, http.StatusOK, status)
}
func (a *AdminServer) guangYaPanQRClient() *guangyapan.QRClient {
return guangyapan.NewQRClient(guangyapan.QRConfig{
AccountBaseURL: a.GuangYaPanAccountBaseURL,
HTTPClient: a.GuangYaPanHTTPClient,
})
}
func (a *AdminServer) handleGuangYaPanQRStart(w http.ResponseWriter, r *http.Request) {
session, err := a.guangYaPanQRClient().Generate(r.Context())
if err != nil {
writeErr(w, http.StatusBadGateway, err)
return
}
w.Header().Set("Cache-Control", "no-store")
writeJSON(w, http.StatusOK, session)
}
func (a *AdminServer) handleGuangYaPanQRStatus(w http.ResponseWriter, r *http.Request) {
deviceCode := r.URL.Query().Get("deviceCode")
if strings.TrimSpace(deviceCode) == "" {
http.Error(w, "deviceCode is required", http.StatusBadRequest)
return
}
status, err := a.guangYaPanQRClient().Poll(r.Context(), deviceCode)
if err != nil {
writeErr(w, http.StatusBadGateway, err)
return
}
w.Header().Set("Cache-Control", "no-store")
writeJSON(w, http.StatusOK, status)
}
// handleRunNightlyJob 触发一次完整的凌晨流水线(不论当前时间,不论今日是否已跑)。
// 立即返回 202;进度通过 backend 日志和下次 GET /admin/api/drives 的状态变化观察。
// 流水线已在跑或已排队时,Runner 会拒绝重复触发。
@@ -1882,6 +2051,14 @@ func (a *AdminServer) handleAdminListVideos(w http.ResponseWriter, r *http.Reque
writeErr(w, http.StatusInternalServerError, err)
return
}
if a.GetPreviewGenerationVideoIDs != nil {
generating := a.GetPreviewGenerationVideoIDs()
for _, item := range items {
if item != nil && generating[item.ID] {
item.PreviewStatus = "generating"
}
}
}
writeJSON(w, http.StatusOK, map[string]any{
"items": items,
"total": total,
+279 -23
View File
@@ -732,6 +732,31 @@ func TestHandleUpsertGoogleDriveMergesOAuthCredentials(t *testing.T) {
if got.Credentials["client_id"] != "google-client-id" || got.Credentials["client_secret"] != "google-client-secret" {
t.Fatalf("oauth client credentials = %#v, want saved", got.Credentials)
}
if got.Credentials["api_url_address"] != "https://api.oplist.org/googleui/renewapi" {
t.Fatalf("api_url_address = %q, want preserved", got.Credentials["api_url_address"])
}
clearReq := httptest.NewRequest(http.MethodPost, "/admin/api/drives", bytes.NewBufferString(`{
"id": "google-main",
"kind": "googledrive",
"name": "Google Drive",
"rootId": "root",
"credentials": {
"api_url_address": ""
}
}`))
clearRR := httptest.NewRecorder()
(&AdminServer{Catalog: cat}).handleUpsertDrive(clearRR, clearReq)
if clearRR.Code != http.StatusOK {
t.Fatalf("clear status = %d, body = %s", clearRR.Code, clearRR.Body.String())
}
cleared, err := cat.GetDrive(ctx, "google-main")
if err != nil {
t.Fatalf("get cleared drive: %v", err)
}
if _, ok := cleared.Credentials["api_url_address"]; ok {
t.Fatalf("api_url_address was not cleared: %#v", cleared.Credentials)
}
}
func TestHandleUpsertSpider91DriveIsRejected(t *testing.T) {
@@ -754,7 +779,7 @@ func TestHandleUpsertSpider91DriveIsRejected(t *testing.T) {
Credentials: map[string]string{
"last_crawl_at": "1800000000",
"proxy": "http://old-proxy.local:7890",
"script_path": "/opt/video-site-91/91VideoSpider/spider_91porn.py",
"script_path": "/opt/video-site-91/data/crawler-scripts/legacy-spider.py",
},
Status: "ok",
}); err != nil {
@@ -944,7 +969,8 @@ func TestHandleListCrawlersOnlyIncludesCrawlerPageScripts(t *testing.T) {
"script_path": scriptPath,
"upload_drive_id": "p115-target",
},
Status: "ok",
Status: "ok",
TeaserEnabled: false,
},
{
ID: "p115-target",
@@ -1027,6 +1053,7 @@ func TestHandleListCrawlersOnlyIncludesCrawlerPageScripts(t *testing.T) {
Kind string `json:"kind"`
Proxy string `json:"proxy"`
UploadDriveID string `json:"uploadDriveId"`
TeaserEnabled bool `json:"teaserEnabled"`
LastCrawlAt int64 `json:"lastCrawlAt"`
TotalCrawled int `json:"totalCrawledCount"`
LocalVideos int `json:"localVideoCount"`
@@ -1038,11 +1065,12 @@ func TestHandleListCrawlersOnlyIncludesCrawlerPageScripts(t *testing.T) {
if err := json.NewDecoder(rr.Body).Decode(&got); err != nil {
t.Fatalf("decode: %v", err)
}
byID := map[string]struct {
type crawlerListRow struct {
Name string
Kind string
Proxy string
UploadDriveID string
TeaserEnabled bool
LastCrawlAt int64
TotalCrawled int
LocalVideos int
@@ -1050,25 +1078,15 @@ func TestHandleListCrawlersOnlyIncludesCrawlerPageScripts(t *testing.T) {
ThumbnailReady int
TeaserReady int
FingerprintReady int
}{}
}
byID := map[string]crawlerListRow{}
for _, d := range got {
byID[d.ID] = struct {
Name string
Kind string
Proxy string
UploadDriveID string
LastCrawlAt int64
TotalCrawled int
LocalVideos int
MigratedVideo int
ThumbnailReady int
TeaserReady int
FingerprintReady int
}{
byID[d.ID] = crawlerListRow{
Name: d.Name,
Kind: d.Kind,
Proxy: d.Proxy,
UploadDriveID: d.UploadDriveID,
TeaserEnabled: d.TeaserEnabled,
LastCrawlAt: d.LastCrawlAt,
TotalCrawled: d.TotalCrawled,
LocalVideos: d.LocalVideos,
@@ -1096,6 +1114,9 @@ func TestHandleListCrawlersOnlyIncludesCrawlerPageScripts(t *testing.T) {
if byID["crawler-spider91"].UploadDriveID != "p115-target" {
t.Fatalf("uploadDriveId = %q, want p115-target", byID["crawler-spider91"].UploadDriveID)
}
if byID["crawler-spider91"].TeaserEnabled {
t.Fatal("teaserEnabled = true, want false from crawler drive")
}
if byID["crawler-spider91"].LastCrawlAt != 1800000000 {
t.Fatalf("lastCrawlAt = %d, want 1800000000", byID["crawler-spider91"].LastCrawlAt)
}
@@ -1171,7 +1192,8 @@ func TestHandleUpsertCrawlerRequiresScriptPath(t *testing.T) {
"id": "spider91-main",
"builtin": "spider91",
"scriptPath": "`+scriptPath+`",
"targetNew": "15"
"targetNew": "15",
"teaserEnabled": false
}`))
rr = httptest.NewRecorder()
srv.handleUpsertCrawler(rr, req)
@@ -1195,6 +1217,9 @@ func TestHandleUpsertCrawlerRequiresScriptPath(t *testing.T) {
if got.Credentials["script_path"] != scriptPath {
t.Fatalf("script_path = %q, want %q", got.Credentials["script_path"], scriptPath)
}
if got.TeaserEnabled {
t.Fatal("teaserEnabled = true, want false from request")
}
}
func TestHandleUpsertCrawlerGeneratesIDFromScriptName(t *testing.T) {
@@ -1271,18 +1296,28 @@ func TestHandleUpsertCrawlerPersistsAndValidatesUploadDrive(t *testing.T) {
for _, d := range []*catalog.Drive{
{ID: "p115-target", Kind: "p115", Name: "115", RootID: "0", Credentials: map[string]string{"cookie": "x"}},
{ID: "wopan-target", Kind: "wopan", Name: "沃盘", RootID: "0", Credentials: map[string]string{"access_token": "a", "refresh_token": "r"}},
{ID: "guangyapan-target", Kind: "guangyapan", Name: "光鸭", RootID: "", Credentials: map[string]string{"access_token": "a", "refresh_token": "r"}},
{ID: "local-target", Kind: "localstorage", Name: "Local", RootID: "/", Credentials: map[string]string{"path": tmp}},
} {
if err := cat.UpsertDrive(ctx, d); err != nil {
t.Fatalf("seed drive %s: %v", d.ID, err)
}
}
srv := &AdminServer{Catalog: cat}
var teaserCallbackID string
var teaserCallbackEnabled bool
srv := &AdminServer{
Catalog: cat,
OnTeaserEnabledChanged: func(id string, enabled bool) {
teaserCallbackID = id
teaserCallbackEnabled = enabled
},
}
req := httptest.NewRequest(http.MethodPost, "/admin/api/crawlers", strings.NewReader(`{
"id": "crawler-upload",
"scriptPath": "`+scriptPath+`",
"uploadDriveId": "p115-target"
"uploadDriveId": "p115-target",
"teaserEnabled": false
}`))
rr := httptest.NewRecorder()
srv.handleUpsertCrawler(rr, req)
@@ -1296,6 +1331,12 @@ func TestHandleUpsertCrawlerPersistsAndValidatesUploadDrive(t *testing.T) {
if got.Credentials["upload_drive_id"] != "p115-target" {
t.Fatalf("upload_drive_id = %q, want p115-target", got.Credentials["upload_drive_id"])
}
if got.TeaserEnabled {
t.Fatal("teaserEnabled = true, want false")
}
if teaserCallbackID != "" {
t.Fatalf("teaser callback on create = %q, want none", teaserCallbackID)
}
req = httptest.NewRequest(http.MethodPost, "/admin/api/crawlers", strings.NewReader(`{
"id": "crawler-upload",
@@ -1314,6 +1355,52 @@ func TestHandleUpsertCrawlerPersistsAndValidatesUploadDrive(t *testing.T) {
if got.Credentials["upload_drive_id"] != "wopan-target" {
t.Fatalf("upload_drive_id = %q, want wopan-target", got.Credentials["upload_drive_id"])
}
if got.TeaserEnabled {
t.Fatal("teaserEnabled after edit without field = true, want preserved false")
}
if teaserCallbackID != "" {
t.Fatalf("teaser callback after preserved edit = %q, want none", teaserCallbackID)
}
req = httptest.NewRequest(http.MethodPost, "/admin/api/crawlers", strings.NewReader(`{
"id": "crawler-upload",
"scriptPath": "`+scriptPath+`",
"uploadDriveId": "guangyapan-target"
}`))
rr = httptest.NewRecorder()
srv.handleUpsertCrawler(rr, req)
if rr.Code != http.StatusOK {
t.Fatalf("guangyapan target status = %d, body = %s", rr.Code, rr.Body.String())
}
got, err = cat.GetDrive(ctx, "crawler-upload")
if err != nil {
t.Fatalf("get crawler after guangyapan target: %v", err)
}
if got.Credentials["upload_drive_id"] != "guangyapan-target" {
t.Fatalf("upload_drive_id = %q, want guangyapan-target", got.Credentials["upload_drive_id"])
}
req = httptest.NewRequest(http.MethodPost, "/admin/api/crawlers", strings.NewReader(`{
"id": "crawler-upload",
"scriptPath": "`+scriptPath+`",
"uploadDriveId": "wopan-target",
"teaserEnabled": true
}`))
rr = httptest.NewRecorder()
srv.handleUpsertCrawler(rr, req)
if rr.Code != http.StatusOK {
t.Fatalf("enable teaser status = %d, body = %s", rr.Code, rr.Body.String())
}
got, err = cat.GetDrive(ctx, "crawler-upload")
if err != nil {
t.Fatalf("get crawler after teaser enable: %v", err)
}
if !got.TeaserEnabled {
t.Fatal("teaserEnabled after explicit enable = false, want true")
}
if teaserCallbackID != "crawler-upload" || !teaserCallbackEnabled {
t.Fatalf("teaser callback = %q/%v, want crawler-upload/true", teaserCallbackID, teaserCallbackEnabled)
}
req = httptest.NewRequest(http.MethodPost, "/admin/api/crawlers", strings.NewReader(`{
"id": "crawler-upload",
@@ -1704,6 +1791,94 @@ func TestHandleWopanQRStatus(t *testing.T) {
}
}
func TestHandleGuangYaPanQRStart(t *testing.T) {
upstream := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
w.Header().Set("Content-Type", "application/json")
if r.URL.Path != "/v1/auth/device/code" {
http.NotFound(w, r)
return
}
var body map[string]any
if err := json.NewDecoder(r.Body).Decode(&body); err != nil {
t.Fatalf("decode body: %v", err)
}
if body["scope"] != "user" {
t.Fatalf("scope = %#v, want user", body["scope"])
}
_ = json.NewEncoder(w).Encode(map[string]any{
"device_code": "device-1",
"verification_uri_complete": "https://account.guangyapan.example/device?code=abc",
"interval": 5,
"expires_in": 300,
})
}))
t.Cleanup(upstream.Close)
req := httptest.NewRequest(http.MethodPost, "/admin/api/drives/guangyapan/qr", nil)
rr := httptest.NewRecorder()
(&AdminServer{GuangYaPanAccountBaseURL: upstream.URL}).handleGuangYaPanQRStart(rr, req)
if rr.Code != http.StatusOK {
t.Fatalf("status = %d, body = %s", rr.Code, rr.Body.String())
}
var got struct {
DeviceCode string `json:"deviceCode"`
QRCodeURL string `json:"qrCodeUrl"`
QRImageDataURL string `json:"qrImageDataUrl"`
}
if err := json.NewDecoder(rr.Body).Decode(&got); err != nil {
t.Fatalf("decode: %v", err)
}
if got.DeviceCode != "device-1" || got.QRCodeURL != "https://account.guangyapan.example/device?code=abc" {
t.Fatalf("response = %#v", got)
}
if !strings.HasPrefix(got.QRImageDataURL, "data:image/png;base64,") {
t.Fatalf("qr image = %q", got.QRImageDataURL)
}
}
func TestHandleGuangYaPanQRStatus(t *testing.T) {
upstream := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
w.Header().Set("Content-Type", "application/json")
if r.URL.Path != "/v1/auth/token" {
http.NotFound(w, r)
return
}
var body map[string]any
if err := json.NewDecoder(r.Body).Decode(&body); err != nil {
t.Fatalf("decode body: %v", err)
}
if body["device_code"] != "device-1" {
t.Fatalf("device_code = %#v, want device-1", body["device_code"])
}
_ = json.NewEncoder(w).Encode(map[string]any{
"access_token": "access-1",
"refresh_token": "refresh-1",
"token_type": "Bearer",
})
}))
t.Cleanup(upstream.Close)
req := httptest.NewRequest(http.MethodGet, "/admin/api/drives/guangyapan/qr/status?deviceCode=device-1", nil)
rr := httptest.NewRecorder()
(&AdminServer{GuangYaPanAccountBaseURL: upstream.URL}).handleGuangYaPanQRStatus(rr, req)
if rr.Code != http.StatusOK {
t.Fatalf("status = %d, body = %s", rr.Code, rr.Body.String())
}
var got struct {
State string `json:"state"`
AccessToken string `json:"accessToken"`
RefreshToken string `json:"refreshToken"`
}
if err := json.NewDecoder(rr.Body).Decode(&got); err != nil {
t.Fatalf("decode: %v", err)
}
if got.State != "success" || got.AccessToken != "access-1" || got.RefreshToken != "refresh-1" {
t.Fatalf("response = %#v", got)
}
}
func TestHandleTestCrawlerScriptRunsImportedScript(t *testing.T) {
if _, err := exec.LookPath("python3"); err != nil {
t.Skip("python3 is required for crawler script dry-run")
@@ -1798,7 +1973,8 @@ func TestHandleListDrivesIncludesGoogleDriveOnlineAPIMode(t *testing.T) {
Name: "Google Legacy",
RootID: "root",
Credentials: map[string]string{
"refresh_token": "legacy-refresh",
"refresh_token": "legacy-refresh",
"api_url_address": "https://openlist-api.example/googleui/renewapi",
},
Status: "ok",
},
@@ -1829,15 +2005,18 @@ func TestHandleListDrivesIncludesGoogleDriveOnlineAPIMode(t *testing.T) {
}
var got []struct {
ID string `json:"id"`
GoogleDriveUseOnlineAPI bool `json:"googleDriveUseOnlineAPI"`
ID string `json:"id"`
GoogleDriveUseOnlineAPI bool `json:"googleDriveUseOnlineAPI"`
GoogleDriveOpenListAPIURL string `json:"googleDriveOpenListApiUrl"`
}
if err := json.NewDecoder(rr.Body).Decode(&got); err != nil {
t.Fatalf("decode: %v", err)
}
byID := map[string]bool{}
byAPIURL := map[string]string{}
for _, d := range got {
byID[d.ID] = d.GoogleDriveUseOnlineAPI
byAPIURL[d.ID] = d.GoogleDriveOpenListAPIURL
}
if !byID["google-legacy"] {
t.Fatalf("legacy google drive use_online_api = false, want true")
@@ -1845,6 +2024,9 @@ func TestHandleListDrivesIncludesGoogleDriveOnlineAPIMode(t *testing.T) {
if byID["google-oauth"] {
t.Fatalf("oauth google drive use_online_api = true, want false")
}
if byAPIURL["google-legacy"] != "https://openlist-api.example/googleui/renewapi" {
t.Fatalf("legacy google drive openlist api url = %q, want custom URL", byAPIURL["google-legacy"])
}
}
func TestHandleListDrivesIncludesTeaserCounts(t *testing.T) {
@@ -2373,6 +2555,80 @@ func TestHandleAdminListVideosPaginates(t *testing.T) {
}
}
func TestHandleAdminListVideosMarksActivePreviewGeneration(t *testing.T) {
ctx := context.Background()
cat, err := catalog.Open(t.TempDir() + "/catalog.db")
if err != nil {
t.Fatalf("open catalog: %v", err)
}
t.Cleanup(func() {
if err := cat.Close(); err != nil {
t.Fatalf("close catalog: %v", err)
}
})
now := time.Now()
for _, v := range []*catalog.Video{
{
ID: "active-video",
DriveID: "OneDrive",
FileID: "active-file",
Title: "Active video",
PreviewStatus: "ready",
PublishedAt: now,
CreatedAt: now,
UpdatedAt: now,
},
{
ID: "idle-video",
DriveID: "OneDrive",
FileID: "idle-file",
Title: "Idle video",
PreviewStatus: "ready",
PublishedAt: now.Add(-time.Hour),
CreatedAt: now,
UpdatedAt: now,
},
} {
if err := cat.UpsertVideo(ctx, v); err != nil {
t.Fatalf("seed video %s: %v", v.ID, err)
}
}
req := httptest.NewRequest(http.MethodGet, "/admin/api/videos?driveId=OneDrive", nil)
rr := httptest.NewRecorder()
(&AdminServer{
Catalog: cat,
GetPreviewGenerationVideoIDs: func() map[string]bool {
return map[string]bool{"active-video": true}
},
}).handleAdminListVideos(rr, req)
if rr.Code != http.StatusOK {
t.Fatalf("status = %d, body = %s", rr.Code, rr.Body.String())
}
var got struct {
Items []catalog.Video `json:"items"`
Total int `json:"total"`
}
if err := json.NewDecoder(rr.Body).Decode(&got); err != nil {
t.Fatalf("decode: %v", err)
}
if got.Total != 2 || len(got.Items) != 2 {
t.Fatalf("response total/items = %d/%d, want 2/2", got.Total, len(got.Items))
}
statusByID := map[string]string{}
for _, item := range got.Items {
statusByID[item.ID] = item.PreviewStatus
}
if statusByID["active-video"] != "generating" {
t.Fatalf("active status = %q, want generating", statusByID["active-video"])
}
if statusByID["idle-video"] != "ready" {
t.Fatalf("idle status = %q, want ready", statusByID["idle-video"])
}
}
func TestHandleRegenAllPreviewsInvokesHook(t *testing.T) {
called := false
server := &AdminServer{
+5 -3
View File
@@ -64,7 +64,7 @@ type Server struct {
tagCacheUntil time.Time
tagCache []TagDTO
// GetTheme 返回当前生效的主题("dark" | "pink")。前台 /api/settings/theme 用,
// GetTheme 返回当前生效的主题("dark" | "pink" | "sky")。前台 /api/settings/theme 用,
// 不需要登录。无注入时返回 "dark"。
GetTheme func() string
}
@@ -160,11 +160,11 @@ func (s *Server) RegisterRoutes(r chi.Router, a *auth.Authenticator) {
}
// handleGetTheme 返回当前生效的主题。无需登录。响应永远是
// {"theme": "dark"} 或 {"theme": "pink"},便于前端无脑解析。
// {"theme": "dark" | "pink" | "sky"},便于前端无脑解析。
func (s *Server) handleGetTheme(w http.ResponseWriter, r *http.Request) {
theme := "dark"
if s.GetTheme != nil {
if v := s.GetTheme(); v == "pink" || v == "dark" {
if v := s.GetTheme(); v == "pink" || v == "dark" || v == "sky" {
theme = v
}
}
@@ -1068,6 +1068,8 @@ func driveKindLabel(kind string) string {
return "PikPak"
case "wopan":
return "联通网盘"
case "guangyapan":
return "光鸭网盘"
case "onedrive":
return "OneDrive"
case "googledrive":
+38 -16
View File
@@ -78,6 +78,7 @@ type Video struct {
TranscodedFileID string `json:"transcodedFileId"`
TranscodedSize int64 `json:"transcodedSize"`
Views int `json:"views"`
LastViewedAt time.Time `json:"lastViewedAt"`
Favorites int `json:"favorites"`
Comments int `json:"comments"`
Likes int `json:"likes"`
@@ -112,13 +113,13 @@ INSERT INTO videos (
id, drive_id, file_id, file_name, content_hash, sampled_sha256, fingerprint_status, fingerprint_error, parent_id, title, author, tags,
duration_seconds, size_bytes, ext, quality, thumbnail_url, thumbnail_status,
preview_file_id, preview_local, preview_status,
views, favorites, comments, likes, dislikes,
views, last_viewed_at, favorites, comments, likes, dislikes,
category, hidden, badges, description, published_at, created_at, updated_at
) VALUES (
?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?,
?, ?, ?, ?, ?, CASE WHEN COALESCE(?, '') != '' THEN 'ready' ELSE 'pending' END,
?, ?, ?,
?, ?, ?, ?, ?,
?, ?, ?, ?, ?, ?,
?, ?, ?, ?, ?, ?, ?
)
ON CONFLICT(id) DO UPDATE SET
@@ -169,7 +170,7 @@ ON CONFLICT(id) DO UPDATE SET
v.ID, v.DriveID, v.FileID, v.FileName, v.ContentHash, v.SampledSHA256, fingerprintStatus, v.FingerprintError, v.ParentID, v.Title, v.Author, string(tagsJSON),
v.DurationSeconds, v.Size, v.Ext, v.Quality, v.ThumbnailURL, v.ThumbnailURL,
v.PreviewFileID, v.PreviewLocal, nullableStatus(v.PreviewStatus),
v.Views, v.Favorites, v.Comments, v.Likes, v.Dislikes,
v.Views, unixMilliOrZero(v.LastViewedAt), v.Favorites, v.Comments, v.Likes, v.Dislikes,
v.Category, boolToInt(v.Hidden), string(badgesJSON), v.Description,
v.PublishedAt.UnixMilli(), v.CreatedAt.UnixMilli(), v.UpdatedAt.UnixMilli(),
)
@@ -423,9 +424,10 @@ func (c *Catalog) IncrementView(ctx context.Context, id string) (int, error) {
return 0, err
}
defer tx.Rollback()
now := time.Now().UnixMilli()
res, err := tx.ExecContext(ctx,
`UPDATE videos SET views = views + 1, updated_at = ? WHERE id = ?`,
time.Now().UnixMilli(), id)
`UPDATE videos SET views = views + 1, last_viewed_at = ?, updated_at = ? WHERE id = ?`,
now, now, id)
if err != nil {
return 0, err
}
@@ -451,6 +453,10 @@ type VideoMetaPatch struct {
Category string
ContentHash string
FileName string
Title string
TitleSet bool
Author string
AuthorSet bool
Tags []string
TagsSet bool
}
@@ -500,6 +506,14 @@ func (c *Catalog) UpdateVideoMeta(ctx context.Context, id string, p VideoMetaPat
parts = append(parts, "file_name = ?")
args = append(args, p.FileName)
}
if p.TitleSet {
parts = append(parts, "title = ?")
args = append(args, p.Title)
}
if p.AuthorSet {
parts = append(parts, "author = ?")
args = append(args, p.Author)
}
if p.TagsSet {
tagsJSON, _ := json.Marshal(p.Tags)
parts = append(parts, "tags = ?")
@@ -1352,7 +1366,7 @@ type ListParams struct {
DriveID string
Tag string
Category string
Sort string // latest | hot | week | long
Sort string // latest | hot | recent
ThumbnailReadyOnly bool
PreferReadyThumbnails bool
SkipTotal bool
@@ -1407,10 +1421,8 @@ func (c *Catalog) ListVideos(ctx context.Context, p ListParams) ([]*Video, int,
case "hot":
// 热度 = 点赞数,点赞相同按最新
orderBy = " ORDER BY " + readyOrderPrefix + "likes DESC, published_at DESC"
case "week":
orderBy = " ORDER BY " + readyOrderPrefix + "likes DESC"
case "long":
orderBy = " ORDER BY " + readyOrderPrefix + "duration_seconds DESC"
case "recent":
orderBy = " ORDER BY " + readyOrderPrefix + "COALESCE(last_viewed_at, 0) DESC, published_at DESC"
}
var total int
@@ -1937,7 +1949,7 @@ type Drive struct {
Credentials map[string]string `json:"credentials,omitempty"`
Status string `json:"status"`
LastError string `json:"lastError,omitempty"`
// TeaserEnabled 控制是否给本盘生成预览视频/封面。
// TeaserEnabled 控制是否给本盘生成预览视频封面生成不受影响
// 替代早期的全局 preview.enabled 开关;新建 drive 时 UpsertDrive 默认置 true。
TeaserEnabled bool `json:"teaserEnabled"`
// SkipDirIDs 是用户在管理后台为该盘选定的"扫描跳过目录"集合(网盘侧的目录 fileID)。
@@ -1992,7 +2004,7 @@ func normalizeDriveRootFields(d *Drive) {
func normalizeDriveRootID(kind, rootID string) string {
rootID = strings.TrimSpace(rootID)
switch kind {
case "pikpak":
case "pikpak", "guangyapan":
if rootID == "0" {
return ""
}
@@ -2070,7 +2082,7 @@ func (c *Catalog) DeleteDrive(ctx context.Context, id string) error {
return err
}
// SetDriveTeaserEnabled 切换某盘的预览视频/封面生成开关。
// SetDriveTeaserEnabled 切换某盘的预览视频生成开关。
//
// 与 UpsertDrive 的区别:只动 teaser_enabled + updated_at 一列,不要求调用方
// 重传 kind / name / credentials 等容易踩坑的字段。
@@ -2203,7 +2215,7 @@ COALESCE(parent_id, ''), title, COALESCE(author, ''), COALESCE(tags, '[]'),
duration_seconds, size_bytes, COALESCE(ext, ''), COALESCE(quality, ''), COALESCE(thumbnail_url, ''),
COALESCE(preview_file_id, ''), COALESCE(preview_local, ''), COALESCE(preview_status, 'pending'),
COALESCE(transcode_status, ''), COALESCE(transcode_error, ''), COALESCE(transcoded_file_id, ''), COALESCE(transcoded_size, 0),
views, favorites, comments, likes, dislikes,
views, COALESCE(last_viewed_at, 0), favorites, comments, likes, dislikes,
COALESCE(category, ''), COALESCE(hidden, 0), COALESCE(badges, '[]'), COALESCE(description, ''),
published_at, created_at, updated_at
`
@@ -2266,7 +2278,7 @@ type rowScanner interface {
func scanVideo(row rowScanner) (*Video, error) {
v := &Video{}
var tagsJSON, badgesJSON string
var publishedAt, createdAt, updatedAt int64
var publishedAt, createdAt, updatedAt, lastViewedAt int64
var hidden int
err := row.Scan(
&v.ID, &v.DriveID, &v.FileID, &v.FileName, &v.ContentHash,
@@ -2275,7 +2287,7 @@ func scanVideo(row rowScanner) (*Video, error) {
&v.DurationSeconds, &v.Size, &v.Ext, &v.Quality, &v.ThumbnailURL,
&v.PreviewFileID, &v.PreviewLocal, &v.PreviewStatus,
&v.TranscodeStatus, &v.TranscodeError, &v.TranscodedFileID, &v.TranscodedSize,
&v.Views, &v.Favorites, &v.Comments, &v.Likes, &v.Dislikes,
&v.Views, &lastViewedAt, &v.Favorites, &v.Comments, &v.Likes, &v.Dislikes,
&v.Category, &hidden, &badgesJSON, &v.Description,
&publishedAt, &createdAt, &updatedAt,
)
@@ -2288,6 +2300,9 @@ func scanVideo(row rowScanner) (*Video, error) {
v.PublishedAt = time.UnixMilli(publishedAt)
v.CreatedAt = time.UnixMilli(createdAt)
v.UpdatedAt = time.UnixMilli(updatedAt)
if lastViewedAt > 0 {
v.LastViewedAt = time.UnixMilli(lastViewedAt)
}
return v, nil
}
@@ -2295,6 +2310,13 @@ func normalizeContentHash(hash string) string {
return strings.ToLower(strings.TrimSpace(hash))
}
func unixMilliOrZero(t time.Time) int64 {
if t.IsZero() {
return 0
}
return t.UnixMilli()
}
func boolToInt(v bool) int {
if v {
return 1
+1
View File
@@ -58,6 +58,7 @@ func TestUpsertDriveDefaultsRootIDByKind(t *testing.T) {
}{
{id: "p115", kind: "p115", want: "0"},
{id: "pikpak", kind: "pikpak", want: ""},
{id: "guangyapan", kind: "guangyapan", want: ""},
{id: "onedrive", kind: "onedrive", want: "root"},
{id: "googledrive", kind: "googledrive", want: "root"},
{id: "localstorage", kind: "localstorage", want: "/"},
@@ -0,0 +1,97 @@
package catalog
import (
"context"
"testing"
"time"
)
func TestIncrementViewStoresLastViewedAt(t *testing.T) {
ctx := context.Background()
cat, err := Open(t.TempDir() + "/catalog.db")
if err != nil {
t.Fatalf("open catalog: %v", err)
}
t.Cleanup(func() {
if err := cat.Close(); err != nil {
t.Fatalf("close catalog: %v", err)
}
})
now := time.Now()
if err := cat.UpsertVideo(ctx, &Video{
ID: "video-1",
DriveID: "drive",
FileID: "file-1",
Title: "Video 1",
PublishedAt: now,
CreatedAt: now,
UpdatedAt: now,
}); err != nil {
t.Fatalf("seed video: %v", err)
}
if _, err := cat.IncrementView(ctx, "video-1"); err != nil {
t.Fatalf("increment view: %v", err)
}
got, err := cat.GetVideo(ctx, "video-1")
if err != nil {
t.Fatalf("get video: %v", err)
}
if got.Views != 1 {
t.Fatalf("views = %d, want 1", got.Views)
}
if got.LastViewedAt.IsZero() {
t.Fatal("last viewed time was not stored")
}
}
func TestListVideosRecentSortUsesLastViewedAt(t *testing.T) {
ctx := context.Background()
cat, err := Open(t.TempDir() + "/catalog.db")
if err != nil {
t.Fatalf("open catalog: %v", err)
}
t.Cleanup(func() {
if err := cat.Close(); err != nil {
t.Fatalf("close catalog: %v", err)
}
})
now := time.Now()
for _, v := range []*Video{
{ID: "old-view", DriveID: "drive", FileID: "old-view", Title: "Old View", PublishedAt: now.Add(3 * time.Hour), CreatedAt: now, UpdatedAt: now},
{ID: "recent-view", DriveID: "drive", FileID: "recent-view", Title: "Recent View", PublishedAt: now, CreatedAt: now, UpdatedAt: now},
{ID: "unviewed", DriveID: "drive", FileID: "unviewed", Title: "Unviewed", PublishedAt: now.Add(4 * time.Hour), CreatedAt: now, UpdatedAt: now},
} {
if err := cat.UpsertVideo(ctx, v); err != nil {
t.Fatalf("seed %s: %v", v.ID, err)
}
}
if _, err := cat.db.ExecContext(ctx,
`UPDATE videos SET last_viewed_at = CASE id
WHEN 'old-view' THEN ?
WHEN 'recent-view' THEN ?
ELSE 0
END`,
now.Add(-time.Hour).UnixMilli(),
now.Add(time.Hour).UnixMilli(),
); err != nil {
t.Fatalf("seed last_viewed_at: %v", err)
}
items, _, err := cat.ListVideos(ctx, ListParams{Sort: "recent", Page: 1, PageSize: 3})
if err != nil {
t.Fatalf("list recent videos: %v", err)
}
if len(items) != 3 {
t.Fatalf("items = %d, want 3", len(items))
}
got := []string{items[0].ID, items[1].ID, items[2].ID}
want := []string{"recent-view", "old-view", "unviewed"}
for i := range want {
if got[i] != want[i] {
t.Fatalf("recent order = %#v, want %#v", got, want)
}
}
}
+4 -3
View File
@@ -21,12 +21,13 @@ CREATE TABLE IF NOT EXISTS videos (
thumbnail_failures INTEGER DEFAULT 0, -- consecutive transient thumbnail generation failures
preview_file_id TEXT, -- deprecated: 旧版回写网盘后的预览视频 file id
preview_local TEXT, -- 本地预览视频路径(兜底)
preview_status TEXT DEFAULT 'pending', -- pending / ready / failed
preview_status TEXT DEFAULT 'pending', -- pending / ready / failed / disabled
transcode_status TEXT DEFAULT '', -- '' / pending / ready / skipped / failed(浏览器兼容性转码)
transcode_error TEXT DEFAULT '',
transcoded_file_id TEXT DEFAULT '', -- 转码产物在同一 drive 上的 fileID,播放源优先用它
transcoded_size INTEGER DEFAULT 0,
views INTEGER DEFAULT 0,
last_viewed_at INTEGER DEFAULT 0,
favorites INTEGER DEFAULT 0,
comments INTEGER DEFAULT 0,
likes INTEGER DEFAULT 0,
@@ -114,14 +115,14 @@ CREATE INDEX IF NOT EXISTS idx_crawler_seen_sources_drive
-- 网盘账户
CREATE TABLE IF NOT EXISTS drives (
id TEXT PRIMARY KEY,
kind TEXT NOT NULL, -- quark / p115 / p123 / pikpak / wopan / onedrive / googledrive / localstorage / spider91
kind TEXT NOT NULL, -- quark / p115 / p123 / pikpak / wopan / guangyapan / onedrive / googledrive / localstorage / spider91
name TEXT NOT NULL,
root_id TEXT NOT NULL DEFAULT '0',
scan_root_id TEXT, -- deprecated: 扫描起点固定等于 root_id
credentials TEXT, -- JSON: cookie / refresh_token 等
status TEXT DEFAULT 'disconnected', -- disconnected / ok / error
last_error TEXT,
-- 是否给该盘生成预览视频/封面1 开 / 0 关。
-- 是否给该盘生成预览视频:1 开 / 0 关。封面生成不受影响。
-- 替代了早期的全局 preview.enabled 设置(保留旧 setting 行不再读)。
teaser_enabled INTEGER NOT NULL DEFAULT 1,
-- 扫描时要跳过的目录 ID 集合(JSON array of string)。命中其中任意一个的目录及其
+27
View File
@@ -66,6 +66,9 @@ func (c *Catalog) migrate(ctx context.Context) error {
if err := c.addColumnIfMissing(ctx, "videos", "thumbnail_failures", "INTEGER DEFAULT 0"); err != nil {
return err
}
if err := c.addColumnIfMissing(ctx, "videos", "last_viewed_at", "INTEGER DEFAULT 0"); err != nil {
return err
}
// videos.transcode_*:浏览器兼容性转码状态。
// status''=未检测 / pending=已入队 / ready=已转码 / skipped=检测后无需转码 / failed=失败。
// transcoded_file_id 指向转码产物在同一 drive 上的 fileID,播放源优先使用它。
@@ -124,6 +127,9 @@ CREATE TABLE IF NOT EXISTS deleted_videos (
if err := c.reconcileThumbnailStatusOnce(ctx); err != nil {
return err
}
if err := c.requeueSkippedPreviews(ctx); err != nil {
return err
}
if _, err := c.db.ExecContext(ctx, `CREATE INDEX IF NOT EXISTS idx_videos_content_hash ON videos(content_hash)`); err != nil {
return err
}
@@ -142,6 +148,9 @@ CREATE TABLE IF NOT EXISTS deleted_videos (
if _, err := c.db.ExecContext(ctx, `CREATE INDEX IF NOT EXISTS idx_videos_visible_pub ON videos(COALESCE(hidden, 0), published_at DESC)`); err != nil {
return err
}
if _, err := c.db.ExecContext(ctx, `CREATE INDEX IF NOT EXISTS idx_videos_last_viewed ON videos(last_viewed_at DESC)`); err != nil {
return err
}
if _, err := c.db.ExecContext(ctx, `CREATE INDEX IF NOT EXISTS idx_videos_file_name_size ON videos(file_name, size_bytes)`); err != nil {
return err
}
@@ -296,6 +305,24 @@ UPDATE videos
return nil
}
func (c *Catalog) requeueSkippedPreviews(ctx context.Context) error {
res, err := c.db.ExecContext(ctx, `
UPDATE videos
SET preview_file_id = '',
preview_local = '',
preview_status = 'pending',
updated_at = ?
WHERE COALESCE(preview_status, 'pending') = 'skipped'
`, time.Now().UnixMilli())
if err != nil {
return fmt.Errorf("requeue skipped previews: %w", err)
}
if affected, err := res.RowsAffected(); err == nil && affected > 0 {
log.Printf("[catalog] requeued %d skipped preview(s) for generation", affected)
}
return nil
}
func (c *Catalog) clearVolatileOneDriveThumbnails(ctx context.Context) error {
// 把 OneDrive 过期的 mediap.svc.ms thumb URL 清空,让 worker 重新抽帧生成本地封面。
// 同步把 thumbnail_status 重置为 'pending':清空后 url 是空的,本应进 worker 重做,
+64
View File
@@ -1539,6 +1539,70 @@ func TestReconcileThumbnailStatusOnce(t *testing.T) {
}
}
func TestRequeueSkippedPreviews(t *testing.T) {
ctx := context.Background()
cat, err := Open(t.TempDir() + "/catalog.db")
if err != nil {
t.Fatalf("open: %v", err)
}
t.Cleanup(func() { cat.Close() })
now := time.Now()
cases := []struct {
id string
status string
local string
fileID string
wantStatus string
wantLocal string
wantFileID string
}{
{"preview-skipped", "skipped", "/tmp/old-preview.mp4", "old-preview-file", "pending", "", ""},
{"preview-ready", "ready", "/tmp/ready-preview.mp4", "ready-preview-file", "ready", "/tmp/ready-preview.mp4", "ready-preview-file"},
{"preview-failed", "failed", "/tmp/failed-preview.mp4", "failed-preview-file", "failed", "/tmp/failed-preview.mp4", "failed-preview-file"},
}
for _, c := range cases {
if err := cat.UpsertVideo(ctx, &Video{
ID: c.id, DriveID: "d", FileID: "source-" + c.id, Title: c.id,
PreviewStatus: c.status, PreviewLocal: c.local, PreviewFileID: c.fileID,
PublishedAt: now, CreatedAt: now, UpdatedAt: now,
}); err != nil {
t.Fatalf("seed %s: %v", c.id, err)
}
}
if err := cat.requeueSkippedPreviews(ctx); err != nil {
t.Fatalf("requeue skipped previews: %v", err)
}
if err := cat.requeueSkippedPreviews(ctx); err != nil {
t.Fatalf("second requeue skipped previews: %v", err)
}
for _, c := range cases {
got, err := cat.GetVideo(ctx, c.id)
if err != nil {
t.Fatalf("get %s: %v", c.id, err)
}
if got.PreviewStatus != c.wantStatus {
t.Errorf("%s: preview status = %q, want %q", c.id, got.PreviewStatus, c.wantStatus)
}
if got.PreviewLocal != c.wantLocal {
t.Errorf("%s: preview local = %q, want %q", c.id, got.PreviewLocal, c.wantLocal)
}
if got.PreviewFileID != c.wantFileID {
t.Errorf("%s: preview file id = %q, want %q", c.id, got.PreviewFileID, c.wantFileID)
}
}
pending, err := cat.ListVideosByPreviewStatus(ctx, "d", "pending", 0)
if err != nil {
t.Fatalf("list pending previews: %v", err)
}
if len(pending) != 1 || pending[0].ID != "preview-skipped" {
t.Fatalf("pending previews = %#v, want only preview-skipped", pending)
}
}
// TestUpsertVideoSyncsThumbnailStatus 验证 scanner 创建/补回视频时
// thumbnail_status 跟随 thumbnail_url 自动设。这是历史 bug 的修复回归测试 ——
// 之前 UpsertVideo 的 SQL 不带 thumbnail_status 列,所有新视频都依赖
+1 -1
View File
@@ -207,7 +207,7 @@ type Nightly struct {
// 这里保留 yaml 中的静态定义,用于启动时预置盘。生产建议只在 DB 里维护。
type Drive struct {
ID string `yaml:"id"`
Kind string `yaml:"kind"` // quark / p115 / p123 / pikpak / wopan / onedrive / googledrive / localstorage
Kind string `yaml:"kind"` // quark / p115 / p123 / pikpak / wopan / guangyapan / onedrive / googledrive / localstorage
Name string `yaml:"name"`
RootID string `yaml:"root_id"`
Params map[string]string `yaml:"params,omitempty"`
+4 -31
View File
@@ -647,7 +647,7 @@ func isGoogleUploadHTTPRateLimit(status int, header http.Header, body []byte, ap
if isGoogleRateLimit(nil, apiErr) {
return true
}
return googleLimitText(string(body))
return false
}
func googleUploadRateLimitError(status int, header http.Header, body []byte, message string) error {
@@ -910,7 +910,7 @@ func isGoogleRateLimit(res *resty.Response, body apiErrorBody) bool {
return true
}
for _, e := range body.Errors {
if googleLimitReason(e.Reason) || googleLimitText(e.Message) {
if googleLimitReason(e.Reason) {
return true
}
domain := compactGoogleLimitText(e.Domain)
@@ -918,7 +918,7 @@ func isGoogleRateLimit(res *resty.Response, body apiErrorBody) bool {
return true
}
}
return googleLimitText(body.Message)
return false
}
func isGoogleTokenRateLimit(res *resty.Response, out tokenResp) bool {
@@ -930,9 +930,7 @@ func isGoogleTokenRateLimit(res *resty.Response, out tokenResp) bool {
return true
}
}
return googleLimitText(out.Text) ||
googleLimitText(out.Error) ||
googleLimitText(out.ErrorDescription)
return googleLimitReason(out.Error)
}
func googleLimitReason(reason string) bool {
@@ -953,31 +951,6 @@ func googleLimitReason(reason string) bool {
}
}
func googleLimitText(text string) bool {
text = strings.ToLower(strings.TrimSpace(text))
if text == "" {
return false
}
compact := compactGoogleLimitText(text)
if strings.Contains(compact, "ratelimitexceeded") ||
strings.Contains(compact, "userratelimitexceeded") ||
strings.Contains(compact, "dailylimitexceeded") ||
strings.Contains(compact, "downloadquotaexceeded") ||
strings.Contains(compact, "sharingratelimitexceeded") ||
strings.Contains(compact, "quotaexceeded") ||
strings.Contains(compact, "toomanyrequests") {
return true
}
return strings.Contains(text, "rate limit") ||
strings.Contains(text, "too many requests") ||
strings.Contains(text, "quota exceeded") ||
strings.Contains(text, "download quota") ||
strings.Contains(text, "sharing rate") ||
strings.Contains(text, "daily limit") ||
strings.Contains(text, "user rate") ||
strings.Contains(text, "usage limit")
}
func compactGoogleLimitText(text string) string {
text = strings.ToLower(strings.TrimSpace(text))
replacer := strings.NewReplacer("_", "", "-", "", " ", "", ".", "", ":", "")
File diff suppressed because it is too large Load Diff
@@ -0,0 +1,300 @@
package guangyapan
import (
"context"
"encoding/json"
"errors"
"net/http"
"net/http/httptest"
"strings"
"testing"
"time"
"github.com/video-site/backend/internal/drives"
)
func TestDriverRefreshListAndStream(t *testing.T) {
var refreshed bool
var listedRoot bool
updates := map[string]string{}
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
switch r.URL.Path {
case "/v1/auth/token":
refreshed = true
writeTestJSON(w, map[string]any{
"access_token": "new-access",
"refresh_token": "new-refresh",
})
case "/v1/user/me":
if got := r.Header.Get("Authorization"); got != "Bearer new-access" {
t.Fatalf("auth header = %q, want new access token", got)
}
writeTestJSON(w, map[string]any{"sub": "user-1"})
case "/userres/v1/file/get_file_list":
if got := r.Header.Get("Authorization"); got != "Bearer new-access" {
t.Fatalf("api auth header = %q, want new access token", got)
}
var body map[string]any
if err := json.NewDecoder(r.Body).Decode(&body); err != nil {
t.Fatalf("decode list body: %v", err)
}
if body["parentId"] != "" {
t.Fatalf("parentId = %#v, want root empty string", body["parentId"])
}
listedRoot = true
writeTestJSON(w, map[string]any{
"code": 0,
"msg": "success",
"data": map[string]any{
"total": 2,
"list": []map[string]any{
{"fileId": "dir-1", "parentId": "", "fileName": "Movies", "resType": 2},
{"fileId": "file-1", "parentId": "", "fileName": "clip.mp4", "fileSize": 123, "resType": 1, "utime": 1700000000},
},
},
})
case "/nd.bizuserres.s/v1/get_res_download_url":
writeTestJSON(w, map[string]any{
"code": 0,
"msg": "success",
"data": map[string]any{"signedURL": "https://cdn.example.test/clip.mp4"},
})
default:
t.Fatalf("unexpected path %s", r.URL.Path)
}
}))
defer srv.Close()
d := New(Config{
ID: "gy",
RefreshToken: "old-refresh",
AccountBaseURL: srv.URL,
APIBaseURL: srv.URL,
OnCredentialsUpdate: func(values map[string]string) {
for k, v := range values {
updates[k] = v
}
},
})
if err := d.Init(context.Background()); err != nil {
t.Fatalf("init: %v", err)
}
if !refreshed {
t.Fatal("refresh token endpoint was not called")
}
if updates["access_token"] != "new-access" || updates["refresh_token"] != "new-refresh" {
t.Fatalf("updates = %#v, want refreshed tokens", updates)
}
entries, err := d.List(context.Background(), "")
if err != nil {
t.Fatalf("list: %v", err)
}
if !listedRoot || len(entries) != 2 {
t.Fatalf("listedRoot=%v entries=%#v", listedRoot, entries)
}
if !entries[0].IsDir || entries[1].ID != "file-1" || entries[1].Size != 123 {
t.Fatalf("entries = %#v", entries)
}
link, err := d.StreamURL(context.Background(), "file-1")
if err != nil {
t.Fatalf("stream url: %v", err)
}
if link.URL != "https://cdn.example.test/clip.mp4" {
t.Fatalf("stream url = %q", link.URL)
}
}
func TestDriverResolvesRootPath(t *testing.T) {
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
switch r.URL.Path {
case "/v1/user/me":
writeTestJSON(w, map[string]any{"sub": "user-1"})
case "/userres/v1/file/get_file_list":
var body map[string]any
if err := json.NewDecoder(r.Body).Decode(&body); err != nil {
t.Fatalf("decode list body: %v", err)
}
parent, _ := body["parentId"].(string)
switch parent {
case "":
writeTestJSON(w, listTestResponse([]map[string]any{
{"fileId": "folder-a", "parentId": "", "fileName": "影视", "resType": 2},
}))
case "folder-a":
writeTestJSON(w, listTestResponse([]map[string]any{
{"fileId": "folder-b", "parentId": "folder-a", "fileName": "电影", "resType": 2},
}))
case "folder-b":
writeTestJSON(w, listTestResponse([]map[string]any{
{"fileId": "file-1", "parentId": "folder-b", "fileName": "movie.mp4", "fileSize": 456, "resType": 1},
}))
default:
t.Fatalf("unexpected parent %q", parent)
}
default:
t.Fatalf("unexpected path %s", r.URL.Path)
}
}))
defer srv.Close()
d := New(Config{
ID: "gy",
RootID: "configured-root",
RootPath: "影视/电影",
AccessToken: "access",
AccountBaseURL: srv.URL,
APIBaseURL: srv.URL,
})
if err := d.Init(context.Background()); err != nil {
t.Fatalf("init: %v", err)
}
if d.RootID() != "folder-b" {
t.Fatalf("root id = %q, want folder-b", d.RootID())
}
entries, err := d.List(context.Background(), "")
if err != nil {
t.Fatalf("list resolved root: %v", err)
}
if len(entries) != 1 || entries[0].ID != "file-1" {
t.Fatalf("entries = %#v", entries)
}
}
func TestDriverSendSMSCodeUpdatesVerificationState(t *testing.T) {
updates := map[string]string{}
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
switch r.URL.Path {
case "/v1/shield/captcha/init":
writeTestJSON(w, map[string]any{"captcha_token": "captcha-1"})
case "/v1/auth/verification":
writeTestJSON(w, map[string]any{"verification_id": "verify-1"})
default:
t.Fatalf("unexpected path %s", r.URL.Path)
}
}))
defer srv.Close()
d := New(Config{
ID: "gy",
PhoneNumber: "13800000000",
SendCode: true,
AccountBaseURL: srv.URL,
APIBaseURL: srv.URL,
OnCredentialsUpdate: func(values map[string]string) {
for k, v := range values {
updates[k] = v
}
},
})
err := d.Init(context.Background())
if err == nil || !strings.Contains(err.Error(), "验证码已发送") {
t.Fatalf("init err = %v, want verification prompt", err)
}
if updates["captcha_token"] != "captcha-1" || updates["verification_id"] != "verify-1" || updates["send_code"] != "false" {
t.Fatalf("updates = %#v, want sms state saved", updates)
}
if updates["device_id"] == "" {
t.Fatalf("updates = %#v, want generated device id saved", updates)
}
}
func TestListHTTP429ReturnsRateLimitError(t *testing.T) {
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
if r.URL.Path != "/userres/v1/file/get_file_list" {
t.Fatalf("unexpected path %s", r.URL.Path)
}
w.Header().Set("Retry-After", "120")
w.WriteHeader(http.StatusTooManyRequests)
writeTestJSON(w, map[string]any{"code": 429, "msg": "操作频繁,请稍后重试"})
}))
defer srv.Close()
d := New(Config{
ID: "gy",
AccessToken: "access",
AccountBaseURL: srv.URL,
APIBaseURL: srv.URL,
})
_, err := d.List(context.Background(), "")
if err == nil {
t.Fatal("list succeeded, want rate limit error")
}
var rateLimit *drives.RateLimitError
if !errors.As(err, &rateLimit) {
t.Fatalf("error = %T %[1]v, want RateLimitError", err)
}
if rateLimit.RetryAfter != 2*time.Minute {
t.Fatalf("retry after = %s, want 2m", rateLimit.RetryAfter)
}
}
func TestListCode429ReturnsRateLimitError(t *testing.T) {
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
if r.URL.Path != "/userres/v1/file/get_file_list" {
t.Fatalf("unexpected path %s", r.URL.Path)
}
writeTestJSON(w, map[string]any{"code": 429, "msg": "操作频繁,请稍后再试"})
}))
defer srv.Close()
d := New(Config{
ID: "gy",
AccessToken: "access",
AccountBaseURL: srv.URL,
APIBaseURL: srv.URL,
})
_, err := d.List(context.Background(), "")
if err == nil {
t.Fatal("list succeeded, want rate limit error")
}
var rateLimit *drives.RateLimitError
if !errors.As(err, &rateLimit) {
t.Fatalf("error = %T %[1]v, want RateLimitError", err)
}
}
func TestListInvalidToken403DoesNotReturnRateLimitError(t *testing.T) {
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
if r.URL.Path != "/userres/v1/file/get_file_list" {
t.Fatalf("unexpected path %s", r.URL.Path)
}
w.WriteHeader(http.StatusForbidden)
writeTestJSON(w, map[string]any{"code": 401, "msg": "invalid access token"})
}))
defer srv.Close()
d := New(Config{
ID: "gy",
AccessToken: "access",
AccountBaseURL: srv.URL,
APIBaseURL: srv.URL,
})
_, err := d.List(context.Background(), "")
if err == nil {
t.Fatal("list succeeded, want auth error")
}
var rateLimit *drives.RateLimitError
if errors.As(err, &rateLimit) {
t.Fatalf("error = %T %[1]v, want non-rate-limit error", err)
}
}
func listTestResponse(items []map[string]any) map[string]any {
return map[string]any{
"code": 0,
"msg": "success",
"data": map[string]any{
"total": len(items),
"list": items,
},
}
}
func writeTestJSON(w http.ResponseWriter, v any) {
w.Header().Set("Content-Type", "application/json")
if err := json.NewEncoder(w).Encode(v); err != nil {
panic(err)
}
}
+244
View File
@@ -0,0 +1,244 @@
package guangyapan
import (
"context"
"encoding/base64"
"encoding/json"
"errors"
"fmt"
"net/http"
"strings"
"time"
"github.com/go-resty/resty/v2"
"github.com/skip2/go-qrcode"
)
const (
defaultQRScope = "user"
deviceCodeGrantType = "urn:ietf:params:oauth:grant-type:device_code"
defaultQRUserAgent = "GuangYaPan-Login/1.0"
)
type QRConfig struct {
AccountBaseURL string
HTTPClient *http.Client
Now func() time.Time
}
type QRClient struct {
accountBaseURL string
client *resty.Client
now func() time.Time
}
type QRCodeSession struct {
DeviceCode string `json:"deviceCode"`
QRCodeURL string `json:"qrCodeUrl"`
QRImageDataURL string `json:"qrImageDataUrl"`
IntervalSeconds int `json:"intervalSeconds"`
ExpiresAt string `json:"expiresAt,omitempty"`
}
type QRCodeStatus struct {
State string `json:"state"`
StatusText string `json:"statusText"`
IntervalSeconds int `json:"intervalSeconds,omitempty"`
AccessToken string `json:"accessToken,omitempty"`
RefreshToken string `json:"refreshToken,omitempty"`
TokenType string `json:"tokenType,omitempty"`
ExpiresIn int64 `json:"expiresIn,omitempty"`
}
type deviceCodeResp struct {
DeviceCode string `json:"device_code"`
VerificationURIComplete string `json:"verification_uri_complete"`
ShortURIComplete string `json:"short_uri_complete"`
Interval int `json:"interval"`
ExpiresIn int `json:"expires_in"`
Error string `json:"error"`
ErrorCode int `json:"error_code"`
ErrorDesc string `json:"error_description"`
}
type deviceTokenResp struct {
AccessToken string `json:"access_token"`
RefreshToken string `json:"refresh_token"`
TokenType string `json:"token_type"`
ExpiresIn int64 `json:"expires_in"`
Scope string `json:"scope"`
Error string `json:"error"`
ErrorCode int `json:"error_code"`
ErrorDesc string `json:"error_description"`
}
func NewQRClient(c QRConfig) *QRClient {
accountBaseURL := strings.TrimRight(strings.TrimSpace(c.AccountBaseURL), "/")
if accountBaseURL == "" {
accountBaseURL = defaultAccountBaseURL
}
httpClient := c.HTTPClient
if httpClient == nil {
httpClient = &http.Client{Timeout: 20 * time.Second}
}
now := c.Now
if now == nil {
now = time.Now
}
return &QRClient{
accountBaseURL: accountBaseURL,
client: resty.NewWithClient(httpClient).
SetTimeout(20*time.Second).
SetBaseURL(accountBaseURL).
SetHeader("User-Agent", defaultQRUserAgent).
SetHeader("Accept", "application/json").
SetHeader("Content-Type", "application/json"),
now: now,
}
}
func (c *QRClient) Generate(ctx context.Context) (QRCodeSession, error) {
var out deviceCodeResp
var errOut deviceCodeResp
resp, err := c.client.R().
SetContext(ctx).
SetBody(map[string]any{
"client_id": defaultClientID,
"scope": defaultQRScope,
}).
SetResult(&out).
SetError(&errOut).
Post("/v1/auth/device/code")
if err != nil {
return QRCodeSession{}, err
}
if resp.IsError() || out.Error != "" {
if out.Error == "" {
out = errOut
}
return QRCodeSession{}, fmt.Errorf("guangyapan qr: %s", deviceAPIError(out.ErrorDesc, out.Error, resp))
}
deviceCode := strings.TrimSpace(out.DeviceCode)
if deviceCode == "" {
return QRCodeSession{}, errors.New("guangyapan qr: empty device_code")
}
qrURL := strings.TrimSpace(out.VerificationURIComplete)
if qrURL == "" {
qrURL = strings.TrimSpace(out.ShortURIComplete)
}
if qrURL == "" {
return QRCodeSession{}, errors.New("guangyapan qr: empty verification uri")
}
interval := out.Interval
if interval <= 0 {
interval = 5
}
expiresIn := out.ExpiresIn
if expiresIn <= 0 {
expiresIn = 300
}
png, err := qrcode.Encode(qrURL, qrcode.Medium, 220)
if err != nil {
return QRCodeSession{}, err
}
return QRCodeSession{
DeviceCode: deviceCode,
QRCodeURL: qrURL,
QRImageDataURL: "data:image/png;base64," + base64.StdEncoding.EncodeToString(png),
IntervalSeconds: interval,
ExpiresAt: c.now().Add(time.Duration(expiresIn) * time.Second).Format(time.RFC3339),
}, nil
}
func (c *QRClient) Poll(ctx context.Context, deviceCode string) (QRCodeStatus, error) {
deviceCode = strings.TrimSpace(deviceCode)
if deviceCode == "" {
return QRCodeStatus{}, errors.New("deviceCode is required")
}
var out deviceTokenResp
var errOut deviceTokenResp
resp, err := c.client.R().
SetContext(ctx).
SetBody(map[string]any{
"client_id": defaultClientID,
"grant_type": deviceCodeGrantType,
"device_code": deviceCode,
}).
SetResult(&out).
SetError(&errOut).
Post("/v1/auth/token")
if err != nil {
return QRCodeStatus{}, err
}
if resp.IsError() && out.Error == "" {
out = errOut
}
if resp.IsError() && out.Error == "" {
_ = json.Unmarshal(resp.Body(), &out)
}
if out.Error != "" {
return qrStatusForDeviceError(out), nil
}
if resp.IsError() {
return QRCodeStatus{}, fmt.Errorf("guangyapan qr: status=%d body=%s", resp.StatusCode(), resp.String())
}
access := strings.TrimSpace(out.AccessToken)
refresh := strings.TrimSpace(out.RefreshToken)
if access == "" || refresh == "" {
return QRCodeStatus{}, errors.New("guangyapan qr: login succeeded but token response is incomplete")
}
tokenType := strings.TrimSpace(out.TokenType)
if tokenType == "" {
tokenType = "Bearer"
}
return QRCodeStatus{
State: "success",
StatusText: "登录成功",
AccessToken: access,
RefreshToken: refresh,
TokenType: tokenType,
ExpiresIn: out.ExpiresIn,
}, nil
}
func qrStatusForDeviceError(out deviceTokenResp) QRCodeStatus {
errCode := strings.TrimSpace(out.Error)
switch errCode {
case "authorization_pending":
return QRCodeStatus{State: "pending", StatusText: "等待扫码确认"}
case "slow_down":
return QRCodeStatus{State: "pending", StatusText: "等待扫码确认,已降低查询频率", IntervalSeconds: 10}
case "expired_token":
return QRCodeStatus{State: "expired", StatusText: "二维码已过期"}
case "access_denied":
return QRCodeStatus{State: "denied", StatusText: "用户拒绝了授权"}
default:
msg := strings.TrimSpace(out.ErrorDesc)
if msg == "" {
msg = errCode
}
if msg == "" {
msg = "未知错误"
}
return QRCodeStatus{State: "error", StatusText: msg}
}
}
func deviceAPIError(desc, short string, resp *resty.Response) string {
msg := strings.TrimSpace(desc)
if msg == "" {
msg = strings.TrimSpace(short)
}
if msg == "" && resp != nil {
msg = strings.TrimSpace(resp.String())
}
if msg == "" && resp != nil {
msg = fmt.Sprintf("status=%d", resp.StatusCode())
}
if msg == "" {
msg = "unknown error"
}
return msg
}
@@ -0,0 +1,102 @@
package guangyapan
import (
"context"
"encoding/json"
"net/http"
"net/http/httptest"
"strings"
"testing"
"time"
)
func TestQRClientGenerate(t *testing.T) {
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
if r.URL.Path != "/v1/auth/device/code" {
t.Fatalf("path = %s, want device code endpoint", r.URL.Path)
}
var body map[string]any
if err := json.NewDecoder(r.Body).Decode(&body); err != nil {
t.Fatalf("decode body: %v", err)
}
if body["client_id"] != defaultClientID || body["scope"] != defaultQRScope {
t.Fatalf("body = %#v", body)
}
writeTestJSON(w, map[string]any{
"device_code": "device-1",
"verification_uri_complete": "https://account.guangyapan.com/device?code=abc",
"interval": 7,
"expires_in": 180,
})
}))
defer srv.Close()
client := NewQRClient(QRConfig{
AccountBaseURL: srv.URL,
Now: func() time.Time { return time.Unix(1700000000, 0) },
})
session, err := client.Generate(context.Background())
if err != nil {
t.Fatalf("generate: %v", err)
}
if session.DeviceCode != "device-1" || session.QRCodeURL != "https://account.guangyapan.com/device?code=abc" {
t.Fatalf("session = %#v", session)
}
if session.IntervalSeconds != 7 {
t.Fatalf("interval = %d, want 7", session.IntervalSeconds)
}
if session.ExpiresAt != time.Unix(1700000180, 0).Format(time.RFC3339) {
t.Fatalf("expiresAt = %q", session.ExpiresAt)
}
if !strings.HasPrefix(session.QRImageDataURL, "data:image/png;base64,") {
t.Fatalf("qr image = %q", session.QRImageDataURL)
}
}
func TestQRClientPollPendingAndSuccess(t *testing.T) {
var calls int
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
if r.URL.Path != "/v1/auth/token" {
t.Fatalf("path = %s, want token endpoint", r.URL.Path)
}
var body map[string]any
if err := json.NewDecoder(r.Body).Decode(&body); err != nil {
t.Fatalf("decode body: %v", err)
}
if body["client_id"] != defaultClientID ||
body["grant_type"] != deviceCodeGrantType ||
body["device_code"] != "device-1" {
t.Fatalf("body = %#v", body)
}
calls++
if calls == 1 {
w.WriteHeader(http.StatusBadRequest)
writeTestJSON(w, map[string]any{"error": "authorization_pending"})
return
}
writeTestJSON(w, map[string]any{
"access_token": "access-1",
"refresh_token": "refresh-1",
"token_type": "Bearer",
"expires_in": 7200,
})
}))
defer srv.Close()
client := NewQRClient(QRConfig{AccountBaseURL: srv.URL})
pending, err := client.Poll(context.Background(), "device-1")
if err != nil {
t.Fatalf("poll pending: %v", err)
}
if pending.State != "pending" || pending.AccessToken != "" {
t.Fatalf("pending = %#v", pending)
}
success, err := client.Poll(context.Background(), "device-1")
if err != nil {
t.Fatalf("poll success: %v", err)
}
if success.State != "success" || success.AccessToken != "access-1" || success.RefreshToken != "refresh-1" {
t.Fatalf("success = %#v", success)
}
}
+129
View File
@@ -0,0 +1,129 @@
package guangyapan
import "time"
type tokenResp struct {
AccessToken string `json:"access_token"`
RefreshToken string `json:"refresh_token"`
Error string `json:"error"`
ErrorCode int `json:"error_code"`
ErrorDesc string `json:"error_description"`
}
type verificationResp struct {
VerificationID string `json:"verification_id"`
Error string `json:"error"`
ErrorCode int `json:"error_code"`
ErrorDesc string `json:"error_description"`
}
type captchaInitResp struct {
CaptchaToken string `json:"captcha_token"`
Error string `json:"error"`
ErrorCode int `json:"error_code"`
ErrorDesc string `json:"error_description"`
}
type verifyResp struct {
VerificationToken string `json:"verification_token"`
Error string `json:"error"`
ErrorCode int `json:"error_code"`
ErrorDesc string `json:"error_description"`
}
type userMeResp struct {
Sub string `json:"sub"`
}
type listResp struct {
Code int `json:"code"`
Msg string `json:"msg"`
Data struct {
Total int `json:"total"`
List []fileItem `json:"list"`
} `json:"data"`
}
type fileItem struct {
FileID string `json:"fileId"`
ParentID string `json:"parentId"`
FileName string `json:"fileName"`
FileSize int64 `json:"fileSize"`
ResType int `json:"resType"`
CTime int64 `json:"ctime"`
UTime int64 `json:"utime"`
}
type downloadResp struct {
Code int `json:"code"`
Msg string `json:"msg"`
Data struct {
SignedURL string `json:"signedURL"`
DownloadURL string `json:"downloadUrl"`
} `json:"data"`
}
type createDirResp struct {
Code int `json:"code"`
Msg string `json:"msg"`
Data struct {
FileID string `json:"fileId"`
FileName string `json:"fileName"`
ResType int `json:"resType"`
CTime int64 `json:"ctime"`
UTime int64 `json:"utime"`
} `json:"data"`
}
type deleteResp struct {
Code int `json:"code"`
Msg string `json:"msg"`
Data struct {
TaskID string `json:"taskId"`
} `json:"data"`
}
type taskStatusResp struct {
Code int `json:"code"`
Msg string `json:"msg"`
Data struct {
Status int `json:"status"`
} `json:"data"`
}
type uploadTokenResp struct {
Code int `json:"code"`
Msg string `json:"msg"`
Data uploadTokenData `json:"data"`
}
type uploadTokenData struct {
TaskID string `json:"taskId"`
ObjectPath string `json:"objectPath"`
BucketName string `json:"bucketName"`
EndPoint string `json:"endPoint"`
FullEndPoint string `json:"fullEndPoint"`
AccessKeyID string `json:"accessKeyID"`
SecretAccessKey string `json:"secretAccessKey"`
SessionToken string `json:"sessionToken"`
Creds struct {
AccessKeyID string `json:"accessKeyID"`
SecretAccessKey string `json:"secretAccessKey"`
SessionToken string `json:"sessionToken"`
} `json:"creds"`
}
type taskInfoResp struct {
Code int `json:"code"`
Msg string `json:"msg"`
Data struct {
FileID string `json:"fileId"`
} `json:"data"`
}
func unixOrZero(v int64) time.Time {
if v <= 0 {
return time.Time{}
}
return time.Unix(v, 0)
}
+42 -1
View File
@@ -5,12 +5,14 @@ import (
"errors"
"io"
"net/http"
"strconv"
"strings"
"time"
)
// Drive 是多家网盘统一抽象。上层不区分盘,只区分 Kind。
type Drive interface {
// Kind 返回驱动代号:"quark" / "p115" / "p123" / "pikpak" / "wopan" / "onedrive" / "googledrive" / "localstorage"
// Kind 返回驱动代号:"quark" / "p115" / "p123" / "pikpak" / "wopan" / "guangyapan" / "onedrive" / "googledrive" / "localstorage"
Kind() string
// ID 返回该盘在 catalog 中的唯一标识
@@ -119,3 +121,42 @@ func RateLimitRetryAfter(err error) (time.Duration, bool) {
}
return 0, false
}
// TextMentionsHTTPStatus only looks for explicit numeric HTTP status contexts
// in errors from tools that do not expose structured response metadata.
func TextMentionsHTTPStatus(text string, statuses ...int) bool {
text = strings.ToLower(strings.TrimSpace(text))
if text == "" {
return false
}
for _, status := range statuses {
if status <= 0 {
continue
}
code := strconv.Itoa(status)
if strings.HasPrefix(text, code+" ") ||
strings.Contains(text, "status="+code) ||
strings.Contains(text, "status: "+code) ||
strings.Contains(text, "status "+code) ||
strings.Contains(text, "status code "+code) ||
strings.Contains(text, "http "+code) ||
strings.Contains(text, "http status="+code) ||
strings.Contains(text, "http status: "+code) ||
strings.Contains(text, "http status "+code) ||
strings.Contains(text, "server returned "+code) ||
strings.Contains(text, "code="+code) ||
strings.Contains(text, "code: "+code) ||
strings.Contains(text, "error_code="+code) ||
strings.Contains(text, "error_code: "+code) {
return true
}
}
return false
}
func ErrorMentionsHTTPStatus(err error, statuses ...int) bool {
if err == nil {
return false
}
return TextMentionsHTTPStatus(err.Error(), statuses...)
}
+24
View File
@@ -0,0 +1,24 @@
package drives
import "testing"
func TestTextMentionsHTTPStatus(t *testing.T) {
tests := []struct {
name string
text string
want bool
}{
{name: "status context", text: "request failed with status: 429 Too Many Requests", want: true},
{name: "http context", text: "http 503 service unavailable", want: true},
{name: "server returned context", text: "Server returned 403 Forbidden", want: true},
{name: "message only", text: "操作频繁,请稍后重试", want: false},
{name: "unrelated number", text: "generated 429 bytes", want: false},
}
for _, tc := range tests {
t.Run(tc.name, func(t *testing.T) {
if got := TextMentionsHTTPStatus(tc.text, 403, 429, 503); got != tc.want {
t.Fatalf("TextMentionsHTTPStatus(%q) = %v, want %v", tc.text, got, tc.want)
}
})
}
}
+2 -14
View File
@@ -594,8 +594,8 @@ func (d *Driver) refresh(ctx context.Context) error {
return nil
}
func isRateLimitResponse(res *resty.Response, code, message string) bool {
if isRateLimitCode(code) || isRateLimitMessage(message) {
func isRateLimitResponse(res *resty.Response, code, _ string) bool {
if isRateLimitCode(code) {
return true
}
if res == nil {
@@ -632,18 +632,6 @@ func isRateLimitCode(code string) bool {
}
}
func isRateLimitMessage(message string) bool {
text := strings.ToLower(strings.TrimSpace(message))
if text == "" {
return false
}
return strings.Contains(text, "too many requests") ||
strings.Contains(text, "throttl") ||
strings.Contains(text, "rate limit") ||
strings.Contains(text, "activity limit") ||
strings.Contains(text, "temporarily blocked")
}
func onedriveRateLimitError(res *resty.Response, message string) error {
if strings.TrimSpace(message) == "" {
message = "onedrive rate limited"
@@ -214,7 +214,7 @@ func TestGraph429ReturnsRateLimitErrorWithRetryAfter(t *testing.T) {
}
}
func TestGraphThrottleMessageReturnsRateLimitError(t *testing.T) {
func TestGraphThrottleMessageDoesNotReturnRateLimitError(t *testing.T) {
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
w.Header().Set("Content-Type", "application/json")
w.WriteHeader(http.StatusForbidden)
@@ -238,11 +238,11 @@ func TestGraphThrottleMessageReturnsRateLimitError(t *testing.T) {
_, err := d.StreamURL(context.Background(), "file-id")
if err == nil {
t.Fatal("list succeeded, want rate limit error")
t.Fatal("list succeeded, want graph error")
}
var rateLimit *drives.RateLimitError
if !errors.As(err, &rateLimit) {
t.Fatalf("error = %T %[1]v, want RateLimitError", err)
if errors.As(err, &rateLimit) {
t.Fatalf("error = %T %[1]v, want non-rate-limit error", err)
}
}
+28 -29
View File
@@ -20,11 +20,12 @@ import (
)
type Driver struct {
id string
cookie string
rootID string
client *sdk.Pan115Client
ua string
id string
cookie string
rootID string
client *sdk.Pan115Client
ua string
uploadTempDir string
listMu sync.Mutex
lastListAt time.Time
@@ -32,10 +33,11 @@ type Driver struct {
}
type Config struct {
ID string
Cookie string // 形如 "UID=xxx; CID=xxx; SEID=xxx; KID=xxx"
RootID string // 默认 "0"
UA string // 默认 UA115Browser
ID string
Cookie string // 形如 "UID=xxx; CID=xxx; SEID=xxx; KID=xxx"
RootID string // 默认 "0"
UA string // 默认 UA115Browser
UploadTempDir string
}
func New(c Config) *Driver {
@@ -48,11 +50,12 @@ func New(c Config) *Driver {
ua = sdk.UA115Browser
}
return &Driver{
id: c.ID,
cookie: c.Cookie,
rootID: rootID,
ua: ua,
listInterval: 2 * time.Second,
id: c.ID,
cookie: c.Cookie,
rootID: rootID,
ua: ua,
uploadTempDir: strings.TrimSpace(c.UploadTempDir),
listInterval: 2 * time.Second,
}
}
@@ -87,7 +90,7 @@ func (d *Driver) List(ctx context.Context, dirID string) ([]drives.Entry, error)
// p115ListCooldown 是列目录触发疑似风控错误时的冷却时长。
//
// 历史上是 [30min × 3],3 次都失败就放弃;新策略改为 10 分钟无限重试 ——
// 只要错误仍属 transient429 / 405 / WAF / blocked / 安全威胁 / unexpected),
// 只要错误仍属明确 HTTP transient 状态429 / 405),
// 就持续等 10 分钟再发一次列目录请求,直到成功或 ctx 取消。这样即使 115
// 风控持续较长时间,扫描会自然延后到风控结束,不再丢半棵子树。
const p115ListCooldown = 10 * time.Minute
@@ -156,17 +159,7 @@ func isTransient115UpstreamError(err error) bool {
if err == nil {
return false
}
text := strings.ToLower(err.Error())
return strings.Contains(text, "405") ||
strings.Contains(text, "429") ||
strings.Contains(text, "too many request") ||
strings.Contains(text, "too many requests") ||
strings.Contains(text, "blocked") ||
strings.Contains(text, "security") ||
strings.Contains(text, "waf") ||
strings.Contains(text, "unexpected error") ||
strings.Contains(text, "访问被阻断") ||
strings.Contains(text, "安全威胁")
return drives.ErrorMentionsHTTPStatus(err, http.StatusMethodNotAllowed, http.StatusTooManyRequests)
}
// ListDirsOnly 只列指定目录的直接**子目录**,不返回文件条目。专为 admin 后台
@@ -357,7 +350,7 @@ func (d *Driver) UploadAndReportSha1(ctx context.Context, parentID, name string,
parentID = d.rootID
}
tmp, sha1Hex, written, err := bufferAndHashSha1(r, size)
tmp, sha1Hex, written, err := bufferAndHashSha1(d.uploadTempDir, r, size)
if err != nil {
return UploadResult{}, err
}
@@ -482,8 +475,14 @@ func (d *Driver) Remove(ctx context.Context, fileID string) error {
// 返回临时文件(位置在末尾,需调用方 Seek 回 0)、SHA1 hex 大写、实际字节数。
//
// 调用方负责 Close + Remove 临时文件。
func bufferAndHashSha1(r io.Reader, declaredSize int64) (*os.File, string, int64, error) {
tmp, err := os.CreateTemp("", "p115-upload-*.bin")
func bufferAndHashSha1(tempDir string, r io.Reader, declaredSize int64) (*os.File, string, int64, error) {
tempDir = strings.TrimSpace(tempDir)
if tempDir != "" {
if err := os.MkdirAll(tempDir, 0o755); err != nil {
return nil, "", 0, fmt.Errorf("p115 upload: create tmp dir: %w", err)
}
}
tmp, err := os.CreateTemp(tempDir, "p115-upload-*.bin")
if err != nil {
return nil, "", 0, fmt.Errorf("p115 upload: create tmp: %w", err)
}
+21 -7
View File
@@ -8,6 +8,7 @@ import (
"errors"
"io"
"os"
"path/filepath"
"strings"
"testing"
"time"
@@ -22,8 +23,9 @@ func TestIsTransient115ListError(t *testing.T) {
want bool
}{
{name: "nil", err: nil, want: false},
{name: "blocked html", err: errors.New(`<!doctype html><title>405</title>Sorry, your request has been blocked as it may cause potential threats to the server's security.`), want: true},
{name: "chinese waf", err: errors.New("很抱歉,由于您访问的URL有可能对网站造成安全威胁,您的访问被阻断。"), want: true},
{name: "blocked html without status context", err: errors.New(`<!doctype html><title>405</title>Sorry, your request has been blocked as it may cause potential threats to the server's security.`), want: false},
{name: "chinese waf", err: errors.New("很抱歉,由于您访问的URL有可能对网站造成安全威胁,您的访问被阻断。"), want: false},
{name: "status 405", err: errors.New("request failed with status: 405"), want: true},
{name: "rate limit", err: errors.New("429 too many requests"), want: true},
{name: "regular auth error", err: errors.New("invalid credential"), want: false},
}
@@ -43,10 +45,10 @@ func TestWrap115StreamTransientError(t *testing.T) {
err error
wantRateLimit bool
}{
{name: "unexpected", err: errors.New("unexpected error"), wantRateLimit: true},
{name: "unexpected", err: errors.New("unexpected error"), wantRateLimit: false},
{name: "405 blocked", err: errors.New("405 request has been blocked"), wantRateLimit: true},
{name: "429", err: errors.New("429 too many requests"), wantRateLimit: true},
{name: "blocked", err: errors.New("blocked by waf"), wantRateLimit: true},
{name: "blocked", err: errors.New("blocked by waf"), wantRateLimit: false},
{name: "auth", err: errors.New("invalid credential"), wantRateLimit: false},
}
@@ -85,7 +87,7 @@ func TestBufferAndHashSha1(t *testing.T) {
wantHex := strings.ToUpper(hex.EncodeToString(want[:]))
t.Run("declared size matches", func(t *testing.T) {
tmp, gotHex, n, err := bufferAndHashSha1(bytes.NewReader(body), int64(len(body)))
tmp, gotHex, n, err := bufferAndHashSha1("", bytes.NewReader(body), int64(len(body)))
if err != nil {
t.Fatalf("bufferAndHashSha1 returned error: %v", err)
}
@@ -110,14 +112,14 @@ func TestBufferAndHashSha1(t *testing.T) {
})
t.Run("declared size mismatch returns error", func(t *testing.T) {
_, _, _, err := bufferAndHashSha1(bytes.NewReader(body), int64(len(body))+1)
_, _, _, err := bufferAndHashSha1("", bytes.NewReader(body), int64(len(body))+1)
if err == nil {
t.Fatal("expected size mismatch error, got nil")
}
})
t.Run("declared size zero is unchecked", func(t *testing.T) {
tmp, gotHex, n, err := bufferAndHashSha1(bytes.NewReader(body), 0)
tmp, gotHex, n, err := bufferAndHashSha1("", bytes.NewReader(body), 0)
if err != nil {
t.Fatalf("bufferAndHashSha1 returned error: %v", err)
}
@@ -129,6 +131,18 @@ func TestBufferAndHashSha1(t *testing.T) {
t.Errorf("written = %d, want %d", n, len(body))
}
})
t.Run("uses configured temp dir", func(t *testing.T) {
tempDir := filepath.Join(t.TempDir(), "upload-tmp")
tmp, _, _, err := bufferAndHashSha1(tempDir, bytes.NewReader(body), int64(len(body)))
if err != nil {
t.Fatalf("bufferAndHashSha1 returned error: %v", err)
}
defer cleanup(tmp)
if gotDir := filepath.Dir(tmp.Name()); gotDir != tempDir {
t.Fatalf("tmp dir = %q, want %q", gotDir, tempDir)
}
})
}
// TestUploadAndReportSha1RejectsInvalidArgs 检查空 reader / 空 name / 负 size 在
+15 -32
View File
@@ -70,6 +70,7 @@ type Driver struct {
httpClient *http.Client
onTokenUpdate func(access string)
uploadTempDir string
tokenMu sync.RWMutex
@@ -90,6 +91,7 @@ type Config struct {
MainAPIBaseURL string
LoginAPIBaseURL string
UploadTempDir string
OnTokenUpdate func(access string)
}
@@ -123,6 +125,7 @@ func New(c Config) *Driver {
referer: defaultReferer,
userAgent: defaultUserAgent,
onTokenUpdate: c.OnTokenUpdate,
uploadTempDir: strings.TrimSpace(c.UploadTempDir),
client: resty.New().
SetTimeout(30*time.Second).
SetHeader("Accept", "application/json, text/plain, */*"),
@@ -289,7 +292,7 @@ func (d *Driver) UploadAndReportHash(ctx context.Context, parentID, name string,
parentID = d.rootID
}
tmp, md5Hex, actualSize, err := bufferAndHashMD5(r, size)
tmp, md5Hex, actualSize, err := bufferAndHashMD5(d.uploadTempDir, r, size)
if err != nil {
return UploadResult{}, err
}
@@ -754,8 +757,8 @@ func (d *Driver) request(ctx context.Context, endpoint, method string, configure
return nil, errors.New("123pan request: unauthorized")
}
func isP123RateLimitResponse(res *resty.Response, code int, message string) bool {
if code == http.StatusTooManyRequests || isP123RateLimitMessage(message) {
func isP123RateLimitResponse(res *resty.Response, code int, _ string) bool {
if code == http.StatusTooManyRequests {
return true
}
if res == nil {
@@ -764,7 +767,7 @@ func isP123RateLimitResponse(res *resty.Response, code int, message string) bool
return isP123RateLimitHTTPResponse(res.StatusCode(), res.Header().Get("Retry-After"), res.String())
}
func isP123RateLimitHTTPResponse(status int, retryAfter, body string) bool {
func isP123RateLimitHTTPResponse(status int, retryAfter, _ string) bool {
if status == http.StatusTooManyRequests {
return true
}
@@ -774,35 +777,9 @@ func isP123RateLimitHTTPResponse(status int, retryAfter, body string) bool {
return true
}
}
if isP123RateLimitMessage(body) {
return true
}
return false
}
func isP123RateLimitMessage(message string) bool {
text := strings.ToLower(strings.TrimSpace(message))
if text == "" {
return false
}
return strings.Contains(text, "请求太频繁") ||
strings.Contains(text, "请求过于频繁") ||
strings.Contains(text, "请求频繁") ||
strings.Contains(text, "操作频繁") ||
strings.Contains(text, "频率限制") ||
strings.Contains(text, "请求次数过多") ||
strings.Contains(text, "too many request") ||
strings.Contains(text, "too many requests") ||
strings.Contains(text, "rate limit") ||
strings.Contains(text, "rate-limit") ||
strings.Contains(text, "ratelimit") ||
strings.Contains(text, "throttl") ||
strings.Contains(text, "temporarily blocked") ||
strings.Contains(text, "request has been blocked") ||
strings.Contains(text, "blocked") ||
strings.Contains(text, "访问被阻断")
}
func p123RateLimitError(res *resty.Response, code int, message string) error {
if strings.TrimSpace(message) == "" {
message = "123pan rate limited"
@@ -1084,8 +1061,14 @@ func splitPath(p string) []string {
return strings.Split(p, "/")
}
func bufferAndHashMD5(r io.Reader, declaredSize int64) (*os.File, string, int64, error) {
tmp, err := os.CreateTemp("", "p123-upload-*.bin")
func bufferAndHashMD5(tempDir string, r io.Reader, declaredSize int64) (*os.File, string, int64, error) {
tempDir = strings.TrimSpace(tempDir)
if tempDir != "" {
if err := os.MkdirAll(tempDir, 0o755); err != nil {
return nil, "", 0, fmt.Errorf("123pan upload: create tmp dir: %w", err)
}
}
tmp, err := os.CreateTemp(tempDir, "p123-upload-*.bin")
if err != nil {
return nil, "", 0, fmt.Errorf("123pan upload: create tmp: %w", err)
}
@@ -11,6 +11,8 @@ import (
"io"
"net/http"
"net/http/httptest"
"os"
"path/filepath"
"strings"
"testing"
"time"
@@ -458,6 +460,29 @@ func TestUploadPresignedPUT429ReturnsRateLimitError(t *testing.T) {
}
}
func TestBufferAndHashMD5UsesConfiguredTempDir(t *testing.T) {
body := []byte("hello-123-upload-test")
tempDir := filepath.Join(t.TempDir(), "upload-tmp")
tmp, gotHex, n, err := bufferAndHashMD5(tempDir, bytes.NewReader(body), int64(len(body)))
if err != nil {
t.Fatalf("bufferAndHashMD5 returned error: %v", err)
}
defer func() {
_ = tmp.Close()
_ = os.Remove(tmp.Name())
}()
if gotDir := filepath.Dir(tmp.Name()); gotDir != tempDir {
t.Fatalf("tmp dir = %q, want %q", gotDir, tempDir)
}
want := md5.Sum(body)
if gotHex != fmt.Sprintf("%x", want) {
t.Fatalf("md5 = %s, want %x", gotHex, want)
}
if n != int64(len(body)) {
t.Fatalf("written = %d, want %d", n, len(body))
}
}
func TestRenameSendsExpectedBody(t *testing.T) {
var renameRequest map[string]any
api := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
+13 -19
View File
@@ -47,6 +47,7 @@ type Driver struct {
client *resty.Client
onTokenUpdate func(access, refresh, captcha, deviceID string)
uploadToOSSFunc func(context.Context, *s3Params, io.Reader) error
uploadTempDir string
// captchaMu serializes captcha-token refreshes triggered by 4002 / 9
// recovery in requestOnce. Without it, N concurrent callers all hitting
@@ -77,6 +78,7 @@ type Config struct {
DeviceID string
RootID string
DisableMediaLink bool
UploadTempDir string
OnTokenUpdate func(access, refresh, captcha, deviceID string)
}
@@ -109,6 +111,7 @@ func New(c Config) *Driver {
deviceID: deviceID,
disableMediaLink: c.DisableMediaLink,
onTokenUpdate: c.OnTokenUpdate,
uploadTempDir: strings.TrimSpace(c.UploadTempDir),
client: resty.New().
SetTimeout(30*time.Second).
SetHeader("Accept", "application/json, text/plain, */*"),
@@ -175,8 +178,8 @@ func (d *Driver) List(ctx context.Context, dirID string) ([]drives.Entry, error)
// pikpakListCooldown 是列目录触发疑似限流错误时的冷却时长。
//
// 与 p115 driver 的 listCooldown 同语义:只要错误属 transient
// error_code=10 / HTTP 429 / 5xx / 通用 "rate limit" 文本),就持续
// 与 p115 driver 的 listCooldown 同语义:只要错误属明确限流/临时状态
// 结构化 error_code=10 / HTTP 429 / 5xx),就持续
// 等 10 分钟再发一次列目录请求,直到成功或 ctx 取消。这样即使 PikPak
// 风控持续较长时间,扫描会自然延后到风控结束,不再丢半棵子树。
const pikpakListCooldown = 10 * time.Minute
@@ -242,7 +245,6 @@ func pikpakSleepContext(ctx context.Context, d time.Duration) error {
//
// - PikPak 业务码 error_code=10 ("操作频繁",见 OpenList drivers/pikpak/util.go)
// - HTTP 429 / 500 / 502 / 503 / 504 / 509rclone 也把这些归为 retry
// - 通用文本:rate limit / too many requests / blocked / temporarily unavailable
//
// 不包含 4122/4121/16access_token 过期)和 9/4002captcha 过期)—— 这些
// 由 requestOnce 内部已经做过一次自动恢复重试;如果恢复后仍然报这类错误,
@@ -259,22 +261,14 @@ func isTransientPikPakListError(err error) bool {
return true
}
}
text := strings.ToLower(err.Error())
return strings.Contains(text, "error_code=10") ||
strings.Contains(text, "429") ||
strings.Contains(text, "http 500") ||
strings.Contains(text, "http 502") ||
strings.Contains(text, "http 503") ||
strings.Contains(text, "http 504") ||
strings.Contains(text, "http 509") ||
strings.Contains(text, "too many request") ||
strings.Contains(text, "too many requests") ||
strings.Contains(text, "rate limit") ||
strings.Contains(text, "operation frequent") ||
strings.Contains(text, "操作频繁") ||
strings.Contains(text, "blocked") ||
strings.Contains(text, "temporarily unavailable") ||
strings.Contains(text, "service unavailable")
return drives.ErrorMentionsHTTPStatus(err,
http.StatusTooManyRequests,
http.StatusInternalServerError,
http.StatusBadGateway,
http.StatusServiceUnavailable,
http.StatusGatewayTimeout,
509,
)
}
func (d *Driver) Stat(ctx context.Context, fileID string) (*drives.Entry, error) {
+78 -14
View File
@@ -79,6 +79,20 @@ type UploadResult struct {
Size int64
}
type preparedUploadBody struct {
reader io.ReadSeeker
start int64
cleanup func()
}
func (b preparedUploadBody) rewind() error {
if b.reader == nil {
return errors.New("pikpak upload: nil upload body")
}
_, err := b.reader.Seek(b.start, io.SeekStart)
return err
}
// Upload 实现 drives.Drive 接口;只返回 fileID。
// 完整上传元数据见 UploadAndReportHash。
func (d *Driver) Upload(ctx context.Context, parentID, name string, r io.Reader, size int64) (string, error) {
@@ -125,15 +139,15 @@ func (d *Driver) UploadAndReportHash(ctx context.Context, parentID, name string,
parentID = d.rootID
}
// 1) 把 r 全量缓冲到临时文件,同时算 GCID。
tmp, gcidHex, actualSize, err := bufferAndHashGCID(r, size)
// 1) 算 GCID,并准备一个可重试读取的 body。爬虫迁移传入的是
// *os.File,可直接复用原文件,避免再占用一份视频大小的临时空间。
body, gcidHex, actualSize, err := d.prepareUploadBody(r, size)
if err != nil {
return UploadResult{}, err
}
defer func() {
_ = tmp.Close()
_ = os.Remove(tmp.Name())
}()
if body.cleanup != nil {
defer body.cleanup()
}
result := UploadResult{Hash: gcidHex, Size: actualSize}
var lastErr error
@@ -155,7 +169,7 @@ func (d *Driver) UploadAndReportHash(ctx context.Context, parentID, name string,
continue
}
out, err := d.completeUploadAttempt(ctx, tmp, parentID, name, result, resp)
out, err := d.completeUploadAttempt(ctx, body, parentID, name, result, resp)
if err == nil {
return out, nil
}
@@ -190,7 +204,7 @@ func (d *Driver) requestUploadSession(ctx context.Context, parentID, name string
return resp, nil
}
func (d *Driver) completeUploadAttempt(ctx context.Context, tmp *os.File, parentID, name string, result UploadResult, resp uploadTaskData) (UploadResult, error) {
func (d *Driver) completeUploadAttempt(ctx context.Context, body preparedUploadBody, parentID, name string, result UploadResult, resp uploadTaskData) (UploadResult, error) {
// 命中秒传:服务端已经知道这个 hash,直接返回新文件 ID。
if resp.Resumable == nil {
if resp.File.ID != "" {
@@ -207,10 +221,10 @@ func (d *Driver) completeUploadAttempt(ctx context.Context, tmp *os.File, parent
}
// 未命中秒传:把字节传到 S3 兼容存储。
if _, err := tmp.Seek(0, io.SeekStart); err != nil {
return UploadResult{}, fmt.Errorf("pikpak upload: seek tmp: %w", err)
if err := body.rewind(); err != nil {
return UploadResult{}, fmt.Errorf("pikpak upload: rewind body: %w", err)
}
if err := d.uploadToOSS(ctx, &resp.Resumable.Params, tmp); err != nil {
if err := d.uploadToOSS(ctx, &resp.Resumable.Params, body.reader); err != nil {
return UploadResult{}, fmt.Errorf("pikpak upload: oss put: %w", err)
}
@@ -279,12 +293,62 @@ func isRetryablePikPakUploadError(err error) bool {
strings.Contains(text, "service unavailable")
}
func (d *Driver) prepareUploadBody(r io.Reader, size int64) (preparedUploadBody, string, int64, error) {
if rs, ok := r.(io.ReadSeeker); ok {
gcidHex, actualSize, start, err := hashGCIDFromReadSeeker(rs, size)
if err != nil {
return preparedUploadBody{}, "", 0, err
}
return preparedUploadBody{reader: rs, start: start, cleanup: func() {}}, gcidHex, actualSize, nil
}
tmp, gcidHex, actualSize, err := bufferAndHashGCID(d.uploadTempDir, r, size)
if err != nil {
return preparedUploadBody{}, "", 0, err
}
return preparedUploadBody{
reader: tmp,
start: 0,
cleanup: func() {
_ = tmp.Close()
_ = os.Remove(tmp.Name())
},
}, gcidHex, actualSize, nil
}
func hashGCIDFromReadSeeker(r io.ReadSeeker, size int64) (string, int64, int64, error) {
start, err := r.Seek(0, io.SeekCurrent)
if err != nil {
return "", 0, 0, fmt.Errorf("pikpak upload: seek body: %w", err)
}
h := NewGCID(size)
written, copyErr := io.Copy(h, r)
_, seekErr := r.Seek(start, io.SeekStart)
if copyErr != nil {
return "", 0, start, fmt.Errorf("pikpak upload: hash body: %w", copyErr)
}
if seekErr != nil {
return "", 0, start, fmt.Errorf("pikpak upload: rewind body: %w", seekErr)
}
if size > 0 && written != size {
return "", 0, start, fmt.Errorf("pikpak upload: size mismatch: declared %d, copied %d", size, written)
}
return strings.ToUpper(hex.EncodeToString(h.Sum(nil))), written, start, nil
}
// bufferAndHashGCID 把 r 复制到一个临时文件,同时计算 GCID。
// 返回临时文件(位置在末尾,需要调用方 Seek 回 0)、GCID hex 大写、实际写入字节数。
// 返回临时文件(位置在末尾,需要调用方 Seek 回 start)、GCID hex 大写、实际写入字节数。
//
// 调用方负责 Close + Remove 临时文件。
func bufferAndHashGCID(r io.Reader, size int64) (*os.File, string, int64, error) {
tmp, err := os.CreateTemp("", "pikpak-upload-*.bin")
func bufferAndHashGCID(tempDir string, r io.Reader, size int64) (*os.File, string, int64, error) {
tempDir = strings.TrimSpace(tempDir)
if tempDir != "" {
if err := os.MkdirAll(tempDir, 0o755); err != nil {
return nil, "", 0, fmt.Errorf("pikpak upload: create tmp dir: %w", err)
}
}
tmp, err := os.CreateTemp(tempDir, "pikpak-upload-*.bin")
if err != nil {
return nil, "", 0, fmt.Errorf("pikpak upload: create tmp: %w", err)
}
+78 -2
View File
@@ -11,6 +11,8 @@ import (
"net"
"net/http"
"net/http/httptest"
"os"
"path/filepath"
"strings"
"testing"
@@ -142,6 +144,80 @@ func TestUploadInstantSuccessReturnsFileID(t *testing.T) {
}
}
func TestUploadUsesReadSeekerWithoutTempCopy(t *testing.T) {
mux := http.NewServeMux()
mux.HandleFunc("/drive/v1/files", func(w http.ResponseWriter, r *http.Request) {
w.Header().Set("Content-Type", "application/json")
_, _ = w.Write([]byte(`{
"upload_type": "UPLOAD_TYPE_RESUMABLE",
"resumable": null,
"file": {"id": "instant-file-id", "name": "test.mp4", "kind": "drive#file"}
}`))
})
server := httptest.NewServer(mux)
defer server.Close()
d := newTestDriver(t, server)
uploadTempDir := filepath.Join(t.TempDir(), "upload-tmp")
d.uploadTempDir = uploadTempDir
data := bytes.Repeat([]byte{0x31}, 1024)
path := filepath.Join(t.TempDir(), "video.bin")
if err := os.WriteFile(path, data, 0o644); err != nil {
t.Fatalf("write source: %v", err)
}
f, err := os.Open(path)
if err != nil {
t.Fatalf("open source: %v", err)
}
defer f.Close()
id, err := d.Upload(context.Background(), "parent-id", "test.mp4", f, int64(len(data)))
if err != nil {
t.Fatalf("upload: %v", err)
}
if id != "instant-file-id" {
t.Fatalf("file id = %q, want instant-file-id", id)
}
if _, err := os.Stat(uploadTempDir); !os.IsNotExist(err) {
t.Fatalf("upload temp dir stat err = %v, want not created for read seeker input", err)
}
}
func TestUploadBuffersNonSeekReaderInConfiguredTempDir(t *testing.T) {
mux := http.NewServeMux()
mux.HandleFunc("/drive/v1/files", func(w http.ResponseWriter, r *http.Request) {
w.Header().Set("Content-Type", "application/json")
_, _ = w.Write([]byte(`{
"upload_type": "UPLOAD_TYPE_RESUMABLE",
"resumable": null,
"file": {"id": "instant-file-id", "name": "test.mp4", "kind": "drive#file"}
}`))
})
server := httptest.NewServer(mux)
defer server.Close()
d := newTestDriver(t, server)
uploadTempDir := filepath.Join(t.TempDir(), "upload-tmp")
d.uploadTempDir = uploadTempDir
data := bytes.Repeat([]byte{0x42}, 1024)
id, err := d.Upload(context.Background(), "parent-id", "test.mp4", bytes.NewBuffer(data), int64(len(data)))
if err != nil {
t.Fatalf("upload: %v", err)
}
if id != "instant-file-id" {
t.Fatalf("file id = %q, want instant-file-id", id)
}
entries, err := os.ReadDir(uploadTempDir)
if err != nil {
t.Fatalf("read upload temp dir: %v", err)
}
if len(entries) != 0 {
t.Fatalf("upload temp dir entries = %d, want cleaned", len(entries))
}
}
func TestUploadInstantSuccessFallsBackToListWhenFileIDMissing(t *testing.T) {
listCalled := false
mux := http.NewServeMux()
@@ -304,7 +380,7 @@ func TestUploadRejectsInvalidArguments(t *testing.T) {
func TestBufferAndHashGCIDDetectsSizeMismatch(t *testing.T) {
src := bytes.NewReader([]byte("hello"))
// 声明 size=10 但实际只有 5 字节
_, _, _, err := bufferAndHashGCID(src, 10)
_, _, _, err := bufferAndHashGCID("", src, 10)
if err == nil {
t.Fatal("expected size mismatch error")
}
@@ -315,7 +391,7 @@ func TestBufferAndHashGCIDDetectsSizeMismatch(t *testing.T) {
func TestBufferAndHashGCIDComputesCorrectHash(t *testing.T) {
data := bytes.Repeat([]byte{0x55}, 1024)
tmp, hex, written, err := bufferAndHashGCID(bytes.NewReader(data), int64(len(data)))
tmp, hex, written, err := bufferAndHashGCID("", bytes.NewReader(data), int64(len(data)))
if err != nil {
t.Fatalf("buffer: %v", err)
}
@@ -50,6 +50,7 @@ type CrawlerConfig struct {
CommonThumbDir string
ProxyURL string
ConfigJSON string
DisablePreview bool
HTTPClient *http.Client
DownloadTimeout time.Duration
OnProgress func(CrawlProgress)
@@ -562,6 +563,10 @@ func (c *Crawler) processItem(ctx context.Context, item Item) (bool, error) {
if quality == "" {
quality = "HD"
}
previewStatus := "pending"
if c.previewDisabled(ctx) {
previewStatus = "disabled"
}
v := &catalog.Video{
ID: videoID,
DriveID: c.cfg.Driver.ID(),
@@ -576,7 +581,7 @@ func (c *Crawler) processItem(ctx context.Context, item Item) (bool, error) {
Quality: quality,
Category: strings.TrimSpace(item.Category),
Description: strings.TrimSpace(item.Description),
PreviewStatus: "pending",
PreviewStatus: previewStatus,
PublishedAt: publishedAt,
CreatedAt: now,
UpdatedAt: now,
@@ -632,6 +637,18 @@ func (c *Crawler) processItem(ctx context.Context, item Item) (bool, error) {
return true, nil
}
func (c *Crawler) previewDisabled(ctx context.Context) bool {
if c == nil {
return false
}
if c.cfg.Catalog != nil && c.cfg.Driver != nil {
if d, err := c.cfg.Catalog.GetDrive(ctx, c.cfg.Driver.ID()); err == nil && d != nil {
return !d.TeaserEnabled
}
}
return c.cfg.DisablePreview
}
func (c *Crawler) materializeMedia(ctx context.Context, ref MediaRef, dst, referer string, required bool) (int64, error) {
if local := strings.TrimSpace(ref.LocalFile); local != "" {
return c.copyLocalOutput(local, dst)
@@ -114,6 +114,128 @@ func TestCrawlerRunOnceImportsLocalFileAndSkipsExisting(t *testing.T) {
}
}
func TestCrawlerRunOnceMarksPreviewDisabledWhenConfigured(t *testing.T) {
ctx := context.Background()
tmp := t.TempDir()
cat, err := catalog.Open(filepath.Join(tmp, "catalog.db"))
if err != nil {
t.Fatalf("open catalog: %v", err)
}
t.Cleanup(func() {
if err := cat.Close(); err != nil {
t.Fatalf("close catalog: %v", err)
}
})
drv := New(Config{ID: "demo", RootDir: filepath.Join(tmp, "crawler")})
if err := drv.Init(ctx); err != nil {
t.Fatalf("driver init: %v", err)
}
dummyScript := filepath.Join(tmp, "helper-script")
if err := os.WriteFile(dummyScript, []byte("helper"), 0o755); err != nil {
t.Fatalf("write dummy script: %v", err)
}
wrapper := filepath.Join(tmp, "helper-wrapper.sh")
wrapperScript := fmt.Sprintf("#!/bin/sh\nexec %q -test.run=TestScriptCrawlerHelperProcess \"$@\"\n", os.Args[0])
if err := os.WriteFile(wrapper, []byte(wrapperScript), 0o755); err != nil {
t.Fatalf("write helper wrapper: %v", err)
}
t.Setenv("GO_WANT_SCRIPTCRAWLER_HELPER", "1")
c := NewCrawler(CrawlerConfig{
Driver: drv,
Catalog: cat,
PythonPath: wrapper,
FFprobePath: writeScriptCrawlerFFprobeStub(t, tmp, true),
ScriptPath: dummyScript,
DisablePreview: true,
})
res, err := c.RunOnce(ctx, 1)
if err != nil {
t.Fatalf("run once: %v", err)
}
if res.NewVideos != 1 || res.Failed != 0 {
t.Fatalf("result = new:%d failed:%d, want 1/0", res.NewVideos, res.Failed)
}
v, err := cat.GetVideo(ctx, BuildVideoID("demo", "abc-123"))
if err != nil {
t.Fatalf("get video: %v", err)
}
if v.PreviewStatus != "disabled" {
t.Fatalf("preview status = %q, want disabled", v.PreviewStatus)
}
if v.FingerprintStatus != "ready" || v.SampledSHA256 == "" {
t.Fatalf("fingerprint status=%q sampled=%q, want ready and sampled hash", v.FingerprintStatus, v.SampledSHA256)
}
pending, err := cat.ListVideosByPreviewStatus(ctx, "demo", "pending", 0)
if err != nil {
t.Fatalf("list pending previews: %v", err)
}
if len(pending) != 0 {
t.Fatalf("pending previews = %d, want 0", len(pending))
}
}
func TestCrawlerRunOnceUsesCurrentDrivePreviewSwitch(t *testing.T) {
ctx := context.Background()
tmp := t.TempDir()
cat, err := catalog.Open(filepath.Join(tmp, "catalog.db"))
if err != nil {
t.Fatalf("open catalog: %v", err)
}
t.Cleanup(func() {
if err := cat.Close(); err != nil {
t.Fatalf("close catalog: %v", err)
}
})
drv := New(Config{ID: "demo", RootDir: filepath.Join(tmp, "crawler")})
if err := drv.Init(ctx); err != nil {
t.Fatalf("driver init: %v", err)
}
if err := cat.UpsertDrive(ctx, &catalog.Drive{
ID: drv.ID(),
Kind: Kind,
Name: "Demo",
RootID: "/",
Credentials: map[string]string{"script_path": "/tmp/crawler.py"},
TeaserEnabled: true,
}); err != nil {
t.Fatalf("seed drive: %v", err)
}
dummyScript := filepath.Join(tmp, "helper-script")
if err := os.WriteFile(dummyScript, []byte("helper"), 0o755); err != nil {
t.Fatalf("write dummy script: %v", err)
}
wrapper := filepath.Join(tmp, "helper-wrapper.sh")
wrapperScript := fmt.Sprintf("#!/bin/sh\nexec %q -test.run=TestScriptCrawlerHelperProcess \"$@\"\n", os.Args[0])
if err := os.WriteFile(wrapper, []byte(wrapperScript), 0o755); err != nil {
t.Fatalf("write helper wrapper: %v", err)
}
t.Setenv("GO_WANT_SCRIPTCRAWLER_HELPER", "1")
c := NewCrawler(CrawlerConfig{
Driver: drv,
Catalog: cat,
PythonPath: wrapper,
FFprobePath: writeScriptCrawlerFFprobeStub(t, tmp, true),
ScriptPath: dummyScript,
DisablePreview: true,
})
res, err := c.RunOnce(ctx, 1)
if err != nil {
t.Fatalf("run once: %v", err)
}
if res.NewVideos != 1 || res.Failed != 0 {
t.Fatalf("result = new:%d failed:%d, want 1/0", res.NewVideos, res.Failed)
}
v, err := cat.GetVideo(ctx, BuildVideoID("demo", "abc-123"))
if err != nil {
t.Fatalf("get video: %v", err)
}
if v.PreviewStatus != "pending" {
t.Fatalf("preview status = %q, want pending from current drive switch", v.PreviewStatus)
}
}
func TestCrawlerRunOnceUsesSourceKindNamespace(t *testing.T) {
ctx := context.Background()
tmp := t.TempDir()
+52 -33
View File
@@ -67,6 +67,55 @@ type DryRunResult struct {
DurationMs int64 `json:"durationMs"`
}
type dryRunLogTail struct {
mu sync.Mutex
lines []string
partial string
}
func newDryRunLogTail() *dryRunLogTail {
return &dryRunLogTail{lines: make([]string, 0, dryRunLogTailLines)}
}
func (t *dryRunLogTail) Write(p []byte) (int, error) {
t.mu.Lock()
defer t.mu.Unlock()
chunk := strings.ReplaceAll(string(p), "\r\n", "\n")
parts := strings.Split(t.partial+chunk, "\n")
t.partial = parts[len(parts)-1]
for _, line := range parts[:len(parts)-1] {
t.appendLocked(line)
}
return len(p), nil
}
func (t *dryRunLogTail) snapshot() []string {
t.mu.Lock()
defer t.mu.Unlock()
lines := append([]string{}, t.lines...)
if partial := strings.TrimSpace(t.partial); partial != "" {
lines = appendDryRunLogLine(lines, partial)
}
return lines
}
func (t *dryRunLogTail) appendLocked(line string) {
t.lines = appendDryRunLogLine(t.lines, line)
}
func appendDryRunLogLine(lines []string, line string) []string {
line = strings.TrimSpace(line)
if line == "" {
return lines
}
if len(lines) >= dryRunLogTailLines {
lines = lines[1:]
}
return append(lines, line)
}
func DryRun(ctx context.Context, cfg DryRunConfig) *DryRunResult {
started := time.Now()
result := &DryRunResult{Items: []DryRunItem{}}
@@ -169,41 +218,14 @@ func DryRun(ctx context.Context, cfg DryRunConfig) *DryRunResult {
result.Error = fmt.Sprintf("启动脚本失败: %v", err)
return result
}
stderr, err := cmd.StderrPipe()
if err != nil {
_ = stdout.Close()
result.Error = fmt.Sprintf("启动脚本失败: %v", err)
return result
}
logTail := newDryRunLogTail()
cmd.Stderr = logTail
if err := cmd.Start(); err != nil {
_ = stdout.Close()
_ = stderr.Close()
result.Error = fmt.Sprintf("启动脚本失败: %v", err)
return result
}
// stderr 是脚本日志,保留尾部若干行用于排错回显。
var logMu sync.Mutex
logTail := make([]string, 0, dryRunLogTailLines)
stderrDone := make(chan struct{})
go func() {
defer close(stderrDone)
scanner := bufio.NewScanner(stderr)
scanner.Buffer(make([]byte, 64*1024), 1024*1024)
for scanner.Scan() {
line := strings.TrimSpace(scanner.Text())
if line == "" {
continue
}
logMu.Lock()
if len(logTail) >= dryRunLogTailLines {
logTail = logTail[1:]
}
logTail = append(logTail, line)
logMu.Unlock()
}
}()
items := []DryRunItem{}
var firstMediaHeaders map[string]string
parseFailures := 0
@@ -264,11 +286,8 @@ func DryRun(ctx context.Context, cfg DryRunConfig) *DryRunResult {
_ = killDryRunProcess(cmd)
<-waitDone
}
<-stderrDone
logMu.Lock()
result.Log = append([]string{}, logTail...)
logMu.Unlock()
result.Log = logTail.snapshot()
result.Items = items
if len(items) == 0 {
@@ -48,6 +48,29 @@ echo '{"type":"done","stats":{"emitted":1}}'
}
}
func TestDryRunCapturesStderrWhenStoppingAfterFirstItem(t *testing.T) {
script := writeDryRunScript(t, `
echo '[log] first item ready' >&2
echo '{"type":"item","item":{"title":"Early Stop Video","media_url":"https://cdn.example.test/v.mp4","source_id":"early-stop"}}'
sleep 30
`)
start := time.Now()
result := DryRun(context.Background(), DryRunConfig{
PythonPath: "/bin/sh",
ScriptPath: script,
SkipMediaProbe: true,
})
if !result.OK {
t.Fatalf("ok = false, error = %q, log = %v", result.Error, result.Log)
}
if elapsed := time.Since(start); elapsed > 5*time.Second {
t.Fatalf("dry run took %s, script was not stopped after first item", elapsed)
}
if len(result.Log) == 0 || !strings.Contains(result.Log[0], "first item ready") {
t.Fatalf("log tail = %v, want stderr captured before early stop", result.Log)
}
}
func TestDryRunProbesMediaURL(t *testing.T) {
var gotRange, gotReferer string
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
+22 -42
View File
@@ -27,6 +27,7 @@ type Driver struct {
refreshToken string
client *sdk.WoClient
onTokenUpdate func(access, refresh string)
uploadTempDir string
listMu sync.Mutex
lastListAt time.Time
@@ -38,11 +39,12 @@ type Driver struct {
}
type Config struct {
ID string
AccessToken string
RefreshToken string
FamilyID string // 空则走个人空间,有值则走家庭空间
RootID string // 根目录 ID,默认 "0"
ID string
AccessToken string
RefreshToken string
FamilyID string // 空则走个人空间,有值则走家庭空间
RootID string // 根目录 ID,默认 "0"
UploadTempDir string
// 当 SDK 刷新 token 时回调,便于持久化
OnTokenUpdate func(access, refresh string)
}
@@ -59,6 +61,7 @@ func New(c Config) *Driver {
accessToken: c.AccessToken,
refreshToken: c.RefreshToken,
onTokenUpdate: c.OnTokenUpdate,
uploadTempDir: strings.TrimSpace(c.UploadTempDir),
listInterval: 800 * time.Millisecond,
listCooldown: 5 * time.Minute,
fidToID: make(map[string]string),
@@ -162,7 +165,12 @@ func (d *Driver) StreamURL(ctx context.Context, fileID string) (*drives.StreamLi
func (d *Driver) Upload(ctx context.Context, parentID, name string, r io.Reader, size int64) (string, error) {
// wopan SDK 要求 *os.File,先把流落到临时文件再上传
tmp, err := os.CreateTemp("", "wopan-upload-*.tmp")
if d.uploadTempDir != "" {
if err := os.MkdirAll(d.uploadTempDir, 0o755); err != nil {
return "", fmt.Errorf("wopan upload: create tmp dir: %w", err)
}
}
tmp, err := os.CreateTemp(d.uploadTempDir, "wopan-upload-*.tmp")
if err != nil {
return "", err
}
@@ -510,42 +518,14 @@ func isWopanRateLimitError(err error) bool {
if err == nil || errors.Is(err, context.Canceled) || errors.Is(err, context.DeadlineExceeded) {
return false
}
text := strings.ToLower(strings.TrimSpace(err.Error()))
if text == "" {
return false
}
return strings.Contains(text, "status: 429") ||
strings.Contains(text, "status 429") ||
strings.Contains(text, "http status: 429") ||
strings.Contains(text, "status: 500") ||
strings.Contains(text, "status 500") ||
strings.Contains(text, "status: 502") ||
strings.Contains(text, "status 502") ||
strings.Contains(text, "status: 503") ||
strings.Contains(text, "status 503") ||
strings.Contains(text, "status: 504") ||
strings.Contains(text, "status 504") ||
strings.Contains(text, "status: 509") ||
strings.Contains(text, "status 509") ||
strings.Contains(text, "too many request") ||
strings.Contains(text, "too many requests") ||
strings.Contains(text, "rate limit") ||
strings.Contains(text, "rate-limit") ||
strings.Contains(text, "throttl") ||
strings.Contains(text, "blocked") ||
strings.Contains(text, "request has been blocked") ||
strings.Contains(text, "操作频繁") ||
strings.Contains(text, "请求频繁") ||
strings.Contains(text, "请求太频繁") ||
strings.Contains(text, "请求过于频繁") ||
strings.Contains(text, "频率限制") ||
strings.Contains(text, "请求次数过多") ||
strings.Contains(text, "系统繁忙") ||
strings.Contains(text, "服务繁忙") ||
strings.Contains(text, "稍后再试") ||
strings.Contains(text, "稍后重试") ||
strings.Contains(text, "访问被阻断") ||
strings.Contains(text, "风控")
return drives.ErrorMentionsHTTPStatus(err,
http.StatusTooManyRequests,
http.StatusInternalServerError,
http.StatusBadGateway,
http.StatusServiceUnavailable,
http.StatusGatewayTimeout,
509,
)
}
func guessMime(name string) string {
+14 -36
View File
@@ -372,37 +372,10 @@ func remoteRangeResponseLooksRateLimited(rawURL string, status int, body []byte)
status == 509) {
return true
}
text := strings.ToLower(strings.TrimSpace(string(body)))
compact := compactRemoteRangeErrorText(text)
if strings.Contains(text, "too many request") ||
strings.Contains(text, "too many requests") ||
strings.Contains(text, "rate limit") ||
strings.Contains(text, "quota exceeded") ||
strings.Contains(text, "操作频繁") ||
strings.Contains(text, "请求频繁") ||
strings.Contains(text, "请求太频繁") ||
strings.Contains(text, "请求过于频繁") ||
strings.Contains(text, "频率限制") ||
strings.Contains(text, "请求次数过多") ||
strings.Contains(text, "系统繁忙") ||
strings.Contains(text, "服务繁忙") ||
strings.Contains(text, "稍后再试") ||
strings.Contains(text, "稍后重试") ||
strings.Contains(text, "访问被阻断") ||
strings.Contains(text, "风控") ||
strings.Contains(text, "download quota") ||
strings.Contains(text, "sharing rate") ||
strings.Contains(text, "daily limit") ||
strings.Contains(text, "user rate") ||
strings.Contains(text, "usage limit") ||
strings.Contains(compact, "ratelimitexceeded") ||
strings.Contains(compact, "userratelimitexceeded") ||
strings.Contains(compact, "dailylimitexceeded") ||
strings.Contains(compact, "downloadquotaexceeded") ||
strings.Contains(compact, "sharingratelimitexceeded") ||
strings.Contains(compact, "quotaexceeded") ||
strings.Contains(compact, "toomanyrequests") ||
strings.Contains(compact, "usagelimits") {
if isGuangYaPanMediaURL(rawURL) && (status == http.StatusForbidden || status == http.StatusTooManyRequests ||
status == http.StatusInternalServerError || status == http.StatusBadGateway ||
status == http.StatusServiceUnavailable || status == http.StatusGatewayTimeout ||
status == 509) {
return true
}
if status == http.StatusForbidden && isGoogleDriveMediaURL(rawURL) {
@@ -424,6 +397,16 @@ func isWopanMediaURL(rawURL string) bool {
strings.Contains(path, "/openapi/download")
}
func isGuangYaPanMediaURL(rawURL string) bool {
u, err := url.Parse(rawURL)
if err != nil {
return false
}
host := strings.ToLower(u.Hostname())
return strings.HasSuffix(host, "guangyacdn.com") ||
strings.HasSuffix(host, "guangyapan.com")
}
func isGoogleDriveMediaURL(rawURL string) bool {
u, err := url.Parse(rawURL)
if err != nil {
@@ -434,11 +417,6 @@ func isGoogleDriveMediaURL(rawURL string) bool {
return strings.Contains(host, "googleapis.com") && strings.Contains(path, "/drive/")
}
func compactRemoteRangeErrorText(text string) string {
replacer := strings.NewReplacer("_", "", "-", "", " ", "", ".", "", ":", "")
return replacer.Replace(strings.ToLower(strings.TrimSpace(text)))
}
func parseRetryAfter(raw string) time.Duration {
raw = strings.TrimSpace(raw)
if raw == "" {
+28 -4
View File
@@ -86,16 +86,16 @@ func TestComputeRemoteUsesRangeSamples(t *testing.T) {
}
}
func TestComputeRemoteGoogleQuotaExceededReturnsRateLimit(t *testing.T) {
func TestComputeRemote429ReturnsRateLimit(t *testing.T) {
ctx := context.Background()
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
w.Header().Set("Retry-After", "60")
w.WriteHeader(http.StatusForbidden)
_, _ = w.Write([]byte(`{"error":{"code":403,"message":"The download quota for this file has been exceeded.","errors":[{"domain":"usageLimits","reason":"downloadQuotaExceeded","message":"The download quota for this file has been exceeded."}]}}`))
w.WriteHeader(http.StatusTooManyRequests)
_, _ = w.Write([]byte(`{"error":{"code":429}}`))
}))
defer srv.Close()
drv := &fakeDrive{paths: map[string]string{"remote": srv.URL + "/drive/v3/files/file-1?alt=media"}}
drv := &fakeDrive{paths: map[string]string{"remote": srv.URL + "/video.mp4"}}
_, err := Compute(ctx, drv, &catalog.Video{ID: "remote", FileID: "remote", Size: 1024 * 1024}, Config{
SampleSizeBytes: 4,
FullHashMaxSize: 8,
@@ -131,6 +131,30 @@ func TestWopanRemoteRangeErrorsLookRateLimited(t *testing.T) {
}
}
func TestGuangYaPanRemoteRangeErrorsLookRateLimited(t *testing.T) {
for _, tc := range []struct {
rawURL string
status int
}{
{rawURL: "https://txgz02-httpdown.guangyacdn.com/download/?fid=encoded", status: http.StatusForbidden},
{rawURL: "https://txgz02-httpdown.guangyacdn.com/download/?fid=encoded", status: http.StatusServiceUnavailable},
{rawURL: "https://txgz02-httpdown.guangyacdn.com/download/?fid=encoded", status: 509},
} {
if !remoteRangeResponseLooksRateLimited(tc.rawURL, tc.status, nil) {
t.Fatalf("remoteRangeResponseLooksRateLimited(%q, %d) = false, want true", tc.rawURL, tc.status)
}
}
if remoteRangeResponseLooksRateLimited("https://example.com/video.mp4", http.StatusForbidden, nil) {
t.Fatal("generic 403 should not be treated as guangyapan rate limit")
}
}
func TestGoogleDriveRemoteRangeForbiddenLooksRateLimitedByURL(t *testing.T) {
if !remoteRangeResponseLooksRateLimited("https://www.googleapis.com/drive/v3/files/file-1?alt=media", http.StatusForbidden, nil) {
t.Fatal("google drive media 403 should be treated as rate limit by URL and status")
}
}
type fakeDrive struct {
paths map[string]string
}
+32 -159
View File
@@ -952,15 +952,7 @@ func redactURLs(text string) string {
}
func ffmpegOutputLooksRateLimited(output []byte) bool {
text := strings.ToLower(string(output))
if !strings.Contains(text, "429") {
return false
}
return strings.Contains(text, "too many requests") ||
strings.Contains(text, "throttl") ||
strings.Contains(text, "rate limit") ||
strings.Contains(text, "rate-limit") ||
strings.Contains(text, "server returned 429")
return drives.TextMentionsHTTPStatus(string(output), http.StatusTooManyRequests)
}
// --- 本地落盘 ---
@@ -1064,12 +1056,10 @@ type ThumbWorker struct {
}
const (
defaultTransientMediaCooldown = 5 * time.Minute
defaultGenerationRateLimitCooldown = 5 * time.Minute
defaultThumbTransientMediaMaxFailures = 3
defaultWorkerQueueSize = 10000
maxPreviewTeaserSizeBytes int64 = 5 * 1024 * 1024 * 1024
previewStatusSkipped = "skipped"
defaultTransientMediaCooldown = 5 * time.Minute
defaultGenerationRateLimitCooldown = 5 * time.Minute
defaultThumbTransientMediaMaxFailures = 3
defaultWorkerQueueSize = 10000
)
type rateLimitState struct {
@@ -1124,6 +1114,19 @@ func (q *videoQueue) release(v *catalog.Video) {
q.mu.Unlock()
}
func (q *videoQueue) idsSnapshot() []string {
q.mu.Lock()
defer q.mu.Unlock()
if len(q.ids) == 0 {
return nil
}
out := make([]string, 0, len(q.ids))
for id := range q.ids {
out = append(out, id)
}
return out
}
func (q *videoQueue) lengthExcluding(currentID string) int {
q.mu.Lock()
defer q.mu.Unlock()
@@ -1251,6 +1254,13 @@ func (w *Worker) Status() TaskStatus {
return taskStatus(&w.activity, &w.rateLimit, w.queue.lengthExcluding(currentID))
}
func (w *Worker) ActiveVideoIDs() []string {
if w == nil {
return nil
}
return w.queue.idsSnapshot()
}
func (w *ThumbWorker) Status() TaskStatus {
if w == nil {
return TaskStatus{State: "idle"}
@@ -1518,145 +1528,21 @@ func driveErrorShouldCooldown(d drives.Drive, err error) bool {
}
switch d.Kind() {
case "p115":
text := strings.ToLower(err.Error())
return strings.Contains(text, "server returned 403") ||
strings.Contains(text, "403 forbidden") ||
strings.Contains(text, "server returned 405") ||
strings.Contains(text, "405 method") ||
strings.Contains(text, "access denied") ||
strings.Contains(text, "moov atom not found") ||
strings.Contains(text, "partial file") ||
strings.Contains(text, "request has been blocked") ||
strings.Contains(text, "访问被阻断")
return drives.ErrorMentionsHTTPStatus(err, http.StatusForbidden, http.StatusMethodNotAllowed, http.StatusTooManyRequests)
case "pikpak":
// PikPak 在预览视频 / 封面生成阶段(取链或拉直链字节)可能命中:
// - error_code=10 操作频繁
// - HTTP 429 / 5xx / 509 限流和服务端不可用
// - 通用文本:rate limit / too many requests / blocked
// 命中时让 worker 冷却 5 分钟,避免连续请求加重风控。
text := strings.ToLower(err.Error())
return strings.Contains(text, "error_code=10") ||
strings.Contains(text, "操作频繁") ||
strings.Contains(text, "429") ||
strings.Contains(text, "http 500") ||
strings.Contains(text, "http 502") ||
strings.Contains(text, "http 503") ||
strings.Contains(text, "http 504") ||
strings.Contains(text, "http 509") ||
strings.Contains(text, "too many request") ||
strings.Contains(text, "too many requests") ||
strings.Contains(text, "rate limit") ||
strings.Contains(text, "blocked") ||
strings.Contains(text, "partial file") ||
strings.Contains(text, "service unavailable")
return drives.ErrorMentionsHTTPStatus(err, http.StatusTooManyRequests, http.StatusInternalServerError, http.StatusBadGateway, http.StatusServiceUnavailable, http.StatusGatewayTimeout, 509)
case "p123":
// 123网盘直链解析 / ffmpeg 读取阶段可能返回 429、5xx,或 WAF 类
// blocked / 访问阻断文本。命中时冷却,避免封面和预览视频生成连续打接口。
text := strings.ToLower(err.Error())
return strings.Contains(text, "请求太频繁") ||
strings.Contains(text, "请求过于频繁") ||
strings.Contains(text, "请求频繁") ||
strings.Contains(text, "操作频繁") ||
strings.Contains(text, "频率限制") ||
strings.Contains(text, "请求次数过多") ||
strings.Contains(text, "429") ||
strings.Contains(text, "http 500") ||
strings.Contains(text, "http 502") ||
strings.Contains(text, "http 503") ||
strings.Contains(text, "http 504") ||
strings.Contains(text, "server returned 403") ||
strings.Contains(text, "403 forbidden") ||
strings.Contains(text, "too many request") ||
strings.Contains(text, "too many requests") ||
strings.Contains(text, "rate limit") ||
strings.Contains(text, "blocked") ||
strings.Contains(text, "访问被阻断") ||
strings.Contains(text, "service unavailable")
return drives.ErrorMentionsHTTPStatus(err, http.StatusForbidden, http.StatusTooManyRequests, http.StatusInternalServerError, http.StatusBadGateway, http.StatusServiceUnavailable, http.StatusGatewayTimeout)
case "wopan":
// 联通网盘的取链接口和下载直链都可能返回"操作频繁"、429、5xx
// 或 WAF 阻断文本。封面/预览失败时先冷却,避免持续触发风控。
text := strings.ToLower(err.Error())
return strings.Contains(text, "请求太频繁") ||
strings.Contains(text, "请求过于频繁") ||
strings.Contains(text, "请求频繁") ||
strings.Contains(text, "操作频繁") ||
strings.Contains(text, "频率限制") ||
strings.Contains(text, "请求次数过多") ||
strings.Contains(text, "系统繁忙") ||
strings.Contains(text, "服务繁忙") ||
strings.Contains(text, "稍后再试") ||
strings.Contains(text, "稍后重试") ||
strings.Contains(text, "429") ||
strings.Contains(text, "http 500") ||
strings.Contains(text, "http 502") ||
strings.Contains(text, "http 503") ||
strings.Contains(text, "http 504") ||
strings.Contains(text, "http 509") ||
strings.Contains(text, "server returned 403") ||
strings.Contains(text, "403 forbidden") ||
strings.Contains(text, "server returned 429") ||
strings.Contains(text, "server returned 500") ||
strings.Contains(text, "server returned 502") ||
strings.Contains(text, "server returned 503") ||
strings.Contains(text, "server returned 504") ||
strings.Contains(text, "too many request") ||
strings.Contains(text, "too many requests") ||
strings.Contains(text, "rate limit") ||
strings.Contains(text, "rate-limit") ||
strings.Contains(text, "throttl") ||
strings.Contains(text, "blocked") ||
strings.Contains(text, "request has been blocked") ||
strings.Contains(text, "访问被阻断") ||
strings.Contains(text, "风控") ||
strings.Contains(text, "service unavailable")
return drives.ErrorMentionsHTTPStatus(err, http.StatusForbidden, http.StatusTooManyRequests, http.StatusInternalServerError, http.StatusBadGateway, http.StatusServiceUnavailable, http.StatusGatewayTimeout, 509)
case "guangyapan":
return drives.ErrorMentionsHTTPStatus(err, http.StatusForbidden, http.StatusTooManyRequests, http.StatusInternalServerError, http.StatusBadGateway, http.StatusServiceUnavailable, http.StatusGatewayTimeout, 509)
case "googledrive":
// Google Drive 下载/取样阶段常把频控和配额问题包装成 403,
// 具体标识在 error.errors[].reason/message 里(OpenList 也按该结构解析)。
// ffmpeg/ffprobe 只能看到 stderr 文本时,按这些 reason/文本兜底冷却。
text := strings.ToLower(err.Error())
return googleDriveMediaErrorShouldCooldown(text)
return drives.ErrorMentionsHTTPStatus(err, http.StatusForbidden, http.StatusTooManyRequests, http.StatusInternalServerError, http.StatusBadGateway, http.StatusServiceUnavailable, http.StatusGatewayTimeout)
}
return false
}
func googleDriveMediaErrorShouldCooldown(text string) bool {
if text == "" {
return false
}
compact := compactGoogleDriveErrorText(text)
return strings.Contains(text, "server returned 403") ||
strings.Contains(text, "403 forbidden") ||
strings.Contains(text, "server returned 429") ||
strings.Contains(text, "http 429") ||
strings.Contains(text, "http 500") ||
strings.Contains(text, "http 502") ||
strings.Contains(text, "http 503") ||
strings.Contains(text, "http 504") ||
strings.Contains(text, "too many request") ||
strings.Contains(text, "too many requests") ||
strings.Contains(text, "rate limit") ||
strings.Contains(text, "quota exceeded") ||
strings.Contains(text, "download quota") ||
strings.Contains(text, "sharing rate") ||
strings.Contains(text, "daily limit") ||
strings.Contains(text, "user rate") ||
strings.Contains(text, "usage limit") ||
strings.Contains(text, "service unavailable") ||
strings.Contains(compact, "ratelimitexceeded") ||
strings.Contains(compact, "userratelimitexceeded") ||
strings.Contains(compact, "dailylimitexceeded") ||
strings.Contains(compact, "downloadquotaexceeded") ||
strings.Contains(compact, "sharingratelimitexceeded") ||
strings.Contains(compact, "quotaexceeded") ||
strings.Contains(compact, "toomanyrequests") ||
strings.Contains(compact, "usagelimits")
}
func compactGoogleDriveErrorText(text string) string {
replacer := strings.NewReplacer("_", "", "-", "", " ", "", ".", "", ":", "")
return replacer.Replace(strings.ToLower(strings.TrimSpace(text)))
}
func (w *ThumbWorker) process(ctx context.Context, v *catalog.Video) bool {
if w.skipIfRateLimited(v) {
return false
@@ -1806,15 +1692,6 @@ func localPreviewLink(v *catalog.Video) (*drives.StreamLink, bool) {
}
func (w *Worker) process(ctx context.Context, v *catalog.Video) {
if shouldSkipTeaser(v) {
removePreviousLocalTeaser(v.PreviewLocal, "")
if err := w.Catalog.UpdatePreview(ctx, v.ID, "", previewStatusSkipped); err != nil {
log.Printf("[preview] skip %s: update status: %v", v.Title, err)
return
}
log.Printf("[preview] skip %s: size=%d exceeds 5GiB teaser limit", v.Title, v.Size)
return
}
if w.skipIfRateLimited(v) {
return
}
@@ -1867,10 +1744,6 @@ func (w *Worker) process(ctx context.Context, v *catalog.Video) {
log.Printf("[preview] ready %s (duration=%.1fs)", v.Title, duration)
}
func shouldSkipTeaser(v *catalog.Video) bool {
return v != nil && v.Size > maxPreviewTeaserSizeBytes
}
func (w *Worker) generateTeaser(ctx context.Context, v *catalog.Video, link *drives.StreamLink, duration float64) (string, error) {
gen, ok := w.Gen.(refreshingTeaserGenerator)
if !ok || w.Drive == nil || w.Drive.Kind() != "p115" {
+56 -105
View File
@@ -349,42 +349,10 @@ func TestPreviewWorkerNeverCallsDriveUploadOrEnsureDir(t *testing.T) {
}
}
func TestPreviewWorkerSkipsTeaserForVideoLargerThanFiveGiB(t *testing.T) {
func TestPreviewWorkerGeneratesTeaserForLargeVideo(t *testing.T) {
ctx := context.Background()
cat, video := seedPreviewTestVideo(t, "preview-large-video")
video.Size = maxPreviewTeaserSizeBytes + 1
if err := cat.UpsertVideo(ctx, video); err != nil {
t.Fatalf("update video: %v", err)
}
gen := &fakeTeaserGenerator{}
drv := &previewFakeDrive{}
worker := NewWorker(gen, cat, drv)
worker.process(ctx, video)
got, err := cat.GetVideo(ctx, video.ID)
if err != nil {
t.Fatalf("get video: %v", err)
}
if got.PreviewStatus != previewStatusSkipped {
t.Fatalf("preview status = %q, want skipped", got.PreviewStatus)
}
if got.PreviewLocal != "" {
t.Fatalf("preview local = %q, want empty", got.PreviewLocal)
}
if drv.streamCalls != 0 {
t.Fatalf("stream calls = %d, want 0", drv.streamCalls)
}
if gen.generateCalls != 0 {
t.Fatalf("generate calls = %d, want 0", gen.generateCalls)
}
}
func TestPreviewWorkerGeneratesTeaserAtFiveGiBBoundary(t *testing.T) {
ctx := context.Background()
cat, video := seedPreviewTestVideo(t, "preview-five-gib-video")
video.Size = maxPreviewTeaserSizeBytes
video.Size = 6 * 1024 * 1024 * 1024
if err := cat.UpsertVideo(ctx, video); err != nil {
t.Fatalf("update video: %v", err)
}
@@ -485,9 +453,9 @@ func TestThumbWorkerRateLimitHonorsRetryAfter(t *testing.T) {
assertCooldownAround(t, worker.Status().CooldownUntil, before, 2*time.Hour)
}
func TestThumbWorkerP115TransientErrorFailsAfterRetryLimit(t *testing.T) {
func TestThumbWorkerP115MessageOnlyErrorFailsWithoutCooldown(t *testing.T) {
ctx := context.Background()
cat, video := seedPreviewTestVideo(t, "thumb-p115-transient")
cat, video := seedPreviewTestVideo(t, "thumb-p115-message-only")
gen := &fakeThumbGenerator{
generateErr: errors.New("ffmpeg thumb: exit status 183, stderr: partial file Cannot determine format of input 0:0 after EOF"),
@@ -495,69 +463,26 @@ func TestThumbWorkerP115TransientErrorFailsAfterRetryLimit(t *testing.T) {
drv := &previewFakeDrive{kind: "p115"}
worker := NewThumbWorker(gen, cat, drv)
for attempt := 1; attempt <= defaultThumbTransientMediaMaxFailures; attempt++ {
worker.rateLimit = rateLimitState{}
worker.process(ctx, video)
if attempt < defaultThumbTransientMediaMaxFailures {
pending, err := cat.ListVideosByThumbnailStatus(ctx, video.DriveID, "pending", 0)
if err != nil {
t.Fatalf("list pending thumbnails: %v", err)
}
if len(pending) != 1 || pending[0].ID != video.ID {
t.Fatalf("attempt %d pending thumbnails = %#v, want only %s", attempt, pending, video.ID)
}
missing, err := cat.CountVideosNeedingThumbnail(ctx, video.DriveID)
if err != nil {
t.Fatalf("count missing thumbnails: %v", err)
}
if missing != 1 {
t.Fatalf("attempt %d missing thumbnails = %d, want 1 before retry limit", attempt, missing)
}
continue
}
failed, err := cat.ListVideosByThumbnailStatus(ctx, video.DriveID, "failed", 0)
if err != nil {
t.Fatalf("list failed thumbnails: %v", err)
}
if len(failed) != 1 || failed[0].ID != video.ID {
t.Fatalf("failed thumbnails = %#v, want only %s", failed, video.ID)
}
missing, err := cat.CountVideosNeedingThumbnail(ctx, video.DriveID)
if err != nil {
t.Fatalf("count missing thumbnails: %v", err)
}
if missing != 0 {
t.Fatalf("missing thumbnails = %d, want 0 after retry limit marks failed", missing)
}
}
if gen.generateCalls != defaultThumbTransientMediaMaxFailures {
t.Fatalf("generate calls = %d, want %d", gen.generateCalls, defaultThumbTransientMediaMaxFailures)
}
if err := cat.UpdateVideoMeta(ctx, video.ID, catalog.VideoMetaPatch{
ThumbnailStatus: "pending",
ResetThumbnailFailures: true,
}); err != nil {
t.Fatalf("reset thumbnail status: %v", err)
}
worker.rateLimit = rateLimitState{}
worker.process(ctx, video)
pending, err := cat.ListVideosByThumbnailStatus(ctx, video.DriveID, "pending", 0)
failed, err := cat.ListVideosByThumbnailStatus(ctx, video.DriveID, "failed", 0)
if err != nil {
t.Fatalf("list pending thumbnails after reset: %v", err)
t.Fatalf("list failed thumbnails: %v", err)
}
if len(pending) != 1 || pending[0].ID != video.ID {
t.Fatalf("pending thumbnails after reset = %#v, want only %s", pending, video.ID)
if len(failed) != 1 || failed[0].ID != video.ID {
t.Fatalf("failed thumbnails = %#v, want only %s", failed, video.ID)
}
if !worker.Status().CooldownUntil.IsZero() {
t.Fatalf("cooldown until = %s, want no cooldown for message-only media error", worker.Status().CooldownUntil)
}
if gen.generateCalls != 1 {
t.Fatalf("generate calls = %d, want 1", gen.generateCalls)
}
}
func TestThumbWorkerRequeuesP115TransientErrorBeforeRetryLimit(t *testing.T) {
func TestThumbWorkerDoesNotRequeueP115MessageOnlyError(t *testing.T) {
ctx := context.Background()
cat, video := seedPreviewTestVideo(t, "thumb-p115-requeue")
cat, video := seedPreviewTestVideo(t, "thumb-p115-no-requeue")
gen := &fakeThumbGenerator{
generateErr: errors.New("ffmpeg thumb: partial file Cannot determine format of input 0:0 after EOF"),
@@ -569,11 +494,8 @@ func TestThumbWorkerRequeuesP115TransientErrorBeforeRetryLimit(t *testing.T) {
select {
case queued := <-worker.ch:
if queued.ID != video.ID {
t.Fatalf("requeued video id = %q, want %q", queued.ID, video.ID)
}
t.Fatalf("unexpected requeued video id = %q", queued.ID)
default:
t.Fatal("expected transient thumbnail failure to requeue the same video")
}
got, err := cat.GetVideo(ctx, video.ID)
@@ -581,14 +503,14 @@ func TestThumbWorkerRequeuesP115TransientErrorBeforeRetryLimit(t *testing.T) {
t.Fatalf("get video: %v", err)
}
if got.ThumbnailURL != "" {
t.Fatalf("thumbnail = %q, want empty after transient failure", got.ThumbnailURL)
t.Fatalf("thumbnail = %q, want empty after message-only failure", got.ThumbnailURL)
}
pending, err := cat.ListVideosByThumbnailStatus(ctx, video.DriveID, "pending", 0)
failed, err := cat.ListVideosByThumbnailStatus(ctx, video.DriveID, "failed", 0)
if err != nil {
t.Fatalf("list pending thumbnails: %v", err)
t.Fatalf("list failed thumbnails: %v", err)
}
if len(pending) != 1 || pending[0].ID != video.ID {
t.Fatalf("pending thumbnails = %#v, want only %s", pending, video.ID)
if len(failed) != 1 || failed[0].ID != video.ID {
t.Fatalf("failed thumbnails = %#v, want only %s", failed, video.ID)
}
}
@@ -649,13 +571,15 @@ func TestP123TransientErrorsShouldCooldown(t *testing.T) {
drv := &previewFakeDrive{kind: "p123"}
for _, err := range []error{
errors.New("Server returned 403 Forbidden"),
errors.New("请求太频繁"),
errors.New("http 503 service unavailable"),
} {
if !driveErrorShouldCooldown(drv, err) {
t.Fatalf("driveErrorShouldCooldown(%v) = false, want true", err)
}
}
if driveErrorShouldCooldown(drv, errors.New("请求太频繁")) {
t.Fatal("message-only throttling text should not trigger p123 cooldown")
}
if driveErrorShouldCooldown(drv, errors.New("invalid credential")) {
t.Fatal("invalid credential should not trigger p123 cooldown")
}
@@ -666,31 +590,58 @@ func TestWopanTransientErrorsShouldCooldown(t *testing.T) {
for _, err := range []error{
errors.New("ffmpeg: Server returned 403 Forbidden"),
errors.New("wopan download url: request failed with status: 429 Too Many Requests"),
errors.New("操作频繁,请稍后重试"),
errors.New("http 503 service unavailable"),
} {
if !driveErrorShouldCooldown(drv, err) {
t.Fatalf("driveErrorShouldCooldown(%v) = false, want true", err)
}
}
if driveErrorShouldCooldown(drv, errors.New("操作频繁,请稍后重试")) {
t.Fatal("message-only throttling text should not trigger wopan cooldown")
}
if driveErrorShouldCooldown(drv, errors.New("invalid access token")) {
t.Fatal("invalid access token should not trigger wopan cooldown")
}
}
func TestGuangYaPanTransientErrorsShouldCooldown(t *testing.T) {
drv := &previewFakeDrive{kind: "guangyapan"}
for _, err := range []error{
errors.New("ffmpeg: Server returned 403 Forbidden"),
errors.New("guangyapan api rate limited: status=429 msg=操作频繁,请稍后重试"),
errors.New("http 503 service unavailable"),
} {
if !driveErrorShouldCooldown(drv, err) {
t.Fatalf("driveErrorShouldCooldown(%v) = false, want true", err)
}
}
if driveErrorShouldCooldown(drv, errors.New("操作频繁,请稍后重试")) {
t.Fatal("message-only throttling text should not trigger guangyapan cooldown")
}
if driveErrorShouldCooldown(drv, errors.New("invalid access token")) {
t.Fatal("invalid access token should not trigger guangyapan cooldown")
}
}
func TestGoogleDriveMediaErrorsShouldCooldown(t *testing.T) {
drv := &previewFakeDrive{kind: "googledrive"}
for _, err := range []error{
errors.New("google drive api error: usageLimits userRateLimitExceeded"),
errors.New("ffmpeg: Server returned 403 Forbidden"),
errors.New("downloadQuotaExceeded: The download quota for this file has been exceeded"),
errors.New("sharingRateLimitExceeded"),
errors.New("http 503 service unavailable"),
} {
if !driveErrorShouldCooldown(drv, err) {
t.Fatalf("driveErrorShouldCooldown(%v) = false, want true", err)
}
}
for _, err := range []error{
errors.New("google drive api error: usageLimits userRateLimitExceeded"),
errors.New("downloadQuotaExceeded: The download quota for this file has been exceeded"),
errors.New("sharingRateLimitExceeded"),
} {
if driveErrorShouldCooldown(drv, err) {
t.Fatalf("message-only google drive error %v should not trigger cooldown", err)
}
}
if driveErrorShouldCooldown(drv, errors.New("invalid credentials")) {
t.Fatal("invalid credentials should not trigger googledrive cooldown")
}
+3 -1
View File
@@ -151,13 +151,15 @@ func (p *Proxy) ServeStream(w http.ResponseWriter, r *http.Request, driveID, fil
// 先解出最终 Location,浏览器可直接 302 到该短期地址
// - wopan:联通网盘 GetDownloadUrlV2 返回的是短期直链,OpenList 也是直接
// 将该 URL 交给客户端使用;不需要后端持续代传视频字节
// - guangyapan:光鸭 get_res_download_url 返回 signedURL / downloadUrl
// 浏览器可直接访问,不需要后端持续代传视频字节
//
// 其余网盘(如夸克等)仍走反代,因为它们的下载
// 链接通常需要随请求带上后端持有的 Cookie / Authorization / Range
// 的特殊处理,浏览器拿不到这些上下文。
func shouldRedirect(d drives.Drive) bool {
switch d.Kind() {
case "p115", "pikpak", "onedrive", "p123", "wopan":
case "p115", "pikpak", "onedrive", "p123", "wopan", "guangyapan":
return true
}
return false
+25
View File
@@ -226,6 +226,31 @@ func TestServeStreamRedirectsWopan(t *testing.T) {
}
}
func TestServeStreamRedirectsGuangYaPan(t *testing.T) {
reg := NewRegistry()
drv := &proxyFakeSimpleDrive{
kind: "guangyapan",
url: "https://cdn.guangyapan.example/video.mp4?sign=encoded",
}
reg.Set("guangyapan", drv)
p := New(reg)
req := httptest.NewRequest(http.MethodGet, "/p/stream/guangyapan/file-1", nil)
rr := httptest.NewRecorder()
p.ServeStream(rr, req, "guangyapan", "file-1")
if rr.Code != http.StatusFound {
t.Fatalf("status = %d, want %d", rr.Code, http.StatusFound)
}
if got := rr.Header().Get("Location"); got != "https://cdn.guangyapan.example/video.mp4?sign=encoded" {
t.Fatalf("Location = %q", got)
}
if drv.calls != 1 {
t.Fatalf("link calls = %d, want 1", drv.calls)
}
}
func TestServeStreamServesLocalFilePath(t *testing.T) {
path := filepath.Join(t.TempDir(), "video.mp4")
if err := os.WriteFile(path, []byte("0123456789"), 0o644); err != nil {
+6 -2
View File
@@ -206,15 +206,19 @@ func (s *Scanner) walk(ctx context.Context, dirID, dirName string, stats *Stats,
patch.ContentHash = e.Hash
existing.ContentHash = e.Hash
}
if e.Name != "" && existing.FileName == "" {
if e.Name != "" && existing.FileName != e.Name {
patch.FileName = e.Name
existing.FileName = e.Name
patch.Title = parsed.Title
patch.TitleSet = true
patch.Author = parsed.Author
patch.AuthorSet = true
}
// 已存在但轻量元数据空缺时,顺便补齐。
if existing.Category == "" && dirName != "" {
patch.Category = dirName
}
if patch.Category != "" || patch.ContentHash != "" || patch.FileName != "" {
if patch.Category != "" || patch.ContentHash != "" || patch.FileName != "" || patch.TitleSet || patch.AuthorSet {
_ = s.Catalog.UpdateVideoMeta(ctx, id, patch)
if err := ctx.Err(); err != nil {
return err
+61
View File
@@ -323,6 +323,67 @@ func TestRunDoesNotBackfillRemoteThumbnailForExistingVideo(t *testing.T) {
}
}
func TestRunSyncsRenamedExistingVideoMetadata(t *testing.T) {
ctx := context.Background()
cat, err := catalog.Open(t.TempDir() + "/catalog.db")
if err != nil {
t.Fatalf("open catalog: %v", err)
}
t.Cleanup(func() {
if err := cat.Close(); err != nil {
t.Fatalf("close catalog: %v", err)
}
})
now := time.Now()
if err := cat.UpsertVideo(ctx, &catalog.Video{
ID: "fake-drive-file-1",
DriveID: "drive",
FileID: "file-1",
FileName: "old-name - Old Author.mp4",
Title: "old-name",
Author: "Old Author",
PreviewStatus: "pending",
PublishedAt: now,
CreatedAt: now,
UpdatedAt: now,
}); err != nil {
t.Fatalf("seed video: %v", err)
}
drv := &scannerFakeDrive{
entries: []drives.Entry{{
ID: "file-1",
Name: "[4K] renamed clip.mp4",
Size: 123,
ModTime: now,
}},
}
sc := New(cat, drv, []string{".mp4"}, nil, nil)
stats, err := sc.Run(ctx, "")
if err != nil {
t.Fatalf("scan: %v", err)
}
if stats.Added != 0 {
t.Fatalf("added = %d, want existing video to be updated in place", stats.Added)
}
got, err := cat.GetVideo(ctx, "fake-drive-file-1")
if err != nil {
t.Fatalf("get video: %v", err)
}
if got.FileName != "[4K] renamed clip.mp4" {
t.Fatalf("file_name = %q, want remote name", got.FileName)
}
if got.Title != "renamed clip" {
t.Fatalf("title = %q, want parsed title from remote name", got.Title)
}
if got.Author != "" {
t.Fatalf("author = %q, want cleared author from remote name without author suffix", got.Author)
}
}
func TestRunReplacesExistingVideoTagsWithFixedFilenameTags(t *testing.T) {
ctx := context.Background()
cat, err := catalog.Open(t.TempDir() + "/catalog.db")
+54 -24
View File
@@ -1,5 +1,5 @@
// Package spider91migrate 周期性把 spider91 drive 下载到本地的视频
// 上传到一个指定的目标 drive 目录(PikPak、115、123、OneDrive、Google Drive 或联通网盘),上传成功后:
// 上传到一个指定的目标 drive 目录(PikPak、115、123、OneDrive、Google Drive、联通网盘或光鸭网盘),上传成功后:
//
// - 改写 catalog 行:drive_id / file_id / content_hash 改成目标盘的;
// 视频自身的 id 不变(仍是 spider91-<driveID>-<viewkey>),video_tags、
@@ -31,6 +31,7 @@ import (
"github.com/video-site/backend/internal/catalog"
"github.com/video-site/backend/internal/drives"
"github.com/video-site/backend/internal/drives/googledrive"
"github.com/video-site/backend/internal/drives/guangyapan"
"github.com/video-site/backend/internal/drives/onedrive"
"github.com/video-site/backend/internal/drives/p115"
"github.com/video-site/backend/internal/drives/p123"
@@ -42,7 +43,7 @@ import (
)
// uploadTarget 是 migrator 调用目标 drive 的最小接口。任何一种"接收 spider91 上传"的
// 网盘都要实现它;当前 PikPak、115、123、OneDrive、Google Drive 和联通网盘各自通过适配器满足。
// 网盘都要实现它;当前 PikPak、115、123、OneDrive、Google Drive、联通网盘和光鸭网盘各自通过适配器满足。
//
// 这一层抽象把"迁移调用方"和"具体盘的 SDK 协议"解耦:
// - PikPak 走 GCID + OSS PutObjectpikpak.UploadResult
@@ -51,6 +52,7 @@ import (
// - OneDrive 走 SHA1 + 小文件 PUT / 大文件 upload session
// - Google Drive 走 MD5 + resumable upload session
// - 联通网盘 走 SDK Upload2C,当前上游不返回内容 hash
// - 光鸭网盘 走 OSS 分片上传,当前上游不返回内容 hash
//
// 各家返回值都被归一成本地的 UploadResult,并在 catalog 改写阶段统一处理。
type uploadTarget interface {
@@ -76,7 +78,7 @@ type Spider91LocalSource interface {
// UploadResult 是 uploadTarget.UploadAndReportHash 的归一返回。
//
// FileID 目标盘上的新文件 ID;
// Hash GCIDPikPak)、MD5 HEX123 / Google Drive)或 SHA1 HEX115 / OneDrive),写入 catalog.content_hash 用于跨盘去重;联通网盘暂为空;
// Hash GCIDPikPak)、MD5 HEX123 / Google Drive)或 SHA1 HEX115 / OneDrive),写入 catalog.content_hash 用于跨盘去重;联通网盘和光鸭网盘暂为空;
// Size 实际上传字节数。
type UploadResult struct {
FileID string
@@ -99,18 +101,19 @@ const (
)
type migrationPlan struct {
source Spider91LocalSource
row *catalog.Drive
sourceKinds []string
targetDriveID string
target uploadTarget
uploadDir string
keepLatestN int
requireAssetsReady bool
legacyBackfill bool
source Spider91LocalSource
row *catalog.Drive
sourceKinds []string
targetDriveID string
target uploadTarget
uploadDir string
keepLatestN int
requireAssetsReady bool
requirePreviewReady bool
legacyBackfill bool
}
// pikpakAdapter / p115Adapter / p123Adapter / onedriveAdapter / googledriveAdapter / wopanAdapter 把具体 driver 包装成 uploadTarget。
// pikpakAdapter / p115Adapter / p123Adapter / onedriveAdapter / googledriveAdapter / wopanAdapter / guangyapanAdapter 把具体 driver 包装成 uploadTarget。
//
// 之所以不让 driver 直接实现 uploadTarget
//
@@ -243,6 +246,27 @@ func (a *wopanAdapter) Rename(ctx context.Context, fileID, newName string) error
return a.d.Rename(ctx, fileID, newName)
}
type guangyapanAdapter struct {
d *guangyapan.Driver
}
func (a *guangyapanAdapter) ID() string { return a.d.ID() }
func (a *guangyapanAdapter) Kind() string { return a.d.Kind() }
func (a *guangyapanAdapter) RootID() string { return a.d.RootID() }
func (a *guangyapanAdapter) EnsureDir(ctx context.Context, pathFromRoot string) (string, error) {
return a.d.EnsureDir(ctx, pathFromRoot)
}
func (a *guangyapanAdapter) UploadAndReportHash(ctx context.Context, parentID, name string, r io.Reader, size int64) (UploadResult, error) {
fileID, err := a.d.Upload(ctx, parentID, name, r, size)
if err != nil {
return UploadResult{}, err
}
return UploadResult{FileID: fileID, Size: size}, nil
}
func (a *guangyapanAdapter) Rename(ctx context.Context, fileID, newName string) error {
return a.d.Rename(ctx, fileID, newName)
}
// adaptUploadTarget 把通用 drive 包装成 uploadTarget。
// 不支持的盘 kind 返回 error;调用方静默跳过。
func adaptUploadTarget(d drives.Drive) (uploadTarget, error) {
@@ -259,6 +283,8 @@ func adaptUploadTarget(d drives.Drive) (uploadTarget, error) {
return &googledriveAdapter{d: v}, nil
case *wopan.Driver:
return &wopanAdapter{d: v}, nil
case *guangyapan.Driver:
return &guangyapanAdapter{d: v}, nil
case uploadTarget:
// 测试或自定义实现可以直接传入;优先使用具体类型分支以拿到适配器。
return v, nil
@@ -572,14 +598,15 @@ func (m *Migrator) migrationPlans(ctx context.Context) []migrationPlan {
continue
}
out = append(out, migrationPlan{
source: src,
row: row,
sourceKinds: crawlerSourceKindsForRow(row),
targetDriveID: resolvedID,
target: target,
uploadDir: scriptCrawlerUploadDir(row.ID),
keepLatestN: 0,
requireAssetsReady: true,
source: src,
row: row,
sourceKinds: crawlerSourceKindsForRow(row),
targetDriveID: resolvedID,
target: target,
uploadDir: scriptCrawlerUploadDir(row.ID),
keepLatestN: 0,
requireAssetsReady: true,
requirePreviewReady: row.TeaserEnabled,
})
case spider91.Kind:
if m.cfg.GetTargetDriveID == nil {
@@ -813,7 +840,7 @@ func (m *Migrator) migrateDrive(ctx context.Context, plan migrationPlan) (int, e
}
if plan.requireAssetsReady {
ready, err := m.crawlerVideoAssetsReady(ctx, v)
ready, err := m.crawlerVideoAssetsReady(ctx, v, plan.requirePreviewReady)
if err != nil {
log.Printf("[spider91migrate] %s check generated assets: %v", v.ID, err)
continue
@@ -889,7 +916,7 @@ func (m *Migrator) findVideoForLocalFile(ctx context.Context, plan migrationPlan
return nil
}
func (m *Migrator) crawlerVideoAssetsReady(ctx context.Context, v *catalog.Video) (bool, error) {
func (m *Migrator) crawlerVideoAssetsReady(ctx context.Context, v *catalog.Video, requirePreview bool) (bool, error) {
if v == nil {
return false, nil
}
@@ -897,6 +924,9 @@ func (m *Migrator) crawlerVideoAssetsReady(ctx context.Context, v *catalog.Video
if !fingerprintReady {
return false, nil
}
if !requirePreview {
return true, nil
}
if strings.EqualFold(strings.TrimSpace(v.PreviewStatus), "ready") {
return true, nil
}
@@ -1183,7 +1213,7 @@ func (m *Migrator) cleanupOldLocalVideos(ctx context.Context, plan migrationPlan
return deleted, nil
}
// backfillFileNames 扫描目标 drivePikPak、115、123、OneDrive、Google Drive 或联通网盘)下所有 spider91-* 起始 ID 的视频,
// backfillFileNames 扫描目标 drivePikPak、115、123、OneDrive、Google Drive、联通网盘或光鸭网盘)下所有 spider91-* 起始 ID 的视频,
// 对文件名不是 desiredPikPakName(...) 期望格式的,调 target.Rename 修正,
// 并把 catalog.file_name 同步到新名字。
//
@@ -365,11 +365,19 @@ func seedScriptCrawlerDrive(t *testing.T, cat *catalog.Catalog, d *scriptcrawler
"script_path": "/tmp/crawler.py",
"upload_drive_id": uploadDriveID,
},
TeaserEnabled: true,
}); err != nil {
t.Fatalf("seed scriptcrawler drive: %v", err)
}
}
func setScriptCrawlerTeaserEnabled(t *testing.T, cat *catalog.Catalog, driveID string, enabled bool) {
t.Helper()
if err := cat.SetDriveTeaserEnabled(context.Background(), driveID, enabled); err != nil {
t.Fatalf("set scriptcrawler teaser enabled: %v", err)
}
}
func writeScriptCrawlerVideo(t *testing.T, cat *catalog.Catalog, d *scriptcrawler.Driver, sourceID, ext string, content []byte, readyAssets bool) string {
t.Helper()
fileID := sourceID + ext
@@ -587,6 +595,47 @@ func TestRunOnceSkipsScriptCrawlerVideoUntilPreviewAndFingerprintReady(t *testin
}
}
func TestRunOnceMigratesScriptCrawlerVideoWithoutPreviewWhenTeaserDisabled(t *testing.T) {
cat := setupCatalog(t)
src := setupScriptCrawler(t, "crawler-no-preview")
pp := newFakePikPak("pikpak-target", "pikpak-root-id")
seedScriptCrawlerDrive(t, cat, src, pp.ID())
setScriptCrawlerTeaserEnabled(t, cat, src.ID(), false)
reg := newFakeRegistry()
reg.Add(src)
reg.Add(pp)
id := writeScriptCrawlerVideo(t, cat, src, "fingerprint-ready", ".mp4", []byte("script video bytes"), false)
if err := cat.UpdateVideoFingerprint(context.Background(), id, "sampled-fingerprint-ready", "ready", ""); err != nil {
t.Fatalf("mark fingerprint ready: %v", err)
}
if err := cat.UpdatePreview(context.Background(), id, "", "disabled"); err != nil {
t.Fatalf("mark preview disabled: %v", err)
}
m := New(Config{Catalog: cat, Registry: reg})
m.runOnce(context.Background())
if pp.uploadCalls != 1 {
t.Fatalf("upload calls = %d, want 1 when preview generation is disabled", pp.uploadCalls)
}
got, err := cat.GetVideo(context.Background(), id)
if err != nil {
t.Fatalf("get migrated video: %v", err)
}
if got.DriveID != pp.ID() {
t.Fatalf("drive_id = %q, want %q", got.DriveID, pp.ID())
}
if got.PreviewStatus != "disabled" || got.FingerprintStatus != "ready" || got.SampledSHA256 == "" {
t.Fatalf("asset status after migration = preview %q fingerprint %q sampled %q, want disabled/ready/non-empty", got.PreviewStatus, got.FingerprintStatus, got.SampledSHA256)
}
videoPath, _ := src.VideoPath("fingerprint-ready.mp4")
if _, err := os.Stat(videoPath); !os.IsNotExist(err) {
t.Fatalf("local scriptcrawler video still exists or stat error %v", err)
}
}
func TestRunOnceBindsScriptCrawlerDuplicateToExistingTargetWithoutUpload(t *testing.T) {
cat := setupCatalog(t)
src := setupScriptCrawler(t, "crawler-duplicate")
@@ -1464,7 +1513,7 @@ func TestAdaptUploadTargetSupportsWopanDriver(t *testing.T) {
}
}
// TestResolveTargetRejectsUnsupportedKind 验证当目标 drive 既不是 PikPak、115、123、OneDrive、Google Drive 也不是联通网盘时,
// TestResolveTargetRejectsUnsupportedKind 验证当目标 drive 既不是 PikPak、115、123、OneDrive、Google Drive、联通网盘也不是光鸭网盘时,
// resolveTarget 拒绝并返回 error,让 runOnce 静默跳过(不会做破坏性变更)。
func TestResolveTargetRejectsUnsupportedKind(t *testing.T) {
cat := setupCatalog(t)
+283
View File
@@ -0,0 +1,283 @@
#!/usr/bin/env python3
# -*- coding: utf-8 -*-
"""
光鸭网盘 - 扫码登录脚本
========================
1. 调用 API 获取登录二维码
2. 保存二维码图片等待用户扫描
3. 扫描成功后保存用户凭证信息
"""
import io
import sys
# 修复 Windows 终端 GBK 编码问题
if sys.platform == 'win32':
sys.stdout = io.TextIOWrapper(sys.stdout.buffer, encoding='utf-8', errors='replace')
sys.stderr = io.TextIOWrapper(sys.stderr.buffer, encoding='utf-8', errors='replace')
import requests
import json
import time
import os
import sys
from datetime import datetime
# ========== 配置 ==========
API_ORIGIN = "https://account.guangyapan.com"
CLIENT_ID = "aMe-8VSlkrbQXpUR"
SCOPE = "user"
QR_IMAGE_PATH = "login_qr.png"
CREDENTIALS_PATH = "credentials.json"
# ========== 可选依赖 ==========
try:
import qrcode
HAS_QRCODE = True
except ImportError:
HAS_QRCODE = False
try:
from PIL import Image
HAS_PIL = True
except ImportError:
HAS_PIL = False
def generate_qr_image(url: str, path: str):
"""生成二维码图片"""
if HAS_QRCODE:
qr = qrcode.QRCode(
version=1,
error_correction=qrcode.constants.ERROR_CORRECT_M,
box_size=10,
border=4,
)
qr.add_data(url)
qr.make(fit=True)
img = qr.make_image(fill_color="black", back_color="white")
img.save(path)
print(f"[✓] 二维码已保存到: {path}")
else:
# Fallback: 使用 qrencode 命令行工具
import subprocess
try:
subprocess.run(["qrencode", "-o", path, url], check=True)
print(f"[✓] 二维码已保存到: {path}")
except FileNotFoundError:
print("[✗] 需要安装 qrcode 库: pip install qrcode[pil]")
print(f"[!] 请手动访问以下链接扫码:")
print(f" {url}")
return
# 尝试直接显示二维码到终端
try:
if HAS_PIL:
img = Image.open(path)
img.show()
print("[✓] 二维码已在图片查看器中打开")
except Exception:
pass
# 终端内显示小二维码
if HAS_QRCODE:
try:
qr.print_ascii(invert=True)
except Exception:
pass
def main():
session = requests.Session()
session.headers.update({
"User-Agent": "GuangYaPan-Login/1.0",
"Accept": "application/json",
"Content-Type": "application/json",
})
# ====== Step 1: 获取设备码和二维码链接 ======
print("=" * 60)
print("Step 1: 请求登录二维码...")
print("=" * 60)
device_code_url = f"{API_ORIGIN}/v1/auth/device/code"
device_payload = {
"client_id": CLIENT_ID,
"scope": SCOPE,
}
try:
resp = session.post(device_code_url, json=device_payload, timeout=30)
resp.raise_for_status()
device_data = resp.json()
except requests.exceptions.RequestException as e:
print(f"[✗] 请求失败: {e}")
if hasattr(e, 'response') and e.response is not None:
print(f" 响应内容: {e.response.text[:500]}")
sys.exit(1)
print(f"[✓] 设备码获取成功")
print(f" device_code: {device_data.get('device_code', 'N/A')[:30]}...")
print(f" interval: {device_data.get('interval', 5)}")
print(f" expires_in: {device_data.get('expires_in', 'N/A')}")
device_code = device_data["device_code"]
interval = int(device_data.get("interval", 5))
expires_in = int(device_data.get("expires_in", 300))
# 二维码链接
qr_url = device_data.get("verification_uri_complete") or device_data.get("short_uri_complete")
if not qr_url:
print("[✗] 响应中没有找到二维码链接")
print(f" 完整响应: {json.dumps(device_data, indent=2, ensure_ascii=False)}")
sys.exit(1)
print(f" qr_url: {qr_url}")
print()
# ====== Step 2: 生成并保存二维码 ======
print("=" * 60)
print("Step 2: 生成二维码图片...")
print("=" * 60)
generate_qr_image(qr_url, QR_IMAGE_PATH)
print()
print("!" * 60)
print("! 请使用「光鸭APP」扫描二维码登录")
print("!" * 60)
print()
# ====== Step 3: 轮询等待用户扫描 ======
print("=" * 60)
print("Step 3: 等待扫码授权...")
print("=" * 60)
token_url = f"{API_ORIGIN}/v1/auth/token"
token_payload = {
"client_id": CLIENT_ID,
"grant_type": "urn:ietf:params:oauth:grant-type:device_code",
"device_code": device_code,
}
start_time = time.time()
attempt = 0
while True:
attempt += 1
elapsed = time.time() - start_time
# 检查是否超时
if elapsed > expires_in:
print(f"\n[✗] 二维码已过期({expires_in}秒),请重新运行脚本")
sys.exit(1)
time.sleep(interval)
try:
resp = session.post(token_url, json=token_payload, timeout=30)
token_data = resp.json()
except requests.exceptions.RequestException as e:
print(f"\n[!] 网络错误: {e},重试中...")
continue
if "error" in token_data:
error = token_data["error"]
if error in ("authorization_pending", "slow_down"):
# 用户还未扫描或确认
dots = "." * ((attempt % 10) + 1)
print(f"\r 等待中{dots:<10} ({int(elapsed)}s / {expires_in}s)", end="", flush=True)
if error == "slow_down":
interval = min(interval * 2, 60)
continue
elif error == "expired_token":
print(f"\n[✗] 二维码已过期,请重新运行脚本")
sys.exit(1)
elif error == "access_denied":
print(f"\n[✗] 用户拒绝了授权")
sys.exit(1)
else:
print(f"\n[✗] 未知错误: {error}")
print(f" 完整响应: {json.dumps(token_data, indent=2, ensure_ascii=False)}")
sys.exit(1)
else:
# 成功!
print(f"\n[✓] 扫码授权成功!({int(elapsed)}s)")
break
# ====== Step 4: 保存凭证 ======
print()
print("=" * 60)
print("Step 4: 保存用户凭证...")
print("=" * 60)
# 保存完整 token 响应
credentials = {
"saved_at": datetime.now().isoformat(),
"api_origin": API_ORIGIN,
"client_id": CLIENT_ID,
"token_response": token_data,
"cookies": dict(session.cookies),
}
with open(CREDENTIALS_PATH, "w", encoding="utf-8") as f:
json.dump(credentials, f, indent=2, ensure_ascii=False)
print(f"[✓] 完整凭证已保存到: {CREDENTIALS_PATH}")
# 提取关键信息
access_token = token_data.get("access_token", "")
refresh_token = token_data.get("refresh_token", "")
id_token = token_data.get("id_token", "")
token_type = token_data.get("token_type", "Bearer")
expires_in = token_data.get("expires_in", 0)
print()
print("-" * 60)
print("凭证摘要:")
print("-" * 60)
print(f" access_token: {access_token[:50]}..." if access_token else " access_token: (无)")
print(f" refresh_token: {refresh_token[:50]}..." if refresh_token else " refresh_token: (无)")
print(f" id_token: {id_token[:50]}..." if id_token else " id_token: (无)")
print(f" token_type: {token_type}")
print(f" expires_in: {expires_in}")
print(f" scope: {token_data.get('scope', SCOPE)}")
print("-" * 60)
# 尝试获取用户信息
print()
print("=" * 60)
print("Step 5: 获取用户信息...")
print("=" * 60)
user_info_url = f"{API_ORIGIN}/v1/user/me"
try:
user_headers = {
"Authorization": f"{token_type} {access_token}",
}
user_resp = requests.get(user_info_url, headers=user_headers, timeout=15)
if user_resp.status_code == 200:
user_data = user_resp.json()
print("[✓] 用户信息获取成功:")
print(json.dumps(user_data, indent=2, ensure_ascii=False))
# 追加用户信息到凭证文件
credentials["user_info"] = user_data
with open(CREDENTIALS_PATH, "w", encoding="utf-8") as f:
json.dump(credentials, f, indent=2, ensure_ascii=False)
else:
print(f"[!] 获取用户信息返回 {user_resp.status_code}: {user_resp.text[:200]}")
except Exception as e:
print(f"[!] 获取用户信息失败: {e}")
print()
print("=" * 60)
print("完成!凭证文件: " + CREDENTIALS_PATH)
print("=" * 60)
if __name__ == "__main__":
main()
+9 -2
View File
@@ -3,8 +3,15 @@
<head>
<meta charset="UTF-8" />
<meta name="referrer" content="no-referrer" />
<link rel="icon" type="image/svg+xml" href="/favicon.svg" />
<link rel="icon" type="image/png" href="/icon.png" />
<link rel="apple-touch-icon" sizes="180x180" href="/apple-touch-icon.png" />
<link rel="manifest" href="/manifest.webmanifest" />
<meta name="viewport" content="width=device-width, initial-scale=1.0, viewport-fit=cover" />
<meta name="mobile-web-app-capable" content="yes" />
<meta name="apple-mobile-web-app-capable" content="yes" />
<meta name="apple-mobile-web-app-status-bar-style" content="black-translucent" />
<meta name="apple-mobile-web-app-title" content="91" />
<meta name="theme-color" content="#000000" />
<meta name="description" content="91 视频站" />
<title>91</title>
<!-- Premium Fonts Preconnect & Links -->
@@ -19,7 +26,7 @@
(function () {
try {
var t = localStorage.getItem("video-site:theme");
if (t === "pink" || t === "dark") {
if (t === "pink" || t === "dark" || t === "sky") {
document.documentElement.setAttribute("data-theme", t);
} else {
document.documentElement.setAttribute("data-theme", "dark");
+2 -7
View File
@@ -194,7 +194,7 @@ backup_install_files() {
local backup="$1"
mkdir -p "$backup"
cp -a "$INSTALL_PATH/server" "$backup/server"
for item in dist config.example.yaml 91VideoSpider config.yaml .version; do
for item in dist config.example.yaml config.yaml .version; do
if [[ -e "$INSTALL_PATH/$item" ]]; then
cp -a "$INSTALL_PATH/$item" "$backup/$item"
fi
@@ -205,7 +205,7 @@ restore_install_files() {
local backup="$1"
mkdir -p "$INSTALL_PATH"
cp -a "$backup/server" "$INSTALL_PATH/server"
for item in dist config.example.yaml 91VideoSpider config.yaml .version; do
for item in dist config.example.yaml config.yaml .version; do
rm -rf "${INSTALL_PATH:?}/$item"
if [[ -e "$backup/$item" ]]; then
cp -a "$backup/$item" "$INSTALL_PATH/$item"
@@ -441,7 +441,6 @@ process_looks_like_app() {
[[ "$cmd" == *"VIDEO_FRONTEND_DIR=$INSTALL_PATH/dist"* ]] && return 0
[[ "$cmd" == *"VIDEO_CONFIG=$INSTALL_PATH/config.yaml"* ]] && return 0
[[ "$cmd" == *"video-site-91"* ]] && return 0
[[ "$cmd" == *"91VideoSpider"* ]] && return 0
return 1
}
@@ -595,10 +594,6 @@ fetch_and_unpack() {
rm -rf "$INSTALL_PATH/dist"
cp -R "$root/dist" "$INSTALL_PATH/dist"
cp "$root/config.example.yaml" "$INSTALL_PATH/config.example.yaml"
if [[ -d "$root/91VideoSpider" ]]; then
rm -rf "$INSTALL_PATH/91VideoSpider"
cp -R "$root/91VideoSpider" "$INSTALL_PATH/91VideoSpider"
fi
chmod +x "$INSTALL_PATH/server"
rm -rf "$tmp"
}
+2 -2
View File
@@ -1,12 +1,12 @@
{
"name": "video-site",
"version": "0.1.7",
"version": "0.2.1",
"lockfileVersion": 3,
"requires": true,
"packages": {
"": {
"name": "video-site",
"version": "0.1.7",
"version": "0.2.1",
"license": "MIT",
"dependencies": {
"artplayer": "^5.4.0",
+1 -1
View File
@@ -2,7 +2,7 @@
"name": "video-site",
"private": true,
"license": "MIT",
"version": "0.1.7",
"version": "0.2.1",
"type": "module",
"scripts": {
"dev": "vite",
Binary file not shown.

After

Width:  |  Height:  |  Size: 21 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 136 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 114 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 19 KiB

-28
View File
@@ -1,28 +0,0 @@
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 32 32">
<defs>
<!-- Background Gradient: Warm Orange to Sakura Pink, representing both themes -->
<linearGradient id="bg-grad" x1="0%" y1="0%" x2="100%" y2="100%">
<stop offset="0%" stop-color="#FF7E40" />
<stop offset="100%" stop-color="#FF4B91" />
</linearGradient>
<!-- Subtle drop shadow for the play button to give it depth -->
<filter id="shadow" x="-20%" y="-20%" width="140%" height="140%">
<feDropShadow dx="0" dy="1.5" stdDeviation="1" flood-opacity="0.25" />
</filter>
</defs>
<!-- Main Squircle Background -->
<rect x="2" y="2" width="28" height="28" rx="8" fill="url(#bg-grad)" />
<!-- Inner border for a premium, glassmorphic feel -->
<rect x="3" y="3" width="26" height="26" rx="7" fill="none" stroke="#ffffff" stroke-width="1" opacity="0.2" />
<!-- Stylized Play Button Icon, perfectly centered with rounded corners and drop shadow -->
<path d="M13 10.5 L21.5 16 L13 21.5 Z"
fill="#ffffff"
stroke="#ffffff"
stroke-width="2.5"
stroke-linejoin="round"
filter="url(#shadow)" />
</svg>

Before

Width:  |  Height:  |  Size: 1.1 KiB

BIN
View File
Binary file not shown.

After

Width:  |  Height:  |  Size: 212 KiB

+30
View File
@@ -0,0 +1,30 @@
{
"name": "91",
"short_name": "91",
"start_url": "/",
"scope": "/",
"display": "standalone",
"display_override": ["fullscreen", "standalone"],
"background_color": "#000000",
"theme_color": "#000000",
"icons": [
{
"src": "/app-icon-192.png",
"sizes": "192x192",
"type": "image/png",
"purpose": "any"
},
{
"src": "/app-icon-512.png",
"sizes": "512x512",
"type": "image/png",
"purpose": "any"
},
{
"src": "/app-icon-maskable-512.png",
"sizes": "512x512",
"type": "image/png",
"purpose": "maskable"
}
]
}
Binary file not shown.

After

Width:  |  Height:  |  Size: 864 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 855 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 1.3 MiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 2.5 MiB

+101 -73
View File
@@ -1,83 +1,111 @@
import { Suspense, lazy } from "react";
import { Navigate, Route, Routes } from "react-router-dom";
import HomePage from "@/pages/HomePage";
import ListingPage from "@/pages/ListingPage";
import ShortsPage from "@/pages/ShortsPage";
import UploadPage from "@/pages/UploadPage";
import VideoDetailPage from "@/pages/VideoDetailPage";
import { AdminLayout } from "@/admin/AdminLayout";
import { LoginPage } from "@/admin/LoginPage";
import { SkyStarfield } from "@/components/SkyStarfield";
import { RequireAuth } from "@/admin/RequireAuth";
import { DrivesPage } from "@/admin/DrivesPage";
import { CrawlersPage } from "@/admin/CrawlersPage";
import { VideosPage } from "@/admin/VideosPage";
import { TagsPage } from "@/admin/TagsPage";
import { ThemePage } from "@/admin/ThemePage";
const HomePage = lazy(() => import("@/pages/HomePage"));
const ListingPage = lazy(() => import("@/pages/ListingPage"));
const ShortsPage = lazy(() => import("@/pages/ShortsPage"));
const UploadPage = lazy(() => import("@/pages/UploadPage"));
const VideoDetailPage = lazy(() => import("@/pages/VideoDetailPage"));
const LoginPage = lazy(() =>
import("@/admin/LoginPage").then((module) => ({ default: module.LoginPage }))
);
const AdminLayout = lazy(() =>
import("@/admin/AdminLayout").then((module) => ({
default: module.AdminLayout,
}))
);
const DrivesPage = lazy(() =>
import("@/admin/DrivesPage").then((module) => ({ default: module.DrivesPage }))
);
const CrawlersPage = lazy(() =>
import("@/admin/CrawlersPage").then((module) => ({
default: module.CrawlersPage,
}))
);
const VideosPage = lazy(() =>
import("@/admin/VideosPage").then((module) => ({ default: module.VideosPage }))
);
const TagsPage = lazy(() =>
import("@/admin/TagsPage").then((module) => ({ default: module.TagsPage }))
);
const ThemePage = lazy(() =>
import("@/admin/ThemePage").then((module) => ({ default: module.ThemePage }))
);
export default function App() {
return (
<Routes>
<Route path="/login" element={<LoginPage />} />
<>
{/* 星空蓝主题的固定位置星星层,仅在 data-theme="sky" 下可见 */}
<SkyStarfield />
<Suspense fallback={null}>
<Routes>
<Route path="/login" element={<LoginPage />} />
{/* 主站需要登录 */}
<Route
path="/"
element={
<RequireAuth>
<HomePage />
</RequireAuth>
}
/>
<Route
path="/list"
element={
<RequireAuth>
<ListingPage />
</RequireAuth>
}
/>
<Route
path="/shorts"
element={
<RequireAuth>
<ShortsPage />
</RequireAuth>
}
/>
<Route
path="/upload"
element={
<RequireAuth>
<UploadPage />
</RequireAuth>
}
/>
<Route
path="/video/:id"
element={
<RequireAuth>
<VideoDetailPage />
</RequireAuth>
}
/>
{/* 主站需要登录 */}
<Route
path="/"
element={
<RequireAuth>
<HomePage />
</RequireAuth>
}
/>
<Route
path="/list"
element={
<RequireAuth>
<ListingPage />
</RequireAuth>
}
/>
<Route
path="/shorts"
element={
<RequireAuth>
<ShortsPage />
</RequireAuth>
}
/>
<Route
path="/upload"
element={
<RequireAuth>
<UploadPage />
</RequireAuth>
}
/>
<Route
path="/video/:id"
element={
<RequireAuth>
<VideoDetailPage />
</RequireAuth>
}
/>
{/* 管理后台也需要登录 */}
<Route
path="/admin"
element={
<RequireAuth>
<AdminLayout />
</RequireAuth>
}
>
<Route index element={<Navigate to="/admin/drives" replace />} />
<Route path="drives" element={<DrivesPage />} />
<Route path="crawlers" element={<CrawlersPage />} />
<Route path="videos" element={<VideosPage />} />
<Route path="tags" element={<TagsPage />} />
<Route path="theme" element={<ThemePage />} />
</Route>
{/* 管理后台也需要登录 */}
<Route
path="/admin"
element={
<RequireAuth>
<AdminLayout />
</RequireAuth>
}
>
<Route index element={<Navigate to="/admin/drives" replace />} />
<Route path="drives" element={<DrivesPage />} />
<Route path="crawlers" element={<CrawlersPage />} />
<Route path="videos" element={<VideosPage />} />
<Route path="tags" element={<TagsPage />} />
<Route path="theme" element={<ThemePage />} />
</Route>
<Route path="*" element={<Navigate to="/" replace />} />
</Routes>
<Route path="*" element={<Navigate to="/" replace />} />
</Routes>
</Suspense>
</>
);
}
-7
View File
@@ -4,7 +4,6 @@ import {
HardDrive,
Film,
LogOut,
Play,
Home,
Tags,
Palette,
@@ -71,12 +70,6 @@ export function AdminLayout() {
return (
<div className="admin-shell">
<aside className="admin-sidebar">
<div className="admin-sidebar__brand">
<span className="admin-sidebar__brand-mark">
<Play size={14} fill="#000" />
</span>
<span className="admin-sidebar__brand-text">91</span>
</div>
<nav className="admin-nav">
<div className="admin-nav__group admin-nav__group--home">
<span className="admin-nav__group-label"></span>
+71 -50
View File
@@ -18,6 +18,8 @@ import {
Link as LinkIcon,
Pencil,
Plus,
Power,
PowerOff,
RefreshCw,
TestTube,
Trash2,
@@ -33,7 +35,7 @@ import { SpiderIcon } from "./icons/SpiderIcon";
const BUSY_STATES = new Set(["scanning", "generating", "uploading", "queued"]);
const POLL_INTERVAL_MS = 5000;
const UPLOAD_TARGET_KINDS = new Set(["p115", "pikpak", "p123", "googledrive", "onedrive", "wopan"]);
const UPLOAD_TARGET_KINDS = new Set(["p115", "pikpak", "p123", "googledrive", "onedrive", "wopan", "guangyapan"]);
function statusBusy(status?: api.DriveGenerationStatus) {
return BUSY_STATES.has(status?.state ?? "");
@@ -55,7 +57,9 @@ export function CrawlersPage() {
const [loading, setLoading] = useState(true);
const [expandedId, setExpandedId] = useState("");
const [runningId, setRunningId] = useState("");
const [uploadingId, setUploadingId] = useState("");
const [stoppingId, setStoppingId] = useState("");
const [togglingTeaserId, setTogglingTeaserId] = useState("");
// undefined = 编辑器关闭;null = 新建;其余 = 编辑已有爬虫
const [editorTarget, setEditorTarget] = useState<api.AdminCrawler | null | undefined>(undefined);
const [deleteTarget, setDeleteTarget] = useState<api.AdminCrawler | null>(null);
@@ -123,6 +127,23 @@ export function CrawlersPage() {
}
}
async function uploadVideos(crawler: api.AdminCrawler) {
setUploadingId(crawler.id);
try {
const resp = await api.uploadCrawlerVideos(crawler.id);
if (!resp.accepted) {
show(resp.message || "当前爬虫暂不满足上传条件", "info");
return;
}
show("已触发上传任务", "success");
await refresh(true);
} catch (e) {
show(e instanceof Error ? e.message : "触发上传失败", "error");
} finally {
setUploadingId("");
}
}
async function stop(crawler: api.AdminCrawler) {
setStoppingId(crawler.id);
try {
@@ -136,6 +157,23 @@ export function CrawlersPage() {
}
}
async function toggleTeaser(crawler: api.AdminCrawler) {
const next = !crawler.teaserEnabled;
setTogglingTeaserId(crawler.id);
setList((prev) => prev.map((item) => (item.id === crawler.id ? { ...item, teaserEnabled: next } : item)));
try {
const resp = await api.setDriveTeaserEnabled(crawler.id, next);
setList((prev) => prev.map((item) => (item.id === crawler.id ? { ...item, teaserEnabled: resp.teaserEnabled } : item)));
show(resp.teaserEnabled ? `已开启「${crawler.name}」预览视频生成` : `已关闭「${crawler.name}」预览视频生成`, "success");
await refresh(true);
} catch (e) {
setList((prev) => prev.map((item) => (item.id === crawler.id ? { ...item, teaserEnabled: crawler.teaserEnabled } : item)));
show(e instanceof Error ? e.message : "切换预览视频失败", "error");
} finally {
setTogglingTeaserId("");
}
}
async function confirmDelete() {
if (!deleteTarget) return;
setDeleting(true);
@@ -213,10 +251,14 @@ export function CrawlersPage() {
crawler={crawler}
expanded={expandedId === crawler.id}
running={runningId === crawler.id}
uploading={uploadingId === crawler.id}
stopping={stoppingId === crawler.id}
togglingTeaser={togglingTeaserId === crawler.id}
onToggle={() => setExpandedId(expandedId === crawler.id ? "" : crawler.id)}
onRun={() => run(crawler)}
onUpload={() => uploadVideos(crawler)}
onStop={() => stop(crawler)}
onToggleTeaser={() => toggleTeaser(crawler)}
onEdit={() => setEditorTarget(crawler)}
onDelete={() => setDeleteTarget(crawler)}
/>
@@ -262,51 +304,37 @@ function CrawlerMetric({ label, value, icon, tone }: { label: string; value: num
);
}
type StageInfo = {
key: string;
label: string;
status?: api.DriveGenerationStatus;
};
function crawlerStages(crawler: api.AdminCrawler): StageInfo[] {
return [
{ key: "scan", label: "抓取", status: crawler.scanGenerationStatus },
{ key: "thumbnail", label: "封面", status: crawler.thumbnailGenerationStatus },
{ key: "preview", label: "预览", status: crawler.previewGenerationStatus },
{ key: "fingerprint", label: "指纹", status: crawler.fingerprintGenerationStatus },
{ key: "upload", label: "上传", status: crawler.uploadGenerationStatus },
];
}
function stageStateLabel(stage: StageInfo): string {
const state = stage.status?.state || "idle";
if (stage.key === "scan" && state === "scanning") return "抓取中";
if (stage.key === "upload" && state === "uploading") return "上传中";
return generationStateLabel(state);
}
function CrawlerRow({
crawler,
expanded,
running,
uploading,
stopping,
togglingTeaser,
onToggle,
onRun,
onUpload,
onStop,
onToggleTeaser,
onEdit,
onDelete,
}: {
crawler: api.AdminCrawler;
expanded: boolean;
running: boolean;
uploading: boolean;
stopping: boolean;
togglingTeaser: boolean;
onToggle: () => void;
onRun: () => void;
onUpload: () => void;
onStop: () => void;
onToggleTeaser: () => void;
onEdit: () => void;
onDelete: () => void;
}) {
const busy = crawlerBusy(crawler);
const uploadButtonTitle = uploading ? "上传请求处理中" : "上传本地爬虫视频到已配置的上传网盘";
return (
<div className={`admin-crawler-row ${expanded ? "is-expanded" : ""}`}>
<div className="admin-crawler-row__line">
@@ -320,29 +348,20 @@ function CrawlerRow({
{formatLastCrawl(crawler.lastCrawlAt)} · {crawler.targetNew || "10"} · {crawler.totalCrawledCount ?? 0}
</span>
</span>
<span className="admin-crawler-pipeline">
{crawlerStages(crawler).map((stage) => {
const state = stage.status?.state || "idle";
const active = BUSY_STATES.has(state) || state === "cooling";
return (
<span
key={stage.key}
className={`admin-crawler-stage is-${generationStateClass(state)}`}
title={`${stage.label}${stageStateLabel(stage)}`}
>
<span className="admin-crawler-stage__dot" />
{stage.label}
{active && <em>{stageStateLabel(stage)}</em>}
</span>
);
})}
</span>
<span className={`admin-status is-${crawler.status === "ok" ? "ok" : crawler.status === "error" ? "error" : "pending"}`}>
{crawlerStatusLabel(crawler)}
</span>
<ChevronDown size={16} className="admin-crawler-row__chevron" />
</button>
<div className="admin-crawler-row__actions">
<button
className="admin-btn admin-crawler-preview-card-toggle"
type="button"
onClick={onToggleTeaser}
disabled={togglingTeaser}
aria-pressed={crawler.teaserEnabled}
title={crawler.teaserEnabled ? "关闭后,该爬虫新爬取的视频不再生成预览视频" : "开启后,该爬虫新爬取的视频会生成预览视频"}
>
{crawler.teaserEnabled ? <Power size={13} /> : <PowerOff size={13} />}
<span>{crawler.teaserEnabled ? "预览:开" : "预览:关"}</span>
</button>
{busy ? (
<button className="admin-btn is-stop" type="button" onClick={onStop} disabled={stopping}>
<CircleStop size={13} /> {stopping ? "停止中..." : "停止"}
@@ -352,6 +371,14 @@ function CrawlerRow({
<Download size={13} /> {running ? "触发中..." : "立即抓取"}
</button>
)}
<button
className="admin-btn"
type="button"
onClick={onUpload}
title={uploadButtonTitle}
>
<Upload size={13} /> {uploading ? "上传中..." : "上传视频"}
</button>
<button className="admin-btn" type="button" onClick={onEdit}>
<Pencil size={13} />
</button>
@@ -1038,12 +1065,6 @@ function crawlerTestFailure(result: api.CrawlerDryRunResult) {
return result.error || result.mediaCheck?.error || "";
}
function crawlerStatusLabel(crawler: api.AdminCrawler) {
if (crawler.status === "ok") return "已就绪";
if (crawler.status === "error") return "错误";
return "未连接";
}
function formatLastCrawl(ts?: number) {
if (!ts) return "从未";
return new Date(ts * 1000).toLocaleString("zh-CN", {
+8 -4
View File
@@ -102,7 +102,8 @@ export function DrivesPage() {
d.kind === "p123" ||
d.kind === "onedrive" ||
d.kind === "googledrive" ||
d.kind === "wopan"
d.kind === "wopan" ||
d.kind === "guangyapan"
),
[list]
);
@@ -216,7 +217,10 @@ export function DrivesPage() {
d.kind === "spider91"
? { proxy: d.spider91Proxy ?? "" }
: d.kind === "googledrive"
? { use_online_api: (d.googleDriveUseOnlineAPI ?? true) ? "true" : "false" }
? {
use_online_api: (d.googleDriveUseOnlineAPI ?? true) ? "true" : "false",
api_url_address: d.googleDriveOpenListApiUrl ?? "",
}
: d.kind === "localstorage"
? { strm_allow_outside_root: (d.strmAllowOutsideRoot ?? false) ? "true" : "false" }
: {},
@@ -631,7 +635,7 @@ export function DrivesPage() {
</button>
<button
type="button"
className="admin-btn is-stop"
className="admin-btn is-primary"
onClick={() => handleStopDriveTasks(d)}
disabled={!!stoppingDriveId}
title="停止此网盘当前的扫描、封面、预览视频和视频指纹生成任务。"
@@ -641,7 +645,7 @@ export function DrivesPage() {
</button>
</div>
{d.kind !== "spider91" && (
<button type="button" className="admin-btn" onClick={() => openEdit(d)}>
<button type="button" className="admin-btn is-primary" onClick={() => openEdit(d)}>
</button>
)}
+5 -3
View File
@@ -79,9 +79,11 @@ export function LoginPage() {
return (
<div className="admin-login">
<form className="admin-login__card" onSubmit={handleSubmit}>
<h1 className="admin-login__title">
<Play size={18} fill="currentColor" /> {setupRequired ? "首次设置管理员" : "登录"}
</h1>
{setupRequired && (
<h1 className="admin-login__title">
<Play size={18} fill="currentColor" />
</h1>
)}
<div className="admin-form">
<div className="admin-form__row">
<label htmlFor="admin-login-username"></label>
+9 -2
View File
@@ -1,12 +1,12 @@
import { useEffect, useState } from "react";
import { Check, Loader2, Moon, Sparkles } from "lucide-react";
import { Check, Loader2, Moon, Sparkles, Star } from "lucide-react";
import * as api from "./api";
import type { Theme } from "./api";
import { useToast } from "./ToastContext";
import { applyTheme, getCurrentTheme } from "@/lib/theme";
function isTheme(value: unknown): value is Theme {
return value === "dark" || value === "pink";
return value === "dark" || value === "pink" || value === "sky";
}
type Option = {
@@ -32,6 +32,13 @@ const OPTIONS: Option[] = [
description: "柔和奶白底 + 樱花粉主色,清爽温柔,日间使用更舒适。",
icon: Sparkles,
},
{
id: "sky",
title: "星空蓝 + 暖星黄",
subtitle: "Starry Sky",
description: "浅天空蓝底 + 暖星黄主色,配上淡淡的网格与点点星光,顶级美感。",
icon: Star,
},
];
/**
+128 -20
View File
@@ -9,35 +9,125 @@ import {
} from "react";
type ToastKind = "info" | "success" | "error";
type Toast = { id: number; kind: ToastKind; text: string };
type Toast = { id: number; kind: ToastKind; text: string; copyable: boolean };
type Ctx = {
show: (text: string, kind?: ToastKind) => void;
};
const ToastCtx = createContext<Ctx | null>(null);
const TOAST_DISMISS_MS = 2500;
const TOAST_COPY_SUCCESS_TEXT = "已复制到剪贴板";
const TOAST_COPY_ERROR_TEXT = "复制失败,请手动复制";
async function copyTextToClipboard(text: string) {
try {
if (navigator.clipboard?.writeText) {
await navigator.clipboard.writeText(text);
return;
}
} catch {
// Fall back to the legacy copy command below.
}
if (!fallbackCopyText(text)) {
throw new Error("copy failed");
}
}
function fallbackCopyText(text: string) {
if (!document.body) return false;
const textarea = document.createElement("textarea");
textarea.value = text;
textarea.setAttribute("readonly", "");
textarea.style.position = "fixed";
textarea.style.left = "-9999px";
textarea.style.top = "0";
document.body.appendChild(textarea);
textarea.select();
try {
return document.execCommand("copy");
} finally {
document.body.removeChild(textarea);
}
}
export function ToastProvider({ children }: { children: ReactNode }) {
const [items, setItems] = useState<Toast[]>([]);
const timers = useRef<Record<string, ReturnType<typeof window.setTimeout>>>({});
const timers = useRef(new Map<number, ReturnType<typeof window.setTimeout>>());
const idsByText = useRef(new Map<string, number>());
const clearDismissTimer = useCallback((id: number) => {
const timer = timers.current.get(id);
if (!timer) return;
window.clearTimeout(timer);
timers.current.delete(id);
}, []);
const removeToast = useCallback(
(id: number, text: string) => {
clearDismissTimer(id);
if (idsByText.current.get(text) === id) {
idsByText.current.delete(text);
}
setItems((list) => list.filter((t) => t.id !== id));
},
[clearDismissTimer]
);
const scheduleDismiss = useCallback(
(id: number, text: string) => {
clearDismissTimer(id);
timers.current.set(
id,
window.setTimeout(() => removeToast(id, text), TOAST_DISMISS_MS)
);
},
[clearDismissTimer, removeToast]
);
const addToast = useCallback(
(text: string, kind: ToastKind = "info", copyable = true) => {
const existingID = idsByText.current.get(text);
if (existingID !== undefined) {
setItems((list) =>
list.map((t) => (t.id === existingID ? { ...t, kind, copyable } : t))
);
scheduleDismiss(existingID, text);
return;
}
const id = Date.now() + Math.random();
idsByText.current.set(text, id);
setItems((list) => [...list, { id, kind, text, copyable }]);
scheduleDismiss(id, text);
},
[scheduleDismiss]
);
// Deduplicate: same text won't stack, just resets the dismiss timer
const show = useCallback((text: string, kind: ToastKind = "info") => {
// Reset timer if duplicate
if (timers.current[text]) {
window.clearTimeout(timers.current[text]);
timers.current[text] = window.setTimeout(() => {
setItems((list) => list.filter((t) => t.text !== text));
delete timers.current[text];
}, 2600);
return;
}
const id = Date.now() + Math.random();
timers.current[text] = window.setTimeout(() => {
setItems((list) => list.filter((t) => t.id !== id));
delete timers.current[text];
}, 2600);
setItems((list) => [...list, { id, kind, text }]);
const show = useCallback(
(text: string, kind: ToastKind = "info") => {
addToast(text, kind, true);
},
[addToast]
);
const copyToastText = useCallback(
(text: string) => {
void copyTextToClipboard(text)
.then(() => addToast(TOAST_COPY_SUCCESS_TEXT, "success", false))
.catch(() => addToast(TOAST_COPY_ERROR_TEXT, "error", false));
},
[addToast]
);
useEffect(() => {
return () => {
for (const timer of timers.current.values()) {
window.clearTimeout(timer);
}
timers.current.clear();
idsByText.current.clear();
};
}, []);
return (
@@ -45,8 +135,26 @@ export function ToastProvider({ children }: { children: ReactNode }) {
{children}
<div className="admin-toast-stack" role="status" aria-live="polite">
{items.map((t) => (
<div key={t.id} className={`admin-toast is-${t.kind}`}>
{t.text}
<div
key={t.id}
className={`admin-toast is-${t.kind}${
t.copyable ? " is-copyable" : ""
}`}
role={t.copyable ? "button" : undefined}
tabIndex={t.copyable ? 0 : undefined}
aria-label={t.copyable ? `复制提示:${t.text}` : undefined}
onClick={t.copyable ? () => copyToastText(t.text) : undefined}
onKeyDown={
t.copyable
? (event) => {
if (event.key !== "Enter" && event.key !== " ") return;
event.preventDefault();
copyToastText(t.text);
}
: undefined
}
>
<span className="admin-toast__text">{t.text}</span>
</div>
))}
</div>
+96 -6
View File
@@ -21,9 +21,17 @@ import { formatBytes } from "./storageFormat";
const DESKTOP_VIDEOS_PAGE_SIZE = 50;
const MOBILE_VIDEOS_PAGE_SIZE = 20;
const VIDEOS_MOBILE_QUERY = "(max-width: 640px)";
const REGEN_PREVIEW_STATUS = "generating";
const REGEN_PREVIEW_POLL_INTERVAL_MS = 2000;
const REGEN_PREVIEW_TRACK_TIMEOUT_MS = 30 * 60 * 1000;
type TabKey = "current" | "blacklist";
type RegenPreviewState = {
expiresAt: number;
originalUpdatedAt: number;
};
const TABS: { key: TabKey; label: string }[] = [
{ key: "current", label: "当前视频" },
{ key: "blacklist", label: "拉黑视频" },
@@ -121,6 +129,7 @@ function CurrentVideosTab({ onStatsChanged }: { onStatsChanged: () => void }) {
const [deleteTarget, setDeleteTarget] = useState<api.AdminVideo | null>(null);
const [deleting, setDeleting] = useState(false);
const [deleteSource, setDeleteSource] = useState(false);
const [regenPreviewById, setRegenPreviewById] = useState<Record<string, RegenPreviewState>>({});
const pageSize = useVideosPageSize();
const { show } = useToast();
@@ -147,6 +156,19 @@ function CurrentVideosTab({ onStatsChanged }: { onStatsChanged: () => void }) {
}
}
async function refreshListOnly() {
try {
const r = await api.listVideos({ driveId, page, size: pageSize, keyword: searchKeyword });
setList(r.items ?? []);
setTotal(r.total ?? 0);
} catch {
// Polling is only used to clear optimistic preview-generation state.
}
}
const trackedRegenCount = Object.keys(regenPreviewById).length;
const hasGeneratingPreview = list.some((v) => v.previewStatus === REGEN_PREVIEW_STATUS);
useEffect(() => {
refresh();
}, [driveId, page, searchKeyword, pageSize]);
@@ -164,6 +186,33 @@ function CurrentVideosTab({ onStatsChanged }: { onStatsChanged: () => void }) {
return () => window.clearTimeout(timer);
}, [keyword]);
useEffect(() => {
if (trackedRegenCount === 0 && !hasGeneratingPreview) return;
const timer = window.setInterval(() => {
refreshListOnly();
}, REGEN_PREVIEW_POLL_INTERVAL_MS);
return () => window.clearInterval(timer);
}, [trackedRegenCount, hasGeneratingPreview, driveId, page, pageSize, searchKeyword]);
useEffect(() => {
if (trackedRegenCount === 0) return;
const now = Date.now();
setRegenPreviewById((current) => {
const next = { ...current };
let changed = false;
const byId = new Map(list.map((v) => [v.id, v]));
for (const [id, state] of Object.entries(current)) {
const video = byId.get(id);
const updatedAt = videoUpdatedAtMs(video);
if (!video || now >= state.expiresAt || updatedAt > state.originalUpdatedAt) {
delete next[id];
changed = true;
}
}
return changed ? next : current;
});
}, [list, trackedRegenCount]);
const driveNameMap = new Map(drives.map((d) => [d.id, d.name || d.id]));
const listItems = list;
@@ -177,6 +226,7 @@ function CurrentVideosTab({ onStatsChanged }: { onStatsChanged: () => void }) {
async function handleRegen(v: api.AdminVideo) {
try {
await api.regenPreview(v.id);
trackRegeneratingPreview([v]);
show("已触发预览视频重生", "success");
} catch (e) {
show(e instanceof Error ? e.message : "触发失败", "error");
@@ -196,13 +246,20 @@ function CurrentVideosTab({ onStatsChanged }: { onStatsChanged: () => void }) {
async function confirmBatchRegen() {
const ids = [...selectedIds];
const videoById = new Map(listItems.map((v) => [v.id, v]));
setBatchRegening(true);
let success = 0;
try {
const results = await Promise.allSettled(ids.map((id) => api.regenPreview(id)));
for (const r of results) {
if (r.status === "fulfilled") success++;
}
const acceptedVideos: api.AdminVideo[] = [];
results.forEach((r, index) => {
if (r.status === "fulfilled") {
const video = videoById.get(ids[index]);
if (video) acceptedVideos.push(video);
success++;
}
});
trackRegeneratingPreview(acceptedVideos);
show(
`批量触发完成,成功 ${success} / ${ids.length}`,
success === ids.length ? "success" : "info"
@@ -214,6 +271,25 @@ function CurrentVideosTab({ onStatsChanged }: { onStatsChanged: () => void }) {
}
}
function trackRegeneratingPreview(videos: api.AdminVideo[]) {
if (videos.length === 0) return;
const startedAt = Date.now();
setRegenPreviewById((current) => {
const next = { ...current };
for (const v of videos) {
next[v.id] = {
expiresAt: startedAt + REGEN_PREVIEW_TRACK_TIMEOUT_MS,
originalUpdatedAt: videoUpdatedAtMs(v),
};
}
return next;
});
}
function isPreviewGenerating(v: api.AdminVideo) {
return !!regenPreviewById[v.id] || v.previewStatus === REGEN_PREVIEW_STATUS;
}
async function confirmDeleteVideo() {
if (!deleteTarget) return;
const target = deleteTarget;
@@ -398,7 +474,7 @@ function CurrentVideosTab({ onStatsChanged }: { onStatsChanged: () => void }) {
<td data-label="作者">{v.author || <span className="admin-text-faint"></span>}</td>
<td data-label="时长">{formatDur(v.durationSeconds)}</td>
<td data-label="预览视频">
<PreviewStatus s={v.previewStatus} />
<PreviewStatus s={isPreviewGenerating(v) ? REGEN_PREVIEW_STATUS : v.previewStatus} />
</td>
<td data-label="来源" className="admin-mono-cell">
{driveNameMap.get(v.driveId) ?? v.driveId}
@@ -407,8 +483,14 @@ function CurrentVideosTab({ onStatsChanged }: { onStatsChanged: () => void }) {
<button type="button" className="admin-btn" onClick={() => setEditing(v)} title="编辑视频">
<Edit size={13} />
</button>{" "}
<button type="button" className="admin-btn" onClick={() => handleRegen(v)} title="重生预览视频">
<RefreshCw size={13} />
<button
type="button"
className="admin-btn"
onClick={() => handleRegen(v)}
disabled={isPreviewGenerating(v)}
title={isPreviewGenerating(v) ? "预览视频正在生成" : "重生预览视频"}
>
<RefreshCw size={13} className={isPreviewGenerating(v) ? "admin-spin" : undefined} />
</button>{" "}
<button
type="button"
@@ -832,8 +914,10 @@ function VideoTitleCell({ video: v }: { video: api.AdminVideo }) {
}
function PreviewStatus({ s }: { s: string }) {
if (s === REGEN_PREVIEW_STATUS) return <span className="admin-status is-generating"></span>;
if (s === "ready") return <span className="admin-status is-ok"></span>;
if (s === "failed") return <span className="admin-status is-error"></span>;
if (s === "disabled") return <span className="admin-status"></span>;
if (s === "skipped") return <span className="admin-status"></span>;
return <span className="admin-status is-pending"></span>;
}
@@ -870,6 +954,12 @@ function formatDateTime(ms: number): string {
return `${d.getFullYear()}-${pad(d.getMonth() + 1)}-${pad(d.getDate())} ${pad(d.getHours())}:${pad(d.getMinutes())}`;
}
function videoUpdatedAtMs(video?: api.AdminVideo): number {
if (!video?.updatedAt) return 0;
const value = Date.parse(video.updatedAt);
return Number.isFinite(value) ? value : 0;
}
function useVideosPageSize() {
const [pageSize, setPageSize] = useState(() =>
window.matchMedia(VIDEOS_MOBILE_QUERY).matches ? MOBILE_VIDEOS_PAGE_SIZE : DESKTOP_VIDEOS_PAGE_SIZE
+41 -4
View File
@@ -78,13 +78,13 @@ export function checkUpdate() {
export type AdminDrive = {
id: string;
kind: "quark" | "p115" | "p123" | "pikpak" | "wopan" | "onedrive" | "googledrive" | "localstorage" | "spider91";
kind: "quark" | "p115" | "p123" | "pikpak" | "wopan" | "guangyapan" | "onedrive" | "googledrive" | "localstorage" | "spider91";
name: string;
rootId: string;
status: string;
lastError?: string;
hasCredential: boolean;
/** 当前是否给该盘生成预览视频/封面(per-drive 开关,替代旧的全局 preview.enabled)。 */
/** 当前是否给该盘生成预览视频(per-drive 开关,替代旧的全局 preview.enabled;封面不受影响)。 */
teaserEnabled: boolean;
/**
* admin "扫描跳过目录"drive fileID
@@ -98,6 +98,8 @@ export type AdminDrive = {
spider91Proxy?: string;
// Google Drive 是否使用 OpenList 在线续期 API;未配置时后端按 true 返回。
googleDriveUseOnlineAPI?: boolean;
// Google Drive OpenList 在线续期 API 地址;为空时后端使用驱动默认值。
googleDriveOpenListApiUrl?: string;
// localstorage 的 .strm 是否允许指向存储根目录之外;未配置时后端按 false 返回。
strmAllowOutsideRoot?: boolean;
scanGenerationStatus?: DriveGenerationStatus;
@@ -155,7 +157,7 @@ export function getDriveStorage() {
export type UpsertDriveInput = {
id: string;
kind: "quark" | "p115" | "p123" | "pikpak" | "wopan" | "onedrive" | "googledrive" | "localstorage" | "spider91";
kind: "quark" | "p115" | "p123" | "pikpak" | "wopan" | "guangyapan" | "onedrive" | "googledrive" | "localstorage" | "spider91";
name: string;
rootId: string;
credentials: Record<string, string>;
@@ -212,6 +214,7 @@ export type AdminCrawler = {
proxy?: string;
targetNew?: string;
uploadDriveId?: string;
teaserEnabled: boolean;
lastCrawlAt?: number;
scanGenerationStatus?: DriveGenerationStatus;
thumbnailGenerationStatus?: DriveGenerationStatus;
@@ -314,6 +317,13 @@ export function runCrawler(id: string) {
);
}
export function uploadCrawlerVideos(id: string) {
return request<{ ok: boolean; accepted: boolean; message?: string; status?: NightlyJobStatus }>(
`/crawlers/${encodeURIComponent(id)}/upload`,
{ method: "POST" }
);
}
export function stopCrawlerTasks(id: string) {
return request<{ ok: boolean; stopped: boolean }>(
`/crawlers/${encodeURIComponent(id)}/tasks/stop`,
@@ -376,6 +386,33 @@ export function getWopanQRStatus(uuid: string) {
return request<WopanQRStatus>(`/drives/wopan/qr/${encodeURIComponent(uuid)}`);
}
export type GuangYaPanQRSession = {
deviceCode: string;
qrCodeUrl: string;
qrImageDataUrl: string;
intervalSeconds: number;
expiresAt?: string;
};
export type GuangYaPanQRStatus = {
state: "pending" | "success" | "expired" | "denied" | "error";
statusText: string;
intervalSeconds?: number;
accessToken?: string;
refreshToken?: string;
tokenType?: string;
expiresIn?: number;
};
export function startGuangYaPanQRLogin() {
return request<GuangYaPanQRSession>("/drives/guangyapan/qr", { method: "POST" });
}
export function getGuangYaPanQRStatus(deviceCode: string) {
const qs = new URLSearchParams({ deviceCode });
return request<GuangYaPanQRStatus>(`/drives/guangyapan/qr/status?${qs.toString()}`);
}
/**
* toggle
*
@@ -632,7 +669,7 @@ export function deleteTag(id: number) {
// ---------- Settings ----------
export type Theme = "dark" | "pink";
export type Theme = "dark" | "pink" | "sky";
export type Settings = {
theme: Theme;
+17
View File
@@ -2,6 +2,7 @@ import { useId, useMemo, useState } from "react";
import { ArrowLeft, ChevronDown } from "lucide-react";
import { P123QRCodeLogin } from "./P123QRCodeLogin";
import { WopanQRCodeLogin } from "./WopanQRCodeLogin";
import { GuangYaPanQRCodeLogin } from "./GuangYaPanQRCodeLogin";
import { Spider91UploadTargetField } from "./Spider91UploadTargetField";
import {
FormState,
@@ -24,6 +25,7 @@ const DRIVE_OPTIONS: DriveOption[] = [
{ kind: "p115", label: "115 网盘", abbr: "115", desc: "302直链,不占带宽" },
{ kind: "p123", label: "123网盘", abbr: "123", desc: "扫码登录,302直链" },
{ kind: "pikpak", label: "PikPak", abbr: "Pk", desc: "302直链,稳定快速" },
{ kind: "guangyapan", label: "光鸭网盘", abbr: "GY", desc: "扫码登录,302直链" },
{ kind: "onedrive", label: "OneDrive", abbr: "OD", desc: "302直链,微软网盘" },
{ kind: "googledrive", label: "Google Drive", abbr: "GD", desc: "服务器中转模式" },
{ kind: "localstorage", label: "本地存储", abbr: "Lo", desc: "本机文件目录" },
@@ -194,6 +196,21 @@ export function DriveForm({
/>
)}
{form.kind === "guangyapan" && (
<GuangYaPanQRCodeLogin
onCredentials={(credentials) =>
onChange({
...form,
creds: {
...form.creds,
access_token: credentials.accessToken,
refresh_token: credentials.refreshToken,
},
})
}
/>
)}
{fields.map((f) => (
<div key={f.key} className="admin-form__row">
{f.type === "select" ? (
+150
View File
@@ -0,0 +1,150 @@
import { useEffect, useState } from "react";
import { QrCode } from "lucide-react";
import * as api from "../api";
import { useToast } from "../ToastContext";
function guangYaPanQRStatusClass(
status: api.GuangYaPanQRStatus | null,
completed: boolean,
error: string
): string {
if (completed || status?.state === "success") return "is-ok";
if (error || status?.state === "expired" || status?.state === "denied" || status?.state === "error")
return "is-error";
return "is-pending";
}
export function GuangYaPanQRCodeLogin({
onCredentials,
}: {
onCredentials: (credentials: {
accessToken: string;
refreshToken: string;
}) => void;
}) {
const { show } = useToast();
const [session, setSession] = useState<api.GuangYaPanQRSession | null>(null);
const [status, setStatus] = useState<api.GuangYaPanQRStatus | null>(null);
const [starting, setStarting] = useState(false);
const [pollingError, setPollingError] = useState("");
const [completed, setCompleted] = useState(false);
async function start() {
setStarting(true);
setPollingError("");
setCompleted(false);
setStatus(null);
try {
const next = await api.startGuangYaPanQRLogin();
setSession(next);
} catch (e) {
setSession(null);
show(e instanceof Error ? e.message : "生成二维码失败", "error");
} finally {
setStarting(false);
}
}
useEffect(() => {
if (!session || completed) return;
const activeSession = session;
let stopped = false;
let timer: number | undefined;
let delayMs = Math.max(1000, (activeSession.intervalSeconds || 5) * 1000);
async function poll() {
if (stopped) return;
try {
const next = await api.getGuangYaPanQRStatus(activeSession.deviceCode);
if (stopped) return;
setStatus(next);
setPollingError("");
if (next.intervalSeconds && next.intervalSeconds > 0) {
delayMs = Math.max(1000, next.intervalSeconds * 1000);
}
if (next.accessToken && next.refreshToken) {
stopped = true;
if (timer) window.clearTimeout(timer);
setCompleted(true);
onCredentials({
accessToken: next.accessToken,
refreshToken: next.refreshToken,
});
show("扫码成功,已填入 access_token 和 refresh_token,保存后生效", "success");
return;
}
if (next.state === "expired" || next.state === "denied" || next.state === "error") {
stopped = true;
if (timer) window.clearTimeout(timer);
return;
}
} catch (e) {
if (stopped) return;
setPollingError(e instanceof Error ? e.message : "查询扫码状态失败");
}
if (!stopped) {
timer = window.setTimeout(poll, delayMs);
}
}
poll();
return () => {
stopped = true;
if (timer) window.clearTimeout(timer);
};
}, [session, completed, onCredentials, show]);
const statusText = completed
? "已获取凭证"
: pollingError || status?.statusText || (session ? "等待扫码" : "未生成二维码");
const statusClass = guangYaPanQRStatusClass(status, completed, pollingError);
return (
<div className="admin-form__row">
<label></label>
<div className="admin-p123-qr">
<div className="admin-p123-qr__actions">
<button
type="button"
className="admin-btn"
onClick={start}
disabled={starting}
>
<QrCode size={14} />
{starting ? "生成中..." : session ? "重新生成二维码" : "生成二维码"}
</button>
<span className={`admin-status ${statusClass}`}>{statusText}</span>
</div>
{session && (
<div className="admin-p123-qr__body">
<img
className="admin-p123-qr__image"
src={session.qrImageDataUrl}
alt="光鸭网盘扫码登录二维码"
/>
<div className="admin-p123-qr__meta">
<div className="admin-form__help">
使 App access_token refresh_token
</div>
{session.expiresAt && (
<div className="admin-form__help">
{new Date(session.expiresAt).toLocaleTimeString("zh-CN", {
hour: "2-digit",
minute: "2-digit",
second: "2-digit",
})}
</div>
)}
{(status?.state === "expired" || status?.state === "denied") && (
<div className="admin-form__help">
{status.state === "denied" ? "已被拒绝" : "已过期"}
</div>
)}
</div>
</div>
)}
</div>
</div>
);
}
+44 -9
View File
@@ -1,4 +1,4 @@
export type Kind = "quark" | "p115" | "p123" | "pikpak" | "wopan" | "onedrive" | "googledrive" | "localstorage" | "spider91";
export type Kind = "quark" | "p115" | "p123" | "pikpak" | "wopan" | "guangyapan" | "onedrive" | "googledrive" | "localstorage" | "spider91";
export const kindAbbr: Record<string, string> = {
quark: "Qk",
@@ -6,6 +6,7 @@ export const kindAbbr: Record<string, string> = {
p123: "123",
pikpak: "Pk",
wopan: "Wo",
guangyapan: "GY",
onedrive: "OD",
googledrive: "GD",
localstorage: "Lo",
@@ -28,6 +29,7 @@ export const kindLabel: Record<string, string> = {
p123: "123网盘",
pikpak: "PikPak",
wopan: "联通网盘",
guangyapan: "光鸭网盘",
onedrive: "OneDrive",
googledrive: "Google Drive",
localstorage: "本地存储",
@@ -126,6 +128,7 @@ export function formatClock(value: string): string {
export function defaultRootId(kind: Kind): string {
if (kind === "pikpak") return "";
if (kind === "guangyapan") return "";
if (kind === "onedrive") return "root";
if (kind === "googledrive") return "root";
if (kind === "localstorage") return "/";
@@ -155,6 +158,8 @@ export function credentialHelp(kind: Kind, isEdit: boolean): string {
return `填写 PikPak 账号和密码即可。平台、设备 ID、验证码 token 和 refresh token 会由服务端自动处理并保存。${note}`;
case "wopan":
return `推荐使用扫码登录自动获取 access_token 和 refresh_token;也可以手工粘贴已有凭证。${note}`;
case "guangyapan":
return `推荐使用扫码登录自动获取 access_token 和 refresh_token;也可以手工粘贴已有 token。${note}`;
case "onedrive":
return `按 OpenList 默认应用在线挂载,只需要 refresh_token;保存时会自动刷新并保存 token。${note}`;
case "googledrive":
@@ -272,6 +277,29 @@ export function credentialFields(kind: Kind, creds: Record<string, string> = {})
placeholder: "留空走个人空间",
},
];
case "guangyapan":
return [
{
key: "root_path",
label: "根目录路径(可选)",
placeholder: "例如:影视/电影;留空使用上方根目录 ID",
help: "如果填写 root_path,服务端会按路径解析光鸭目录,并优先作为扫描根目录。",
},
{
key: "refresh_token",
label: "refresh_token",
placeholder: "推荐填写,服务端会自动刷新 access_token",
multiline: true,
help: "扫码成功后会自动填入该字段。",
},
{
key: "access_token",
label: "access_token",
placeholder: "Bearer eyJ... 或直接粘贴 token",
multiline: true,
help: "扫码成功后会自动填入该字段;如果 token 过期,重新扫码后保存即可。",
},
];
case "onedrive":
return [
{
@@ -295,15 +323,15 @@ export function credentialFields(kind: Kind, creds: Record<string, string> = {})
{ value: "false", label: "自建 Google OAuth 客户端" },
],
},
{
key: "refresh_token",
label: "refresh_token",
placeholder: "OpenList Google Drive refresh_token",
multiline: true,
required: true,
},
...(googleDriveUsesOnlineAPI(creds)
? []
? [
{
key: "api_url_address",
label: "OpenList 在线 API URL",
placeholder: "默认:https://api.oplist.org/googleui/renewapi",
help: "留空时使用 OpenList 官方在线 API,填写后会使用自定义续期 API。",
},
]
: [
{
key: "client_id",
@@ -320,6 +348,13 @@ export function credentialFields(kind: Kind, creds: Record<string, string> = {})
help: "Google Cloud Console 中同一个 OAuth 客户端的 Client Secret",
},
]),
{
key: "refresh_token",
label: "refresh_token",
placeholder: "OpenList Google Drive refresh_token",
multiline: true,
required: true,
},
];
case "localstorage":
return [
+1 -3
View File
@@ -3,7 +3,6 @@ import { NavLink } from "react-router-dom";
import {
Film,
Menu,
Play,
Settings,
Sparkles,
Upload,
@@ -25,9 +24,8 @@ export function MainNav() {
<div className="container main-nav__inner">
<NavLink to="/" className="main-nav__logo">
<span className="main-nav__logo-mark">
<Play size={16} fill="#000" />
<img src="/icon.png" alt="" className="main-nav__logo-img" />
</span>
<span className="main-nav__logo-text">91</span>
</NavLink>
<ul className="main-nav__list" role="menubar">
+116
View File
@@ -0,0 +1,116 @@
import type { CSSProperties } from "react";
/**
*
*
* vip.215.im GIF GIF
* CSS opacity
*
*
* - public/stickers/star-*.gif dist/stickers/
* - App
* - data-theme!=="sky" CSS display: none
* - aria-hidden + pointer-events: none访
* - / / DESKTOP_STARS / MOBILE_STARS
*/
const STICKERS = [
"/stickers/star-gold.gif",
"/stickers/star-pink.gif",
"/stickers/star-sparkle.gif",
"/stickers/star-mini.gif",
];
type StarSpec = {
/** 锚点用百分号写,CSS 直接当 top/left/right/bottom 用 */
top?: string;
bottom?: string;
left?: string;
right?: string;
/** 像素,控制 GIF 渲染尺寸 */
size: number;
};
/**
*
*
*/
const DESKTOP_STARS: StarSpec[] = [
{ top: "6%", left: "5%", size: 44 },
{ top: "4%", left: "24%", size: 26 },
{ top: "8%", right: "12%", size: 48 },
{ top: "17%", right: "31%", size: 30 },
{ top: "24%", left: "8%", size: 34 },
{ top: "28%", right: "5%", size: 38 },
{ top: "43%", left: "3%", size: 24 },
{ top: "49%", right: "9%", size: 28 },
{ top: "63%", left: "11%", size: 32 },
{ top: "66%", right: "18%", size: 44 },
{ bottom: "14%", left: "5%", size: 36 },
{ bottom: "10%", right: "6%", size: 42 },
{ bottom: "4%", left: "33%", size: 24 },
{ bottom: "6%", right: "34%", size: 28 },
{ top: "13%", left: "52%", size: 22 },
{ bottom: "24%", right: "41%", size: 22 },
];
/**
*
*/
const MOBILE_STARS: StarSpec[] = [
{ top: "7%", left: "6%", size: 30 },
{ top: "11%", right: "7%", size: 28 },
{ top: "24%", right: "3%", size: 22 },
{ top: "39%", left: "4%", size: 22 },
{ top: "57%", right: "6%", size: 26 },
{ bottom: "23%", left: "9%", size: 24 },
{ bottom: "12%", right: "12%", size: 30 },
{ bottom: "5%", left: "48%", size: 20 },
];
export function SkyStarfield() {
return (
<div className="sky-starfield" aria-hidden="true">
{DESKTOP_STARS.map((s, i) => {
const style: CSSProperties = {
top: s.top,
bottom: s.bottom,
left: s.left,
right: s.right,
width: s.size,
height: s.size,
};
const src = STICKERS[i % STICKERS.length];
return (
<img
key={`desktop-${i}`}
className="sky-star sky-star--desktop"
src={src}
alt=""
style={style}
/>
);
})}
{MOBILE_STARS.map((s, i) => {
const style: CSSProperties = {
top: s.top,
bottom: s.bottom,
left: s.left,
right: s.right,
width: s.size,
height: s.size,
};
const src = STICKERS[(i + 1) % STICKERS.length];
return (
<img
key={`mobile-${i}`}
className="sky-star sky-star--mobile"
src={src}
alt=""
style={style}
/>
);
})}
</div>
);
}
+1 -4
View File
@@ -13,10 +13,7 @@ type Props = {
const sortOptions: { key: SortKey; label: string }[] = [
{ key: "latest", label: "最新" },
{ key: "hot", label: "最热" },
{ key: "week", label: "本周" },
{ key: "long", label: "最长" },
{ key: "hd", label: "高清" },
{ key: "featured", label: "精选" },
{ key: "recent", label: "最近观看" },
];
export function SortToolbar({ sort, view, onSortChange, onViewChange }: Props) {
+1
View File
@@ -295,6 +295,7 @@ function sourceKindFromLabel(label: string): string {
if (value.includes("123") || value.includes("p123")) return "p123";
if (value.includes("pikpak")) return "pikpak";
if (value.includes("沃盘") || value.includes("wopan") || value.includes("联通")) return "wopan";
if (value.includes("光鸭") || value.includes("guangyapan") || value.includes("guangya")) return "guangyapan";
if (value.includes("onedrive") || value.includes("one drive")) return "onedrive";
if (value.includes("本地") || value.includes("localstorage") || value.includes("local storage")) return "localstorage";
return "";
+2
View File
@@ -74,6 +74,8 @@ function sourceKindFromLabel(label: string): string {
if (value.includes("pikpak")) return "pikpak";
if (value.includes("沃盘") || value.includes("wopan") || value.includes("联通"))
return "wopan";
if (value.includes("光鸭") || value.includes("guangyapan") || value.includes("guangya"))
return "guangyapan";
if (value.includes("onedrive") || value.includes("one drive")) return "onedrive";
if (value.includes("本地") || value.includes("localstorage") || value.includes("local storage"))
return "localstorage";
+4 -1
View File
@@ -92,8 +92,11 @@ const LONG_PRESS_MS = 400;
const FAST_RATE = 2;
/** 默认倍速。 */
const NORMAL_RATE = 1;
/** ArtPlayer 内部播放失败自动重连次数。 */
const ARTPLAYER_RECONNECT_TIME_MAX = 3;
Artplayer.FAST_FORWARD_VALUE = FAST_RATE;
Artplayer.RECONNECT_TIME_MAX = ARTPLAYER_RECONNECT_TIME_MAX;
const DEFAULT_SETTINGS: PlayerSettings = {
volume: 0.7,
@@ -983,7 +986,7 @@ function createHlsSourceLoader(
destroyHls(target);
onError(null);
void import("hls.js")
void import("hls.js/light")
.then((hlsModule) => {
if (art.isDestroy || !video.isConnected) return;
loadHlsSourceWith(video, url, art, hlsModule.default, onError);
+4
View File
@@ -0,0 +1,4 @@
declare module "hls.js/light" {
export { default } from "hls.js";
export * from "hls.js";
}
+3 -3
View File
@@ -10,13 +10,13 @@
// 公开端点 /api/settings/theme 不需要登录,原因见 backend/internal/api/api.go 中
// 的注释——登录页本身就要在用户登录之前正确显示主题。
export type Theme = "dark" | "pink";
export type Theme = "dark" | "pink" | "sky";
export const THEMES: Theme[] = ["dark", "pink"];
export const THEMES: Theme[] = ["dark", "pink", "sky"];
const STORAGE_KEY = "video-site:theme";
function isTheme(value: unknown): value is Theme {
return value === "dark" || value === "pink";
return value === "dark" || value === "pink" || value === "sky";
}
/**
+1 -4
View File
@@ -226,9 +226,6 @@ function isSortKey(value: unknown): value is SortKey {
return (
value === "latest" ||
value === "hot" ||
value === "week" ||
value === "long" ||
value === "hd" ||
value === "featured"
value === "recent"
);
}
+321 -81
View File
@@ -7,11 +7,8 @@ import {
Minimize,
Volume2,
VolumeX,
Play,
Pause,
EyeOff,
Info,
Loader2,
Sparkles,
AlertCircle,
} from "lucide-react";
@@ -89,16 +86,34 @@ export default function ShortsPage() {
}, 1500);
}, []);
const stopHeaderControlPropagation = useCallback((e: React.SyntheticEvent) => {
e.stopPropagation();
}, []);
const handleVolumeButtonClick = useCallback(() => {
const activeVideo = videoRefs.current.get(activeIndex);
const canResumeActiveVideo = () =>
Boolean(activeVideo) &&
videoRefs.current.get(activeIndexRef.current) === activeVideo &&
userPausedIndexRef.current !== activeIndexRef.current;
const wasPlaying = Boolean(activeVideo) && canResumeActiveVideo() && !activeVideo?.paused;
setMuted((v) => {
const next = !v;
if (activeVideo) {
normalizeVideoPlaybackRate(activeVideo);
applyVideoAudioState(activeVideo, next, volume);
stabilizeVideoAfterAudioToggle(
activeVideo,
() => wasPlaying && canResumeActiveVideo()
);
}
showHud(
next ? "已静音" : "音量已开启",
next ? <VolumeX size={16} /> : <Volume2 size={16} />
);
return next;
});
}, [showHud]);
}, [activeIndex, showHud, volume]);
const handleVolumeSliderChange = useCallback((e: React.ChangeEvent<HTMLInputElement>) => {
const val = parseFloat(e.target.value);
@@ -111,8 +126,19 @@ export default function ShortsPage() {
// Update active video volume directly
const activeVideo = videoRefs.current.get(activeIndex);
if (activeVideo) {
activeVideo.volume = val;
activeVideo.muted = val === 0;
normalizeVideoPlaybackRate(activeVideo);
applyVideoAudioState(activeVideo, val === 0, val);
const wasPlaying =
videoRefs.current.get(activeIndexRef.current) === activeVideo &&
userPausedIndexRef.current !== activeIndexRef.current &&
!activeVideo.paused;
stabilizeVideoAfterAudioToggle(
activeVideo,
() =>
wasPlaying &&
videoRefs.current.get(activeIndexRef.current) === activeVideo &&
userPausedIndexRef.current !== activeIndexRef.current
);
}
}, [activeIndex]);
@@ -139,20 +165,24 @@ export default function ShortsPage() {
// index → video element,用来精确控制播放/暂停
const videoRefs = useRef<Map<number, HTMLVideoElement>>(new Map());
const activeIndexRef = useRef(0);
const userPausedIndexRef = useRef<number | null>(null);
const ignoreIntersectionUntilRef = useRef(0);
const fullscreenRestoreTimersRef = useRef<number[]>([]);
const fullscreenPointerHandledRef = useRef(false);
const [activeReadyForPreload, setActiveReadyForPreload] = useState(false);
const [userPausedIndex, setUserPausedIndexState] = useState<number | null>(null);
const [cacheableSourceIds, setCacheableSourceIds] = useState<Set<string>>(
() => new Set()
);
const [cacheWindowHighIndex, setCacheWindowHighIndex] = useState(-1);
// 当前是否处在浏览器全屏(Fullscreen API)状态。
// iOS Safari 不支持元素级 Fullscreen API,这里会一直保持 false
// 全屏按钮在那种环境下点了也无效(按钮仍展示"进入全屏"图标)。
// iPhone Safari 不支持网页元素级全屏;那种环境下改用页面滚动让浏览器栏随刷动收起。
const useDocumentScroll = shouldUseDocumentScrollForShorts();
const [canRequestFullscreen, setCanRequestFullscreen] = useState(() =>
supportsElementFullscreenAPI()
);
const [isFullscreen, setIsFullscreen] = useState(false);
// 自动尝试进入全屏只做一次,避免反复打扰用户
const autoFullscreenAttemptedRef = useRef(false);
// 本次会话内已经点过赞的视频 id 集合。
// 与后端的真实 likes 字段同步——后端是单纯计数器,前端在这里防重避免连发。
@@ -163,6 +193,38 @@ export default function ShortsPage() {
activeIndexRef.current = activeIndex;
}, [activeIndex]);
useEffect(() => {
const page = pageRef.current;
if (page && supportsElementFullscreenAPI(page)) {
setCanRequestFullscreen(true);
}
}, []);
const updateUserPausedIndex = useCallback((index: number | null) => {
userPausedIndexRef.current = index;
setUserPausedIndexState(index);
}, []);
const setUserPausedForIndex = useCallback(
(index: number, isPaused: boolean) => {
if (isPaused) {
updateUserPausedIndex(index);
} else if (userPausedIndexRef.current === index) {
updateUserPausedIndex(null);
}
},
[updateUserPausedIndex]
);
const isVideoPausedByUser = useCallback(
(index: number) => userPausedIndexRef.current === index,
[]
);
useEffect(() => {
updateUserPausedIndex(null);
}, [activeIndex, updateUserPausedIndex]);
const handleActiveReadyForPreload = useCallback((index: number) => {
if (index === activeIndexRef.current) {
setActiveReadyForPreload(true);
@@ -294,7 +356,8 @@ export default function ShortsPage() {
}
}, [activeIndex, items, loading, roundComplete, loadMore]);
// 用 IntersectionObserver 找出当前进入视口的 item
// 用 IntersectionObserver 找出当前进入视口的 item
// root 直接用 viewport:普通模式和 iPhone 页面滚动模式都能正确观测。
useEffect(() => {
const root = containerRef.current;
if (!root) return;
@@ -321,7 +384,7 @@ export default function ShortsPage() {
}
},
{
root,
root: null,
threshold: [0.6, 0.85],
}
);
@@ -331,20 +394,28 @@ export default function ShortsPage() {
return () => observer.disconnect();
}, [items.length]);
// 控制每个 video 的播放状态与音量:只有 activeIndex 对应的在播
// 控制每个 video 的播放状态:只有 activeIndex 对应的在播
// 声音切换不要进入这里,否则移动端切换 muted 时可能额外触发 play/pause。
useEffect(() => {
videoRefs.current.forEach((video, idx) => {
if (idx === activeIndex) {
video.muted = muted;
video.volume = volume;
if (video.paused) {
if (userPausedIndex === idx) {
if (!video.paused) video.pause();
} else if (video.paused) {
video.play().catch(() => undefined);
}
} else {
if (!video.paused) video.pause();
}
});
}, [activeIndex, muted, volume, items.length]);
}, [activeIndex, items.length, userPausedIndex]);
// 单独同步音频属性。这里不做 play/pause,避免手机端切换静音时打断播放节奏。
useEffect(() => {
videoRefs.current.forEach((video) => {
applyVideoAudioState(video, muted, volume);
});
}, [muted, volume, items.length]);
// 键盘快捷键监听
useEffect(() => {
@@ -376,12 +447,15 @@ export default function ShortsPage() {
e.preventDefault();
const activeVideo = videoRefs.current.get(activeIndex);
if (activeVideo) {
if (activeVideo.paused) {
const shouldResume =
userPausedIndexRef.current === activeIndex ||
(activeVideo.paused && activeVideo.readyState >= 3);
if (shouldResume) {
setUserPausedForIndex(activeIndex, false);
activeVideo.play().catch(() => undefined);
showHud("播放", <Play size={16} fill="currentColor" />);
} else {
setUserPausedForIndex(activeIndex, true);
activeVideo.pause();
showHud("暂停", <Pause size={16} fill="currentColor" />);
}
}
} else if (e.key === "m" || e.key === "M") {
@@ -417,7 +491,7 @@ export default function ShortsPage() {
window.addEventListener("keydown", handleKeyDown);
return () => window.removeEventListener("keydown", handleKeyDown);
}, [activeIndex, items, toggleFullscreen, showHud, handleVolumeButtonClick]);
}, [activeIndex, items, toggleFullscreen, showHud, handleVolumeButtonClick, setUserPausedForIndex]);
// 页面卸载时暂停所有
useEffect(() => {
@@ -444,16 +518,21 @@ export default function ShortsPage() {
document.title = "短视频 · 91";
}, []);
// 沉浸式:进入页面后锁住 body 滚动 + 把主题色改黑(Android Chrome 状态栏会变黑)
// 沉浸式:默认锁住 body 滚动;iPhone 浏览器里放开根页面滚动,让 Safari 工具栏能随刷动收起。
useEffect(() => {
const html = document.documentElement;
const body = document.body;
const prevHtmlOverflow = html.style.overflow;
const prevBodyOverflow = body.style.overflow;
const prevBodyBg = body.style.background;
html.style.overflow = "hidden";
body.style.overflow = "hidden";
body.style.background = "#000";
if (useDocumentScroll) {
html.classList.add("shorts-document-scroll");
body.classList.add("shorts-document-scroll");
} else {
html.style.overflow = "hidden";
body.style.overflow = "hidden";
body.style.background = "#000";
}
let prevThemeColor: string | null = null;
let themeMeta = document.querySelector<HTMLMetaElement>(
@@ -470,6 +549,8 @@ export default function ShortsPage() {
themeMeta.content = "#000000";
return () => {
html.classList.remove("shorts-document-scroll");
body.classList.remove("shorts-document-scroll");
html.style.overflow = prevHtmlOverflow;
body.style.overflow = prevBodyOverflow;
body.style.background = prevBodyBg;
@@ -481,7 +562,7 @@ export default function ShortsPage() {
}
}
};
}, []);
}, [useDocumentScroll]);
function clearFullscreenRestoreTimers() {
for (const timer of fullscreenRestoreTimersRef.current) {
@@ -543,29 +624,8 @@ export default function ShortsPage() {
};
}, []);
// 进入页面后第一次任意触摸时尝试自动进入全屏。
// 浏览器要求 requestFullscreen 必须在用户手势内调用;进页面时直接调
// 一定会被拒绝,所以挂在 pointerdown 上利用第一次手势。
// iOS Safari 不支持元素级 Fullscreen API,这里 catch 后保持原样,
// 退化为已经做的 100svh 沉浸样式。
useEffect(() => {
const page = pageRef.current;
if (!page) return;
function onFirstPointer() {
if (autoFullscreenAttemptedRef.current) return;
autoFullscreenAttemptedRef.current = true;
requestPageFullscreen();
}
page.addEventListener("pointerdown", onFirstPointer, {
once: true,
passive: true,
});
return () => {
page.removeEventListener("pointerdown", onFirstPointer);
};
}, []);
function requestPageFullscreen() {
if (!canRequestFullscreen) return;
const page = pageRef.current;
if (!page) return;
// eslint-disable-next-line @typescript-eslint/no-explicit-any
@@ -604,8 +664,33 @@ export default function ShortsPage() {
function toggleFullscreen() {
scheduleFullscreenActiveRestore();
if (isFullscreen) exitPageFullscreen();
else requestPageFullscreen();
if (canRequestFullscreen) {
if (isFullscreen) exitPageFullscreen();
else requestPageFullscreen();
return;
}
if (useDocumentScroll) {
restoreActiveSlideIntoView();
}
}
function handleFullscreenButtonPointerDown(
e: React.PointerEvent<HTMLButtonElement>
) {
e.preventDefault();
e.stopPropagation();
fullscreenPointerHandledRef.current = true;
toggleFullscreen();
}
function handleFullscreenButtonClick(e: React.MouseEvent<HTMLButtonElement>) {
e.preventDefault();
e.stopPropagation();
if (fullscreenPointerHandledRef.current) {
fullscreenPointerHandledRef.current = false;
return;
}
toggleFullscreen();
}
const handleHideSuccess = useCallback((idx: number) => {
@@ -624,7 +709,10 @@ export default function ShortsPage() {
const videoWindow = getVideoWindowBounds(cacheWindowHighIndex, items.length);
return (
<div className="shorts-page" ref={pageRef}>
<div
className={`shorts-page${useDocumentScroll ? " is-document-scroll" : ""}`}
ref={pageRef}
>
<header className="shorts-header">
<Link to="/" className="shorts-header__back" aria-label="返回首页">
<ChevronLeft size={22} />
@@ -634,7 +722,8 @@ export default function ShortsPage() {
type="button"
className="shorts-header__icon-btn"
aria-label={isFullscreen ? "退出全屏" : "进入全屏"}
onClick={toggleFullscreen}
onPointerDown={handleFullscreenButtonPointerDown}
onClick={handleFullscreenButtonClick}
>
{isFullscreen ? <Minimize size={20} /> : <Maximize size={20} />}
</button>
@@ -656,7 +745,16 @@ export default function ShortsPage() {
type="button"
className="shorts-header__icon-btn"
aria-label={muted ? "取消静音" : "静音"}
onClick={handleVolumeButtonClick}
onPointerDownCapture={stopHeaderControlPropagation}
onTouchStartCapture={stopHeaderControlPropagation}
onMouseDownCapture={stopHeaderControlPropagation}
onPointerDown={stopHeaderControlPropagation}
onTouchStart={stopHeaderControlPropagation}
onMouseDown={stopHeaderControlPropagation}
onClick={(e) => {
e.stopPropagation();
handleVolumeButtonClick();
}}
>
{muted || volume === 0 ? <VolumeX size={20} /> : <Volume2 size={20} />}
</button>
@@ -720,17 +818,12 @@ export default function ShortsPage() {
onActiveReadyForPreload={handleActiveReadyForPreload}
onActiveNeedsPriority={handleActiveNeedsPriority}
onSourceCached={handleSourceCached}
onUserPausedChange={setUserPausedForIndex}
isVideoPausedByUser={isVideoPausedByUser}
showHud={showHud}
/>
);
})}
{!empty && items.length > 0 && loading && (
<div className="shorts-loading">
<Loader2 size={16} className="shorts-slide__buffering-icon" />
<span></span>
</div>
)}
</div>
</div>
);
@@ -760,6 +853,8 @@ type SlideProps = {
onActiveNeedsPriority: (index: number) => void;
/** 本条视频在浏览器里已有可复用缓冲,之后在视频窗口内保留 src */
onSourceCached: (videoId: string) => void;
onUserPausedChange: (index: number, isPaused: boolean) => void;
isVideoPausedByUser: (index: number) => boolean;
showHud: (text: string, icon?: React.ReactNode) => void;
};
@@ -788,6 +883,8 @@ function ShortsSlide({
onActiveReadyForPreload,
onActiveNeedsPriority,
onSourceCached,
onUserPausedChange,
isVideoPausedByUser,
showHud,
}: SlideProps) {
const localRef = useRef<HTMLVideoElement | null>(null);
@@ -796,8 +893,6 @@ function ShortsSlide({
// 视频缓冲状态
const [isBuffering, setIsBuffering] = useState(false);
// 单击播放暂停的瞬间 HUD 动效
const [playPauseHud, setPlayPauseHud] = useState<{ id: number; type: "play" | "pause" } | null>(null);
// 是否已经被隐藏/拉黑
const [isMarkedHidden, setIsMarkedHidden] = useState(false);
@@ -865,7 +960,6 @@ function ShortsSlide({
setScrubbing(false);
scrubbingRef.current = false;
setIsBuffering(false);
setPlayPauseHud(null);
}
}, [isActive]);
@@ -873,8 +967,7 @@ function ShortsSlide({
useEffect(() => {
const video = localRef.current;
if (video && isActive) {
video.muted = muted;
video.volume = volume;
applyVideoAudioState(video, muted, volume);
}
}, [muted, volume, isActive]);
@@ -916,13 +1009,23 @@ function ShortsSlide({
syncActivePreloadReadiness(video);
};
const handleWaiting = () => {
if (video.paused || isVideoPausedByUser(index)) {
setIsBuffering(false);
return;
}
setIsBuffering(true);
if (isActive) onActiveNeedsPriority(index);
};
const handlePlayingOrCanPlay = () => {
setIsBuffering(false);
// 已经能解码播放,说明浏览器里有了值得复用的数据。
if (shouldLoad) onSourceCached(item.id);
if (isActive && isVideoPausedByUser(index)) {
video.pause();
setPaused(true);
setIsBuffering(false);
return;
}
setIsBuffering(false);
syncActivePreloadReadiness(video);
};
const handleProgress = () => {
@@ -944,6 +1047,21 @@ function ShortsSlide({
setVolume(video.volume);
}
};
const handlePlay = () => {
if (!isActive) return;
if (isVideoPausedByUser(index)) {
video.pause();
setPaused(true);
setIsBuffering(false);
return;
}
setPaused(false);
};
const handlePause = () => {
if (!isActive || video.ended) return;
setPaused(true);
setIsBuffering(false);
};
function syncActivePreloadReadiness(currentVideo: HTMLVideoElement) {
if (!isActive) return;
@@ -966,6 +1084,8 @@ function ShortsSlide({
video.addEventListener("canplay", handlePlayingOrCanPlay);
video.addEventListener("progress", handleProgress);
video.addEventListener("volumechange", handleVolumeChange);
video.addEventListener("play", handlePlay);
video.addEventListener("pause", handlePause);
// 挂载时如果已经在播放但是状态不到 ready 则置 buffering
if (video.readyState < 3 && !video.paused) {
@@ -981,8 +1101,10 @@ function ShortsSlide({
video.removeEventListener("canplay", handlePlayingOrCanPlay);
video.removeEventListener("progress", handleProgress);
video.removeEventListener("volumechange", handleVolumeChange);
video.removeEventListener("play", handlePlay);
video.removeEventListener("pause", handlePause);
};
}, [shouldMount, shouldLoad, item.id, index, isActive, muted, volume, setMuted, setVolume, onActiveReadyForPreload, onActiveNeedsPriority, onSourceCached]);
}, [shouldMount, shouldLoad, item.id, index, isActive, muted, volume, setMuted, setVolume, onActiveReadyForPreload, onActiveNeedsPriority, onSourceCached, isVideoPausedByUser]);
// 长按 2 倍速:直接绑原生事件
useEffect(() => {
@@ -1047,16 +1169,18 @@ function ShortsSlide({
function togglePlayInternal() {
const video = localRef.current;
if (!video) return;
if (video.paused) {
const shouldResume =
isVideoPausedByUser(index) || (video.paused && paused && !isBuffering);
if (shouldResume) {
onUserPausedChange(index, false);
video.play().catch(() => undefined);
setPaused(false);
setPlayPauseHud({ id: Date.now(), type: "play" });
setTimeout(() => setPlayPauseHud(null), 450);
if (video.readyState < 3) setIsBuffering(true);
} else {
onUserPausedChange(index, true);
video.pause();
setPaused(true);
setPlayPauseHud({ id: Date.now(), type: "pause" });
setTimeout(() => setPlayPauseHud(null), 450);
setIsBuffering(false);
}
}
@@ -1311,7 +1435,7 @@ function ShortsSlide({
{paused && isActive && !scrubbing && !playPauseHud && (
{paused && isActive && !scrubbing && (
<div className="shorts-slide__paused" aria-hidden="true">
</div>
@@ -1320,14 +1444,7 @@ function ShortsSlide({
{/* 视频加载/缓冲旋转器 */}
{isBuffering && isActive && shouldLoad && !isMarkedHidden && (
<div className="shorts-slide__buffering" aria-hidden="true">
<Loader2 size={30} className="shorts-slide__buffering-icon" />
</div>
)}
{/* 播放暂停瞬间 HUD 动效 */}
{playPauseHud && isActive && (
<div key={playPauseHud.id} className="shorts-slide__hud-pulse" aria-hidden="true">
{playPauseHud.type === "play" ? <Play size={30} fill="currentColor" /> : <Pause size={30} fill="currentColor" />}
<ShortsLoadingSpinner size={30} />
</div>
)}
@@ -1451,6 +1568,128 @@ function ShortsSlide({
);
}
function ShortsLoadingSpinner({ size }: { size: number }) {
const ref = useRef<HTMLSpanElement | null>(null);
useEffect(() => {
let frame = 0;
const startedAt = performance.now();
const tick = (now: number) => {
const spinner = ref.current;
if (spinner) {
const rotation = ((now - startedAt) / 800) * 360;
spinner.style.transform = `rotate(${rotation}deg)`;
}
frame = window.requestAnimationFrame(tick);
};
frame = window.requestAnimationFrame(tick);
return () => window.cancelAnimationFrame(frame);
}, []);
return (
<span
ref={ref}
className="shorts-slide__loading-spinner"
style={{
"--shorts-spinner-size": `${size}px`,
} as React.CSSProperties}
aria-hidden="true"
/>
);
}
function applyVideoAudioState(
video: HTMLVideoElement,
nextMuted: boolean,
nextVolume: number
) {
const safeVolume = clamp(nextVolume, 0, 1);
const syncVolume = () => {
try {
if (Math.abs(video.volume - safeVolume) > 0.001) {
video.volume = safeVolume;
}
} catch {
// Some mobile browsers expose volume as effectively read-only.
}
};
if (!nextMuted) syncVolume();
try {
if (video.muted !== nextMuted) {
video.muted = nextMuted;
}
} catch {
// ignore
}
if (nextMuted) syncVolume();
}
function normalizeVideoPlaybackRate(video: HTMLVideoElement) {
try {
if (video.defaultPlaybackRate !== 1) {
video.defaultPlaybackRate = 1;
}
if (video.playbackRate !== 1) {
video.playbackRate = 1;
}
} catch {
// ignore
}
}
function stabilizeVideoAfterAudioToggle(
video: HTMLVideoElement,
shouldResume: () => boolean
) {
const stabilize = () => {
normalizeVideoPlaybackRate(video);
if (shouldResume() && video.paused && !video.ended) {
video.play().catch(() => undefined);
}
};
stabilize();
for (const delay of [80, 240, 600]) {
window.setTimeout(stabilize, delay);
}
}
function shouldUseDocumentScrollForShorts() {
return isIPhoneBrowserShell();
}
function isIPhoneBrowserShell() {
if (typeof window === "undefined" || typeof navigator === "undefined") {
return false;
}
const ua = navigator.userAgent || "";
return /\biPhone\b|\biPod\b/.test(ua) && !isStandaloneDisplayMode();
}
function isStandaloneDisplayMode() {
if (typeof window === "undefined" || typeof navigator === "undefined") {
return false;
}
const nav = navigator as Navigator & { standalone?: boolean };
return (
nav.standalone === true ||
window.matchMedia?.("(display-mode: standalone)").matches === true ||
window.matchMedia?.("(display-mode: fullscreen)").matches === true
);
}
function supportsElementFullscreenAPI(target?: Element | null) {
if (typeof document === "undefined") return false;
const el = (target ?? document.documentElement) as HTMLElement & {
webkitRequestFullscreen?: () => Promise<void> | void;
};
return (
typeof el.requestFullscreen === "function" ||
typeof el.webkitRequestFullscreen === "function"
);
}
function clamp(n: number, min: number, max: number) {
return n < min ? min : n > max ? max : n;
}
@@ -1532,6 +1771,7 @@ function getDriveShortName(source: string): string {
if (s.includes("quark") || s.includes("夸克")) return "Quak";
if (s.includes("onedrive")) return "OneDrive";
if (s.includes("wopan") || s.includes("沃盘")) return "沃盘";
if (s.includes("guangyapan") || s.includes("guangya") || s.includes("光鸭")) return "光鸭";
if (s.includes("localstorage") || s.includes("本地")) return "本地";
if (s.includes("spider") || s.includes("爬虫")) return "爬虫";
return source.substring(0, 4);
+104 -96
View File
@@ -32,8 +32,9 @@
.admin-sidebar__brand {
display: flex;
align-items: center;
gap: 12px;
padding: 0 12px var(--space-6);
justify-content: center;
gap: 0;
padding: 0 12px var(--space-5);
font-size: var(--font-xl);
font-weight: var(--weight-bold);
color: var(--text-strong);
@@ -52,22 +53,29 @@
.admin-sidebar__brand-mark {
display: grid;
place-items: center;
width: 38px;
height: 38px;
width: 40px;
height: 40px;
border-radius: var(--radius-sm);
background: var(--accent-gradient);
color: var(--text-on-accent);
box-shadow: 0 4px 14px var(--accent-glow), var(--shadow-inset);
background: transparent;
box-shadow: 0 4px 14px rgba(0, 0, 0, 0.16);
overflow: hidden;
animation: admin-brand-pulse 3s infinite ease-in-out;
}
.admin-sidebar__brand-img {
width: 100%;
height: 100%;
object-fit: cover;
border-radius: inherit;
}
@keyframes admin-brand-pulse {
0%, 100% {
box-shadow: 0 4px 12px var(--accent-glow), var(--shadow-inset);
box-shadow: 0 4px 12px rgba(0, 0, 0, 0.14);
transform: scale(1);
}
50% {
box-shadow: 0 6px 20px rgba(255, 138, 60, 0.45), var(--shadow-inset);
box-shadow: 0 6px 20px var(--accent-glow);
transform: scale(1.04);
}
}
@@ -76,8 +84,10 @@
.admin-nav {
display: flex;
flex-direction: column;
gap: 22px;
padding: var(--space-5) 0;
flex: 1;
justify-content: space-evenly;
gap: var(--space-4);
padding: var(--space-2) 0 var(--space-4);
}
.admin-nav__group {
@@ -616,7 +626,7 @@
width: 100%;
min-width: 0;
display: grid;
grid-template-columns: 38px minmax(160px, 1.1fr) minmax(0, 1.4fr) auto 18px;
grid-template-columns: 38px minmax(160px, 1fr) 18px;
align-items: center;
gap: var(--space-3);
padding: 0;
@@ -666,77 +676,6 @@
font-size: var(--font-xs);
}
.admin-crawler-pipeline {
display: flex;
flex-wrap: wrap;
align-items: center;
gap: 6px;
min-width: 0;
}
.admin-crawler-stage {
display: inline-flex;
align-items: center;
gap: 6px;
min-height: 24px;
padding: 3px 9px;
border-radius: var(--radius-pill);
border: 1px solid var(--border-subtle);
background: var(--bg-sunken);
color: var(--text-muted);
font-size: var(--font-xs);
font-weight: var(--weight-medium);
white-space: nowrap;
}
.admin-crawler-stage em {
font-style: normal;
color: inherit;
}
.admin-crawler-stage__dot {
width: 7px;
height: 7px;
border-radius: 50%;
background: var(--text-faint);
flex: 0 0 auto;
}
.admin-crawler-stage.is-generating {
color: var(--info);
border-color: transparent;
background: var(--info-soft);
}
.admin-crawler-stage.is-generating .admin-crawler-stage__dot {
background: var(--info);
animation: admin-crawler-stage-pulse 1.2s ease-in-out infinite;
}
.admin-crawler-stage.is-cooling {
color: var(--warning);
border-color: transparent;
background: var(--warning-soft);
}
.admin-crawler-stage.is-cooling .admin-crawler-stage__dot {
background: var(--warning);
}
.admin-crawler-stage.is-queued {
color: var(--text-muted);
background: rgba(255, 255, 255, 0.06);
}
.admin-crawler-stage.is-queued .admin-crawler-stage__dot {
background: var(--text-muted);
}
@keyframes admin-crawler-stage-pulse {
0%, 100% { opacity: 1; transform: scale(1); }
50% { opacity: 0.4; transform: scale(0.75); }
}
.admin-crawler-row__chevron {
color: var(--text-faint);
transition: transform var(--transition-fast), color var(--transition-fast);
@@ -758,6 +697,11 @@
gap: var(--space-2);
}
.admin-crawler-preview-card-toggle {
min-width: 96px;
padding-inline: 10px;
}
.admin-crawler-row__delete {
padding-inline: 10px;
}
@@ -1185,13 +1129,7 @@
@media (max-width: 1180px) {
.admin-crawler-row__main {
grid-template-columns: 38px minmax(0, 1fr) auto 18px;
row-gap: var(--space-2);
}
.admin-crawler-pipeline {
grid-column: 2 / 4;
grid-row: 2;
grid-template-columns: 38px minmax(0, 1fr) 18px;
}
}
@@ -1869,6 +1807,7 @@
.admin-status.is-ok { background: var(--success-soft); color: var(--success); }
.admin-status.is-error { background: var(--danger-soft); color: var(--danger); }
.admin-status.is-pending { background: var(--warning-soft); color: var(--warning); }
.admin-status.is-generating { background: var(--info-soft); color: var(--info); }
.admin-generation-state.is-generating { background: var(--info-soft); color: var(--info); }
.admin-generation-state.is-cooling { background: var(--warning-soft); color: var(--warning); }
@@ -2164,18 +2103,32 @@
}
.admin-toast {
padding: 12px 18px;
max-width: min(520px, calc(100vw - 48px));
padding: 14px 18px;
background: var(--bg-elevated);
color: var(--text-strong);
border: 1px solid var(--border-default);
border-radius: var(--radius-sm);
position: relative;
font-size: var(--font-sm);
font-weight: var(--weight-medium);
line-height: 1.5;
overflow-wrap: anywhere;
touch-action: manipulation;
box-shadow: var(--shadow-lg);
animation: toast-in var(--duration-normal) var(--ease-out);
pointer-events: auto;
}
.admin-toast.is-copyable {
cursor: pointer;
}
.admin-toast__text {
display: block;
min-width: 0;
}
@keyframes toast-in {
from { opacity: 0; transform: translateY(10px); }
to { opacity: 1; transform: none; }
@@ -2853,7 +2806,8 @@
}
.admin-toast {
text-align: center;
max-width: 100%;
text-align: left;
}
}
@@ -3654,6 +3608,7 @@
.admin-drive-type-card[data-kind="spider91"]:hover { border-color: var(--accent); box-shadow: 0 4px 18px var(--accent-glow); }
.admin-drive-type-card[data-kind="quark"]:hover { border-color: var(--drive-quark); box-shadow: 0 4px 18px rgba(91,141,239,.2); }
.admin-drive-type-card[data-kind="wopan"]:hover { border-color: var(--drive-wopan); box-shadow: 0 4px 18px rgba(255,138,60,.2); }
.admin-drive-type-card[data-kind="guangyapan"]:hover { border-color: var(--drive-guangyapan); box-shadow: 0 4px 18px rgba(48,195,168,.2); }
.admin-drive-type-card__icon {
display: grid;
@@ -3679,6 +3634,7 @@
.admin-drive-type-card__icon[data-kind="spider91"] { background: var(--accent-soft); color: var(--accent); }
.admin-drive-type-card__icon[data-kind="quark"] { background: rgba(91,141,239,.14); color: var(--drive-quark); }
.admin-drive-type-card__icon[data-kind="wopan"] { background: rgba(255,138,60,.14); color: var(--drive-wopan); }
.admin-drive-type-card__icon[data-kind="guangyapan"] { background: rgba(48,195,168,.14); color: var(--drive-guangyapan); }
.admin-drive-type-card__label {
font-size: var(--font-sm);
@@ -3737,6 +3693,7 @@
.admin-drive-selected-bar__icon[data-kind="spider91"] { background: var(--accent-soft); color: var(--accent); }
.admin-drive-selected-bar__icon[data-kind="quark"] { background: rgba(91,141,239,.14); color: var(--drive-quark); }
.admin-drive-selected-bar__icon[data-kind="wopan"] { background: rgba(255,138,60,.14); color: var(--drive-wopan); }
.admin-drive-selected-bar__icon[data-kind="guangyapan"] { background: rgba(48,195,168,.14); color: var(--drive-guangyapan); }
.admin-drive-selected-bar__text {
flex: 1;
@@ -3891,12 +3848,16 @@
.theme-card[data-preview="dark"] .theme-card__preview {
background:
radial-gradient(80% 60% at 50% 0%, rgba(255, 138, 60, 0.18), transparent 70%),
linear-gradient(to right, rgba(255, 138, 60, 0.14) 1px, transparent 1px) 0 0 / 18px 18px,
linear-gradient(to bottom, rgba(255, 138, 60, 0.14) 1px, transparent 1px) 0 0 / 18px 18px,
linear-gradient(180deg, #14161c 0%, #0b0c10 100%);
}
.theme-card[data-preview="pink"] .theme-card__preview {
background:
radial-gradient(80% 60% at 50% 0%, rgba(255, 91, 138, 0.16), transparent 70%),
linear-gradient(to right, rgba(255, 91, 138, 0.18) 1px, transparent 1px) 0 0 / 18px 18px,
linear-gradient(to bottom, rgba(255, 91, 138, 0.18) 1px, transparent 1px) 0 0 / 18px 18px,
linear-gradient(180deg, #ffffff 0%, #fff5f7 100%);
}
@@ -4017,6 +3978,43 @@
border-color: rgba(255, 91, 138, 0.6);
}
/* ----- 星空蓝预览:浅蓝渐变 + 隐约网格 + 几颗黄星点缀 ----- */
.theme-card[data-preview="sky"] .theme-card__preview {
background:
radial-gradient(80% 60% at 50% 0%, rgba(255, 200, 61, 0.22), transparent 70%),
linear-gradient(to right, rgba(255, 255, 255, 0.5) 1px, transparent 1px) 0 0 / 18px 18px,
linear-gradient(to bottom, rgba(255, 255, 255, 0.5) 1px, transparent 1px) 0 0 / 18px 18px,
linear-gradient(180deg, #d6ecff 0%, #b8dcff 100%);
}
.theme-card[data-preview="sky"] .theme-card__bar {
background: linear-gradient(135deg, #ffe28a 0%, #ffc83d 100%);
box-shadow: 0 0 12px rgba(255, 200, 61, 0.5);
}
.theme-card[data-preview="sky"] .theme-card__player {
box-shadow: 0 0 0 1px rgba(255, 200, 61, 0.5),
0 12px 28px rgba(40, 80, 160, 0.22);
}
.theme-card[data-preview="sky"] .theme-card__line {
background: rgba(40, 70, 140, 0.22);
}
.theme-card[data-preview="sky"] .theme-card__line--lg {
background: rgba(27, 37, 71, 0.55);
}
.theme-card[data-preview="sky"] .theme-card__chip {
background: rgba(40, 70, 140, 0.08);
border: 1px solid rgba(40, 70, 140, 0.22);
}
.theme-card[data-preview="sky"] .theme-card__chip--accent {
background: rgba(255, 200, 61, 0.28);
border-color: rgba(255, 200, 61, 0.7);
}
/* 卡片底部信息区 */
.theme-card__body {
display: flex;
@@ -4280,6 +4278,7 @@
.admin-drive-card__brand-icon[data-kind="p123"] { background: var(--drive-p123); }
.admin-drive-card__brand-icon[data-kind="pikpak"] { background: var(--drive-pikpak); }
.admin-drive-card__brand-icon[data-kind="wopan"] { background: var(--drive-wopan); }
.admin-drive-card__brand-icon[data-kind="guangyapan"] { background: var(--drive-guangyapan); }
.admin-drive-card__brand-icon[data-kind="onedrive"] { background: var(--drive-onedrive); }
.admin-drive-card__brand-icon[data-kind="googledrive"] { background: #4285f4; }
.admin-drive-card__brand-icon[data-kind="localstorage"] { background: var(--drive-localstorage); }
@@ -4878,14 +4877,23 @@
}
.admin-tag-card__alias-pill {
font-size: 10px;
padding: 1px 6px;
font-size: var(--font-xs);
font-weight: var(--weight-semibold);
line-height: 1.3;
padding: 2px 7px;
background: var(--bg-sunken);
color: var(--text-muted);
border: 1px solid var(--border-subtle);
color: var(--text-default);
border: 1px solid var(--border-default);
border-radius: var(--radius-pill);
}
:root[data-theme="sky"] .admin-tag-card__alias-pill {
background: rgba(47, 111, 214, 0.13);
border-color: rgba(47, 111, 214, 0.2);
color: #2f436f;
box-shadow: inset 0 1px 0 rgba(255, 255, 255, 0.5);
}
.admin-tag-card__footer {
display: flex;
align-items: center;
+11 -15
View File
@@ -49,33 +49,29 @@
}
.main-nav__logo:hover .main-nav__logo-mark {
transform: rotate(6deg) scale(1.05);
box-shadow: 0 6px 20px rgba(255, 138, 60, 0.45), var(--shadow-inset);
transform: rotate(3deg) scale(1.04);
box-shadow: 0 6px 20px var(--accent-glow);
}
.main-nav__logo-mark {
display: grid;
place-items: center;
width: 32px;
height: 32px;
width: 34px;
height: 34px;
border-radius: var(--radius-sm);
background: var(--accent-gradient);
color: var(--text-on-accent);
box-shadow:
0 4px 14px var(--accent-glow),
var(--shadow-inset);
background: transparent;
box-shadow: 0 4px 14px rgba(0, 0, 0, 0.16);
position: relative;
isolation: isolate;
overflow: hidden;
transition: transform var(--transition-fast), box-shadow var(--transition-fast);
}
.main-nav__logo-mark::after {
content: "";
position: absolute;
inset: 0;
background: linear-gradient(180deg, rgba(255, 255, 255, 0.18), transparent 60%);
pointer-events: none;
.main-nav__logo-img {
width: 100%;
height: 100%;
object-fit: cover;
border-radius: inherit;
}
/* ----- 链接列表 ----- */
+2
View File
@@ -173,6 +173,7 @@
gap: var(--space-2);
padding: var(--space-3) var(--space-2);
margin-top: var(--space-3);
margin-bottom: var(--space-4);
background: var(--bg-surface);
border: 1px solid var(--border-subtle);
border-radius: var(--radius-md);
@@ -285,6 +286,7 @@
.sort-toolbar {
padding: var(--space-2);
gap: var(--space-2);
margin-bottom: var(--space-3);
}
.sort-toolbar__group {
+81 -61
View File
@@ -2,8 +2,14 @@
滚动靠原生 scroll-snap 实现 TikTok 式吸附切屏 */
.shorts-page {
position: fixed;
inset: 0;
/* 不能用 fixed 定位iOS Safari/WebKit 不会合成渲染嵌套在 fixed
祖先里的内联 <video>视频能解码播放但画面始终不显示黑屏
桌面 Chrome 正常所以只在 iOS 上表现为黑屏
这里改用普通流里的满屏块高度锁视口沉浸态的滚动锁定已由组件在
mount 时给 html/body 设置 overflow:hidden 实现效果等价 */
position: relative;
width: 100%;
height: 100svh;
background: #000;
color: #fff;
z-index: 50;
@@ -18,6 +24,59 @@
overscroll-behavior: none;
}
html.shorts-document-scroll {
overflow-y: auto;
scroll-snap-type: y mandatory;
scrollbar-gutter: auto;
overscroll-behavior-y: none;
background: #000;
}
body.shorts-document-scroll {
min-height: 100%;
overflow-y: visible;
scroll-snap-type: y mandatory;
background: #000;
}
body.shorts-document-scroll::before {
display: none;
}
.shorts-page.is-document-scroll {
min-height: 100dvh;
height: auto;
display: block;
overflow: visible;
}
.shorts-page.is-document-scroll .shorts-feed {
min-height: 100dvh;
overflow-y: visible;
scroll-snap-type: none;
}
.shorts-page.is-document-scroll .shorts-header,
.shorts-page.is-document-scroll .shorts-hud-toast {
position: fixed;
}
.shorts-page.is-document-scroll .shorts-slide {
height: 100dvh;
min-height: 100dvh;
}
@supports not (height: 100svh) {
.shorts-page {
height: 100dvh;
}
}
@supports not (height: 100dvh) {
.shorts-page {
height: 100vh;
}
}
/* ---------- 顶部条 (高阶毛玻璃渐变遮罩) ---------- */
.shorts-header {
position: absolute;
@@ -214,48 +273,6 @@
filter: drop-shadow(0 20px 50px rgba(0, 0, 0, 0.95));
}
/* 播放状态脉冲 HUD (磨砂玻璃大图标弹出动效) */
.shorts-slide__hud-pulse {
position: absolute;
left: 50%;
top: 50%;
transform: translate(-50%, -50%);
width: 80px;
height: 80px;
border-radius: 50%;
background: rgba(0, 0, 0, 0.45);
border: 1.5px solid rgba(255, 255, 255, 0.25);
backdrop-filter: blur(10px);
-webkit-backdrop-filter: blur(10px);
display: flex;
align-items: center;
justify-content: center;
color: #fff;
z-index: 15;
pointer-events: none;
box-shadow: 0 8px 32px rgba(0, 0, 0, 0.4);
animation: shorts-hud-pop 450ms cubic-bezier(0.34, 1.56, 0.64, 1) forwards;
}
@keyframes shorts-hud-pop {
0% {
opacity: 0;
transform: translate(-50%, -50%) scale(0.6);
}
25% {
opacity: 1;
transform: translate(-50%, -50%) scale(1.1);
}
75% {
opacity: 0.85;
transform: translate(-50%, -50%) scale(1);
}
100% {
opacity: 0;
transform: translate(-50%, -50%) scale(0.9);
}
}
/* 视频加载/卡顿缓冲磨砂提示圈 */
.shorts-slide__buffering {
position: absolute;
@@ -278,13 +295,27 @@
box-shadow: 0 4px 20px rgba(0, 0, 0, 0.5);
}
.shorts-slide__buffering-icon {
animation: shorts-spin 1.1s linear infinite;
.shorts-slide__loading-spinner {
display: block;
flex: 0 0 auto;
width: var(--shorts-spinner-size, 30px);
height: var(--shorts-spinner-size, 30px);
border: 3px solid rgba(255, 255, 255, 0.24);
border-top-color: rgba(255, 255, 255, 0.98);
border-radius: 50%;
will-change: transform;
}
@keyframes shorts-spin {
0% { transform: rotate(0deg); }
100% { transform: rotate(360deg); }
@media (max-width: 640px) {
.shorts-slide__buffering {
--shorts-spinner-size: 24px;
width: 56px;
height: 56px;
}
.shorts-slide__buffering .shorts-slide__loading-spinner {
border-width: 2px;
}
}
/* 原始大三角形的遗留样式(做兼容处理) */
@@ -765,7 +796,7 @@
font-weight: 600;
display: flex;
align-items: center;
gap: 8px;
gap: 4px;
box-shadow: 0 8px 24px rgba(0, 0, 0, 0.4);
backdrop-filter: blur(10px);
-webkit-backdrop-filter: blur(10px);
@@ -810,14 +841,3 @@
.shorts-empty__link:hover {
background: rgba(255, 255, 255, 0.25);
}
.shorts-loading {
height: 70px;
display: flex;
align-items: center;
justify-content: center;
gap: 8px;
color: rgba(255, 255, 255, 0.6);
font-size: 13px;
font-weight: 500;
}
+408 -5
View File
@@ -1,15 +1,16 @@
/* =========================================================
* Design Tokens
*
* 套主题
* 套主题
* [data-theme="dark"] 暗黑 + 暖橙默认 / 兜底
* [data-theme="pink"] 奶油白 + 樱花粉
* [data-theme="sky"] 星空蓝 + 暖星黄
*
* 切换方式
* document.documentElement.setAttribute("data-theme", "pink" | "dark")
* document.documentElement.setAttribute("data-theme", "dark" | "pink" | "sky")
*
* 设计约束
* - 套用相同的 token key组件 CSS 一行不动
* - 套用相同的 token key组件 CSS 一行不动
* - 间距 / 圆角 / 字号 / 过渡等"非主题"变量挂在 :root
* ========================================================= */
@@ -134,6 +135,7 @@
--drive-p123: #22b8c8;
--drive-pikpak: #8a6dff;
--drive-wopan: #ff8a3c;
--drive-guangyapan: #30c3a8;
--drive-onedrive: #4cabea;
--drive-localstorage: #35b88f;
@@ -152,6 +154,17 @@
--shadow-inset: inset 0 1px 0 rgba(255, 255, 255, 0.04);
}
/* 暗黑主题:保留暖橙光晕,再叠一层低对比度网格。 */
:root body::before,
:root[data-theme="dark"] body::before {
background:
linear-gradient(to right, rgba(255, 138, 60, 0.09) 1px, transparent 1px) 0 0 / 88px 88px,
linear-gradient(to bottom, rgba(255, 138, 60, 0.09) 1px, transparent 1px) 0 0 / 88px 88px,
radial-gradient(1200px 600px at 85% -10%, rgba(255, 138, 60, 0.12), transparent 60%),
radial-gradient(900px 500px at 10% 110%, rgba(90, 120, 255, 0.06), transparent 60%),
linear-gradient(180deg, rgba(255, 255, 255, 0.02), transparent 44%);
}
/* =========================================================
* 奶油白 + 樱花粉
*
@@ -215,6 +228,7 @@
--drive-p123: #1596a8;
--drive-pikpak: #8466e6;
--drive-wopan: #e57a36;
--drive-guangyapan: #229f8b;
--drive-onedrive: #2f95cf;
--drive-localstorage: #239978;
@@ -233,11 +247,13 @@
--shadow-inset: inset 0 1px 0 rgba(255, 255, 255, 0.7);
}
/* 粉白主题下 body::before "暖色光晕"改得更柔粉
/* 粉白主题下 body::before 改成柔粉网格 + 轻光晕
base.css 用的是写死的 rgba 渐变这里放一层覆盖 */
:root[data-theme="pink"] body::before {
background:
radial-gradient(1200px 600px at 85% -10%, rgba(255, 91, 138, 0.1), transparent 60%),
linear-gradient(to right, rgba(255, 91, 138, 0.13) 1px, transparent 1px) 0 0 / 82px 82px,
linear-gradient(to bottom, rgba(255, 91, 138, 0.13) 1px, transparent 1px) 0 0 / 82px 82px,
radial-gradient(1200px 600px at 85% -10%, rgba(255, 91, 138, 0.12), transparent 60%),
radial-gradient(900px 500px at 10% 110%, rgba(190, 130, 200, 0.07), transparent 60%);
}
@@ -251,3 +267,390 @@
:root[data-theme="pink"] * {
scrollbar-color: rgba(180, 90, 120, 0.28) transparent;
}
/* =========================================================
* 星空蓝 + 暖星黄
*
* 设计要点
* - 页底用浅天空蓝带一点点紫的清透蓝不用纯白避免缺少氛围
* - 卡片提亮到接近白色但带蓝调避免和背景太接近
* - 主色用暖星黄#ffc83d与天蓝形成互补关系呼应背景里的星点
* - 文本用深夜蓝#1b2547避免纯黑割裂梦幻氛围
* - 边框/阴影用蓝调让卡片看起来像漂在天空里
* ========================================================= */
:root[data-theme="sky"] {
/* ----- 表面色 ----- */
--bg-page: #c9e4ff;
--bg-surface: #ffffff;
--bg-elevated: #eaf4ff;
--bg-sunken: #b8d8f5;
--bg-overlay: rgba(210, 230, 250, 0.82);
/* 半透明玻璃(蓝白叠透明) */
--glass-nav: rgba(232, 244, 255, 0.78);
--glass-card: rgba(255, 255, 255, 0.72);
/* ----- 边框(深蓝描边) ----- */
--border-subtle: rgba(60, 100, 170, 0.1);
--border-default: rgba(60, 100, 170, 0.18);
--border-strong: rgba(40, 70, 140, 0.26);
--border-accent: rgba(255, 200, 61, 0.6);
/* ----- 文本 ----- */
--text-strong: #1b2547;
--text-default: #324063;
--text-muted: #6a7898;
--text-faint: #a5b1c8;
--text-disabled: #ccd4e2;
--text-on-accent: #3a2400;
--text-on-dark: #1b2547;
/* ----- 主色(暖星黄) ----- */
--accent: #ffc83d;
--accent-hover: #ffd76b;
--accent-strong: #f0b21f;
--accent-soft: rgba(255, 200, 61, 0.16);
--accent-softer: rgba(255, 200, 61, 0.08);
--accent-glow: rgba(255, 200, 61, 0.35);
--accent-gradient: linear-gradient(135deg, #ffe28a 0%, #ffc83d 100%);
--accent-gradient-strong: linear-gradient(135deg, #ffeca8 0%, #ffc83d 55%, #f0a514 100%);
/* ----- 状态色(浅蓝底下加深一些) ----- */
--success: #1ea974;
--success-soft: rgba(30, 169, 116, 0.16);
--warning: #d99022;
--warning-soft: rgba(217, 144, 34, 0.16);
--danger: #e43b5c;
--danger-soft: rgba(228, 59, 92, 0.14);
--info: #2f6fd6;
--info-soft: rgba(47, 111, 214, 0.14);
/* ----- 网盘品牌色(浅蓝底下重新调谐,整体加深) ----- */
--drive-quark: #4467c8;
--drive-p115: #d8485d;
--drive-p123: #128da0;
--drive-pikpak: #6b4ed4;
--drive-wopan: #dc6d28;
--drive-guangyapan: #158b7a;
--drive-onedrive: #1f7fc0;
--drive-localstorage: #198866;
/* ----- 阴影(蓝色柔投影) ----- */
--shadow-sm: 0 1px 2px rgba(40, 80, 160, 0.1);
--shadow-md:
0 2px 4px rgba(40, 80, 160, 0.1),
0 8px 20px rgba(40, 80, 160, 0.12);
--shadow-lg:
0 4px 10px rgba(40, 80, 160, 0.12),
0 18px 40px rgba(40, 80, 160, 0.16);
--shadow-xl:
0 8px 16px rgba(40, 80, 160, 0.12),
0 28px 60px rgba(40, 80, 160, 0.2);
--shadow-glow: 0 0 0 1px var(--border-accent), 0 8px 28px var(--accent-glow);
--shadow-inset: inset 0 1px 0 rgba(255, 255, 255, 0.7);
}
/* 星空蓝主题下body::before 改为承载"网格"base.css 里它画的是暖光晕这里覆写
* 星星 / 闪光不再用 CSS 瓦片改由 React 组件 <SkyStarfield /> 放固定位置元素
* 见本文件下方 .sky-starfield / .sky-star 规则与 src/components/SkyStarfield.tsx */
:root[data-theme="sky"] body::before {
background:
linear-gradient(to right, rgba(255, 255, 255, 0.55) 1px, transparent 1px) 0 0 / 80px 80px,
linear-gradient(to bottom, rgba(255, 255, 255, 0.55) 1px, transparent 1px) 0 0 / 80px 80px;
}
/* 星空蓝主题:自定义滚动条(覆盖 base.css 的硬编码白色透明) */
:root[data-theme="sky"] ::-webkit-scrollbar-thumb {
background: rgba(60, 100, 170, 0.22);
}
:root[data-theme="sky"] ::-webkit-scrollbar-thumb:hover {
background: rgba(60, 100, 170, 0.36);
}
:root[data-theme="sky"] * {
scrollbar-color: rgba(60, 100, 170, 0.28) transparent;
}
/* =========================================================
* 星空蓝主题专属视口级星星贴纸SkyStarfield 组件
*
* 设计原则
* - 不用 CSS 瓦片 pattern那会让星星在屏幕上"重复堆叠"显脏
* - 桌面和手机分开点位桌面围绕四周手机减少数量并缩小尺寸
* - 使用 fixed 定位让首屏星星密度稳定不被长页面高度稀释
* - pointer-events: none对滚动和点击都透明
* - sky 主题下整层 display: none不渲染也不占位
* ========================================================= */
.sky-starfield {
display: none;
}
:root[data-theme="sky"] .sky-starfield {
/* fixed + 四边显式 0星星锚定视口滚动时保持稳定的背景氛围
* top/left/right/bottom 而不是 inset 是为了避免某些旧浏览器对 inset
* 简写的支持差异Safari 14.1 之前
* overflow: hidden 防止 left 百分比在窄屏下溢出造成水平滚动条 */
display: block;
position: fixed;
top: 0;
left: 0;
right: 0;
bottom: 0;
pointer-events: none;
z-index: 0;
overflow: hidden;
}
.sky-star {
position: absolute;
pointer-events: none;
display: block;
user-select: none;
max-width: none;
opacity: 0.72;
/* GIF 自身就是动画 + 彩色,不再叠 CSS twinkle / drop-shadow */
}
.sky-star--mobile {
display: none;
}
/* 每颗星的 top/left/right/bottom + width/height 都走 inline style。 */
:root .app-shell,
:root .admin-shell,
:root .admin-main,
:root .admin-login,
:root .admin-loading-screen {
position: relative;
z-index: 1;
}
@media (max-width: 768px) {
:root[data-theme="sky"] .sky-star--desktop {
display: none;
}
:root[data-theme="sky"] .sky-star--mobile {
display: block;
opacity: 0.64;
}
}
@media (min-width: 769px) {
:root .admin-sidebar,
:root[data-theme="dark"] .admin-sidebar {
background:
linear-gradient(90deg, rgba(18, 20, 27, 0.86) 0%, rgba(18, 20, 27, 0.66) 72%, rgba(18, 20, 27, 0.24) 100%),
linear-gradient(180deg, rgba(255, 255, 255, 0.045) 0%, rgba(255, 138, 60, 0.008) 100%);
border-right: 0;
box-shadow: none;
backdrop-filter: blur(10px);
-webkit-backdrop-filter: blur(10px);
}
:root .admin-sidebar__brand,
:root .admin-sidebar__footer,
:root[data-theme="dark"] .admin-sidebar__brand,
:root[data-theme="dark"] .admin-sidebar__footer {
border-color: rgba(255, 255, 255, 0.07);
}
:root .admin-nav__icon,
:root .admin-sidebar__footer .admin-sidebar__home svg,
:root .admin-sidebar__footer .admin-sidebar__check-update svg,
:root .admin-sidebar__footer .admin-sidebar__logout svg,
:root[data-theme="dark"] .admin-nav__icon,
:root[data-theme="dark"] .admin-sidebar__footer .admin-sidebar__home svg,
:root[data-theme="dark"] .admin-sidebar__footer .admin-sidebar__check-update svg,
:root[data-theme="dark"] .admin-sidebar__footer .admin-sidebar__logout svg {
background: rgba(255, 255, 255, 0.055);
border-color: rgba(255, 255, 255, 0.08);
}
:root .admin-nav__link:hover,
:root .admin-sidebar__footer .admin-sidebar__home:hover,
:root .admin-sidebar__footer .admin-sidebar__check-update:hover:not(:disabled),
:root[data-theme="dark"] .admin-nav__link:hover,
:root[data-theme="dark"] .admin-sidebar__footer .admin-sidebar__home:hover,
:root[data-theme="dark"] .admin-sidebar__footer .admin-sidebar__check-update:hover:not(:disabled) {
background: rgba(255, 255, 255, 0.055);
border-color: rgba(255, 138, 60, 0.065);
box-shadow: 0 8px 22px rgba(0, 0, 0, 0.18);
}
:root .admin-nav__link:hover .admin-nav__icon,
:root .admin-sidebar__footer .admin-sidebar__home:hover svg,
:root .admin-sidebar__footer .admin-sidebar__check-update:hover:not(:disabled) svg,
:root[data-theme="dark"] .admin-nav__link:hover .admin-nav__icon,
:root[data-theme="dark"] .admin-sidebar__footer .admin-sidebar__home:hover svg,
:root[data-theme="dark"] .admin-sidebar__footer .admin-sidebar__check-update:hover:not(:disabled) svg {
background: rgba(255, 255, 255, 0.085);
border-color: rgba(255, 255, 255, 0.13);
}
:root .admin-nav__link.is-active,
:root[data-theme="dark"] .admin-nav__link.is-active {
background: rgba(255, 255, 255, 0.08);
border-color: rgba(255, 138, 60, 0.14);
box-shadow:
0 10px 26px rgba(0, 0, 0, 0.22),
inset 0 1px 0 rgba(255, 255, 255, 0.08);
}
:root .admin-nav__link.is-active .admin-nav__icon,
:root[data-theme="dark"] .admin-nav__link.is-active .admin-nav__icon {
background: rgba(255, 138, 60, 0.07);
border-color: rgba(255, 138, 60, 0.2);
color: rgba(255, 176, 112, 0.76);
}
:root .admin-nav__link.is-active::before,
:root[data-theme="dark"] .admin-nav__link.is-active::before {
left: 8px;
width: 2px;
opacity: 0.45;
box-shadow: none;
}
:root .admin-sidebar__footer .admin-sidebar__logout:hover,
:root[data-theme="dark"] .admin-sidebar__footer .admin-sidebar__logout:hover {
background: rgba(241, 85, 108, 0.12);
border-color: rgba(241, 85, 108, 0.25);
}
:root[data-theme="pink"] .admin-sidebar {
background:
linear-gradient(90deg, rgba(255, 255, 255, 0.58) 0%, rgba(255, 245, 249, 0.34) 72%, transparent 100%),
linear-gradient(180deg, rgba(255, 91, 138, 0.08) 0%, rgba(255, 255, 255, 0.18) 100%);
border-right: 0;
box-shadow: none;
backdrop-filter: blur(10px);
-webkit-backdrop-filter: blur(10px);
}
:root[data-theme="pink"] .admin-sidebar__brand,
:root[data-theme="pink"] .admin-sidebar__footer {
border-color: rgba(255, 91, 138, 0.11);
}
:root[data-theme="pink"] .admin-nav__icon,
:root[data-theme="pink"] .admin-sidebar__footer .admin-sidebar__home svg,
:root[data-theme="pink"] .admin-sidebar__footer .admin-sidebar__check-update svg,
:root[data-theme="pink"] .admin-sidebar__footer .admin-sidebar__logout svg {
background: rgba(255, 255, 255, 0.38);
border-color: rgba(255, 91, 138, 0.12);
}
:root[data-theme="pink"] .admin-nav__link:hover,
:root[data-theme="pink"] .admin-sidebar__footer .admin-sidebar__home:hover,
:root[data-theme="pink"] .admin-sidebar__footer .admin-sidebar__check-update:hover:not(:disabled) {
background: rgba(255, 255, 255, 0.42);
border-color: rgba(255, 91, 138, 0.16);
box-shadow: 0 8px 22px rgba(180, 90, 120, 0.08);
}
:root[data-theme="pink"] .admin-nav__link:hover .admin-nav__icon,
:root[data-theme="pink"] .admin-sidebar__footer .admin-sidebar__home:hover svg,
:root[data-theme="pink"] .admin-sidebar__footer .admin-sidebar__check-update:hover:not(:disabled) svg {
background: rgba(255, 255, 255, 0.58);
border-color: rgba(255, 91, 138, 0.2);
}
:root[data-theme="pink"] .admin-nav__link.is-active {
background: rgba(255, 255, 255, 0.58);
border-color: rgba(255, 91, 138, 0.34);
box-shadow:
0 10px 26px rgba(180, 90, 120, 0.1),
inset 0 1px 0 rgba(255, 255, 255, 0.7);
}
:root[data-theme="pink"] .admin-nav__link.is-active .admin-nav__icon {
background: rgba(255, 91, 138, 0.16);
border-color: rgba(255, 91, 138, 0.42);
}
:root[data-theme="pink"] .admin-nav__link.is-active::before {
left: 8px;
width: 2px;
opacity: 0.72;
box-shadow: none;
}
:root[data-theme="pink"] .admin-sidebar__footer .admin-sidebar__logout:hover {
background: rgba(228, 59, 92, 0.1);
border-color: rgba(228, 59, 92, 0.22);
}
:root[data-theme="sky"] .admin-sidebar {
background:
linear-gradient(90deg, rgba(255, 255, 255, 0.34) 0%, rgba(255, 255, 255, 0.18) 72%, transparent 100%),
linear-gradient(180deg, rgba(255, 255, 255, 0.22) 0%, rgba(232, 244, 255, 0.08) 100%);
border-right: 0;
box-shadow: none;
backdrop-filter: blur(10px);
-webkit-backdrop-filter: blur(10px);
}
:root[data-theme="sky"] .admin-sidebar__brand,
:root[data-theme="sky"] .admin-sidebar__footer {
border-color: rgba(60, 100, 170, 0.1);
}
:root[data-theme="sky"] .admin-nav__icon,
:root[data-theme="sky"] .admin-sidebar__footer .admin-sidebar__home svg,
:root[data-theme="sky"] .admin-sidebar__footer .admin-sidebar__check-update svg,
:root[data-theme="sky"] .admin-sidebar__footer .admin-sidebar__logout svg {
background: rgba(255, 255, 255, 0.28);
border-color: rgba(60, 100, 170, 0.12);
}
:root[data-theme="sky"] .admin-nav__link:hover,
:root[data-theme="sky"] .admin-sidebar__footer .admin-sidebar__home:hover,
:root[data-theme="sky"] .admin-sidebar__footer .admin-sidebar__check-update:hover:not(:disabled) {
background: rgba(255, 255, 255, 0.26);
border-color: rgba(60, 100, 170, 0.14);
box-shadow: 0 8px 22px rgba(40, 80, 160, 0.08);
}
:root[data-theme="sky"] .admin-nav__link:hover .admin-nav__icon,
:root[data-theme="sky"] .admin-sidebar__footer .admin-sidebar__home:hover svg,
:root[data-theme="sky"] .admin-sidebar__footer .admin-sidebar__check-update:hover:not(:disabled) svg {
background: rgba(255, 255, 255, 0.42);
border-color: rgba(60, 100, 170, 0.18);
}
:root[data-theme="sky"] .admin-nav__link.is-active {
background: rgba(255, 255, 255, 0.42);
border-color: rgba(255, 200, 61, 0.42);
box-shadow:
0 10px 26px rgba(40, 80, 160, 0.09),
inset 0 1px 0 rgba(255, 255, 255, 0.58);
}
:root[data-theme="sky"] .admin-nav__link.is-active .admin-nav__icon {
background: rgba(255, 200, 61, 0.2);
border-color: rgba(255, 200, 61, 0.58);
}
:root[data-theme="sky"] .admin-nav__link.is-active::before {
left: 8px;
width: 2px;
opacity: 0.72;
box-shadow: none;
}
:root[data-theme="sky"] .admin-sidebar__footer .admin-sidebar__logout:hover {
background: rgba(228, 59, 92, 0.1);
border-color: rgba(228, 59, 92, 0.22);
}
}
/* 后台和登录页默认会用 var(--bg-page) 填满网格会被遮挡
* 三套主题都让外壳透明露出 body::before 的底纹卡片本身仍是实心表面 */
:root .admin-shell,
:root .admin-main,
:root .admin-login,
:root .admin-loading-screen {
background: transparent;
}
+33 -7
View File
@@ -305,6 +305,14 @@
--drive-shadow: rgba(255, 138, 60, 0.15);
--drive-shadow-strong: rgba(255, 138, 60, 0.4);
}
.source-badge[data-kind="guangyapan"] {
--drive-color: var(--drive-guangyapan);
--drive-bg: rgba(48, 195, 168, 0.12);
--drive-text: #9debdc;
--drive-border: rgba(48, 195, 168, 0.35);
--drive-shadow: rgba(48, 195, 168, 0.15);
--drive-shadow-strong: rgba(48, 195, 168, 0.4);
}
.source-badge[data-kind="onedrive"] {
--drive-color: var(--drive-onedrive);
--drive-bg: rgba(76, 171, 234, 0.12);
@@ -456,6 +464,7 @@
.video-card__source[data-kind="p123"] { --source-color: var(--drive-p123); }
.video-card__source[data-kind="pikpak"] { --source-color: var(--drive-pikpak); }
.video-card__source[data-kind="wopan"] { --source-color: var(--drive-wopan); }
.video-card__source[data-kind="guangyapan"] { --source-color: var(--drive-guangyapan); }
.video-card__source[data-kind="onedrive"] { --source-color: var(--drive-onedrive); }
.video-card__source[data-kind="localstorage"] { --source-color: var(--drive-localstorage); }
@@ -483,10 +492,15 @@
}
.skeleton-card {
--skeleton-card-bg: var(--bg-surface);
--skeleton-card-border: var(--border-subtle);
--skeleton-shimmer-base: rgba(255, 255, 255, 0.03);
--skeleton-shimmer-highlight: rgba(255, 255, 255, 0.08);
position: relative;
aspect-ratio: 16 / 12.5;
background: var(--bg-surface);
border: 1px solid var(--border-subtle);
background: var(--skeleton-card-bg);
border: 1px solid var(--skeleton-card-border);
border-radius: var(--radius-md);
padding: 8px;
box-sizing: border-box;
@@ -495,6 +509,18 @@
flex-direction: column;
}
:root[data-theme="pink"] .skeleton-card {
--skeleton-card-border: rgba(255, 91, 138, 0.18);
--skeleton-shimmer-base: rgba(255, 91, 138, 0.12);
--skeleton-shimmer-highlight: rgba(255, 91, 138, 0.26);
}
:root[data-theme="sky"] .skeleton-card {
--skeleton-card-border: rgba(60, 100, 170, 0.18);
--skeleton-shimmer-base: rgba(60, 100, 170, 0.13);
--skeleton-shimmer-highlight: rgba(60, 100, 170, 0.26);
}
/* Skeleton image thumbnail area */
.skeleton-card::before {
content: "";
@@ -504,9 +530,9 @@
border-radius: var(--radius-sm);
background: linear-gradient(
90deg,
rgba(255, 255, 255, 0.03) 25%,
rgba(255, 255, 255, 0.08) 50%,
rgba(255, 255, 255, 0.03) 75%
var(--skeleton-shimmer-base) 25%,
var(--skeleton-shimmer-highlight) 50%,
var(--skeleton-shimmer-base) 75%
);
background-size: 200% 100%;
animation: skeleton-shimmer 1.6s ease-in-out infinite;
@@ -520,8 +546,8 @@
width: 100%;
height: 28px;
background:
linear-gradient(90deg, rgba(255, 255, 255, 0.03) 25%, rgba(255, 255, 255, 0.08) 50%, rgba(255, 255, 255, 0.03) 75%) 0 0 / 70% 12px no-repeat,
linear-gradient(90deg, rgba(255, 255, 255, 0.03) 25%, rgba(255, 255, 255, 0.08) 50%, rgba(255, 255, 255, 0.03) 75%) 0 20px / 42% 8px no-repeat;
linear-gradient(90deg, var(--skeleton-shimmer-base) 25%, var(--skeleton-shimmer-highlight) 50%, var(--skeleton-shimmer-base) 75%) 0 0 / 70% 12px no-repeat,
linear-gradient(90deg, var(--skeleton-shimmer-base) 25%, var(--skeleton-shimmer-highlight) 50%, var(--skeleton-shimmer-base) 75%) 0 20px / 42% 8px no-repeat;
background-size: 200% 100%;
animation: skeleton-shimmer 1.6s ease-in-out infinite;
}
+20
View File
@@ -94,6 +94,13 @@
--video-player-progress-hover: rgba(255, 197, 216, 0.42);
}
:root[data-theme="sky"] .video-player {
--video-player-progress: #58b8ff;
--video-player-progress-loaded: rgba(88, 184, 255, 0.36);
--video-player-progress-track: rgba(210, 236, 255, 0.32);
--video-player-progress-hover: rgba(166, 220, 255, 0.45);
}
.video-player__mount {
position: absolute;
inset: 0;
@@ -669,6 +676,12 @@
color: var(--drive-wopan);
}
.vd-meta__chip[data-tone="guangyapan"] {
background: rgba(48, 195, 168, 0.14);
border-color: rgba(48, 195, 168, 0.3);
color: var(--drive-guangyapan);
}
.vd-meta__chip[data-tone="onedrive"] {
background: rgba(76, 171, 234, 0.14);
border-color: rgba(76, 171, 234, 0.3);
@@ -1441,6 +1454,13 @@
letter-spacing: 0.04em;
}
:root[data-theme="sky"] .vd-rail__hd {
background: var(--accent);
color: var(--text-on-accent);
border: 0;
box-shadow: 0 4px 10px var(--accent-glow);
}
.vd-rail__body {
min-width: 0;
display: flex;
+1 -1
View File
@@ -57,7 +57,7 @@ export type VideoDetail = VideoItem & {
export type PreviewState = "idle" | "intent" | "loading" | "playing" | "error";
export type SortKey = "latest" | "hot" | "week" | "long" | "hd" | "featured";
export type SortKey = "latest" | "hot" | "recent";
export type TagItem = {
id: string;
+68 -3
View File
@@ -78,9 +78,9 @@ test("spider91 upload target uses explicit local-save option instead of auto tar
assert.match(combinedSource, /本地保存,不上传/);
assert.match(
combinedSource,
/d\.kind === "pikpak"[\s\S]*d\.kind === "p115"[\s\S]*d\.kind === "p123"[\s\S]*d\.kind === "onedrive"[\s\S]*d\.kind === "googledrive"[\s\S]*d\.kind === "wopan"/
/d\.kind === "pikpak"[\s\S]*d\.kind === "p115"[\s\S]*d\.kind === "p123"[\s\S]*d\.kind === "onedrive"[\s\S]*d\.kind === "googledrive"[\s\S]*d\.kind === "wopan"[\s\S]*d\.kind === "guangyapan"/
);
assert.match(crawlerPageSource, /UPLOAD_TARGET_KINDS[\s\S]*"wopan"/);
assert.match(crawlerPageSource, /UPLOAD_TARGET_KINDS[\s\S]*"wopan"[\s\S]*"guangyapan"/);
assert.doesNotMatch(combinedSource, /自动:唯一/);
assert.doesNotMatch(combinedSource, /自动模式/);
assert.doesNotMatch(combinedSource, /较早的视频会上传到该云盘根目录下的 91 Spider 文件夹/);
@@ -145,7 +145,8 @@ test("googledrive drive form supports online API and custom OAuth client modes",
assert.match(fields, /key: "client_id"/);
assert.match(fields, /key: "client_secret"/);
assert.match(fields, /googleDriveUsesOnlineAPI\(creds\)/);
assert.doesNotMatch(fields, /key: "api_url_address"/);
assert.match(fields, /key: "api_url_address"/);
assert.match(fields, /OpenList 在线 API URL/);
assert.doesNotMatch(fields, /在线 API 模式填写 OpenList 获取的 refresh_token/);
assert.doesNotMatch(constantsSource, /请参考OpenList文档中关于谷歌云盘的配置方法。/);
assert.doesNotMatch(constantsSource, /选择自建 Google OAuth 客户端后,服务端会直接请求 Google OAuth token 接口续期。/);
@@ -154,7 +155,9 @@ test("googledrive drive form supports online API and custom OAuth client modes",
assert.match(driveFormSource, /className="admin-form-select"/);
assert.match(driveFormSource, /ChevronDown/);
assert.match(drivesPageSource, /googleDriveUseOnlineAPI/);
assert.match(drivesPageSource, /googleDriveOpenListApiUrl/);
assert.match(apiSource, /googleDriveUseOnlineAPI\?: boolean/);
assert.match(apiSource, /googleDriveOpenListApiUrl\?: string/);
assert.doesNotMatch(fields, /key: "access_token"/);
});
@@ -175,6 +178,32 @@ test("pikpak drive form only exposes account login fields", () => {
assert.doesNotMatch(fields, /key: "disable_media_link"/);
});
test("guangyapan drive form exposes qr login and token fields", () => {
assertDriveTypeOption("guangyapan", "光鸭网盘");
assert.match(driveFormSource, /GuangYaPanQRCodeLogin/);
assert.match(driveFormSource, /form\.kind === "guangyapan"/);
assert.match(apiSource, /startGuangYaPanQRLogin/);
assert.match(apiSource, /getGuangYaPanQRStatus/);
const match =
/case "guangyapan":\s*return \[([\s\S]*?)\];\s*case "onedrive":/.exec(
combinedSource
);
assert.ok(match, "guangyapan credential field block should be present");
const fields = match[1];
assert.match(fields, /key: "root_path"/);
assert.match(fields, /key: "refresh_token"/);
assert.match(fields, /key: "access_token"/);
assert.doesNotMatch(fields, /key: "phone_number"/);
assert.doesNotMatch(fields, /key: "send_code"/);
assert.doesNotMatch(fields, /key: "verify_code"/);
assert.doesNotMatch(fields, /key: "captcha_token"/);
assert.doesNotMatch(fields, /key: "client_id"/);
assert.doesNotMatch(fields, /key: "device_id"/);
assert.match(combinedSource, /if \(kind === "guangyapan"\) return ""/);
});
test("localstorage drive form asks for a server directory path", () => {
assertDriveTypeOption("localstorage", "本地存储");
@@ -196,6 +225,7 @@ test("drive type selector keeps primary source order", () => {
{ value: "p115", label: "115 网盘" },
{ value: "p123", label: "123网盘" },
{ value: "pikpak", label: "PikPak" },
{ value: "guangyapan", label: "光鸭网盘" },
{ value: "onedrive", label: "OneDrive" },
{ value: "googledrive", label: "Google Drive" },
{ value: "localstorage", label: "本地存储" },
@@ -221,6 +251,7 @@ test("crawler management is a separate admin section", () => {
assert.match(crawlerPageSource, /api\.listDrives/);
assert.match(crawlerPageSource, /api\.upsertCrawler/);
assert.match(crawlerPageSource, /api\.runCrawler/);
assert.match(crawlerPageSource, /api\.uploadCrawlerVideos/);
assert.match(crawlerPageSource, /api\.stopCrawlerTasks/);
assert.match(crawlerPageSource, /api\.deleteCrawler/);
assert.match(crawlerPageSource, /api\.importCrawlerScriptFile/);
@@ -232,6 +263,21 @@ test("crawler management is a separate admin section", () => {
assert.match(crawlerPageSource, /测试通过/);
assert.match(crawlerPageSource, /Spider91UploadTargetField/);
assert.match(crawlerPageSource, /uploadDriveId/);
assert.match(crawlerPageSource, /api\.setDriveTeaserEnabled/);
assert.match(crawlerPageSource, /admin-crawler-preview-card-toggle/);
assert.match(crawlerPageSource, /预览:开/);
assert.match(crawlerPageSource, /预览:关/);
assert.match(crawlerPageSource, /上传视频/);
assert.match(crawlerPageSource, /aria-pressed=\{crawler\.teaserEnabled\}/);
assert.doesNotMatch(crawlerPageSource, /crawlerUploadBlockedReason/);
assert.doesNotMatch(crawlerPageSource, /disabled=\{uploading/);
assert.doesNotMatch(crawlerPageSource, /crawlerStatusLabel/);
assert.doesNotMatch(crawlerPageSource, /admin-crawler-preview-card-toggle \$\{crawler\.teaserEnabled/);
assert.doesNotMatch(adminCss, /admin-crawler-preview-card-toggle\.is-on/);
assert.doesNotMatch(crawlerPageSource, /admin-crawler-pipeline/);
assert.doesNotMatch(adminCss, /admin-crawler-(pipeline|stage)/);
assert.doesNotMatch(crawlerPageSource, /teaserEnabled: form\.teaserEnabled/);
assert.doesNotMatch(crawlerPageSource, /aria-pressed=\{form\.teaserEnabled\}/);
assert.match(crawlerPageSource, /UPLOAD_TARGET_KINDS/);
assert.doesNotMatch(crawlerPageSource, /新建脚本/);
assert.doesNotMatch(crawlerPageSource, /爬虫 ID/);
@@ -247,7 +293,10 @@ test("crawler management is a separate admin section", () => {
assert.doesNotMatch(crawlerPageSource, /内置 91/);
assert.match(apiSource, /type AdminCrawler/);
assert.match(apiSource, /uploadDriveId\?: string/);
assert.match(apiSource, /teaserEnabled: boolean/);
assert.doesNotMatch(apiSource, /teaserEnabled\?: boolean/);
assert.match(apiSource, /"\/crawlers"/);
assert.match(apiSource, /\/crawlers\/\$\{encodeURIComponent\(id\)\}\/upload/);
assert.match(apiSource, /"\/crawlers\/import-file"/);
assert.match(apiSource, /"\/crawlers\/import-url"/);
assert.match(apiSource, /"\/crawlers\/test-script"/);
@@ -264,6 +313,7 @@ test("drive cards use configured abbreviations and visible fallback icon colors"
assert.match(drivesPageSource, /driveKindAbbr\(d\.kind\)/);
assert.match(adminCss, /\.admin-drive-card__brand-icon\s*\{[^}]*background:\s*var\(--accent\);/s);
assert.match(adminCss, /\.admin-drive-card__brand-icon\[data-kind="googledrive"\]\s*\{\s*background:\s*#4285f4;\s*\}/);
assert.match(adminCss, /\.admin-drive-card__brand-icon\[data-kind="guangyapan"\]\s*\{\s*background:\s*var\(--drive-guangyapan\);/);
});
test("drive management exposes stop task controls", () => {
@@ -276,6 +326,21 @@ test("drive management exposes stop task controls", () => {
assert.match(drivesPageSource, /停止所有网盘任务/);
});
test("drive detail primary actions use the rescan button color", () => {
assert.match(
drivesPageSource,
/className="admin-btn is-primary"\s+onClick=\{\(\) => handleRescan\(d\)\}/
);
assert.match(
drivesPageSource,
/className="admin-btn is-primary"\s+onClick=\{\(\) => handleStopDriveTasks\(d\)\}/
);
assert.match(
drivesPageSource,
/className="admin-btn is-primary"\s+onClick=\{\(\) => openEdit\(d\)\}/
);
});
test("drive rescan reports busy storage tasks instead of queueing duplicates", () => {
assert.match(apiSource, /accepted:\s*boolean;\s*message\?:\s*string/);
assert.match(apiSource, /scanGenerationStatus\?: DriveGenerationStatus/);
+74
View File
@@ -0,0 +1,74 @@
import assert from "node:assert/strict";
import { readFileSync } from "node:fs";
import test from "node:test";
const toastSource = readFileSync(
new URL("../src/admin/ToastContext.tsx", import.meta.url),
"utf8"
);
const adminCss = readFileSync(
new URL("../src/styles/admin.css", import.meta.url),
"utf8"
);
function ruleBody(css: string, selector: string): string {
const escapedSelector = selector.replace(/[.*+?^${}()|[\]\\]/g, "\\$&");
const match = css.match(new RegExp(`${escapedSelector}\\s*\\{([^}]*)\\}`));
assert.ok(match, `Expected CSS rule for ${selector}`);
return match[1];
}
function mobileCss(): string {
const marker = "@media (max-width: 768px)";
const start = adminCss.indexOf(marker);
assert.notEqual(start, -1, "Expected mobile admin media query");
return adminCss.slice(start);
}
test("admin toasts auto-dismiss and copy their text when clicked", () => {
assert.match(toastSource, /const TOAST_DISMISS_MS = 2500/);
assert.match(toastSource, /const TOAST_COPY_SUCCESS_TEXT = "已复制到剪贴板"/);
assert.match(toastSource, /const TOAST_COPY_ERROR_TEXT = "复制失败,请手动复制"/);
assert.match(toastSource, /navigator\.clipboard\?\.writeText/);
assert.match(toastSource, /fallbackCopyText\(text\)/);
assert.match(toastSource, /document\.execCommand\("copy"\)/);
assert.match(toastSource, /addToast\(TOAST_COPY_SUCCESS_TEXT,\s*"success",\s*false\)/);
assert.match(toastSource, /addToast\(TOAST_COPY_ERROR_TEXT,\s*"error",\s*false\)/);
assert.match(toastSource, /t\.copyable\s*\?\s*" is-copyable"\s*:\s*""/);
assert.match(toastSource, /onClick=\{t\.copyable \? \(\) => copyToastText\(t\.text\) : undefined\}/);
assert.match(toastSource, /aria-label=\{t\.copyable \? `复制提示:\$\{t\.text\}` : undefined\}/);
assert.match(toastSource, /event\.key !== "Enter" && event\.key !== " "/);
assert.doesNotMatch(toastSource, /onClick=\{\(\) => scheduleDismiss/);
assert.doesNotMatch(toastSource, /pinnedToastIDs/);
assert.doesNotMatch(toastSource, /isDismissPaused/);
assert.doesNotMatch(toastSource, /pinDismiss/);
assert.doesNotMatch(toastSource, /className="admin-toast__close"/);
assert.doesNotMatch(toastSource, /aria-label="关闭提示"/);
assert.doesNotMatch(toastSource, /<X size=/);
assert.doesNotMatch(toastSource, /event\.stopPropagation\(\)/);
assert.doesNotMatch(toastSource, /onPointerEnter/);
assert.doesNotMatch(toastSource, /onPointerLeave/);
});
test("admin toasts show long messages without internal scrolling", () => {
const baseToast = ruleBody(adminCss, ".admin-toast");
const baseText = ruleBody(adminCss, ".admin-toast__text");
const mobileToast = ruleBody(mobileCss(), ".admin-toast");
assert.match(baseToast, /max-width\s*:\s*min\(520px,\s*calc\(100vw - 48px\)\)/);
assert.match(baseToast, /padding\s*:\s*14px\s+18px/);
assert.match(baseToast, /position\s*:\s*relative/);
assert.match(baseToast, /overflow-wrap\s*:\s*anywhere/);
assert.match(baseToast, /touch-action\s*:\s*manipulation/);
assert.doesNotMatch(baseToast, /cursor\s*:\s*pointer/);
assert.match(ruleBody(adminCss, ".admin-toast.is-copyable"), /cursor\s*:\s*pointer/);
assert.match(baseText, /display\s*:\s*block/);
assert.doesNotMatch(adminCss, /\.admin-toast__close/);
assert.match(mobileToast, /max-width\s*:\s*100%/);
assert.match(mobileToast, /text-align\s*:\s*left/);
assert.doesNotMatch(baseToast, /max-height/);
assert.doesNotMatch(baseText, /max-height/);
assert.doesNotMatch(baseText, /overflow-y\s*:\s*auto/);
assert.doesNotMatch(mobileToast, /max-height/);
assert.doesNotMatch(mobileCss(), /\.admin-toast__text\s*\{/);
});
+16
View File
@@ -20,3 +20,19 @@ test("admin videos batch delete runs deletions sequentially", () => {
/Promise\.allSettled\(\s*ids\.map\(\(id\) => api\.deleteVideo\(id(?:, [^)]+)?\)\)\s*\)/
);
});
test("admin videos show generating status after preview regeneration is accepted", () => {
assert.match(videosPageSource, /const REGEN_PREVIEW_STATUS = "generating";/);
assert.match(videosPageSource, /const \[regenPreviewById, setRegenPreviewById\]/);
assert.match(videosPageSource, /trackRegeneratingPreview\(\[v\]\)/);
assert.match(videosPageSource, /<PreviewStatus s=\{isPreviewGenerating\(v\) \? REGEN_PREVIEW_STATUS : v\.previewStatus\} \/>/);
assert.match(videosPageSource, /refreshListOnly\(\)/);
});
test("admin videos keep generating status after page refresh", () => {
assert.match(videosPageSource, /const hasGeneratingPreview = list\.some\(\(v\) => v\.previewStatus === REGEN_PREVIEW_STATUS\);/);
assert.match(videosPageSource, /if \(trackedRegenCount === 0 && !hasGeneratingPreview\) return;/);
assert.match(videosPageSource, /function isPreviewGenerating\(v: api\.AdminVideo\)/);
assert.match(videosPageSource, /return !!regenPreviewById\[v\.id\] \|\| v\.previewStatus === REGEN_PREVIEW_STATUS;/);
assert.match(videosPageSource, /disabled=\{isPreviewGenerating\(v\)\}/);
});
+20
View File
@@ -0,0 +1,20 @@
import assert from "node:assert/strict";
import { readFileSync } from "node:fs";
import test from "node:test";
const sortToolbarSource = readFileSync(
new URL("../src/components/SortToolbar.tsx", import.meta.url),
"utf8"
);
const typesSource = readFileSync(new URL("../src/types.ts", import.meta.url), "utf8");
test("list page sort toolbar only exposes active sort options", () => {
assert.match(sortToolbarSource, /\{ key: "latest", label: "最新" \}/);
assert.match(sortToolbarSource, /\{ key: "hot", label: "最热" \}/);
assert.match(sortToolbarSource, /\{ key: "recent", label: "最近观看" \}/);
for (const removed of ["本周", "最长", "高清", "精选"]) {
assert.doesNotMatch(sortToolbarSource, new RegExp(removed));
}
assert.match(typesSource, /export type SortKey = "latest" \| "hot" \| "recent";/);
});

Some files were not shown because too many files have changed in this diff Show More