mirror of
https://github.com/nianzhibai/91.git
synced 2026-06-15 16:55:42 +08:00
Compare commits
26 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
| fa4ea469c3 | |||
| 92885748fd | |||
| 093724a59d | |||
| 9cd30c8059 | |||
| bec6d9496c | |||
| f187302b8e | |||
| 739baf1294 | |||
| af18bbbf4c | |||
| 309b621084 | |||
| 286329c446 | |||
| 1d5b5c2495 | |||
| fac60b0054 | |||
| 19a939e80f | |||
| 16a2a7e03c | |||
| b9b6c5e098 | |||
| 4200919774 | |||
| 33d970a322 | |||
| 59e9b435a0 | |||
| dcda0e2e36 | |||
| 87709792f1 | |||
| 655da05b94 | |||
| 674a92be16 | |||
| 9ada39debb | |||
| 24d1244bc3 | |||
| 0dd0c45509 | |||
| ef6eadd0a6 |
@@ -26,6 +26,8 @@ jobs:
|
||||
steps:
|
||||
- name: Checkout
|
||||
uses: actions/checkout@v4
|
||||
with:
|
||||
fetch-depth: 0
|
||||
|
||||
- name: Set up QEMU
|
||||
uses: docker/setup-qemu-action@v3
|
||||
@@ -52,7 +54,19 @@ jobs:
|
||||
type=semver,pattern={{version}}
|
||||
type=semver,pattern={{major}}.{{minor}}
|
||||
type=sha,prefix=sha-
|
||||
type=raw,value=latest,enable={{is_default_branch}}
|
||||
type=raw,value=latest,enable=${{ startsWith(github.ref, 'refs/tags/v') }}
|
||||
type=raw,value=stable,enable=${{ startsWith(github.ref, 'refs/tags/v') }}
|
||||
|
||||
- name: Determine image version
|
||||
id: version
|
||||
shell: bash
|
||||
run: |
|
||||
if [[ "$GITHUB_REF" == refs/tags/v* ]]; then
|
||||
version="$GITHUB_REF_NAME"
|
||||
else
|
||||
version="$(git describe --tags --always --dirty 2>/dev/null || git rev-parse --short=12 HEAD)"
|
||||
fi
|
||||
echo "version=$version" >> "$GITHUB_OUTPUT"
|
||||
|
||||
- name: Build and push Docker image
|
||||
uses: docker/build-push-action@v6
|
||||
@@ -63,6 +77,6 @@ jobs:
|
||||
tags: ${{ steps.meta.outputs.tags }}
|
||||
labels: ${{ steps.meta.outputs.labels }}
|
||||
build-args: |
|
||||
VERSION=${{ startsWith(github.ref, 'refs/tags/v') && github.ref_name || '' }}
|
||||
VERSION=${{ steps.version.outputs.version }}
|
||||
cache-from: type=gha
|
||||
cache-to: type=gha,mode=max
|
||||
|
||||
@@ -0,0 +1,135 @@
|
||||
# CLAUDE.md
|
||||
|
||||
This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
|
||||
|
||||
## Common Commands
|
||||
|
||||
### Full Local Development
|
||||
|
||||
```bash
|
||||
npm install
|
||||
./start.sh
|
||||
```
|
||||
|
||||
`./start.sh` starts the Go backend on `127.0.0.1:9192` and the frontend on `0.0.0.0:9191`. By default it builds the frontend and runs Vite preview mode. Use hot reload with:
|
||||
|
||||
```bash
|
||||
FRONTEND_MODE=dev ./start.sh --restart
|
||||
```
|
||||
|
||||
Useful variants:
|
||||
|
||||
```bash
|
||||
./start.sh --status
|
||||
./start.sh --restart
|
||||
./start.sh --stop
|
||||
```
|
||||
|
||||
### Frontend
|
||||
|
||||
Run from the repository root:
|
||||
|
||||
```bash
|
||||
npm run dev # Vite dev server; default port comes from Vite unless overridden
|
||||
npm run dev:raw # Vite dev server on 127.0.0.1:5173
|
||||
npm run build # tsc -b && vite build
|
||||
npm run preview # Vite preview; vite.config.ts uses port 9191
|
||||
npm run lint # TypeScript no-emit check
|
||||
npm test # node --import tsx --test tests/*.test.ts
|
||||
```
|
||||
|
||||
Run one frontend test:
|
||||
|
||||
```bash
|
||||
node --import tsx --test tests/previewIntent.test.ts
|
||||
```
|
||||
|
||||
### Backend
|
||||
|
||||
Run from `backend/` unless noted:
|
||||
|
||||
```bash
|
||||
go run ./cmd/server
|
||||
go test ./... -count=1
|
||||
go build -o video-server ./cmd/server
|
||||
```
|
||||
|
||||
Run one backend package or test:
|
||||
|
||||
```bash
|
||||
go test ./internal/scanner -count=1
|
||||
go test ./internal/scanner -run TestParse -count=1
|
||||
```
|
||||
|
||||
The backend requires Go 1.23+ and uses vendored dependencies in `backend/vendor/`, so keep `go mod vendor` in sync after dependency changes.
|
||||
|
||||
### Release and Deployment
|
||||
|
||||
```bash
|
||||
scripts/build-release.sh # builds Linux amd64/arm64 release tarballs into release/
|
||||
sudo bash install.sh # prebuilt installer flow used by README
|
||||
sudo bash deploy.sh # build from current checkout and install systemd services
|
||||
```
|
||||
|
||||
Docker uses the root `Dockerfile` and `docker-compose.yml`. The runtime image exposes port `9191` and stores persistent data under `/opt/video-site-91/data`.
|
||||
|
||||
## Architecture Overview
|
||||
|
||||
This is a private video aggregation site with a React/Vite frontend and a Go backend.
|
||||
|
||||
### Frontend
|
||||
|
||||
The frontend is a React 18 SPA under `src/`. `src/main.tsx` mounts `BrowserRouter`, `ToastProvider`, and `AuthProvider`, then renders `src/App.tsx`. `App.tsx` defines the public app routes (`/`, `/list`, `/shorts`, `/upload`, `/video/:id`) and admin routes under `/admin`; both main-site and admin pages are wrapped in `RequireAuth`, while `/login` is public.
|
||||
|
||||
Frontend API calls are split by surface:
|
||||
|
||||
- `src/data/videos.ts` calls the main authenticated API under `/api` and upload/proxy-related endpoints.
|
||||
- `src/admin/api.ts` is the admin API client for `/admin/api`, always sending cookies and raising `UnauthorizedError` on `401`.
|
||||
|
||||
`vite.config.ts` proxies `/api`, `/p`, and `/admin/api` to `http://127.0.0.1:9192`, with frontend dev/preview served on port `9191` by default. The alias `@` maps to `src`.
|
||||
|
||||
Styling is plain CSS loaded from `src/main.tsx` in token/base/layout/navigation/search/video/admin layers. Shared UI lives in `src/components`, page-level screens in `src/pages`, and admin screens in `src/admin`.
|
||||
|
||||
### Backend
|
||||
|
||||
The backend entrypoint is `backend/cmd/server/main.go`. It loads `config.yaml` or `VIDEO_CONFIG`, creates the SQLite catalog and preview directories, builds the app state, registers API routes, starts the nightly runner, and then asynchronously attaches configured external drives so slow upstream login checks do not block port binding.
|
||||
|
||||
Important backend packages:
|
||||
|
||||
- `internal/config`: YAML config loading and first-run admin credential setup.
|
||||
- `internal/catalog`: SQLite catalog, schema migration, video metadata, settings, tags, drive records, generation status, and deduplication state. It opens SQLite with WAL and a busy timeout.
|
||||
- `internal/drives`: provider abstraction. Implementations include `quark`, `p115`, `pikpak`, `wopan`, `onedrive`, `localstorage`, `localupload`, and `spider91`.
|
||||
- `internal/scanner`: recursively lists drive directories, parses filenames/tags, upserts catalog videos, applies skip-directory rules, and enqueues newly discovered videos.
|
||||
- `internal/preview`: ffprobe/ffmpeg thumbnail and teaser generation workers. Generated assets are local files under the configured preview directory.
|
||||
- `internal/fingerprint`: asynchronous sampled SHA-256 worker used for cross-drive duplicate detection.
|
||||
- `internal/proxy`: `/p/*` media serving. Some providers redirect with `302` to signed CDN URLs, while providers requiring backend-held headers are reverse-proxied with Range support.
|
||||
- `internal/api`: main API and admin API route handlers.
|
||||
- `internal/nightly`: daily pipeline for drive scans, spider91 crawl, migration, queue drain, and duplicate asset cleanup.
|
||||
- `internal/spider91migrate`: migration from spider91 downloads to a configured cloud drive.
|
||||
|
||||
### Runtime Flow
|
||||
|
||||
1. Admin adds or edits drives through `/admin/drives`, which persists drive config in the catalog.
|
||||
2. The server attaches the drive implementation into the proxy registry and can trigger scans.
|
||||
3. Scans convert provider files into catalog video rows, parse titles/authors/tags from filenames, and queue preview/fingerprint work.
|
||||
4. The frontend lists videos through `/api/home`, `/api/list`, `/api/video/:id`, and streams media through `/p/*` endpoints.
|
||||
5. The nightly runner performs the scheduled end-to-end maintenance pipeline; admins can trigger it manually through `/admin/api/jobs/nightly/run`.
|
||||
|
||||
### Configuration and Data
|
||||
|
||||
Backend defaults come from `backend/config.example.yaml`. On first backend start, `config.yaml` is created automatically if missing. Default local development paths are:
|
||||
|
||||
- Backend listen address: `127.0.0.1:9192`
|
||||
- SQLite DB: `backend/data/video-site.db`
|
||||
- Generated previews/thumbs: `backend/data/previews`
|
||||
|
||||
Docker and installer deployments rewrite config paths so data lives under `/opt/video-site-91/data` or the mounted `./data` directory.
|
||||
|
||||
`VIDEO_FRONTEND_DIR` controls where the Go server looks for built frontend assets. If unset, it serves `./dist` when present. Backend routes (`/api`, `/admin/api`, `/p`) are excluded from the SPA fallback.
|
||||
|
||||
## Notes for Changes
|
||||
|
||||
- Main-site API routes and proxy routes require authentication; only login/setup and `/api/settings/theme` are intentionally public.
|
||||
- When adding a new drive provider, implement `internal/drives.Drive`, persist any needed config through catalog/admin APIs, attach it in `cmd/server`, and decide whether `/p/stream` should redirect or reverse-proxy in `internal/proxy`.
|
||||
- Generated thumbnails and teasers are local runtime assets; do not treat them as source files.
|
||||
- Frontend tests use Node's built-in test runner with `tsx`; TypeScript linting only checks `src` through the root `tsconfig.json`.
|
||||
+1
-1
@@ -48,7 +48,7 @@ COPY backend/config.example.yaml ./config.example.yaml
|
||||
COPY 91VideoSpider/ ./91VideoSpider/
|
||||
COPY docker-entrypoint.sh /usr/local/bin/docker-entrypoint.sh
|
||||
|
||||
ARG VERSION=
|
||||
ARG VERSION=dev
|
||||
|
||||
ENV VIDEO_CONFIG=/opt/video-site-91/data/config.yaml \
|
||||
VIDEO_FRONTEND_DIR=/opt/video-site-91/dist \
|
||||
|
||||
@@ -5,28 +5,27 @@
|
||||
</p>
|
||||
|
||||
<p align="center">
|
||||
😄个人 91 站😄
|
||||
😄 个人私有视频站 😄
|
||||
</p>
|
||||
|
||||
<p align="center">
|
||||
<a href="#快速开始">快速开始</a> ·
|
||||
<a href="#功能特性">功能特性</a> ·
|
||||
<a href="#预览图">预览图</a> ·
|
||||
<a href="#数据存放位置">数据目录</a>
|
||||
</p>
|
||||
|
||||
---
|
||||
|
||||
## 项目说明
|
||||
## 功能特性
|
||||
|
||||
支持 115 云盘、PikPak 云盘和服务器本地目录作为视频播放后端。
|
||||
|
||||
采用 115 云盘和 PikPak 云盘的 302 重定向播放,不占用服务器带宽,也不会因为服务器带宽小而影响视频播放体验。
|
||||
|
||||
服务器只负责扫描云盘或本地目录中的视频文件,并给每个视频生成封面图和预览片段。
|
||||
|
||||
你可以通过封面图和预览片段,在首页快速选择想看的视频。
|
||||
|
||||
支持 91 爬虫,爬取 91 的本月最热视频。
|
||||
|
||||
内置两种主题:黑黄主题(91 经典主题)和粉白主题。
|
||||
|
||||
支持短视频模式,一键切换成熟悉的抖音模式。
|
||||
|
||||
该项目2C2G服务器稳定跑👍👍👍
|
||||
- **多后端支持** — 兼容 115 云盘、PikPak 云盘、OneDrive、Google Drive 和本地存储
|
||||
- **低带宽播放** — 支持 302 的云盘可直连播放;Google Drive 等需鉴权直链的来源走后端代理
|
||||
- **封面 & 预览片段** — 自动为每个视频生成封面图和预览片段,首页快速选片
|
||||
- **91 爬虫** — 内置爬虫,支持抓取 91 本月最热视频
|
||||
- **双主题** — 黑黄经典主题 / 粉白清新主题,随时切换
|
||||
- **短视频模式** — 一键切换抖音风格,沉浸刷片
|
||||
- **低资源占用** — 2C2G 服务器稳定运行,主要性能消耗就是封面图和预览视频的生成
|
||||
|
||||
---
|
||||
|
||||
@@ -35,82 +34,86 @@
|
||||
### 电脑端
|
||||
|
||||
<p>
|
||||
<img width="49%" alt="91 电脑端首页" src="https://github.com/user-attachments/assets/9808fceb-760b-4dd5-b7d2-8622b95b90d5" />
|
||||
<img width="49%" alt="91 电脑端播放页" src="https://github.com/user-attachments/assets/859db4aa-1fba-44f2-bb46-1db07c2f964f" />
|
||||
<img width="49%" alt="首页" src="https://github.com/user-attachments/assets/9808fceb-760b-4dd5-b7d2-8622b95b90d5" />
|
||||
<img width="49%" alt="播放页" src="https://github.com/user-attachments/assets/859db4aa-1fba-44f2-bb46-1db07c2f964f" />
|
||||
</p>
|
||||
|
||||
<p>
|
||||
<img width="49%" alt="91 电脑端主题" src="https://github.com/user-attachments/assets/96bea37a-8764-413e-9b70-1856b4ae0cd2" />
|
||||
<img width="49%" alt="91 电脑端管理页" src="https://github.com/user-attachments/assets/29c1e27a-7651-4dfc-93dd-556331844214" />
|
||||
<img width="49%" alt="主题切换" src="https://github.com/user-attachments/assets/96bea37a-8764-413e-9b70-1856b4ae0cd2" />
|
||||
<img width="49%" alt="管理页" src="https://github.com/user-attachments/assets/29c1e27a-7651-4dfc-93dd-556331844214" />
|
||||
</p>
|
||||
|
||||
### 手机端
|
||||
|
||||
<p align="center">
|
||||
<img width="1284" height="1134" alt="PixPin_2026-05-29_11-54-12" src="https://github.com/user-attachments/assets/bdb7a86c-a4e5-483e-a307-e02c0bb34dac" />
|
||||
<img width="1284" height="1134" alt="手机端" src="https://github.com/user-attachments/assets/bdb7a86c-a4e5-483e-a307-e02c0bb34dac" />
|
||||
</p>
|
||||
|
||||
---
|
||||
|
||||
## 快速开始
|
||||
|
||||
### 一键安装脚本(推荐)
|
||||
### 方式一:一键安装脚本(推荐)
|
||||
|
||||
```bash
|
||||
sudo apt update
|
||||
sudo apt install -y curl ca-certificates
|
||||
sudo apt update && sudo apt install -y curl ca-certificates
|
||||
curl -fsSL https://raw.githubusercontent.com/nianzhibai/91/main/install.sh -o install.sh
|
||||
sudo bash install.sh
|
||||
```
|
||||
|
||||
部署完成后访问:
|
||||
|
||||
- 前台:`http://服务器IP:9191/`
|
||||
- 后台:`http://服务器IP:9191/admin`
|
||||
| 地址 | 说明 |
|
||||
|------|------|
|
||||
| `http://服务器IP:9191/` | 前台 |
|
||||
| `http://服务器IP:9191/admin` | 后台管理 |
|
||||
|
||||
安装后会自动创建 `91` 指令:
|
||||
**注意:如果首次访问,显示502,可以运行 `91 restart` 重启一下服务**
|
||||
|
||||
安装后自动注册 `91` 管理命令:
|
||||
|
||||
```bash
|
||||
91 # 打开管理菜单
|
||||
91 status # 查看状态
|
||||
91 logs # 查看日志
|
||||
91 update # 更新
|
||||
91 restart # 重启
|
||||
91 stop # 停止
|
||||
91 # 打开管理菜单
|
||||
91 status # 查看运行状态
|
||||
91 logs # 查看日志
|
||||
91 update # 更新到最新版本
|
||||
91 restart # 重启服务
|
||||
91 stop # 停止服务
|
||||
```
|
||||
|
||||
同时也保留 `video-site-91` 作为同等别名。
|
||||
> `video-site-91` 为等效别名,两者可互换使用。
|
||||
|
||||
**旧版本用户升级说明:**
|
||||
**自定义端口:**
|
||||
|
||||
如果你是在 `v0.0.2` 之前部署的项目,系统里可能还保留旧的 `91` 管理脚本。旧脚本直接运行 `91 update` 可能更新失败。先执行下面的一次性修复命令,后续再使用 `91 update` 即可:
|
||||
```bash
|
||||
FRONTEND_PORT=8080 sudo -E bash install.sh
|
||||
```
|
||||
|
||||
**旧版本升级(v0.0.2 之前):**
|
||||
|
||||
旧版脚本直接执行 `91 update` 可能失败,先执行以下修复命令:
|
||||
|
||||
```bash
|
||||
curl -fsSL https://raw.githubusercontent.com/nianzhibai/91/main/install.sh -o /tmp/install-91.sh
|
||||
sudo bash /tmp/install-91.sh update
|
||||
```
|
||||
|
||||
想换端口:
|
||||
---
|
||||
|
||||
### 方式二:Docker Compose 部署
|
||||
|
||||
**1. 准备目录**
|
||||
|
||||
```bash
|
||||
FRONTEND_PORT=8080 sudo -E bash install.sh
|
||||
mkdir -p video-site-91 && cd video-site-91
|
||||
```
|
||||
|
||||
### Docker Compose 部署
|
||||
|
||||
准备目录:
|
||||
|
||||
```bash
|
||||
mkdir -p video-site-91
|
||||
cd video-site-91
|
||||
```
|
||||
|
||||
创建 `docker-compose.yml`:
|
||||
**2. 创建 `docker-compose.yml`**
|
||||
|
||||
```yaml
|
||||
services:
|
||||
video-site-91:
|
||||
image: ghcr.io/nianzhibai/91:latest
|
||||
image: ghcr.io/nianzhibai/91:stable
|
||||
container_name: video-site-91
|
||||
ports:
|
||||
- "9191:9191"
|
||||
@@ -118,79 +121,83 @@ services:
|
||||
- ./data:/opt/video-site-91/data
|
||||
restart: unless-stopped
|
||||
```
|
||||
|
||||
启动:
|
||||
|
||||
```bash
|
||||
docker compose up -d
|
||||
```
|
||||
|
||||
部署完成后访问:
|
||||
|
||||
- 前台:`http://服务器IP:9191/`
|
||||
- 后台:`http://服务器IP:9191/admin`
|
||||
|
||||
所有配置、数据库、封面、预览、上传文件都会保存在当前目录的 `./data` 里。更新时执行:
|
||||
|
||||
创建yml文件后运行下面指令
|
||||
```bash
|
||||
docker compose pull
|
||||
docker compose up -d
|
||||
```
|
||||
|
||||
查看日志:
|
||||
如果想固定某个 Release 版本,可以改成明确的 tag,例如:
|
||||
|
||||
```bash
|
||||
docker compose logs -f
|
||||
```yaml
|
||||
image: ghcr.io/nianzhibai/91:v0.0.6
|
||||
```
|
||||
|
||||
如果只想下载仓库内置的 Compose 文件:
|
||||
或直接拉取仓库内置配置:
|
||||
|
||||
```bash
|
||||
curl -fsSL https://raw.githubusercontent.com/nianzhibai/91/main/docker-compose.yml -o docker-compose.yml
|
||||
```
|
||||
|
||||
**3. 启动**
|
||||
|
||||
```bash
|
||||
docker compose up -d
|
||||
```
|
||||
|
||||
**常用命令:**
|
||||
|
||||
```bash
|
||||
docker compose logs -f # 查看日志
|
||||
docker compose pull # 拉取最新正式版 stable 镜像
|
||||
docker compose up -d # 更新并重启
|
||||
```
|
||||
|
||||
> 所有配置、数据库、封面、预览及上传文件均保存在 `./data/` 目录下。
|
||||
|
||||
---
|
||||
|
||||
## 数据存放位置
|
||||
|
||||
一键安装脚本会把运行数据保存在宿主机:
|
||||
### 一键脚本部署
|
||||
|
||||
- `/opt/video-site-91/config.yaml`:本地配置、管理员账号、网盘凭证。
|
||||
- `/opt/video-site-91/data/video-site.db`:SQLite 数据库。
|
||||
- `/opt/video-site-91/data/previews/`:本地生成的封面和 teaser。
|
||||
| 路径 | 内容 |
|
||||
|------|------|
|
||||
| `/opt/video-site-91/config.yaml` | 配置文件、管理员账号、网盘凭证 |
|
||||
| `/opt/video-site-91/data/video-site.db` | SQLite 数据库 |
|
||||
| `/opt/video-site-91/data/previews/` | 封面图和预览片段 |
|
||||
|
||||
Docker Compose 部署会把运行数据保存在当前目录的 `./data/`:
|
||||
### Docker Compose 部署
|
||||
|
||||
- `./data/config.yaml`:本地配置、管理员账号、网盘凭证。
|
||||
- `./data/video-site.db`:SQLite 数据库。
|
||||
- `./data/previews/`:本地生成的封面和 teaser。
|
||||
- `./data/uploads/`:本地上传的视频文件。
|
||||
- `./data/spider91/`:91 爬虫本地保存的视频文件。
|
||||
| 路径 | 内容 |
|
||||
|------|------|
|
||||
| `./data/config.yaml` | 配置文件、管理员账号、网盘凭证 |
|
||||
| `./data/video-site.db` | SQLite 数据库 |
|
||||
| `./data/previews/` | 封面图和预览片段 |
|
||||
| `./data/uploads/` | 本地上传的视频文件 |
|
||||
| `./data/spider91/` | 91 爬虫抓取的视频文件 |
|
||||
|
||||
---
|
||||
|
||||
## 了解更多
|
||||
## 更多文档
|
||||
|
||||
根目录 README 只保留项目介绍和最短上手路径。更细的实现、接口、网盘字段和部署方式可以看:
|
||||
|
||||
- [backend/README.md](backend/README.md)
|
||||
- [video-site-implementation-plan.md](video-site-implementation-plan.md)
|
||||
| 文档 | 内容 |
|
||||
|------|------|
|
||||
| [backend/README.md](backend/README.md) | 后端实现、接口说明、网盘字段 |
|
||||
| [video-site-implementation-plan.md](video-site-implementation-plan.md) | 完整实现方案 |
|
||||
|
||||
---
|
||||
|
||||
## 使用边界
|
||||
## 使用须知
|
||||
|
||||
这个项目面向个人私有部署。请只接入你有权访问和管理的内容,并遵守对应网盘、站点服务条款以及所在地法律法规。
|
||||
本项目面向**个人私有部署**,请仅接入你有权访问和管理的内容,并遵守对应网盘、站点的服务条款及所在地法律法规。
|
||||
|
||||
不要传播,仅限个人使用,个人视频站。
|
||||
> 不对外传播,仅限个人使用。
|
||||
|
||||
---
|
||||
|
||||
## 致谢
|
||||
|
||||
感谢开源项目 OpenList。
|
||||
|
||||
感谢 <a href="https://linux.do/">LinuxDo</a> 社区,学 AI 上 L 站。
|
||||
|
||||
感谢 <a href="https://nodeseek.com/">NodeSeek</a> 社区,MJJ 上 N 站。
|
||||
- [OpenList](https://github.com/OpenListTeam/OpenList) — 优秀的开源项目
|
||||
- [LinuxDo](https://linux.do/) — 学 AI 上 L 站
|
||||
- [NodeSeek](https://nodeseek.com/) — MJJ 上 N 站
|
||||
|
||||
+16
-7
@@ -2,7 +2,7 @@
|
||||
|
||||
视频聚合站的 Go 后端。提供三件事:
|
||||
|
||||
1. 多家网盘统一抽象(夸克 / 115 / PikPak / 联通沃盘 / OneDrive / 本地存储)
|
||||
1. 多家网盘统一抽象(夸克 / 115 / PikPak / 联通沃盘 / OneDrive / Google Drive / 本地存储)
|
||||
2. 视频元数据目录(SQLite)+ 扫描 + teaser 预生成
|
||||
3. REST API(前台)+ 管理后台 + 直链代理
|
||||
4. 标签池、视频隐藏、按网盘统计和详情页来源网盘类型展示能力
|
||||
@@ -21,6 +21,7 @@ internal/
|
||||
pikpak/ PikPak(自己实现,参考 OpenList pikpak)
|
||||
wopan/ 联通沃盘(壳子 + OpenListTeam/wopan-sdk-go)
|
||||
onedrive/ OneDrive(OpenList 在线续期 + Microsoft Graph 文件接口)
|
||||
googledrive/ Google Drive(OpenList 在线续期 + Google Drive API;播放走后端代理)
|
||||
localstorage/ 本地目录扫描(服务器已有视频目录)
|
||||
scanner/ 扫目录 → 落库
|
||||
preview/ ffmpeg 抽封面和生成多段 teaser
|
||||
@@ -92,7 +93,6 @@ go run ./cmd/server 后端 9192
|
||||
"kind": "quark",
|
||||
"name": "我的夸克盘",
|
||||
"rootId": "0",
|
||||
"scanRootId": "0",
|
||||
"credentials": {
|
||||
"cookie": "粘贴浏览器 F12 复制的 pan.quark.cn Cookie"
|
||||
}
|
||||
@@ -109,6 +109,7 @@ go run ./cmd/server 后端 9192
|
||||
| pikpak | `username`、`password`(token、验证码和设备 ID 由服务端自动处理并保存) |
|
||||
| wopan | `access_token`、`refresh_token`,可选 `family_id` |
|
||||
| onedrive | `refresh_token` |
|
||||
| googledrive | `refresh_token` |
|
||||
| localstorage | `path`(服务器上的已有视频目录,如 `/mnt/videos`) |
|
||||
|
||||
### PikPak 速度说明
|
||||
@@ -119,16 +120,24 @@ go run ./cmd/server 后端 9192
|
||||
|
||||
OneDrive 按 OpenList 默认应用方式调用 `https://api.oplist.org/onedrive/renewapi` 在线刷新 token,不需要配置 Azure 应用的 `client_id` / `client_secret` / `redirect_uri`。后台新建 OneDrive 时只需要填 OpenList 代刷得到的 `refresh_token`;服务端会默认挂载根目录并自动回写新 token。
|
||||
|
||||
Google Drive 按 OpenList 在线 API 调用 `https://api.oplist.org/googleui/renewapi` 刷新 token。后台新建 Google Drive 时只需要填 OpenList Google Drive 获取到的 `refresh_token`。Google Drive 下载地址必须携带 `Authorization` 头,浏览器不能直接 302 使用,所以本站会由后端代理 `/p/stream` 播放,不加入零带宽 302 白名单。
|
||||
|
||||
## 文件名约定
|
||||
|
||||
扫描器按以下顺序解析文件名:
|
||||
扫描器按以下顺序解析文件名,用于提取标题和作者:
|
||||
|
||||
1. `[tag1,tag2] 标题 - 作者.mp4`
|
||||
2. `[tag1,tag2] 标题.mp4`
|
||||
1. `[前缀] 标题 - 作者.mp4`
|
||||
2. `[前缀] 标题.mp4`
|
||||
3. `标题 - 作者.mp4`
|
||||
4. `标题.mp4`
|
||||
|
||||
标签分隔符支持 `, , 、` 和空格。解析结果会和系统标签池匹配,常见番号类噪声会归并到 `AV` 等系统标签,避免把每个番号都变成独立标签。解析结果可在管理后台覆盖。
|
||||
开头的 `[前缀]` 只会从标题里剥离,不会按分隔符作为任意标签入库。视频标签来自三类规则:
|
||||
|
||||
1. 文件名、作者和目录名命中系统标签或已有标签的标签名 / 别名。
|
||||
2. 符合条件的目录名会自动创建 `collection` 合集标签,并给同目录视频打上该标签。
|
||||
3. 常见番号类噪声会统一归并到 `AV`,避免把每个番号都变成独立标签。
|
||||
|
||||
当前内置系统标签为:`后入`、`奶子`、`口交`、`臀`、`人妻`、`女大`、`AV`。解析结果可在管理后台覆盖;手动保存后,该视频会标记为人工标签,后续扫描不会再自动覆盖。
|
||||
|
||||
## 视频去重
|
||||
|
||||
@@ -146,7 +155,7 @@ OneDrive 按 OpenList 默认应用方式调用 `https://api.oplist.org/onedrive/
|
||||
|
||||
- `/admin/drives`:新增、编辑、删除网盘,触发扫描。
|
||||
- `/admin/videos`:按网盘筛选视频,每页 100 条分页,查看各网盘 Teaser 统计,编辑标题/作者/分类/标签,单条或全量重生 teaser。
|
||||
- `/admin/tags`:新增标签并用内置规则自动匹配已有视频。
|
||||
- `/admin/tags`:新增标签并用内置规则自动匹配已有视频;删除非系统标签时会从所有视频上同步移除该标签。
|
||||
- 播放页视频信息会展示来源网盘类型;同时提供“不再展示”,点击后会把视频标记为全局隐藏。隐藏视频不会再出现在首页、列表、搜索、相关推荐和详情接口中。目前没有管理后台恢复入口,如需恢复可把数据库里对应视频的 `hidden` 字段改回 `0`。
|
||||
|
||||
## Teaser 生成
|
||||
|
||||
@@ -67,3 +67,40 @@ func TestFrontendHandlerDoesNotSwallowBackendRoutes(t *testing.T) {
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestResolveFrontendDirFallsBackToParentDist(t *testing.T) {
|
||||
workspace := t.TempDir()
|
||||
backendDir := filepath.Join(workspace, "backend")
|
||||
distDir := filepath.Join(workspace, "dist")
|
||||
if err := os.MkdirAll(backendDir, 0o755); err != nil {
|
||||
t.Fatalf("mkdir backend: %v", err)
|
||||
}
|
||||
if err := os.MkdirAll(distDir, 0o755); err != nil {
|
||||
t.Fatalf("mkdir dist: %v", err)
|
||||
}
|
||||
if err := os.WriteFile(filepath.Join(distDir, "index.html"), []byte("<html>app</html>"), 0o644); err != nil {
|
||||
t.Fatalf("write index: %v", err)
|
||||
}
|
||||
|
||||
oldWD, err := os.Getwd()
|
||||
if err != nil {
|
||||
t.Fatalf("getwd: %v", err)
|
||||
}
|
||||
t.Cleanup(func() {
|
||||
if err := os.Chdir(oldWD); err != nil {
|
||||
t.Fatalf("restore wd: %v", err)
|
||||
}
|
||||
})
|
||||
t.Setenv("VIDEO_FRONTEND_DIR", "")
|
||||
if err := os.Chdir(backendDir); err != nil {
|
||||
t.Fatalf("chdir backend: %v", err)
|
||||
}
|
||||
|
||||
got, ok := resolveFrontendDir()
|
||||
if !ok {
|
||||
t.Fatal("resolveFrontendDir ok = false, want true")
|
||||
}
|
||||
if got != "../dist" {
|
||||
t.Fatalf("frontend dir = %q, want ../dist", got)
|
||||
}
|
||||
}
|
||||
|
||||
+79
-22
@@ -25,6 +25,7 @@ import (
|
||||
"github.com/video-site/backend/internal/catalog"
|
||||
"github.com/video-site/backend/internal/config"
|
||||
"github.com/video-site/backend/internal/drives"
|
||||
"github.com/video-site/backend/internal/drives/googledrive"
|
||||
"github.com/video-site/backend/internal/drives/localstorage"
|
||||
"github.com/video-site/backend/internal/drives/localupload"
|
||||
"github.com/video-site/backend/internal/drives/onedrive"
|
||||
@@ -125,6 +126,7 @@ func main() {
|
||||
Catalog: cat,
|
||||
Auth: authr,
|
||||
VersionFilePath: versionFilePath,
|
||||
ImageVersion: strings.TrimSpace(os.Getenv("VIDEO_IMAGE_VERSION")),
|
||||
GitHubRepo: githubRepo,
|
||||
SetupRequired: func() bool {
|
||||
setupMu.Lock()
|
||||
@@ -203,10 +205,14 @@ func main() {
|
||||
SetSpider91UploadDriveID: func(id string) error {
|
||||
return app.SetSpider91UploadDriveID(ctx, id)
|
||||
},
|
||||
OnRunNightlyJob: func() {
|
||||
OnRunNightlyJob: func() bool {
|
||||
if app.nightlyRunner != nil {
|
||||
app.nightlyRunner.TriggerNow()
|
||||
return app.nightlyRunner.TriggerNow()
|
||||
}
|
||||
return false
|
||||
},
|
||||
GetNightlyJobStatus: func() api.NightlyJobStatus {
|
||||
return app.nightlyJobStatus()
|
||||
},
|
||||
ListDriveDirChildren: func(reqCtx context.Context, driveID, parentID string) ([]api.DriveDirEntry, error) {
|
||||
return app.listDriveDirChildren(reqCtx, driveID, parentID)
|
||||
@@ -415,6 +421,27 @@ func (a *App) SetSpider91UploadDriveID(ctx context.Context, driveID string) erro
|
||||
return a.cat.SetSetting(ctx, "spider91.upload_drive_id", driveID)
|
||||
}
|
||||
|
||||
func (a *App) nightlyJobStatus() api.NightlyJobStatus {
|
||||
if a.nightlyRunner == nil {
|
||||
return api.NightlyJobStatus{State: "idle"}
|
||||
}
|
||||
status := a.nightlyRunner.Status()
|
||||
return api.NightlyJobStatus{
|
||||
State: status.State,
|
||||
Running: status.Running,
|
||||
Queued: status.Queued,
|
||||
StartedAt: formatOptionalRFC3339(status.StartedAt),
|
||||
LastFinishedAt: formatOptionalRFC3339(status.LastFinishedAt),
|
||||
}
|
||||
}
|
||||
|
||||
func formatOptionalRFC3339(t time.Time) string {
|
||||
if t.IsZero() {
|
||||
return ""
|
||||
}
|
||||
return t.Format(time.RFC3339)
|
||||
}
|
||||
|
||||
// isSpider91UploadKind 是 spider91 迁移目标盘的 allowlist。
|
||||
// 与 spider91migrate.adaptUploadTarget 的支持范围保持一致。
|
||||
func isSpider91UploadKind(kind string) bool {
|
||||
@@ -633,6 +660,27 @@ func (a *App) attachDriveUnlocked(ctx context.Context, d *catalog.Drive) error {
|
||||
_ = a.cat.UpsertDrive(ctx, d)
|
||||
},
|
||||
})
|
||||
case googledrive.Kind:
|
||||
drv = googledrive.New(googledrive.Config{
|
||||
ID: d.ID,
|
||||
RootID: d.RootID,
|
||||
AccessToken: d.Credentials["access_token"],
|
||||
RefreshToken: d.Credentials["refresh_token"],
|
||||
ClientID: d.Credentials["client_id"],
|
||||
ClientSecret: d.Credentials["client_secret"],
|
||||
UseOnlineAPI: parseBoolDefault(d.Credentials["use_online_api"], true),
|
||||
RenewAPIURL: d.Credentials["api_url_address"],
|
||||
OAuthURL: d.Credentials["oauth_url"],
|
||||
APIBaseURL: d.Credentials["api_base_url"],
|
||||
OnTokenUpdate: func(access, refresh string) {
|
||||
if d.Credentials == nil {
|
||||
d.Credentials = make(map[string]string)
|
||||
}
|
||||
d.Credentials["access_token"] = access
|
||||
d.Credentials["refresh_token"] = refresh
|
||||
_ = a.cat.UpsertDrive(ctx, d)
|
||||
},
|
||||
})
|
||||
case localstorage.Kind:
|
||||
drv = localstorage.New(localstorage.Config{
|
||||
ID: d.ID,
|
||||
@@ -1000,9 +1048,8 @@ func (a *App) detachDrive(id string) {
|
||||
// listDriveDirChildren 实现 AdminServer.ListDriveDirChildren:
|
||||
// 列指定 drive 在 parentID 下的直接子目录,仅返回目录条目(IsDir=true),文件忽略。
|
||||
//
|
||||
// parentID 为空时使用 drive 实例的 RootID(),与扫描起点保持一致 —— 但有意不
|
||||
// 用 ScanRootID:用户在"设置跳过目录"弹窗里浏览的是整个网盘逻辑根,方便从 0
|
||||
// 起逐层挑跳过点;ScanRootID 仅用于实际扫描起点。
|
||||
// parentID 为空时使用 drive 实例的 RootID()。用户在"设置跳过目录"弹窗里
|
||||
// 浏览的是整个网盘逻辑根,方便从根目录起逐层挑跳过点。
|
||||
//
|
||||
// 性能优化:p115 的 Driver.List 走 SDK 的 ListWithLimit,会把目录里全部文件 +
|
||||
// 目录分页拉完才返回;某些 115 根目录累积了几万个视频,单次列目录可能卡几十
|
||||
@@ -1112,7 +1159,7 @@ func (a *App) runScan(ctx context.Context, driveID string) {
|
||||
}
|
||||
}
|
||||
|
||||
// 使用 drive 的 scan_root_id,否则 root_id;同时把 admin 配置的 SkipDirIDs
|
||||
// 扫描入口固定使用 drive 的 root_id;同时把 admin 配置的 SkipDirIDs
|
||||
// 传给 scanner(命中即不递归)。
|
||||
d, err := a.cat.GetDrive(ctx, driveID)
|
||||
if err != nil {
|
||||
@@ -1121,10 +1168,7 @@ func (a *App) runScan(ctx context.Context, driveID string) {
|
||||
}
|
||||
sc := scanner.New(a.cat, drv, a.cfg.Scanner.VideoExtensions, d.SkipDirIDs, onNew)
|
||||
|
||||
startID := d.ScanRootID
|
||||
if startID == "" {
|
||||
startID = d.RootID
|
||||
}
|
||||
startID := d.RootID
|
||||
|
||||
log.Printf("[scan] drive=%s start=%s skip_dirs=%d", driveID, startID, len(d.SkipDirIDs))
|
||||
stats, err := sc.Run(ctx, startID)
|
||||
@@ -1490,8 +1534,9 @@ func (a *App) regenFailedThumbnails(ctx context.Context, driveID string) {
|
||||
// 来判断是否真的要再生)。但既然之前是 failed 说明 url 没写过,所以这里
|
||||
// 把 url 一并清空更稳。
|
||||
if err := a.cat.UpdateVideoMeta(ctx, v.ID, catalog.VideoMetaPatch{
|
||||
ThumbnailURL: "",
|
||||
ThumbnailStatus: "pending",
|
||||
ThumbnailURL: "",
|
||||
ThumbnailStatus: "pending",
|
||||
ResetThumbnailFailures: true,
|
||||
}); err != nil {
|
||||
log.Printf("[thumb] reset failed video %s drive=%s: %v", v.ID, driveID, err)
|
||||
continue
|
||||
@@ -1742,22 +1787,34 @@ func corsMiddleware(allowedOrigins []string) func(http.Handler) http.Handler {
|
||||
}
|
||||
|
||||
func mountFrontend(r chi.Router) {
|
||||
dir := strings.TrimSpace(os.Getenv("VIDEO_FRONTEND_DIR"))
|
||||
if dir == "" {
|
||||
dir = "./dist"
|
||||
}
|
||||
info, err := os.Stat(dir)
|
||||
if err != nil || !info.IsDir() {
|
||||
return
|
||||
}
|
||||
indexPath := filepath.Join(dir, "index.html")
|
||||
if st, err := os.Stat(indexPath); err != nil || st.IsDir() {
|
||||
dir, ok := resolveFrontendDir()
|
||||
if !ok {
|
||||
return
|
||||
}
|
||||
log.Printf("serving frontend from %s", dir)
|
||||
r.NotFound(frontendHandler(dir))
|
||||
}
|
||||
|
||||
func resolveFrontendDir() (string, bool) {
|
||||
candidates := []string{}
|
||||
if dir := strings.TrimSpace(os.Getenv("VIDEO_FRONTEND_DIR")); dir != "" {
|
||||
candidates = append(candidates, dir)
|
||||
} else {
|
||||
candidates = append(candidates, "./dist", "../dist")
|
||||
}
|
||||
for _, dir := range candidates {
|
||||
info, err := os.Stat(dir)
|
||||
if err != nil || !info.IsDir() {
|
||||
continue
|
||||
}
|
||||
indexPath := filepath.Join(dir, "index.html")
|
||||
if st, err := os.Stat(indexPath); err == nil && !st.IsDir() {
|
||||
return dir, true
|
||||
}
|
||||
}
|
||||
return "", false
|
||||
}
|
||||
|
||||
func frontendHandler(dir string) http.HandlerFunc {
|
||||
return func(w http.ResponseWriter, r *http.Request) {
|
||||
if r.Method != http.MethodGet && r.Method != http.MethodHead {
|
||||
|
||||
@@ -59,7 +59,7 @@ preview:
|
||||
width: 480
|
||||
|
||||
# 盘列表。上线后请通过管理后台添加,本文件可留空。
|
||||
# kind 支持 quark / p115 / pikpak / wopan / onedrive / localstorage。
|
||||
# kind 支持 quark / p115 / pikpak / wopan / onedrive / googledrive / localstorage。
|
||||
# OneDrive 示例:
|
||||
# - id: "my-onedrive"
|
||||
# kind: "onedrive"
|
||||
@@ -67,12 +67,18 @@ preview:
|
||||
# root_id: "root"
|
||||
# params:
|
||||
# refresh_token: "..."
|
||||
# Google Drive 示例:
|
||||
# - id: "my-google"
|
||||
# kind: "googledrive"
|
||||
# name: "我的 Google Drive"
|
||||
# root_id: "root"
|
||||
# params:
|
||||
# refresh_token: "..."
|
||||
# 本地存储示例:
|
||||
# - id: "local-media"
|
||||
# kind: "localstorage"
|
||||
# name: "本地视频目录"
|
||||
# root_id: "/"
|
||||
# scan_root_id: "/"
|
||||
# params:
|
||||
# path: "/mnt/videos"
|
||||
drives: []
|
||||
|
||||
+102
-39
@@ -23,6 +23,10 @@ type AdminServer struct {
|
||||
Auth *auth.Authenticator
|
||||
// VersionFilePath points to the installer-written .version file.
|
||||
VersionFilePath string
|
||||
// ImageVersion is the Docker image version injected at build/runtime.
|
||||
// It takes precedence over VersionFilePath because Docker data volumes can
|
||||
// keep an older .version file across image upgrades.
|
||||
ImageVersion string
|
||||
// GitHubRepo is the owner/name repo used for update checks.
|
||||
GitHubRepo string
|
||||
// ReleaseAPIURL and HTTPClient are injectable for tests. Production code leaves them empty.
|
||||
@@ -55,8 +59,10 @@ type AdminServer struct {
|
||||
SetSpider91UploadDriveID func(driveID string) error
|
||||
// OnRunNightlyJob 触发一次完整的凌晨流水线(Phase1 扫盘 + Phase2 91 爬虫 +
|
||||
// Phase3 迁移)。立即返回 —— 实际任务在后台跑,admin 在日志或下次状态查询里
|
||||
// 看进度。若流水线正在跑,Runner 最多保留一个待触发请求,当前轮结束后再跑一轮。
|
||||
OnRunNightlyJob func()
|
||||
// 看进度。若流水线正在跑或已排队,Runner 会拒绝重复触发。
|
||||
OnRunNightlyJob func() bool
|
||||
// GetNightlyJobStatus 返回凌晨流水线当前状态,用于前端禁用重复触发按钮。
|
||||
GetNightlyJobStatus func() NightlyJobStatus
|
||||
// ListDriveDirChildren 列出某个 drive 在 parentID 目录下的直接子目录。
|
||||
// parentID 为空时使用 drive 的 RootID。返回 (子目录列表, error)。
|
||||
// 用于"设置跳过目录"弹窗按需展开浏览网盘目录树;只返回目录条目,文件忽略。
|
||||
@@ -83,6 +89,14 @@ type DriveGenerationStatuses struct {
|
||||
Fingerprint GenerationStatus `json:"fingerprint"`
|
||||
}
|
||||
|
||||
type NightlyJobStatus struct {
|
||||
State string `json:"state"`
|
||||
Running bool `json:"running"`
|
||||
Queued bool `json:"queued"`
|
||||
StartedAt string `json:"startedAt,omitempty"`
|
||||
LastFinishedAt string `json:"lastFinishedAt,omitempty"`
|
||||
}
|
||||
|
||||
func (a *AdminServer) Register(r chi.Router) {
|
||||
r.Route("/admin/api", func(r chi.Router) {
|
||||
// 登录、登出和首次部署初始化不需要鉴权
|
||||
@@ -117,6 +131,7 @@ func (a *AdminServer) Register(r chi.Router) {
|
||||
// 标签
|
||||
r.Get("/tags", a.handleListTags)
|
||||
r.Post("/tags", a.handleCreateTag)
|
||||
r.Delete("/tags/{id}", a.handleDeleteTag)
|
||||
|
||||
// 运行时设置
|
||||
r.Get("/settings", a.handleGetSettings)
|
||||
@@ -124,6 +139,7 @@ func (a *AdminServer) Register(r chi.Router) {
|
||||
|
||||
// 运维任务
|
||||
r.Get("/update/check", a.handleCheckUpdate)
|
||||
r.Get("/jobs/nightly/status", a.handleNightlyJobStatus)
|
||||
r.Post("/jobs/nightly/run", a.handleRunNightlyJob)
|
||||
})
|
||||
})
|
||||
@@ -279,6 +295,9 @@ func (a *AdminServer) checkUpdate(ctx context.Context) (updateCheckDTO, error) {
|
||||
}
|
||||
|
||||
func (a *AdminServer) installedVersion() string {
|
||||
if version := strings.TrimSpace(a.ImageVersion); version != "" {
|
||||
return version
|
||||
}
|
||||
path := strings.TrimSpace(a.VersionFilePath)
|
||||
if path == "" {
|
||||
path = ".version"
|
||||
@@ -374,19 +393,20 @@ func (a *AdminServer) handleListDrives(w http.ResponseWriter, r *http.Request) {
|
||||
SkipDirIDs []string `json:"skipDirIds"`
|
||||
// LastCrawlAt 是 spider91 上次成功爬取的 unix 秒(来自 credentials.last_crawl_at)。
|
||||
// 其它 kind 留 0;前端用它显示"上次抓取: N 小时前"。
|
||||
LastCrawlAt int64 `json:"lastCrawlAt,omitempty"`
|
||||
ThumbnailGenerationStatus GenerationStatus `json:"thumbnailGenerationStatus"`
|
||||
PreviewGenerationStatus GenerationStatus `json:"previewGenerationStatus"`
|
||||
FingerprintGenerationStatus GenerationStatus `json:"fingerprintGenerationStatus"`
|
||||
ThumbnailReadyCount int `json:"thumbnailReadyCount"`
|
||||
ThumbnailPendingCount int `json:"thumbnailPendingCount"`
|
||||
ThumbnailFailedCount int `json:"thumbnailFailedCount"`
|
||||
TeaserReadyCount int `json:"teaserReadyCount"`
|
||||
TeaserPendingCount int `json:"teaserPendingCount"`
|
||||
TeaserFailedCount int `json:"teaserFailedCount"`
|
||||
FingerprintReadyCount int `json:"fingerprintReadyCount"`
|
||||
FingerprintPendingCount int `json:"fingerprintPendingCount"`
|
||||
FingerprintFailedCount int `json:"fingerprintFailedCount"`
|
||||
LastCrawlAt int64 `json:"lastCrawlAt,omitempty"`
|
||||
ThumbnailGenerationStatus GenerationStatus `json:"thumbnailGenerationStatus"`
|
||||
PreviewGenerationStatus GenerationStatus `json:"previewGenerationStatus"`
|
||||
FingerprintGenerationStatus GenerationStatus `json:"fingerprintGenerationStatus"`
|
||||
ThumbnailReadyCount int `json:"thumbnailReadyCount"`
|
||||
ThumbnailPendingCount int `json:"thumbnailPendingCount"`
|
||||
ThumbnailFailedCount int `json:"thumbnailFailedCount"`
|
||||
ThumbnailDurationPendingCount int `json:"thumbnailDurationPendingCount"`
|
||||
TeaserReadyCount int `json:"teaserReadyCount"`
|
||||
TeaserPendingCount int `json:"teaserPendingCount"`
|
||||
TeaserFailedCount int `json:"teaserFailedCount"`
|
||||
FingerprintReadyCount int `json:"fingerprintReadyCount"`
|
||||
FingerprintPendingCount int `json:"fingerprintPendingCount"`
|
||||
FingerprintFailedCount int `json:"fingerprintFailedCount"`
|
||||
}
|
||||
list := make([]out, 0, len(drives))
|
||||
for _, d := range drives {
|
||||
@@ -428,32 +448,34 @@ func (a *AdminServer) handleListDrives(w http.ResponseWriter, r *http.Request) {
|
||||
ID: d.ID, Kind: d.Kind, Name: d.Name,
|
||||
RootID: d.RootID, ScanRootID: d.ScanRootID,
|
||||
Status: d.Status, LastError: d.LastError,
|
||||
HasCredential: hasCred,
|
||||
TeaserEnabled: d.TeaserEnabled,
|
||||
SkipDirIDs: append([]string{}, d.SkipDirIDs...),
|
||||
LastCrawlAt: lastCrawlAt,
|
||||
ThumbnailGenerationStatus: generation.Thumbnail,
|
||||
PreviewGenerationStatus: generation.Preview,
|
||||
FingerprintGenerationStatus: generation.Fingerprint,
|
||||
ThumbnailReadyCount: thumbCounts.Ready,
|
||||
ThumbnailPendingCount: thumbCounts.Pending,
|
||||
ThumbnailFailedCount: thumbCounts.Failed,
|
||||
TeaserReadyCount: counts.Ready,
|
||||
TeaserPendingCount: counts.Pending,
|
||||
TeaserFailedCount: counts.Failed,
|
||||
FingerprintReadyCount: fingerprintCount.Ready,
|
||||
FingerprintPendingCount: fingerprintCount.Pending,
|
||||
FingerprintFailedCount: fingerprintCount.Failed,
|
||||
HasCredential: hasCred,
|
||||
TeaserEnabled: d.TeaserEnabled,
|
||||
SkipDirIDs: append([]string{}, d.SkipDirIDs...),
|
||||
LastCrawlAt: lastCrawlAt,
|
||||
ThumbnailGenerationStatus: generation.Thumbnail,
|
||||
PreviewGenerationStatus: generation.Preview,
|
||||
FingerprintGenerationStatus: generation.Fingerprint,
|
||||
ThumbnailReadyCount: thumbCounts.Ready,
|
||||
ThumbnailPendingCount: thumbCounts.Pending,
|
||||
ThumbnailFailedCount: thumbCounts.Failed,
|
||||
ThumbnailDurationPendingCount: thumbCounts.DurationPending,
|
||||
TeaserReadyCount: counts.Ready,
|
||||
TeaserPendingCount: counts.Pending,
|
||||
TeaserFailedCount: counts.Failed,
|
||||
FingerprintReadyCount: fingerprintCount.Ready,
|
||||
FingerprintPendingCount: fingerprintCount.Pending,
|
||||
FingerprintFailedCount: fingerprintCount.Failed,
|
||||
})
|
||||
}
|
||||
writeJSON(w, http.StatusOK, list)
|
||||
}
|
||||
|
||||
type upsertDriveReq struct {
|
||||
ID string `json:"id"`
|
||||
Kind string `json:"kind"`
|
||||
Name string `json:"name"`
|
||||
RootID string `json:"rootId"`
|
||||
ID string `json:"id"`
|
||||
Kind string `json:"kind"`
|
||||
Name string `json:"name"`
|
||||
RootID string `json:"rootId"`
|
||||
// Deprecated: 扫描起点已固定为 rootId;保留字段只为兼容旧客户端请求体。
|
||||
ScanRootID string `json:"scanRootId"`
|
||||
Credentials map[string]string `json:"credentials"`
|
||||
// TeaserEnabled 是 per-drive teaser/封面生成开关。
|
||||
@@ -511,7 +533,7 @@ func (a *AdminServer) handleUpsertDrive(w http.ResponseWriter, r *http.Request)
|
||||
|
||||
d := &catalog.Drive{
|
||||
ID: body.ID, Kind: body.Kind, Name: body.Name,
|
||||
RootID: body.RootID, ScanRootID: body.ScanRootID,
|
||||
RootID: body.RootID,
|
||||
Credentials: body.Credentials,
|
||||
Status: "disconnected",
|
||||
TeaserEnabled: teaserEnabled,
|
||||
@@ -552,12 +574,32 @@ func (a *AdminServer) handleRescan(w http.ResponseWriter, r *http.Request) {
|
||||
|
||||
// handleRunNightlyJob 触发一次完整的凌晨流水线(不论当前时间,不论今日是否已跑)。
|
||||
// 立即返回 202;进度通过 backend 日志和下次 GET /admin/api/drives 的状态变化观察。
|
||||
// 流水线已在跑时 Runner 最多排队一个后续触发;如果已有待触发请求,新的点击会被忽略。
|
||||
// 流水线已在跑或已排队时,Runner 会拒绝重复触发。
|
||||
func (a *AdminServer) handleRunNightlyJob(w http.ResponseWriter, r *http.Request) {
|
||||
accepted := false
|
||||
if a.OnRunNightlyJob != nil {
|
||||
a.OnRunNightlyJob()
|
||||
accepted = a.OnRunNightlyJob()
|
||||
}
|
||||
writeJSON(w, http.StatusAccepted, map[string]any{"ok": true})
|
||||
writeJSON(w, http.StatusAccepted, map[string]any{
|
||||
"ok": true,
|
||||
"accepted": accepted,
|
||||
"status": a.nightlyJobStatus(),
|
||||
})
|
||||
}
|
||||
|
||||
func (a *AdminServer) handleNightlyJobStatus(w http.ResponseWriter, r *http.Request) {
|
||||
writeJSON(w, http.StatusOK, a.nightlyJobStatus())
|
||||
}
|
||||
|
||||
func (a *AdminServer) nightlyJobStatus() NightlyJobStatus {
|
||||
if a.GetNightlyJobStatus == nil {
|
||||
return NightlyJobStatus{State: "idle"}
|
||||
}
|
||||
status := a.GetNightlyJobStatus()
|
||||
if status.State == "" {
|
||||
status.State = "idle"
|
||||
}
|
||||
return status
|
||||
}
|
||||
|
||||
// teaserEnabledReq 是 POST /admin/api/drives/{id}/teaser-enabled 的入参。
|
||||
@@ -738,6 +780,27 @@ func (a *AdminServer) handleCreateTag(w http.ResponseWriter, r *http.Request) {
|
||||
})
|
||||
}
|
||||
|
||||
func (a *AdminServer) handleDeleteTag(w http.ResponseWriter, r *http.Request) {
|
||||
id, err := strconv.ParseInt(chi.URLParam(r, "id"), 10, 64)
|
||||
if err != nil || id <= 0 {
|
||||
writeErr(w, http.StatusBadRequest, errors.New("invalid tag id"))
|
||||
return
|
||||
}
|
||||
removedVideos, err := a.Catalog.DeleteTag(r.Context(), id)
|
||||
if err != nil {
|
||||
switch {
|
||||
case errors.Is(err, sql.ErrNoRows):
|
||||
writeErr(w, http.StatusNotFound, err)
|
||||
case errors.Is(err, catalog.ErrSystemTag):
|
||||
writeErr(w, http.StatusBadRequest, err)
|
||||
default:
|
||||
writeErr(w, http.StatusInternalServerError, err)
|
||||
}
|
||||
return
|
||||
}
|
||||
writeJSON(w, http.StatusOK, map[string]any{"ok": true, "removedVideos": removedVideos})
|
||||
}
|
||||
|
||||
type updateVideoReq struct {
|
||||
Title string `json:"title"`
|
||||
Author string `json:"author"`
|
||||
|
||||
@@ -8,6 +8,7 @@ import (
|
||||
"net/http/httptest"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"strconv"
|
||||
"strings"
|
||||
"testing"
|
||||
"time"
|
||||
@@ -194,6 +195,106 @@ func TestHandleCheckUpdateReportsUpToDate(t *testing.T) {
|
||||
}
|
||||
}
|
||||
|
||||
func TestHandleCheckUpdateUsesDockerImageVersion(t *testing.T) {
|
||||
releaseServer := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||
writeJSON(w, http.StatusOK, map[string]any{
|
||||
"tag_name": "v0.2.0",
|
||||
"html_url": "https://github.com/nianzhibai/91/releases/tag/v0.2.0",
|
||||
})
|
||||
}))
|
||||
t.Cleanup(releaseServer.Close)
|
||||
|
||||
req := httptest.NewRequest(http.MethodGet, "/admin/api/update/check", nil)
|
||||
rr := httptest.NewRecorder()
|
||||
(&AdminServer{
|
||||
ImageVersion: "v0.1.0",
|
||||
ReleaseAPIURL: releaseServer.URL,
|
||||
}).handleCheckUpdate(rr, req)
|
||||
|
||||
if rr.Code != http.StatusOK {
|
||||
t.Fatalf("status = %d, body = %s", rr.Code, rr.Body.String())
|
||||
}
|
||||
var got updateCheckDTO
|
||||
if err := json.NewDecoder(rr.Body).Decode(&got); err != nil {
|
||||
t.Fatalf("decode: %v", err)
|
||||
}
|
||||
if got.CurrentVersion != "v0.1.0" {
|
||||
t.Fatalf("currentVersion = %q, want v0.1.0", got.CurrentVersion)
|
||||
}
|
||||
if !got.HasUpdate {
|
||||
t.Fatalf("hasUpdate = false, want true")
|
||||
}
|
||||
}
|
||||
|
||||
func TestInstalledVersionPrefersDockerImageVersionOverVersionFile(t *testing.T) {
|
||||
dir := t.TempDir()
|
||||
versionFile := filepath.Join(dir, ".version")
|
||||
if err := os.WriteFile(versionFile, []byte("v0.1.0\n"), 0o644); err != nil {
|
||||
t.Fatalf("write version file: %v", err)
|
||||
}
|
||||
|
||||
got := (&AdminServer{
|
||||
VersionFilePath: versionFile,
|
||||
ImageVersion: "v0.2.0",
|
||||
}).installedVersion()
|
||||
|
||||
if got != "v0.2.0" {
|
||||
t.Fatalf("installedVersion = %q, want v0.2.0", got)
|
||||
}
|
||||
}
|
||||
|
||||
func TestHandleRunNightlyJobReturnsAcceptedStatus(t *testing.T) {
|
||||
called := false
|
||||
req := httptest.NewRequest(http.MethodPost, "/admin/api/jobs/nightly/run", nil)
|
||||
rr := httptest.NewRecorder()
|
||||
|
||||
(&AdminServer{
|
||||
OnRunNightlyJob: func() bool {
|
||||
called = true
|
||||
return true
|
||||
},
|
||||
GetNightlyJobStatus: func() NightlyJobStatus {
|
||||
return NightlyJobStatus{State: "queued", Queued: true}
|
||||
},
|
||||
}).handleRunNightlyJob(rr, req)
|
||||
|
||||
if rr.Code != http.StatusAccepted {
|
||||
t.Fatalf("status = %d, want 202; body = %s", rr.Code, rr.Body.String())
|
||||
}
|
||||
if !called {
|
||||
t.Fatal("OnRunNightlyJob was not called")
|
||||
}
|
||||
var got struct {
|
||||
OK bool `json:"ok"`
|
||||
Accepted bool `json:"accepted"`
|
||||
Status NightlyJobStatus `json:"status"`
|
||||
}
|
||||
if err := json.NewDecoder(rr.Body).Decode(&got); err != nil {
|
||||
t.Fatalf("decode: %v", err)
|
||||
}
|
||||
if !got.OK || !got.Accepted || got.Status.State != "queued" || !got.Status.Queued {
|
||||
t.Fatalf("response = %#v, want accepted queued status", got)
|
||||
}
|
||||
}
|
||||
|
||||
func TestHandleNightlyJobStatusDefaultsToIdle(t *testing.T) {
|
||||
req := httptest.NewRequest(http.MethodGet, "/admin/api/jobs/nightly/status", nil)
|
||||
rr := httptest.NewRecorder()
|
||||
|
||||
(&AdminServer{}).handleNightlyJobStatus(rr, req)
|
||||
|
||||
if rr.Code != http.StatusOK {
|
||||
t.Fatalf("status = %d, want 200; body = %s", rr.Code, rr.Body.String())
|
||||
}
|
||||
var got NightlyJobStatus
|
||||
if err := json.NewDecoder(rr.Body).Decode(&got); err != nil {
|
||||
t.Fatalf("decode: %v", err)
|
||||
}
|
||||
if got.State != "idle" || got.Running || got.Queued {
|
||||
t.Fatalf("status = %#v, want idle", got)
|
||||
}
|
||||
}
|
||||
|
||||
func TestHandleUpsertDrivePreservesExistingCredentialsWhenRequestCredentialsEmpty(t *testing.T) {
|
||||
ctx := context.Background()
|
||||
cat, err := catalog.Open(t.TempDir() + "/catalog.db")
|
||||
@@ -242,14 +343,52 @@ func TestHandleUpsertDrivePreservesExistingCredentialsWhenRequestCredentialsEmpt
|
||||
if got.Name != "New name" {
|
||||
t.Fatalf("name = %q, want New name", got.Name)
|
||||
}
|
||||
if got.ScanRootID != "scan-root" {
|
||||
t.Fatalf("scanRootId = %q, want scan-root", got.ScanRootID)
|
||||
if got.ScanRootID != "0" {
|
||||
t.Fatalf("scanRootId = %q, want rootId 0", got.ScanRootID)
|
||||
}
|
||||
if got.Credentials["cookie"] != "existing-cookie" {
|
||||
t.Fatalf("cookie credential = %q, want existing-cookie", got.Credentials["cookie"])
|
||||
}
|
||||
}
|
||||
|
||||
func TestHandleUpsertDriveDefaultsEmptyRootID(t *testing.T) {
|
||||
ctx := context.Background()
|
||||
cat, err := catalog.Open(t.TempDir() + "/catalog.db")
|
||||
if err != nil {
|
||||
t.Fatalf("open catalog: %v", err)
|
||||
}
|
||||
t.Cleanup(func() {
|
||||
if err := cat.Close(); err != nil {
|
||||
t.Fatalf("close catalog: %v", err)
|
||||
}
|
||||
})
|
||||
|
||||
req := httptest.NewRequest(http.MethodPost, "/admin/api/drives", strings.NewReader(`{
|
||||
"id": "onedrive-main",
|
||||
"kind": "onedrive",
|
||||
"name": "OneDrive",
|
||||
"rootId": "",
|
||||
"credentials": {"refresh_token": "token"}
|
||||
}`))
|
||||
rr := httptest.NewRecorder()
|
||||
|
||||
(&AdminServer{Catalog: cat}).handleUpsertDrive(rr, req)
|
||||
|
||||
if rr.Code != http.StatusOK {
|
||||
t.Fatalf("status = %d, body = %s", rr.Code, rr.Body.String())
|
||||
}
|
||||
got, err := cat.GetDrive(ctx, "onedrive-main")
|
||||
if err != nil {
|
||||
t.Fatalf("get drive: %v", err)
|
||||
}
|
||||
if got.RootID != "root" {
|
||||
t.Fatalf("rootId = %q, want root", got.RootID)
|
||||
}
|
||||
if got.ScanRootID != got.RootID {
|
||||
t.Fatalf("scanRootId = %q, want rootId %q", got.ScanRootID, got.RootID)
|
||||
}
|
||||
}
|
||||
|
||||
func TestHandleUpsertDriveReplacesExistingCredentialsWhenProvided(t *testing.T) {
|
||||
ctx := context.Background()
|
||||
cat, err := catalog.Open(t.TempDir() + "/catalog.db")
|
||||
@@ -363,64 +502,68 @@ func TestHandleListDrivesIncludesTeaserCounts(t *testing.T) {
|
||||
t.Fatalf("status = %d, body = %s", rr.Code, rr.Body.String())
|
||||
}
|
||||
var got []struct {
|
||||
ID string `json:"id"`
|
||||
ThumbnailGenerationStatus GenerationStatus `json:"thumbnailGenerationStatus"`
|
||||
PreviewGenerationStatus GenerationStatus `json:"previewGenerationStatus"`
|
||||
FingerprintGenerationStatus GenerationStatus `json:"fingerprintGenerationStatus"`
|
||||
ThumbnailReadyCount int `json:"thumbnailReadyCount"`
|
||||
ThumbnailPendingCount int `json:"thumbnailPendingCount"`
|
||||
ThumbnailFailedCount int `json:"thumbnailFailedCount"`
|
||||
TeaserReadyCount int `json:"teaserReadyCount"`
|
||||
TeaserPendingCount int `json:"teaserPendingCount"`
|
||||
TeaserFailedCount int `json:"teaserFailedCount"`
|
||||
FingerprintReadyCount int `json:"fingerprintReadyCount"`
|
||||
FingerprintPendingCount int `json:"fingerprintPendingCount"`
|
||||
FingerprintFailedCount int `json:"fingerprintFailedCount"`
|
||||
ID string `json:"id"`
|
||||
ThumbnailGenerationStatus GenerationStatus `json:"thumbnailGenerationStatus"`
|
||||
PreviewGenerationStatus GenerationStatus `json:"previewGenerationStatus"`
|
||||
FingerprintGenerationStatus GenerationStatus `json:"fingerprintGenerationStatus"`
|
||||
ThumbnailReadyCount int `json:"thumbnailReadyCount"`
|
||||
ThumbnailPendingCount int `json:"thumbnailPendingCount"`
|
||||
ThumbnailFailedCount int `json:"thumbnailFailedCount"`
|
||||
ThumbnailDurationPendingCount int `json:"thumbnailDurationPendingCount"`
|
||||
TeaserReadyCount int `json:"teaserReadyCount"`
|
||||
TeaserPendingCount int `json:"teaserPendingCount"`
|
||||
TeaserFailedCount int `json:"teaserFailedCount"`
|
||||
FingerprintReadyCount int `json:"fingerprintReadyCount"`
|
||||
FingerprintPendingCount int `json:"fingerprintPendingCount"`
|
||||
FingerprintFailedCount int `json:"fingerprintFailedCount"`
|
||||
}
|
||||
if err := json.NewDecoder(rr.Body).Decode(&got); err != nil {
|
||||
t.Fatalf("decode: %v", err)
|
||||
}
|
||||
byID := map[string]struct {
|
||||
TeaserReady int
|
||||
TeaserPending int
|
||||
TeaserFailed int
|
||||
ThumbnailReady int
|
||||
ThumbnailPending int
|
||||
ThumbnailFailed int
|
||||
FingerprintReady int
|
||||
FingerprintPending int
|
||||
FingerprintFailed int
|
||||
Thumbnail GenerationStatus
|
||||
Preview GenerationStatus
|
||||
Fingerprint GenerationStatus
|
||||
TeaserReady int
|
||||
TeaserPending int
|
||||
TeaserFailed int
|
||||
ThumbnailReady int
|
||||
ThumbnailPending int
|
||||
ThumbnailFailed int
|
||||
ThumbnailDurationPending int
|
||||
FingerprintReady int
|
||||
FingerprintPending int
|
||||
FingerprintFailed int
|
||||
Thumbnail GenerationStatus
|
||||
Preview GenerationStatus
|
||||
Fingerprint GenerationStatus
|
||||
}{}
|
||||
for _, d := range got {
|
||||
byID[d.ID] = struct {
|
||||
TeaserReady int
|
||||
TeaserPending int
|
||||
TeaserFailed int
|
||||
ThumbnailReady int
|
||||
ThumbnailPending int
|
||||
ThumbnailFailed int
|
||||
FingerprintReady int
|
||||
FingerprintPending int
|
||||
FingerprintFailed int
|
||||
Thumbnail GenerationStatus
|
||||
Preview GenerationStatus
|
||||
Fingerprint GenerationStatus
|
||||
TeaserReady int
|
||||
TeaserPending int
|
||||
TeaserFailed int
|
||||
ThumbnailReady int
|
||||
ThumbnailPending int
|
||||
ThumbnailFailed int
|
||||
ThumbnailDurationPending int
|
||||
FingerprintReady int
|
||||
FingerprintPending int
|
||||
FingerprintFailed int
|
||||
Thumbnail GenerationStatus
|
||||
Preview GenerationStatus
|
||||
Fingerprint GenerationStatus
|
||||
}{
|
||||
TeaserReady: d.TeaserReadyCount,
|
||||
TeaserPending: d.TeaserPendingCount,
|
||||
TeaserFailed: d.TeaserFailedCount,
|
||||
ThumbnailReady: d.ThumbnailReadyCount,
|
||||
ThumbnailPending: d.ThumbnailPendingCount,
|
||||
ThumbnailFailed: d.ThumbnailFailedCount,
|
||||
FingerprintReady: d.FingerprintReadyCount,
|
||||
FingerprintPending: d.FingerprintPendingCount,
|
||||
FingerprintFailed: d.FingerprintFailedCount,
|
||||
Thumbnail: d.ThumbnailGenerationStatus,
|
||||
Preview: d.PreviewGenerationStatus,
|
||||
Fingerprint: d.FingerprintGenerationStatus,
|
||||
TeaserReady: d.TeaserReadyCount,
|
||||
TeaserPending: d.TeaserPendingCount,
|
||||
TeaserFailed: d.TeaserFailedCount,
|
||||
ThumbnailReady: d.ThumbnailReadyCount,
|
||||
ThumbnailPending: d.ThumbnailPendingCount,
|
||||
ThumbnailFailed: d.ThumbnailFailedCount,
|
||||
ThumbnailDurationPending: d.ThumbnailDurationPendingCount,
|
||||
FingerprintReady: d.FingerprintReadyCount,
|
||||
FingerprintPending: d.FingerprintPendingCount,
|
||||
FingerprintFailed: d.FingerprintFailedCount,
|
||||
Thumbnail: d.ThumbnailGenerationStatus,
|
||||
Preview: d.PreviewGenerationStatus,
|
||||
Fingerprint: d.FingerprintGenerationStatus,
|
||||
}
|
||||
}
|
||||
if byID["OneDrive"].TeaserReady != 2 || byID["OneDrive"].TeaserPending != 1 || byID["OneDrive"].TeaserFailed != 0 {
|
||||
@@ -429,6 +572,9 @@ func TestHandleListDrivesIncludesTeaserCounts(t *testing.T) {
|
||||
if byID["OneDrive"].ThumbnailReady != 1 || byID["OneDrive"].ThumbnailPending != 1 || byID["OneDrive"].ThumbnailFailed != 1 {
|
||||
t.Fatalf("OneDrive thumbnail counts = %#v, want ready=1 pending=1 failed=1", byID["OneDrive"])
|
||||
}
|
||||
if byID["OneDrive"].ThumbnailDurationPending != 1 {
|
||||
t.Fatalf("OneDrive thumbnail duration pending = %#v, want 1", byID["OneDrive"])
|
||||
}
|
||||
if byID["OneDrive"].Thumbnail.State != "cooling" || byID["OneDrive"].Preview.State != "generating" {
|
||||
t.Fatalf("OneDrive generation statuses = %#v, want thumbnail cooling and preview generating", byID["OneDrive"])
|
||||
}
|
||||
@@ -615,6 +761,64 @@ func TestHandleCreateTagClassifiesExistingVideos(t *testing.T) {
|
||||
}
|
||||
}
|
||||
|
||||
func TestHandleDeleteTagRemovesTagFromVideos(t *testing.T) {
|
||||
ctx := context.Background()
|
||||
cat, err := catalog.Open(t.TempDir() + "/catalog.db")
|
||||
if err != nil {
|
||||
t.Fatalf("open catalog: %v", err)
|
||||
}
|
||||
t.Cleanup(func() {
|
||||
if err := cat.Close(); err != nil {
|
||||
t.Fatalf("close catalog: %v", err)
|
||||
}
|
||||
})
|
||||
|
||||
now := time.Now()
|
||||
if err := cat.UpsertVideo(ctx, &catalog.Video{
|
||||
ID: "video-1",
|
||||
DriveID: "drive",
|
||||
FileID: "file-1",
|
||||
Title: "清纯短发",
|
||||
PublishedAt: now,
|
||||
CreatedAt: now,
|
||||
UpdatedAt: now,
|
||||
}); err != nil {
|
||||
t.Fatalf("seed video: %v", err)
|
||||
}
|
||||
if _, err := cat.CreateTagAndClassify(ctx, "清纯", nil, "user"); err != nil {
|
||||
t.Fatalf("create tag: %v", err)
|
||||
}
|
||||
tags, err := cat.ListTags(ctx)
|
||||
if err != nil {
|
||||
t.Fatalf("list tags: %v", err)
|
||||
}
|
||||
var tagID int64
|
||||
for _, tag := range tags {
|
||||
if tag.Label == "清纯" {
|
||||
tagID = tag.ID
|
||||
break
|
||||
}
|
||||
}
|
||||
if tagID == 0 {
|
||||
t.Fatal("created tag not found")
|
||||
}
|
||||
|
||||
req := requestWithRouteParam(http.MethodDelete, "/admin/api/tags/1", "id", strconv.FormatInt(tagID, 10), strings.NewReader(``))
|
||||
rr := httptest.NewRecorder()
|
||||
(&AdminServer{Catalog: cat}).handleDeleteTag(rr, req)
|
||||
|
||||
if rr.Code != http.StatusOK {
|
||||
t.Fatalf("status = %d, body = %s", rr.Code, rr.Body.String())
|
||||
}
|
||||
video, err := cat.GetVideo(ctx, "video-1")
|
||||
if err != nil {
|
||||
t.Fatalf("get video: %v", err)
|
||||
}
|
||||
if len(video.Tags) != 0 {
|
||||
t.Fatalf("video tags = %#v, want none", video.Tags)
|
||||
}
|
||||
}
|
||||
|
||||
func TestHandleAdminListVideosFiltersByDriveID(t *testing.T) {
|
||||
ctx := context.Background()
|
||||
cat, err := catalog.Open(t.TempDir() + "/catalog.db")
|
||||
|
||||
@@ -40,7 +40,7 @@ var allowedUploadExtensions = map[string]struct{}{
|
||||
var allowedUploadTags = map[string]struct{}{
|
||||
"奶子": {},
|
||||
"臀": {},
|
||||
"口角": {},
|
||||
"口交": {},
|
||||
"女大": {},
|
||||
"人妻": {},
|
||||
"AV": {},
|
||||
@@ -440,11 +440,12 @@ func (s *Server) handleTags(w http.ResponseWriter, r *http.Request) {
|
||||
writeJSON(w, http.StatusOK, out)
|
||||
}
|
||||
|
||||
// shortsNextReq 客户端把当前轮已看过的 video id 列表传上来,
|
||||
// 服务器从未在列表中的视频里随机抽 count 个返回。
|
||||
// shortsNextReq 客户端把当前轮已看过的 video id 列表传上来。
|
||||
// PreferredFromVideoID 来自短视频页最近一次点赞成功的视频,用于优先推荐相似标签。
|
||||
type shortsNextReq struct {
|
||||
SeenIDs []string `json:"seenIds"`
|
||||
Count int `json:"count"`
|
||||
SeenIDs []string `json:"seenIds"`
|
||||
Count int `json:"count"`
|
||||
PreferredFromVideoID string `json:"preferredFromVideoId"`
|
||||
}
|
||||
|
||||
// ShortsItemDTO 是短视频流单条的精简结构。比 VideoDTO 多 videoSrc / poster,
|
||||
@@ -490,7 +491,12 @@ func (s *Server) handleShortsNext(w http.ResponseWriter, r *http.Request) {
|
||||
exclude = nil
|
||||
}
|
||||
|
||||
items, err := s.Catalog.RandomVideosExcluding(r.Context(), exclude, count)
|
||||
var items []*catalog.Video
|
||||
if strings.TrimSpace(body.PreferredFromVideoID) != "" {
|
||||
items, err = s.Catalog.RandomVideosForPreferredVideoExcluding(r.Context(), body.PreferredFromVideoID, exclude, count)
|
||||
} else {
|
||||
items, err = s.Catalog.RandomVideosExcluding(r.Context(), exclude, count)
|
||||
}
|
||||
if err != nil {
|
||||
writeErr(w, http.StatusInternalServerError, err)
|
||||
return
|
||||
@@ -900,6 +906,8 @@ func driveKindLabel(kind string) string {
|
||||
return "联通沃盘"
|
||||
case "onedrive":
|
||||
return "OneDrive"
|
||||
case "googledrive":
|
||||
return "Google Drive"
|
||||
case localstorage.Kind:
|
||||
return "本地存储"
|
||||
case spider91.Kind:
|
||||
|
||||
@@ -261,7 +261,7 @@ func TestHandleUploadVideoSavesFileVideoTagsAndQueuesPreview(t *testing.T) {
|
||||
}
|
||||
req := multipartUploadRequest(t, map[string]string{
|
||||
"title": "用户上传标题",
|
||||
"tags": "奶子,AV,女大",
|
||||
"tags": "奶子,口交,AV,女大",
|
||||
}, "clip.mp4", "video-bytes")
|
||||
rr := httptest.NewRecorder()
|
||||
|
||||
@@ -287,7 +287,7 @@ func TestHandleUploadVideoSavesFileVideoTagsAndQueuesPreview(t *testing.T) {
|
||||
if got.Title != "用户上传标题" {
|
||||
t.Fatalf("title = %q, want submitted title", got.Title)
|
||||
}
|
||||
if !sameStringSet(got.Tags, []string{"奶子", "AV", "女大"}) {
|
||||
if !sameStringSet(got.Tags, []string{"奶子", "口交", "AV", "女大"}) {
|
||||
t.Fatalf("tags = %#v, want selected tags", got.Tags)
|
||||
}
|
||||
if got.PreviewStatus != "pending" {
|
||||
@@ -523,6 +523,66 @@ func TestHandleTagsReturnsUnifiedTagPool(t *testing.T) {
|
||||
}
|
||||
}
|
||||
|
||||
func TestHandleShortsNextUsesPreferredVideoLeastPopulatedTag(t *testing.T) {
|
||||
ctx := context.Background()
|
||||
cat, err := catalog.Open(t.TempDir() + "/catalog.db")
|
||||
if err != nil {
|
||||
t.Fatalf("open catalog: %v", err)
|
||||
}
|
||||
t.Cleanup(func() {
|
||||
if err := cat.Close(); err != nil {
|
||||
t.Fatalf("close catalog: %v", err)
|
||||
}
|
||||
})
|
||||
|
||||
now := time.Now()
|
||||
for _, v := range []*catalog.Video{
|
||||
{ID: "current", DriveID: "drive", FileID: "f-current", Title: "current", Tags: []string{"common", "rare"}, PublishedAt: now, CreatedAt: now, UpdatedAt: now},
|
||||
{ID: "common-1", DriveID: "drive", FileID: "f-common-1", Title: "common 1", Tags: []string{"common"}, PublishedAt: now, CreatedAt: now, UpdatedAt: now},
|
||||
{ID: "common-2", DriveID: "drive", FileID: "f-common-2", Title: "common 2", Tags: []string{"common"}, PublishedAt: now, CreatedAt: now, UpdatedAt: now},
|
||||
{ID: "rare-1", DriveID: "drive", FileID: "f-rare-1", Title: "rare 1", Tags: []string{"rare"}, PublishedAt: now, CreatedAt: now, UpdatedAt: now},
|
||||
} {
|
||||
if err := cat.UpsertVideo(ctx, v); err != nil {
|
||||
t.Fatalf("seed %s: %v", v.ID, err)
|
||||
}
|
||||
}
|
||||
|
||||
req := httptest.NewRequest(http.MethodPost, "/api/shorts/next", strings.NewReader(`{"seenIds":["current"],"count":3,"preferredFromVideoId":"current"}`))
|
||||
rr := httptest.NewRecorder()
|
||||
(&Server{Catalog: cat}).handleShortsNext(rr, req)
|
||||
|
||||
if rr.Code != http.StatusOK {
|
||||
t.Fatalf("status = %d, body = %s", rr.Code, rr.Body.String())
|
||||
}
|
||||
var got struct {
|
||||
Items []ShortsItemDTO `json:"items"`
|
||||
Total int `json:"total"`
|
||||
RoundComplete bool `json:"roundComplete"`
|
||||
}
|
||||
if err := json.NewDecoder(rr.Body).Decode(&got); err != nil {
|
||||
t.Fatalf("decode: %v", err)
|
||||
}
|
||||
ids := make([]string, 0, len(got.Items))
|
||||
for _, item := range got.Items {
|
||||
ids = append(ids, item.ID)
|
||||
}
|
||||
if got.Total != 4 {
|
||||
t.Fatalf("total = %d, want 4", got.Total)
|
||||
}
|
||||
if got.RoundComplete {
|
||||
t.Fatalf("roundComplete = true, want false with fallback-filled batch")
|
||||
}
|
||||
if !containsString(ids, "rare-1") {
|
||||
t.Fatalf("ids = %#v, want rare-1 from least populated tag", ids)
|
||||
}
|
||||
if containsString(ids, "current") {
|
||||
t.Fatalf("ids = %#v, should exclude current", ids)
|
||||
}
|
||||
if len(ids) != 3 {
|
||||
t.Fatalf("ids = %#v, want 3 items", ids)
|
||||
}
|
||||
}
|
||||
|
||||
func TestHandleUpdateVideoTagsRejectsUnknownTags(t *testing.T) {
|
||||
ctx := context.Background()
|
||||
cat, err := catalog.Open(t.TempDir() + "/catalog.db")
|
||||
|
||||
@@ -321,14 +321,15 @@ func (c *Catalog) IncrementView(ctx context.Context, id string) (int, error) {
|
||||
|
||||
// VideoMetaPatch 轻量更新视频元数据(仅非零值字段会被写入)
|
||||
type VideoMetaPatch struct {
|
||||
ThumbnailURL string
|
||||
ThumbnailStatus string
|
||||
DurationSeconds int
|
||||
Category string
|
||||
ContentHash string
|
||||
FileName string
|
||||
Tags []string
|
||||
TagsSet bool
|
||||
ThumbnailURL string
|
||||
ThumbnailStatus string
|
||||
ResetThumbnailFailures bool
|
||||
DurationSeconds int
|
||||
Category string
|
||||
ContentHash string
|
||||
FileName string
|
||||
Tags []string
|
||||
TagsSet bool
|
||||
}
|
||||
|
||||
func (c *Catalog) UpdateVideoMeta(ctx context.Context, id string, p VideoMetaPatch) error {
|
||||
@@ -342,8 +343,12 @@ func (c *Catalog) UpdateVideoMeta(ctx context.Context, id string, p VideoMetaPat
|
||||
case p.ThumbnailStatus != "":
|
||||
// 调用方显式指定 status —— 信任之;典型是 worker 把状态置 'failed' 或
|
||||
// 在重试时显式置 'pending'。
|
||||
status := nullableStatus(p.ThumbnailStatus)
|
||||
parts = append(parts, "thumbnail_status = ?")
|
||||
args = append(args, nullableStatus(p.ThumbnailStatus))
|
||||
args = append(args, status)
|
||||
if status == "ready" {
|
||||
p.ResetThumbnailFailures = true
|
||||
}
|
||||
case p.ThumbnailURL != "":
|
||||
// 调用方写了 url 但没显式给 status —— 视为"封面就绪"。url 非空意味着
|
||||
// 浏览器访问那个 URL 能拿到图(要么是本地 /p/thumb/<id>,要么是网盘 API
|
||||
@@ -351,6 +356,10 @@ func (c *Catalog) UpdateVideoMeta(ctx context.Context, id string, p VideoMetaPat
|
||||
// 仍是 'pending' 的脏状态(修过的历史 bug)。
|
||||
parts = append(parts, "thumbnail_status = ?")
|
||||
args = append(args, nullableStatus("ready"))
|
||||
p.ResetThumbnailFailures = true
|
||||
}
|
||||
if p.ResetThumbnailFailures {
|
||||
parts = append(parts, "thumbnail_failures = 0")
|
||||
}
|
||||
if p.DurationSeconds > 0 {
|
||||
parts = append(parts, "duration_seconds = ?")
|
||||
@@ -389,6 +398,38 @@ func (c *Catalog) UpdateVideoMeta(ctx context.Context, id string, p VideoMetaPat
|
||||
return nil
|
||||
}
|
||||
|
||||
func (c *Catalog) IncrementThumbnailFailures(ctx context.Context, id string) (int, error) {
|
||||
tx, err := c.db.BeginTx(ctx, nil)
|
||||
if err != nil {
|
||||
return 0, err
|
||||
}
|
||||
defer tx.Rollback()
|
||||
|
||||
res, err := tx.ExecContext(ctx,
|
||||
`UPDATE videos
|
||||
SET thumbnail_failures = COALESCE(thumbnail_failures, 0) + 1,
|
||||
updated_at = ?
|
||||
WHERE id = ?`,
|
||||
time.Now().UnixMilli(), id)
|
||||
if err != nil {
|
||||
return 0, err
|
||||
}
|
||||
if affected, err := res.RowsAffected(); err == nil && affected == 0 {
|
||||
return 0, sql.ErrNoRows
|
||||
}
|
||||
|
||||
var failures int
|
||||
if err := tx.QueryRowContext(ctx,
|
||||
`SELECT COALESCE(thumbnail_failures, 0) FROM videos WHERE id = ?`,
|
||||
id).Scan(&failures); err != nil {
|
||||
return 0, err
|
||||
}
|
||||
if err := tx.Commit(); err != nil {
|
||||
return 0, err
|
||||
}
|
||||
return failures, nil
|
||||
}
|
||||
|
||||
// ListCategories 聚合所有 category,按视频数降序
|
||||
type CategoryStat struct {
|
||||
Category string
|
||||
@@ -869,21 +910,7 @@ func (c *Catalog) RandomVideosExcluding(ctx context.Context, excludeIDs []string
|
||||
return nil, nil
|
||||
}
|
||||
|
||||
// 去重 excludeIDs,过滤空串
|
||||
seen := make(map[string]struct{}, len(excludeIDs))
|
||||
cleaned := make([]string, 0, len(excludeIDs))
|
||||
for _, id := range excludeIDs {
|
||||
id = strings.TrimSpace(id)
|
||||
if id == "" {
|
||||
continue
|
||||
}
|
||||
if _, ok := seen[id]; ok {
|
||||
continue
|
||||
}
|
||||
seen[id] = struct{}{}
|
||||
cleaned = append(cleaned, id)
|
||||
}
|
||||
|
||||
cleaned := cleanVideoIDs(excludeIDs)
|
||||
args := make([]any, 0, len(cleaned)+1)
|
||||
whereSQL := `WHERE COALESCE(hidden, 0) = 0
|
||||
AND ` + uniqueVideoWhereSQL
|
||||
@@ -921,6 +948,175 @@ func (c *Catalog) RandomVideosExcluding(ctx context.Context, excludeIDs []string
|
||||
return out, nil
|
||||
}
|
||||
|
||||
func cleanVideoIDs(ids []string) []string {
|
||||
seen := make(map[string]struct{}, len(ids))
|
||||
cleaned := make([]string, 0, len(ids))
|
||||
for _, id := range ids {
|
||||
id = strings.TrimSpace(id)
|
||||
if id == "" {
|
||||
continue
|
||||
}
|
||||
if _, ok := seen[id]; ok {
|
||||
continue
|
||||
}
|
||||
seen[id] = struct{}{}
|
||||
cleaned = append(cleaned, id)
|
||||
}
|
||||
return cleaned
|
||||
}
|
||||
|
||||
func cleanTagLabels(labels []string) []string {
|
||||
seen := make(map[string]struct{}, len(labels))
|
||||
cleaned := make([]string, 0, len(labels))
|
||||
for _, label := range labels {
|
||||
label = strings.TrimSpace(label)
|
||||
if label == "" {
|
||||
continue
|
||||
}
|
||||
key := strings.ToLower(label)
|
||||
if _, ok := seen[key]; ok {
|
||||
continue
|
||||
}
|
||||
seen[key] = struct{}{}
|
||||
cleaned = append(cleaned, label)
|
||||
}
|
||||
return cleaned
|
||||
}
|
||||
|
||||
func (c *Catalog) LeastPopulatedVisibleUniqueTag(ctx context.Context, labels []string) (string, error) {
|
||||
cleaned := cleanTagLabels(labels)
|
||||
bestLabel := ""
|
||||
bestCount := 0
|
||||
for _, label := range cleaned {
|
||||
var count int
|
||||
if err := c.db.QueryRowContext(ctx,
|
||||
`SELECT COUNT(*)
|
||||
FROM videos
|
||||
WHERE COALESCE(hidden, 0) = 0
|
||||
AND `+uniqueVideoWhereSQL+`
|
||||
AND EXISTS (
|
||||
SELECT 1
|
||||
FROM video_tags vt
|
||||
JOIN tags t ON t.id = vt.tag_id
|
||||
WHERE vt.video_id = videos.id
|
||||
AND t.label = ? COLLATE NOCASE
|
||||
)`,
|
||||
label,
|
||||
).Scan(&count); err != nil {
|
||||
return "", err
|
||||
}
|
||||
if count == 0 {
|
||||
continue
|
||||
}
|
||||
if bestLabel == "" || count < bestCount {
|
||||
bestLabel = label
|
||||
bestCount = count
|
||||
}
|
||||
}
|
||||
return bestLabel, nil
|
||||
}
|
||||
|
||||
func (c *Catalog) RandomVideosByTagExcluding(ctx context.Context, tag string, excludeIDs []string, limit int) ([]*Video, error) {
|
||||
if limit <= 0 {
|
||||
return nil, nil
|
||||
}
|
||||
tag = strings.TrimSpace(tag)
|
||||
if tag == "" {
|
||||
return nil, nil
|
||||
}
|
||||
|
||||
cleaned := cleanVideoIDs(excludeIDs)
|
||||
args := make([]any, 0, len(cleaned)+2)
|
||||
args = append(args, tag)
|
||||
whereSQL := `WHERE COALESCE(hidden, 0) = 0
|
||||
AND ` + uniqueVideoWhereSQL + `
|
||||
AND EXISTS (
|
||||
SELECT 1
|
||||
FROM video_tags vt
|
||||
JOIN tags t ON t.id = vt.tag_id
|
||||
WHERE vt.video_id = videos.id
|
||||
AND t.label = ? COLLATE NOCASE
|
||||
)`
|
||||
if len(cleaned) > 0 {
|
||||
placeholders := strings.Repeat("?,", len(cleaned))
|
||||
placeholders = placeholders[:len(placeholders)-1]
|
||||
whereSQL += " AND id NOT IN (" + placeholders + ")"
|
||||
for _, id := range cleaned {
|
||||
args = append(args, id)
|
||||
}
|
||||
}
|
||||
args = append(args, limit)
|
||||
|
||||
rows, err := c.db.QueryContext(ctx,
|
||||
`SELECT `+allVideoCols+` FROM videos `+whereSQL+`
|
||||
ORDER BY RANDOM() LIMIT ?`,
|
||||
args...,
|
||||
)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
defer rows.Close()
|
||||
|
||||
var out []*Video
|
||||
for rows.Next() {
|
||||
v, err := scanVideo(rows)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
out = append(out, v)
|
||||
}
|
||||
if err := rows.Err(); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return out, nil
|
||||
}
|
||||
|
||||
func (c *Catalog) RandomVideosForPreferredVideoExcluding(ctx context.Context, preferredVideoID string, excludeIDs []string, limit int) ([]*Video, error) {
|
||||
if limit <= 0 {
|
||||
return nil, nil
|
||||
}
|
||||
preferredVideoID = strings.TrimSpace(preferredVideoID)
|
||||
if preferredVideoID == "" {
|
||||
return c.RandomVideosExcluding(ctx, excludeIDs, limit)
|
||||
}
|
||||
|
||||
preferredExclude := append([]string{}, excludeIDs...)
|
||||
preferredExclude = append(preferredExclude, preferredVideoID)
|
||||
|
||||
preferred, err := c.GetVideo(ctx, preferredVideoID)
|
||||
if err != nil || preferred == nil || preferred.Hidden || len(preferred.Tags) == 0 {
|
||||
return c.RandomVideosExcluding(ctx, preferredExclude, limit)
|
||||
}
|
||||
tag, err := c.LeastPopulatedVisibleUniqueTag(ctx, preferred.Tags)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if tag == "" {
|
||||
return c.RandomVideosExcluding(ctx, preferredExclude, limit)
|
||||
}
|
||||
|
||||
items, err := c.RandomVideosByTagExcluding(ctx, tag, preferredExclude, limit)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if len(items) >= limit {
|
||||
return items, nil
|
||||
}
|
||||
|
||||
mergedExclude := make([]string, 0, len(preferredExclude)+len(items))
|
||||
mergedExclude = append(mergedExclude, preferredExclude...)
|
||||
for _, item := range items {
|
||||
if item != nil {
|
||||
mergedExclude = append(mergedExclude, item.ID)
|
||||
}
|
||||
}
|
||||
fallback, err := c.RandomVideosExcluding(ctx, mergedExclude, limit-len(items))
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return append(items, fallback...), nil
|
||||
}
|
||||
|
||||
type DriveTeaserCounts struct {
|
||||
Ready int
|
||||
Pending int
|
||||
@@ -928,9 +1124,10 @@ type DriveTeaserCounts struct {
|
||||
}
|
||||
|
||||
type DriveThumbnailCounts struct {
|
||||
Ready int
|
||||
Pending int
|
||||
Failed int
|
||||
Ready int
|
||||
Pending int
|
||||
Failed int
|
||||
DurationPending int
|
||||
}
|
||||
|
||||
type DriveFingerprintCounts struct {
|
||||
@@ -974,9 +1171,12 @@ func (c *Catalog) CountThumbnailsByDrive(ctx context.Context) (map[string]DriveT
|
||||
`SELECT drive_id,
|
||||
COUNT(CASE WHEN COALESCE(thumbnail_url, '') != '' THEN 1 END) AS ready_count,
|
||||
COUNT(CASE WHEN COALESCE(thumbnail_url, '') = ''
|
||||
AND COALESCE(thumbnail_status, 'pending') != 'failed' THEN 1 END) AS pending_count,
|
||||
AND COALESCE(thumbnail_status, 'pending') NOT IN ('failed', 'skipped') THEN 1 END) AS pending_count,
|
||||
COUNT(CASE WHEN COALESCE(thumbnail_url, '') = ''
|
||||
AND COALESCE(thumbnail_status, 'pending') = 'failed' THEN 1 END) AS failed_count
|
||||
AND COALESCE(thumbnail_status, 'pending') = 'failed' THEN 1 END) AS failed_count,
|
||||
COUNT(CASE WHEN COALESCE(thumbnail_url, '') != ''
|
||||
AND COALESCE(duration_seconds, 0) <= 0
|
||||
AND COALESCE(thumbnail_status, 'pending') NOT IN ('failed', 'skipped') THEN 1 END) AS duration_pending_count
|
||||
FROM videos
|
||||
WHERE COALESCE(hidden, 0) = 0
|
||||
AND `+uniqueVideoWhereSQL+`
|
||||
@@ -990,7 +1190,7 @@ func (c *Catalog) CountThumbnailsByDrive(ctx context.Context) (map[string]DriveT
|
||||
for rows.Next() {
|
||||
var driveID string
|
||||
var counts DriveThumbnailCounts
|
||||
if err := rows.Scan(&driveID, &counts.Ready, &counts.Pending, &counts.Failed); err != nil {
|
||||
if err := rows.Scan(&driveID, &counts.Ready, &counts.Pending, &counts.Failed, &counts.DurationPending); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
out[driveID] = counts
|
||||
@@ -1197,10 +1397,11 @@ func (c *Catalog) ClearGeneratedAssets(ctx context.Context, videoID string, clea
|
||||
// ---------- Drive ----------
|
||||
|
||||
type Drive struct {
|
||||
ID string `json:"id"`
|
||||
Kind string `json:"kind"`
|
||||
Name string `json:"name"`
|
||||
RootID string `json:"rootId"`
|
||||
ID string `json:"id"`
|
||||
Kind string `json:"kind"`
|
||||
Name string `json:"name"`
|
||||
RootID string `json:"rootId"`
|
||||
// Deprecated: 扫描入口固定等于 RootID;字段保留用于兼容旧数据/API。
|
||||
ScanRootID string `json:"scanRootId"`
|
||||
Credentials map[string]string `json:"credentials,omitempty"`
|
||||
Status string `json:"status"`
|
||||
@@ -1218,6 +1419,7 @@ type Drive struct {
|
||||
}
|
||||
|
||||
func (c *Catalog) UpsertDrive(ctx context.Context, d *Drive) error {
|
||||
normalizeDriveRootFields(d)
|
||||
cred, _ := json.Marshal(d.Credentials)
|
||||
skipDirs := d.SkipDirIDs
|
||||
if skipDirs == nil {
|
||||
@@ -1248,6 +1450,49 @@ ON CONFLICT(id) DO UPDATE SET
|
||||
return err
|
||||
}
|
||||
|
||||
func normalizeDriveRootFields(d *Drive) {
|
||||
if d == nil {
|
||||
return
|
||||
}
|
||||
d.RootID = normalizeDriveRootID(d.Kind, d.RootID)
|
||||
d.ScanRootID = d.RootID
|
||||
}
|
||||
|
||||
func normalizeDriveRootID(kind, rootID string) string {
|
||||
rootID = strings.TrimSpace(rootID)
|
||||
switch kind {
|
||||
case "pikpak":
|
||||
if rootID == "0" {
|
||||
return ""
|
||||
}
|
||||
return rootID
|
||||
case "onedrive", "googledrive":
|
||||
if rootID == "" {
|
||||
return "root"
|
||||
}
|
||||
return rootID
|
||||
case "localstorage", "spider91":
|
||||
if rootID == "" {
|
||||
return "/"
|
||||
}
|
||||
return rootID
|
||||
default:
|
||||
if rootID == "" {
|
||||
return "0"
|
||||
}
|
||||
return rootID
|
||||
}
|
||||
}
|
||||
|
||||
func (c *Catalog) syncDriveScanRootIDToRootID(ctx context.Context) error {
|
||||
_, err := c.db.ExecContext(ctx, `
|
||||
UPDATE drives
|
||||
SET scan_root_id = root_id,
|
||||
updated_at = ?
|
||||
WHERE COALESCE(scan_root_id, '') != COALESCE(root_id, '')`, time.Now().UnixMilli())
|
||||
return err
|
||||
}
|
||||
|
||||
func (c *Catalog) ListDrives(ctx context.Context) ([]*Drive, error) {
|
||||
rows, err := c.db.QueryContext(ctx, `SELECT id, kind, name, root_id, COALESCE(scan_root_id, ''), COALESCE(credentials, '{}'), status, COALESCE(last_error, ''), COALESCE(teaser_enabled, 1), COALESCE(skip_dir_ids, '[]'), created_at, updated_at FROM drives ORDER BY created_at ASC`)
|
||||
if err != nil {
|
||||
@@ -1265,6 +1510,7 @@ func (c *Catalog) ListDrives(ctx context.Context) ([]*Drive, error) {
|
||||
}
|
||||
_ = json.Unmarshal([]byte(credsStr), &d.Credentials)
|
||||
_ = json.Unmarshal([]byte(skipDirsStr), &d.SkipDirIDs)
|
||||
normalizeDriveRootFields(d)
|
||||
d.TeaserEnabled = teaserEnabled != 0
|
||||
d.CreatedAt = time.UnixMilli(createdAt)
|
||||
d.UpdatedAt = time.UnixMilli(updatedAt)
|
||||
@@ -1284,6 +1530,7 @@ func (c *Catalog) GetDrive(ctx context.Context, id string) (*Drive, error) {
|
||||
}
|
||||
_ = json.Unmarshal([]byte(credsStr), &d.Credentials)
|
||||
_ = json.Unmarshal([]byte(skipDirsStr), &d.SkipDirIDs)
|
||||
normalizeDriveRootFields(d)
|
||||
d.TeaserEnabled = teaserEnabled != 0
|
||||
d.CreatedAt = time.UnixMilli(createdAt)
|
||||
d.UpdatedAt = time.UnixMilli(updatedAt)
|
||||
|
||||
@@ -0,0 +1,84 @@
|
||||
package catalog
|
||||
|
||||
import (
|
||||
"context"
|
||||
"testing"
|
||||
)
|
||||
|
||||
func TestUpsertDriveUsesRootIDAsScanRootID(t *testing.T) {
|
||||
ctx := context.Background()
|
||||
cat, err := Open(t.TempDir() + "/catalog.db")
|
||||
if err != nil {
|
||||
t.Fatalf("open catalog: %v", err)
|
||||
}
|
||||
t.Cleanup(func() {
|
||||
if err := cat.Close(); err != nil {
|
||||
t.Fatalf("close catalog: %v", err)
|
||||
}
|
||||
})
|
||||
|
||||
if err := cat.UpsertDrive(ctx, &Drive{
|
||||
ID: "drive",
|
||||
Kind: "p115",
|
||||
Name: "115",
|
||||
RootID: "root-folder",
|
||||
ScanRootID: "ignored-scan-root",
|
||||
}); err != nil {
|
||||
t.Fatalf("upsert drive: %v", err)
|
||||
}
|
||||
|
||||
got, err := cat.GetDrive(ctx, "drive")
|
||||
if err != nil {
|
||||
t.Fatalf("get drive: %v", err)
|
||||
}
|
||||
if got.RootID != "root-folder" {
|
||||
t.Fatalf("rootId = %q, want root-folder", got.RootID)
|
||||
}
|
||||
if got.ScanRootID != "root-folder" {
|
||||
t.Fatalf("scanRootId = %q, want root-folder", got.ScanRootID)
|
||||
}
|
||||
}
|
||||
|
||||
func TestUpsertDriveDefaultsRootIDByKind(t *testing.T) {
|
||||
ctx := context.Background()
|
||||
cat, err := Open(t.TempDir() + "/catalog.db")
|
||||
if err != nil {
|
||||
t.Fatalf("open catalog: %v", err)
|
||||
}
|
||||
t.Cleanup(func() {
|
||||
if err := cat.Close(); err != nil {
|
||||
t.Fatalf("close catalog: %v", err)
|
||||
}
|
||||
})
|
||||
|
||||
cases := []struct {
|
||||
id string
|
||||
kind string
|
||||
want string
|
||||
}{
|
||||
{id: "p115", kind: "p115", want: "0"},
|
||||
{id: "pikpak", kind: "pikpak", want: ""},
|
||||
{id: "onedrive", kind: "onedrive", want: "root"},
|
||||
{id: "googledrive", kind: "googledrive", want: "root"},
|
||||
{id: "localstorage", kind: "localstorage", want: "/"},
|
||||
}
|
||||
for _, tc := range cases {
|
||||
if err := cat.UpsertDrive(ctx, &Drive{
|
||||
ID: tc.id,
|
||||
Kind: tc.kind,
|
||||
Name: tc.kind,
|
||||
}); err != nil {
|
||||
t.Fatalf("upsert %s: %v", tc.kind, err)
|
||||
}
|
||||
got, err := cat.GetDrive(ctx, tc.id)
|
||||
if err != nil {
|
||||
t.Fatalf("get %s: %v", tc.kind, err)
|
||||
}
|
||||
if got.RootID != tc.want {
|
||||
t.Fatalf("%s rootId = %q, want %q", tc.kind, got.RootID, tc.want)
|
||||
}
|
||||
if got.ScanRootID != tc.want {
|
||||
t.Fatalf("%s scanRootId = %q, want %q", tc.kind, got.ScanRootID, tc.want)
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -17,7 +17,8 @@ CREATE TABLE IF NOT EXISTS videos (
|
||||
ext TEXT,
|
||||
quality TEXT, -- HD / SD
|
||||
thumbnail_url TEXT,
|
||||
thumbnail_status TEXT DEFAULT 'pending', -- pending / ready / failed
|
||||
thumbnail_status TEXT DEFAULT 'pending', -- pending / ready / failed / skipped
|
||||
thumbnail_failures INTEGER DEFAULT 0, -- consecutive transient thumbnail generation failures
|
||||
preview_file_id TEXT, -- deprecated: 旧版回写网盘后的 teaser file id
|
||||
preview_local TEXT, -- 本地 teaser 路径(兜底)
|
||||
preview_status TEXT DEFAULT 'pending', -- pending / ready / failed
|
||||
@@ -61,13 +62,21 @@ CREATE TABLE IF NOT EXISTS video_tags (
|
||||
CREATE INDEX IF NOT EXISTS idx_video_tags_tag ON video_tags(tag_id);
|
||||
CREATE INDEX IF NOT EXISTS idx_video_tags_video ON video_tags(video_id);
|
||||
|
||||
-- 用户手动删除过的非系统标签。自动扫描/迁移不再重新创建同名标签;
|
||||
-- 管理员手动新建同名标签时会移除这里的记录。
|
||||
CREATE TABLE IF NOT EXISTS deleted_tags (
|
||||
label TEXT PRIMARY KEY COLLATE NOCASE,
|
||||
source TEXT NOT NULL DEFAULT '',
|
||||
deleted_at INTEGER NOT NULL
|
||||
);
|
||||
|
||||
-- 网盘账户
|
||||
CREATE TABLE IF NOT EXISTS drives (
|
||||
id TEXT PRIMARY KEY,
|
||||
kind TEXT NOT NULL, -- quark / p115 / pikpak / wopan / onedrive / localstorage / spider91
|
||||
kind TEXT NOT NULL, -- quark / p115 / pikpak / wopan / onedrive / googledrive / localstorage / spider91
|
||||
name TEXT NOT NULL,
|
||||
root_id TEXT NOT NULL DEFAULT '0',
|
||||
scan_root_id TEXT, -- 扫描起点(默认 root_id)
|
||||
scan_root_id TEXT, -- deprecated: 扫描起点固定等于 root_id
|
||||
credentials TEXT, -- JSON: cookie / refresh_token 等
|
||||
status TEXT DEFAULT 'disconnected', -- disconnected / ok / error
|
||||
last_error TEXT,
|
||||
|
||||
@@ -109,3 +109,171 @@ func TestRandomVideosExcluding(t *testing.T) {
|
||||
t.Fatalf("limit 0 should return nil, got %v", got4)
|
||||
}
|
||||
}
|
||||
|
||||
func TestRandomVideosForPreferredVideoChoosesLeastPopulatedTag(t *testing.T) {
|
||||
ctx := context.Background()
|
||||
cat, err := Open(t.TempDir() + "/catalog.db")
|
||||
if err != nil {
|
||||
t.Fatalf("open catalog: %v", err)
|
||||
}
|
||||
t.Cleanup(func() { _ = cat.Close() })
|
||||
|
||||
now := time.Now()
|
||||
for _, v := range []*Video{
|
||||
{ID: "current", DriveID: "drive", FileID: "f-current", Title: "current", Tags: []string{"common", "rare"}, PublishedAt: now, CreatedAt: now, UpdatedAt: now},
|
||||
{ID: "common-1", DriveID: "drive", FileID: "f-common-1", Title: "common 1", Tags: []string{"common"}, PublishedAt: now, CreatedAt: now, UpdatedAt: now},
|
||||
{ID: "common-2", DriveID: "drive", FileID: "f-common-2", Title: "common 2", Tags: []string{"common"}, PublishedAt: now, CreatedAt: now, UpdatedAt: now},
|
||||
{ID: "rare-1", DriveID: "drive", FileID: "f-rare-1", Title: "rare 1", Tags: []string{"rare"}, PublishedAt: now, CreatedAt: now, UpdatedAt: now},
|
||||
} {
|
||||
if err := cat.UpsertVideo(ctx, v); err != nil {
|
||||
t.Fatalf("seed %s: %v", v.ID, err)
|
||||
}
|
||||
}
|
||||
|
||||
tag, err := cat.LeastPopulatedVisibleUniqueTag(ctx, []string{"common", "rare"})
|
||||
if err != nil {
|
||||
t.Fatalf("least populated tag: %v", err)
|
||||
}
|
||||
if tag != "rare" {
|
||||
t.Fatalf("least populated tag = %q, want rare", tag)
|
||||
}
|
||||
|
||||
got, err := cat.RandomVideosForPreferredVideoExcluding(ctx, "current", []string{"current"}, 1)
|
||||
if err != nil {
|
||||
t.Fatalf("random preferred: %v", err)
|
||||
}
|
||||
if len(got) != 1 || got[0].ID != "rare-1" {
|
||||
t.Fatalf("preferred result = %#v, want rare-1", videoIDs(got))
|
||||
}
|
||||
|
||||
got, err = cat.RandomVideosForPreferredVideoExcluding(ctx, "current", nil, 1)
|
||||
if err != nil {
|
||||
t.Fatalf("random preferred without explicit exclude: %v", err)
|
||||
}
|
||||
if len(got) != 1 || got[0].ID == "current" {
|
||||
t.Fatalf("preferred result without explicit exclude = %#v, should not return current", videoIDs(got))
|
||||
}
|
||||
}
|
||||
|
||||
func TestRandomVideosForPreferredVideoFallsBackToFillBatch(t *testing.T) {
|
||||
ctx := context.Background()
|
||||
cat, err := Open(t.TempDir() + "/catalog.db")
|
||||
if err != nil {
|
||||
t.Fatalf("open catalog: %v", err)
|
||||
}
|
||||
t.Cleanup(func() { _ = cat.Close() })
|
||||
|
||||
now := time.Now()
|
||||
for _, v := range []*Video{
|
||||
{ID: "current", DriveID: "drive", FileID: "f-current", Title: "current", Tags: []string{"common", "rare"}, PublishedAt: now, CreatedAt: now, UpdatedAt: now},
|
||||
{ID: "common-1", DriveID: "drive", FileID: "f-common-1", Title: "common 1", Tags: []string{"common"}, PublishedAt: now, CreatedAt: now, UpdatedAt: now},
|
||||
{ID: "common-2", DriveID: "drive", FileID: "f-common-2", Title: "common 2", Tags: []string{"common"}, PublishedAt: now, CreatedAt: now, UpdatedAt: now},
|
||||
{ID: "rare-1", DriveID: "drive", FileID: "f-rare-1", Title: "rare 1", Tags: []string{"rare"}, PublishedAt: now, CreatedAt: now, UpdatedAt: now},
|
||||
{ID: "hidden-rare", DriveID: "drive", FileID: "f-hidden-rare", Title: "hidden rare", Tags: []string{"rare"}, PublishedAt: now, CreatedAt: now, UpdatedAt: now},
|
||||
} {
|
||||
if err := cat.UpsertVideo(ctx, v); err != nil {
|
||||
t.Fatalf("seed %s: %v", v.ID, err)
|
||||
}
|
||||
}
|
||||
if err := cat.HideVideo(ctx, "hidden-rare"); err != nil {
|
||||
t.Fatalf("hide hidden-rare: %v", err)
|
||||
}
|
||||
|
||||
got, err := cat.RandomVideosForPreferredVideoExcluding(ctx, "current", []string{"current"}, 3)
|
||||
if err != nil {
|
||||
t.Fatalf("random preferred: %v", err)
|
||||
}
|
||||
ids := videoIDs(got)
|
||||
if len(ids) != 3 {
|
||||
t.Fatalf("result ids = %#v, want 3 items", ids)
|
||||
}
|
||||
for _, excluded := range []string{"current", "hidden-rare"} {
|
||||
if hasVideoID(ids, excluded) {
|
||||
t.Fatalf("result ids = %#v, should not include %s", ids, excluded)
|
||||
}
|
||||
}
|
||||
if !hasVideoID(ids, "rare-1") {
|
||||
t.Fatalf("result ids = %#v, want rare-1 from least populated tag", ids)
|
||||
}
|
||||
if len(uniqueVideoIDs(ids)) != len(ids) {
|
||||
t.Fatalf("result ids = %#v, want no duplicates", ids)
|
||||
}
|
||||
}
|
||||
|
||||
func TestRandomVideosForPreferredVideoFallbacksWhenPreferenceUnavailable(t *testing.T) {
|
||||
ctx := context.Background()
|
||||
cat, err := Open(t.TempDir() + "/catalog.db")
|
||||
if err != nil {
|
||||
t.Fatalf("open catalog: %v", err)
|
||||
}
|
||||
t.Cleanup(func() { _ = cat.Close() })
|
||||
|
||||
now := time.Now()
|
||||
for _, v := range []*Video{
|
||||
{ID: "untagged", DriveID: "drive", FileID: "f-untagged", Title: "untagged", PublishedAt: now, CreatedAt: now, UpdatedAt: now},
|
||||
{ID: "visible-1", DriveID: "drive", FileID: "f-visible-1", Title: "visible 1", PublishedAt: now, CreatedAt: now, UpdatedAt: now},
|
||||
{ID: "visible-2", DriveID: "drive", FileID: "f-visible-2", Title: "visible 2", PublishedAt: now, CreatedAt: now, UpdatedAt: now},
|
||||
} {
|
||||
if err := cat.UpsertVideo(ctx, v); err != nil {
|
||||
t.Fatalf("seed %s: %v", v.ID, err)
|
||||
}
|
||||
}
|
||||
|
||||
got, err := cat.RandomVideosForPreferredVideoExcluding(ctx, "missing", []string{"untagged"}, 2)
|
||||
if err != nil {
|
||||
t.Fatalf("random missing preferred: %v", err)
|
||||
}
|
||||
if !sameVideoIDSet(videoIDs(got), []string{"visible-1", "visible-2"}) {
|
||||
t.Fatalf("missing preferred ids = %#v, want visible fallback videos", videoIDs(got))
|
||||
}
|
||||
|
||||
got, err = cat.RandomVideosForPreferredVideoExcluding(ctx, "untagged", []string{"untagged"}, 2)
|
||||
if err != nil {
|
||||
t.Fatalf("random untagged preferred: %v", err)
|
||||
}
|
||||
if !sameVideoIDSet(videoIDs(got), []string{"visible-1", "visible-2"}) {
|
||||
t.Fatalf("untagged preferred ids = %#v, want visible fallback videos", videoIDs(got))
|
||||
}
|
||||
}
|
||||
|
||||
func videoIDs(videos []*Video) []string {
|
||||
ids := make([]string, 0, len(videos))
|
||||
for _, v := range videos {
|
||||
ids = append(ids, v.ID)
|
||||
}
|
||||
return ids
|
||||
}
|
||||
|
||||
func hasVideoID(ids []string, want string) bool {
|
||||
for _, id := range ids {
|
||||
if id == want {
|
||||
return true
|
||||
}
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
func uniqueVideoIDs(ids []string) map[string]struct{} {
|
||||
seen := make(map[string]struct{}, len(ids))
|
||||
for _, id := range ids {
|
||||
seen[id] = struct{}{}
|
||||
}
|
||||
return seen
|
||||
}
|
||||
|
||||
func sameVideoIDSet(a, b []string) bool {
|
||||
if len(a) != len(b) {
|
||||
return false
|
||||
}
|
||||
seen := make(map[string]int, len(a))
|
||||
for _, value := range a {
|
||||
seen[value]++
|
||||
}
|
||||
for _, value := range b {
|
||||
if seen[value] == 0 {
|
||||
return false
|
||||
}
|
||||
seen[value]--
|
||||
}
|
||||
return true
|
||||
}
|
||||
|
||||
@@ -17,6 +17,8 @@ import (
|
||||
)
|
||||
|
||||
var ErrUnknownTag = errors.New("unknown tag")
|
||||
var ErrSystemTag = errors.New("system tag cannot be deleted")
|
||||
var ErrDeletedTag = errors.New("tag was previously deleted")
|
||||
|
||||
const avTagLabel = "AV"
|
||||
|
||||
@@ -61,6 +63,9 @@ func (c *Catalog) migrate(ctx context.Context) error {
|
||||
if err := c.addColumnIfMissing(ctx, "videos", "thumbnail_status", "TEXT DEFAULT 'pending'"); err != nil {
|
||||
return err
|
||||
}
|
||||
if err := c.addColumnIfMissing(ctx, "videos", "thumbnail_failures", "INTEGER DEFAULT 0"); err != nil {
|
||||
return err
|
||||
}
|
||||
// drives.teaser_enabled:每盘 teaser 开关,替代旧的全局 preview.enabled。
|
||||
// 升级路径:直接让 ALTER TABLE 的 DEFAULT 1 兜底 —— 每个现存 drive 都默认开启,
|
||||
// 不读旧的 settings.preview.enabled 字段。这样老用户即便之前关过全局开关,
|
||||
@@ -74,6 +79,9 @@ func (c *Catalog) migrate(ctx context.Context) error {
|
||||
if err := c.addColumnIfMissing(ctx, "drives", "skip_dir_ids", "TEXT NOT NULL DEFAULT '[]'"); err != nil {
|
||||
return err
|
||||
}
|
||||
if err := c.syncDriveScanRootIDToRootID(ctx); err != nil {
|
||||
return err
|
||||
}
|
||||
// 一次性修正:早期版本(短暂存在过)会把现存 drive 的 teaser_enabled 同步成
|
||||
// 旧的全局 preview.enabled 值,导致升级后所有 drive 都是关。"默认开启"约定下,
|
||||
// 这里一次性把所有 drive 强制重置为 1,并用 marker setting 记号,避免之后
|
||||
@@ -201,8 +209,9 @@ func (c *Catalog) resetDriveTeaserEnabledToDefaultOnce(ctx context.Context) erro
|
||||
// - 管理员凭直觉认知字段名时会被误导
|
||||
//
|
||||
// 修正策略:
|
||||
// - thumbnail_url 非空 + status 非 'ready' + status 非 'failed' → 改成 'ready'
|
||||
// - thumbnail_url 非空 + status 非 'ready' + status 非 'failed' + status 非 'skipped' → 改成 'ready'
|
||||
// - status='failed' 不动(这是 worker 显式标的失败,要保留以便管理员手动重生)
|
||||
// - status='skipped' 不动(已有封面但时长探测不可用,避免重启后重复排队)
|
||||
//
|
||||
// 幂等保证:marker setting 写过就不再跑,避免每次重启都 update 一遍。
|
||||
func (c *Catalog) reconcileThumbnailStatusOnce(ctx context.Context) error {
|
||||
@@ -219,7 +228,7 @@ UPDATE videos
|
||||
SET thumbnail_status = 'ready',
|
||||
updated_at = ?
|
||||
WHERE COALESCE(thumbnail_url, '') != ''
|
||||
AND COALESCE(thumbnail_status, 'pending') NOT IN ('ready', 'failed')
|
||||
AND COALESCE(thumbnail_status, 'pending') NOT IN ('ready', 'failed', 'skipped')
|
||||
`, time.Now().UnixMilli())
|
||||
if err != nil {
|
||||
return fmt.Errorf("reconcile thumbnail_status: %w", err)
|
||||
@@ -362,6 +371,9 @@ GROUP BY category`)
|
||||
if !LooksLikeCollectionTag(stat.category) {
|
||||
continue
|
||||
}
|
||||
if c.tagDeleted(ctx, stat.category) {
|
||||
continue
|
||||
}
|
||||
if _, err := c.ensureTag(ctx, stat.category, nil, "collection"); err != nil {
|
||||
return err
|
||||
}
|
||||
@@ -380,6 +392,65 @@ func (c *Catalog) CreateTagAndClassify(ctx context.Context, label string, aliase
|
||||
return c.classifyTag(ctx, tag)
|
||||
}
|
||||
|
||||
func (c *Catalog) DeleteTag(ctx context.Context, tagID int64) (int, error) {
|
||||
tx, err := c.db.BeginTx(ctx, nil)
|
||||
if err != nil {
|
||||
return 0, err
|
||||
}
|
||||
defer tx.Rollback()
|
||||
|
||||
tag, err := c.getTagByIDTx(ctx, tx, tagID)
|
||||
if err != nil {
|
||||
return 0, err
|
||||
}
|
||||
if tag.Source == "system" {
|
||||
return 0, ErrSystemTag
|
||||
}
|
||||
|
||||
rows, err := tx.QueryContext(ctx, `SELECT video_id FROM video_tags WHERE tag_id = ?`, tagID)
|
||||
if err != nil {
|
||||
return 0, err
|
||||
}
|
||||
var videoIDs []string
|
||||
for rows.Next() {
|
||||
var videoID string
|
||||
if err := rows.Scan(&videoID); err != nil {
|
||||
rows.Close()
|
||||
return 0, err
|
||||
}
|
||||
videoIDs = append(videoIDs, videoID)
|
||||
}
|
||||
if err := rows.Err(); err != nil {
|
||||
rows.Close()
|
||||
return 0, err
|
||||
}
|
||||
if err := rows.Close(); err != nil {
|
||||
return 0, err
|
||||
}
|
||||
|
||||
if _, err := tx.ExecContext(ctx, `DELETE FROM video_tags WHERE tag_id = ?`, tagID); err != nil {
|
||||
return 0, err
|
||||
}
|
||||
if _, err := tx.ExecContext(ctx, `DELETE FROM tags WHERE id = ?`, tagID); err != nil {
|
||||
return 0, err
|
||||
}
|
||||
if err := markDeletedTagTx(ctx, tx, tag); err != nil {
|
||||
return 0, err
|
||||
}
|
||||
|
||||
for _, videoID := range videoIDs {
|
||||
manual := hasManualTagsTx(ctx, tx, videoID)
|
||||
if err := syncVideoTagsJSONTx(ctx, tx, videoID, manual); err != nil {
|
||||
return 0, err
|
||||
}
|
||||
}
|
||||
|
||||
if err := tx.Commit(); err != nil {
|
||||
return 0, err
|
||||
}
|
||||
return len(videoIDs), nil
|
||||
}
|
||||
|
||||
func (c *Catalog) ListTags(ctx context.Context) ([]Tag, error) {
|
||||
rows, err := c.db.QueryContext(ctx, `
|
||||
SELECT t.id, t.label, t.aliases, t.source, COUNT(v.id) AS cnt
|
||||
@@ -453,6 +524,9 @@ func (c *Catalog) EnsureCollectionTag(ctx context.Context, label string) (string
|
||||
if !LooksLikeCollectionTag(label) {
|
||||
return "", false, nil
|
||||
}
|
||||
if c.tagDeleted(ctx, label) {
|
||||
return "", false, nil
|
||||
}
|
||||
if !c.tagExists(ctx, label) {
|
||||
count, err := c.categoryVideoCount(ctx, label)
|
||||
if err != nil {
|
||||
@@ -484,6 +558,14 @@ func (c *Catalog) ensureTag(ctx context.Context, label string, aliases []string,
|
||||
if source == "" {
|
||||
source = "user"
|
||||
}
|
||||
if source != "system" && source != "user" && c.tagDeleted(ctx, label) {
|
||||
return Tag{}, ErrDeletedTag
|
||||
}
|
||||
if source == "system" || source == "user" {
|
||||
if err := c.restoreDeletedTag(ctx, label); err != nil {
|
||||
return Tag{}, err
|
||||
}
|
||||
}
|
||||
aliases = cleanAliases(aliases, label)
|
||||
aliasesJSON, _ := json.Marshal(aliases)
|
||||
now := time.Now().UnixMilli()
|
||||
@@ -557,9 +639,15 @@ FROM videos`)
|
||||
|
||||
func (c *Catalog) replaceVideoTags(ctx context.Context, videoID string, labels []string, source string, manual bool, createMissing bool) error {
|
||||
labels = uniqueStrings(cleanLabels(labels))
|
||||
if source != "manual" {
|
||||
labels = c.filterDeletedTagLabels(ctx, labels)
|
||||
}
|
||||
if createMissing {
|
||||
for _, label := range labels {
|
||||
if _, err := c.ensureTag(ctx, label, nil, "legacy"); err != nil {
|
||||
if errors.Is(err, ErrDeletedTag) {
|
||||
continue
|
||||
}
|
||||
return err
|
||||
}
|
||||
}
|
||||
@@ -602,7 +690,11 @@ func (c *Catalog) replaceVideoTags(ctx context.Context, videoID string, labels [
|
||||
}
|
||||
|
||||
func (c *Catalog) addVideoTags(ctx context.Context, videoID string, labels []string, source string, createMissing bool) error {
|
||||
for _, label := range uniqueStrings(cleanLabels(labels)) {
|
||||
labels = uniqueStrings(cleanLabels(labels))
|
||||
if source != "manual" {
|
||||
labels = c.filterDeletedTagLabels(ctx, labels)
|
||||
}
|
||||
for _, label := range labels {
|
||||
if _, err := c.addVideoTag(ctx, videoID, label, source, createMissing); err != nil {
|
||||
return err
|
||||
}
|
||||
@@ -611,8 +703,14 @@ func (c *Catalog) addVideoTags(ctx context.Context, videoID string, labels []str
|
||||
}
|
||||
|
||||
func (c *Catalog) addVideoTag(ctx context.Context, videoID, label, source string, createMissing bool) (bool, error) {
|
||||
if source != "manual" && c.tagDeleted(ctx, label) {
|
||||
return false, nil
|
||||
}
|
||||
if createMissing {
|
||||
if _, err := c.ensureTag(ctx, label, nil, "legacy"); err != nil {
|
||||
if errors.Is(err, ErrDeletedTag) {
|
||||
return false, nil
|
||||
}
|
||||
return false, err
|
||||
}
|
||||
}
|
||||
@@ -804,6 +902,39 @@ func (c *Catalog) tagExists(ctx context.Context, label string) bool {
|
||||
return err == nil
|
||||
}
|
||||
|
||||
func (c *Catalog) tagDeleted(ctx context.Context, label string) bool {
|
||||
label = cleanTagLabel(label)
|
||||
if label == "" {
|
||||
return false
|
||||
}
|
||||
var exists int
|
||||
err := c.db.QueryRowContext(ctx, `SELECT 1 FROM deleted_tags WHERE label = ? COLLATE NOCASE`, label).Scan(&exists)
|
||||
return err == nil
|
||||
}
|
||||
|
||||
func (c *Catalog) filterDeletedTagLabels(ctx context.Context, labels []string) []string {
|
||||
if len(labels) == 0 {
|
||||
return labels
|
||||
}
|
||||
out := labels[:0]
|
||||
for _, label := range labels {
|
||||
if c.tagDeleted(ctx, label) {
|
||||
continue
|
||||
}
|
||||
out = append(out, label)
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
func (c *Catalog) restoreDeletedTag(ctx context.Context, label string) error {
|
||||
label = cleanTagLabel(label)
|
||||
if label == "" {
|
||||
return nil
|
||||
}
|
||||
_, err := c.db.ExecContext(ctx, `DELETE FROM deleted_tags WHERE label = ? COLLATE NOCASE`, label)
|
||||
return err
|
||||
}
|
||||
|
||||
func (c *Catalog) categoryVideoCount(ctx context.Context, category string) (int, error) {
|
||||
var count int
|
||||
err := c.db.QueryRowContext(ctx, `SELECT COUNT(*) FROM videos WHERE category = ?`, category).Scan(&count)
|
||||
@@ -817,6 +948,71 @@ func (c *Catalog) getTagByLabelTx(ctx context.Context, tx *sql.Tx, label string)
|
||||
return scanTag(row)
|
||||
}
|
||||
|
||||
func (c *Catalog) getTagByIDTx(ctx context.Context, tx *sql.Tx, id int64) (Tag, error) {
|
||||
row := tx.QueryRowContext(ctx,
|
||||
`SELECT id, label, aliases, source, 0 FROM tags WHERE id = ?`,
|
||||
id)
|
||||
return scanTag(row)
|
||||
}
|
||||
|
||||
func hasManualTagsTx(ctx context.Context, tx *sql.Tx, videoID string) bool {
|
||||
var manual int
|
||||
err := tx.QueryRowContext(ctx, `SELECT COALESCE(tags_manual, 0) FROM videos WHERE id = ?`, videoID).Scan(&manual)
|
||||
return err == nil && manual == 1
|
||||
}
|
||||
|
||||
func markDeletedTagTx(ctx context.Context, tx *sql.Tx, tag Tag) error {
|
||||
label := cleanTagLabel(tag.Label)
|
||||
if label == "" {
|
||||
return nil
|
||||
}
|
||||
now := time.Now().UnixMilli()
|
||||
_, err := tx.ExecContext(ctx, `
|
||||
INSERT INTO deleted_tags (label, source, deleted_at)
|
||||
VALUES (?, ?, ?)
|
||||
ON CONFLICT(label) DO UPDATE SET
|
||||
source = excluded.source,
|
||||
deleted_at = excluded.deleted_at`, label, tag.Source, now)
|
||||
return err
|
||||
}
|
||||
|
||||
func syncVideoTagsJSONTx(ctx context.Context, tx *sql.Tx, videoID string, manual bool) error {
|
||||
rows, err := tx.QueryContext(ctx, `
|
||||
SELECT t.label
|
||||
FROM video_tags vt
|
||||
JOIN tags t ON t.id = vt.tag_id
|
||||
WHERE vt.video_id = ?
|
||||
ORDER BY t.id ASC`, videoID)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
var labels []string
|
||||
for rows.Next() {
|
||||
var label string
|
||||
if err := rows.Scan(&label); err != nil {
|
||||
rows.Close()
|
||||
return err
|
||||
}
|
||||
labels = append(labels, label)
|
||||
}
|
||||
if err := rows.Err(); err != nil {
|
||||
rows.Close()
|
||||
return err
|
||||
}
|
||||
if err := rows.Close(); err != nil {
|
||||
return err
|
||||
}
|
||||
labelsJSON, _ := json.Marshal(labels)
|
||||
manualValue := 0
|
||||
if manual {
|
||||
manualValue = 1
|
||||
}
|
||||
_, err = tx.ExecContext(ctx,
|
||||
`UPDATE videos SET tags = ?, tags_manual = ?, updated_at = ? WHERE id = ?`,
|
||||
string(labelsJSON), manualValue, time.Now().UnixMilli(), videoID)
|
||||
return err
|
||||
}
|
||||
|
||||
type tagRowScanner interface {
|
||||
Scan(dest ...any) error
|
||||
}
|
||||
|
||||
@@ -3,6 +3,7 @@ package catalog
|
||||
import (
|
||||
"context"
|
||||
"database/sql"
|
||||
"errors"
|
||||
"testing"
|
||||
"time"
|
||||
)
|
||||
@@ -89,6 +90,32 @@ func TestListVideosNeedingThumbnailIncludesExistingThumbnailMissingDuration(t *t
|
||||
if count != 2 {
|
||||
t.Fatalf("count = %d, want 2", count)
|
||||
}
|
||||
|
||||
counts, err := cat.CountThumbnailsByDrive(ctx)
|
||||
if err != nil {
|
||||
t.Fatalf("count thumbnails by drive: %v", err)
|
||||
}
|
||||
if got := counts["drive"]; got.Ready != 2 || got.Pending != 1 || got.Failed != 1 || got.DurationPending != 1 {
|
||||
t.Fatalf("thumbnail counts = %#v, want ready=2 pending=1 failed=1 durationPending=1", got)
|
||||
}
|
||||
|
||||
if err := cat.UpdateVideoMeta(ctx, "duration-only", VideoMetaPatch{ThumbnailStatus: "skipped"}); err != nil {
|
||||
t.Fatalf("mark duration-only skipped: %v", err)
|
||||
}
|
||||
count, err = cat.CountVideosNeedingThumbnail(ctx, "drive")
|
||||
if err != nil {
|
||||
t.Fatalf("count videos needing thumbnail after skip: %v", err)
|
||||
}
|
||||
if count != 1 {
|
||||
t.Fatalf("count after skip = %d, want 1", count)
|
||||
}
|
||||
counts, err = cat.CountThumbnailsByDrive(ctx)
|
||||
if err != nil {
|
||||
t.Fatalf("count thumbnails by drive after skip: %v", err)
|
||||
}
|
||||
if got := counts["drive"]; got.Ready != 2 || got.Pending != 1 || got.Failed != 1 || got.DurationPending != 0 {
|
||||
t.Fatalf("thumbnail counts after skip = %#v, want ready=2 pending=1 failed=1 durationPending=0", got)
|
||||
}
|
||||
}
|
||||
|
||||
func TestCreateTagAndClassifyAddsTagToMatchingExistingVideos(t *testing.T) {
|
||||
@@ -154,6 +181,173 @@ func TestCreateTagAndClassifyAddsTagToMatchingExistingVideos(t *testing.T) {
|
||||
}
|
||||
}
|
||||
|
||||
func TestDeleteTagRemovesTagFromVideos(t *testing.T) {
|
||||
ctx := context.Background()
|
||||
cat, err := Open(t.TempDir() + "/catalog.db")
|
||||
if err != nil {
|
||||
t.Fatalf("open catalog: %v", err)
|
||||
}
|
||||
t.Cleanup(func() {
|
||||
if err := cat.Close(); err != nil {
|
||||
t.Fatalf("close catalog: %v", err)
|
||||
}
|
||||
})
|
||||
|
||||
now := time.Now()
|
||||
if err := cat.UpsertVideo(ctx, &Video{
|
||||
ID: "video-1",
|
||||
DriveID: "drive",
|
||||
FileID: "file-1",
|
||||
Title: "清纯短发",
|
||||
PublishedAt: now,
|
||||
CreatedAt: now,
|
||||
UpdatedAt: now,
|
||||
}); err != nil {
|
||||
t.Fatalf("seed video: %v", err)
|
||||
}
|
||||
if _, err := cat.CreateTagAndClassify(ctx, "清纯", nil, "user"); err != nil {
|
||||
t.Fatalf("create tag: %v", err)
|
||||
}
|
||||
|
||||
tag := mustTagByLabel(t, ctx, cat, "清纯")
|
||||
removed, err := cat.DeleteTag(ctx, tag.ID)
|
||||
if err != nil {
|
||||
t.Fatalf("delete tag: %v", err)
|
||||
}
|
||||
if removed != 1 {
|
||||
t.Fatalf("removed = %d, want 1", removed)
|
||||
}
|
||||
|
||||
got, err := cat.GetVideo(ctx, "video-1")
|
||||
if err != nil {
|
||||
t.Fatalf("get video: %v", err)
|
||||
}
|
||||
if len(got.Tags) != 0 {
|
||||
t.Fatalf("video tags = %#v, want none", got.Tags)
|
||||
}
|
||||
for _, tag := range mustListTags(t, ctx, cat) {
|
||||
if tag.Label == "清纯" {
|
||||
t.Fatal("deleted tag still appears in ListTags")
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestDeleteTagSuppressesAutomaticCollectionRecreation(t *testing.T) {
|
||||
ctx := context.Background()
|
||||
cat, err := Open(t.TempDir() + "/catalog.db")
|
||||
if err != nil {
|
||||
t.Fatalf("open catalog: %v", err)
|
||||
}
|
||||
t.Cleanup(func() {
|
||||
if err := cat.Close(); err != nil {
|
||||
t.Fatalf("close catalog: %v", err)
|
||||
}
|
||||
})
|
||||
|
||||
now := time.Now()
|
||||
for _, id := range []string{"video-1", "video-2"} {
|
||||
if err := cat.UpsertVideo(ctx, &Video{
|
||||
ID: id,
|
||||
DriveID: "drive",
|
||||
FileID: id,
|
||||
Title: "合集视频",
|
||||
Category: "sunny",
|
||||
PublishedAt: now,
|
||||
CreatedAt: now,
|
||||
UpdatedAt: now,
|
||||
}); err != nil {
|
||||
t.Fatalf("seed video %s: %v", id, err)
|
||||
}
|
||||
}
|
||||
|
||||
if label, ok, err := cat.EnsureCollectionTag(ctx, "sunny"); err != nil || !ok || label != "sunny" {
|
||||
t.Fatalf("ensure collection = %q, %v, %v; want sunny true nil", label, ok, err)
|
||||
}
|
||||
tag := mustTagByLabel(t, ctx, cat, "sunny")
|
||||
if _, err := cat.DeleteTag(ctx, tag.ID); err != nil {
|
||||
t.Fatalf("delete tag: %v", err)
|
||||
}
|
||||
|
||||
if label, ok, err := cat.EnsureCollectionTag(ctx, "sunny"); err != nil || ok || label != "" {
|
||||
t.Fatalf("ensure deleted collection = %q, %v, %v; want empty false nil", label, ok, err)
|
||||
}
|
||||
for _, tag := range mustListTags(t, ctx, cat) {
|
||||
if tag.Label == "sunny" {
|
||||
t.Fatal("deleted collection tag was recreated automatically")
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestCreateTagAndClassifyRestoresDeletedTag(t *testing.T) {
|
||||
ctx := context.Background()
|
||||
cat, err := Open(t.TempDir() + "/catalog.db")
|
||||
if err != nil {
|
||||
t.Fatalf("open catalog: %v", err)
|
||||
}
|
||||
t.Cleanup(func() {
|
||||
if err := cat.Close(); err != nil {
|
||||
t.Fatalf("close catalog: %v", err)
|
||||
}
|
||||
})
|
||||
|
||||
now := time.Now()
|
||||
if err := cat.UpsertVideo(ctx, &Video{
|
||||
ID: "video-1",
|
||||
DriveID: "drive",
|
||||
FileID: "file-1",
|
||||
Title: "清纯短发",
|
||||
PublishedAt: now,
|
||||
CreatedAt: now,
|
||||
UpdatedAt: now,
|
||||
}); err != nil {
|
||||
t.Fatalf("seed video: %v", err)
|
||||
}
|
||||
if _, err := cat.CreateTagAndClassify(ctx, "清纯", nil, "user"); err != nil {
|
||||
t.Fatalf("create tag: %v", err)
|
||||
}
|
||||
tag := mustTagByLabel(t, ctx, cat, "清纯")
|
||||
if _, err := cat.DeleteTag(ctx, tag.ID); err != nil {
|
||||
t.Fatalf("delete tag: %v", err)
|
||||
}
|
||||
|
||||
classified, err := cat.CreateTagAndClassify(ctx, "清纯", nil, "user")
|
||||
if err != nil {
|
||||
t.Fatalf("recreate tag: %v", err)
|
||||
}
|
||||
if classified != 1 {
|
||||
t.Fatalf("classified = %d, want 1", classified)
|
||||
}
|
||||
got, err := cat.GetVideo(ctx, "video-1")
|
||||
if err != nil {
|
||||
t.Fatalf("get video: %v", err)
|
||||
}
|
||||
if !sameStrings(got.Tags, []string{"清纯"}) {
|
||||
t.Fatalf("video tags = %#v, want 清纯", got.Tags)
|
||||
}
|
||||
}
|
||||
|
||||
func TestDeleteTagRejectsSystemTags(t *testing.T) {
|
||||
ctx := context.Background()
|
||||
cat, err := Open(t.TempDir() + "/catalog.db")
|
||||
if err != nil {
|
||||
t.Fatalf("open catalog: %v", err)
|
||||
}
|
||||
t.Cleanup(func() {
|
||||
if err := cat.Close(); err != nil {
|
||||
t.Fatalf("close catalog: %v", err)
|
||||
}
|
||||
})
|
||||
|
||||
tag := mustTagByLabel(t, ctx, cat, "AV")
|
||||
if _, err := cat.DeleteTag(ctx, tag.ID); !errors.Is(err, ErrSystemTag) {
|
||||
t.Fatalf("delete system tag err = %v, want ErrSystemTag", err)
|
||||
}
|
||||
|
||||
if tag := mustTagByLabel(t, ctx, cat, "AV"); tag.Source != "system" {
|
||||
t.Fatalf("AV source = %q, want system", tag.Source)
|
||||
}
|
||||
}
|
||||
|
||||
func TestOpenClassifiesSystemTagsForExistingVideos(t *testing.T) {
|
||||
path := t.TempDir() + "/catalog.db"
|
||||
db, err := sql.Open("sqlite", path)
|
||||
@@ -730,6 +924,26 @@ func sameStrings(a, b []string) bool {
|
||||
return true
|
||||
}
|
||||
|
||||
func mustListTags(t *testing.T, ctx context.Context, cat *Catalog) []Tag {
|
||||
t.Helper()
|
||||
tags, err := cat.ListTags(ctx)
|
||||
if err != nil {
|
||||
t.Fatalf("list tags: %v", err)
|
||||
}
|
||||
return tags
|
||||
}
|
||||
|
||||
func mustTagByLabel(t *testing.T, ctx context.Context, cat *Catalog, label string) Tag {
|
||||
t.Helper()
|
||||
for _, tag := range mustListTags(t, ctx, cat) {
|
||||
if tag.Label == label {
|
||||
return tag
|
||||
}
|
||||
}
|
||||
t.Fatalf("tag %q not found", label)
|
||||
return Tag{}
|
||||
}
|
||||
|
||||
// 删除 collection 标签的最后一个引用视频后,标签应当自动从 tags 表里消失。
|
||||
// user/system 标签不受影响:用户/系统标签的语义由人维护,孤儿状态保留。
|
||||
func TestDeleteVideoPrunesOrphanCollectionTag(t *testing.T) {
|
||||
@@ -923,11 +1137,12 @@ func TestReconcileThumbnailStatusOnce(t *testing.T) {
|
||||
id, url, status string
|
||||
wantStatus string
|
||||
}{
|
||||
{"v-pending-url", "/p/thumb/v-pending-url", "pending", "ready"}, // 主要修复目标
|
||||
{"v-empty-url-pending", "", "pending", "pending"}, // 没 url 不动
|
||||
{"v-failed-with-url", "/p/thumb/v-failed-with-url", "failed", "failed"}, // 显式失败保留
|
||||
{"v-empty-url-failed", "", "failed", "failed"}, // 失败 + 没 url 也保留
|
||||
{"v-already-ready", "/p/thumb/v-already-ready", "ready", "ready"}, // 幂等
|
||||
{"v-pending-url", "/p/thumb/v-pending-url", "pending", "ready"}, // 主要修复目标
|
||||
{"v-empty-url-pending", "", "pending", "pending"}, // 没 url 不动
|
||||
{"v-failed-with-url", "/p/thumb/v-failed-with-url", "failed", "failed"}, // 显式失败保留
|
||||
{"v-empty-url-failed", "", "failed", "failed"}, // 失败 + 没 url 也保留
|
||||
{"v-skipped-with-url", "/p/thumb/v-skipped-with-url", "skipped", "skipped"}, // 已跳过的时长补全保留
|
||||
{"v-already-ready", "/p/thumb/v-already-ready", "ready", "ready"}, // 幂等
|
||||
}
|
||||
for _, c := range cases {
|
||||
if err := cat.UpsertVideo(ctx, &Video{
|
||||
|
||||
@@ -202,7 +202,7 @@ type Nightly struct {
|
||||
// 这里保留 yaml 中的静态定义,用于启动时预置盘。生产建议只在 DB 里维护。
|
||||
type Drive struct {
|
||||
ID string `yaml:"id"`
|
||||
Kind string `yaml:"kind"` // quark / p115 / pikpak / wopan / onedrive / localstorage
|
||||
Kind string `yaml:"kind"` // quark / p115 / pikpak / wopan / onedrive / googledrive / localstorage
|
||||
Name string `yaml:"name"`
|
||||
RootID string `yaml:"root_id"`
|
||||
Params map[string]string `yaml:"params,omitempty"`
|
||||
|
||||
@@ -0,0 +1,505 @@
|
||||
package googledrive
|
||||
|
||||
import (
|
||||
"context"
|
||||
"errors"
|
||||
"fmt"
|
||||
"io"
|
||||
"net/http"
|
||||
"net/url"
|
||||
"path"
|
||||
"strconv"
|
||||
"strings"
|
||||
"sync"
|
||||
"time"
|
||||
|
||||
"github.com/go-resty/resty/v2"
|
||||
|
||||
"github.com/video-site/backend/internal/drives"
|
||||
)
|
||||
|
||||
const (
|
||||
Kind = "googledrive"
|
||||
defaultAPIBaseURL = "https://www.googleapis.com/drive/v3"
|
||||
defaultOAuthURL = "https://www.googleapis.com/oauth2/v4/token"
|
||||
defaultRenewAPIURL = "https://api.oplist.org/googleui/renewapi"
|
||||
defaultListInterval = 1 * time.Second
|
||||
defaultListCooldown = 5 * time.Minute
|
||||
|
||||
filesListFields = "files(id,name,mimeType,size,modifiedTime,createdTime,thumbnailLink,shortcutDetails,md5Checksum,sha1Checksum,sha256Checksum),nextPageToken"
|
||||
fileInfoFields = "id,name,mimeType,size,modifiedTime,createdTime,thumbnailLink,shortcutDetails,md5Checksum,sha1Checksum,sha256Checksum"
|
||||
)
|
||||
|
||||
type Driver struct {
|
||||
id string
|
||||
rootID string
|
||||
refreshToken string
|
||||
accessToken string
|
||||
clientID string
|
||||
clientSecret string
|
||||
useOnlineAPI bool
|
||||
renewAPIURL string
|
||||
oauthURL string
|
||||
apiBaseURL string
|
||||
client *resty.Client
|
||||
onTokenUpdate func(access, refresh string)
|
||||
|
||||
listMu sync.Mutex
|
||||
lastListAt time.Time
|
||||
listInterval time.Duration
|
||||
listCooldown time.Duration
|
||||
}
|
||||
|
||||
type Config struct {
|
||||
ID string
|
||||
RootID string
|
||||
RefreshToken string
|
||||
AccessToken string
|
||||
ClientID string
|
||||
ClientSecret string
|
||||
UseOnlineAPI bool
|
||||
RenewAPIURL string
|
||||
OAuthURL string
|
||||
APIBaseURL string
|
||||
|
||||
OnTokenUpdate func(access, refresh string)
|
||||
}
|
||||
|
||||
func New(c Config) *Driver {
|
||||
rootID := strings.TrimSpace(c.RootID)
|
||||
if rootID == "" {
|
||||
rootID = "root"
|
||||
}
|
||||
renewAPIURL := strings.TrimSpace(c.RenewAPIURL)
|
||||
if renewAPIURL == "" {
|
||||
renewAPIURL = defaultRenewAPIURL
|
||||
}
|
||||
oauthURL := strings.TrimSpace(c.OAuthURL)
|
||||
if oauthURL == "" {
|
||||
oauthURL = defaultOAuthURL
|
||||
}
|
||||
apiBaseURL := strings.TrimRight(strings.TrimSpace(c.APIBaseURL), "/")
|
||||
if apiBaseURL == "" {
|
||||
apiBaseURL = defaultAPIBaseURL
|
||||
}
|
||||
return &Driver{
|
||||
id: c.ID,
|
||||
rootID: rootID,
|
||||
refreshToken: strings.TrimSpace(c.RefreshToken),
|
||||
accessToken: strings.TrimSpace(c.AccessToken),
|
||||
clientID: strings.TrimSpace(c.ClientID),
|
||||
clientSecret: strings.TrimSpace(c.ClientSecret),
|
||||
useOnlineAPI: c.UseOnlineAPI,
|
||||
renewAPIURL: renewAPIURL,
|
||||
oauthURL: oauthURL,
|
||||
apiBaseURL: apiBaseURL,
|
||||
onTokenUpdate: c.OnTokenUpdate,
|
||||
client: resty.New().
|
||||
SetTimeout(30*time.Second).
|
||||
SetHeader("Accept", "application/json, text/plain, */*"),
|
||||
listInterval: defaultListInterval,
|
||||
listCooldown: defaultListCooldown,
|
||||
}
|
||||
}
|
||||
|
||||
func (d *Driver) Kind() string { return Kind }
|
||||
func (d *Driver) ID() string { return d.id }
|
||||
func (d *Driver) RootID() string { return d.rootID }
|
||||
|
||||
func (d *Driver) Init(ctx context.Context) error {
|
||||
if d.refreshToken == "" {
|
||||
return errors.New("googledrive init: refresh_token is required")
|
||||
}
|
||||
return d.refresh(ctx)
|
||||
}
|
||||
|
||||
func (d *Driver) List(ctx context.Context, dirID string) ([]drives.Entry, error) {
|
||||
if dirID == "" {
|
||||
dirID = d.rootID
|
||||
}
|
||||
d.listMu.Lock()
|
||||
defer d.listMu.Unlock()
|
||||
|
||||
pageToken := ""
|
||||
out := make([]drives.Entry, 0)
|
||||
for {
|
||||
if err := d.waitForListSlotLocked(ctx); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
var resp filesResp
|
||||
err := d.request(ctx, d.filesURL(), http.MethodGet, func(req *resty.Request) {
|
||||
params := map[string]string{
|
||||
"fields": filesListFields,
|
||||
"pageSize": "1000",
|
||||
"q": fmt.Sprintf("'%s' in parents and trashed = false", strings.ReplaceAll(dirID, "'", "\\'")),
|
||||
"orderBy": "folder,name,modifiedTime desc",
|
||||
}
|
||||
if pageToken != "" {
|
||||
params["pageToken"] = pageToken
|
||||
}
|
||||
req.SetQueryParams(params)
|
||||
}, &resp)
|
||||
if err != nil {
|
||||
if wait, ok := drives.RateLimitRetryAfter(err); ok {
|
||||
if wait <= 0 {
|
||||
wait = d.listCooldown
|
||||
}
|
||||
if sleepErr := sleepContext(ctx, wait); sleepErr != nil {
|
||||
return nil, sleepErr
|
||||
}
|
||||
continue
|
||||
}
|
||||
return nil, fmt.Errorf("googledrive list: %w", err)
|
||||
}
|
||||
if err := d.fillShortcutFileMetadata(ctx, resp.Files); err != nil {
|
||||
return nil, fmt.Errorf("googledrive shortcut metadata: %w", err)
|
||||
}
|
||||
for _, f := range resp.Files {
|
||||
out = append(out, fileToEntry(f, dirID))
|
||||
}
|
||||
pageToken = resp.NextPageToken
|
||||
if pageToken == "" {
|
||||
return out, nil
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func (d *Driver) waitForListSlotLocked(ctx context.Context) error {
|
||||
if d.listInterval <= 0 || d.lastListAt.IsZero() {
|
||||
d.lastListAt = time.Now()
|
||||
return ctx.Err()
|
||||
}
|
||||
next := d.lastListAt.Add(d.listInterval)
|
||||
now := time.Now()
|
||||
if now.Before(next) {
|
||||
if err := sleepContext(ctx, next.Sub(now)); err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
d.lastListAt = time.Now()
|
||||
return ctx.Err()
|
||||
}
|
||||
|
||||
func sleepContext(ctx context.Context, d time.Duration) error {
|
||||
if d <= 0 {
|
||||
return ctx.Err()
|
||||
}
|
||||
timer := time.NewTimer(d)
|
||||
defer timer.Stop()
|
||||
select {
|
||||
case <-ctx.Done():
|
||||
return ctx.Err()
|
||||
case <-timer.C:
|
||||
return nil
|
||||
}
|
||||
}
|
||||
|
||||
func (d *Driver) Stat(ctx context.Context, fileID string) (*drives.Entry, error) {
|
||||
var f driveFile
|
||||
if err := d.request(ctx, d.fileURL(fileID), http.MethodGet, func(req *resty.Request) {
|
||||
req.SetQueryParam("fields", fileInfoFields)
|
||||
}, &f); err != nil {
|
||||
return nil, fmt.Errorf("googledrive stat: %w", err)
|
||||
}
|
||||
e := fileToEntry(f, "")
|
||||
return &e, nil
|
||||
}
|
||||
|
||||
func (d *Driver) StreamURL(ctx context.Context, fileID string) (*drives.StreamLink, error) {
|
||||
if fileID == "" {
|
||||
return nil, errors.New("googledrive stream: empty file id")
|
||||
}
|
||||
if _, err := d.Stat(ctx, fileID); err != nil {
|
||||
return nil, fmt.Errorf("googledrive stream: %w", err)
|
||||
}
|
||||
u := d.fileURL(fileID) + "?alt=media&acknowledgeAbuse=true&supportsAllDrives=true"
|
||||
return &drives.StreamLink{
|
||||
URL: u,
|
||||
Headers: http.Header{
|
||||
"Authorization": []string{"Bearer " + d.accessToken},
|
||||
},
|
||||
Expires: time.Now().Add(30 * time.Minute),
|
||||
}, nil
|
||||
}
|
||||
|
||||
func (d *Driver) Upload(context.Context, string, string, io.Reader, int64) (string, error) {
|
||||
return "", drives.ErrNotSupported
|
||||
}
|
||||
|
||||
func (d *Driver) EnsureDir(context.Context, string) (string, error) {
|
||||
return "", drives.ErrNotSupported
|
||||
}
|
||||
|
||||
func (d *Driver) refresh(ctx context.Context) error {
|
||||
if d.useOnlineAPI && d.renewAPIURL != "" {
|
||||
var out tokenResp
|
||||
res, err := d.client.R().
|
||||
SetContext(ctx).
|
||||
SetQueryParams(map[string]string{
|
||||
"refresh_ui": d.refreshToken,
|
||||
"server_use": "true",
|
||||
"driver_txt": "googleui_go",
|
||||
}).
|
||||
SetResult(&out).
|
||||
SetError(&out).
|
||||
Get(d.renewAPIURL)
|
||||
if err != nil {
|
||||
return fmt.Errorf("googledrive refresh token: %w", err)
|
||||
}
|
||||
if err := tokenResponseError("googledrive refresh token", res, out, true); err != nil {
|
||||
return err
|
||||
}
|
||||
d.applyToken(out)
|
||||
return nil
|
||||
}
|
||||
if d.clientID == "" || d.clientSecret == "" {
|
||||
return errors.New("googledrive refresh token: client_id and client_secret are required when online API is disabled")
|
||||
}
|
||||
var out tokenResp
|
||||
res, err := d.client.R().
|
||||
SetContext(ctx).
|
||||
SetFormData(map[string]string{
|
||||
"client_id": d.clientID,
|
||||
"client_secret": d.clientSecret,
|
||||
"refresh_token": d.refreshToken,
|
||||
"grant_type": "refresh_token",
|
||||
}).
|
||||
SetResult(&out).
|
||||
SetError(&out).
|
||||
Post(d.oauthURL)
|
||||
if err != nil {
|
||||
return fmt.Errorf("googledrive refresh token: %w", err)
|
||||
}
|
||||
if err := tokenResponseError("googledrive refresh token", res, out, false); err != nil {
|
||||
return err
|
||||
}
|
||||
d.applyToken(out)
|
||||
return nil
|
||||
}
|
||||
|
||||
func (d *Driver) applyToken(out tokenResp) {
|
||||
d.accessToken = out.AccessToken
|
||||
if strings.TrimSpace(out.RefreshToken) != "" {
|
||||
d.refreshToken = out.RefreshToken
|
||||
}
|
||||
if d.onTokenUpdate != nil {
|
||||
d.onTokenUpdate(d.accessToken, d.refreshToken)
|
||||
}
|
||||
}
|
||||
|
||||
func tokenResponseError(prefix string, res *resty.Response, out tokenResp, requireRefresh bool) error {
|
||||
if out.Text != "" {
|
||||
return fmt.Errorf("%s: %s", prefix, out.Text)
|
||||
}
|
||||
if out.Error != "" {
|
||||
if out.ErrorDescription != "" {
|
||||
return fmt.Errorf("%s: %s", prefix, out.ErrorDescription)
|
||||
}
|
||||
return fmt.Errorf("%s: %s", prefix, out.Error)
|
||||
}
|
||||
if res != nil && res.IsError() {
|
||||
return fmt.Errorf("%s: status=%d body=%s", prefix, res.StatusCode(), strings.TrimSpace(res.String()))
|
||||
}
|
||||
if out.AccessToken == "" || (requireRefresh && out.RefreshToken == "") {
|
||||
return fmt.Errorf("%s: empty token", prefix)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func (d *Driver) request(ctx context.Context, rawURL, method string, configure func(*resty.Request), out any) error {
|
||||
return d.requestOnce(ctx, rawURL, method, configure, out, true)
|
||||
}
|
||||
|
||||
func (d *Driver) requestOnce(ctx context.Context, rawURL, method string, configure func(*resty.Request), out any, retry bool) error {
|
||||
req := d.client.R().
|
||||
SetContext(ctx).
|
||||
SetHeader("Authorization", "Bearer "+d.accessToken).
|
||||
SetQueryParam("includeItemsFromAllDrives", "true").
|
||||
SetQueryParam("supportsAllDrives", "true")
|
||||
if configure != nil {
|
||||
configure(req)
|
||||
}
|
||||
if out != nil {
|
||||
req.SetResult(out)
|
||||
}
|
||||
var apiErr apiErrorResp
|
||||
req.SetError(&apiErr)
|
||||
res, err := req.Execute(method, rawURL)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if isGoogleRateLimit(res, apiErr.Error) {
|
||||
return googleRateLimitError(res, apiErr.Error.Message)
|
||||
}
|
||||
if apiErr.Error.Code != 0 {
|
||||
if apiErr.Error.Code == http.StatusUnauthorized && retry {
|
||||
if err := d.refresh(ctx); err != nil {
|
||||
return err
|
||||
}
|
||||
return d.requestOnce(ctx, rawURL, method, configure, out, false)
|
||||
}
|
||||
return googleAPIError(apiErr.Error)
|
||||
}
|
||||
if res.IsError() {
|
||||
return fmt.Errorf("google drive api error: status=%d body=%s", res.StatusCode(), strings.TrimSpace(res.String()))
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func (d *Driver) fillShortcutFileMetadata(ctx context.Context, files []driveFile) error {
|
||||
for i := range files {
|
||||
f := &files[i]
|
||||
if f.MimeType != "application/vnd.google-apps.shortcut" ||
|
||||
f.Shortcut.TargetID == "" ||
|
||||
f.Shortcut.TargetMimeType == "application/vnd.google-apps.folder" {
|
||||
continue
|
||||
}
|
||||
var target driveFile
|
||||
if err := d.request(ctx, d.fileURL(f.Shortcut.TargetID), http.MethodGet, func(req *resty.Request) {
|
||||
req.SetQueryParam("fields", fileInfoFields)
|
||||
}, &target); err != nil {
|
||||
return err
|
||||
}
|
||||
if target.Size != "" {
|
||||
f.Size = target.Size
|
||||
}
|
||||
if target.MD5Checksum != "" {
|
||||
f.MD5Checksum = target.MD5Checksum
|
||||
}
|
||||
if target.SHA1Checksum != "" {
|
||||
f.SHA1Checksum = target.SHA1Checksum
|
||||
}
|
||||
if target.SHA256Checksum != "" {
|
||||
f.SHA256Checksum = target.SHA256Checksum
|
||||
}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func (d *Driver) filesURL() string {
|
||||
return d.apiBaseURL + "/files"
|
||||
}
|
||||
|
||||
func (d *Driver) fileURL(fileID string) string {
|
||||
return d.filesURL() + "/" + url.PathEscape(fileID)
|
||||
}
|
||||
|
||||
func fileToEntry(f driveFile, fallbackParentID string) drives.Entry {
|
||||
id := f.ID
|
||||
isDir := f.MimeType == "application/vnd.google-apps.folder"
|
||||
if f.MimeType == "application/vnd.google-apps.shortcut" && f.Shortcut.TargetID != "" {
|
||||
id = f.Shortcut.TargetID
|
||||
isDir = f.Shortcut.TargetMimeType == "application/vnd.google-apps.folder"
|
||||
}
|
||||
size, _ := strconv.ParseInt(f.Size, 10, 64)
|
||||
hash := f.MD5Checksum
|
||||
if hash == "" {
|
||||
hash = f.SHA1Checksum
|
||||
}
|
||||
if hash == "" {
|
||||
hash = f.SHA256Checksum
|
||||
}
|
||||
return drives.Entry{
|
||||
ID: id,
|
||||
Name: f.Name,
|
||||
Size: size,
|
||||
Hash: hash,
|
||||
IsDir: isDir,
|
||||
ParentID: fallbackParentID,
|
||||
MimeType: mimeType(f),
|
||||
ModTime: f.ModifiedTime,
|
||||
ThumbnailURL: f.ThumbnailLink,
|
||||
}
|
||||
}
|
||||
|
||||
func mimeType(f driveFile) string {
|
||||
if f.MimeType != "" && f.MimeType != "application/vnd.google-apps.shortcut" {
|
||||
return f.MimeType
|
||||
}
|
||||
if f.Shortcut.TargetMimeType != "" {
|
||||
return f.Shortcut.TargetMimeType
|
||||
}
|
||||
ext := strings.ToLower(path.Ext(f.Name))
|
||||
switch ext {
|
||||
case ".mp4":
|
||||
return "video/mp4"
|
||||
case ".mkv":
|
||||
return "video/x-matroska"
|
||||
case ".mov":
|
||||
return "video/quicktime"
|
||||
case ".webm":
|
||||
return "video/webm"
|
||||
case ".avi":
|
||||
return "video/x-msvideo"
|
||||
case ".jpg", ".jpeg":
|
||||
return "image/jpeg"
|
||||
case ".png":
|
||||
return "image/png"
|
||||
default:
|
||||
return "application/octet-stream"
|
||||
}
|
||||
}
|
||||
|
||||
func isGoogleRateLimit(res *resty.Response, body apiErrorBody) bool {
|
||||
if res != nil && res.StatusCode() == http.StatusTooManyRequests {
|
||||
return true
|
||||
}
|
||||
if body.Code == http.StatusTooManyRequests {
|
||||
return true
|
||||
}
|
||||
for _, e := range body.Errors {
|
||||
reason := strings.ToLower(strings.TrimSpace(e.Reason))
|
||||
switch reason {
|
||||
case "ratelimitexceeded", "userratelimitexceeded", "downloadquotaexceeded", "sharingratelimitexceeded":
|
||||
return true
|
||||
}
|
||||
}
|
||||
msg := strings.ToLower(body.Message)
|
||||
return strings.Contains(msg, "rate limit") || strings.Contains(msg, "too many requests") || strings.Contains(msg, "quota exceeded")
|
||||
}
|
||||
|
||||
func googleRateLimitError(res *resty.Response, message string) error {
|
||||
if strings.TrimSpace(message) == "" {
|
||||
message = "google drive rate limited"
|
||||
}
|
||||
if res != nil && strings.TrimSpace(res.String()) != "" {
|
||||
message = fmt.Sprintf("%s: status=%d body=%s", message, res.StatusCode(), strings.TrimSpace(res.String()))
|
||||
}
|
||||
return &drives.RateLimitError{
|
||||
Provider: Kind,
|
||||
RetryAfter: parseRetryAfter(res),
|
||||
Err: errors.New(message),
|
||||
}
|
||||
}
|
||||
|
||||
func googleAPIError(body apiErrorBody) error {
|
||||
if body.Message != "" {
|
||||
return errors.New(body.Message)
|
||||
}
|
||||
if body.Code != 0 {
|
||||
return fmt.Errorf("google drive api error: code=%d", body.Code)
|
||||
}
|
||||
return errors.New("google drive api error")
|
||||
}
|
||||
|
||||
func parseRetryAfter(res *resty.Response) time.Duration {
|
||||
if res == nil {
|
||||
return 0
|
||||
}
|
||||
raw := strings.TrimSpace(res.Header().Get("Retry-After"))
|
||||
if raw == "" {
|
||||
return 0
|
||||
}
|
||||
if seconds, err := strconv.Atoi(raw); err == nil && seconds > 0 {
|
||||
return time.Duration(seconds) * time.Second
|
||||
}
|
||||
if when, err := http.ParseTime(raw); err == nil {
|
||||
d := time.Until(when)
|
||||
if d > 0 {
|
||||
return d
|
||||
}
|
||||
}
|
||||
return 0
|
||||
}
|
||||
|
||||
var _ drives.Drive = (*Driver)(nil)
|
||||
@@ -0,0 +1,190 @@
|
||||
package googledrive
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"net/http"
|
||||
"net/http/httptest"
|
||||
"strings"
|
||||
"testing"
|
||||
)
|
||||
|
||||
func TestInitUsesOnlineRenewAPI(t *testing.T) {
|
||||
var savedAccess, savedRefresh string
|
||||
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||
if r.URL.Path != "/renew" {
|
||||
t.Fatalf("unexpected path %s", r.URL.Path)
|
||||
}
|
||||
if got := r.URL.Query().Get("refresh_ui"); got != "old-refresh" {
|
||||
t.Fatalf("refresh_ui = %q", got)
|
||||
}
|
||||
if got := r.URL.Query().Get("server_use"); got != "true" {
|
||||
t.Fatalf("server_use = %q", got)
|
||||
}
|
||||
if got := r.URL.Query().Get("driver_txt"); got != "googleui_go" {
|
||||
t.Fatalf("driver_txt = %q", got)
|
||||
}
|
||||
writeTestJSON(w, tokenResp{
|
||||
AccessToken: "new-access",
|
||||
RefreshToken: "new-refresh",
|
||||
})
|
||||
}))
|
||||
defer srv.Close()
|
||||
|
||||
d := New(Config{
|
||||
ID: "g",
|
||||
RefreshToken: "old-refresh",
|
||||
UseOnlineAPI: true,
|
||||
RenewAPIURL: srv.URL + "/renew",
|
||||
OnTokenUpdate: func(access, refresh string) {
|
||||
savedAccess = access
|
||||
savedRefresh = refresh
|
||||
},
|
||||
})
|
||||
if err := d.Init(context.Background()); err != nil {
|
||||
t.Fatalf("Init() error = %v", err)
|
||||
}
|
||||
if d.accessToken != "new-access" || d.refreshToken != "new-refresh" {
|
||||
t.Fatalf("tokens not applied: access=%q refresh=%q", d.accessToken, d.refreshToken)
|
||||
}
|
||||
if savedAccess != "new-access" || savedRefresh != "new-refresh" {
|
||||
t.Fatalf("tokens not persisted: access=%q refresh=%q", savedAccess, savedRefresh)
|
||||
}
|
||||
}
|
||||
|
||||
func TestListMapsGoogleDriveFiles(t *testing.T) {
|
||||
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||
if got := r.Header.Get("Authorization"); got != "Bearer access" {
|
||||
t.Fatalf("Authorization = %q", got)
|
||||
}
|
||||
if r.URL.Path != "/drive/v3/files" {
|
||||
t.Fatalf("unexpected path %s", r.URL.Path)
|
||||
}
|
||||
if !strings.Contains(r.URL.Query().Get("q"), "'root' in parents") {
|
||||
t.Fatalf("unexpected q = %q", r.URL.Query().Get("q"))
|
||||
}
|
||||
writeTestJSON(w, filesResp{Files: []driveFile{
|
||||
{ID: "folder-1", Name: "Movies", MimeType: "application/vnd.google-apps.folder"},
|
||||
{
|
||||
ID: "file-1",
|
||||
Name: "clip.mp4",
|
||||
MimeType: "video/mp4",
|
||||
Size: "1234",
|
||||
MD5Checksum: "abc",
|
||||
ThumbnailLink: "https://thumb.example/1",
|
||||
},
|
||||
}})
|
||||
}))
|
||||
defer srv.Close()
|
||||
|
||||
d := New(Config{ID: "g", RootID: "root", APIBaseURL: srv.URL + "/drive/v3"})
|
||||
d.accessToken = "access"
|
||||
d.listInterval = -1
|
||||
|
||||
entries, err := d.List(context.Background(), "")
|
||||
if err != nil {
|
||||
t.Fatalf("List() error = %v", err)
|
||||
}
|
||||
if len(entries) != 2 {
|
||||
t.Fatalf("len(entries) = %d", len(entries))
|
||||
}
|
||||
if !entries[0].IsDir || entries[0].ID != "folder-1" {
|
||||
t.Fatalf("folder entry = %+v", entries[0])
|
||||
}
|
||||
if entries[1].ID != "file-1" || entries[1].Size != 1234 || entries[1].Hash != "abc" || entries[1].ThumbnailURL == "" {
|
||||
t.Fatalf("file entry = %+v", entries[1])
|
||||
}
|
||||
}
|
||||
|
||||
func TestStreamURLReturnsAuthenticatedMediaLinkWithoutRedirectRequirement(t *testing.T) {
|
||||
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||
if got := r.Header.Get("Authorization"); got != "Bearer access" {
|
||||
t.Fatalf("Authorization = %q", got)
|
||||
}
|
||||
if r.URL.Path != "/drive/v3/files/file-1" {
|
||||
t.Fatalf("unexpected path %s", r.URL.Path)
|
||||
}
|
||||
writeTestJSON(w, driveFile{
|
||||
ID: "file-1",
|
||||
Name: "clip.mp4",
|
||||
MimeType: "video/mp4",
|
||||
Size: "1234",
|
||||
})
|
||||
}))
|
||||
defer srv.Close()
|
||||
|
||||
d := New(Config{ID: "g", APIBaseURL: srv.URL + "/drive/v3"})
|
||||
d.accessToken = "access"
|
||||
|
||||
link, err := d.StreamURL(context.Background(), "file-1")
|
||||
if err != nil {
|
||||
t.Fatalf("StreamURL() error = %v", err)
|
||||
}
|
||||
if !strings.HasPrefix(link.URL, srv.URL+"/drive/v3/files/file-1?") {
|
||||
t.Fatalf("link URL = %q", link.URL)
|
||||
}
|
||||
if !strings.Contains(link.URL, "alt=media") {
|
||||
t.Fatalf("link URL missing alt=media: %q", link.URL)
|
||||
}
|
||||
if got := link.Headers.Get("Authorization"); got != "Bearer access" {
|
||||
t.Fatalf("link Authorization = %q", got)
|
||||
}
|
||||
}
|
||||
|
||||
func TestRequestRefreshesOnUnauthorized(t *testing.T) {
|
||||
var fileCalls int
|
||||
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||
switch r.URL.Path {
|
||||
case "/renew":
|
||||
writeTestJSON(w, tokenResp{
|
||||
AccessToken: "new-access",
|
||||
RefreshToken: "new-refresh",
|
||||
})
|
||||
case "/drive/v3/files/file-1":
|
||||
fileCalls++
|
||||
if fileCalls == 1 {
|
||||
writeTestJSONStatus(w, http.StatusUnauthorized, apiErrorResp{Error: apiErrorBody{
|
||||
Code: http.StatusUnauthorized,
|
||||
Message: "Invalid Credentials",
|
||||
}})
|
||||
return
|
||||
}
|
||||
if got := r.Header.Get("Authorization"); got != "Bearer new-access" {
|
||||
t.Fatalf("Authorization after refresh = %q", got)
|
||||
}
|
||||
writeTestJSON(w, driveFile{ID: "file-1", Name: "clip.mp4", Size: "1"})
|
||||
default:
|
||||
t.Fatalf("unexpected path %s", r.URL.Path)
|
||||
}
|
||||
}))
|
||||
defer srv.Close()
|
||||
|
||||
d := New(Config{
|
||||
ID: "g",
|
||||
RefreshToken: "old-refresh",
|
||||
UseOnlineAPI: true,
|
||||
RenewAPIURL: srv.URL + "/renew",
|
||||
APIBaseURL: srv.URL + "/drive/v3",
|
||||
})
|
||||
d.accessToken = "old-access"
|
||||
|
||||
if _, err := d.Stat(context.Background(), "file-1"); err != nil {
|
||||
t.Fatalf("Stat() error = %v", err)
|
||||
}
|
||||
if fileCalls != 2 {
|
||||
t.Fatalf("fileCalls = %d", fileCalls)
|
||||
}
|
||||
if d.accessToken != "new-access" || d.refreshToken != "new-refresh" {
|
||||
t.Fatalf("tokens not refreshed: access=%q refresh=%q", d.accessToken, d.refreshToken)
|
||||
}
|
||||
}
|
||||
|
||||
func writeTestJSON(w http.ResponseWriter, v any) {
|
||||
writeTestJSONStatus(w, http.StatusOK, v)
|
||||
}
|
||||
|
||||
func writeTestJSONStatus(w http.ResponseWriter, status int, v any) {
|
||||
w.Header().Set("Content-Type", "application/json")
|
||||
w.WriteHeader(status)
|
||||
_ = json.NewEncoder(w).Encode(v)
|
||||
}
|
||||
@@ -0,0 +1,49 @@
|
||||
package googledrive
|
||||
|
||||
import "time"
|
||||
|
||||
type tokenResp struct {
|
||||
AccessToken string `json:"access_token"`
|
||||
RefreshToken string `json:"refresh_token"`
|
||||
ExpiresIn int64 `json:"expires_in"`
|
||||
Error string `json:"error"`
|
||||
ErrorDescription string `json:"error_description"`
|
||||
Text string `json:"text"`
|
||||
}
|
||||
|
||||
type filesResp struct {
|
||||
NextPageToken string `json:"nextPageToken"`
|
||||
Files []driveFile `json:"files"`
|
||||
Error apiErrorBody `json:"error"`
|
||||
}
|
||||
|
||||
type driveFile struct {
|
||||
ID string `json:"id"`
|
||||
Name string `json:"name"`
|
||||
MimeType string `json:"mimeType"`
|
||||
ModifiedTime time.Time `json:"modifiedTime"`
|
||||
CreatedTime time.Time `json:"createdTime"`
|
||||
Size string `json:"size"`
|
||||
ThumbnailLink string `json:"thumbnailLink"`
|
||||
MD5Checksum string `json:"md5Checksum"`
|
||||
SHA1Checksum string `json:"sha1Checksum"`
|
||||
SHA256Checksum string `json:"sha256Checksum"`
|
||||
Shortcut struct {
|
||||
TargetID string `json:"targetId"`
|
||||
TargetMimeType string `json:"targetMimeType"`
|
||||
} `json:"shortcutDetails"`
|
||||
}
|
||||
|
||||
type apiErrorResp struct {
|
||||
Error apiErrorBody `json:"error"`
|
||||
}
|
||||
|
||||
type apiErrorBody struct {
|
||||
Code int `json:"code"`
|
||||
Message string `json:"message"`
|
||||
Errors []struct {
|
||||
Domain string `json:"domain"`
|
||||
Reason string `json:"reason"`
|
||||
Message string `json:"message"`
|
||||
} `json:"errors"`
|
||||
}
|
||||
@@ -10,7 +10,7 @@ import (
|
||||
|
||||
// Drive 是多家网盘统一抽象。上层不区分盘,只区分 Kind。
|
||||
type Drive interface {
|
||||
// Kind 返回驱动代号:"quark" / "p115" / "pikpak" / "wopan" / "onedrive" / "localstorage"
|
||||
// Kind 返回驱动代号:"quark" / "p115" / "pikpak" / "wopan" / "onedrive" / "googledrive" / "localstorage"
|
||||
Kind() string
|
||||
|
||||
// ID 返回该盘在 catalog 中的唯一标识
|
||||
|
||||
@@ -96,11 +96,25 @@ type Config struct {
|
||||
Now func() time.Time
|
||||
}
|
||||
|
||||
type Status struct {
|
||||
State string
|
||||
Running bool
|
||||
Queued bool
|
||||
StartedAt time.Time
|
||||
LastFinishedAt time.Time
|
||||
}
|
||||
|
||||
// Runner drives the nightly pipeline.
|
||||
type Runner struct {
|
||||
cfg Config
|
||||
trigger chan struct{} // buffered(1); manual "run now"
|
||||
runMu sync.Mutex // prevents overlapping pipeline runs
|
||||
|
||||
stateMu sync.Mutex
|
||||
running bool
|
||||
queued bool
|
||||
startedAt time.Time
|
||||
lastFinishedAt time.Time
|
||||
}
|
||||
|
||||
// New constructs a Runner. cfg is shallow-copied; defaults are applied.
|
||||
@@ -138,14 +152,53 @@ func (r *Runner) Run(ctx context.Context) {
|
||||
}
|
||||
}
|
||||
|
||||
// TriggerNow asks the running loop to fire a pipeline ASAP. The trigger channel
|
||||
// is buffered(1): if a pipeline is already in progress, one follow-up run may
|
||||
// remain pending and will start after the current run finishes. Additional
|
||||
// clicks while that follow-up is pending are ignored.
|
||||
func (r *Runner) TriggerNow() {
|
||||
// TriggerNow asks the running loop to fire a pipeline ASAP. Only one manual
|
||||
// trigger can be active at a time: if a pipeline is already running or waiting
|
||||
// in the trigger channel, the request is ignored and returns false.
|
||||
func (r *Runner) TriggerNow() bool {
|
||||
r.stateMu.Lock()
|
||||
if r.running || r.queued {
|
||||
r.stateMu.Unlock()
|
||||
return false
|
||||
}
|
||||
r.queued = true
|
||||
r.stateMu.Unlock()
|
||||
|
||||
select {
|
||||
case r.trigger <- struct{}{}:
|
||||
return true
|
||||
default:
|
||||
r.stateMu.Lock()
|
||||
r.queued = false
|
||||
r.stateMu.Unlock()
|
||||
return false
|
||||
}
|
||||
}
|
||||
|
||||
func (r *Runner) Status() Status {
|
||||
r.stateMu.Lock()
|
||||
running := r.running
|
||||
queued := r.queued
|
||||
startedAt := r.startedAt
|
||||
lastFinishedAt := r.lastFinishedAt
|
||||
r.stateMu.Unlock()
|
||||
|
||||
state := "idle"
|
||||
switch {
|
||||
case running && queued:
|
||||
state = "running_queued"
|
||||
case running:
|
||||
state = "running"
|
||||
case queued:
|
||||
state = "queued"
|
||||
}
|
||||
|
||||
return Status{
|
||||
State: state,
|
||||
Running: running,
|
||||
Queued: queued,
|
||||
StartedAt: startedAt,
|
||||
LastFinishedAt: lastFinishedAt,
|
||||
}
|
||||
}
|
||||
|
||||
@@ -183,9 +236,13 @@ func (r *Runner) runPipelineLocked(ctx context.Context, manual bool) {
|
||||
log.Printf("[nightly] another pipeline is already running, skipping this trigger")
|
||||
return
|
||||
}
|
||||
defer r.runMu.Unlock()
|
||||
|
||||
started := r.cfg.Now()
|
||||
r.markStarted(started)
|
||||
defer func() {
|
||||
r.markFinished(r.cfg.Now())
|
||||
r.runMu.Unlock()
|
||||
}()
|
||||
|
||||
mode := "scheduled"
|
||||
if manual {
|
||||
@@ -207,6 +264,22 @@ func (r *Runner) runPipelineLocked(ctx context.Context, manual bool) {
|
||||
}
|
||||
}
|
||||
|
||||
func (r *Runner) markStarted(started time.Time) {
|
||||
r.stateMu.Lock()
|
||||
defer r.stateMu.Unlock()
|
||||
r.running = true
|
||||
r.queued = false
|
||||
r.startedAt = started
|
||||
}
|
||||
|
||||
func (r *Runner) markFinished(finished time.Time) {
|
||||
r.stateMu.Lock()
|
||||
defer r.stateMu.Unlock()
|
||||
r.running = false
|
||||
r.startedAt = time.Time{}
|
||||
r.lastFinishedAt = finished
|
||||
}
|
||||
|
||||
// runPipeline executes the three phases. It returns when the pipeline finishes
|
||||
// OR ctx is done (deadline / cancel). Errors are logged but not propagated —
|
||||
// each phase is best-effort; downstream phases still attempt to run unless ctx
|
||||
|
||||
@@ -312,11 +312,14 @@ func TestCtxCancelPreventsLaterPhases(t *testing.T) {
|
||||
func TestTriggerNowIsNonBlocking(t *testing.T) {
|
||||
r := New(Config{Settings: newStubSettings()})
|
||||
// fill the trigger channel
|
||||
r.TriggerNow()
|
||||
if !r.TriggerNow() {
|
||||
t.Fatal("first TriggerNow should be accepted")
|
||||
}
|
||||
// Second call must not block
|
||||
done := make(chan struct{})
|
||||
var accepted bool
|
||||
go func() {
|
||||
r.TriggerNow()
|
||||
accepted = r.TriggerNow()
|
||||
close(done)
|
||||
}()
|
||||
select {
|
||||
@@ -324,4 +327,98 @@ func TestTriggerNowIsNonBlocking(t *testing.T) {
|
||||
case <-time.After(100 * time.Millisecond):
|
||||
t.Fatal("TriggerNow blocked when channel is full")
|
||||
}
|
||||
if accepted {
|
||||
t.Fatal("second TriggerNow should be ignored when trigger channel is full")
|
||||
}
|
||||
}
|
||||
|
||||
func TestStatusTracksQueuedRunningAndFinished(t *testing.T) {
|
||||
blockScan := make(chan struct{})
|
||||
scanStarted := make(chan struct{})
|
||||
var startedOnce sync.Once
|
||||
r := New(Config{
|
||||
Settings: newStubSettings(),
|
||||
ListScanTargets: func(context.Context) []string {
|
||||
return []string{"drive"}
|
||||
},
|
||||
RunScan: func(context.Context, string) {
|
||||
startedOnce.Do(func() { close(scanStarted) })
|
||||
<-blockScan
|
||||
},
|
||||
})
|
||||
|
||||
if got := r.Status(); got.State != "idle" || got.Running || got.Queued {
|
||||
t.Fatalf("initial status = %#v, want idle", got)
|
||||
}
|
||||
|
||||
if !r.TriggerNow() {
|
||||
t.Fatal("TriggerNow should queue a manual run")
|
||||
}
|
||||
if got := r.Status(); got.State != "queued" || got.Running || !got.Queued {
|
||||
t.Fatalf("queued status = %#v, want queued", got)
|
||||
}
|
||||
|
||||
ctx, cancel := context.WithCancel(context.Background())
|
||||
defer cancel()
|
||||
go r.Run(ctx)
|
||||
|
||||
select {
|
||||
case <-scanStarted:
|
||||
case <-time.After(time.Second):
|
||||
t.Fatal("pipeline did not start")
|
||||
}
|
||||
|
||||
if got := r.Status(); got.State != "running" || !got.Running || got.Queued || got.StartedAt.IsZero() {
|
||||
t.Fatalf("running status = %#v, want running with startedAt", got)
|
||||
}
|
||||
|
||||
if r.TriggerNow() {
|
||||
t.Fatal("TriggerNow during a run should be ignored")
|
||||
}
|
||||
if got := r.Status(); got.State != "running" || !got.Running || got.Queued {
|
||||
t.Fatalf("status after ignored trigger = %#v, want running", got)
|
||||
}
|
||||
|
||||
close(blockScan)
|
||||
deadline := time.After(time.Second)
|
||||
for {
|
||||
got := r.Status()
|
||||
if !got.Running && !got.Queued && !got.LastFinishedAt.IsZero() {
|
||||
return
|
||||
}
|
||||
select {
|
||||
case <-deadline:
|
||||
t.Fatalf("status did not finish; got=%#v", got)
|
||||
default:
|
||||
time.Sleep(10 * time.Millisecond)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestTriggerNowAcceptsOnlyOneConcurrentRequest(t *testing.T) {
|
||||
r := New(Config{Settings: newStubSettings()})
|
||||
|
||||
const callers = 16
|
||||
start := make(chan struct{})
|
||||
results := make(chan bool, callers)
|
||||
for i := 0; i < callers; i++ {
|
||||
go func() {
|
||||
<-start
|
||||
results <- r.TriggerNow()
|
||||
}()
|
||||
}
|
||||
close(start)
|
||||
|
||||
accepted := 0
|
||||
for i := 0; i < callers; i++ {
|
||||
if <-results {
|
||||
accepted++
|
||||
}
|
||||
}
|
||||
if accepted != 1 {
|
||||
t.Fatalf("accepted triggers = %d, want 1", accepted)
|
||||
}
|
||||
if got := r.Status(); got.State != "queued" || got.Running || !got.Queued {
|
||||
t.Fatalf("status = %#v, want one queued trigger", got)
|
||||
}
|
||||
}
|
||||
|
||||
@@ -1,6 +1,7 @@
|
||||
package preview
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"context"
|
||||
"encoding/json"
|
||||
"errors"
|
||||
@@ -345,9 +346,15 @@ func (g *Generator) Probe(ctx context.Context, link *drives.StreamLink) (float64
|
||||
args = append(args, ffmpegLink.URL)
|
||||
|
||||
cmd := exec.CommandContext(ctx2, g.cfg.FFprobePath, args...)
|
||||
out, err := cmd.CombinedOutput()
|
||||
var stderr bytes.Buffer
|
||||
cmd.Stderr = &stderr
|
||||
out, err := cmd.Output()
|
||||
if err != nil {
|
||||
return 0, ffmpegCommandError("ffprobe", err, out)
|
||||
errOut := stderr.Bytes()
|
||||
if len(errOut) == 0 {
|
||||
errOut = out
|
||||
}
|
||||
return 0, ffmpegCommandError("ffprobe", err, errOut)
|
||||
}
|
||||
raw := strings.TrimSpace(string(out))
|
||||
if raw == "" || raw == "N/A" {
|
||||
@@ -1033,11 +1040,12 @@ type ThumbWorker struct {
|
||||
}
|
||||
|
||||
const (
|
||||
defaultTransientMediaCooldown = 5 * time.Minute
|
||||
defaultGenerationRateLimitCooldown = 5 * time.Minute
|
||||
defaultWorkerQueueSize = 10000
|
||||
maxPreviewTeaserSizeBytes int64 = 5 * 1024 * 1024 * 1024
|
||||
previewStatusSkipped = "skipped"
|
||||
defaultTransientMediaCooldown = 5 * time.Minute
|
||||
defaultGenerationRateLimitCooldown = 5 * time.Minute
|
||||
defaultThumbTransientMediaMaxFailures = 3
|
||||
defaultWorkerQueueSize = 10000
|
||||
maxPreviewTeaserSizeBytes int64 = 5 * 1024 * 1024 * 1024
|
||||
previewStatusSkipped = "skipped"
|
||||
)
|
||||
|
||||
type rateLimitState struct {
|
||||
@@ -1339,13 +1347,16 @@ func (w *Worker) processQueued(ctx context.Context, v *catalog.Video) {
|
||||
}
|
||||
|
||||
func (w *ThumbWorker) processQueued(ctx context.Context, v *catalog.Video) {
|
||||
defer w.queue.release(v)
|
||||
w.activity.start(v)
|
||||
defer w.activity.done()
|
||||
if !waitForRateLimitCooldown(ctx, &w.rateLimit, "thumb", w.Drive) {
|
||||
return
|
||||
retry := false
|
||||
if waitForRateLimitCooldown(ctx, &w.rateLimit, "thumb", w.Drive) {
|
||||
retry = w.process(ctx, v)
|
||||
}
|
||||
w.activity.done()
|
||||
w.queue.release(v)
|
||||
if retry && ctx.Err() == nil {
|
||||
w.EnqueueBlocking(ctx, v)
|
||||
}
|
||||
w.process(ctx, v)
|
||||
}
|
||||
|
||||
func waitForRateLimitCooldown(ctx context.Context, state *rateLimitState, label string, drive drives.Drive) bool {
|
||||
@@ -1427,15 +1438,34 @@ func (w *ThumbWorker) pauseForRateLimit(err error, step, title string) bool {
|
||||
return true
|
||||
}
|
||||
|
||||
func (w *ThumbWorker) pauseForRecoverableError(err error, step, title string) bool {
|
||||
func (w *ThumbWorker) pauseForRecoverableError(ctx context.Context, v *catalog.Video, err error, step string) bool {
|
||||
title := ""
|
||||
videoID := ""
|
||||
if v != nil {
|
||||
title = v.Title
|
||||
videoID = v.ID
|
||||
}
|
||||
if w.pauseForRateLimit(err, step, title) {
|
||||
return true
|
||||
}
|
||||
if !driveErrorShouldCooldown(w.Drive, err) {
|
||||
return false
|
||||
}
|
||||
failures := 1
|
||||
if w.Catalog != nil && videoID != "" {
|
||||
count, countErr := w.Catalog.IncrementThumbnailFailures(ctx, videoID)
|
||||
if countErr != nil {
|
||||
log.Printf("[thumb] drive=%s transient media source error count failed step=%s video=%s: %v", w.Drive.ID(), step, title, countErr)
|
||||
} else {
|
||||
failures = count
|
||||
}
|
||||
}
|
||||
if failures >= defaultThumbTransientMediaMaxFailures {
|
||||
log.Printf("[thumb] drive=%s transient media source error reached retry limit failures=%d/%d step=%s video=%s: %v", w.Drive.ID(), failures, defaultThumbTransientMediaMaxFailures, step, title, err)
|
||||
return false
|
||||
}
|
||||
until := w.rateLimit.pause(time.Now(), w.RateLimitCooldown)
|
||||
log.Printf("[thumb] drive=%s transient media source error until=%s step=%s video=%s: %v", w.Drive.ID(), until.Format(time.RFC3339), step, title, err)
|
||||
log.Printf("[thumb] drive=%s transient media source error until=%s failures=%d/%d step=%s video=%s: %v", w.Drive.ID(), until.Format(time.RFC3339), failures, defaultThumbTransientMediaMaxFailures, step, title, err)
|
||||
return true
|
||||
}
|
||||
|
||||
@@ -1481,9 +1511,9 @@ func driveErrorShouldCooldown(d drives.Drive, err error) bool {
|
||||
return false
|
||||
}
|
||||
|
||||
func (w *ThumbWorker) process(ctx context.Context, v *catalog.Video) {
|
||||
func (w *ThumbWorker) process(ctx context.Context, v *catalog.Video) bool {
|
||||
if w.skipIfRateLimited(v) {
|
||||
return
|
||||
return false
|
||||
}
|
||||
queued := v
|
||||
current := v
|
||||
@@ -1495,54 +1525,64 @@ func (w *ThumbWorker) process(ctx context.Context, v *catalog.Video) {
|
||||
v = loaded
|
||||
if loaded.ThumbnailURL != "" && loaded.DurationSeconds > 0 {
|
||||
_ = w.Catalog.UpdateVideoMeta(ctx, v.ID, catalog.VideoMetaPatch{ThumbnailStatus: "ready"})
|
||||
return
|
||||
return false
|
||||
}
|
||||
}
|
||||
if current.ThumbnailURL != "" {
|
||||
durationBackfillFailed := false
|
||||
if current.DurationSeconds <= 0 {
|
||||
link, err := w.streamLink(ctx, current)
|
||||
if err != nil {
|
||||
if w.pauseForRecoverableError(err, "streamURL", current.Title) {
|
||||
return
|
||||
if w.pauseForRecoverableError(ctx, current, err, "streamURL") {
|
||||
return true
|
||||
}
|
||||
log.Printf("[thumb] probe streamURL %s: %v", current.Title, err)
|
||||
durationBackfillFailed = true
|
||||
} else if w.probeDuration(ctx, current, link) {
|
||||
return
|
||||
return true
|
||||
} else if current.DurationSeconds <= 0 {
|
||||
durationBackfillFailed = true
|
||||
}
|
||||
}
|
||||
if durationBackfillFailed {
|
||||
log.Printf("[thumb] skip duration backfill %s: thumbnail already exists but duration could not be probed", current.Title)
|
||||
_ = w.Catalog.UpdateVideoMeta(ctx, current.ID, catalog.VideoMetaPatch{ThumbnailStatus: "skipped"})
|
||||
return false
|
||||
}
|
||||
_ = w.Catalog.UpdateVideoMeta(ctx, current.ID, catalog.VideoMetaPatch{ThumbnailStatus: "ready"})
|
||||
return
|
||||
return false
|
||||
}
|
||||
_ = w.Catalog.UpdateVideoMeta(ctx, v.ID, catalog.VideoMetaPatch{ThumbnailStatus: "pending"})
|
||||
link, err := w.streamLink(ctx, v)
|
||||
if err != nil {
|
||||
if w.pauseForRecoverableError(err, "streamURL", v.Title) {
|
||||
return
|
||||
if w.pauseForRecoverableError(ctx, v, err, "streamURL") {
|
||||
return true
|
||||
}
|
||||
log.Printf("[thumb] streamURL %s: %v", v.Title, err)
|
||||
_ = w.Catalog.UpdateVideoMeta(ctx, v.ID, catalog.VideoMetaPatch{ThumbnailStatus: "failed"})
|
||||
return
|
||||
return false
|
||||
}
|
||||
if w.probeDuration(ctx, v, link) {
|
||||
return
|
||||
return true
|
||||
}
|
||||
|
||||
if err := w.generateThumbnailFromLink(ctx, v, link); err != nil {
|
||||
if localLink, ok := localPreviewLink(v); ok && link.URL != localLink.URL {
|
||||
if w.probeDuration(ctx, v, localLink) {
|
||||
return
|
||||
return true
|
||||
}
|
||||
if localErr := w.generateThumbnailFromLink(ctx, v, localLink); localErr == nil {
|
||||
return
|
||||
return false
|
||||
}
|
||||
}
|
||||
if w.pauseForRecoverableError(err, "generate", v.Title) {
|
||||
return
|
||||
if w.pauseForRecoverableError(ctx, v, err, "generate") {
|
||||
return true
|
||||
}
|
||||
log.Printf("[thumb] generate %s: %v", v.Title, err)
|
||||
_ = w.Catalog.UpdateVideoMeta(ctx, v.ID, catalog.VideoMetaPatch{ThumbnailStatus: "failed"})
|
||||
return
|
||||
return false
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
func (w *ThumbWorker) streamLink(ctx context.Context, v *catalog.Video) (*drives.StreamLink, error) {
|
||||
@@ -1570,7 +1610,7 @@ func (w *ThumbWorker) probeDuration(ctx context.Context, v *catalog.Video, link
|
||||
}
|
||||
return false
|
||||
}
|
||||
if w.pauseForRecoverableError(err, "probe", v.Title) {
|
||||
if w.pauseForRecoverableError(ctx, v, err, "probe") {
|
||||
return true
|
||||
}
|
||||
log.Printf("[thumb] probe %s: %v", v.Title, err)
|
||||
|
||||
@@ -5,6 +5,8 @@ import (
|
||||
"errors"
|
||||
"math"
|
||||
"net/http"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
@@ -95,6 +97,24 @@ func TestTinyVideoPreviewPlanUsesWholeVideoAsSingleSegment(t *testing.T) {
|
||||
}
|
||||
}
|
||||
|
||||
func TestProbeIgnoresStderrWarnings(t *testing.T) {
|
||||
dir := t.TempDir()
|
||||
ffprobePath := filepath.Join(dir, "ffprobe")
|
||||
script := "#!/bin/sh\nprintf '%s\\n' 'h264 warning' >&2\nprintf '%s\\n' '364.800000'\n"
|
||||
if err := os.WriteFile(ffprobePath, []byte(script), 0o755); err != nil {
|
||||
t.Fatalf("write ffprobe stub: %v", err)
|
||||
}
|
||||
|
||||
gen := New(Config{FFprobePath: ffprobePath})
|
||||
got, err := gen.Probe(context.Background(), &drives.StreamLink{URL: filepath.Join(dir, "video.mp4")})
|
||||
if err != nil {
|
||||
t.Fatalf("probe: %v", err)
|
||||
}
|
||||
if got != 364.8 {
|
||||
t.Fatalf("duration = %v, want 364.8", got)
|
||||
}
|
||||
}
|
||||
|
||||
func TestTeaserCandidateStartsKeepPrimaryAndAddFallbacks(t *testing.T) {
|
||||
primary := []float64{10.2, 64.65, 119.1, 173.55}
|
||||
got := teaserCandidateStarts(204, primary, 3)
|
||||
|
||||
@@ -89,6 +89,46 @@ func TestThumbWorkerBackfillsDurationWhenThumbnailAlreadyExists(t *testing.T) {
|
||||
}
|
||||
}
|
||||
|
||||
func TestThumbWorkerSkipsDurationBackfillWhenExistingThumbnailCannotBeProbed(t *testing.T) {
|
||||
ctx := context.Background()
|
||||
cat, video := seedPreviewTestVideo(t, "thumb-worker-existing-thumbnail-probe-fails")
|
||||
video.ThumbnailURL = "/p/thumb/" + video.ID
|
||||
if err := cat.UpsertVideo(ctx, video); err != nil {
|
||||
t.Fatalf("update video: %v", err)
|
||||
}
|
||||
|
||||
gen := &fakeThumbGenerator{probeErr: errors.New("invalid media")}
|
||||
drv := &previewFakeDrive{}
|
||||
worker := NewThumbWorker(gen, cat, drv)
|
||||
|
||||
worker.process(ctx, video)
|
||||
|
||||
got, err := cat.GetVideo(ctx, video.ID)
|
||||
if err != nil {
|
||||
t.Fatalf("get video: %v", err)
|
||||
}
|
||||
if got.ThumbnailURL != "/p/thumb/"+video.ID {
|
||||
t.Fatalf("thumbnail = %q, want unchanged existing thumbnail", got.ThumbnailURL)
|
||||
}
|
||||
if got.DurationSeconds != 0 {
|
||||
t.Fatalf("duration = %d, want still unknown", got.DurationSeconds)
|
||||
}
|
||||
skipped, err := cat.ListVideosByThumbnailStatus(ctx, video.DriveID, "skipped", 0)
|
||||
if err != nil {
|
||||
t.Fatalf("list skipped thumbnails: %v", err)
|
||||
}
|
||||
if len(skipped) != 1 || skipped[0].ID != video.ID {
|
||||
t.Fatalf("skipped thumbnails = %#v, want only %s", skipped, video.ID)
|
||||
}
|
||||
missing, err := cat.CountVideosNeedingThumbnail(ctx, video.DriveID)
|
||||
if err != nil {
|
||||
t.Fatalf("count videos needing thumbnail: %v", err)
|
||||
}
|
||||
if missing != 0 {
|
||||
t.Fatalf("missing thumbnails = %d, want 0 after duration backfill is skipped", missing)
|
||||
}
|
||||
}
|
||||
|
||||
func TestThumbWorkerFallsBackToLocalPreviewWhenDriveStreamFails(t *testing.T) {
|
||||
ctx := context.Background()
|
||||
cat, video := seedPreviewTestVideo(t, "thumb-worker-local-preview")
|
||||
@@ -416,6 +456,113 @@ func TestThumbWorkerRateLimitCoolsDownFiveMinutes(t *testing.T) {
|
||||
assertCooldownAround(t, worker.Status().CooldownUntil, before, 5*time.Minute)
|
||||
}
|
||||
|
||||
func TestThumbWorkerP115TransientErrorFailsAfterRetryLimit(t *testing.T) {
|
||||
ctx := context.Background()
|
||||
cat, video := seedPreviewTestVideo(t, "thumb-p115-transient")
|
||||
|
||||
gen := &fakeThumbGenerator{
|
||||
generateErr: errors.New("ffmpeg thumb: exit status 183, stderr: partial file Cannot determine format of input 0:0 after EOF"),
|
||||
}
|
||||
drv := &previewFakeDrive{kind: "p115"}
|
||||
worker := NewThumbWorker(gen, cat, drv)
|
||||
|
||||
for attempt := 1; attempt <= defaultThumbTransientMediaMaxFailures; attempt++ {
|
||||
worker.rateLimit = rateLimitState{}
|
||||
worker.process(ctx, video)
|
||||
|
||||
if attempt < defaultThumbTransientMediaMaxFailures {
|
||||
pending, err := cat.ListVideosByThumbnailStatus(ctx, video.DriveID, "pending", 0)
|
||||
if err != nil {
|
||||
t.Fatalf("list pending thumbnails: %v", err)
|
||||
}
|
||||
if len(pending) != 1 || pending[0].ID != video.ID {
|
||||
t.Fatalf("attempt %d pending thumbnails = %#v, want only %s", attempt, pending, video.ID)
|
||||
}
|
||||
missing, err := cat.CountVideosNeedingThumbnail(ctx, video.DriveID)
|
||||
if err != nil {
|
||||
t.Fatalf("count missing thumbnails: %v", err)
|
||||
}
|
||||
if missing != 1 {
|
||||
t.Fatalf("attempt %d missing thumbnails = %d, want 1 before retry limit", attempt, missing)
|
||||
}
|
||||
continue
|
||||
}
|
||||
|
||||
failed, err := cat.ListVideosByThumbnailStatus(ctx, video.DriveID, "failed", 0)
|
||||
if err != nil {
|
||||
t.Fatalf("list failed thumbnails: %v", err)
|
||||
}
|
||||
if len(failed) != 1 || failed[0].ID != video.ID {
|
||||
t.Fatalf("failed thumbnails = %#v, want only %s", failed, video.ID)
|
||||
}
|
||||
missing, err := cat.CountVideosNeedingThumbnail(ctx, video.DriveID)
|
||||
if err != nil {
|
||||
t.Fatalf("count missing thumbnails: %v", err)
|
||||
}
|
||||
if missing != 0 {
|
||||
t.Fatalf("missing thumbnails = %d, want 0 after retry limit marks failed", missing)
|
||||
}
|
||||
}
|
||||
|
||||
if gen.generateCalls != defaultThumbTransientMediaMaxFailures {
|
||||
t.Fatalf("generate calls = %d, want %d", gen.generateCalls, defaultThumbTransientMediaMaxFailures)
|
||||
}
|
||||
|
||||
if err := cat.UpdateVideoMeta(ctx, video.ID, catalog.VideoMetaPatch{
|
||||
ThumbnailStatus: "pending",
|
||||
ResetThumbnailFailures: true,
|
||||
}); err != nil {
|
||||
t.Fatalf("reset thumbnail status: %v", err)
|
||||
}
|
||||
worker.rateLimit = rateLimitState{}
|
||||
worker.process(ctx, video)
|
||||
|
||||
pending, err := cat.ListVideosByThumbnailStatus(ctx, video.DriveID, "pending", 0)
|
||||
if err != nil {
|
||||
t.Fatalf("list pending thumbnails after reset: %v", err)
|
||||
}
|
||||
if len(pending) != 1 || pending[0].ID != video.ID {
|
||||
t.Fatalf("pending thumbnails after reset = %#v, want only %s", pending, video.ID)
|
||||
}
|
||||
}
|
||||
|
||||
func TestThumbWorkerRequeuesP115TransientErrorBeforeRetryLimit(t *testing.T) {
|
||||
ctx := context.Background()
|
||||
cat, video := seedPreviewTestVideo(t, "thumb-p115-requeue")
|
||||
|
||||
gen := &fakeThumbGenerator{
|
||||
generateErr: errors.New("ffmpeg thumb: partial file Cannot determine format of input 0:0 after EOF"),
|
||||
}
|
||||
drv := &previewFakeDrive{kind: "p115"}
|
||||
worker := NewThumbWorker(gen, cat, drv)
|
||||
|
||||
worker.processQueued(ctx, video)
|
||||
|
||||
select {
|
||||
case queued := <-worker.ch:
|
||||
if queued.ID != video.ID {
|
||||
t.Fatalf("requeued video id = %q, want %q", queued.ID, video.ID)
|
||||
}
|
||||
default:
|
||||
t.Fatal("expected transient thumbnail failure to requeue the same video")
|
||||
}
|
||||
|
||||
got, err := cat.GetVideo(ctx, video.ID)
|
||||
if err != nil {
|
||||
t.Fatalf("get video: %v", err)
|
||||
}
|
||||
if got.ThumbnailURL != "" {
|
||||
t.Fatalf("thumbnail = %q, want empty after transient failure", got.ThumbnailURL)
|
||||
}
|
||||
pending, err := cat.ListVideosByThumbnailStatus(ctx, video.DriveID, "pending", 0)
|
||||
if err != nil {
|
||||
t.Fatalf("list pending thumbnails: %v", err)
|
||||
}
|
||||
if len(pending) != 1 || pending[0].ID != video.ID {
|
||||
t.Fatalf("pending thumbnails = %#v, want only %s", pending, video.ID)
|
||||
}
|
||||
}
|
||||
|
||||
func TestPreviewWorkerP115TransientErrorKeepsVideoPending(t *testing.T) {
|
||||
ctx := context.Background()
|
||||
cat, video := seedPreviewTestVideo(t, "preview-p115-transient")
|
||||
@@ -508,6 +655,7 @@ type fakeThumbGenerator struct {
|
||||
thumbnailDuration float64
|
||||
thumbnailURL string
|
||||
probeCalls int
|
||||
generateCalls int
|
||||
probeDuration float64
|
||||
probeErr error
|
||||
generateErr error
|
||||
@@ -522,6 +670,7 @@ func (g *fakeThumbGenerator) Probe(context.Context, *drives.StreamLink) (float64
|
||||
}
|
||||
|
||||
func (g *fakeThumbGenerator) GenerateThumbnail(_ context.Context, link *drives.StreamLink, videoID string, duration float64) (string, error) {
|
||||
g.generateCalls++
|
||||
g.thumbnailVideoID = videoID
|
||||
g.thumbnailDuration = duration
|
||||
if link != nil {
|
||||
|
||||
@@ -16,11 +16,11 @@ type ParsedName struct {
|
||||
}
|
||||
|
||||
var (
|
||||
reTags = regexp.MustCompile(`^\s*\[([^\]]+)\]\s*`) // [tag1,tag2]
|
||||
reTags = regexp.MustCompile(`^\s*\[([^\]]+)\]\s*`) // [前缀]
|
||||
reAuthor = regexp.MustCompile(`\s*-\s*([^-]+?)\s*$`) // - author
|
||||
)
|
||||
|
||||
// Parse 按约定解析:[tag1,tag2] 标题 - 作者.ext
|
||||
// Parse 按约定解析:[前缀] 标题 - 作者.ext
|
||||
// 任何字段缺失都能降级
|
||||
func Parse(filename string) ParsedName {
|
||||
name := strings.TrimSuffix(filename, path.Ext(filename))
|
||||
|
||||
@@ -254,6 +254,93 @@ func TestRunAddsShortCollectionDirectoryAsTag(t *testing.T) {
|
||||
}
|
||||
}
|
||||
|
||||
func TestRunDoesNotRecreateDeletedCollectionDirectoryTag(t *testing.T) {
|
||||
ctx := context.Background()
|
||||
cat, err := catalog.Open(t.TempDir() + "/catalog.db")
|
||||
if err != nil {
|
||||
t.Fatalf("open catalog: %v", err)
|
||||
}
|
||||
t.Cleanup(func() {
|
||||
if err := cat.Close(); err != nil {
|
||||
t.Fatalf("close catalog: %v", err)
|
||||
}
|
||||
})
|
||||
now := time.Now()
|
||||
for _, id := range []string{"existing-1", "existing-2"} {
|
||||
if err := cat.UpsertVideo(ctx, &catalog.Video{
|
||||
ID: id,
|
||||
DriveID: "drive",
|
||||
FileID: id,
|
||||
Title: "Existing",
|
||||
Category: "sunny",
|
||||
PublishedAt: now,
|
||||
CreatedAt: now,
|
||||
UpdatedAt: now,
|
||||
}); err != nil {
|
||||
t.Fatalf("seed existing sunny video: %v", err)
|
||||
}
|
||||
}
|
||||
if label, ok, err := cat.EnsureCollectionTag(ctx, "sunny"); err != nil || !ok || label != "sunny" {
|
||||
t.Fatalf("ensure collection = %q, %v, %v; want sunny true nil", label, ok, err)
|
||||
}
|
||||
tags, err := cat.ListTags(ctx)
|
||||
if err != nil {
|
||||
t.Fatalf("list tags: %v", err)
|
||||
}
|
||||
var tagID int64
|
||||
for _, tag := range tags {
|
||||
if tag.Label == "sunny" {
|
||||
tagID = tag.ID
|
||||
break
|
||||
}
|
||||
}
|
||||
if tagID == 0 {
|
||||
t.Fatal("sunny tag not found before delete")
|
||||
}
|
||||
if _, err := cat.DeleteTag(ctx, tagID); err != nil {
|
||||
t.Fatalf("delete tag: %v", err)
|
||||
}
|
||||
|
||||
drv := &scannerTreeFakeDrive{
|
||||
entries: map[string][]drives.Entry{
|
||||
"root": {{
|
||||
ID: "dir-1",
|
||||
Name: "sunny",
|
||||
IsDir: true,
|
||||
}},
|
||||
"dir-1": {{
|
||||
ID: "file-1",
|
||||
ParentID: "dir-1",
|
||||
Name: "clip.mp4",
|
||||
Size: 123,
|
||||
ModTime: now,
|
||||
}},
|
||||
},
|
||||
}
|
||||
sc := New(cat, drv, []string{".mp4"}, nil, nil)
|
||||
|
||||
if _, err := sc.Run(ctx, ""); err != nil {
|
||||
t.Fatalf("scan: %v", err)
|
||||
}
|
||||
|
||||
got, err := cat.GetVideo(ctx, "fake-drive-file-1")
|
||||
if err != nil {
|
||||
t.Fatalf("get video: %v", err)
|
||||
}
|
||||
if len(got.Tags) != 0 {
|
||||
t.Fatalf("tags = %#v, want none", got.Tags)
|
||||
}
|
||||
tags, err = cat.ListTags(ctx)
|
||||
if err != nil {
|
||||
t.Fatalf("list tags after scan: %v", err)
|
||||
}
|
||||
for _, tag := range tags {
|
||||
if tag.Label == "sunny" {
|
||||
t.Fatal("deleted collection tag was recreated during scan")
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestRunMapsAVCodeDirectoryToAVTag(t *testing.T) {
|
||||
ctx := context.Background()
|
||||
cat, err := catalog.Open(t.TempDir() + "/catalog.db")
|
||||
|
||||
+1
-1
@@ -1,6 +1,6 @@
|
||||
services:
|
||||
video-site-91:
|
||||
image: ghcr.io/nianzhibai/91:latest
|
||||
image: ghcr.io/nianzhibai/91:stable
|
||||
container_name: video-site-91
|
||||
ports:
|
||||
- "9191:9191"
|
||||
|
||||
@@ -30,7 +30,7 @@ else
|
||||
echo "[entrypoint] using existing $CONFIG"
|
||||
fi
|
||||
|
||||
if [ -n "${VIDEO_VERSION_FILE:-}" ] && [ -n "${VIDEO_IMAGE_VERSION:-}" ] && [ ! -f "$VIDEO_VERSION_FILE" ]; then
|
||||
if [ -n "${VIDEO_VERSION_FILE:-}" ] && [ -n "${VIDEO_IMAGE_VERSION:-}" ]; then
|
||||
mkdir -p "$(dirname "$VIDEO_VERSION_FILE")"
|
||||
printf '%s\n' "$VIDEO_IMAGE_VERSION" > "$VIDEO_VERSION_FILE"
|
||||
fi
|
||||
|
||||
+193
-13
@@ -70,6 +70,7 @@ Options via environment:
|
||||
CONFIGURE_UFW=$CONFIGURE_UFW
|
||||
SELF_UPDATE=$SELF_UPDATE
|
||||
FORCE_UPDATE=$FORCE_UPDATE
|
||||
UNINSTALL_DELETE_FILES=0 Set to 1 for non-interactive uninstall to delete $INSTALL_PATH
|
||||
INSTALL_SCRIPT_REF=$INSTALL_SCRIPT_REF
|
||||
INSTALL_SCRIPT_URL=$INSTALL_SCRIPT_URL
|
||||
SERVICE_READY_TIMEOUT=$SERVICE_READY_TIMEOUT
|
||||
@@ -323,8 +324,8 @@ exec_latest_manager_update() {
|
||||
}
|
||||
|
||||
open_firewall_port() {
|
||||
[[ "$CONFIGURE_UFW" == "1" ]] || return
|
||||
command -v ufw >/dev/null 2>&1 || return
|
||||
[[ "$CONFIGURE_UFW" == "1" ]] || return 0
|
||||
command -v ufw >/dev/null 2>&1 || return 0
|
||||
if ufw status 2>/dev/null | grep -qi "Status: active"; then
|
||||
log "allowing ${FRONTEND_PORT}/tcp in UFW"
|
||||
ufw allow "${FRONTEND_PORT}/tcp"
|
||||
@@ -345,6 +346,171 @@ listen_port_from_config() {
|
||||
printf '%s' "$FRONTEND_PORT"
|
||||
}
|
||||
|
||||
append_unique() {
|
||||
local value="$1"
|
||||
shift
|
||||
for existing in "$@"; do
|
||||
[[ "$existing" == "$value" ]] && return 1
|
||||
done
|
||||
printf '%s' "$value"
|
||||
}
|
||||
|
||||
app_service_names() {
|
||||
local names=()
|
||||
local name
|
||||
for name in "$SERVICE_NAME" "$APP_NAME" video-site-91 video-site-backend video-site-frontend; do
|
||||
[[ -n "$name" ]] || continue
|
||||
if append_unique "$name" "${names[@]}" >/dev/null; then
|
||||
names+=("$name")
|
||||
fi
|
||||
done
|
||||
printf '%s\n' "${names[@]}"
|
||||
}
|
||||
|
||||
stop_app_services() {
|
||||
local name unit
|
||||
while IFS= read -r name; do
|
||||
[[ -n "$name" ]] || continue
|
||||
unit="${name}.service"
|
||||
systemctl disable --now "$unit" 2>/dev/null || systemctl stop "$unit" 2>/dev/null || true
|
||||
rm -f "/etc/systemd/system/$unit"
|
||||
done < <(app_service_names)
|
||||
systemctl daemon-reload
|
||||
}
|
||||
|
||||
remove_app_containers() {
|
||||
command -v docker >/dev/null 2>&1 || return 0
|
||||
|
||||
local names=()
|
||||
local name
|
||||
for name in "$SERVICE_NAME" "$APP_NAME" video-site-91; do
|
||||
[[ -n "$name" ]] || continue
|
||||
if append_unique "$name" "${names[@]}" >/dev/null; then
|
||||
names+=("$name")
|
||||
fi
|
||||
done
|
||||
|
||||
for name in "${names[@]}"; do
|
||||
if docker ps -a --format '{{.Names}}' 2>/dev/null | grep -Fxq "$name"; then
|
||||
log "removing docker container $name"
|
||||
docker rm -f "$name" >/dev/null 2>&1 || true
|
||||
fi
|
||||
done
|
||||
}
|
||||
|
||||
pids_listening_on_port() {
|
||||
local port="$1"
|
||||
[[ "$port" =~ ^[0-9]+$ ]] || return 0
|
||||
command -v ss >/dev/null 2>&1 || return 0
|
||||
|
||||
ss -ltnp 2>/dev/null \
|
||||
| awk -v port="$port" '$4 ~ ":" port "$" {print}' \
|
||||
| grep -oE 'pid=[0-9]+' \
|
||||
| cut -d= -f2 \
|
||||
| sort -u || true
|
||||
}
|
||||
|
||||
process_looks_like_app() {
|
||||
local pid="$1"
|
||||
local exe="" cmd=""
|
||||
exe="$(readlink "/proc/$pid/exe" 2>/dev/null || true)"
|
||||
cmd="$(tr '\0' ' ' <"/proc/$pid/cmdline" 2>/dev/null || true)"
|
||||
|
||||
[[ "$exe" == "$INSTALL_PATH/server" ]] && return 0
|
||||
[[ "$cmd" == *"$INSTALL_PATH"* ]] && return 0
|
||||
[[ "$cmd" == *"VIDEO_FRONTEND_DIR=$INSTALL_PATH/dist"* ]] && return 0
|
||||
[[ "$cmd" == *"VIDEO_CONFIG=$INSTALL_PATH/config.yaml"* ]] && return 0
|
||||
[[ "$cmd" == *"video-site-91"* ]] && return 0
|
||||
[[ "$cmd" == *"91VideoSpider"* ]] && return 0
|
||||
return 1
|
||||
}
|
||||
|
||||
stop_lingering_app_processes() {
|
||||
local ports=("$@")
|
||||
local port pid pids=()
|
||||
|
||||
for port in "${ports[@]}"; do
|
||||
[[ "$port" =~ ^[0-9]+$ ]] || continue
|
||||
while IFS= read -r pid; do
|
||||
[[ -n "$pid" ]] || continue
|
||||
process_looks_like_app "$pid" || continue
|
||||
if append_unique "$pid" "${pids[@]}" >/dev/null; then
|
||||
pids+=("$pid")
|
||||
fi
|
||||
done < <(pids_listening_on_port "$port")
|
||||
done
|
||||
|
||||
if (( ${#pids[@]} == 0 )); then
|
||||
return
|
||||
fi
|
||||
|
||||
warn "stopping lingering app process(es): ${pids[*]}"
|
||||
kill "${pids[@]}" 2>/dev/null || true
|
||||
sleep 1
|
||||
|
||||
local alive=()
|
||||
for pid in "${pids[@]}"; do
|
||||
if kill -0 "$pid" 2>/dev/null; then
|
||||
alive+=("$pid")
|
||||
fi
|
||||
done
|
||||
if (( ${#alive[@]} > 0 )); then
|
||||
warn "force killing lingering app process(es): ${alive[*]}"
|
||||
kill -9 "${alive[@]}" 2>/dev/null || true
|
||||
fi
|
||||
}
|
||||
|
||||
warn_remaining_listeners() {
|
||||
local ports=("$@")
|
||||
local port pid cmd
|
||||
for port in "${ports[@]}"; do
|
||||
[[ "$port" =~ ^[0-9]+$ ]] || continue
|
||||
while IFS= read -r pid; do
|
||||
[[ -n "$pid" ]] || continue
|
||||
cmd="$(tr '\0' ' ' <"/proc/$pid/cmdline" 2>/dev/null || true)"
|
||||
warn "port $port is still listening after uninstall: pid=$pid ${cmd:-unknown}"
|
||||
done < <(pids_listening_on_port "$port")
|
||||
done
|
||||
}
|
||||
|
||||
has_interactive_tty() {
|
||||
[[ -t 0 ]]
|
||||
}
|
||||
|
||||
confirm_uninstall_app() {
|
||||
if ! has_interactive_tty; then
|
||||
return 0
|
||||
fi
|
||||
|
||||
local confirm=""
|
||||
printf '确认卸载 91 吗?这会停止服务、移除管理命令,并可选择是否删除项目文件。[y/N]: ' >/dev/tty
|
||||
IFS= read -r confirm </dev/tty || confirm=""
|
||||
case "$confirm" in
|
||||
[yY]) return 0 ;;
|
||||
*)
|
||||
log "uninstall cancelled"
|
||||
return 1
|
||||
;;
|
||||
esac
|
||||
}
|
||||
|
||||
delete_install_path_requested() {
|
||||
if [[ "${UNINSTALL_DELETE_FILES:-0}" == "1" ]]; then
|
||||
return 0
|
||||
fi
|
||||
if ! has_interactive_tty; then
|
||||
return 1
|
||||
fi
|
||||
|
||||
local confirm=""
|
||||
printf '删除 %s 里的程序、配置和数据吗?[y/N]: ' "$INSTALL_PATH" >/dev/tty
|
||||
IFS= read -r confirm </dev/tty || confirm=""
|
||||
case "$confirm" in
|
||||
[yY]) return 0 ;;
|
||||
*) return 1 ;;
|
||||
esac
|
||||
}
|
||||
|
||||
service_health_url() {
|
||||
printf 'http://127.0.0.1:%s/admin/api/setup' "$(listen_port_from_config)"
|
||||
}
|
||||
@@ -557,20 +723,30 @@ update_app() {
|
||||
}
|
||||
|
||||
uninstall_app() {
|
||||
systemctl disable --now "${SERVICE_NAME}.service" 2>/dev/null || true
|
||||
rm -f "/etc/systemd/system/${SERVICE_NAME}.service"
|
||||
systemctl daemon-reload
|
||||
local listen_port port ports=()
|
||||
confirm_uninstall_app || return 1
|
||||
|
||||
listen_port="$(listen_port_from_config)"
|
||||
for port in "$listen_port" "$FRONTEND_PORT" 9191 9192; do
|
||||
[[ "$port" =~ ^[0-9]+$ ]] || continue
|
||||
if append_unique "$port" "${ports[@]}" >/dev/null; then
|
||||
ports+=("$port")
|
||||
fi
|
||||
done
|
||||
|
||||
stop_app_services
|
||||
remove_app_containers
|
||||
stop_lingering_app_processes "${ports[@]}"
|
||||
rm -f "$COMMAND_LINK" "$APP_COMMAND_LINK" "$MANAGER_PATH"
|
||||
|
||||
if [[ -t 0 ]]; then
|
||||
read -r -p "删除 $INSTALL_PATH 里的程序、配置和数据吗?[y/N]: " confirm
|
||||
case "$confirm" in
|
||||
[yY]) rm -rf "$INSTALL_PATH" ;;
|
||||
*) log "kept $INSTALL_PATH" ;;
|
||||
esac
|
||||
if delete_install_path_requested; then
|
||||
rm -rf "$INSTALL_PATH"
|
||||
log "removed $INSTALL_PATH"
|
||||
else
|
||||
log "removed service; kept $INSTALL_PATH"
|
||||
log "kept $INSTALL_PATH"
|
||||
fi
|
||||
|
||||
warn_remaining_listeners "${ports[@]}"
|
||||
}
|
||||
|
||||
show_menu() {
|
||||
@@ -600,7 +776,11 @@ show_menu() {
|
||||
3) main update ;;
|
||||
4) main restart ;;
|
||||
5) main stop ;;
|
||||
6) main uninstall ;;
|
||||
6)
|
||||
if main uninstall; then
|
||||
exit 0
|
||||
fi
|
||||
;;
|
||||
0) exit 0 ;;
|
||||
*) echo "无效的选项" ;;
|
||||
esac
|
||||
|
||||
@@ -110,7 +110,7 @@ export function AdminLayout() {
|
||||
{checkingUpdate ? "检查中" : "检查更新"}
|
||||
</button>
|
||||
<button className="admin-sidebar__logout" onClick={handleLogout}>
|
||||
<LogOut size={14} style={{ verticalAlign: -2, marginRight: 4 }} />
|
||||
<LogOut size={14} />
|
||||
退出登录
|
||||
</button>
|
||||
</div>
|
||||
|
||||
+93
-48
@@ -26,6 +26,7 @@ const kindLabel: Record<string, string> = {
|
||||
pikpak: "PikPak",
|
||||
wopan: "联通沃盘",
|
||||
onedrive: "OneDrive",
|
||||
googledrive: "Google Drive",
|
||||
localstorage: "本地存储",
|
||||
spider91: "91 爬虫",
|
||||
};
|
||||
@@ -41,7 +42,6 @@ type FormState = {
|
||||
kind: Kind;
|
||||
name: string;
|
||||
rootId: string;
|
||||
scanRootId: string;
|
||||
creds: Record<string, string>;
|
||||
/**
|
||||
* spider91 专用字段:把视频迁移到云盘的目标 drive ID。
|
||||
@@ -58,16 +58,36 @@ const emptyForm: FormState = {
|
||||
id: "",
|
||||
kind: "p115",
|
||||
name: "",
|
||||
rootId: "0",
|
||||
scanRootId: "0",
|
||||
rootId: "",
|
||||
creds: {},
|
||||
spider91UploadDriveId: "",
|
||||
};
|
||||
|
||||
const idleNightlyStatus: api.NightlyJobStatus = {
|
||||
state: "idle",
|
||||
running: false,
|
||||
queued: false,
|
||||
};
|
||||
|
||||
function nightlyButtonText(status: api.NightlyJobStatus, triggering: boolean) {
|
||||
if (triggering) return "触发中...";
|
||||
if (status.running) return "扫描运行中";
|
||||
if (status.queued) return "扫描已排队";
|
||||
return "扫描所有网盘";
|
||||
}
|
||||
|
||||
function nightlyBusyText(status: api.NightlyJobStatus) {
|
||||
if (status.running) return "扫描任务正在运行";
|
||||
if (status.queued) return "扫描任务已排队";
|
||||
return "";
|
||||
}
|
||||
|
||||
export function DrivesPage() {
|
||||
const [list, setList] = useState<api.AdminDrive[]>([]);
|
||||
const [storage, setStorage] = useState<api.AdminDriveStorage | null>(null);
|
||||
const [settings, setSettings] = useState<api.Settings | null>(null);
|
||||
const [nightlyStatus, setNightlyStatus] =
|
||||
useState<api.NightlyJobStatus>(idleNightlyStatus);
|
||||
const [loading, setLoading] = useState(true);
|
||||
const [modalOpen, setModalOpen] = useState(false);
|
||||
const [form, setForm] = useState<FormState>(emptyForm);
|
||||
@@ -78,8 +98,10 @@ export function DrivesPage() {
|
||||
const [regenFailedThumbId, setRegenFailedThumbId] = useState("");
|
||||
// togglingTeaserId 在请求未返回前禁用按钮,避免连点导致两次切换互相覆盖。
|
||||
const [togglingTeaserId, setTogglingTeaserId] = useState("");
|
||||
const [scanningAll, setScanningAll] = useState(false);
|
||||
const [selectedDriveId, setSelectedDriveId] = useState<string | null>(null);
|
||||
const { show } = useToast();
|
||||
const nightlyBusy = scanningAll || nightlyStatus.running || nightlyStatus.queued;
|
||||
|
||||
// 当前系统中可作为 spider91 上传目标的 drive 列表(pikpak ∪ p115 ∪ onedrive)。
|
||||
// 用户保存 spider91 drive 时从这里挑一个;空表示本地保存不上传。
|
||||
@@ -91,14 +113,16 @@ export function DrivesPage() {
|
||||
async function refresh() {
|
||||
setLoading(true);
|
||||
try {
|
||||
const [data, storageData, settingsData] = await Promise.all([
|
||||
const [data, storageData, settingsData, jobStatus] = await Promise.all([
|
||||
api.listDrives(),
|
||||
api.getDriveStorage(),
|
||||
api.getSettings().catch(() => null),
|
||||
api.getNightlyJobStatus().catch(() => null),
|
||||
]);
|
||||
setList(data ?? []);
|
||||
setStorage(storageData);
|
||||
if (settingsData) setSettings(settingsData);
|
||||
if (jobStatus) setNightlyStatus(jobStatus);
|
||||
} catch (e) {
|
||||
show(e instanceof Error ? e.message : "加载失败", "error");
|
||||
} finally {
|
||||
@@ -108,8 +132,12 @@ export function DrivesPage() {
|
||||
|
||||
async function refreshDriveList() {
|
||||
try {
|
||||
const data = await api.listDrives();
|
||||
const [data, jobStatus] = await Promise.all([
|
||||
api.listDrives(),
|
||||
api.getNightlyJobStatus().catch(() => null),
|
||||
]);
|
||||
setList(data ?? []);
|
||||
if (jobStatus) setNightlyStatus(jobStatus);
|
||||
} catch {
|
||||
// 保持当前页面状态,下一次轮询或手动操作再刷新。
|
||||
}
|
||||
@@ -141,7 +169,6 @@ export function DrivesPage() {
|
||||
kind: d.kind,
|
||||
name: d.name,
|
||||
rootId: d.rootId,
|
||||
scanRootId: d.scanRootId || d.rootId,
|
||||
creds: {},
|
||||
spider91UploadDriveId: settings?.spider91UploadDriveId ?? "",
|
||||
});
|
||||
@@ -158,6 +185,7 @@ export function DrivesPage() {
|
||||
const driveID = existing
|
||||
? form.id
|
||||
: makeUniqueDriveId(form.kind, name, list);
|
||||
const rootId = form.rootId.trim() || defaultRootId(form.kind);
|
||||
// 若编辑且没有提供凭证,提示一下但仍允许保存(不改凭证)
|
||||
setSaving(true);
|
||||
try {
|
||||
@@ -165,8 +193,7 @@ export function DrivesPage() {
|
||||
id: driveID,
|
||||
kind: form.kind,
|
||||
name,
|
||||
rootId: form.rootId || defaultRootId(form.kind),
|
||||
scanRootId: form.scanRootId || form.rootId || defaultRootId(form.kind),
|
||||
rootId,
|
||||
credentials: form.creds,
|
||||
});
|
||||
|
||||
@@ -233,14 +260,26 @@ export function DrivesPage() {
|
||||
/**
|
||||
* 立即触发完整凌晨流水线(Phase1 扫所有云盘 → Phase2 spider91 爬虫 →
|
||||
* Phase3 spider91 → 云盘迁移)。后端立即返回 202;进度看 backend 日志。
|
||||
* 如果当前已有流水线在跑,后端最多保留一个待触发请求,当前轮结束后再跑一轮。
|
||||
* 如果当前已有流水线在跑或已排队,前端只提示,不再提交新任务。
|
||||
*/
|
||||
async function handleRunNightly() {
|
||||
if (nightlyBusy) {
|
||||
show(nightlyBusyText(nightlyStatus) || "当前已有扫描所有网盘任务", "info");
|
||||
return;
|
||||
}
|
||||
setScanningAll(true);
|
||||
try {
|
||||
await api.runNightlyJob();
|
||||
show("已触发扫描所有网盘,耗时较长,可在 backend 日志观察进度", "success");
|
||||
const resp = await api.runNightlyJob();
|
||||
setNightlyStatus(resp.status);
|
||||
if (resp.accepted) {
|
||||
show("已触发扫描所有网盘,耗时较长,可在任务状态和 backend 日志观察进度", "success");
|
||||
} else {
|
||||
show("当前已有扫描所有网盘任务", "info");
|
||||
}
|
||||
} catch (e) {
|
||||
show(e instanceof Error ? e.message : "触发失败", "error");
|
||||
} finally {
|
||||
setScanningAll(false);
|
||||
}
|
||||
}
|
||||
|
||||
@@ -362,10 +401,6 @@ export function DrivesPage() {
|
||||
<span className="admin-detail-label">根目录 ID</span>
|
||||
<span className="admin-detail-value admin-mono-cell">{d.rootId}</span>
|
||||
</div>
|
||||
<div className="admin-detail-row">
|
||||
<span className="admin-detail-label">扫描起点 ID</span>
|
||||
<span className="admin-detail-value admin-mono-cell">{d.scanRootId || d.rootId}</span>
|
||||
</div>
|
||||
</>
|
||||
)}
|
||||
{d.kind === "spider91" && (
|
||||
@@ -463,6 +498,7 @@ export function DrivesPage() {
|
||||
ready={d.thumbnailReadyCount}
|
||||
pending={d.thumbnailPendingCount}
|
||||
failed={d.thumbnailFailedCount}
|
||||
durationPending={d.thumbnailDurationPendingCount}
|
||||
/>
|
||||
</div>
|
||||
</div>
|
||||
@@ -587,9 +623,10 @@ export function DrivesPage() {
|
||||
type="button"
|
||||
className="admin-btn"
|
||||
onClick={handleRunNightly}
|
||||
title="立即扫描所有网盘。耗时较长,期间不要重复触发。"
|
||||
disabled={scanningAll}
|
||||
title={nightlyBusyText(nightlyStatus) || "立即扫描所有网盘。耗时较长,期间不要重复触发。"}
|
||||
>
|
||||
<PlayCircle size={14} /> 扫描所有网盘
|
||||
<PlayCircle size={14} /> {nightlyButtonText(nightlyStatus, scanningAll)}
|
||||
</button>
|
||||
<button className="admin-btn is-primary" onClick={openCreate}>
|
||||
<Plus size={14} /> 新建网盘
|
||||
@@ -721,10 +758,12 @@ function GenerationCounts({
|
||||
ready,
|
||||
pending,
|
||||
failed,
|
||||
durationPending,
|
||||
}: {
|
||||
ready?: number;
|
||||
pending?: number;
|
||||
failed?: number;
|
||||
durationPending?: number;
|
||||
}) {
|
||||
return (
|
||||
<div className="admin-generation-counts">
|
||||
@@ -737,6 +776,11 @@ function GenerationCounts({
|
||||
<span className="admin-drive-teaser__metric is-failed">
|
||||
失败 {failed ?? 0}
|
||||
</span>
|
||||
{(durationPending ?? 0) > 0 && (
|
||||
<span className="admin-drive-teaser__metric">
|
||||
待补时长 {durationPending}
|
||||
</span>
|
||||
)}
|
||||
</div>
|
||||
);
|
||||
}
|
||||
@@ -752,7 +796,7 @@ function GenerationStatusLine({
|
||||
const queueLength = status?.queueLength ?? 0;
|
||||
const detail = generationDetail(status);
|
||||
const title = generationTitle(status, detail);
|
||||
const countText = queueLength > 0 ? `${label === "封面" ? "剩余" : "队列"} ${queueLength}` : "";
|
||||
const countText = queueLength > 0 ? `${label === "封面" ? "待处理" : "队列"} ${queueLength}` : "";
|
||||
|
||||
return (
|
||||
<div className="admin-generation-row" title={title}>
|
||||
@@ -863,11 +907,6 @@ function DriveForm({
|
||||
}) {
|
||||
const fields = useMemo(() => credentialFields(form.kind), [form.kind]);
|
||||
const help = credentialHelp(form.kind, isEdit);
|
||||
const showDirectoryFields =
|
||||
form.kind !== "spider91" &&
|
||||
form.kind !== "onedrive" &&
|
||||
form.kind !== "localstorage" &&
|
||||
form.kind !== "pikpak";
|
||||
|
||||
function set<K extends keyof FormState>(k: K, v: FormState[K]) {
|
||||
onChange({ ...form, [k]: v });
|
||||
@@ -879,8 +918,7 @@ function DriveForm({
|
||||
onChange({
|
||||
...form,
|
||||
kind: v,
|
||||
rootId: defaultRootId(v),
|
||||
scanRootId: defaultRootId(v),
|
||||
rootId: "",
|
||||
creds: {},
|
||||
});
|
||||
}
|
||||
@@ -905,35 +943,24 @@ function DriveForm({
|
||||
<option value="p115">115 网盘</option>
|
||||
<option value="pikpak">PikPak</option>
|
||||
<option value="onedrive">OneDrive</option>
|
||||
<option value="googledrive">Google Drive</option>
|
||||
<option value="localstorage">本地存储</option>
|
||||
<option value="spider91">91 Spider</option>
|
||||
<option value="quark">夸克网盘</option>
|
||||
<option value="wopan">联通沃盘</option>
|
||||
</select>
|
||||
</div>
|
||||
{showDirectoryFields && (
|
||||
<>
|
||||
<div className="admin-form__row">
|
||||
<label>根目录 ID</label>
|
||||
<input
|
||||
value={form.rootId}
|
||||
onChange={(e) => set("rootId", e.target.value)}
|
||||
placeholder={form.kind === "pikpak" ? "留空表示根目录" : form.kind === "onedrive" ? "root" : "0"}
|
||||
/>
|
||||
</div>
|
||||
<div className="admin-form__row">
|
||||
<label>扫描起点目录 ID</label>
|
||||
<input
|
||||
value={form.scanRootId}
|
||||
onChange={(e) => set("scanRootId", e.target.value)}
|
||||
placeholder="留空则使用根目录"
|
||||
/>
|
||||
<div className="admin-form__help">
|
||||
可以指定一个子目录作为视频库入口,避免扫描整个网盘
|
||||
</div>
|
||||
</div>
|
||||
</>
|
||||
)}
|
||||
<div className="admin-form__row">
|
||||
<label>根目录 ID</label>
|
||||
<input
|
||||
value={form.rootId}
|
||||
onChange={(e) => set("rootId", e.target.value)}
|
||||
placeholder={rootIdPlaceholder(form.kind)}
|
||||
/>
|
||||
<div className="admin-form__help">
|
||||
留空时使用该网盘类型的默认根目录,具体目录ID获取方式请参考OpenList文档
|
||||
</div>
|
||||
</div>
|
||||
|
||||
{(help || fields.length > 0) && (
|
||||
<>
|
||||
@@ -1031,6 +1058,8 @@ function credentialHelp(kind: Kind, isEdit: boolean): string {
|
||||
return `需要 access_token 和 refresh_token。后续会加扫码/短信登录入口,第一版只能手工粘贴。${note}`;
|
||||
case "onedrive":
|
||||
return `按 OpenList 默认应用在线挂载,只需要 refresh_token;保存时会自动刷新并保存 token。${note}`;
|
||||
case "googledrive":
|
||||
return `按 OpenList 在线 API 挂载,只需要 Google Drive refresh_token;保存时会自动刷新并保存 token。播放不走 302,会由后端带 Authorization 代理转发。${note}`;
|
||||
case "localstorage":
|
||||
return `把服务器上的一个已有目录作为视频来源扫描。填写绝对路径,例如 /mnt/videos;系统会读取该目录及子目录中的视频,并生成封面、Teaser 和指纹。${note}`;
|
||||
case "spider91":
|
||||
@@ -1114,6 +1143,16 @@ function credentialFields(kind: Kind): Array<{
|
||||
required: true,
|
||||
},
|
||||
];
|
||||
case "googledrive":
|
||||
return [
|
||||
{
|
||||
key: "refresh_token",
|
||||
label: "refresh_token",
|
||||
placeholder: "OpenList Google Drive refresh_token",
|
||||
multiline: true,
|
||||
required: true,
|
||||
},
|
||||
];
|
||||
case "localstorage":
|
||||
return [
|
||||
{
|
||||
@@ -1132,11 +1171,17 @@ function credentialFields(kind: Kind): Array<{
|
||||
function defaultRootId(kind: Kind): string {
|
||||
if (kind === "pikpak") return "";
|
||||
if (kind === "onedrive") return "root";
|
||||
if (kind === "googledrive") return "root";
|
||||
if (kind === "localstorage") return "/";
|
||||
if (kind === "spider91") return "/";
|
||||
return "0";
|
||||
}
|
||||
|
||||
function rootIdPlaceholder(kind: Kind): string {
|
||||
const rootId = defaultRootId(kind);
|
||||
return rootId ? `默认:${rootId}` : "留空表示根目录";
|
||||
}
|
||||
|
||||
|
||||
// ---------- SkipDirsModal ----------
|
||||
//
|
||||
|
||||
@@ -82,11 +82,6 @@ export function LoginPage() {
|
||||
<Play size={18} fill="currentColor" /> {setupRequired ? "首次设置管理员" : "登录"}
|
||||
</h1>
|
||||
<div className="admin-form">
|
||||
{setupRequired && (
|
||||
<div className="admin-form__help admin-form__help--lead">
|
||||
请先设置后台管理员账号。保存后会写入本机配置文件,之后使用该账号登录。
|
||||
</div>
|
||||
)}
|
||||
<div className="admin-form__row">
|
||||
<label>用户名</label>
|
||||
<input
|
||||
|
||||
+268
-37
@@ -1,16 +1,26 @@
|
||||
import { useEffect, useMemo, useState } from "react";
|
||||
import { Film, Plus, RefreshCw, Search, Tags } from "lucide-react";
|
||||
import { CheckSquare, Film, Plus, RefreshCw, Search, Tags, Trash2 } from "lucide-react";
|
||||
import * as api from "./api";
|
||||
import { useToast } from "./ToastContext";
|
||||
|
||||
const DESKTOP_TAGS_PAGE_SIZE = 25;
|
||||
const MOBILE_TAGS_PAGE_SIZE = 8;
|
||||
const TAGS_MOBILE_QUERY = "(max-width: 640px)";
|
||||
|
||||
export function TagsPage() {
|
||||
const [tags, setTags] = useState<api.AdminTag[]>([]);
|
||||
const [label, setLabel] = useState("");
|
||||
const [aliases, setAliases] = useState("");
|
||||
const [loading, setLoading] = useState(true);
|
||||
const [saving, setSaving] = useState(false);
|
||||
const [deletingId, setDeletingId] = useState<number | null>(null);
|
||||
const [searchQuery, setSearchQuery] = useState("");
|
||||
const [filterSource, setFilterSource] = useState<string>("all");
|
||||
const [selectMode, setSelectMode] = useState(false);
|
||||
const [selected, setSelected] = useState<Set<number>>(new Set());
|
||||
const [bulkDeleting, setBulkDeleting] = useState(false);
|
||||
const pageSize = useTagsPageSize();
|
||||
const [page, setPage] = useState(1);
|
||||
const { show } = useToast();
|
||||
|
||||
async function refresh() {
|
||||
@@ -45,6 +55,63 @@ export function TagsPage() {
|
||||
}
|
||||
}
|
||||
|
||||
async function handleDelete(tag: api.AdminTag) {
|
||||
if (tag.source === "system") return;
|
||||
if (!window.confirm(`确定删除标签「${tag.label}」吗?此操作会从所有视频上移除该标签。`)) {
|
||||
return;
|
||||
}
|
||||
setDeletingId(tag.id);
|
||||
try {
|
||||
const r = await api.deleteTag(tag.id);
|
||||
show(`已删除标签,并从 ${r.removedVideos} 个视频移除`, "success");
|
||||
await refresh();
|
||||
} catch (e) {
|
||||
show(e instanceof Error ? e.message : "删除标签失败", "error");
|
||||
} finally {
|
||||
setDeletingId(null);
|
||||
}
|
||||
}
|
||||
|
||||
function toggleSelectMode() {
|
||||
setSelectMode((m) => !m);
|
||||
setSelected(new Set());
|
||||
}
|
||||
|
||||
function toggleSelect(id: number) {
|
||||
setSelected((prev) => {
|
||||
const next = new Set(prev);
|
||||
next.has(id) ? next.delete(id) : next.add(id);
|
||||
return next;
|
||||
});
|
||||
}
|
||||
|
||||
async function handleBulkDelete() {
|
||||
const ids = [...selected];
|
||||
if (ids.length === 0) return;
|
||||
if (!window.confirm(`确定删除选中的 ${ids.length} 个标签吗?此操作会从所有视频上移除这些标签。`)) {
|
||||
return;
|
||||
}
|
||||
setBulkDeleting(true);
|
||||
try {
|
||||
let ok = 0;
|
||||
for (const id of ids) {
|
||||
try {
|
||||
await api.deleteTag(id);
|
||||
ok += 1;
|
||||
} catch {
|
||||
/* 统计失败数,继续删除其余标签 */
|
||||
}
|
||||
}
|
||||
const failed = ids.length - ok;
|
||||
show(failed ? `已删除 ${ok} 个,${failed} 个失败` : `已删除 ${ok} 个标签`, failed ? "error" : "success");
|
||||
setSelected(new Set());
|
||||
setSelectMode(false);
|
||||
await refresh();
|
||||
} finally {
|
||||
setBulkDeleting(false);
|
||||
}
|
||||
}
|
||||
|
||||
const stats = useMemo(() => {
|
||||
let totalVideos = 0;
|
||||
let systemCount = 0;
|
||||
@@ -82,6 +149,41 @@ export function TagsPage() {
|
||||
});
|
||||
}, [tags, searchQuery, filterSource]);
|
||||
|
||||
const totalPages = Math.max(1, Math.ceil(filteredTags.length / pageSize));
|
||||
const currentPage = Math.min(page, totalPages);
|
||||
const pageStartIndex = (currentPage - 1) * pageSize;
|
||||
const pageEndIndex = pageStartIndex + pageSize;
|
||||
const pagedTags = useMemo(
|
||||
() => filteredTags.slice(pageStartIndex, pageEndIndex),
|
||||
[filteredTags, pageStartIndex, pageEndIndex]
|
||||
);
|
||||
const pageStart = filteredTags.length === 0 ? 0 : pageStartIndex + 1;
|
||||
const pageEnd = Math.min(filteredTags.length, pageEndIndex);
|
||||
|
||||
useEffect(() => {
|
||||
setPage(1);
|
||||
}, [searchQuery, filterSource, pageSize]);
|
||||
|
||||
useEffect(() => {
|
||||
setPage((p) => Math.min(Math.max(1, p), totalPages));
|
||||
}, [totalPages]);
|
||||
|
||||
const deletablePageTags = useMemo(
|
||||
() => pagedTags.filter((t) => t.source !== "system"),
|
||||
[pagedTags]
|
||||
);
|
||||
const allSelected =
|
||||
deletablePageTags.length > 0 && deletablePageTags.every((t) => selected.has(t.id));
|
||||
|
||||
function toggleSelectAll() {
|
||||
setSelected((prev) => {
|
||||
const next = new Set(prev);
|
||||
if (allSelected) deletablePageTags.forEach((t) => next.delete(t.id));
|
||||
else deletablePageTags.forEach((t) => next.add(t.id));
|
||||
return next;
|
||||
});
|
||||
}
|
||||
|
||||
return (
|
||||
<section>
|
||||
<header className="admin-page__header">
|
||||
@@ -130,7 +232,7 @@ export function TagsPage() {
|
||||
|
||||
<div className="admin-card">
|
||||
<div className="admin-card__title">
|
||||
<Tags size={15} /> 系统标签统计
|
||||
<Tags size={15} /> 标签总览
|
||||
</div>
|
||||
<div className="admin-tag-stats-list">
|
||||
<div className="admin-tag-stat-item">
|
||||
@@ -141,14 +243,6 @@ export function TagsPage() {
|
||||
<span>关联视频次</span>
|
||||
<strong>{stats.totalVideos}</strong>
|
||||
</div>
|
||||
<div className="admin-tag-stat-item">
|
||||
<span>系统提取</span>
|
||||
<strong>{stats.systemCount}</strong>
|
||||
</div>
|
||||
<div className="admin-tag-stat-item">
|
||||
<span>用户创建</span>
|
||||
<strong>{stats.userCount}</strong>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
@@ -195,9 +289,52 @@ export function TagsPage() {
|
||||
>
|
||||
合集 ({stats.collectionCount})
|
||||
</button>
|
||||
{stats.legacyCount > 0 && (
|
||||
<button
|
||||
type="button"
|
||||
className={`admin-tags-filter-tab ${filterSource === "legacy" ? "is-active" : ""}`}
|
||||
onClick={() => setFilterSource("legacy")}
|
||||
>
|
||||
旧数据 ({stats.legacyCount})
|
||||
</button>
|
||||
)}
|
||||
</div>
|
||||
|
||||
<button
|
||||
type="button"
|
||||
className={`admin-btn ${selectMode ? "is-primary" : ""}`}
|
||||
onClick={toggleSelectMode}
|
||||
>
|
||||
<CheckSquare size={13} /> {selectMode ? "退出批量" : "批量删除"}
|
||||
</button>
|
||||
</div>
|
||||
|
||||
{selectMode && (
|
||||
<div className="admin-tags-bulkbar">
|
||||
<label className="admin-check">
|
||||
<input type="checkbox" checked={allSelected} onChange={toggleSelectAll} />
|
||||
<span>全选本页 ({deletablePageTags.length})</span>
|
||||
</label>
|
||||
<span className="admin-tags-bulkbar__count">已选 {selected.size} 个</span>
|
||||
<button
|
||||
type="button"
|
||||
className="admin-btn"
|
||||
onClick={() => setSelected(new Set())}
|
||||
disabled={selected.size === 0}
|
||||
>
|
||||
清空
|
||||
</button>
|
||||
<button
|
||||
type="button"
|
||||
className="admin-btn is-danger"
|
||||
onClick={handleBulkDelete}
|
||||
disabled={selected.size === 0 || bulkDeleting}
|
||||
>
|
||||
<Trash2 size={13} /> {bulkDeleting ? "删除中..." : `删除选中 (${selected.size})`}
|
||||
</button>
|
||||
</div>
|
||||
)}
|
||||
|
||||
{loading ? (
|
||||
<div className="admin-empty">加载中...</div>
|
||||
) : filteredTags.length === 0 ? (
|
||||
@@ -205,36 +342,110 @@ export function TagsPage() {
|
||||
没有找到匹配的标签。
|
||||
</div>
|
||||
) : (
|
||||
<div className="admin-tags-grid">
|
||||
{filteredTags.map((tag) => (
|
||||
<div key={tag.id} className="admin-tag-card">
|
||||
<div className="admin-tag-card__head">
|
||||
<span className="admin-tag-card__title">{tag.label}</span>
|
||||
<span className="admin-tag-card__source-badge" data-source={tag.source}>
|
||||
{sourceLabel(tag.source)}
|
||||
</span>
|
||||
</div>
|
||||
|
||||
{tag.aliases && tag.aliases.length > 0 && (
|
||||
<div className="admin-tag-card__aliases">
|
||||
{tag.aliases.map((alias) => (
|
||||
<span key={alias} className="admin-tag-card__alias-pill">
|
||||
{alias}
|
||||
<>
|
||||
<div className="admin-tags-grid">
|
||||
{pagedTags.map((tag) => {
|
||||
const selectable = selectMode && tag.source !== "system";
|
||||
const isSelected = selected.has(tag.id);
|
||||
return (
|
||||
<div
|
||||
key={tag.id}
|
||||
className={`admin-tag-card${selectable ? " is-selectable" : ""}${
|
||||
selectable && isSelected ? " is-selected" : ""
|
||||
}`}
|
||||
onClick={selectable ? () => toggleSelect(tag.id) : undefined}
|
||||
>
|
||||
<div className="admin-tag-card__head">
|
||||
{selectable && (
|
||||
<input
|
||||
type="checkbox"
|
||||
className="admin-tag-card__check"
|
||||
checked={isSelected}
|
||||
readOnly
|
||||
/>
|
||||
)}
|
||||
<span className="admin-tag-card__title">{tag.label}</span>
|
||||
<span className="admin-tag-card__source-badge" data-source={tag.source}>
|
||||
{sourceLabel(tag.source)}
|
||||
</span>
|
||||
))}
|
||||
</div>
|
||||
)}
|
||||
</div>
|
||||
|
||||
<div className="admin-tag-card__footer">
|
||||
<span>ID: {tag.id}</span>
|
||||
<span className="admin-tag-card__count">
|
||||
<Film size={11} />
|
||||
<strong>{tag.count} 视频</strong>
|
||||
</span>
|
||||
</div>
|
||||
{tag.aliases && tag.aliases.length > 0 && (
|
||||
<div className="admin-tag-card__aliases">
|
||||
{tag.aliases.map((alias) => (
|
||||
<span key={alias} className="admin-tag-card__alias-pill">
|
||||
{alias}
|
||||
</span>
|
||||
))}
|
||||
</div>
|
||||
)}
|
||||
|
||||
<div className="admin-tag-card__footer">
|
||||
<span className="admin-tag-card__count">
|
||||
<Film size={13} />
|
||||
<strong>{tag.count}</strong> 视频
|
||||
</span>
|
||||
<div className="admin-tag-card__footer-actions">
|
||||
<span className="admin-tag-card__id">#{tag.id}</span>
|
||||
{!selectMode && tag.source !== "system" && (
|
||||
<button
|
||||
type="button"
|
||||
className="admin-tag-card__delete"
|
||||
onClick={() => handleDelete(tag)}
|
||||
disabled={deletingId === tag.id}
|
||||
aria-label={`删除标签 ${tag.label}`}
|
||||
>
|
||||
<Trash2 size={11} />
|
||||
<span>{deletingId === tag.id ? "删除中" : "删除"}</span>
|
||||
</button>
|
||||
)}
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
);
|
||||
})}
|
||||
</div>
|
||||
|
||||
{totalPages > 1 && (
|
||||
<div className="admin-table-pagination admin-tags-pagination">
|
||||
<button
|
||||
type="button"
|
||||
className="admin-btn"
|
||||
onClick={() => setPage(1)}
|
||||
disabled={currentPage <= 1}
|
||||
>
|
||||
首页
|
||||
</button>
|
||||
<button
|
||||
type="button"
|
||||
className="admin-btn"
|
||||
onClick={() => setPage((p) => Math.max(1, p - 1))}
|
||||
disabled={currentPage <= 1}
|
||||
>
|
||||
上一页
|
||||
</button>
|
||||
<span className="admin-table-pagination__info">
|
||||
第 {currentPage} / {totalPages} 页,显示 {pageStart}-{pageEnd} / {filteredTags.length},每页 {pageSize} 个
|
||||
</span>
|
||||
<button
|
||||
type="button"
|
||||
className="admin-btn"
|
||||
onClick={() => setPage((p) => Math.min(totalPages, p + 1))}
|
||||
disabled={currentPage >= totalPages}
|
||||
>
|
||||
下一页
|
||||
</button>
|
||||
<button
|
||||
type="button"
|
||||
className="admin-btn"
|
||||
onClick={() => setPage(totalPages)}
|
||||
disabled={currentPage >= totalPages}
|
||||
>
|
||||
末页
|
||||
</button>
|
||||
</div>
|
||||
))}
|
||||
</div>
|
||||
)}
|
||||
</>
|
||||
)}
|
||||
</div>
|
||||
</div>
|
||||
@@ -242,6 +453,26 @@ export function TagsPage() {
|
||||
);
|
||||
}
|
||||
|
||||
function useTagsPageSize() {
|
||||
const [pageSize, setPageSize] = useState(() =>
|
||||
window.matchMedia(TAGS_MOBILE_QUERY).matches
|
||||
? MOBILE_TAGS_PAGE_SIZE
|
||||
: DESKTOP_TAGS_PAGE_SIZE
|
||||
);
|
||||
|
||||
useEffect(() => {
|
||||
const media = window.matchMedia(TAGS_MOBILE_QUERY);
|
||||
const update = () => {
|
||||
setPageSize(media.matches ? MOBILE_TAGS_PAGE_SIZE : DESKTOP_TAGS_PAGE_SIZE);
|
||||
};
|
||||
update();
|
||||
media.addEventListener("change", update);
|
||||
return () => media.removeEventListener("change", update);
|
||||
}, []);
|
||||
|
||||
return pageSize;
|
||||
}
|
||||
|
||||
function splitList(s: string): string[] {
|
||||
return s
|
||||
.split(/[,,、\s]+/)
|
||||
|
||||
@@ -4,6 +4,7 @@ import {
|
||||
useCallback,
|
||||
useContext,
|
||||
useEffect,
|
||||
useRef,
|
||||
useState,
|
||||
} from "react";
|
||||
|
||||
@@ -18,13 +19,25 @@ const ToastCtx = createContext<Ctx | null>(null);
|
||||
|
||||
export function ToastProvider({ children }: { children: ReactNode }) {
|
||||
const [items, setItems] = useState<Toast[]>([]);
|
||||
const timers = useRef<Record<string, ReturnType<typeof window.setTimeout>>>({});
|
||||
|
||||
// Deduplicate: same text won't stack, just resets the dismiss timer
|
||||
const show = useCallback((text: string, kind: ToastKind = "info") => {
|
||||
// Reset timer if duplicate
|
||||
if (timers.current[text]) {
|
||||
window.clearTimeout(timers.current[text]);
|
||||
timers.current[text] = window.setTimeout(() => {
|
||||
setItems((list) => list.filter((t) => t.text !== text));
|
||||
delete timers.current[text];
|
||||
}, 2600);
|
||||
return;
|
||||
}
|
||||
const id = Date.now() + Math.random();
|
||||
setItems((list) => [...list, { id, kind, text }]);
|
||||
window.setTimeout(() => {
|
||||
timers.current[text] = window.setTimeout(() => {
|
||||
setItems((list) => list.filter((t) => t.id !== id));
|
||||
delete timers.current[text];
|
||||
}, 2600);
|
||||
setItems((list) => [...list, { id, kind, text }]);
|
||||
}, []);
|
||||
|
||||
return (
|
||||
|
||||
+29
-8
@@ -77,10 +77,9 @@ export function checkUpdate() {
|
||||
|
||||
export type AdminDrive = {
|
||||
id: string;
|
||||
kind: "quark" | "p115" | "pikpak" | "wopan" | "onedrive" | "localstorage" | "spider91";
|
||||
kind: "quark" | "p115" | "pikpak" | "wopan" | "onedrive" | "googledrive" | "localstorage" | "spider91";
|
||||
name: string;
|
||||
rootId: string;
|
||||
scanRootId: string;
|
||||
status: string;
|
||||
lastError?: string;
|
||||
hasCredential: boolean;
|
||||
@@ -100,6 +99,7 @@ export type AdminDrive = {
|
||||
thumbnailReadyCount: number;
|
||||
thumbnailPendingCount: number;
|
||||
thumbnailFailedCount: number;
|
||||
thumbnailDurationPendingCount: number;
|
||||
teaserReadyCount: number;
|
||||
teaserPendingCount: number;
|
||||
teaserFailedCount: number;
|
||||
@@ -137,10 +137,9 @@ export function getDriveStorage() {
|
||||
|
||||
export type UpsertDriveInput = {
|
||||
id: string;
|
||||
kind: "quark" | "p115" | "pikpak" | "wopan" | "onedrive" | "localstorage" | "spider91";
|
||||
kind: "quark" | "p115" | "pikpak" | "wopan" | "onedrive" | "googledrive" | "localstorage" | "spider91";
|
||||
name: string;
|
||||
rootId: string;
|
||||
scanRootId: string;
|
||||
credentials: Record<string, string>;
|
||||
/**
|
||||
* 可选:写入"扫描跳过目录"集合。`undefined` 表示不变(沿用服务端旧值),
|
||||
@@ -333,6 +332,13 @@ export function createTag(label: string, aliases: string[]) {
|
||||
});
|
||||
}
|
||||
|
||||
export function deleteTag(id: number) {
|
||||
return request<{ ok: boolean; removedVideos: number }>(
|
||||
`/tags/${encodeURIComponent(String(id))}`,
|
||||
{ method: "DELETE" }
|
||||
);
|
||||
}
|
||||
|
||||
// ---------- Settings ----------
|
||||
|
||||
export type Theme = "dark" | "pink";
|
||||
@@ -369,10 +375,25 @@ export function updateSettings(body: Partial<Settings>) {
|
||||
|
||||
/**
|
||||
* 立即触发一次完整的凌晨流水线(Phase1 扫盘 + Phase2 91 爬虫 + Phase3 迁移),
|
||||
* 不论当前时间或今日是否已跑。立即返回 202;进度通过 backend 日志观察。
|
||||
* 不论当前时间或今日是否已跑。立即返回 202;进度通过任务状态和 backend 日志观察。
|
||||
*
|
||||
* 流水线已在跑时后端最多保留一个待触发请求;已有待触发请求时,新的点击会被忽略。
|
||||
* 流水线已在跑或已排队时,后端会拒绝重复触发。
|
||||
*/
|
||||
export function runNightlyJob() {
|
||||
return request<{ ok: boolean }>("/jobs/nightly/run", { method: "POST" });
|
||||
export type NightlyJobStatus = {
|
||||
state: "idle" | "queued" | "running" | "running_queued";
|
||||
running: boolean;
|
||||
queued: boolean;
|
||||
startedAt?: string;
|
||||
lastFinishedAt?: string;
|
||||
};
|
||||
|
||||
export function getNightlyJobStatus() {
|
||||
return request<NightlyJobStatus>("/jobs/nightly/status");
|
||||
}
|
||||
|
||||
export function runNightlyJob() {
|
||||
return request<{ ok: boolean; accepted: boolean; status: NightlyJobStatus }>(
|
||||
"/jobs/nightly/run",
|
||||
{ method: "POST" }
|
||||
);
|
||||
}
|
||||
|
||||
+5
-3
@@ -93,17 +93,19 @@ export type ShortsNextResponse = {
|
||||
|
||||
/**
|
||||
* 拉取短视频流的下一批候选。把当前轮已看过的 video id 列表传给后端,
|
||||
* 服务器从未在列表中的视频里随机抽 count 条返回。
|
||||
* 服务器从未在列表中的视频里随机抽 count 条返回。preferredFromVideoId
|
||||
* 来自用户最近一次点赞成功的视频,用于按相似标签优先推荐。
|
||||
*
|
||||
* 失败时返回空批 + roundComplete=false,由调用方决定是否重试。
|
||||
*/
|
||||
export function fetchShortsNext(
|
||||
seenIds: string[],
|
||||
count: number
|
||||
count: number,
|
||||
preferredFromVideoId?: string
|
||||
): Promise<ShortsNextResponse> {
|
||||
return apiJSON<ShortsNextResponse>("/api/shorts/next", {
|
||||
method: "POST",
|
||||
body: JSON.stringify({ seenIds, count }),
|
||||
body: JSON.stringify({ seenIds, count, preferredFromVideoId }),
|
||||
}).catch(() => ({ items: [], total: 0, roundComplete: false }));
|
||||
}
|
||||
|
||||
|
||||
+82
-12
@@ -120,12 +120,16 @@ export default function ShortsPage() {
|
||||
|
||||
// seenIds 用 ref 维护,方便在异步 callback 里读到最新值
|
||||
const seenIdsRef = useRef<string[]>(loadSeenIds());
|
||||
const preferredFromVideoIdRef = useRef<string | null>(null);
|
||||
|
||||
const containerRef = useRef<HTMLDivElement | null>(null);
|
||||
// 整个页面根元素,用于 requestFullscreen
|
||||
const pageRef = useRef<HTMLDivElement | null>(null);
|
||||
// index → video element,用来精确控制播放/暂停
|
||||
const videoRefs = useRef<Map<number, HTMLVideoElement>>(new Map());
|
||||
const activeIndexRef = useRef(0);
|
||||
const ignoreIntersectionUntilRef = useRef(0);
|
||||
const fullscreenRestoreTimersRef = useRef<number[]>([]);
|
||||
|
||||
// 当前是否处在浏览器全屏(Fullscreen API)状态。
|
||||
// iOS Safari 不支持元素级 Fullscreen API,这里会一直保持 false,
|
||||
@@ -139,6 +143,10 @@ export default function ShortsPage() {
|
||||
// 用户在操作栏点取消时会从这里移除,允许之后再次点赞。
|
||||
const likedIdsRef = useRef<Set<string>>(new Set());
|
||||
|
||||
useEffect(() => {
|
||||
activeIndexRef.current = activeIndex;
|
||||
}, [activeIndex]);
|
||||
|
||||
/**
|
||||
* 切换点赞状态。
|
||||
* - liked=true:发 POST /api/video/:id/like
|
||||
@@ -163,6 +171,11 @@ export default function ShortsPage() {
|
||||
);
|
||||
if (!res.ok) throw new Error(`HTTP ${res.status}`);
|
||||
const data = (await res.json()) as { likes?: number };
|
||||
if (liked) {
|
||||
preferredFromVideoIdRef.current = videoId;
|
||||
} else if (preferredFromVideoIdRef.current === videoId) {
|
||||
preferredFromVideoIdRef.current = null;
|
||||
}
|
||||
return typeof data.likes === "number" ? data.likes : null;
|
||||
} catch {
|
||||
// 请求失败:回滚集合,让 Slide 自己回滚 UI
|
||||
@@ -191,7 +204,11 @@ export default function ShortsPage() {
|
||||
setLoading(true);
|
||||
try {
|
||||
const seen = seenIdsRef.current;
|
||||
const resp = await fetchShortsNext(seen, BATCH_SIZE);
|
||||
const resp = await fetchShortsNext(
|
||||
seen,
|
||||
BATCH_SIZE,
|
||||
preferredFromVideoIdRef.current ?? undefined
|
||||
);
|
||||
if (resp.items.length === 0) {
|
||||
setEmpty((prev) => prev || true /* 维持 true 即可 */);
|
||||
setRoundComplete(true);
|
||||
@@ -250,6 +267,8 @@ export default function ShortsPage() {
|
||||
|
||||
const observer = new IntersectionObserver(
|
||||
(entries) => {
|
||||
if (Date.now() < ignoreIntersectionUntilRef.current) return;
|
||||
|
||||
let bestIndex = -1;
|
||||
let bestRatio = 0.6;
|
||||
for (const entry of entries) {
|
||||
@@ -437,11 +456,37 @@ export default function ShortsPage() {
|
||||
};
|
||||
}, []);
|
||||
|
||||
function clearFullscreenRestoreTimers() {
|
||||
for (const timer of fullscreenRestoreTimersRef.current) {
|
||||
window.clearTimeout(timer);
|
||||
}
|
||||
fullscreenRestoreTimersRef.current = [];
|
||||
}
|
||||
|
||||
function restoreActiveSlideIntoView() {
|
||||
const idx = activeIndexRef.current;
|
||||
const slide = containerRef.current?.querySelector<HTMLElement>(
|
||||
`[data-index="${idx}"]`
|
||||
);
|
||||
if (!slide) return;
|
||||
slide.scrollIntoView({ block: "start", inline: "nearest", behavior: "auto" });
|
||||
}
|
||||
|
||||
function scheduleFullscreenActiveRestore() {
|
||||
ignoreIntersectionUntilRef.current = Date.now() + 700;
|
||||
clearFullscreenRestoreTimers();
|
||||
restoreActiveSlideIntoView();
|
||||
fullscreenRestoreTimersRef.current = [80, 220, 520].map((delay) =>
|
||||
window.setTimeout(restoreActiveSlideIntoView, delay)
|
||||
);
|
||||
}
|
||||
|
||||
// ---- 浏览器全屏(Fullscreen API) ----
|
||||
// 监听全屏状态变化,保持 React state 同步。
|
||||
// 用户按 ESC / 系统返回 / 浏览器退出全屏按钮 时也会走这里。
|
||||
useEffect(() => {
|
||||
function handleChange() {
|
||||
scheduleFullscreenActiveRestore();
|
||||
setIsFullscreen(
|
||||
document.fullscreenElement !== null ||
|
||||
// Safari (desktop) 旧前缀
|
||||
@@ -454,6 +499,7 @@ export default function ShortsPage() {
|
||||
return () => {
|
||||
document.removeEventListener("fullscreenchange", handleChange);
|
||||
document.removeEventListener("webkitfullscreenchange", handleChange);
|
||||
clearFullscreenRestoreTimers();
|
||||
};
|
||||
}, []);
|
||||
|
||||
@@ -530,6 +576,7 @@ export default function ShortsPage() {
|
||||
}
|
||||
|
||||
function toggleFullscreen() {
|
||||
scheduleFullscreenActiveRestore();
|
||||
if (isFullscreen) exitPageFullscreen();
|
||||
else requestPageFullscreen();
|
||||
}
|
||||
@@ -695,6 +742,7 @@ function ShortsSlide({
|
||||
const [duration, setDuration] = useState(0);
|
||||
const [currentTime, setCurrentTime] = useState(0);
|
||||
const [scrubbing, setScrubbing] = useState(false);
|
||||
const scrubbingRef = useRef(false);
|
||||
// 拖动开始时是否在播:用于拖完后判断要不要 resume
|
||||
const wasPlayingRef = useRef(true);
|
||||
|
||||
@@ -735,6 +783,7 @@ function ShortsSlide({
|
||||
if (!isActive) {
|
||||
setPaused(false);
|
||||
setScrubbing(false);
|
||||
scrubbingRef.current = false;
|
||||
setIsBuffering(false);
|
||||
setPlayPauseHud(null);
|
||||
}
|
||||
@@ -769,7 +818,7 @@ function ShortsSlide({
|
||||
};
|
||||
const handleTime = () => {
|
||||
// 拖动期间不要被 timeupdate 覆盖 UI
|
||||
if (!scrubbing) setCurrentTime(video.currentTime);
|
||||
if (!scrubbingRef.current) setCurrentTime(video.currentTime);
|
||||
};
|
||||
const handleWaiting = () => {
|
||||
setIsBuffering(true);
|
||||
@@ -811,7 +860,7 @@ function ShortsSlide({
|
||||
video.removeEventListener("canplay", handlePlayingOrCanPlay);
|
||||
video.removeEventListener("volumechange", handleVolumeChange);
|
||||
};
|
||||
}, [shouldMount, scrubbing, muted, volume, setMuted, setVolume]);
|
||||
}, [muted, volume, setMuted, setVolume]);
|
||||
|
||||
// 长按 2 倍速:直接绑原生事件
|
||||
useEffect(() => {
|
||||
@@ -1021,11 +1070,16 @@ function ShortsSlide({
|
||||
// ---- 进度条拖动 ----
|
||||
// 触摸进度条时:暂停 → 跟随手指更新 currentTime → 松手 resume
|
||||
function handleProgressPointerDown(e: React.PointerEvent<HTMLDivElement>) {
|
||||
const video = localRef.current;
|
||||
if (!video || !duration) return;
|
||||
e.preventDefault();
|
||||
e.stopPropagation();
|
||||
(e.currentTarget as HTMLElement).setPointerCapture(e.pointerId);
|
||||
const video = localRef.current;
|
||||
const seekDuration = getSeekDuration(video);
|
||||
if (!video || !seekDuration) return;
|
||||
try {
|
||||
(e.currentTarget as HTMLElement).setPointerCapture(e.pointerId);
|
||||
} catch {
|
||||
// ignore
|
||||
}
|
||||
wasPlayingRef.current = !video.paused;
|
||||
if (!video.paused) {
|
||||
try {
|
||||
@@ -1034,17 +1088,19 @@ function ShortsSlide({
|
||||
// ignore
|
||||
}
|
||||
}
|
||||
scrubbingRef.current = true;
|
||||
setScrubbing(true);
|
||||
applyProgressFromEvent(e);
|
||||
applyProgressFromEvent(e, seekDuration);
|
||||
}
|
||||
function handleProgressPointerMove(e: React.PointerEvent<HTMLDivElement>) {
|
||||
if (!scrubbing) return;
|
||||
if (!scrubbingRef.current) return;
|
||||
e.preventDefault();
|
||||
e.stopPropagation();
|
||||
applyProgressFromEvent(e);
|
||||
}
|
||||
function handleProgressPointerEnd(e: React.PointerEvent<HTMLDivElement>) {
|
||||
if (!scrubbing) return;
|
||||
if (!scrubbingRef.current) return;
|
||||
e.preventDefault();
|
||||
e.stopPropagation();
|
||||
try {
|
||||
(e.currentTarget as HTMLElement).releasePointerCapture(e.pointerId);
|
||||
@@ -1052,17 +1108,30 @@ function ShortsSlide({
|
||||
// ignore
|
||||
}
|
||||
const video = localRef.current;
|
||||
scrubbingRef.current = false;
|
||||
setScrubbing(false);
|
||||
if (video && wasPlayingRef.current) {
|
||||
video.play().catch(() => undefined);
|
||||
}
|
||||
}
|
||||
function applyProgressFromEvent(e: React.PointerEvent<HTMLDivElement>) {
|
||||
function getSeekDuration(video: HTMLVideoElement | null) {
|
||||
if (duration > 0) return duration;
|
||||
if (video && Number.isFinite(video.duration) && video.duration > 0) {
|
||||
setDuration(video.duration);
|
||||
return video.duration;
|
||||
}
|
||||
return 0;
|
||||
}
|
||||
function applyProgressFromEvent(
|
||||
e: React.PointerEvent<HTMLDivElement>,
|
||||
knownDuration?: number
|
||||
) {
|
||||
const video = localRef.current;
|
||||
if (!video || !duration) return;
|
||||
const seekDuration = knownDuration ?? getSeekDuration(video);
|
||||
if (!video || !seekDuration) return;
|
||||
const rect = e.currentTarget.getBoundingClientRect();
|
||||
const ratio = clamp((e.clientX - rect.left) / rect.width, 0, 1);
|
||||
const next = ratio * duration;
|
||||
const next = ratio * seekDuration;
|
||||
setCurrentTime(next);
|
||||
try {
|
||||
video.currentTime = next;
|
||||
@@ -1235,6 +1304,7 @@ function ShortsSlide({
|
||||
onPointerMove={handleProgressPointerMove}
|
||||
onPointerUp={handleProgressPointerEnd}
|
||||
onPointerCancel={handleProgressPointerEnd}
|
||||
onLostPointerCapture={handleProgressPointerEnd}
|
||||
onClick={(e) => e.stopPropagation()}
|
||||
>
|
||||
<div
|
||||
|
||||
@@ -7,7 +7,7 @@ import { uploadVideo } from "@/data/videos";
|
||||
import { defaultUploadTitleFromFileName } from "@/lib/uploadTitle";
|
||||
import type { VideoItem } from "@/types";
|
||||
|
||||
const UPLOAD_TAGS = ["奶子", "臀", "口角", "女大", "人妻", "AV"];
|
||||
const UPLOAD_TAGS = ["奶子", "臀", "口交", "女大", "人妻", "AV"];
|
||||
|
||||
export default function UploadPage() {
|
||||
const [file, setFile] = useState<File | null>(null);
|
||||
|
||||
+105
-18
@@ -2128,28 +2128,51 @@
|
||||
}
|
||||
|
||||
.admin-tags-grid {
|
||||
column-count: 3;
|
||||
column-gap: var(--space-3);
|
||||
display: grid;
|
||||
grid-template-columns: repeat(auto-fill, minmax(240px, 1fr));
|
||||
gap: var(--space-3);
|
||||
align-items: stretch;
|
||||
}
|
||||
|
||||
@media (max-width: 1200px) {
|
||||
.admin-tags-grid {
|
||||
column-count: 2;
|
||||
}
|
||||
.admin-tags-bulkbar {
|
||||
display: flex;
|
||||
align-items: center;
|
||||
gap: var(--space-3);
|
||||
flex-wrap: wrap;
|
||||
margin-bottom: var(--space-4);
|
||||
padding: var(--space-2) var(--space-3);
|
||||
background: var(--bg-sunken);
|
||||
border: 1px solid var(--border-subtle);
|
||||
border-radius: var(--radius-sm);
|
||||
}
|
||||
|
||||
@media (max-width: 600px) {
|
||||
.admin-tags-grid {
|
||||
column-count: 1;
|
||||
}
|
||||
.admin-tags-bulkbar__count {
|
||||
font-size: var(--font-xs);
|
||||
color: var(--text-muted);
|
||||
margin-left: auto;
|
||||
}
|
||||
|
||||
.admin-tag-card.is-selectable {
|
||||
cursor: pointer;
|
||||
user-select: none;
|
||||
}
|
||||
|
||||
.admin-tag-card.is-selected {
|
||||
border-color: var(--border-accent);
|
||||
box-shadow: 0 0 0 1px var(--border-accent), var(--shadow-md);
|
||||
}
|
||||
|
||||
.admin-tag-card__check {
|
||||
width: 15px;
|
||||
height: 15px;
|
||||
accent-color: var(--accent);
|
||||
cursor: pointer;
|
||||
flex-shrink: 0;
|
||||
}
|
||||
|
||||
.admin-tag-card {
|
||||
break-inside: avoid;
|
||||
display: inline-flex;
|
||||
display: flex;
|
||||
flex-direction: column;
|
||||
width: 100%;
|
||||
margin-bottom: var(--space-3);
|
||||
background: var(--bg-surface);
|
||||
border: 1px solid var(--border-subtle);
|
||||
border-radius: var(--radius-sm);
|
||||
@@ -2176,6 +2199,7 @@
|
||||
font-size: var(--font-md);
|
||||
font-weight: var(--weight-bold);
|
||||
color: var(--text-strong);
|
||||
margin-right: auto;
|
||||
}
|
||||
|
||||
.admin-tag-card__source-badge {
|
||||
@@ -2228,7 +2252,7 @@
|
||||
display: flex;
|
||||
align-items: center;
|
||||
justify-content: space-between;
|
||||
margin-top: var(--space-2);
|
||||
margin-top: auto;
|
||||
padding-top: var(--space-2);
|
||||
border-top: 1px solid var(--border-subtle);
|
||||
font-size: var(--font-xs);
|
||||
@@ -2237,8 +2261,71 @@
|
||||
|
||||
.admin-tag-card__count {
|
||||
display: inline-flex;
|
||||
align-items: center;
|
||||
align-items: baseline;
|
||||
gap: 4px;
|
||||
font-weight: var(--weight-semibold);
|
||||
color: var(--text-default);
|
||||
font-size: var(--font-xs);
|
||||
color: var(--text-muted);
|
||||
}
|
||||
|
||||
.admin-tag-card__count svg {
|
||||
align-self: center;
|
||||
color: var(--accent);
|
||||
}
|
||||
|
||||
.admin-tag-card__count strong {
|
||||
font-size: var(--font-md);
|
||||
font-weight: var(--weight-bold);
|
||||
color: var(--text-strong);
|
||||
}
|
||||
|
||||
.admin-tag-card__id {
|
||||
font-size: 10px;
|
||||
color: var(--text-faint);
|
||||
font-variant-numeric: tabular-nums;
|
||||
}
|
||||
|
||||
.admin-tag-card__footer-actions {
|
||||
display: inline-flex;
|
||||
align-items: center;
|
||||
gap: var(--space-2);
|
||||
}
|
||||
|
||||
.admin-tag-card__delete {
|
||||
border: 1px solid var(--danger);
|
||||
background: var(--danger-soft);
|
||||
color: var(--danger);
|
||||
border-radius: var(--radius-xs);
|
||||
padding: 3px 6px;
|
||||
display: inline-flex;
|
||||
align-items: center;
|
||||
gap: 3px;
|
||||
font-size: 10px;
|
||||
font-weight: var(--weight-semibold);
|
||||
cursor: pointer;
|
||||
transition: all var(--transition-fast);
|
||||
}
|
||||
|
||||
.admin-tag-card__delete:hover:not(:disabled) {
|
||||
background: var(--danger);
|
||||
color: white;
|
||||
}
|
||||
|
||||
/* 仅在支持悬停的设备上隐藏删除按钮,悬停或键盘聚焦卡片时显示,避免满屏删除按钮 */
|
||||
@media (hover: hover) {
|
||||
.admin-tag-card__delete {
|
||||
opacity: 0;
|
||||
pointer-events: none;
|
||||
transition: opacity var(--transition-fast);
|
||||
}
|
||||
|
||||
.admin-tag-card:hover .admin-tag-card__delete,
|
||||
.admin-tag-card:focus-within .admin-tag-card__delete {
|
||||
opacity: 1;
|
||||
pointer-events: auto;
|
||||
}
|
||||
}
|
||||
|
||||
.admin-tag-card__delete:disabled {
|
||||
opacity: 0.55;
|
||||
cursor: not-allowed;
|
||||
}
|
||||
|
||||
@@ -24,14 +24,16 @@ test("spider91 upload target uses explicit local-save option instead of auto tar
|
||||
assert.doesNotMatch(drivesPageSource, /自动模式/);
|
||||
});
|
||||
|
||||
test("onedrive drive form only exposes required default-app fields", () => {
|
||||
assert.match(
|
||||
drivesPageSource,
|
||||
/form\.kind !== "spider91" &&\s*form\.kind !== "onedrive" &&\s*form\.kind !== "localstorage" &&\s*form\.kind !== "pikpak"/
|
||||
);
|
||||
test("drive form shows a root directory id field for all drive kinds", () => {
|
||||
assert.match(drivesPageSource, /<label>根目录 ID<\/label>/);
|
||||
assert.match(drivesPageSource, /placeholder=\{rootIdPlaceholder\(form\.kind\)\}/);
|
||||
assert.doesNotMatch(drivesPageSource, /扫描起点目录 ID/);
|
||||
assert.doesNotMatch(drivesPageSource, /set\("scanRootId"/);
|
||||
});
|
||||
|
||||
test("onedrive drive form only exposes required default-app fields", () => {
|
||||
const match =
|
||||
/function credentialFields[\s\S]*?case "onedrive":\s*return \[([\s\S]*?)\];\s*case "spider91":/.exec(
|
||||
/function credentialFields[\s\S]*?case "onedrive":\s*return \[([\s\S]*?)\];\s*case "googledrive":/.exec(
|
||||
drivesPageSource
|
||||
);
|
||||
assert.ok(match, "onedrive credential field block should be present");
|
||||
@@ -45,6 +47,23 @@ test("onedrive drive form only exposes required default-app fields", () => {
|
||||
assert.doesNotMatch(fields, /key: "site_id"/);
|
||||
});
|
||||
|
||||
test("googledrive drive form only exposes refresh token", () => {
|
||||
assert.match(drivesPageSource, /<option value="googledrive">Google Drive<\/option>/);
|
||||
|
||||
const match =
|
||||
/case "googledrive":\s*return \[([\s\S]*?)\];\s*case "localstorage":/.exec(
|
||||
drivesPageSource
|
||||
);
|
||||
assert.ok(match, "googledrive credential field block should be present");
|
||||
const fields = match[1];
|
||||
|
||||
assert.match(fields, /key: "refresh_token"/);
|
||||
assert.doesNotMatch(fields, /key: "access_token"/);
|
||||
assert.doesNotMatch(fields, /key: "api_url_address"/);
|
||||
assert.doesNotMatch(fields, /key: "client_id"/);
|
||||
assert.doesNotMatch(fields, /key: "client_secret"/);
|
||||
});
|
||||
|
||||
test("pikpak drive form only exposes account login fields", () => {
|
||||
const match =
|
||||
/case "pikpak":\s*return \[([\s\S]*?)\];\s*case "wopan":/.exec(
|
||||
@@ -82,12 +101,13 @@ test("drive type selector keeps primary source order", () => {
|
||||
drivesPageSource.matchAll(/<option value="([^"]+)">([^<]+)<\/option>/g),
|
||||
(match) => ({ value: match[1], label: match[2] })
|
||||
);
|
||||
const driveOptions = options.slice(0, 7);
|
||||
const driveOptions = options.slice(0, 8);
|
||||
|
||||
assert.deepEqual(driveOptions, [
|
||||
{ value: "p115", label: "115 网盘" },
|
||||
{ value: "pikpak", label: "PikPak" },
|
||||
{ value: "onedrive", label: "OneDrive" },
|
||||
{ value: "googledrive", label: "Google Drive" },
|
||||
{ value: "localstorage", label: "本地存储" },
|
||||
{ value: "spider91", label: "91 Spider" },
|
||||
{ value: "quark", label: "夸克网盘" },
|
||||
|
||||
@@ -0,0 +1,22 @@
|
||||
import assert from "node:assert/strict";
|
||||
import { readFileSync } from "node:fs";
|
||||
import test from "node:test";
|
||||
|
||||
const tagsPageSource = readFileSync(
|
||||
new URL("../src/admin/TagsPage.tsx", import.meta.url),
|
||||
"utf8"
|
||||
);
|
||||
|
||||
test("admin tags page limits visible tags by viewport", () => {
|
||||
assert.match(tagsPageSource, /const DESKTOP_TAGS_PAGE_SIZE = 25;/);
|
||||
assert.match(tagsPageSource, /const MOBILE_TAGS_PAGE_SIZE = 8;/);
|
||||
assert.match(tagsPageSource, /const TAGS_MOBILE_QUERY = "\(max-width: 640px\)";/);
|
||||
assert.match(tagsPageSource, /window\.matchMedia\(TAGS_MOBILE_QUERY\)/);
|
||||
});
|
||||
|
||||
test("admin tags page renders only the current page", () => {
|
||||
assert.match(tagsPageSource, /filteredTags\.slice\(pageStartIndex, pageEndIndex\)/);
|
||||
assert.match(tagsPageSource, /pagedTags\.map\(\(tag\) =>/);
|
||||
assert.doesNotMatch(tagsPageSource, /filteredTags\.map\(\(tag\) =>/);
|
||||
assert.match(tagsPageSource, /全选本页/);
|
||||
});
|
||||
@@ -0,0 +1,48 @@
|
||||
import assert from "node:assert/strict";
|
||||
import { readFileSync } from "node:fs";
|
||||
import test from "node:test";
|
||||
|
||||
const shortsPageSource = readFileSync(
|
||||
new URL("../src/pages/ShortsPage.tsx", import.meta.url),
|
||||
"utf8"
|
||||
);
|
||||
|
||||
test("shorts recommendation preference follows successful likes instead of watch time", () => {
|
||||
assert.doesNotMatch(shortsPageSource, /currentTime\s*>=\s*3/);
|
||||
assert.doesNotMatch(shortsPageSource, /onPreferenceReady/);
|
||||
|
||||
const match = /const handleLikeToggle[\s\S]*?const hasLiked/.exec(
|
||||
shortsPageSource
|
||||
);
|
||||
assert.ok(match, "handleLikeToggle block should be present");
|
||||
|
||||
assert.match(
|
||||
match[0],
|
||||
/if \(liked\) \{\s*preferredFromVideoIdRef\.current = videoId;\s*\} else if \(preferredFromVideoIdRef\.current === videoId\) \{\s*preferredFromVideoIdRef\.current = null;/
|
||||
);
|
||||
});
|
||||
|
||||
test("shorts progress dragging uses immediate pointer state", () => {
|
||||
assert.match(shortsPageSource, /const scrubbingRef = useRef\(false\)/);
|
||||
assert.match(shortsPageSource, /scrubbingRef\.current = true;/);
|
||||
assert.match(shortsPageSource, /if \(!scrubbingRef\.current\) return;/);
|
||||
assert.doesNotMatch(shortsPageSource, /if \(!scrubbing\) return;/);
|
||||
assert.match(shortsPageSource, /function getSeekDuration/);
|
||||
assert.match(shortsPageSource, /onLostPointerCapture=\{handleProgressPointerEnd\}/);
|
||||
});
|
||||
|
||||
test("shorts fullscreen changes preserve the active slide", () => {
|
||||
assert.match(shortsPageSource, /const activeIndexRef = useRef\(0\)/);
|
||||
assert.match(shortsPageSource, /const ignoreIntersectionUntilRef = useRef\(0\)/);
|
||||
assert.match(
|
||||
shortsPageSource,
|
||||
/if \(Date\.now\(\) < ignoreIntersectionUntilRef\.current\) return;/
|
||||
);
|
||||
assert.match(shortsPageSource, /function scheduleFullscreenActiveRestore\(\)/);
|
||||
assert.match(shortsPageSource, /scheduleFullscreenActiveRestore\(\);\s*setIsFullscreen/);
|
||||
assert.match(
|
||||
shortsPageSource,
|
||||
/function toggleFullscreen\(\) \{\s*scheduleFullscreenActiveRestore\(\);/
|
||||
);
|
||||
assert.match(shortsPageSource, /scrollIntoView\(\{ block: "start", inline: "nearest", behavior: "auto" \}\)/);
|
||||
});
|
||||
@@ -1441,7 +1441,7 @@ VideoProject/
|
||||
| 项 | 决定 |
|
||||
|---|---|
|
||||
| 登录方式 | **B**:管理后台做完整登录流程。115 扫码、夸克扫码或 Cookie 导入、沃盘手机号 + 短信验证。Token 持久化到 SQLite 并自动刷新。 |
|
||||
| 元数据来源 | **默认文件名解析**:`标题.mp4` 或 `[tag1,tag2] 标题 - 作者.mp4`;同时提供后台录入 API 覆盖字段 |
|
||||
| 元数据来源 | **默认文件名解析**:`标题.mp4`、`标题 - 作者.mp4`,或带前缀的 `[前缀] 标题 - 作者.mp4`;前缀只用于标题清理,不作为任意标签列表入库。标签来自系统 / 用户标签匹配和目录合集规则;同时提供后台录入 API 覆盖字段 |
|
||||
| Hover teaser | **C 预生成**:scanner 发现新视频时异步生成 10s teaser 并存回网盘的 `previews/` 目录,详情页和列表页 hover 都秒开 |
|
||||
| 部署目标 | Linux 服务器;本地 Windows 开发 |
|
||||
| 扫描策略 | 启动时全量 + 每 6 小时增量 + 支持手动触发 |
|
||||
@@ -1485,14 +1485,14 @@ type StreamLink struct {
|
||||
|
||||
### 15.5 文件名解析规则
|
||||
|
||||
默认解析顺序(取第一个匹配):
|
||||
默认解析顺序(取第一个匹配),用于提取 `title` / `author`:
|
||||
|
||||
1. 完整格式:`[tag1,tag2] 标题 - 作者.ext`
|
||||
2. 去作者:`[tag1,tag2] 标题.ext`
|
||||
3. 去标签:`标题 - 作者.ext`
|
||||
1. 带前缀和作者:`[前缀] 标题 - 作者.ext`
|
||||
2. 带前缀:`[前缀] 标题.ext`
|
||||
3. 带作者:`标题 - 作者.ext`
|
||||
4. 最简单:`标题.ext`
|
||||
|
||||
解析出的字段:`title` / `author` / `tags[]`。其余字段(`duration` / `views` / `favorites` 等)由 scanner 读取文件元数据或置默认值。
|
||||
开头的 `[前缀]` 只会从标题里剥离,不会按 `,` / `,` / `、` / 空格拆成任意标签入库。`tags[]` 由 scanner 另行生成:文件名、作者和目录名命中系统标签或已有标签的标签名 / 别名时自动打标;符合条件的目录名会创建 `collection` 合集标签;常见番号类文本会归并为 `AV`。当前内置系统标签是 `后入`、`奶子`、`口交`、`臀`、`人妻`、`女大`、`AV`。其余字段(`duration` / `views` / `favorites` 等)由 scanner 读取文件元数据或置默认值。
|
||||
|
||||
后台录入接口可用来覆盖解析结果:
|
||||
|
||||
@@ -1570,6 +1570,10 @@ POST /admin/api/videos # 手动新建
|
||||
PUT /admin/api/videos/:id # 修改元数据
|
||||
DELETE /admin/api/videos/:id
|
||||
POST /admin/api/videos/:id/regen-preview
|
||||
|
||||
GET /admin/api/tags # 标签列表
|
||||
POST /admin/api/tags # 新增标签并自动归类历史视频
|
||||
DELETE /admin/api/tags/:id # 删除非系统标签,并从所有视频上移除
|
||||
```
|
||||
|
||||
登录流程三家各不相同:
|
||||
@@ -1684,9 +1688,10 @@ Teaser 不再是"固定从第 10 秒抽 10 秒",改为按视频时长分段挑
|
||||
- `backend/internal/catalog/tags.go` `migrate` / `pruneOrphanCollectionTags` / `pruneOrphanCollectionTagsByID` / `collectVideoTagIDs`
|
||||
- 测试:`backend/internal/catalog/tags_test.go` `TestDeleteVideoPrunesOrphanCollectionTag` / `TestMigratePrunesPreexistingOrphanCollectionTags`
|
||||
|
||||
**已知不在本次范围**:
|
||||
- `/admin/api/tags` 仍只有 `GET` / `POST`,没有 `DELETE`。如果将来要让管理员手动删 `user` 标签,再加 endpoint。
|
||||
- 数据迁移:上线时对运行中数据库一次性执行同样的 `DELETE` 即可(已对当前实例执行:清掉 10 条 `Season N` / `Better Call Saul SXX` / `东京爱情故事(1991)`,`tags` 总数 153 → 143)。
|
||||
**手动删除标签**:
|
||||
- `/admin/api/tags/{id}` 支持 `DELETE`。管理员可手动删除非系统标签,删除时同步清理 `video_tags` 并刷新相关视频的 `videos.tags` JSON。
|
||||
- `system` 标签由固定标签池维护,不开放删除;`user` / `collection` / `legacy` 标签可由管理员按需删除。
|
||||
- 历史孤儿 `collection` 标签仍由迁移自愈逻辑自动清理。
|
||||
|
||||
### 14.7 取消浏览器内本机转码,全部走 302 直链 + VLC 外部播放器按钮(2026-05-21)
|
||||
|
||||
|
||||
Reference in New Issue
Block a user