admin/91
1
0
Fork 0
mirror of https://github.com/nianzhibai/91.git synced 2026-06-15 08:45:41 +08:00
Code Issues Packages Projects Releases Wiki Activity

Initial commit: video aggregator with quark/115/wopan drivers

Browse Source
This commit is contained in:
nianzhibai
2026-05-10 17:07:20 +08:00
commit 87866858b8
2028 changed files with 5803208 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
.gitignore
+21
View File
@@ -0,0 +1,21 @@
node_modules
dist
.vite
*.log
.DS_Store
.vscode/
.idea/
# 第三方源码参考，可选，详见 vendor-refs/README.md
vendor-refs/
# 后端数据目录（SQLite + teaser/封面）
backend/data/
backend/config.yaml
# 工具链
tools/
# 注意：backend/vendor/ 是 Go modules vendor 目录，故意入库
# 目的是让任何人 clone 后断网也能直接 go build
# 不要加 `vendor/` 或 `backend/vendor/` 到这里
README.md
+111
View File
@@ -0,0 +1,111 @@
# 视频聚合站
把夸克 / 115 / 联通沃盘作为存储后端的视频聚合前台。按 `video-site-implementation-plan.md` 的设计实现。
- 前端：React 18 + Vite + TypeScript
- 后端：Go 1.23，SQLite（纯 Go 驱动，无 CGO），ffmpeg 生成 teaser 和封面
- 三家网盘接入：夸克自研 + 115driver SDK + wopan-sdk-go SDK
## 快速开始
### 环境要求
- Node.js 18+ 和 npm
- Go 1.23+
- ffmpeg 和 ffprobe（用于生成预览 teaser 和抽封面）
Windows 用户可以把 Go 和 ffmpeg 解压到 `%USERPROFILE%\tools\`，然后把 `\tools\go\bin` 和 `\tools\ffmpeg\bin` 加到 PATH 即可，不需要管理员权限。
### 运行
```bash
# 前端
npm install
npm run dev # 监听 http://127.0.0.1:5173
# 后端（另开终端）
cd backend
go run ./cmd/server # 监听 :8080，依赖已 vendor 入库，无需 go mod tidy
```
首次启动后端会自动生成：
- `backend/config.yaml`（从 `config.example.yaml` 复制）
- `backend/data/video-site.db`（SQLite）
- `backend/data/previews/`（teaser 和封面本地目录）
Vite dev server 已配置把 `/api`、`/p`、`/admin/api` 反代到 `:8080`。浏览器访问 `http://127.0.0.1:5173/` 进入前台，`/admin` 进入管理后台（默认 `admin` / `admin123`，请在 `config.yaml` 里改）。
## 目录
```
.
├─ src/ React 前端
├─ backend/ Go 后端（单体服务）
│ └─ vendor/ Go 依赖全量源码，入库，支持完全离线构建
├─ vendor-refs/ 可选的阅读资料，.gitignore 忽略
│ └─ OpenList-4.2.1/ OpenList 完整源码，网盘协议对接参考
├─ video-site-implementation-plan.md 完整的设计和实现记录
└─ README.md
```
### 依赖管理
所有 Go 依赖都已通过 `go mod vendor` 打包进 `backend/vendor/` 并入库。别人 clone 仓库后，**无需联网**，直接 `go run ./cmd/server` 就能编译运行。
升级依赖的流程：
```bash
cd backend
go get github.com/SheltonZhu/115driver@<新版本>
go mod tidy
go mod vendor # 把新依赖同步到 vendor 目录
git add vendor/ # 入库
```
### `vendor-refs/` 要不要在意？
不需要。它只存 OpenList 源码作协议参考，删除或保留都不影响项目编译。
## 加一个网盘
1. 登录 `/admin` → 网盘管理 → 新建
2. 选类型（夸克 / 115 / 沃盘），填名称 + 凭证
3. 保存后会自动触发一次扫描
4. 在 `/admin/videos` 里看扫到了多少视频
5. 侧栏底部 **Teaser 生成** 开关开着，就会按配置给每个视频生成封面和 10 秒 teaser
三家盘的凭证字段：
| 类型 | 凭证字段 | 获取方式 |
|---|---|---|
| 夸克 | `cookie` | pan.quark.cn 登录后 F12 拷 Cookie |
| 115 | `cookie` | 115.com 登录后拷 Cookie（`UID=...; CID=...; SEID=...; KID=...`） |
| 沃盘 | `access_token`、`refresh_token`、可选 `family_id` | 第一版只能手动粘贴 token；后续会加扫码/短信登录 |
## Teaser 和封面生成策略
- 封面：根据视频时长从 20% 或 30% 位置抽一帧 jpg
- Teaser：3 段拼接（`20% / 50% / 80%` 位置各 3 秒，带 0.2s fade-in/out），总长约 9 秒，目标体积 500 KB - 1.5 MB
- 短视频 (< 30s) 自动降级为单段
- 首次失败的任务标 `preview_status = failed`，不再自动重试；管理后台可手动重生
- 详见 plan 15.12 节
## 部署到 Linux
```bash
# 本机交叉编译
cd backend
GOOS=linux GOARCH=amd64 go build -o video-server ./cmd/server
# 目标服务器
sudo apt install ffmpeg
scp video-server user@host:/opt/video-site/
# 配 systemd + nginx 反代到 /、/api、/p、/admin
```
完整部署方式见 plan 15.10 节。
## 贡献
任何代码改动请保持和 `video-site-implementation-plan.md` 同步；重要的设计决策追加到第 14 节（实现备注）或第 15 节（后端）。
backend/README.md
+133
View File
@@ -0,0 +1,133 @@
# backend
视频聚合站的 Go 后端。提供三件事：
1. 三家网盘统一抽象（夸克 / 115 / 联通沃盘）
2. 视频元数据目录（SQLite）+ 扫描 + teaser 预生成
3. REST API（前台）+ 管理后台 + 直链代理
## 目录
```
cmd/server/main.go 入口
internal/
config/ YAML 配置
catalog/ SQLite 元数据
drives/
iface.go Drive 接口
quark/ 夸克（自己实现，参考 OpenList quark_uc）
p115/ 115（壳子 + SheltonZhu/115driver）
wopan/ 联通沃盘（壳子 + OpenListTeam/wopan-sdk-go）
scanner/ 扫目录 → 落库
preview/ ffmpeg 抽 10s teaser
proxy/ /p/stream/*、/p/preview/* 代理
auth/ 管理员 session
api/ REST 路由
config.example.yaml 配置模板
```
## 开发环境（Windows）
本仓库假设工具都装在用户目录，不需要管理员权限。
```
C:\Users\<you>\tools\
go\bin\go.exe Go 1.23+
ffmpeg\bin\ffmpeg.exe 任意 ≥ 4.x 版本
```
并加到 `PATH`。
### 第一次启动
```powershell
cd F:\VideoProject\backend
go mod tidy
go run ./cmd/server
```
首次启动会在当前目录创建：
- `config.yaml`（从 `config.example.yaml` 复制）
- `data/video-site.db`
- `data/previews/`
默认监听 `:8080`，默认管理员 `admin / admin123`（务必在 `config.yaml` 里改）。
### 连接前端
`vite.config.ts` 已经把 `/api`、`/p`、`/admin` 代理到 `8080`。
```
npm run dev 前端 5173
go run ./cmd/server 后端 8080
```
## 添加一个盘
1. 登录管理后台：`POST /admin/api/login` body `{"username":"admin","password":"admin123"}`
2. 新建盘：`POST /admin/api/drives`
```json
{
"id": "my-quark",
"kind": "quark",
"name": "我的夸克盘",
"rootId": "0",
"scanRootId": "0",
"credentials": {
"cookie": "粘贴浏览器 F12 复制的 pan.quark.cn Cookie"
}
}
```
3. 手动触发扫描：`POST /admin/api/drives/my-quark/rescan`
三家盘的凭证字段：
| kind | credentials 字段 |
|--------|---------------------------------------------------------------|
| quark | `cookie` |
| p115 | `cookie`（形如 `UID=...; CID=...; SEID=...; KID=...`） |
| wopan | `access_token`、`refresh_token`，可选 `family_id` |
## 文件名约定
扫描器按以下顺序解析文件名：
1. `[tag1,tag2] 标题 - 作者.mp4`
2. `[tag1,tag2] 标题.mp4`
3. `标题 - 作者.mp4`
4. `标题.mp4`
标签分隔符支持 `, ， 、` 和空格。解析结果可在管理后台覆盖。
## Teaser 生成
scanner 扫到新视频会把 `(driveID, videoID)` 丢进 worker 队列，调用：
```
ffmpeg -ss 10 -headers "UA/Cookie/Referer" -i <直链> \
-t 10 -an -vf scale=480:-2 -c:v libx264 -preset veryfast -crf 28 \
-movflags +faststart -y <local>.mp4
```
优先把 teaser 上传回网盘的 `previews/` 目录；失败时保留本地 `data/previews/<videoID>.mp4` 作为兜底。
前端卡片的 `previewSrc` 统一指向 `/p/preview/<videoID>`，后端自动选择网盘代理或本地文件。
## 部署到 Linux
```bash
# 交叉编译
GOOS=linux GOARCH=amd64 go build -o video-server ./cmd/server
# 目标机
sudo apt install ffmpeg
scp video-server user@host:/opt/video-site/
ssh user@host
cd /opt/video-site
cp config.example.yaml config.yaml
# 改密码、监听地址
./video-server
```
配 systemd + nginx 反代到 `/` 和 `/api`、`/p`、`/admin`。
backend/cmd/server/main.go
+425
View File
@@ -0,0 +1,425 @@
package main
import (
"context"
"fmt"
"log"
"net/http"
"os"
"os/signal"
"path/filepath"
"sync"
"syscall"
"time"
"github.com/go-chi/chi/v5"
"github.com/go-chi/chi/v5/middleware"
"github.com/video-site/backend/internal/api"
"github.com/video-site/backend/internal/auth"
"github.com/video-site/backend/internal/catalog"
"github.com/video-site/backend/internal/config"
"github.com/video-site/backend/internal/drives"
"github.com/video-site/backend/internal/drives/p115"
"github.com/video-site/backend/internal/drives/quark"
"github.com/video-site/backend/internal/drives/wopan"
"github.com/video-site/backend/internal/preview"
"github.com/video-site/backend/internal/proxy"
"github.com/video-site/backend/internal/scanner"
)
func main() {
cfgPath := "./config.yaml"
if v := os.Getenv("VIDEO_CONFIG"); v != "" {
cfgPath = v
}
cfg, err := config.Load(cfgPath)
if err != nil {
log.Fatalf("load config: %v", err)
}
if err := os.MkdirAll(filepath.Dir(cfg.Storage.DBPath), 0o755); err != nil {
log.Fatalf("mkdir db dir: %v", err)
}
if err := os.MkdirAll(cfg.Storage.LocalPreviewDir, 0o755); err != nil {
log.Fatalf("mkdir preview dir: %v", err)
}
cat, err := catalog.Open(cfg.Storage.DBPath)
if err != nil {
log.Fatalf("open catalog: %v", err)
}
defer cat.Close()
app := &App{
cfg: cfg,
cat: cat,
registry: proxy.NewRegistry(),
workers: make(map[string]*preview.Worker),
}
app.proxy = proxy.New(app.registry)
// 初始化现有 drives
ctx, cancel := context.WithCancel(context.Background())
defer cancel()
app.loadPreviewEnabled(ctx)
existing, err := cat.ListDrives(ctx)
if err != nil {
log.Fatalf("list drives: %v", err)
}
for _, d := range existing {
if err := app.attachDrive(ctx, d); err != nil {
log.Printf("[drive %s] attach failed: %v", d.ID, err)
}
}
authr := &auth.Authenticator{
Username: cfg.Server.Admin.Username,
Password: cfg.Server.Admin.Password,
Catalog: cat,
}
apiServer := &api.Server{
Catalog: cat,
Proxy: app.proxy,
LocalDir: cfg.Storage.LocalPreviewDir,
}
adminServer := &api.AdminServer{
Catalog: cat,
Auth: authr,
OnDriveSaved: func(driveID string) error {
d, err := cat.GetDrive(ctx, driveID)
if err != nil {
return err
}
return app.attachDrive(ctx, d)
},
OnDriveRemoved: func(driveID string) {
app.detachDrive(driveID)
},
OnScanRequested: func(driveID string) {
go app.runScan(ctx, driveID)
},
OnRegenPreview: func(videoID string) {
go app.regenPreview(ctx, videoID)
},
GetPreviewEnabled: func() bool { return app.PreviewEnabled() },
SetPreviewEnabled: func(enabled bool) error {
return app.SetPreviewEnabled(ctx, enabled)
},
}
r := chi.NewRouter()
r.Use(middleware.Logger)
r.Use(middleware.Recoverer)
r.Use(corsMiddleware)
apiServer.RegisterRoutes(r, authr)
adminServer.Register(r)
// 启动定时扫描
go app.scanLoop(ctx)
srv := &http.Server{
Addr: cfg.Server.Listen,
Handler: r,
}
go func() {
log.Printf("video-site backend listening on %s", cfg.Server.Listen)
if err := srv.ListenAndServe(); err != nil && err != http.ErrServerClosed {
log.Fatalf("server error: %v", err)
}
}()
// 等待退出信号
sigs := make(chan os.Signal, 1)
signal.Notify(sigs, syscall.SIGINT, syscall.SIGTERM)
<-sigs
log.Println("shutting down...")
shutCtx, shutCancel := context.WithTimeout(context.Background(), 5*time.Second)
defer shutCancel()
_ = srv.Shutdown(shutCtx)
}
// ---------- App ----------
type App struct {
cfg *config.Config
cat *catalog.Catalog
registry *proxy.Registry
proxy *proxy.Proxy
mu sync.Mutex
workers map[string]*preview.Worker
cancels map[string]context.CancelFunc
// 运行时 preview 开关（从 DB 读）
previewEnabled bool
}
// PreviewEnabled 线程安全读
func (a *App) PreviewEnabled() bool {
a.mu.Lock()
defer a.mu.Unlock()
return a.previewEnabled
}
// SetPreviewEnabled 切换开关，写库 + 若开启则立刻补扫 pending
func (a *App) SetPreviewEnabled(ctx context.Context, enabled bool) error {
a.mu.Lock()
a.previewEnabled = enabled
a.mu.Unlock()
val := "0"
if enabled {
val = "1"
}
if err := a.cat.SetSetting(ctx, "preview.enabled", val); err != nil {
return err
}
if enabled {
// 异步补扫所有盘
go func() {
for _, d := range a.registry.All() {
a.mu.Lock()
w := a.workers[d.ID()]
a.mu.Unlock()
if w != nil {
a.enqueuePending(ctx, d.ID(), w)
}
}
}()
}
return nil
}
// loadPreviewEnabled 从 DB 读运行时开关，首次启动取 config 默认值
func (a *App) loadPreviewEnabled(ctx context.Context) {
def := "0"
if a.cfg.Preview.Enabled {
def = "1"
}
v, err := a.cat.GetSetting(ctx, "preview.enabled", def)
if err != nil {
log.Printf("[preview] load setting: %v (fallback to config)", err)
a.mu.Lock()
a.previewEnabled = a.cfg.Preview.Enabled
a.mu.Unlock()
return
}
a.mu.Lock()
a.previewEnabled = v == "1"
a.mu.Unlock()
}
func (a *App) attachDrive(ctx context.Context, d *catalog.Drive) error {
var drv drives.Drive
switch d.Kind {
case "quark":
drv = quark.New(quark.Config{
ID: d.ID,
Cookie: d.Credentials["cookie"],
RootID: d.RootID,
OnCookieUpdate: func(cookie string) {
d.Credentials["cookie"] = cookie
_ = a.cat.UpsertDrive(ctx, d)
},
})
case "p115":
drv = p115.New(p115.Config{
ID: d.ID,
Cookie: d.Credentials["cookie"],
RootID: d.RootID,
})
case "wopan":
drv = wopan.New(wopan.Config{
ID: d.ID,
AccessToken: d.Credentials["access_token"],
RefreshToken: d.Credentials["refresh_token"],
FamilyID: d.Credentials["family_id"],
RootID: d.RootID,
OnTokenUpdate: func(access, refresh string) {
d.Credentials["access_token"] = access
d.Credentials["refresh_token"] = refresh
_ = a.cat.UpsertDrive(ctx, d)
},
})
default:
return fmt.Errorf("unknown drive kind: %s", d.Kind)
}
if err := drv.Init(ctx); err != nil {
d.Status = "error"
d.LastError = err.Error()
_ = a.cat.UpsertDrive(ctx, d)
return err
}
d.Status = "ok"
d.LastError = ""
_ = a.cat.UpsertDrive(ctx, d)
a.registry.Set(d.ID, drv)
// preview worker
gen := preview.New(preview.Config{
FFmpegPath: a.cfg.Preview.FFmpegPath,
FFprobePath: a.cfg.Preview.FFprobePath,
DurationSeconds: a.cfg.Preview.DurationSeconds,
Width: a.cfg.Preview.Width,
Segments: a.cfg.Preview.Segments,
LocalDir: a.cfg.Storage.LocalPreviewDir,
RemoteDir: a.cfg.Preview.RemoteDir,
})
worker := preview.NewWorker(gen, a.cat, drv, a.cfg.Preview.RemoteDir)
workerCtx, cancel := context.WithCancel(ctx)
go worker.Run(workerCtx)
a.mu.Lock()
if a.cancels == nil {
a.cancels = make(map[string]context.CancelFunc)
}
if old, ok := a.cancels[d.ID]; ok {
old()
}
a.workers[d.ID] = worker
a.cancels[d.ID] = cancel
a.mu.Unlock()
// 启动补扫：把这个盘下所有 pending 的视频塞进 worker 队列
// 使用 goroutine 因为队列可能比预期的小，Enqueue 直接丢弃，调用方也无需等待
if a.PreviewEnabled() {
go a.enqueuePending(workerCtx, d.ID, worker)
}
return nil
}
func (a *App) enqueuePending(ctx context.Context, driveID string, w *preview.Worker) {
pending, err := a.cat.ListVideosByPreviewStatus(ctx, driveID, "pending", 0)
if err != nil {
log.Printf("[preview] list pending %s: %v", driveID, err)
return
}
if len(pending) == 0 {
return
}
log.Printf("[preview] enqueue %d pending videos for drive=%s", len(pending), driveID)
for _, v := range pending {
w.Enqueue(v)
}
}
func (a *App) detachDrive(id string) {
a.registry.Remove(id)
a.mu.Lock()
if cancel, ok := a.cancels[id]; ok {
cancel()
delete(a.cancels, id)
}
delete(a.workers, id)
a.mu.Unlock()
}
func (a *App) runScan(ctx context.Context, driveID string) {
drv, ok := a.registry.Get(driveID)
if !ok {
log.Printf("[scan] drive %s not attached", driveID)
return
}
a.mu.Lock()
worker := a.workers[driveID]
a.mu.Unlock()
var onNew func(v *catalog.Video)
if a.PreviewEnabled() && worker != nil {
onNew = worker.Enqueue
}
sc := scanner.New(a.cat, drv, a.cfg.Scanner.VideoExtensions, a.cfg.Scanner.MaxDepth, onNew)
// 使用 drive 的 scan_root_id，否则 root_id
d, err := a.cat.GetDrive(ctx, driveID)
if err != nil {
log.Printf("[scan] get drive %s: %v", driveID, err)
return
}
startID := d.ScanRootID
if startID == "" {
startID = d.RootID
}
log.Printf("[scan] drive=%s start=%s", driveID, startID)
stats, err := sc.Run(ctx, startID)
if err != nil {
log.Printf("[scan] drive=%s error: %v", driveID, err)
return
}
log.Printf("[scan] drive=%s done scanned=%d added=%d", driveID, stats.Scanned, stats.Added)
}
func (a *App) regenPreview(ctx context.Context, videoID string) {
v, err := a.cat.GetVideo(ctx, videoID)
if err != nil {
return
}
a.mu.Lock()
worker := a.workers[v.DriveID]
a.mu.Unlock()
if worker != nil {
worker.Enqueue(v)
}
}
func (a *App) scanLoop(ctx context.Context) {
// 启动后立刻扫一次
a.scanAllOnce(ctx)
if a.cfg.Scanner.IntervalSeconds <= 0 {
return
}
ticker := time.NewTicker(time.Duration(a.cfg.Scanner.IntervalSeconds) * time.Second)
defer ticker.Stop()
for {
select {
case <-ctx.Done():
return
case <-ticker.C:
a.scanAllOnce(ctx)
}
}
}
func (a *App) scanAllOnce(ctx context.Context) {
for _, d := range a.registry.All() {
a.runScan(ctx, d.ID())
}
}
// ---------- middleware ----------
func corsMiddleware(next http.Handler) http.Handler {
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
w.Header().Set("Access-Control-Allow-Origin", originOr(r, "*"))
w.Header().Set("Access-Control-Allow-Credentials", "true")
w.Header().Set("Access-Control-Allow-Methods", "GET,POST,PUT,DELETE,OPTIONS")
w.Header().Set("Access-Control-Allow-Headers", "Content-Type, Authorization")
if r.Method == http.MethodOptions {
w.WriteHeader(http.StatusNoContent)
return
}
next.ServeHTTP(w, r)
})
}
func originOr(r *http.Request, fallback string) string {
if o := r.Header.Get("Origin"); o != "" {
return o
}
return fallback
}
backend/config.example.yaml
+41
View File
@@ -0,0 +1,41 @@
# backend 配置示例。首次启动若未发现 config.yaml，会基于此文件自动创建。
server:
listen: ":8080"
# 管理后台用户，生产环境请务必修改
admin:
username: "admin"
password: "admin123"
# 用于签发 admin session cookie，生产请改成随机字符串
session_secret: "change-me-to-a-random-string"
storage:
# SQLite 数据库文件路径
db_path: "./data/video-site.db"
# 本地 teaser 兜底目录（网盘写入失败时使用）
local_preview_dir: "./data/previews"
scanner:
# 扫描间隔（秒），0 表示只启动时扫一次
interval_seconds: 21600
# 单次扫描每家网盘目录递归层数上限
max_depth: 5
# 被扫描的扩展名
video_extensions: [".mp4", ".mkv", ".mov", ".webm", ".avi"]
preview:
# 是否启用 ffmpeg 抽帧生成 teaser
enabled: true
# ffmpeg / ffprobe 可执行文件名或绝对路径
ffmpeg_path: "ffmpeg"
ffprobe_path: "ffprobe"
# teaser 总时长（秒）。多段模式下会均分给每段
duration_seconds: 9
# teaser 段数。1=从视频 25% 位置取单段；>=2 按时长自适应切段并拼接
segments: 3
# teaser 视频宽度
width: 480
# teaser 上传到网盘的目录（相对网盘根）。空值则写本地
remote_dir: "/previews"
# 盘列表。上线后请通过管理后台添加，本文件可留空
drives: []
backend/go.mod
+46
View File
@@ -0,0 +1,46 @@
module github.com/video-site/backend
go 1.23.0
toolchain go1.23.4
require (
github.com/OpenListTeam/wopan-sdk-go v0.2.0
github.com/SheltonZhu/115driver v1.3.2
github.com/go-chi/chi/v5 v5.1.0
github.com/go-resty/resty/v2 v2.14.0
gopkg.in/yaml.v3 v3.0.1
modernc.org/sqlite v1.33.1
)
require (
github.com/aead/ecdh v0.2.0 // indirect
github.com/aliyun/aliyun-oss-go-sdk v3.0.2+incompatible // indirect
github.com/andreburgaud/crypt2go v1.1.0 // indirect
github.com/dustin/go-humanize v1.0.1 // indirect
github.com/google/uuid v1.6.0 // indirect
github.com/hashicorp/golang-lru/v2 v2.0.7 // indirect
github.com/kr/pretty v0.3.1 // indirect
github.com/mattn/go-isatty v0.0.20 // indirect
github.com/ncruces/go-strftime v0.1.9 // indirect
github.com/pierrec/lz4/v4 v4.1.17 // indirect
github.com/pkg/errors v0.9.1 // indirect
github.com/remyoudompheng/bigfft v0.0.0-20230129092748-24d4a6f8daec // indirect
github.com/skip2/go-qrcode v0.0.0-20200617195104-da1b6568686e // indirect
golang.org/x/crypto v0.25.0 // indirect
golang.org/x/net v0.27.0 // indirect
golang.org/x/sys v0.30.0 // indirect
golang.org/x/time v0.8.0 // indirect
modernc.org/gc/v3 v3.0.0-20240107210532-573471604cb6 // indirect
modernc.org/libc v1.55.3 // indirect
modernc.org/mathutil v1.6.0 // indirect
modernc.org/memory v1.8.0 // indirect
modernc.org/strutil v1.2.0 // indirect
modernc.org/token v1.1.0 // indirect
)
// 依赖已通过 go mod vendor 打包进 backend/vendor/ 并入库，支持离线构建。
// 升级 SDK 请使用标准流程：
// go get github.com/SheltonZhu/115driver@<版本>
// go mod tidy
// go mod vendor
backend/go.sum
+155
View File
@@ -0,0 +1,155 @@
github.com/OpenListTeam/wopan-sdk-go v0.2.0 h1:i7nxcdDTIWEbMDlmZ4bFAjX4Dd3mWmf8SEfcateicSk=
github.com/OpenListTeam/wopan-sdk-go v0.2.0/go.mod h1:otynv0CgSNUClPpUgZ44qCZGcMRe0dc83Pkk65xAunI=
github.com/SheltonZhu/115driver v1.3.2 h1:GmIYqivI1Y647FHGwiAoQce1tDaY8YWNjk7hGLhbsqs=
github.com/SheltonZhu/115driver v1.3.2/go.mod h1:OujS7azslg1/bn85sPSHnNsp4/WBI9/TiijtZL9kuSQ=
github.com/aead/ecdh v0.2.0 h1:pYop54xVaq/CEREFEcukHRZfTdjiWvYIsZDXXrBapQQ=
github.com/aead/ecdh v0.2.0/go.mod h1:a9HHtXuSo8J1Js1MwLQx2mBhkXMT6YwUmVVEY4tTB8U=
github.com/aliyun/aliyun-oss-go-sdk v3.0.2+incompatible h1:8psS8a+wKfiLt1iVDX79F7Y6wUM49Lcha2FMXt4UM8g=
github.com/aliyun/aliyun-oss-go-sdk v3.0.2+incompatible/go.mod h1:T/Aws4fEfogEE9v+HPhhw+CntffsBHJ8nXQCwKr0/g8=
github.com/andreburgaud/crypt2go v1.1.0 h1:eitZxTPY1krUsxinsng3Qvt/Ud7q/aQmmYRh8p4hyPw=
github.com/andreburgaud/crypt2go v1.1.0/go.mod h1:4qhZPzarj1dCIRmCkpdgCklwp+hBq9yEt0zPe9Ayuhc=
github.com/creack/pty v1.1.9/go.mod h1:oKZEueFk5CKHvIhNR5MUki03XCEU+Q6VDXinZuGJ33E=
github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c=
github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/dustin/go-humanize v1.0.1 h1:GzkhY7T5VNhEkwH0PVJgjz+fX1rhBrR7pRT3mDkpeCY=
github.com/dustin/go-humanize v1.0.1/go.mod h1:Mu1zIs6XwVuF/gI1OepvI0qD18qycQx+mFykh5fBlto=
github.com/go-chi/chi/v5 v5.1.0 h1:acVI1TYaD+hhedDJ3r54HyA6sExp3HfXq7QWEEY/xMw=
github.com/go-chi/chi/v5 v5.1.0/go.mod h1:DslCQbL2OYiznFReuXYUmQ2hGd1aDpCnlMNITLSKoi8=
github.com/go-resty/resty/v2 v2.14.0 h1:/rhkzsAqGQkozwfKS5aFAbb6TyKd3zyFRWcdRXLPCAU=
github.com/go-resty/resty/v2 v2.14.0/go.mod h1:IW6mekUOsElt9C7oWr0XRt9BNSD6D5rr9mhk6NjmNHg=
github.com/google/go-cmp v0.6.0/go.mod h1:17dUlkBOakJ0+DkrSSNjCkIjxS6bF9zb3elmeNGIjoY=
github.com/google/pprof v0.0.0-20240409012703-83162a5b38cd h1:gbpYu9NMq8jhDVbvlGkMFWCjLFlqqEZjEmObmhUy6Vo=
github.com/google/pprof v0.0.0-20240409012703-83162a5b38cd/go.mod h1:kf6iHlnVGwgKolg33glAes7Yg/8iWP8ukqeldJSO7jw=
github.com/google/uuid v1.6.0 h1:NIvaJDMOsjHA8n1jAhLSgzrAzy1Hgr+hNrb57e+94F0=
github.com/google/uuid v1.6.0/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo=
github.com/hashicorp/golang-lru/v2 v2.0.7 h1:a+bsQ5rvGLjzHuww6tVxozPZFVghXaHOwFs4luLUK2k=
github.com/hashicorp/golang-lru/v2 v2.0.7/go.mod h1:QeFd9opnmA6QUJc5vARoKUSoFhyfM2/ZepoAG6RGpeM=
github.com/kr/pretty v0.3.1 h1:flRD4NNwYAUpkphVc1HcthR4KEIFJ65n8Mw5qdRn3LE=
github.com/kr/pretty v0.3.1/go.mod h1:hoEshYVHaxMs3cyo3Yncou5ZscifuDolrwPKZanG3xk=
github.com/kr/text v0.2.0 h1:5Nx0Ya0ZqY2ygV366QzturHI13Jq95ApcVaJBhpS+AY=
github.com/kr/text v0.2.0/go.mod h1:eLer722TekiGuMkidMxC/pM04lWEeraHUUmBw8l2grE=
github.com/mattn/go-isatty v0.0.20 h1:xfD0iDuEKnDkl03q4limB+vH+GxLEtL/jb4xVJSWWEY=
github.com/mattn/go-isatty v0.0.20/go.mod h1:W+V8PltTTMOvKvAeJH7IuucS94S2C6jfK/D7dTCTo3Y=
github.com/ncruces/go-strftime v0.1.9 h1:bY0MQC28UADQmHmaF5dgpLmImcShSi2kHU9XLdhx/f4=
github.com/ncruces/go-strftime v0.1.9/go.mod h1:Fwc5htZGVVkseilnfgOVb9mKy6w1naJmn9CehxcKcls=
github.com/pierrec/lz4/v4 v4.1.17 h1:kV4Ip+/hUBC+8T6+2EgburRtkE9ef4nbY3f4dFhGjMc=
github.com/pierrec/lz4/v4 v4.1.17/go.mod h1:gZWDp/Ze/IJXGXf23ltt2EXimqmTUXEy0GFuRQyBid4=
github.com/pkg/diff v0.0.0-20210226163009-20ebb0f2a09e/go.mod h1:pJLUxLENpZxwdsKMEsNbx1VGcRFpLqf3715MtcvvzbA=
github.com/pkg/errors v0.9.1 h1:FEBLx1zS214owpjy7qsBeixbURkuhQAwrK5UwLGTwt4=
github.com/pkg/errors v0.9.1/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0=
github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM=
github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
github.com/remyoudompheng/bigfft v0.0.0-20230129092748-24d4a6f8daec h1:W09IVJc94icq4NjY3clb7Lk8O1qJ8BdBEF8z0ibU0rE=
github.com/remyoudompheng/bigfft v0.0.0-20230129092748-24d4a6f8daec/go.mod h1:qqbHyh8v60DhA7CoWK5oRCqLrMHRGoxYCSS9EjAz6Eo=
github.com/rogpeppe/go-internal v1.9.0 h1:73kH8U+JUqXU8lRuOHeVHaa/SZPifC7BkcraZVejAe8=
github.com/rogpeppe/go-internal v1.9.0/go.mod h1:WtVeX8xhTBvf0smdhujwtBcq4Qrzq/fJaraNFVN+nFs=
github.com/skip2/go-qrcode v0.0.0-20200617195104-da1b6568686e h1:MRM5ITcdelLK2j1vwZ3Je0FKVCfqOLp5zO6trqMLYs0=
github.com/skip2/go-qrcode v0.0.0-20200617195104-da1b6568686e/go.mod h1:XV66xRDqSt+GTGFMVlhk3ULuV0y9ZmzeVGR4mloJI3M=
github.com/stretchr/testify v1.11.1 h1:7s2iGBzp5EwR7/aIZr8ao5+dra3wiQyKjjFuvgVKu7U=
github.com/stretchr/testify v1.11.1/go.mod h1:wZwfW3scLgRK+23gO65QZefKpKQRnfz6sD981Nm4B6U=
github.com/yuin/goldmark v1.4.13/go.mod h1:6yULJ656Px+3vBD8DxQVa3kxgyrAnzto9xy5taEt/CY=
golang.org/x/crypto v0.0.0-20190211182817-74369b46fc67/go.mod h1:6SG95UA2DQfeDnfUPMdvaQW0Q7yPrPDi9nlGo2tz2b4=
golang.org/x/crypto v0.0.0-20190308221718-c2843e01d9a2/go.mod h1:djNgcEr1/C05ACkg1iLfiJU5Ep61QUkGW8qpdssI0+w=
golang.org/x/crypto v0.0.0-20210921155107-089bfa567519/go.mod h1:GvvjBRRGRdwPK5ydBHafDWAxML/pGHZbMvKqRZ5+Abc=
golang.org/x/crypto v0.13.0/go.mod h1:y6Z2r+Rw4iayiXXAIxJIDAJ1zMW4yaTpebo8fPOliYc=
golang.org/x/crypto v0.19.0/go.mod h1:Iy9bg/ha4yyC70EfRS8jz+B6ybOBKMaSxLj6P6oBDfU=
golang.org/x/crypto v0.23.0/go.mod h1:CKFgDieR+mRhux2Lsu27y0fO304Db0wZe70UKqHu0v8=
golang.org/x/crypto v0.25.0 h1:ypSNr+bnYL2YhwoMt2zPxHFmbAN1KZs/njMG3hxUp30=
golang.org/x/crypto v0.25.0/go.mod h1:T+wALwcMOSE0kXgUAnPAHqTLW+XHgcELELW8VaDgm/M=
golang.org/x/mod v0.6.0-dev.0.20220419223038-86c51ed26bb4/go.mod h1:jJ57K6gSWd91VN4djpZkiMVwK6gcyfeH4XE8wZrZaV4=
golang.org/x/mod v0.8.0/go.mod h1:iBbtSCu2XBx23ZKBPSOrRkjjQPZFPuis4dIYUhu/chs=
golang.org/x/mod v0.12.0/go.mod h1:iBbtSCu2XBx23ZKBPSOrRkjjQPZFPuis4dIYUhu/chs=
golang.org/x/mod v0.15.0/go.mod h1:hTbmBsO62+eylJbnUtE2MGJUyE7QWk4xUqPFrRgJ+7c=
golang.org/x/mod v0.17.0 h1:zY54UmvipHiNd+pm+m0x9KhZ9hl1/7QNMyxXbc6ICqA=
golang.org/x/mod v0.17.0/go.mod h1:hTbmBsO62+eylJbnUtE2MGJUyE7QWk4xUqPFrRgJ+7c=
golang.org/x/net v0.0.0-20190620200207-3b0461eec859/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
golang.org/x/net v0.0.0-20210226172049-e18ecbb05110/go.mod h1:m0MpNAwzfU5UDzcl9v0D8zg8gWTRqZa9RBIspLL5mdg=
golang.org/x/net v0.0.0-20220722155237-a158d28d115b/go.mod h1:XRhObCWvk6IyKnWLug+ECip1KBveYUHfp+8e9klMJ9c=
golang.org/x/net v0.6.0/go.mod h1:2Tu9+aMcznHK/AK1HMvgo6xiTLG5rD5rZLDS+rp2Bjs=
golang.org/x/net v0.10.0/go.mod h1:0qNGK6F8kojg2nk9dLZ2mShWaEBan6FAoqfSigmmuDg=
golang.org/x/net v0.15.0/go.mod h1:idbUs1IY1+zTqbi8yxTbhexhEEk5ur9LInksu6HrEpk=
golang.org/x/net v0.21.0/go.mod h1:bIjVDfnllIU7BJ2DNgfnXvpSvtn8VRwhlsaeUTyUS44=
golang.org/x/net v0.25.0/go.mod h1:JkAGAh7GEvH74S6FOH42FLoXpXbE/aqXSrIQjXgsiwM=
golang.org/x/net v0.27.0 h1:5K3Njcw06/l2y9vpGCSdcxWOYHOUk3dVNGDXN+FvAys=
golang.org/x/net v0.27.0/go.mod h1:dDi0PyhWNoiUOrAS8uXv/vnScO4wnHQO4mj9fn/RytE=
golang.org/x/sync v0.0.0-20190423024810-112230192c58/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.0.0-20220722155255-886fb9371eb4/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.1.0/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.3.0/go.mod h1:FU7BRWz2tNW+3quACPkgCx/L+uEAv1htQ0V83Z9Rj+Y=
golang.org/x/sync v0.6.0/go.mod h1:Czt+wKu1gCyEFDUtn0jG5QVvpJ6rzVqr5aXyt9drQfk=
golang.org/x/sync v0.7.0 h1:YsImfSBoP9QPYL0xyKJPq0gcaJdG3rInoqxTWbfQu9M=
golang.org/x/sync v0.7.0/go.mod h1:Czt+wKu1gCyEFDUtn0jG5QVvpJ6rzVqr5aXyt9drQfk=
golang.org/x/sys v0.0.0-20190215142949-d0b11bdaac8a/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
golang.org/x/sys v0.0.0-20201119102817-f84b799fce68/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20210615035016-665e8c7367d1/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.0.0-20220520151302-bc2c85ada10a/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.0.0-20220722155257-8c9f86f7a55f/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.5.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.6.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.8.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.12.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.17.0/go.mod h1:/VUhepiaJMQUp4+oa/7Zr1D23ma6VTLIYjOOTFZPUcA=
golang.org/x/sys v0.20.0/go.mod h1:/VUhepiaJMQUp4+oa/7Zr1D23ma6VTLIYjOOTFZPUcA=
golang.org/x/sys v0.22.0/go.mod h1:/VUhepiaJMQUp4+oa/7Zr1D23ma6VTLIYjOOTFZPUcA=
golang.org/x/sys v0.30.0 h1:QjkSwP/36a20jFYWkSue1YwXzLmsV5Gfq7Eiy72C1uc=
golang.org/x/sys v0.30.0/go.mod h1:/VUhepiaJMQUp4+oa/7Zr1D23ma6VTLIYjOOTFZPUcA=
golang.org/x/telemetry v0.0.0-20240228155512-f48c80bd79b2/go.mod h1:TeRTkGYfJXctD9OcfyVLyj2J3IxLnKwHJR8f4D8a3YE=
golang.org/x/term v0.0.0-20201126162022-7de9c90e9dd1/go.mod h1:bj7SfCRtBDWHUb9snDiAeCFNEtKQo2Wmx5Cou7ajbmo=
golang.org/x/term v0.0.0-20210927222741-03fcf44c2211/go.mod h1:jbD1KX2456YbFQfuXm/mYQcufACuNUgVhRMnK/tPxf8=
golang.org/x/term v0.5.0/go.mod h1:jMB1sMXY+tzblOD4FWmEbocvup2/aLOaQEp7JmGp78k=
golang.org/x/term v0.8.0/go.mod h1:xPskH00ivmX89bAKVGSKKtLOWNx2+17Eiy94tnKShWo=
golang.org/x/term v0.12.0/go.mod h1:owVbMEjm3cBLCHdkQu9b1opXd4ETQWc3BhuQGKgXgvU=
golang.org/x/term v0.17.0/go.mod h1:lLRBjIVuehSbZlaOtGMbcMncT+aqLLLmKrsjNrUguwk=
golang.org/x/term v0.20.0/go.mod h1:8UkIAJTvZgivsXaD6/pH6U9ecQzZ45awqEOzuCvwpFY=
golang.org/x/term v0.22.0/go.mod h1:F3qCibpT5AMpCRfhfT53vVJwhLtIVHhB9XDjfFvnMI4=
golang.org/x/text v0.3.0/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
golang.org/x/text v0.3.3/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ=
golang.org/x/text v0.3.7/go.mod h1:u+2+/6zg+i71rQMx5EYifcz6MCKuco9NR6JIITiCfzQ=
golang.org/x/text v0.7.0/go.mod h1:mrYo+phRRbMaCq/xk9113O4dZlRixOauAjOtrjsXDZ8=
golang.org/x/text v0.9.0/go.mod h1:e1OnstbJyHTd6l/uOt8jFFHp6TRDWZR/bV3emEE/zU8=
golang.org/x/text v0.13.0/go.mod h1:TvPlkZtksWOMsz7fbANvkp4WM8x/WCo/om8BMLbz+aE=
golang.org/x/text v0.14.0/go.mod h1:18ZOQIKpY8NJVqYksKHtTdi31H5itFRjB5/qKTNYzSU=
golang.org/x/text v0.15.0/go.mod h1:18ZOQIKpY8NJVqYksKHtTdi31H5itFRjB5/qKTNYzSU=
golang.org/x/text v0.16.0/go.mod h1:GhwF1Be+LQoKShO3cGOHzqOgRrGaYc9AvblQOmPVHnI=
golang.org/x/time v0.6.0/go.mod h1:3BpzKBy/shNhVucY/MWOyx10tF3SFh9QdLuxbVysPQM=
golang.org/x/time v0.8.0 h1:9i3RxcPv3PZnitoVGMPDKZSq1xW1gK1Xy3ArNOGZfEg=
golang.org/x/time v0.8.0/go.mod h1:3BpzKBy/shNhVucY/MWOyx10tF3SFh9QdLuxbVysPQM=
golang.org/x/tools v0.0.0-20180917221912-90fa682c2a6e/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ=
golang.org/x/tools v0.0.0-20191119224855-298f0cb1881e/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo=
golang.org/x/tools v0.1.12/go.mod h1:hNGJHUnrk76NpqgfD5Aqm5Crs+Hm0VOH/i9J2+nxYbc=
golang.org/x/tools v0.6.0/go.mod h1:Xwgl3UAJ/d3gWutnCtw505GrjyAbvKui8lOU390QaIU=
golang.org/x/tools v0.13.0/go.mod h1:HvlwmtVNQAhOuCjW7xxvovg8wbNq7LwfXh/k7wXUl58=
golang.org/x/tools v0.21.1-0.20240508182429-e35e4ccd0d2d h1:vU5i/LfpvrRCpgM/VPfJLg5KjxD3E+hfT1SH+d9zLwg=
golang.org/x/tools v0.21.1-0.20240508182429-e35e4ccd0d2d/go.mod h1:aiJjzUbINMkxbQROHiO6hDPo2LHcIPhhQsa9DLh0yGk=
golang.org/x/xerrors v0.0.0-20190717185122-a985d3407aa7/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c h1:Hei/4ADfdWqJk1ZMxUNpqntNwaWcugrBjAiHlqqRiVk=
gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c/go.mod h1:JHkPIbrfpd72SG/EVd6muEfDQjcINNoR0C8j2r3qZ4Q=
gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA=
gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
modernc.org/cc/v4 v4.21.4 h1:3Be/Rdo1fpr8GrQ7IVw9OHtplU4gWbb+wNgeoBMmGLQ=
modernc.org/cc/v4 v4.21.4/go.mod h1:HM7VJTZbUCR3rV8EYBi9wxnJ0ZBRiGE5OeGXNA0IsLQ=
modernc.org/ccgo/v4 v4.19.2 h1:lwQZgvboKD0jBwdaeVCTouxhxAyN6iawF3STraAal8Y=
modernc.org/ccgo/v4 v4.19.2/go.mod h1:ysS3mxiMV38XGRTTcgo0DQTeTmAO4oCmJl1nX9VFI3s=
modernc.org/fileutil v1.3.0 h1:gQ5SIzK3H9kdfai/5x41oQiKValumqNTDXMvKo62HvE=
modernc.org/fileutil v1.3.0/go.mod h1:XatxS8fZi3pS8/hKG2GH/ArUogfxjpEKs3Ku3aK4JyQ=
modernc.org/gc/v2 v2.4.1 h1:9cNzOqPyMJBvrUipmynX0ZohMhcxPtMccYgGOJdOiBw=
modernc.org/gc/v2 v2.4.1/go.mod h1:wzN5dK1AzVGoH6XOzc3YZ+ey/jPgYHLuVckd62P0GYU=
modernc.org/gc/v3 v3.0.0-20240107210532-573471604cb6 h1:5D53IMaUuA5InSeMu9eJtlQXS2NxAhyWQvkKEgXZhHI=
modernc.org/gc/v3 v3.0.0-20240107210532-573471604cb6/go.mod h1:Qz0X07sNOR1jWYCrJMEnbW/X55x206Q7Vt4mz6/wHp4=
modernc.org/libc v1.55.3 h1:AzcW1mhlPNrRtjS5sS+eW2ISCgSOLLNyFzRh/V3Qj/U=
modernc.org/libc v1.55.3/go.mod h1:qFXepLhz+JjFThQ4kzwzOjA/y/artDeg+pcYnY+Q83w=
modernc.org/mathutil v1.6.0 h1:fRe9+AmYlaej+64JsEEhoWuAYBkOtQiMEU7n/XgfYi4=
modernc.org/mathutil v1.6.0/go.mod h1:Ui5Q9q1TR2gFm0AQRqQUaBWFLAhQpCwNcuhBOSedWPo=
modernc.org/memory v1.8.0 h1:IqGTL6eFMaDZZhEWwcREgeMXYwmW83LYW8cROZYkg+E=
modernc.org/memory v1.8.0/go.mod h1:XPZ936zp5OMKGWPqbD3JShgd/ZoQ7899TUuQqxY+peU=
modernc.org/opt v0.1.3 h1:3XOZf2yznlhC+ibLltsDGzABUGVx8J6pnFMS3E4dcq4=
modernc.org/opt v0.1.3/go.mod h1:WdSiB5evDcignE70guQKxYUl14mgWtbClRi5wmkkTX0=
modernc.org/sortutil v1.2.0 h1:jQiD3PfS2REGJNzNCMMaLSp/wdMNieTbKX920Cqdgqc=
modernc.org/sortutil v1.2.0/go.mod h1:TKU2s7kJMf1AE84OoiGppNHJwvB753OYfNl2WRb++Ss=
modernc.org/sqlite v1.33.1 h1:trb6Z3YYoeM9eDL1O8do81kP+0ejv+YzgyFo+Gwy0nM=
modernc.org/sqlite v1.33.1/go.mod h1:pXV2xHxhzXZsgT/RtTFAPY6JJDEvOTcTdwADQCCWD4k=
modernc.org/strutil v1.2.0 h1:agBi9dp1I+eOnxXeiZawM8F4LawKv4NzGWSaLfyeNZA=
modernc.org/strutil v1.2.0/go.mod h1:/mdcBmfOibveCTBxUl5B5l6W+TTH1FXPLHZE6bTosX0=
modernc.org/token v1.1.0 h1:Xl7Ap9dKaEs5kLoOQeQmPWevfnk/DM5qcLcYlA8ys6Y=
modernc.org/token v1.1.0/go.mod h1:UGzOrNV1mAFSEB63lOFHIpNRUVMvYTc6yu1SMY/XTDM=
backend/internal/api/admin.go
+284
View File
@@ -0,0 +1,284 @@
package api
import (
"encoding/json"
"net/http"
"github.com/go-chi/chi/v5"
"github.com/video-site/backend/internal/auth"
"github.com/video-site/backend/internal/catalog"
)
type AdminServer struct {
Catalog *catalog.Catalog
Auth *auth.Authenticator
// Hooks：外层注入实际执行者
OnDriveSaved func(driveID string) error
OnDriveRemoved func(driveID string)
OnScanRequested func(driveID string)
OnRegenPreview func(videoID string)
// Preview 开关读写
GetPreviewEnabled func() bool
SetPreviewEnabled func(enabled bool) error
}
func (a *AdminServer) Register(r chi.Router) {
r.Route("/admin/api", func(r chi.Router) {
// 登录、登出不需要鉴权
r.Post("/login", a.handleLogin)
r.Post("/logout", a.handleLogout)
r.Get("/me", a.handleMe)
// 其余路由需鉴权
r.Group(func(r chi.Router) {
r.Use(a.Auth.Required)
// 网盘
r.Get("/drives", a.handleListDrives)
r.Post("/drives", a.handleUpsertDrive)
r.Delete("/drives/{id}", a.handleDeleteDrive)
r.Post("/drives/{id}/rescan", a.handleRescan)
// 视频
r.Get("/videos", a.handleAdminListVideos)
r.Put("/videos/{id}", a.handleUpdateVideo)
r.Post("/videos/{id}/regen-preview", a.handleRegenPreview)
// 运行时设置
r.Get("/settings", a.handleGetSettings)
r.Put("/settings", a.handlePutSettings)
})
})
}
type loginReq struct {
Username string `json:"username"`
Password string `json:"password"`
}
func (a *AdminServer) handleLogin(w http.ResponseWriter, r *http.Request) {
var body loginReq
if err := json.NewDecoder(r.Body).Decode(&body); err != nil {
writeErr(w, http.StatusBadRequest, err)
return
}
ok, err := a.Auth.Login(w, r, body.Username, body.Password)
if err != nil {
writeErr(w, http.StatusInternalServerError, err)
return
}
if !ok {
http.Error(w, "invalid credentials", http.StatusUnauthorized)
return
}
writeJSON(w, http.StatusOK, map[string]any{"ok": true})
}
func (a *AdminServer) handleLogout(w http.ResponseWriter, r *http.Request) {
a.Auth.Logout(w, r)
writeJSON(w, http.StatusOK, map[string]any{"ok": true})
}
func (a *AdminServer) handleMe(w http.ResponseWriter, r *http.Request) {
c, err := r.Cookie("vs_admin")
if err != nil {
writeJSON(w, http.StatusOK, map[string]any{"authenticated": false})
return
}
ok, _ := a.Catalog.ValidateSession(r.Context(), c.Value)
writeJSON(w, http.StatusOK, map[string]any{"authenticated": ok})
}
func (a *AdminServer) handleListDrives(w http.ResponseWriter, r *http.Request) {
drives, err := a.Catalog.ListDrives(r.Context())
if err != nil {
writeErr(w, http.StatusInternalServerError, err)
return
}
// 出参不返回凭证明文，只告诉前端是否已配置
type out struct {
ID string `json:"id"`
Kind string `json:"kind"`
Name string `json:"name"`
RootID string `json:"rootId"`
ScanRootID string `json:"scanRootId"`
Status string `json:"status"`
LastError string `json:"lastError,omitempty"`
HasCredential bool `json:"hasCredential"`
}
list := make([]out, 0, len(drives))
for _, d := range drives {
list = append(list, out{
ID: d.ID, Kind: d.Kind, Name: d.Name,
RootID: d.RootID, ScanRootID: d.ScanRootID,
Status: d.Status, LastError: d.LastError,
HasCredential: len(d.Credentials) > 0,
})
}
writeJSON(w, http.StatusOK, list)
}
type upsertDriveReq struct {
ID string `json:"id"`
Kind string `json:"kind"`
Name string `json:"name"`
RootID string `json:"rootId"`
ScanRootID string `json:"scanRootId"`
Credentials map[string]string `json:"credentials"`
}
func (a *AdminServer) handleUpsertDrive(w http.ResponseWriter, r *http.Request) {
var body upsertDriveReq
if err := json.NewDecoder(r.Body).Decode(&body); err != nil {
writeErr(w, http.StatusBadRequest, err)
return
}
if body.ID == "" || body.Kind == "" {
http.Error(w, "id and kind are required", http.StatusBadRequest)
return
}
d := &catalog.Drive{
ID: body.ID, Kind: body.Kind, Name: body.Name,
RootID: body.RootID, ScanRootID: body.ScanRootID,
Credentials: body.Credentials,
Status: "disconnected",
}
if err := a.Catalog.UpsertDrive(r.Context(), d); err != nil {
writeErr(w, http.StatusInternalServerError, err)
return
}
if a.OnDriveSaved != nil {
if err := a.OnDriveSaved(body.ID); err != nil {
writeJSON(w, http.StatusOK, map[string]any{"ok": true, "warning": err.Error()})
return
}
}
writeJSON(w, http.StatusOK, map[string]any{"ok": true})
}
func (a *AdminServer) handleDeleteDrive(w http.ResponseWriter, r *http.Request) {
id := chi.URLParam(r, "id")
if err := a.Catalog.DeleteDrive(r.Context(), id); err != nil {
writeErr(w, http.StatusInternalServerError, err)
return
}
if a.OnDriveRemoved != nil {
a.OnDriveRemoved(id)
}
writeJSON(w, http.StatusOK, map[string]any{"ok": true})
}
func (a *AdminServer) handleRescan(w http.ResponseWriter, r *http.Request) {
id := chi.URLParam(r, "id")
if a.OnScanRequested != nil {
a.OnScanRequested(id)
}
writeJSON(w, http.StatusAccepted, map[string]any{"ok": true})
}
func (a *AdminServer) handleAdminListVideos(w http.ResponseWriter, r *http.Request) {
items, total, err := a.Catalog.ListVideos(r.Context(), catalog.ListParams{
Page: 1, PageSize: 100,
})
if err != nil {
writeErr(w, http.StatusInternalServerError, err)
return
}
writeJSON(w, http.StatusOK, map[string]any{"items": items, "total": total})
}
type updateVideoReq struct {
Title string `json:"title"`
Author string `json:"author"`
Tags []string `json:"tags"`
Category string `json:"category"`
Badges []string `json:"badges"`
Description string `json:"description"`
Thumbnail string `json:"thumbnail"`
Quality string `json:"quality"`
DurationSec int `json:"durationSeconds"`
}
func (a *AdminServer) handleUpdateVideo(w http.ResponseWriter, r *http.Request) {
id := chi.URLParam(r, "id")
var body updateVideoReq
if err := json.NewDecoder(r.Body).Decode(&body); err != nil {
writeErr(w, http.StatusBadRequest, err)
return
}
v, err := a.Catalog.GetVideo(r.Context(), id)
if err != nil {
writeErr(w, http.StatusNotFound, err)
return
}
if body.Title != "" {
v.Title = body.Title
}
if body.Author != "" {
v.Author = body.Author
}
if body.Tags != nil {
v.Tags = body.Tags
}
if body.Category != "" {
v.Category = body.Category
}
if body.Badges != nil {
v.Badges = body.Badges
}
if body.Description != "" {
v.Description = body.Description
}
if body.Thumbnail != "" {
v.ThumbnailURL = body.Thumbnail
}
if body.Quality != "" {
v.Quality = body.Quality
}
if body.DurationSec > 0 {
v.DurationSeconds = body.DurationSec
}
if err := a.Catalog.UpsertVideo(r.Context(), v); err != nil {
writeErr(w, http.StatusInternalServerError, err)
return
}
writeJSON(w, http.StatusOK, v)
}
func (a *AdminServer) handleRegenPreview(w http.ResponseWriter, r *http.Request) {
id := chi.URLParam(r, "id")
if a.OnRegenPreview != nil {
a.OnRegenPreview(id)
}
writeJSON(w, http.StatusAccepted, map[string]any{"ok": true})
}
// ---------- Settings ----------
type settingsDTO struct {
PreviewEnabled bool `json:"previewEnabled"`
}
func (a *AdminServer) handleGetSettings(w http.ResponseWriter, r *http.Request) {
enabled := false
if a.GetPreviewEnabled != nil {
enabled = a.GetPreviewEnabled()
}
writeJSON(w, http.StatusOK, settingsDTO{PreviewEnabled: enabled})
}
func (a *AdminServer) handlePutSettings(w http.ResponseWriter, r *http.Request) {
var body settingsDTO
if err := json.NewDecoder(r.Body).Decode(&body); err != nil {
writeErr(w, http.StatusBadRequest, err)
return
}
if a.SetPreviewEnabled != nil {
if err := a.SetPreviewEnabled(body.PreviewEnabled); err != nil {
writeErr(w, http.StatusInternalServerError, err)
return
}
}
writeJSON(w, http.StatusOK, settingsDTO{PreviewEnabled: body.PreviewEnabled})
}
backend/internal/api/api.go
+305
View File
@@ -0,0 +1,305 @@
package api
import (
"encoding/json"
"fmt"
"net/http"
"os"
"path/filepath"
"strconv"
"strings"
"github.com/go-chi/chi/v5"
"github.com/video-site/backend/internal/auth"
"github.com/video-site/backend/internal/catalog"
"github.com/video-site/backend/internal/proxy"
)
type Server struct {
Catalog *catalog.Catalog
Proxy *proxy.Proxy
LocalDir string
}
// VideoDTO 是返回给前端的视频对象，字段名跟前端 VideoItem 对齐
type VideoDTO struct {
ID string `json:"id"`
Href string `json:"href"`
Title string `json:"title"`
Thumbnail string `json:"thumbnail"`
PreviewSrc string `json:"previewSrc"`
PreviewDuration int `json:"previewDuration"`
PreviewStrategy string `json:"previewStrategy"`
Duration string `json:"duration"`
Badges []string `json:"badges"`
Quality string `json:"quality,omitempty"`
Author string `json:"author"`
Views int `json:"views"`
Favorites int `json:"favorites"`
Comments int `json:"comments"`
Likes int `json:"likes"`
Dislikes int `json:"dislikes"`
PublishedAt string `json:"publishedAt"`
Tags []string `json:"tags,omitempty"`
Category string `json:"category,omitempty"`
}
type VideoDetailDTO struct {
VideoDTO
VideoSrc string `json:"videoSrc"`
Poster string `json:"poster"`
Description string `json:"description"`
EmbedURL string `json:"embedUrl"`
Points int `json:"points,omitempty"`
AuthorProfile AuthorProfile `json:"authorProfile"`
RelatedVideos []VideoDTO `json:"relatedVideos"`
CommentsList []Comment `json:"commentsList"`
}
type AuthorProfile struct {
ID string `json:"id"`
Name string `json:"name"`
Href string `json:"href"`
Badges []string `json:"badges"`
}
type Comment struct {
ID string `json:"id"`
Author string `json:"author"`
Body string `json:"body"`
CreatedAt string `json:"createdAt"`
Likes int `json:"likes,omitempty"`
}
// RegisterRoutes 挂载前台 REST 路由。前台接口需要登录态。
func (s *Server) RegisterRoutes(r chi.Router, a *auth.Authenticator) {
r.Group(func(r chi.Router) {
r.Use(a.Required)
r.Get("/api/home", s.handleHome)
r.Get("/api/list", s.handleList)
r.Get("/api/video/{id}", s.handleVideoDetail)
r.Post("/api/video/{id}/like", s.handleLike)
r.Get("/api/tags", s.handleTags)
// 代理路由同样需要鉴权，防止绕过
r.Get("/p/stream/{driveID}/{fileID}", s.handleStream)
r.Get("/p/preview/{videoID}", s.handlePreview)
r.Get("/p/thumb/{videoID}", s.handleThumb)
})
}
func (s *Server) handleHome(w http.ResponseWriter, r *http.Request) {
items, _, err := s.Catalog.ListVideos(r.Context(), catalog.ListParams{
Sort: "hot", Page: 1, PageSize: 24,
})
if err != nil {
writeErr(w, http.StatusInternalServerError, err)
return
}
writeJSON(w, http.StatusOK, mapVideos(items))
}
func (s *Server) handleList(w http.ResponseWriter, r *http.Request) {
q := r.URL.Query()
page, _ := strconv.Atoi(q.Get("page"))
size, _ := strconv.Atoi(q.Get("size"))
if size <= 0 {
size = 24
}
params := catalog.ListParams{
Keyword: q.Get("q"),
Tag: q.Get("tag"),
Category: q.Get("cat"),
Sort: q.Get("sort"),
Page: page,
PageSize: size,
}
items, total, err := s.Catalog.ListVideos(r.Context(), params)
if err != nil {
writeErr(w, http.StatusInternalServerError, err)
return
}
writeJSON(w, http.StatusOK, map[string]any{
"items": mapVideos(items),
"total": total,
"page": params.Page,
"size": params.PageSize,
})
}
func (s *Server) handleVideoDetail(w http.ResponseWriter, r *http.Request) {
id := chi.URLParam(r, "id")
v, err := s.Catalog.GetVideo(r.Context(), id)
if err != nil {
writeErr(w, http.StatusNotFound, err)
return
}
related, _, _ := s.Catalog.ListVideos(r.Context(), catalog.ListParams{
Sort: "hot", Page: 1, PageSize: 8,
})
detail := VideoDetailDTO{
VideoDTO: mapVideo(v),
VideoSrc: fmt.Sprintf("/p/stream/%s/%s", v.DriveID, v.FileID),
Poster: v.ThumbnailURL,
Description: v.Description,
EmbedURL: fmt.Sprintf(`<iframe src="/embed/%s" width="640" height="360" frameborder="0" allowfullscreen></iframe>`, v.ID),
AuthorProfile: AuthorProfile{
ID: "author-" + v.Author,
Name: v.Author,
Href: "/author/" + v.Author,
Badges: []string{},
},
RelatedVideos: filterVideos(mapVideos(related), v.ID),
CommentsList: []Comment{},
}
writeJSON(w, http.StatusOK, detail)
}
func (s *Server) handleTags(w http.ResponseWriter, r *http.Request) {
cats, err := s.Catalog.ListCategories(r.Context())
if err != nil {
writeErr(w, http.StatusInternalServerError, err)
return
}
type tag struct {
ID string `json:"id"`
Label string `json:"label"`
Count int `json:"count"`
}
out := make([]tag, 0, len(cats))
for _, c := range cats {
out = append(out, tag{ID: c.Category, Label: c.Category, Count: c.Count})
}
writeJSON(w, http.StatusOK, out)
}
func (s *Server) handleLike(w http.ResponseWriter, r *http.Request) {
id := chi.URLParam(r, "id")
likes, err := s.Catalog.IncrementLike(r.Context(), id)
if err != nil {
writeErr(w, http.StatusInternalServerError, err)
return
}
writeJSON(w, http.StatusOK, map[string]any{"likes": likes})
}
func (s *Server) handleStream(w http.ResponseWriter, r *http.Request) {
driveID := chi.URLParam(r, "driveID")
fileID := chi.URLParam(r, "fileID")
s.Proxy.ServeStream(w, r, driveID, fileID)
}
func (s *Server) handlePreview(w http.ResponseWriter, r *http.Request) {
videoID := chi.URLParam(r, "videoID")
v, err := s.Catalog.GetVideo(r.Context(), videoID)
if err != nil {
http.NotFound(w, r)
return
}
if v.PreviewStatus != "ready" {
http.Error(w, "preview not ready", http.StatusNotFound)
return
}
if v.PreviewFileID != "" {
s.Proxy.ServeStream(w, r, v.DriveID, v.PreviewFileID)
return
}
if v.PreviewLocal != "" {
if !strings.HasPrefix(filepath.Clean(v.PreviewLocal), filepath.Clean(s.LocalDir)) {
http.Error(w, "invalid local path", http.StatusForbidden)
return
}
s.Proxy.ServeLocal(w, r, v.PreviewLocal)
return
}
http.NotFound(w, r)
}
func (s *Server) handleThumb(w http.ResponseWriter, r *http.Request) {
videoID := chi.URLParam(r, "videoID")
// 直接读本地 thumbs 目录中 <videoID>.jpg
path := filepath.Join(s.LocalDir, "thumbs", videoID+".jpg")
clean := filepath.Clean(path)
if !strings.HasPrefix(clean, filepath.Clean(s.LocalDir)) {
http.Error(w, "invalid path", http.StatusForbidden)
return
}
if _, err := os.Stat(clean); err != nil {
http.NotFound(w, r)
return
}
s.Proxy.ServeLocal(w, r, clean)
}
// ---------- helpers ----------
func mapVideo(v *catalog.Video) VideoDTO {
badges := v.Badges
if badges == nil {
badges = []string{}
}
tags := v.Tags
if tags == nil {
tags = []string{}
}
return VideoDTO{
ID: v.ID,
Href: "/video/" + v.ID,
Title: v.Title,
Thumbnail: v.ThumbnailURL,
PreviewSrc: "/p/preview/" + v.ID,
PreviewDuration: 10,
PreviewStrategy: "teaser-file",
Duration: formatDuration(v.DurationSeconds),
Badges: badges,
Quality: v.Quality,
Author: v.Author,
Views: v.Views,
Favorites: v.Favorites,
Comments: v.Comments,
Likes: v.Likes,
Dislikes: v.Dislikes,
PublishedAt: v.PublishedAt.Format("2006-01-02"),
Tags: tags,
Category: v.Category,
}
}
func mapVideos(vs []*catalog.Video) []VideoDTO {
out := make([]VideoDTO, 0, len(vs))
for _, v := range vs {
out = append(out, mapVideo(v))
}
return out
}
func filterVideos(vs []VideoDTO, exclude string) []VideoDTO {
out := make([]VideoDTO, 0, len(vs))
for _, v := range vs {
if v.ID != exclude {
out = append(out, v)
}
}
return out
}
func formatDuration(sec int) string {
if sec <= 0 {
return "00:00"
}
m := sec / 60
s := sec % 60
return fmt.Sprintf("%02d:%02d", m, s)
}
func writeJSON(w http.ResponseWriter, code int, body any) {
w.Header().Set("Content-Type", "application/json; charset=utf-8")
w.WriteHeader(code)
_ = json.NewEncoder(w).Encode(body)
}
func writeErr(w http.ResponseWriter, code int, err error) {
writeJSON(w, code, map[string]string{"error": err.Error()})
}
backend/internal/auth/auth.go
+81
View File
@@ -0,0 +1,81 @@
package auth
import (
"crypto/rand"
"crypto/subtle"
"encoding/hex"
"net/http"
"time"
"github.com/video-site/backend/internal/catalog"
)
const (
sessionCookie = "vs_admin"
sessionTTL = 24 * time.Hour
)
type Authenticator struct {
Username string
Password string
Catalog *catalog.Catalog
}
func (a *Authenticator) Login(w http.ResponseWriter, r *http.Request, user, pass string) (bool, error) {
if subtle.ConstantTimeCompare([]byte(user), []byte(a.Username)) != 1 ||
subtle.ConstantTimeCompare([]byte(pass), []byte(a.Password)) != 1 {
return false, nil
}
token, err := randomToken()
if err != nil {
return false, err
}
if err := a.Catalog.CreateSession(r.Context(), token, sessionTTL); err != nil {
return false, err
}
http.SetCookie(w, &http.Cookie{
Name: sessionCookie,
Value: token,
Path: "/",
HttpOnly: true,
SameSite: http.SameSiteLaxMode,
Expires: time.Now().Add(sessionTTL),
})
return true, nil
}
func (a *Authenticator) Logout(w http.ResponseWriter, r *http.Request) {
if c, err := r.Cookie(sessionCookie); err == nil {
_ = a.Catalog.DeleteSession(r.Context(), c.Value)
}
http.SetCookie(w, &http.Cookie{
Name: sessionCookie,
Value: "",
Path: "/",
Expires: time.Unix(0, 0),
})
}
func (a *Authenticator) Required(next http.Handler) http.Handler {
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
c, err := r.Cookie(sessionCookie)
if err != nil {
http.Error(w, "unauthorized", http.StatusUnauthorized)
return
}
ok, err := a.Catalog.ValidateSession(r.Context(), c.Value)
if err != nil || !ok {
http.Error(w, "unauthorized", http.StatusUnauthorized)
return
}
next.ServeHTTP(w, r)
})
}
func randomToken() (string, error) {
b := make([]byte, 32)
if _, err := rand.Read(b); err != nil {
return "", err
}
return hex.EncodeToString(b), nil
}
backend/internal/catalog/catalog.go
+479
View File
@@ -0,0 +1,479 @@
package catalog
import (
"context"
"database/sql"
_ "embed"
"encoding/json"
"fmt"
"strings"
"time"
_ "modernc.org/sqlite"
)
//go:embed schema.sql
var schemaSQL string
type Catalog struct {
db *sql.DB
}
func Open(path string) (*Catalog, error) {
db, err := sql.Open("sqlite", path+"?_pragma=journal_mode(WAL)&_pragma=busy_timeout(5000)")
if err != nil {
return nil, err
}
if _, err := db.Exec(schemaSQL); err != nil {
db.Close()
return nil, fmt.Errorf("apply schema: %w", err)
}
return &Catalog{db: db}, nil
}
func (c *Catalog) Close() error { return c.db.Close() }
// ---------- Video ----------
type Video struct {
ID string `json:"id"`
DriveID string `json:"driveId"`
FileID string `json:"fileId"`
ParentID string `json:"parentId"`
Title string `json:"title"`
Author string `json:"author"`
Tags []string `json:"tags"`
DurationSeconds int `json:"durationSeconds"`
Size int64 `json:"size"`
Ext string `json:"ext"`
Quality string `json:"quality"`
ThumbnailURL string `json:"thumbnailUrl"`
PreviewFileID string `json:"previewFileId"`
PreviewLocal string `json:"previewLocal"`
PreviewStatus string `json:"previewStatus"`
Views int `json:"views"`
Favorites int `json:"favorites"`
Comments int `json:"comments"`
Likes int `json:"likes"`
Dislikes int `json:"dislikes"`
Category string `json:"category"`
Badges []string `json:"badges"`
Description string `json:"description"`
PublishedAt time.Time `json:"publishedAt"`
CreatedAt time.Time `json:"createdAt"`
UpdatedAt time.Time `json:"updatedAt"`
}
func (c *Catalog) UpsertVideo(ctx context.Context, v *Video) error {
tagsJSON, _ := json.Marshal(v.Tags)
badgesJSON, _ := json.Marshal(v.Badges)
now := time.Now().UnixMilli()
if v.CreatedAt.IsZero() {
v.CreatedAt = time.UnixMilli(now)
}
v.UpdatedAt = time.UnixMilli(now)
_, err := c.db.ExecContext(ctx, `
INSERT INTO videos (
id, drive_id, file_id, parent_id, title, author, tags,
duration_seconds, size_bytes, ext, quality, thumbnail_url,
preview_file_id, preview_local, preview_status,
views, favorites, comments, likes, dislikes,
category, badges, description, published_at, created_at, updated_at
) VALUES (
?, ?, ?, ?, ?, ?, ?,
?, ?, ?, ?, ?,
?, ?, ?,
?, ?, ?, ?, ?,
?, ?, ?, ?, ?, ?
)
ON CONFLICT(id) DO UPDATE SET
title = excluded.title,
author = excluded.author,
tags = excluded.tags,
duration_seconds= excluded.duration_seconds,
size_bytes = excluded.size_bytes,
ext = excluded.ext,
quality = excluded.quality,
thumbnail_url = excluded.thumbnail_url,
category = excluded.category,
badges = excluded.badges,
description = excluded.description,
updated_at = excluded.updated_at
`,
v.ID, v.DriveID, v.FileID, v.ParentID, v.Title, v.Author, string(tagsJSON),
v.DurationSeconds, v.Size, v.Ext, v.Quality, v.ThumbnailURL,
v.PreviewFileID, v.PreviewLocal, nullableStatus(v.PreviewStatus),
v.Views, v.Favorites, v.Comments, v.Likes, v.Dislikes,
v.Category, string(badgesJSON), v.Description,
v.PublishedAt.UnixMilli(), v.CreatedAt.UnixMilli(), v.UpdatedAt.UnixMilli(),
)
return err
}
func nullableStatus(s string) string {
if s == "" {
return "pending"
}
return s
}
func (c *Catalog) UpdatePreview(ctx context.Context, id, previewFileID, previewLocal, status string) error {
_, err := c.db.ExecContext(ctx,
`UPDATE videos SET preview_file_id = ?, preview_local = ?, preview_status = ?, updated_at = ? WHERE id = ?`,
previewFileID, previewLocal, status, time.Now().UnixMilli(), id)
return err
}
// IncrementLike 原子 +1，返回最新点赞数
func (c *Catalog) IncrementLike(ctx context.Context, id string) (int, error) {
tx, err := c.db.BeginTx(ctx, nil)
if err != nil {
return 0, err
}
defer tx.Rollback()
if _, err := tx.ExecContext(ctx,
`UPDATE videos SET likes = likes + 1, updated_at = ? WHERE id = ?`,
time.Now().UnixMilli(), id); err != nil {
return 0, err
}
var likes int
if err := tx.QueryRowContext(ctx, `SELECT likes FROM videos WHERE id = ?`, id).Scan(&likes); err != nil {
return 0, err
}
if err := tx.Commit(); err != nil {
return 0, err
}
return likes, nil
}
// VideoMetaPatch 轻量更新视频元数据（仅非零值字段会被写入）
type VideoMetaPatch struct {
ThumbnailURL string
DurationSeconds int
Category string
}
func (c *Catalog) UpdateVideoMeta(ctx context.Context, id string, p VideoMetaPatch) error {
parts := []string{}
args := []any{}
if p.ThumbnailURL != "" {
parts = append(parts, "thumbnail_url = ?")
args = append(args, p.ThumbnailURL)
}
if p.DurationSeconds > 0 {
parts = append(parts, "duration_seconds = ?")
args = append(args, p.DurationSeconds)
}
if p.Category != "" {
parts = append(parts, "category = ?")
args = append(args, p.Category)
}
if len(parts) == 0 {
return nil
}
parts = append(parts, "updated_at = ?")
args = append(args, time.Now().UnixMilli())
args = append(args, id)
q := `UPDATE videos SET ` + strings.Join(parts, ", ") + ` WHERE id = ?`
_, err := c.db.ExecContext(ctx, q, args...)
return err
}
// ListCategories 聚合所有 category，按视频数降序
type CategoryStat struct {
Category string
Count int
}
func (c *Catalog) ListCategories(ctx context.Context) ([]CategoryStat, error) {
rows, err := c.db.QueryContext(ctx,
`SELECT COALESCE(category, '') AS c, COUNT(*) AS cnt
FROM videos
WHERE category IS NOT NULL AND category != ''
GROUP BY c
ORDER BY cnt DESC, c ASC`)
if err != nil {
return nil, err
}
defer rows.Close()
var out []CategoryStat
for rows.Next() {
var s CategoryStat
if err := rows.Scan(&s.Category, &s.Count); err != nil {
return nil, err
}
out = append(out, s)
}
return out, nil
}
// ListVideosByPreviewStatus 按预览状态列出全部视频，通常用于启动补扫
func (c *Catalog) ListVideosByPreviewStatus(ctx context.Context, driveID, status string, limit int) ([]*Video, error) {
if limit <= 0 {
limit = 10000
}
rows, err := c.db.QueryContext(ctx,
`SELECT `+allVideoCols+` FROM videos WHERE drive_id = ? AND preview_status = ? ORDER BY created_at ASC LIMIT ?`,
driveID, status, limit)
if err != nil {
return nil, err
}
defer rows.Close()
var out []*Video
for rows.Next() {
v, err := scanVideo(rows)
if err != nil {
return nil, err
}
out = append(out, v)
}
return out, nil
}
func (c *Catalog) GetVideo(ctx context.Context, id string) (*Video, error) {
row := c.db.QueryRowContext(ctx, `SELECT `+allVideoCols+` FROM videos WHERE id = ?`, id)
return scanVideo(row)
}
type ListParams struct {
Keyword string
Tag string
Category string
Sort string // latest | hot | week | long
Page int
PageSize int
}
func (c *Catalog) ListVideos(ctx context.Context, p ListParams) ([]*Video, int, error) { if p.PageSize <= 0 {
p.PageSize = 24
}
if p.Page <= 0 {
p.Page = 1
}
var where []string
var args []any
if p.Keyword != "" {
where = append(where, "(title LIKE ? OR author LIKE ?)")
like := "%" + p.Keyword + "%"
args = append(args, like, like)
}
if p.Tag != "" {
where = append(where, "tags LIKE ?")
args = append(args, "%\""+p.Tag+"\"%")
}
if p.Category != "" && p.Category != "all" {
where = append(where, "category = ?")
args = append(args, p.Category)
}
whereSQL := ""
if len(where) > 0 {
whereSQL = " WHERE " + strings.Join(where, " AND ")
}
orderBy := " ORDER BY published_at DESC"
switch p.Sort {
case "hot":
// 热度 = 点赞数，点赞相同按最新
orderBy = " ORDER BY likes DESC, published_at DESC"
case "week":
orderBy = " ORDER BY likes DESC"
case "long":
orderBy = " ORDER BY duration_seconds DESC"
}
// count
var total int
if err := c.db.QueryRowContext(ctx, "SELECT COUNT(*) FROM videos"+whereSQL, args...).Scan(&total); err != nil {
return nil, 0, err
}
// list
offset := (p.Page - 1) * p.PageSize
rows, err := c.db.QueryContext(ctx,
"SELECT "+allVideoCols+" FROM videos"+whereSQL+orderBy+" LIMIT ? OFFSET ?",
append(args, p.PageSize, offset)...)
if err != nil {
return nil, 0, err
}
defer rows.Close()
var out []*Video
for rows.Next() {
v, err := scanVideo(rows)
if err != nil {
return nil, 0, err
}
out = append(out, v)
}
return out, total, nil
}
// ---------- Drive ----------
type Drive struct {
ID string `json:"id"`
Kind string `json:"kind"`
Name string `json:"name"`
RootID string `json:"rootId"`
ScanRootID string `json:"scanRootId"`
Credentials map[string]string `json:"credentials,omitempty"`
Status string `json:"status"`
LastError string `json:"lastError,omitempty"`
CreatedAt time.Time `json:"createdAt"`
UpdatedAt time.Time `json:"updatedAt"`
}
func (c *Catalog) UpsertDrive(ctx context.Context, d *Drive) error {
cred, _ := json.Marshal(d.Credentials)
now := time.Now().UnixMilli()
if d.CreatedAt.IsZero() {
d.CreatedAt = time.UnixMilli(now)
}
d.UpdatedAt = time.UnixMilli(now)
_, err := c.db.ExecContext(ctx, `
INSERT INTO drives (id, kind, name, root_id, scan_root_id, credentials, status, last_error, created_at, updated_at)
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
ON CONFLICT(id) DO UPDATE SET
kind = excluded.kind,
name = excluded.name,
root_id = excluded.root_id,
scan_root_id = excluded.scan_root_id,
credentials = excluded.credentials,
status = excluded.status,
last_error = excluded.last_error,
updated_at = excluded.updated_at
`, d.ID, d.Kind, d.Name, d.RootID, d.ScanRootID, string(cred), d.Status, d.LastError,
d.CreatedAt.UnixMilli(), d.UpdatedAt.UnixMilli())
return err
}
func (c *Catalog) ListDrives(ctx context.Context) ([]*Drive, error) {
rows, err := c.db.QueryContext(ctx, `SELECT id, kind, name, root_id, COALESCE(scan_root_id, ''), COALESCE(credentials, '{}'), status, COALESCE(last_error, ''), created_at, updated_at FROM drives ORDER BY created_at ASC`)
if err != nil {
return nil, err
}
defer rows.Close()
var out []*Drive
for rows.Next() {
d := &Drive{}
var credsStr string
var createdAt, updatedAt int64
if err := rows.Scan(&d.ID, &d.Kind, &d.Name, &d.RootID, &d.ScanRootID, &credsStr, &d.Status, &d.LastError, &createdAt, &updatedAt); err != nil {
return nil, err
}
_ = json.Unmarshal([]byte(credsStr), &d.Credentials)
d.CreatedAt = time.UnixMilli(createdAt)
d.UpdatedAt = time.UnixMilli(updatedAt)
out = append(out, d)
}
return out, nil
}
func (c *Catalog) GetDrive(ctx context.Context, id string) (*Drive, error) {
row := c.db.QueryRowContext(ctx, `SELECT id, kind, name, root_id, COALESCE(scan_root_id, ''), COALESCE(credentials, '{}'), status, COALESCE(last_error, ''), created_at, updated_at FROM drives WHERE id = ?`, id)
d := &Drive{}
var credsStr string
var createdAt, updatedAt int64
if err := row.Scan(&d.ID, &d.Kind, &d.Name, &d.RootID, &d.ScanRootID, &credsStr, &d.Status, &d.LastError, &createdAt, &updatedAt); err != nil {
return nil, err
}
_ = json.Unmarshal([]byte(credsStr), &d.Credentials)
d.CreatedAt = time.UnixMilli(createdAt)
d.UpdatedAt = time.UnixMilli(updatedAt)
return d, nil
}
func (c *Catalog) DeleteDrive(ctx context.Context, id string) error {
_, err := c.db.ExecContext(ctx, `DELETE FROM drives WHERE id = ?`, id)
return err
}
// ---------- Admin session ----------
func (c *Catalog) CreateSession(ctx context.Context, token string, ttl time.Duration) error {
now := time.Now()
_, err := c.db.ExecContext(ctx,
`INSERT INTO admin_sessions (token, created_at, expires_at) VALUES (?, ?, ?)`,
token, now.UnixMilli(), now.Add(ttl).UnixMilli())
return err
}
func (c *Catalog) ValidateSession(ctx context.Context, token string) (bool, error) {
var expires int64
err := c.db.QueryRowContext(ctx, `SELECT expires_at FROM admin_sessions WHERE token = ?`, token).Scan(&expires)
if err == sql.ErrNoRows {
return false, nil
}
if err != nil {
return false, err
}
return time.Now().UnixMilli() < expires, nil
}
func (c *Catalog) DeleteSession(ctx context.Context, token string) error {
_, err := c.db.ExecContext(ctx, `DELETE FROM admin_sessions WHERE token = ?`, token)
return err
}
// ---------- Settings ----------
func (c *Catalog) GetSetting(ctx context.Context, key, defaultValue string) (string, error) {
var v string
err := c.db.QueryRowContext(ctx, `SELECT value FROM settings WHERE key = ?`, key).Scan(&v)
if err == sql.ErrNoRows {
return defaultValue, nil
}
if err != nil {
return "", err
}
return v, nil
}
func (c *Catalog) SetSetting(ctx context.Context, key, value string) error {
_, err := c.db.ExecContext(ctx, `
INSERT INTO settings (key, value, updated_at) VALUES (?, ?, ?)
ON CONFLICT(key) DO UPDATE SET value = excluded.value, updated_at = excluded.updated_at
`, key, value, time.Now().UnixMilli())
return err
}
// ---------- helpers ----------
const allVideoCols = `
id, drive_id, file_id, COALESCE(parent_id, ''), title, COALESCE(author, ''), COALESCE(tags, '[]'),
duration_seconds, size_bytes, COALESCE(ext, ''), COALESCE(quality, ''), COALESCE(thumbnail_url, ''),
COALESCE(preview_file_id, ''), COALESCE(preview_local, ''), COALESCE(preview_status, 'pending'),
views, favorites, comments, likes, dislikes,
COALESCE(category, ''), COALESCE(badges, '[]'), COALESCE(description, ''),
published_at, created_at, updated_at
`
type rowScanner interface {
Scan(dest ...any) error
}
func scanVideo(row rowScanner) (*Video, error) {
v := &Video{}
var tagsJSON, badgesJSON string
var publishedAt, createdAt, updatedAt int64
err := row.Scan(
&v.ID, &v.DriveID, &v.FileID, &v.ParentID, &v.Title, &v.Author, &tagsJSON,
&v.DurationSeconds, &v.Size, &v.Ext, &v.Quality, &v.ThumbnailURL,
&v.PreviewFileID, &v.PreviewLocal, &v.PreviewStatus,
&v.Views, &v.Favorites, &v.Comments, &v.Likes, &v.Dislikes,
&v.Category, &badgesJSON, &v.Description,
&publishedAt, &createdAt, &updatedAt,
)
if err != nil {
return nil, err
}
_ = json.Unmarshal([]byte(tagsJSON), &v.Tags)
_ = json.Unmarshal([]byte(badgesJSON), &v.Badges)
v.PublishedAt = time.UnixMilli(publishedAt)
v.CreatedAt = time.UnixMilli(createdAt)
v.UpdatedAt = time.UnixMilli(updatedAt)
return v, nil
}
backend/internal/catalog/schema.sql
+72
View File
@@ -0,0 +1,72 @@
-- 视频元数据主表
CREATE TABLE IF NOT EXISTS videos (
id TEXT PRIMARY KEY, -- <drive>-<fileID> 拼接的稳定 ID
drive_id TEXT NOT NULL,
file_id TEXT NOT NULL,
parent_id TEXT,
title TEXT NOT NULL,
author TEXT,
tags TEXT, -- JSON array
duration_seconds INTEGER DEFAULT 0,
size_bytes INTEGER DEFAULT 0,
ext TEXT,
quality TEXT, -- HD / SD
thumbnail_url TEXT,
preview_file_id TEXT, -- 回写网盘后的 teaser file id
preview_local TEXT, -- 本地 teaser 路径（兜底）
preview_status TEXT DEFAULT 'pending', -- pending / ready / failed
views INTEGER DEFAULT 0,
favorites INTEGER DEFAULT 0,
comments INTEGER DEFAULT 0,
likes INTEGER DEFAULT 0,
dislikes INTEGER DEFAULT 0,
category TEXT,
badges TEXT, -- JSON array
description TEXT,
published_at INTEGER NOT NULL, -- unix ms
created_at INTEGER NOT NULL,
updated_at INTEGER NOT NULL
);
CREATE INDEX IF NOT EXISTS idx_videos_drive ON videos(drive_id, file_id);
CREATE INDEX IF NOT EXISTS idx_videos_pub ON videos(published_at DESC);
CREATE INDEX IF NOT EXISTS idx_videos_views ON videos(views DESC);
-- 网盘账户
CREATE TABLE IF NOT EXISTS drives (
id TEXT PRIMARY KEY,
kind TEXT NOT NULL, -- quark / p115 / wopan
name TEXT NOT NULL,
root_id TEXT NOT NULL DEFAULT '0',
scan_root_id TEXT, -- 扫描起点（默认 root_id）
credentials TEXT, -- JSON: cookie / refresh_token 等
status TEXT DEFAULT 'disconnected', -- disconnected / ok / error
last_error TEXT,
created_at INTEGER NOT NULL,
updated_at INTEGER NOT NULL
);
-- 扫描任务状态
CREATE TABLE IF NOT EXISTS scans (
id INTEGER PRIMARY KEY AUTOINCREMENT,
drive_id TEXT NOT NULL,
started_at INTEGER NOT NULL,
finished_at INTEGER,
scanned INTEGER DEFAULT 0,
added INTEGER DEFAULT 0,
error TEXT
);
-- 管理后台 session（简单 token 存储）
CREATE TABLE IF NOT EXISTS admin_sessions (
token TEXT PRIMARY KEY,
created_at INTEGER NOT NULL,
expires_at INTEGER NOT NULL
);
-- 全局 key-value 设置（preview 开关等）
CREATE TABLE IF NOT EXISTS settings (
key TEXT PRIMARY KEY,
value TEXT NOT NULL,
updated_at INTEGER NOT NULL
);
backend/internal/config/config.go
+117
View File
@@ -0,0 +1,117 @@
package config
import (
"fmt"
"os"
"path/filepath"
"gopkg.in/yaml.v3"
)
type Config struct {
Server Server `yaml:"server"`
Storage Storage `yaml:"storage"`
Scanner Scanner `yaml:"scanner"`
Preview Preview `yaml:"preview"`
Drives []Drive `yaml:"drives"`
}
type Server struct {
Listen string `yaml:"listen"`
Admin Admin `yaml:"admin"`
SessionSecret string `yaml:"session_secret"`
}
type Admin struct {
Username string `yaml:"username"`
Password string `yaml:"password"`
}
type Storage struct {
DBPath string `yaml:"db_path"`
LocalPreviewDir string `yaml:"local_preview_dir"`
}
type Scanner struct {
IntervalSeconds int `yaml:"interval_seconds"`
MaxDepth int `yaml:"max_depth"`
VideoExtensions []string `yaml:"video_extensions"`
}
type Preview struct {
Enabled bool `yaml:"enabled"`
FFmpegPath string `yaml:"ffmpeg_path"`
FFprobePath string `yaml:"ffprobe_path"`
DurationSeconds int `yaml:"duration_seconds"`
Width int `yaml:"width"`
Segments int `yaml:"segments"`
RemoteDir string `yaml:"remote_dir"`
}
// Drive 配置项中的敏感字段（Cookie / RefreshToken 等）最终由管理后台写入 DB
// 这里保留 yaml 中的静态定义，用于启动时预置盘。生产建议只在 DB 里维护。
type Drive struct {
ID string `yaml:"id"`
Kind string `yaml:"kind"` // quark / p115 / wopan
Name string `yaml:"name"`
RootID string `yaml:"root_id"`
Params map[string]string `yaml:"params,omitempty"`
}
// Load 读取配置；若不存在则从 config.example.yaml 复制一份并返回
func Load(path string) (*Config, error) {
if _, err := os.Stat(path); os.IsNotExist(err) {
example := filepath.Join(filepath.Dir(path), "config.example.yaml")
data, err := os.ReadFile(example)
if err != nil {
return nil, fmt.Errorf("config not found and example missing: %w", err)
}
if err := os.WriteFile(path, data, 0o644); err != nil {
return nil, fmt.Errorf("write default config: %w", err)
}
}
b, err := os.ReadFile(path)
if err != nil {
return nil, fmt.Errorf("read config: %w", err)
}
var c Config
if err := yaml.Unmarshal(b, &c); err != nil {
return nil, fmt.Errorf("parse config: %w", err)
}
c.applyDefaults()
return &c, nil
}
func (c *Config) applyDefaults() {
if c.Server.Listen == "" {
c.Server.Listen = ":8080"
}
if c.Storage.DBPath == "" {
c.Storage.DBPath = "./data/video-site.db"
}
if c.Storage.LocalPreviewDir == "" {
c.Storage.LocalPreviewDir = "./data/previews"
}
if c.Scanner.MaxDepth == 0 {
c.Scanner.MaxDepth = 5
}
if len(c.Scanner.VideoExtensions) == 0 {
c.Scanner.VideoExtensions = []string{".mp4", ".mkv", ".mov", ".webm", ".avi"}
}
if c.Preview.FFmpegPath == "" {
c.Preview.FFmpegPath = "ffmpeg"
}
if c.Preview.FFprobePath == "" {
c.Preview.FFprobePath = "ffprobe"
}
if c.Preview.DurationSeconds == 0 {
c.Preview.DurationSeconds = 9
}
if c.Preview.Width == 0 {
c.Preview.Width = 480
}
if c.Preview.Segments == 0 {
c.Preview.Segments = 3
}
}
backend/internal/drives/iface.go
+64
View File
@@ -0,0 +1,64 @@
package drives
import (
"context"
"errors"
"io"
"net/http"
"time"
)
// Drive 是三家网盘统一抽象。上层不区分盘，只区分 Kind。
type Drive interface {
// Kind 返回驱动代号："quark" / "p115" / "wopan"
Kind() string
// ID 返回该盘在 catalog 中的唯一标识
ID() string
// Init 完成登录态校验；登录态由 Authenticator 另行获取后注入
Init(ctx context.Context) error
// List 列指定目录下的直接子项
List(ctx context.Context, dirID string) ([]Entry, error)
// Stat 拿到单个文件的元数据
Stat(ctx context.Context, fileID string) (*Entry, error)
// StreamURL 返回一次性直链 + 必须的请求头
// 代理层据此回源，透传 Range
StreamURL(ctx context.Context, fileID string) (*StreamLink, error)
// Upload 把本地流写入指定目录，返回新文件 fileID
// 用于 scanner 把 teaser 写回网盘
Upload(ctx context.Context, parentID, name string, r io.Reader, size int64) (string, error)
// EnsureDir 保证指定路径存在（相对根目录），返回最终目录 fileID
// 例如传 "/previews" 会保证根下有一个 previews 目录
EnsureDir(ctx context.Context, pathFromRoot string) (string, error)
// RootID 返回根目录 fileID
RootID() string
}
type Entry struct {
ID string
Name string
Size int64
IsDir bool
ParentID string
MimeType string
ModTime time.Time
// 部分网盘额外信息
Category int // 1=视频 (quark)
}
type StreamLink struct {
URL string
Headers http.Header
Expires time.Time
}
// ErrNotSupported 代表某家盘不支持某操作
var ErrNotSupported = errors.New("operation not supported by this drive")
backend/internal/drives/p115/driver.go
+217
View File
@@ -0,0 +1,217 @@
package p115
import (
"context"
"errors"
"fmt"
"io"
"net/http"
"path"
"strings"
"time"
sdk "github.com/SheltonZhu/115driver/pkg/driver"
"github.com/video-site/backend/internal/drives"
)
type Driver struct {
id string
cookie string
rootID string
client *sdk.Pan115Client
ua string
}
type Config struct {
ID string
Cookie string // 形如 "UID=xxx; CID=xxx; SEID=xxx; KID=xxx"
RootID string // 默认 "0"
UA string // 默认 UA115Browser
}
func New(c Config) *Driver {
rootID := c.RootID
if rootID == "" {
rootID = "0"
}
ua := c.UA
if ua == "" {
ua = sdk.UA115Browser
}
return &Driver{
id: c.ID,
cookie: c.Cookie,
rootID: rootID,
ua: ua,
}
}
func (d *Driver) Kind() string { return "p115" }
func (d *Driver) ID() string { return d.id }
func (d *Driver) RootID() string { return d.rootID }
func (d *Driver) Init(ctx context.Context) error {
cr := &sdk.Credential{}
if err := cr.FromCookie(d.cookie); err != nil {
return fmt.Errorf("parse cookie: %w", err)
}
d.client = sdk.New(sdk.UA(d.ua)).ImportCredential(cr)
return d.client.LoginCheck()
}
func (d *Driver) List(ctx context.Context, dirID string) ([]drives.Entry, error) {
files, err := d.client.ListWithLimit(dirID, sdk.FileListLimit)
if err != nil {
return nil, fmt.Errorf("115 list: %w", err)
}
if files == nil {
return nil, nil
}
out := make([]drives.Entry, 0, len(*files))
for _, f := range *files {
out = append(out, fileToEntry(&f, dirID))
}
return out, nil
}
func (d *Driver) Stat(ctx context.Context, fileID string) (*drives.Entry, error) {
f, err := d.client.GetFile(fileID)
if err != nil {
return nil, fmt.Errorf("115 stat: %w", err)
}
if f == nil {
return nil, errors.New("115 stat: not found")
}
e := fileToEntry(f, f.ParentID)
return &e, nil
}
func (d *Driver) StreamURL(ctx context.Context, fileID string) (*drives.StreamLink, error) {
// 需要先拿到 pickCode
f, err := d.client.GetFile(fileID)
if err != nil {
return nil, fmt.Errorf("115 get file: %w", err)
}
info, err := d.client.DownloadWithUA(f.PickCode, d.ua)
if err != nil {
return nil, fmt.Errorf("115 download url: %w", err)
}
if info == nil || info.Url.Url == "" {
return nil, errors.New("115 download url: empty")
}
headers := http.Header{}
headers.Set("User-Agent", d.ua)
// 115 直链会返回一组 Cookie / Referer，info.Header 里带了
for k, vs := range info.Header {
for _, v := range vs {
headers.Add(k, v)
}
}
return &drives.StreamLink{
URL: info.Url.Url,
Headers: headers,
Expires: time.Now().Add(25 * time.Minute), // 115 直链 30 分钟过期，留余量
}, nil
}
func (d *Driver) Upload(ctx context.Context, parentID, name string, r io.Reader, size int64) (string, error) {
// 115 上传流程比较复杂：RapidUpload -> OSS 分片
// 第一版 teaser 文件小（<2MB），直接读全量写 seeker，走 RapidUploadOrByOSS
buf, err := io.ReadAll(r)
if err != nil {
return "", err
}
rs := strings.NewReader(string(buf))
if err := d.client.RapidUploadOrByOSS(parentID, name, size, rs); err != nil {
return "", fmt.Errorf("115 upload: %w", err)
}
// RapidUploadOrByOSS 目前没返回 fileID，需要回查
files, err := d.client.ListWithLimit(parentID, sdk.FileListLimit)
if err != nil {
return "", fmt.Errorf("115 upload verify: %w", err)
}
if files != nil {
for _, f := range *files {
if !f.IsDirectory && f.Name == name {
return f.FileID, nil
}
}
}
return "", errors.New("115 upload: file not found after upload")
}
func (d *Driver) EnsureDir(ctx context.Context, pathFromRoot string) (string, error) {
parts := splitPath(pathFromRoot)
currentID := d.rootID
for _, name := range parts {
childID, err := d.findChildDir(ctx, currentID, name)
if err != nil {
return "", err
}
if childID == "" {
id, err := d.client.Mkdir(currentID, name)
if err != nil {
return "", fmt.Errorf("115 mkdir %s: %w", name, err)
}
childID = id
}
currentID = childID
}
return currentID, nil
}
func (d *Driver) findChildDir(ctx context.Context, parent, name string) (string, error) {
entries, err := d.List(ctx, parent)
if err != nil {
return "", err
}
for _, e := range entries {
if e.IsDir && e.Name == name {
return e.ID, nil
}
}
return "", nil
}
func splitPath(p string) []string {
p = strings.Trim(p, "/")
if p == "" {
return nil
}
return strings.Split(p, "/")
}
func fileToEntry(f *sdk.File, parentID string) drives.Entry {
return drives.Entry{
ID: f.FileID,
Name: f.Name,
Size: f.Size,
IsDir: f.IsDirectory,
ParentID: parentID,
MimeType: guessMime(f.Name),
ModTime: f.UpdateTime,
}
}
func guessMime(name string) string {
ext := strings.ToLower(path.Ext(name))
switch ext {
case ".mp4":
return "video/mp4"
case ".mkv":
return "video/x-matroska"
case ".mov":
return "video/quicktime"
case ".webm":
return "video/webm"
case ".jpg", ".jpeg":
return "image/jpeg"
case ".png":
return "image/png"
}
return "application/octet-stream"
}
var _ drives.Drive = (*Driver)(nil)
backend/internal/drives/quark/driver.go
+345
View File
@@ -0,0 +1,345 @@
package quark
import (
"context"
"errors"
"fmt"
"io"
"net/http"
"path"
"strconv"
"strings"
"time"
"github.com/go-resty/resty/v2"
"github.com/video-site/backend/internal/drives"
)
const (
defaultUA = "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) quark-cloud-drive/2.5.20 Chrome/100.0.4896.160 Electron/18.3.5.4-b478491100 Safari/537.36 Channel/pckk_other_ch"
defaultReferer = "https://pan.quark.cn"
defaultAPI = "https://drive.quark.cn/1/clouddrive"
defaultPR = "ucpro"
)
type Driver struct {
id string
cookie string
rootID string
ua string
referer string
apiBase string
pr string
client *resty.Client
onCookieUpdate func(string)
useTranscodingAddress bool
}
type Config struct {
ID string
Cookie string
RootID string
UseTranscodingAddress bool // 开启后对视频文件返回转码直链（支持 302），但可能画质不一致
OnCookieUpdate func(cookie string)
}
func New(c Config) *Driver {
rootID := c.RootID
if rootID == "" {
rootID = "0"
}
d := &Driver{
id: c.ID,
cookie: c.Cookie,
rootID: rootID,
ua: defaultUA,
referer: defaultReferer,
apiBase: defaultAPI,
pr: defaultPR,
useTranscodingAddress: c.UseTranscodingAddress,
onCookieUpdate: c.OnCookieUpdate,
}
d.client = resty.New().
SetTimeout(30 * time.Second).
SetHeader("Accept", "application/json, text/plain, */*").
SetHeader("Referer", d.referer).
SetHeader("User-Agent", d.ua)
return d
}
func (d *Driver) Kind() string { return "quark" }
func (d *Driver) ID() string { return d.id }
func (d *Driver) RootID() string { return d.rootID }
// ---------- 公共请求 ----------
type resp struct {
Status int `json:"status"`
Code int `json:"code"`
Message string `json:"message"`
}
func (d *Driver) request(ctx context.Context, path, method string, query map[string]string, body any, out any) error {
req := d.client.R().
SetContext(ctx).
SetHeader("Cookie", d.cookie).
SetQueryParam("pr", d.pr).
SetQueryParam("fr", "pc")
if query != nil {
req.SetQueryParams(query)
}
if body != nil {
req.SetBody(body)
}
if out != nil {
req.SetResult(out)
}
var e resp
req.SetError(&e)
res, err := req.Execute(method, d.apiBase+path)
if err != nil {
return err
}
// 处理 cookie 刷新（__puus）
for _, ck := range res.Cookies() {
if ck.Name == "__puus" {
d.cookie = setCookieValue(d.cookie, "__puus", ck.Value)
if d.onCookieUpdate != nil {
d.onCookieUpdate(d.cookie)
}
}
}
if e.Status >= 400 || e.Code != 0 {
if e.Message == "" {
return fmt.Errorf("quark api error: status=%d code=%d", e.Status, e.Code)
}
return errors.New(e.Message)
}
return nil
}
func (d *Driver) Init(ctx context.Context) error {
return d.request(ctx, "/config", http.MethodGet, nil, nil, nil)
}
// ---------- 列目录 ----------
type file struct {
Fid string `json:"fid"`
FileName string `json:"file_name"`
Size int64 `json:"size"`
Category int `json:"category"`
File bool `json:"file"`
UpdatedAt int64 `json:"updated_at"`
}
type sortResp struct {
Data struct {
List []file `json:"list"`
} `json:"data"`
Metadata struct {
Total int `json:"_total"`
} `json:"metadata"`
}
func (d *Driver) List(ctx context.Context, dirID string) ([]drives.Entry, error) {
var out []drives.Entry
page := 1
size := 100
for {
q := map[string]string{
"pdir_fid": dirID,
"_size": strconv.Itoa(size),
"_page": strconv.Itoa(page),
"_fetch_total": "1",
"fetch_all_file": "1",
"fetch_risk_file_name": "1",
}
var r sortResp
if err := d.request(ctx, "/file/sort", http.MethodGet, q, nil, &r); err != nil {
return nil, fmt.Errorf("quark list: %w", err)
}
for _, f := range r.Data.List {
out = append(out, fileToEntry(&f, dirID))
}
if page*size >= r.Metadata.Total {
break
}
page++
}
return out, nil
}
func (d *Driver) Stat(ctx context.Context, fileID string) (*drives.Entry, error) {
// 夸克没提供单文件查询接口，回退到父目录遍历需要额外信息
return nil, drives.ErrNotSupported
}
// ---------- 下载直链 ----------
type downResp struct {
Data []struct {
DownloadUrl string `json:"download_url"`
} `json:"data"`
}
func (d *Driver) StreamURL(ctx context.Context, fileID string) (*drives.StreamLink, error) {
body := map[string]any{"fids": []string{fileID}}
var r downResp
if err := d.request(ctx, "/file/download", http.MethodPost, nil, body, &r); err != nil {
return nil, fmt.Errorf("quark download: %w", err)
}
if len(r.Data) == 0 || r.Data[0].DownloadUrl == "" {
return nil, errors.New("quark download: empty url")
}
headers := http.Header{}
headers.Set("User-Agent", d.ua)
headers.Set("Referer", d.referer)
headers.Set("Cookie", d.cookie)
return &drives.StreamLink{
URL: r.Data[0].DownloadUrl,
Headers: headers,
Expires: time.Now().Add(10 * time.Minute),
}, nil
}
// ---------- 创建目录 ----------
type mkdirResp struct {
Data struct {
Fid string `json:"fid"`
} `json:"data"`
}
func (d *Driver) MakeDir(ctx context.Context, parentID, name string) (string, error) {
body := map[string]any{
"dir_init_lock": false,
"dir_path": "",
"file_name": name,
"pdir_fid": parentID,
}
var r mkdirResp
if err := d.request(ctx, "/file", http.MethodPost, nil, body, &r); err != nil {
return "", fmt.Errorf("quark mkdir: %w", err)
}
return r.Data.Fid, nil
}
func (d *Driver) EnsureDir(ctx context.Context, pathFromRoot string) (string, error) {
parts := splitPath(pathFromRoot)
currentID := d.rootID
for _, name := range parts {
childID, err := d.findChildDir(ctx, currentID, name)
if err != nil {
return "", err
}
if childID == "" {
id, err := d.MakeDir(ctx, currentID, name)
if err != nil {
return "", err
}
childID = id
}
currentID = childID
}
return currentID, nil
}
func (d *Driver) findChildDir(ctx context.Context, parent, name string) (string, error) {
entries, err := d.List(ctx, parent)
if err != nil {
return "", err
}
for _, e := range entries {
if e.IsDir && e.Name == name {
return e.ID, nil
}
}
return "", nil
}
// ---------- 上传（第一版不实现，走本地 teaser 兜底） ----------
func (d *Driver) Upload(ctx context.Context, parentID, name string, r io.Reader, size int64) (string, error) {
return "", drives.ErrNotSupported
}
// ---------- helpers ----------
func fileToEntry(f *file, parentID string) drives.Entry {
return drives.Entry{
ID: f.Fid,
Name: f.FileName,
Size: f.Size,
IsDir: !f.File,
ParentID: parentID,
MimeType: guessMime(f.FileName),
ModTime: time.UnixMilli(f.UpdatedAt),
Category: f.Category,
}
}
func guessMime(name string) string {
ext := strings.ToLower(path.Ext(name))
switch ext {
case ".mp4":
return "video/mp4"
case ".mkv":
return "video/x-matroska"
case ".mov":
return "video/quicktime"
case ".webm":
return "video/webm"
case ".jpg", ".jpeg":
return "image/jpeg"
case ".png":
return "image/png"
}
return "application/octet-stream"
}
func splitPath(p string) []string {
p = strings.Trim(p, "/")
if p == "" {
return nil
}
return strings.Split(p, "/")
}
// setCookieValue 替换 cookie 字符串中某个 key 的值，不存在则追加
func setCookieValue(cookie, key, value string) string {
if cookie == "" {
return key + "=" + value
}
parts := strings.Split(cookie, ";")
var out []string
found := false
for _, p := range parts {
kv := strings.TrimSpace(p)
if kv == "" {
continue
}
eq := strings.IndexByte(kv, '=')
if eq < 0 {
out = append(out, kv)
continue
}
if kv[:eq] == key {
out = append(out, key+"="+value)
found = true
} else {
out = append(out, kv)
}
}
if !found {
out = append(out, key+"="+value)
}
return strings.Join(out, "; ")
}
var _ drives.Drive = (*Driver)(nil)
backend/internal/drives/wopan/driver.go
+231
View File
@@ -0,0 +1,231 @@
package wopan
import (
"context"
"fmt"
"io"
"net/http"
"os"
"path"
"strings"
"time"
sdk "github.com/OpenListTeam/wopan-sdk-go"
"github.com/video-site/backend/internal/drives"
)
// Driver 封装联通沃盘
type Driver struct {
id string
rootID string
familyID string
accessToken string
refreshToken string
client *sdk.WoClient
onTokenUpdate func(access, refresh string)
}
type Config struct {
ID string
AccessToken string
RefreshToken string
FamilyID string // 空则走个人空间，有值则走家庭空间
RootID string // 根目录 ID，默认 "0"
// 当 SDK 刷新 token 时回调，便于持久化
OnTokenUpdate func(access, refresh string)
}
func New(c Config) *Driver {
rootID := c.RootID
if rootID == "" {
rootID = "0"
}
return &Driver{
id: c.ID,
rootID: rootID,
familyID: c.FamilyID,
accessToken: c.AccessToken,
refreshToken: c.RefreshToken,
onTokenUpdate: c.OnTokenUpdate,
}
}
func (d *Driver) Kind() string { return "wopan" }
func (d *Driver) ID() string { return d.id }
func (d *Driver) RootID() string {
return d.rootID
}
func (d *Driver) Init(ctx context.Context) error {
d.client = sdk.DefaultWithRefreshToken(d.refreshToken)
d.client.SetAccessToken(d.accessToken)
d.client.OnRefreshToken(func(access, refresh string) {
d.accessToken = access
d.refreshToken = refresh
if d.onTokenUpdate != nil {
d.onTokenUpdate(access, refresh)
}
})
// InitData 会触发一次 token 校验
return d.client.InitData()
}
func (d *Driver) spaceType() string {
if d.familyID != "" {
return sdk.SpaceTypeFamily
}
return sdk.SpaceTypePersonal
}
func (d *Driver) List(ctx context.Context, dirID string) ([]drives.Entry, error) {
var result []drives.Entry
pageNum := 0
pageSize := 100
for {
data, err := d.client.QueryAllFiles(d.spaceType(), dirID, pageNum, pageSize, 0, d.familyID)
if err != nil {
return nil, fmt.Errorf("wopan list: %w", err)
}
for _, f := range data.Files {
result = append(result, fileToEntry(f, dirID))
}
if len(data.Files) < pageSize {
break
}
pageNum++
}
return result, nil
}
func (d *Driver) Stat(ctx context.Context, fileID string) (*drives.Entry, error) {
// 沃盘 SDK 没有单文件查询，退化为遍历父目录 —— 这里第一版只在 scanner 路径使用 List，Stat 保留 stub
return nil, drives.ErrNotSupported
}
func (d *Driver) StreamURL(ctx context.Context, fileID string) (*drives.StreamLink, error) {
data, err := d.client.GetDownloadUrlV2([]string{fileID})
if err != nil {
return nil, fmt.Errorf("wopan download url: %w", err)
}
if len(data.List) == 0 {
return nil, fmt.Errorf("wopan download url: empty response")
}
return &drives.StreamLink{
URL: data.List[0].DownloadUrl,
Headers: http.Header{},
Expires: time.Now().Add(10 * time.Minute),
}, nil
}
func (d *Driver) Upload(ctx context.Context, parentID, name string, r io.Reader, size int64) (string, error) {
// wopan SDK 要求 *os.File，先把流落到临时文件再上传
tmp, err := os.CreateTemp("", "wopan-upload-*.tmp")
if err != nil {
return "", err
}
defer func() {
tmp.Close()
os.Remove(tmp.Name())
}()
if _, err := io.Copy(tmp, r); err != nil {
return "", err
}
if _, err := tmp.Seek(0, 0); err != nil {
return "", err
}
fid, err := d.client.Upload2C(d.spaceType(), sdk.Upload2CFile{
Name: name,
Size: size,
Content: tmp,
ContentType: "application/octet-stream",
}, parentID, d.familyID, sdk.Upload2COption{Ctx: ctx})
if err != nil {
return "", fmt.Errorf("wopan upload: %w", err)
}
return fid, nil
}
func (d *Driver) EnsureDir(ctx context.Context, pathFromRoot string) (string, error) {
parts := splitPath(pathFromRoot)
currentID := d.rootID
for _, name := range parts {
childID, err := d.findChildDir(ctx, currentID, name)
if err != nil {
return "", err
}
if childID == "" {
resp, err := d.client.CreateDirectory(d.spaceType(), currentID, name, d.familyID)
if err != nil {
return "", fmt.Errorf("wopan mkdir %s: %w", name, err)
}
childID = resp.Id
}
currentID = childID
}
return currentID, nil
}
func (d *Driver) findChildDir(ctx context.Context, parent, name string) (string, error) {
entries, err := d.List(ctx, parent)
if err != nil {
return "", err
}
for _, e := range entries {
if e.IsDir && e.Name == name {
return e.ID, nil
}
}
return "", nil
}
func splitPath(p string) []string {
p = strings.Trim(p, "/")
if p == "" {
return nil
}
return strings.Split(p, "/")
}
func fileToEntry(f *sdk.File, parentID string) drives.Entry {
mod, _ := time.Parse("2006-01-02 15:04:05", f.CreateTime)
name := f.Name
isDir := f.Type == 0
id := f.Fid
if id == "" {
id = f.Id
}
if isDir && !strings.HasSuffix(name, "/") {
// 不改 name，只标志
}
return drives.Entry{
ID: id,
Name: name,
Size: f.Size,
IsDir: isDir,
ParentID: parentID,
MimeType: guessMime(name),
ModTime: mod,
}
}
func guessMime(name string) string {
ext := strings.ToLower(path.Ext(name))
switch ext {
case ".mp4":
return "video/mp4"
case ".mkv":
return "video/x-matroska"
case ".mov":
return "video/quicktime"
case ".webm":
return "video/webm"
case ".jpg", ".jpeg":
return "image/jpeg"
case ".png":
return "image/png"
}
return "application/octet-stream"
}
// 确保实现接口
var _ drives.Drive = (*Driver)(nil)
backend/internal/preview/ffmpeg.go
+493
View File
@@ -0,0 +1,493 @@
package preview
import (
"context"
"fmt"
"io"
"log"
"math"
"os"
"os/exec"
"path/filepath"
"strconv"
"strings"
"time"
"github.com/video-site/backend/internal/catalog"
"github.com/video-site/backend/internal/drives"
)
type Config struct {
FFmpegPath string
FFprobePath string
DurationSeconds int // 单段时长（秒），用于单段 fallback；拼接模式下每段 = DurationSeconds / 段数
Width int
Segments int // teaser 段数，1=单段，推荐 3
LocalDir string // 本地兜底
RemoteDir string // 远端目录路径（相对盘根）
}
type Generator struct {
cfg Config
}
func New(cfg Config) *Generator {
if cfg.FFmpegPath == "" {
cfg.FFmpegPath = "ffmpeg"
}
if cfg.FFprobePath == "" {
cfg.FFprobePath = "ffprobe"
}
if cfg.DurationSeconds == 0 {
cfg.DurationSeconds = 9 // 3 段 × 3 秒
}
if cfg.Width == 0 {
cfg.Width = 480
}
if cfg.Segments <= 0 {
cfg.Segments = 3
}
return &Generator{cfg: cfg}
}
// --- 选段策略 ---
// pickSegmentStarts 根据视频总时长选出 N 段起点秒数（按时间升序）
//
// 规则：
// - duration < 30s → 单段从 max(2, duration*0.1) 起
// - 30s ≤ duration < 10min → N 段：前段跳过片头、末段避开片尾
// - duration ≥ 10min → 20% / 50% / 80%（或按 N 等距分布）
func pickSegmentStarts(duration float64, n int, eachSec float64) []float64 {
if n <= 0 {
n = 1
}
if duration <= 0 {
// 未知时长，用保守默认
return []float64{10}
}
// 余量：保证最后一段结束前留 1 秒，避免切到文件末尾
usable := duration - eachSec - 1
if usable < 0 {
usable = 0
}
if duration < 30 {
start := math.Max(2, duration*0.1)
if start > usable {
start = math.Max(0, usable)
}
return []float64{start}
}
if duration < 600 {
// 30s ~ 10min：20% 起，均匀分段
starts := make([]float64, 0, n)
// 保证第一段跳过片头（>= 5% 或 3s）
firstMin := math.Max(3, duration*0.05)
// 最后一段结束 <= 85%，避开结尾
lastMax := duration * 0.85
if lastMax < firstMin {
lastMax = firstMin
}
if n == 1 {
return []float64{duration * 0.25}
}
step := (lastMax - firstMin) / float64(n-1)
for i := 0; i < n; i++ {
s := firstMin + step*float64(i)
if s > usable {
s = usable
}
starts = append(starts, s)
}
return starts
}
// 长视频：按 20% / 50% / 80% 布置
if n == 1 {
return []float64{duration * 0.3}
}
starts := make([]float64, 0, n)
pct := make([]float64, 0, n)
// 均匀在 [0.2, 0.8] 区间取 N 个点
lo, hi := 0.2, 0.8
if n == 1 {
pct = append(pct, 0.3)
} else {
step := (hi - lo) / float64(n-1)
for i := 0; i < n; i++ {
pct = append(pct, lo+step*float64(i))
}
}
for _, p := range pct {
s := duration * p
if s > usable {
s = usable
}
starts = append(starts, s)
}
return starts
}
// pickThumbnailOffset 选封面抽帧的时间点（秒）。独立于 teaser。
func pickThumbnailOffset(duration float64) float64 {
if duration <= 0 {
return 5
}
// 短视频从 30% 抽；长视频从 20% 抽，避开片头
if duration < 60 {
return math.Max(1, duration*0.3)
}
return math.Max(5, math.Min(duration*0.2, 120))
}
// --- 封面 ---
// GenerateThumbnail 抽一张 jpg 封面。偏移点由 duration 决定（独立于 teaser）。
func (g *Generator) GenerateThumbnail(ctx context.Context, link *drives.StreamLink, videoID string, duration float64) (string, error) {
dir := filepath.Join(g.cfg.LocalDir, "thumbs")
if err := os.MkdirAll(dir, 0o755); err != nil {
return "", err
}
dst := filepath.Join(dir, videoID+".jpg")
offset := pickThumbnailOffset(duration)
ctx2, cancel := context.WithTimeout(ctx, 60*time.Second)
defer cancel()
args := []string{
"-hide_banner",
"-loglevel", "error",
"-ss", fmt.Sprintf("%.2f", offset),
}
if h := buildHeaders(link.Headers); h != "" {
args = append(args, "-headers", h)
}
args = append(args,
"-i", link.URL,
"-frames:v", "1",
"-vf", fmt.Sprintf("scale=%d:-2", g.cfg.Width),
"-q:v", "3",
"-y", dst,
)
cmd := exec.CommandContext(ctx2, g.cfg.FFmpegPath, args...)
out, err := cmd.CombinedOutput()
if err != nil {
os.Remove(dst)
return "", fmt.Errorf("ffmpeg thumb: %w, stderr: %s", err, string(out))
}
if info, statErr := os.Stat(dst); statErr != nil || info.Size() == 0 {
os.Remove(dst)
return "", fmt.Errorf("ffmpeg thumb produced empty file, stderr: %s", string(out))
}
return dst, nil
}
// --- 时长 ---
// Probe 用 ffprobe 拿视频时长（秒，浮点）
func (g *Generator) Probe(ctx context.Context, link *drives.StreamLink) (float64, error) {
ctx2, cancel := context.WithTimeout(ctx, 30*time.Second)
defer cancel()
args := []string{
"-hide_banner",
"-loglevel", "error",
"-show_entries", "format=duration",
"-of", "default=noprint_wrappers=1:nokey=1",
}
if h := buildHeaders(link.Headers); h != "" {
args = append(args, "-headers", h)
}
args = append(args, link.URL)
cmd := exec.CommandContext(ctx2, g.cfg.FFprobePath, args...)
out, err := cmd.Output()
if err != nil {
return 0, fmt.Errorf("ffprobe: %w", err)
}
raw := strings.TrimSpace(string(out))
if raw == "" || raw == "N/A" {
return 0, nil
}
return strconv.ParseFloat(raw, 64)
}
// --- Teaser ---
// Generate 拉取 teaser 到本地临时文件，返回路径。
// 根据 Config.Segments 和视频时长决定是单段还是多段拼接。
func (g *Generator) Generate(ctx context.Context, link *drives.StreamLink, duration float64) (string, error) {
if err := os.MkdirAll(g.cfg.LocalDir, 0o755); err != nil {
return "", err
}
segs := g.cfg.Segments
// 视频太短直接单段
if duration > 0 && duration < 30 {
segs = 1
}
eachSec := float64(g.cfg.DurationSeconds)
if segs > 1 {
eachSec = float64(g.cfg.DurationSeconds) / float64(segs)
if eachSec < 2 {
eachSec = 2
}
}
starts := pickSegmentStarts(duration, segs, eachSec)
ctx2, cancel := context.WithTimeout(ctx, 4*time.Minute)
defer cancel()
// 用 ffmpeg 的 concat 滤镜一次输出：多个 -ss input 再 concat + fade
tmp, err := os.CreateTemp(g.cfg.LocalDir, "teaser-*.mp4")
if err != nil {
return "", err
}
tmpPath := tmp.Name()
tmp.Close()
args := []string{
"-hide_banner",
"-loglevel", "error",
}
headers := buildHeaders(link.Headers)
// 每段独立 -ss + -i，精确 seek 重新解码保证拼接帧准
for _, s := range starts {
if headers != "" {
args = append(args, "-headers", headers)
}
args = append(args,
"-ss", fmt.Sprintf("%.2f", s),
"-t", fmt.Sprintf("%.2f", eachSec),
"-i", link.URL,
)
}
if len(starts) == 1 {
// 单段：无需 concat，直接缩放 + 无音
args = append(args,
"-an",
"-vf", fmt.Sprintf("scale=%d:-2", g.cfg.Width),
"-c:v", "libx264",
"-preset", "veryfast",
"-crf", "28",
"-movflags", "+faststart",
"-y", tmpPath,
)
} else {
// 多段：各段缩放 + 0.2s 黑场淡入淡出，concat 拼接
// filter_complex: [0:v]scale,fade=in:0:5,fade=out:start=eachSec-0.2:d=0.2[v0]; ...; [v0][v1][v2]concat=n=3:v=1:a=0[v]
fadeIn := 0.2
fadeOutStart := eachSec - 0.2
if fadeOutStart < 0 {
fadeOutStart = 0
}
var filter strings.Builder
for i := range starts {
if i > 0 {
filter.WriteString(";")
}
fmt.Fprintf(&filter,
"[%d:v]scale=%d:-2,fade=t=in:st=0:d=%.2f,fade=t=out:st=%.2f:d=0.2[v%d]",
i, g.cfg.Width, fadeIn, fadeOutStart, i)
}
filter.WriteString(";")
for i := range starts {
fmt.Fprintf(&filter, "[v%d]", i)
}
fmt.Fprintf(&filter, "concat=n=%d:v=1:a=0[v]", len(starts))
args = append(args,
"-filter_complex", filter.String(),
"-map", "[v]",
"-an",
"-c:v", "libx264",
"-preset", "veryfast",
"-crf", "28",
"-movflags", "+faststart",
"-y", tmpPath,
)
}
cmd := exec.CommandContext(ctx2, g.cfg.FFmpegPath, args...)
out, err := cmd.CombinedOutput()
if err != nil {
os.Remove(tmpPath)
return "", fmt.Errorf("ffmpeg: %w, stderr: %s", err, string(out))
}
if info, statErr := os.Stat(tmpPath); statErr != nil || info.Size() == 0 {
os.Remove(tmpPath)
return "", fmt.Errorf("ffmpeg produced empty file, stderr: %s", string(out))
}
return tmpPath, nil
}
// --- 本地落盘 ---
// MoveToLocal 把临时文件改名到稳定位置，返回最终路径
func (g *Generator) MoveToLocal(tmpPath, videoID string) (string, error) {
dst := filepath.Join(g.cfg.LocalDir, videoID+".mp4")
if err := os.Rename(tmpPath, dst); err != nil {
// 跨盘 rename 可能失败，fallback 到 copy
if cerr := copyFile(tmpPath, dst); cerr != nil {
return "", cerr
}
_ = os.Remove(tmpPath)
}
return dst, nil
}
func copyFile(src, dst string) error {
in, err := os.Open(src)
if err != nil {
return err
}
defer in.Close()
out, err := os.Create(dst)
if err != nil {
return err
}
defer out.Close()
_, err = io.Copy(out, in)
return err
}
// --- Worker ---
type Worker struct {
Gen *Generator
Catalog *catalog.Catalog
Drive drives.Drive
RemoteDir string
ch chan *catalog.Video
}
func NewWorker(gen *Generator, cat *catalog.Catalog, drv drives.Drive, remoteDir string) *Worker {
return &Worker{
Gen: gen,
Catalog: cat,
Drive: drv,
RemoteDir: remoteDir,
ch: make(chan *catalog.Video, 4096),
}
}
func (w *Worker) Enqueue(v *catalog.Video) {
select {
case w.ch <- v:
default:
}
}
// Run 阻塞运行直到 ctx 取消
func (w *Worker) Run(ctx context.Context) {
for {
select {
case <-ctx.Done():
return
case v := <-w.ch:
w.process(ctx, v)
select {
case <-ctx.Done():
return
case <-time.After(500 * time.Millisecond):
}
}
}
}
func (w *Worker) process(ctx context.Context, v *catalog.Video) {
link, err := w.Drive.StreamURL(ctx, v.FileID)
if err != nil {
log.Printf("[preview] streamURL %s: %v", v.Title, err)
w.Catalog.UpdatePreview(ctx, v.ID, "", "", "failed")
return
}
// 1) 探时长（失败用 0 继续）
duration := float64(v.DurationSeconds)
if duration <= 0 {
if dur, err := w.Gen.Probe(ctx, link); err == nil && dur > 0 {
duration = dur
_ = w.Catalog.UpdateVideoMeta(ctx, v.ID, catalog.VideoMetaPatch{
DurationSeconds: int(dur),
})
}
}
// 2) 封面（独立时间点，失败不致命）
if _, err := w.Gen.GenerateThumbnail(ctx, link, v.ID, duration); err != nil {
log.Printf("[preview] thumbnail %s: %v", v.Title, err)
} else {
_ = w.Catalog.UpdateVideoMeta(ctx, v.ID, catalog.VideoMetaPatch{
ThumbnailURL: "/p/thumb/" + v.ID,
})
}
// 3) teaser
tmp, err := w.Gen.Generate(ctx, link, duration)
if err != nil {
log.Printf("[preview] generate %s: %v", v.Title, err)
w.Catalog.UpdatePreview(ctx, v.ID, "", "", "failed")
return
}
local, err := w.Gen.MoveToLocal(tmp, v.ID)
if err != nil {
log.Printf("[preview] move %s: %v", v.Title, err)
w.Catalog.UpdatePreview(ctx, v.ID, "", "", "failed")
return
}
previewFileID := ""
if w.RemoteDir != "" {
if fid, uerr := w.uploadToDrive(ctx, v.ID, local); uerr == nil {
previewFileID = fid
} else {
log.Printf("[preview] upload %s: %v (local only)", v.Title, uerr)
}
}
w.Catalog.UpdatePreview(ctx, v.ID, previewFileID, local, "ready")
log.Printf("[preview] ready %s (duration=%.1fs)", v.Title, duration)
}
func (w *Worker) uploadToDrive(ctx context.Context, videoID, localPath string) (string, error) {
parentID, err := w.Drive.EnsureDir(ctx, w.RemoteDir)
if err != nil {
return "", err
}
f, err := os.Open(localPath)
if err != nil {
return "", err
}
defer f.Close()
stat, err := f.Stat()
if err != nil {
return "", err
}
return w.Drive.Upload(ctx, parentID, videoID+".mp4", f, stat.Size())
}
// --- utils ---
func buildHeaders(h map[string][]string) string {
if len(h) == 0 {
return ""
}
var sb strings.Builder
for k, vs := range h {
for _, v := range vs {
sb.WriteString(k)
sb.WriteString(": ")
sb.WriteString(v)
sb.WriteString("\r\n")
}
}
return sb.String()
}
backend/internal/proxy/proxy.go
+169
View File
@@ -0,0 +1,169 @@
package proxy
import (
"context"
"io"
"net/http"
"net/url"
"sync"
"time"
"github.com/video-site/backend/internal/drives"
)
// Registry 管理多个 Drive 实例
type Registry struct {
mu sync.RWMutex
drives map[string]drives.Drive
}
func NewRegistry() *Registry {
return &Registry{drives: make(map[string]drives.Drive)}
}
func (r *Registry) Set(id string, d drives.Drive) {
r.mu.Lock()
defer r.mu.Unlock()
r.drives[id] = d
}
func (r *Registry) Get(id string) (drives.Drive, bool) {
r.mu.RLock()
defer r.mu.RUnlock()
d, ok := r.drives[id]
return d, ok
}
func (r *Registry) All() []drives.Drive {
r.mu.RLock()
defer r.mu.RUnlock()
out := make([]drives.Drive, 0, len(r.drives))
for _, d := range r.drives {
out = append(out, d)
}
return out
}
func (r *Registry) Remove(id string) {
r.mu.Lock()
defer r.mu.Unlock()
delete(r.drives, id)
}
// Proxy 根据 driveID + fileID 反向代理到真实网盘直链
type Proxy struct {
Registry *Registry
// linkCache key: driveID + "/" + fileID，value: cachedLink
cacheMu sync.Mutex
cache map[string]cachedLink
http *http.Client
}
type cachedLink struct {
link *drives.StreamLink
fetched time.Time
}
func New(r *Registry) *Proxy {
return &Proxy{
Registry: r,
cache: make(map[string]cachedLink),
http: &http.Client{
Timeout: 0, // 流式不设超时
},
}
}
func (p *Proxy) getLink(ctx context.Context, driveID, fileID string) (*drives.StreamLink, error) {
key := driveID + "/" + fileID
p.cacheMu.Lock()
if c, ok := p.cache[key]; ok {
// 缓存 30 秒，且不超过 link.Expires
if time.Since(c.fetched) < 30*time.Second && time.Now().Before(c.link.Expires) {
p.cacheMu.Unlock()
return c.link, nil
}
}
p.cacheMu.Unlock()
d, ok := p.Registry.Get(driveID)
if !ok {
return nil, errDriveNotFound
}
link, err := d.StreamURL(ctx, fileID)
if err != nil {
return nil, err
}
p.cacheMu.Lock()
p.cache[key] = cachedLink{link: link, fetched: time.Now()}
p.cacheMu.Unlock()
return link, nil
}
func (p *Proxy) ServeStream(w http.ResponseWriter, r *http.Request, driveID, fileID string) {
link, err := p.getLink(r.Context(), driveID, fileID)
if err != nil {
http.Error(w, err.Error(), http.StatusBadGateway)
return
}
p.serve(w, r, link)
}
func (p *Proxy) serve(w http.ResponseWriter, r *http.Request, link *drives.StreamLink) {
// 构造上游请求
u, err := url.Parse(link.URL)
if err != nil {
http.Error(w, "bad upstream url", http.StatusBadGateway)
return
}
req, err := http.NewRequestWithContext(r.Context(), r.Method, u.String(), nil)
if err != nil {
http.Error(w, err.Error(), http.StatusInternalServerError)
return
}
// 复制上游请求头
for k, vs := range link.Headers {
for _, v := range vs {
req.Header.Add(k, v)
}
}
// 透传 Range
if rng := r.Header.Get("Range"); rng != "" {
req.Header.Set("Range", rng)
}
resp, err := p.http.Do(req)
if err != nil {
http.Error(w, err.Error(), http.StatusBadGateway)
return
}
defer resp.Body.Close()
// 透传响应头
for _, k := range []string{
"Content-Type", "Content-Length", "Content-Range",
"Accept-Ranges", "Last-Modified", "Etag",
} {
if v := resp.Header.Get(k); v != "" {
w.Header().Set(k, v)
}
}
w.Header().Set("Cache-Control", "private, max-age=300")
w.WriteHeader(resp.StatusCode)
_, _ = io.Copy(w, resp.Body)
}
// ServeLocal 服务本地 teaser 文件
func (p *Proxy) ServeLocal(w http.ResponseWriter, r *http.Request, path string) {
http.ServeFile(w, r, path)
}
var errDriveNotFound = &httpError{Code: http.StatusNotFound, Msg: "drive not found"}
type httpError struct {
Code int
Msg string
}
func (e *httpError) Error() string { return e.Msg }
backend/internal/scanner/filename.go
+49
View File
@@ -0,0 +1,49 @@
package scanner
import (
"path"
"regexp"
"strings"
)
// ParsedName 从文件名里解析出的视频元数据
type ParsedName struct {
Title string
Author string
Tags []string
}
var (
reTags = regexp.MustCompile(`^\s*\[([^\]]+)\]\s*`) // [tag1,tag2]
reAuthor = regexp.MustCompile(`\s*-\s*([^-]+?)\s*$`) // - author
)
// Parse 按约定解析：[tag1,tag2] 标题 - 作者.ext
// 任何字段缺失都能降级
func Parse(filename string) ParsedName {
name := strings.TrimSuffix(filename, path.Ext(filename))
var out ParsedName
if m := reTags.FindStringSubmatch(name); m != nil {
raw := m[1]
parts := strings.FieldsFunc(raw, func(r rune) bool {
return r == ',' || r == '，' || r == '、' || r == ' '
})
for _, p := range parts {
p = strings.TrimSpace(p)
if p != "" {
out.Tags = append(out.Tags, p)
}
}
name = strings.TrimSpace(name[len(m[0]):])
}
if m := reAuthor.FindStringSubmatch(name); m != nil {
out.Author = strings.TrimSpace(m[1])
name = strings.TrimSpace(name[:len(name)-len(m[0])])
}
out.Title = strings.TrimSpace(name)
return out
}
backend/internal/scanner/scanner.go
+139
View File
@@ -0,0 +1,139 @@
package scanner
import (
"context"
"fmt"
"log"
"path"
"strings"
"time"
"github.com/video-site/backend/internal/catalog"
"github.com/video-site/backend/internal/drives"
)
type Scanner struct {
Catalog *catalog.Catalog
Drive drives.Drive
Exts map[string]bool
MaxDepth int
// 回调：新视频被加入后触发 teaser 生成
OnNewVideo func(v *catalog.Video)
}
func New(cat *catalog.Catalog, drv drives.Drive, exts []string, maxDepth int, onNew func(v *catalog.Video)) *Scanner {
m := make(map[string]bool, len(exts))
for _, e := range exts {
m[strings.ToLower(e)] = true
}
if maxDepth == 0 {
maxDepth = 5
}
return &Scanner{
Catalog: cat,
Drive: drv,
Exts: m,
MaxDepth: maxDepth,
OnNewVideo: onNew,
}
}
type Stats struct {
Scanned int
Added int
}
// Run 从 Drive.RootID 开始扫描
func (s *Scanner) Run(ctx context.Context, startDirID string) (Stats, error) {
if startDirID == "" {
startDirID = s.Drive.RootID()
}
stats := Stats{}
if err := s.walk(ctx, startDirID, "", 0, &stats); err != nil {
return stats, err
}
return stats, nil
}
func (s *Scanner) walk(ctx context.Context, dirID, dirName string, depth int, stats *Stats) error {
if depth >= s.MaxDepth {
return nil
}
if err := ctx.Err(); err != nil {
return err
}
entries, err := s.Drive.List(ctx, dirID)
if err != nil {
return fmt.Errorf("list %s: %w", dirID, err)
}
for _, e := range entries {
if e.IsDir {
// 跳过 previews 目录，避免扫到自己生成的 teaser
if strings.EqualFold(e.Name, "previews") {
continue
}
if err := s.walk(ctx, e.ID, e.Name, depth+1, stats); err != nil {
log.Printf("[scanner] walk %s error: %v", e.Name, err)
}
continue
}
stats.Scanned++
ext := strings.ToLower(path.Ext(e.Name))
if !s.Exts[ext] {
continue
}
id := s.Drive.Kind() + "-" + s.Drive.ID() + "-" + e.ID
existing, _ := s.Catalog.GetVideo(ctx, id)
if existing != nil {
// 已存在但 category 空缺时，顺便补 category
if existing.Category == "" && dirName != "" {
_ = s.Catalog.UpdateVideoMeta(ctx, id, catalog.VideoMetaPatch{Category: dirName})
}
continue
}
parsed := Parse(e.Name)
if parsed.Title == "" {
parsed.Title = strings.TrimSuffix(e.Name, ext)
}
now := time.Now()
v := &catalog.Video{
ID: id,
DriveID: s.Drive.ID(),
FileID: e.ID,
ParentID: e.ParentID,
Title: parsed.Title,
Author: parsed.Author,
Tags: parsed.Tags,
Ext: strings.TrimPrefix(ext, "."),
Quality: "HD",
Size: e.Size,
PreviewStatus: "pending",
Category: dirName,
PublishedAt: orDefault(e.ModTime, now),
CreatedAt: now,
UpdatedAt: now,
}
if err := s.Catalog.UpsertVideo(ctx, v); err != nil {
log.Printf("[scanner] upsert %s error: %v", v.Title, err)
continue
}
stats.Added++
if s.OnNewVideo != nil {
s.OnNewVideo(v)
}
}
return nil
}
func orDefault(t time.Time, d time.Time) time.Time {
if t.IsZero() {
return d
}
return t
}
backend/vendor/github.com/OpenListTeam/wopan-sdk-go/.gitattributes
Generated
+2
View File
@@ -0,0 +1,2 @@
# Auto detect text files and perform LF normalization
* text=auto
backend/vendor/github.com/OpenListTeam/wopan-sdk-go/.gitignore
Generated
+24
View File
@@ -0,0 +1,24 @@
# If you prefer the allow list template instead of the deny list, see community template:
# https://github.com/github/gitignore/blob/main/community/Golang/Go.AllowList.gitignore
#
# Binaries for programs and plugins
*.exe
*.exe~
*.dll
*.so
*.dylib
# Test binary, built with `go test -c`
*.test
# Output of the go coverage tool, specifically when used with LiteIDE
*.out
# Dependency directories (remove the comment below to include it)
# vendor/
# Go workspace file
go.work
# ide files
.idea/
backend/vendor/github.com/OpenListTeam/wopan-sdk-go/LICENSE
Generated Vendored
+201
View File
@@ -0,0 +1,201 @@
Apache License
Version 2.0, January 2004
http://www.apache.org/licenses/
TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
1. Definitions.
"License" shall mean the terms and conditions for use, reproduction,
and distribution as defined by Sections 1 through 9 of this document.
"Licensor" shall mean the copyright owner or entity authorized by
the copyright owner that is granting the License.
"Legal Entity" shall mean the union of the acting entity and all
other entities that control, are controlled by, or are under common
control with that entity. For the purposes of this definition,
"control" means (i) the power, direct or indirect, to cause the
direction or management of such entity, whether by contract or
otherwise, or (ii) ownership of fifty percent (50%) or more of the
outstanding shares, or (iii) beneficial ownership of such entity.
"You" (or "Your") shall mean an individual or Legal Entity
exercising permissions granted by this License.
"Source" form shall mean the preferred form for making modifications,
including but not limited to software source code, documentation
source, and configuration files.
"Object" form shall mean any form resulting from mechanical
transformation or translation of a Source form, including but
not limited to compiled object code, generated documentation,
and conversions to other media types.
"Work" shall mean the work of authorship, whether in Source or
Object form, made available under the License, as indicated by a
copyright notice that is included in or attached to the work
(an example is provided in the Appendix below).
"Derivative Works" shall mean any work, whether in Source or Object
form, that is based on (or derived from) the Work and for which the
editorial revisions, annotations, elaborations, or other modifications
represent, as a whole, an original work of authorship. For the purposes
of this License, Derivative Works shall not include works that remain
separable from, or merely link (or bind by name) to the interfaces of,
the Work and Derivative Works thereof.
"Contribution" shall mean any work of authorship, including
the original version of the Work and any modifications or additions
to that Work or Derivative Works thereof, that is intentionally
submitted to Licensor for inclusion in the Work by the copyright owner
or by an individual or Legal Entity authorized to submit on behalf of
the copyright owner. For the purposes of this definition, "submitted"
means any form of electronic, verbal, or written communication sent
to the Licensor or its representatives, including but not limited to
communication on electronic mailing lists, source code control systems,
and issue tracking systems that are managed by, or on behalf of, the
Licensor for the purpose of discussing and improving the Work, but
excluding communication that is conspicuously marked or otherwise
designated in writing by the copyright owner as "Not a Contribution."
"Contributor" shall mean Licensor and any individual or Legal Entity
on behalf of whom a Contribution has been received by Licensor and
subsequently incorporated within the Work.
2. Grant of Copyright License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
copyright license to reproduce, prepare Derivative Works of,
publicly display, publicly perform, sublicense, and distribute the
Work and such Derivative Works in Source or Object form.
3. Grant of Patent License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
(except as stated in this section) patent license to make, have made,
use, offer to sell, sell, import, and otherwise transfer the Work,
where such license applies only to those patent claims licensable
by such Contributor that are necessarily infringed by their
Contribution(s) alone or by combination of their Contribution(s)
with the Work to which such Contribution(s) was submitted. If You
institute patent litigation against any entity (including a
cross-claim or counterclaim in a lawsuit) alleging that the Work
or a Contribution incorporated within the Work constitutes direct
or contributory patent infringement, then any patent licenses
granted to You under this License for that Work shall terminate
as of the date such litigation is filed.
4. Redistribution. You may reproduce and distribute copies of the
Work or Derivative Works thereof in any medium, with or without
modifications, and in Source or Object form, provided that You
meet the following conditions:
(a) You must give any other recipients of the Work or
Derivative Works a copy of this License; and
(b) You must cause any modified files to carry prominent notices
stating that You changed the files; and
(c) You must retain, in the Source form of any Derivative Works
that You distribute, all copyright, patent, trademark, and
attribution notices from the Source form of the Work,
excluding those notices that do not pertain to any part of
the Derivative Works; and
(d) If the Work includes a "NOTICE" text file as part of its
distribution, then any Derivative Works that You distribute must
include a readable copy of the attribution notices contained
within such NOTICE file, excluding those notices that do not
pertain to any part of the Derivative Works, in at least one
of the following places: within a NOTICE text file distributed
as part of the Derivative Works; within the Source form or
documentation, if provided along with the Derivative Works; or,
within a display generated by the Derivative Works, if and
wherever such third-party notices normally appear. The contents
of the NOTICE file are for informational purposes only and
do not modify the License. You may add Your own attribution
notices within Derivative Works that You distribute, alongside
or as an addendum to the NOTICE text from the Work, provided
that such additional attribution notices cannot be construed
as modifying the License.
You may add Your own copyright statement to Your modifications and
may provide additional or different license terms and conditions
for use, reproduction, or distribution of Your modifications, or
for any such Derivative Works as a whole, provided Your use,
reproduction, and distribution of the Work otherwise complies with
the conditions stated in this License.
5. Submission of Contributions. Unless You explicitly state otherwise,
any Contribution intentionally submitted for inclusion in the Work
by You to the Licensor shall be under the terms and conditions of
this License, without any additional terms or conditions.
Notwithstanding the above, nothing herein shall supersede or modify
the terms of any separate license agreement you may have executed
with Licensor regarding such Contributions.
6. Trademarks. This License does not grant permission to use the trade
names, trademarks, service marks, or product names of the Licensor,
except as required for reasonable and customary use in describing the
origin of the Work and reproducing the content of the NOTICE file.
7. Disclaimer of Warranty. Unless required by applicable law or
agreed to in writing, Licensor provides the Work (and each
Contributor provides its Contributions) on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
implied, including, without limitation, any warranties or conditions
of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
PARTICULAR PURPOSE. You are solely responsible for determining the
appropriateness of using or redistributing the Work and assume any
risks associated with Your exercise of permissions under this License.
8. Limitation of Liability. In no event and under no legal theory,
whether in tort (including negligence), contract, or otherwise,
unless required by applicable law (such as deliberate and grossly
negligent acts) or agreed to in writing, shall any Contributor be
liable to You for damages, including any direct, indirect, special,
incidental, or consequential damages of any character arising as a
result of this License or out of the use or inability to use the
Work (including but not limited to damages for loss of goodwill,
work stoppage, computer failure or malfunction, or any and all
other commercial damages or losses), even if such Contributor
has been advised of the possibility of such damages.
9. Accepting Warranty or Additional Liability. While redistributing
the Work or Derivative Works thereof, You may choose to offer,
and charge a fee for, acceptance of support, warranty, indemnity,
or other liability obligations and/or rights consistent with this
License. However, in accepting such obligations, You may act only
on Your own behalf and on Your sole responsibility, not on behalf
of any other Contributor, and only if You agree to indemnify,
defend, and hold each Contributor harmless for any liability
incurred by, or claims asserted against, such Contributor by reason
of your accepting any such warranty or additional liability.
END OF TERMS AND CONDITIONS
APPENDIX: How to apply the Apache License to your work.
To apply the Apache License to your work, attach the following
boilerplate notice, with the fields enclosed by brackets "[]"
replaced with your own identifying information. (Don't include
the brackets!) The text should be enclosed in the appropriate
comment syntax for the file format. We also recommend that a
file or class name and description of purpose be included on the
same "printed page" as the copyright notice for easier
identification within third-party archives.
Copyright [yyyy] [name of copyright owner]
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
backend/vendor/github.com/OpenListTeam/wopan-sdk-go/README.md
Generated Vendored
+30
View File
@@ -0,0 +1,30 @@
# wopan-sdk-go
Wopan SDK for the Go programming language
## Installation
```bash
go get github.com/OpenListTeam/wopan-sdk-go
```
## Usage
```go
package main
import (
"fmt"
"github.com/OpenListTeam/wopan-sdk-go"
)
func main() {
w := wopan.DefaultWithRefreshToken("91d4b946-xxxx-4909-bac1-d9914e45f2de")
res, err := w.AppQueryUser()
if err != nil {
fmt.Printf("AppQueryUser() error = %v", err)
} else {
fmt.Printf("AppQueryUser() = %+v", res)
}
}
```
backend/vendor/github.com/OpenListTeam/wopan-sdk-go/aes.go
Generated Vendored
+74
View File
@@ -0,0 +1,74 @@
package wopan
import (
"bytes"
"crypto/aes"
"crypto/cipher"
"errors"
)
// pkcs7Padding 填充
func pkcs7Padding(data []byte, blockSize int) []byte {
//判断缺少几位长度。最少1，最多 blockSize
padding := blockSize - len(data)%blockSize
//补足位数。把切片[]byte{byte(padding)}复制padding个
padText := bytes.Repeat([]byte{byte(padding)}, padding)
return append(data, padText...)
}
// pkcs7UnPadding 填充的反向操作
func pkcs7UnPadding(data []byte) ([]byte, error) {
length := len(data)
if length == 0 {
return nil, errors.New("加密字符串错误！")
}
//获取填充的个数
unPadding := int(data[length-1])
return data[:(length - unPadding)], nil
}
// AesEncrypt 加密
func AesEncrypt(data []byte, key []byte, iv []byte) ([]byte, error) {
//创建加密实例
block, err := aes.NewCipher(key)
if err != nil {
return nil, err
}
//判断加密快的大小
blockSize := block.BlockSize()
//填充
encryptBytes := pkcs7Padding(data, blockSize)
//初始化加密数据接收切片
encrypted := make([]byte, len(encryptBytes))
//使用cbc加密模式
//blockMode := cipher.NewCBCEncrypter(block, key[:blockSize])
blockMode := cipher.NewCBCEncrypter(block, iv)
//执行加密
blockMode.CryptBlocks(encrypted, encryptBytes)
return encrypted, nil
}
// AesDecrypt 解密
func AesDecrypt(data []byte, key []byte, iv []byte) ([]byte, error) {
//创建实例
block, err := aes.NewCipher(key)
if err != nil {
return nil, err
}
//获取块的大小
//blockSize := block.BlockSize()
//使用cbc
//blockMode := cipher.NewCBCDecrypter(block, key[:blockSize])
blockMode := cipher.NewCBCDecrypter(block, iv)
//初始化解密数据接收切片
decrypted := make([]byte, len(data))
//执行解密
blockMode.CryptBlocks(decrypted, data)
//去除填充
decrypted, err = pkcs7UnPadding(decrypted)
//decrypted = pkcs7Padding(decrypted, blockSize)
if err != nil {
return nil, err
}
return decrypted, nil
}
backend/vendor/github.com/OpenListTeam/wopan-sdk-go/api-fs.go
Generated Vendored
+218
View File
@@ -0,0 +1,218 @@
package wopan
type File struct {
FamilyId int `json:"familyId"`
Fid string `json:"fid"`
Creator string `json:"creator"`
Size int64 `json:"size"`
CreateTime string `json:"createTime"`
Name string `json:"name"`
ShootingTime string `json:"shootingTime"`
Id string `json:"id"`
Type int `json:"type"`
ThumbUrl string `json:"thumbUrl"`
FileType string `json:"fileType"`
}
type QueryAllFilesData struct {
Files []*File `json:"files"`
}
func (w *WoClient) QueryAllFiles(spaceType, parentDirectoryId string, pageNum, pageSize int, sortRule int, familyId string, opts ...RestyOption) (*QueryAllFilesData, error) {
var resp QueryAllFilesData
param := Json{
"spaceType": spaceType,
"parentDirectoryId": parentDirectoryId,
"pageNum": pageNum,
"pageSize": pageSize,
"sortRule": sortRule,
"clientId": DefaultClientID,
}
if spaceType == SpaceTypeFamily {
param["familyId"] = familyId
}
if spaceType == SpaceTypePrivate {
if w.psToken == "" {
return nil, ErrInvalidPsToken
}
param["psToken"] = w.psToken
}
_, err := w.RequestWoHome(KeyQueryAllFiles, param, JsonSecret, &resp, opts...)
if err != nil {
return nil, err
}
return &resp, nil
}
func (w *WoClient) QueryAllFilesPersonal(parentDirectoryId string, pageNum, pageSize int, sortRule int, opts ...RestyOption) (*QueryAllFilesData, error) {
return w.QueryAllFiles(SpaceTypePersonal, parentDirectoryId, pageNum, pageSize, sortRule, "", opts...)
}
func (w *WoClient) QueryAllFilesFamily(parentDirectoryId string, pageNum, pageSize int, sortRule int, familyId string, opts ...RestyOption) (*QueryAllFilesData, error) {
return w.QueryAllFiles(SpaceTypeFamily, parentDirectoryId, pageNum, pageSize, sortRule, familyId, opts...)
}
// GetSearchDirectory??
type GetDownloadUrlV2Data struct {
Type int `json:"type"`
List []struct {
Fid string `json:"fid"`
DownloadUrl string `json:"downloadUrl"`
} `json:"list"`
}
func (w *WoClient) GetDownloadUrlV2(fidList []string, opts ...RestyOption) (*GetDownloadUrlV2Data, error) {
var resp GetDownloadUrlV2Data
param := Json{
"type": "1",
"fidList": fidList,
"clientId": DefaultClientID,
}
_, err := w.RequestWoHome(KeyGetDownloadUrlV2, param, JsonSecret, &resp, opts...)
if err != nil {
return nil, err
}
return &resp, nil
}
type GetDownloadUrlData struct {
Fid string `json:"fid"`
DownloadUrl string `json:"downloadUrl"`
}
func (w *WoClient) GetDownloadUrl(spaceType string, fidList []string, opts ...RestyOption) ([]GetDownloadUrlData, error) {
var resp []GetDownloadUrlData
param := Json{
"fidList": fidList,
"clientId": DefaultClientID, // ???
"spaceType": spaceType,
}
_, err := w.RequestWoHome(KeyGetDownloadUrl, param, JsonSecret, &resp, opts...)
if err != nil {
return nil, err
}
return resp, nil
}
type CreateDirectoryData struct {
Id string `json:"id"`
}
func (w *WoClient) CreateDirectory(spaceType, parentDirectoryId string, directoryName, familyId string, opts ...RestyOption) (*CreateDirectoryData, error) {
var resp CreateDirectoryData
param := Json{
"spaceType": spaceType,
"familyId": familyId,
"parentDirectoryId": parentDirectoryId,
"directoryName": directoryName,
"clientId": DefaultClientID,
}
if spaceType == SpaceTypePrivate {
if w.psToken == "" {
return nil, ErrInvalidPsToken
}
param["psToken"] = w.psToken
}
_, err := w.RequestWoHome(KeyCreateDirectory, param, JsonSecret, &resp, opts...)
if err != nil {
return nil, err
}
return &resp, nil
}
// RenameFileOrDirectory
// _type: 1: file, 0: directory
func (w *WoClient) RenameFileOrDirectory(spaceType string, _type int, id string, name string, familyId string, opts ...RestyOption) error {
fileType := "0"
if _type != 0 {
fileType = w.GetFileType(name)
}
param := Json{
"spaceType": spaceType,
"type": _type,
"fileType": fileType,
"id": id,
"name": name,
"clientId": DefaultClientID,
}
if spaceType == SpaceTypeFamily {
param["familyId"] = familyId
}
if spaceType == SpaceTypePrivate {
if w.psToken == "" {
return ErrInvalidPsToken
}
param["psToken"] = w.psToken
}
_, err := w.RequestWoHome(KeyRenameFileOrDirectory, param, JsonSecret, nil, opts...)
return err
}
func (w *WoClient) RenameFileOrDirectoryPersonal(_type int, id string, name string, opts ...RestyOption) error {
return w.RenameFileOrDirectory(SpaceTypePersonal, _type, id, name, "", opts...)
}
func (w *WoClient) RenameFileOrDirectoryFamily(_type int, id string, name string, familyId string, opts ...RestyOption) error {
return w.RenameFileOrDirectory(SpaceTypeFamily, _type, id, name, familyId, opts...)
}
func (w *WoClient) MoveFile(dirList, fileList []string, targetDirId string, sourceType, targetType string, fromFamilyId, targetFamilyId string, opts ...RestyOption) error {
param := Json{
"targetDirId": targetDirId,
"sourceType": sourceType,
"targetType": targetType,
"dirList": dirList,
"fileList": fileList,
"secret": false,
"clientId": DefaultClientID,
}
if sourceType == SpaceTypeFamily {
param["fromFamilyId"] = fromFamilyId
}
if targetType == SpaceTypeFamily {
param["familyId"] = targetFamilyId
}
_, err := w.RequestWoHome(KeyMoveFile, param, JsonSecret, nil, opts...)
return err
}
func (w *WoClient) CopyFile(dirList, fileList []string, targetDirId string, sourceType, targetType string, fromFamilyId, targetFamilyId string, opts ...RestyOption) error {
param := Json{
"targetDirId": targetDirId,
"sourceType": sourceType,
"targetType": targetType,
"dirList": dirList,
"fileList": fileList,
"secret": false,
"clientId": DefaultClientID,
}
if sourceType == SpaceTypeFamily {
param["fromFamilyId"] = fromFamilyId
}
if targetType == SpaceTypeFamily {
param["familyId"] = targetFamilyId
}
_, err := w.RequestWoHome(KeyCopyFile, param, JsonSecret, nil, opts...)
return err
}
func (w *WoClient) DeleteFile(spaceType string, dirList, fileList []string, opts ...RestyOption) error {
param := Json{
"spaceType": spaceType,
"vipLevel": "0",
"dirList": dirList,
"fileList": fileList,
"clientId": DefaultClientID,
}
_, err := w.RequestWoHome(KeyDeleteFile, param, JsonSecret, nil, opts...)
return err
}
func (w *WoClient) EmptyRecycleData(opts ...RestyOption) error {
param := Json{
"clientId": DefaultClientID,
}
_, err := w.RequestWoHome(KeyEmptyRecycleData, param, JsonSecret, nil, opts...)
return err
}
backend/vendor/github.com/OpenListTeam/wopan-sdk-go/api-user.go
Generated Vendored
+95
View File
@@ -0,0 +1,95 @@
package wopan
// PcWebLoginData no encrypt
type PcWebLoginData struct {
NeedSmsCode string `json:"needSmsCode"`
}
func (w *WoClient) PcWebLogin(phone, password string, opts ...RestyOption) (*PcWebLoginData, error) {
var resp PcWebLoginData
_, err := w.RequestApiUser(KeyPcWebLogin, Json{
"phone": phone,
"password": password,
"uuid": "",
"verifyCode": "",
"clientSecret": DefaultClientSecret,
}, JsonClientIDSecret, &resp, opts...)
if err != nil {
return nil, err
}
return &resp, nil
}
// PcLoginVerifyCodeData no encrypt
type PcLoginVerifyCodeData struct {
AccessToken string `json:"access_token"`
RefreshToken string `json:"refresh_token"`
ExpiresIn int `json:"expires_in"`
}
func (w *WoClient) PcLoginVerifyCode(phone, password, messageCode string, opts ...RestyOption) (*PcLoginVerifyCodeData, error) {
var resp PcLoginVerifyCodeData
_, err := w.RequestApiUser(KeyPcLoginVerifyCode, Json{
"phone": phone,
"messageCode": messageCode,
"verifyCode": nil,
"uuid": nil,
"clientSecret": DefaultClientSecret,
"password": password,
}, JsonClientIDSecret, &resp, opts...)
if err != nil {
return nil, err
}
return &resp, nil
}
type AppQueryUserData struct {
UserId string `json:"userId"`
HeadUrl string `json:"headUrl"`
UserName string `json:"userName"`
Sex string `json:"sex"`
Birthday string `json:"birthday"`
IsModify string `json:"isModify"`
IsHeadModify string `json:"isHeadModify"`
IsSetPassword string `json:"isSetPassword"`
RegisterTime string `json:"registerTime"`
}
func (w *WoClient) AppQueryUser(opts ...RestyOption) (*AppQueryUserData, error) {
var resp AppQueryUserData
_, err := w.RequestApiUser(KeyAppQueryUser, Json{
"accessToken": w.accessToken,
}, JsonClientIDSecret, &resp, opts...)
if err != nil {
return nil, err
}
return &resp, nil
}
// AppRefreshTokenData no encrypt
type AppRefreshTokenData struct {
AccessToken string `json:"access_token"`
TokenType string `json:"token_type"`
RefreshToken string `json:"refresh_token"`
ExpiresIn int `json:"expires_in"`
Scope string `json:"scope"`
}
func (w *WoClient) AppRefreshToken(opts ...RestyOption) (*AppRefreshTokenData, error) {
var resp AppRefreshTokenData
_, err := w.RequestApiUser(KeyAppRefreshToken, Json{
"refreshToken": w.refreshToken,
"clientSecret": DefaultClientSecret,
}, JsonClientIDSecret, &resp, opts...)
if err != nil {
return nil, err
}
return &resp, err
}
func (w *WoClient) AppLogout(opts ...RestyOption) error {
_, err := w.RequestApiUser(KeyAppLogout, Json{
"accessToken": w.accessToken,
}, JsonClientIDSecret, nil, opts...)
return err
}
backend/vendor/github.com/OpenListTeam/wopan-sdk-go/api-wohome.go
Generated Vendored
+202
View File
@@ -0,0 +1,202 @@
package wopan
type FCloudProductOrdListQryCtxData struct {
FcloudProductOrds []struct {
ActiviteCode string `json:"activiteCode"`
AppStorePackageDesc string `json:"appStorePackageDesc"`
AppStorePackageFee string `json:"appStorePackageFee"`
AppStoreProductId string `json:"appStoreProductId"`
ApplyTime string `json:"applyTime"`
ApplyTimeFormate string `json:"applyTimeFormate"`
CbssOrderId string `json:"cbssOrderId"`
City string `json:"city"`
ClientId string `json:"clientId"`
Days string `json:"days"`
DescUrl string `json:"descUrl"`
EffectState string `json:"effectState"`
EffectiveDays int `json:"effectiveDays"`
ExpireTime string `json:"expireTime"`
ExpireTimeFormate string `json:"expireTimeFormate"`
Fee string `json:"fee"`
IsAppStorePay string `json:"isAppStorePay"`
IsAutoSub string `json:"isAutoSub"`
IsExpire string `json:"isExpire"`
IsNewPackage string `json:"isNewPackage"`
IsOnline string `json:"isOnline"`
IsPlus string `json:"isPlus"`
IsShowExpireTips string `json:"isShowExpireTips"`
OrderId string `json:"orderId"`
OrderState string `json:"orderState"`
OrderStatus string `json:"orderStatus"`
PackageDesc string `json:"packageDesc"`
PackageProductCode string `json:"packageProductCode"`
PackageProductId string `json:"packageProductId"`
PayMethod string `json:"payMethod"`
PayTransactionId string `json:"payTransactionId"`
PayType string `json:"payType"`
Province string `json:"province"`
RemainDays string `json:"remainDays"`
SignStatus string `json:"signStatus"`
Source string `json:"source"`
SubTime string `json:"subTime"`
SubTimeFormate string `json:"subTimeFormate"`
SubType string `json:"subType"`
UserId string `json:"userId"`
VipDesc string `json:"vipDesc"`
VipDescNew string `json:"vipDescNew"`
VipExpireTimeLabel string `json:"vipExpireTimeLabel"`
VipLevel string `json:"vipLevel"`
} `json:"fcloudProductOrds"`
MaxVipLevel string `json:"maxVipLevel"`
IsShowInlet string `json:"isShowInlet"`
}
func (w *WoClient) FCloudProductOrdListQry(opts ...RestyOption) (*FCloudProductOrdListQryCtxData, error) {
var resp FCloudProductOrdListQryCtxData
_, err := w.RequestWoHome(KeyFCloudProductOrdListQry, nil, Json{
"qryType": "1",
"clientId": DefaultClientID,
}, &resp, opts...)
if err != nil {
return nil, err
}
return &resp, nil
}
type QueryCloudUsageInfoData struct {
Code string `json:"code"`
UsageInfo struct {
TotalSize string `json:"totalSize"`
UsedSize int64 `json:"usedSize"`
ImageSize int64 `json:"imageSize"`
VideoSize int64 `json:"videoSize"`
AudioSize int64 `json:"audioSize"`
TextSize int64 `json:"textSize"`
OtherSize int64 `json:"otherSize"`
ByteUsedSize int64 `json:"byteUsedSize"`
ByteTotalSize string `json:"byteTotalSize"`
} `json:"usageInfo"`
VipLevel string `json:"vipLevel"`
ExpireTime string `json:"expireTime"`
ApplyTime string `json:"applyTime"`
PayType string `json:"payType"`
Source string `json:"source"`
OrderState string `json:"orderState"`
Status string `json:"status"`
}
func (w *WoClient) QueryCloudUsageInfo(opts ...RestyOption) (*QueryCloudUsageInfoData, error) {
err := w.InitPhone()
if err != nil {
return nil, err
}
var resp QueryCloudUsageInfoData
_, err = w.RequestWoHome(KeyQueryCloudUsageInfo, Json{
"phoneNum": w.Phone,
"clientId": DefaultClientID,
}, JsonSecret, &resp, opts...)
if err != nil {
return nil, err
}
return &resp, nil
}
// FCloudProductPackage is not required
type ClassifyRuleData struct {
FileTypes map[string]struct {
SubType string `json:"subType"`
Ability string `json:"ability"`
Type string `json:"type"`
} `json:"fileTypes"`
FileIcons struct {
App struct {
Zip string `json:"zip"`
Image string `json:"image"`
Other string `json:"other"`
Xmind string `json:"xmind"`
Gif string `json:"gif"`
Csv string `json:"csv"`
Video string `json:"video"`
Excel string `json:"excel"`
Pdf string `json:"pdf"`
Ppt string `json:"ppt"`
Doc string `json:"doc"`
Audio string `json:"audio"`
Text string `json:"text"`
Word string `json:"word"`
} `json:"app"`
H5 struct {
} `json:"h5"`
} `json:"fileIcons"`
}
func (w *WoClient) ClassifyRule(opts ...RestyOption) (*ClassifyRuleData, error) {
var resp ClassifyRuleData
_, err := w.RequestWoHome(KeyClassifyRule, Json{}, Json{
"key": true,
}, &resp, opts...)
if err != nil {
return nil, err
}
return &resp, nil
}
type GetZoneInfoData struct {
Url string `json:"url"`
}
func (w *WoClient) GetZoneInfo(opts ...RestyOption) (*GetZoneInfoData, error) {
var resp GetZoneInfoData
_, err := w.RequestWoHome(KeyGetZoneInfo, Json{
"appId": DefaultAppID,
}, Json{
"key": true,
}, &resp, opts...)
if err != nil {
return nil, err
}
return &resp, nil
}
type FamilyUserCurrentEncodeData struct {
Count string `json:"count"`
DefaultHomeId int `json:"defaultHomeId"`
DefaultHomeName string `json:"defaultHomeName"`
GroupHeadUrl string `json:"groupHeadUrl"`
GroupName string `json:"groupName"`
Id int `json:"id"`
MemberRole string `json:"memberRole"`
OwnerId string `json:"owner Id"`
UnreadFlag string `json:"unreadFlag"`
}
func (w *WoClient) FamilyUserCurrentEncode(opts ...RestyOption) (*FamilyUserCurrentEncodeData, error) {
var resp FamilyUserCurrentEncodeData
_, err := w.RequestWoHome(KeyFamilyUserCurrentEncode, Json{
"clientId": DefaultClientID,
}, JsonSecret, &resp, opts...)
if err != nil {
return nil, err
}
return &resp, nil
}
type PrivateSpaceLoginData struct {
PsToken string `json:"psToken"`
IsPass string `json:"isPass"`
Desc string `json:"desc"`
}
func (w *WoClient) PrivateSpaceLogin(pwd string, opts ...RestyOption) error {
var resp PrivateSpaceLoginData
_, err := w.RequestWoHome(KeyPrivateSpaceLogin, Json{
"pwd": pwd,
"clientId": DefaultClientID,
}, JsonSecret, &resp, opts...)
if err != nil {
return err
}
w.SetPsToken(resp.PsToken)
return nil
}
backend/vendor/github.com/OpenListTeam/wopan-sdk-go/client.go
Generated Vendored
+143
View File
@@ -0,0 +1,143 @@
package wopan
import (
"encoding/json"
"net/http"
"path"
"sync"
"github.com/go-resty/resty/v2"
)
type WoClient struct {
accessToken string
refreshToken string
psToken string
client *resty.Client
crypto *Crypto
ua string
jsonMarshalFunc func(v interface{}) ([]byte, error)
jsonUnmarshalFunc func(data []byte, v interface{}) error
Phone string
ZoneURL string
zoneURLOnce sync.Once
ClassifyRuleData *ClassifyRuleData
onRefreshToken func(accessToken, refreshToken string)
}
func New(opts ...Option) *WoClient {
w := &WoClient{
client: resty.New(),
crypto: NewCrypto(),
jsonMarshalFunc: json.Marshal,
jsonUnmarshalFunc: json.Unmarshal,
}
for _, opt := range opts {
opt(w)
}
return w
}
func DefaultWithAccessToken(accessToken string) *WoClient {
w := Default()
w.SetAccessToken(accessToken)
return w
}
func DefaultWithRefreshToken(refreshToken string) *WoClient {
w := Default()
w.SetRefreshToken(refreshToken)
return w
}
func DefaultWithAccessAndPsToken(refreshToken, psToken string) *WoClient {
w := Default()
w.SetAccessToken(refreshToken)
w.SetPsToken(psToken)
return w
}
func Default() *WoClient {
return New(WithUA(DefaultUA))
}
func (w *WoClient) SetUA(ua string) {
w.ua = ua
}
func (w *WoClient) SetJsonMarshalFunc(f func(v interface{}) ([]byte, error)) {
w.jsonMarshalFunc = f
}
func (w *WoClient) SetJsonUnmarshalFunc(f func(data []byte, v interface{}) error) {
w.jsonUnmarshalFunc = f
}
func (w *WoClient) SetAccessToken(token string) {
w.accessToken = token
_ = w.crypto.SetAccessToken(token)
}
func (w *WoClient) SetRefreshToken(token string) {
w.refreshToken = token
}
func (w *WoClient) SetPsToken(psToken string) {
w.psToken = psToken
}
func (w *WoClient) GetToken() (string, string) {
return w.accessToken, w.refreshToken
}
func (w *WoClient) SetHttpClient(httpClient *http.Client) *WoClient {
w.client = resty.NewWithClient(httpClient)
return w
}
func (w *WoClient) SetUserAgent(userAgent string) *WoClient {
w.client.SetHeader("User-Agent", userAgent)
return w
}
func (w *WoClient) SetDebug(d bool) *WoClient {
w.client.SetDebug(d)
return w
}
func (w *WoClient) EnableTrace() *WoClient {
w.client.EnableTrace()
return w
}
func (w *WoClient) SetProxy(proxy string) *WoClient {
w.client.SetProxy(proxy)
return w
}
func (w *WoClient) NewRequest() *resty.Request {
return w.client.R()
}
func (w *WoClient) GetFileType(filename string) string {
ext := path.Ext(filename)
if ext == "" {
return "5"
}
ext = ext[1:]
err := w.InitClassifyRule()
if err != nil {
return "5"
}
if _type, ok := w.ClassifyRuleData.FileTypes[ext]; ok {
return _type.Type
}
return "5"
}
func (w *WoClient) OnRefreshToken(f func(accessToken, refreshToken string)) {
w.onRefreshToken = f
}
backend/vendor/github.com/OpenListTeam/wopan-sdk-go/consts.go
Generated Vendored
+69
View File
@@ -0,0 +1,69 @@
package wopan
const (
// app:
// 1001000035
// iELf0UL07o6I8eRK
DefaultClientID = "1001000021"
DefaultClientSecret = "XFmi9GS2hzk98jGX"
DefaultAppID = "10000001"
DefaultBaseURL = "https://panservice.mail.wo.cn"
// DefaultZoneURL old https://gxupload.pan.wo.cn:8443
DefaultZoneURL = "https://tjupload.pan.wo.cn"
DefaultUA = "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/114.0.0.0 Safari/537.36 Edg/114.0.1823.37"
DefaultPartSize = int64(8 * 1024 * 1024)
)
const (
ChannelAPIUser = "api-user"
ChannelWoHome = "wohome"
ChannelWoCloud = "wocloud"
)
const (
SpaceTypePersonal = "0"
SpaceTypeFamily = "1"
SpaceTypePrivate = "4"
)
// api-user methods
const (
KeyPcWebLogin = "PcWebLogin"
KeyPcLoginVerifyCode = "PcLoginVerifyCode"
KeyAppQueryUser = "AppQueryUser"
KeyAppRefreshToken = "AppRefreshToken"
KeyAppLogout = "AppLogout"
)
// wohome methods
const (
KeyFCloudProductOrdListQry = "FCloudProductOrdListQry"
KeyQueryCloudUsageInfo = "QueryCloudUsageInfo"
KeyFCloudProductPackage = "FCloudProductPackage"
KeyClassifyRule = "ClassifyRule"
KeyGetZoneInfo = "GetZoneInfo"
KeyQuerySysConfig = "QuerySysConfig"
KeyFamilyUserCurrentEncode = "FamilyUserCurrentEncode"
KeyQueryAllFiles = "QueryAllFiles"
KeyGetSearchDirectory = "GetSearchDirectory"
KeyGetDownloadUrlV2 = "GetDownloadUrlV2"
KeyGetDownloadUrl = "GetDownloadUrl"
KeyCreateDirectory = "CreateDirectory"
KeyRenameFileOrDirectory = "RenameFileOrDirectory"
KeyMoveFile = "MoveFile"
KeyCopyFile = "CopyFile"
KeyDeleteFile = "DeleteFile"
KeyEmptyRecycleData = "EmptyRecycleData"
KeyUpload2C = "upload2C"
KeyPrivateSpaceLogin = "PrivateSpaceLogin"
)
const (
SortNameAsc = iota + 1
SortNameDesc
SortSizeAsc
SortSizeDesc
SortTimeAsc
SortTimeDesc
)
backend/vendor/github.com/OpenListTeam/wopan-sdk-go/crypto.go
Generated Vendored
+73
View File
@@ -0,0 +1,73 @@
package wopan
import "encoding/base64"
type Crypto struct {
key []byte
iv []byte
accessKey []byte
}
func NewCrypto() *Crypto {
c := &Crypto{
key: []byte(DefaultClientSecret),
iv: []byte("wNSOYIB1k1DjY5lA"),
}
return c
}
func (c *Crypto) SetAccessToken(token string) error {
if len(token) < 16 {
return ErrInvalidAccessToken
}
c.accessKey = []byte(token[:16])
return nil
}
func (c *Crypto) EncryptBytes(bs []byte, channel string) (string, error) {
key := c.accessKey
if channel == ChannelAPIUser {
key = c.key
}
res, err := AesEncrypt(bs, key, c.iv)
if err != nil {
return "", err
}
return base64.StdEncoding.EncodeToString(res), nil
}
func (c *Crypto) Encrypt(content string, channel string) (string, error) {
return c.EncryptBytes([]byte(content), channel)
}
func (c *Crypto) Decrypt(content string, channel string) (string, error) {
bs, err := base64.StdEncoding.DecodeString(content)
if err != nil {
return "", err
}
key := c.accessKey
if channel == ChannelAPIUser {
key = c.key
}
res, err := AesDecrypt(bs, key, c.iv)
if err != nil {
return "", err
}
return string(res), nil
}
func (c *Crypto) UserEncrypt(content string) (string, error) {
return c.Encrypt(content, ChannelAPIUser)
}
func (c *Crypto) UserDecrypt(content string) (string, error) {
return c.Decrypt(content, ChannelAPIUser)
}
func (c *Crypto) WoHomeEncrypt(content string) (string, error) {
return c.Encrypt(content, ChannelWoHome)
}
func (c *Crypto) WoHomeDecrypt(content string) (string, error) {
return c.Decrypt(content, ChannelWoHome)
}
backend/vendor/github.com/OpenListTeam/wopan-sdk-go/header.go
Generated Vendored
+26
View File
@@ -0,0 +1,26 @@
package wopan
import (
"crypto/md5"
"encoding/hex"
"fmt"
"math/rand"
"time"
)
func calHeader(channel, key string) Header {
resTime := time.Now().UnixMilli()
reqSeq := rand.Int31n(8999) + 1e5
version := ""
m := md5.New()
m.Write([]byte(fmt.Sprintf("%s%d%d%s%s", key, resTime, reqSeq, channel, version)))
sign := hex.EncodeToString(m.Sum(nil))
return Header{
Key: key,
ResTime: resTime,
ReqSeq: int(reqSeq),
Channel: channel,
Sign: sign,
Version: version,
}
}
backend/vendor/github.com/OpenListTeam/wopan-sdk-go/operation.go
Generated Vendored
+65
View File
@@ -0,0 +1,65 @@
package wopan
func (w *WoClient) InitPhone() error {
if w.Phone == "" {
_resp, err := w.AppQueryUser()
if err != nil {
return err
}
w.Phone = _resp.UserId
}
return nil
}
func (w *WoClient) InitClassifyRule() error {
if w.ClassifyRuleData != nil {
return nil
}
data, err := w.ClassifyRule()
if err != nil {
return err
}
w.ClassifyRuleData = data
return nil
}
func (w *WoClient) InitZoneURL() error {
if w.ZoneURL != "" {
return nil
}
data, err := w.GetZoneInfo()
if err != nil {
return err
}
w.ZoneURL = data.Url
return nil
}
func (w *WoClient) RefreshToken() error {
resp, err := w.AppRefreshToken()
if err != nil {
return err
}
w.onRefreshToken(resp.AccessToken, resp.RefreshToken)
w.SetAccessToken(resp.AccessToken)
w.SetRefreshToken(resp.RefreshToken)
return nil
}
func (w *WoClient) InitData() error {
if w.accessToken == "" && w.refreshToken != "" {
if err := w.RefreshToken(); err != nil {
return err
}
}
if err := w.InitPhone(); err != nil {
return err
}
if err := w.InitClassifyRule(); err != nil {
return err
}
if err := w.InitZoneURL(); err != nil {
return err
}
return nil
}
backend/vendor/github.com/OpenListTeam/wopan-sdk-go/option.go
Generated Vendored
+77
View File
@@ -0,0 +1,77 @@
package wopan
import (
"net/http"
"github.com/go-resty/resty/v2"
)
type Option func(w *WoClient)
func WithAccessToken(token string) Option {
return func(w *WoClient) {
w.SetAccessToken(token)
}
}
func WithRefreshToken(token string) Option {
return func(w *WoClient) {
w.SetRefreshToken(token)
}
}
func WithPsToken(psToken string) Option {
return func(w *WoClient) {
w.SetPsToken(psToken)
}
}
func WithUA(ua string) Option {
return func(w *WoClient) {
w.SetUA(ua)
}
}
func WithJsonMarshalFunc(f func(v interface{}) ([]byte, error)) Option {
return func(w *WoClient) {
w.SetJsonMarshalFunc(f)
}
}
func WithJsonUnmarshalFunc(f func(data []byte, v interface{}) error) Option {
return func(w *WoClient) {
w.SetJsonUnmarshalFunc(f)
}
}
func WithClient(hc *http.Client) Option {
return func(c *WoClient) {
c.SetHttpClient(hc)
}
}
func WithRestyClient(rc *resty.Client) Option {
return func(c *WoClient) {
c.client = rc
}
}
func WithDebug() Option {
return func(c *WoClient) {
c.SetDebug(true)
}
}
func WithTrace() Option {
return func(c *WoClient) {
c.EnableTrace()
}
}
func WithProxy(proxy string) Option {
return func(c *WoClient) {
c.SetProxy(proxy)
}
}
type RestyOption func(request *resty.Request)
backend/vendor/github.com/OpenListTeam/wopan-sdk-go/req_body.go
Generated Vendored
+30
View File
@@ -0,0 +1,30 @@
package wopan
func (w *WoClient) EncryptParam(channel string, param Json) (string, error) {
if param == nil {
return "", nil
}
jsonBytes, err := w.jsonMarshalFunc(param)
if err != nil {
return "", err
}
encrypted, err := w.crypto.EncryptBytes(jsonBytes, channel)
if err != nil {
return "", err
}
return encrypted, nil
}
func (w *WoClient) NewBody(channel string, param, other Json) (Json, error) {
if param == nil {
return other, nil
}
encrypted, err := w.EncryptParam(channel, param)
if err != nil {
return nil, err
}
// copy other to avoid modifying the original and concurrent map writes
_other := copyJson(other)
_other["param"] = encrypted
return _other, nil
}
backend/vendor/github.com/OpenListTeam/wopan-sdk-go/request.go
Generated Vendored
+77
View File
@@ -0,0 +1,77 @@
package wopan
import (
"fmt"
"strings"
)
func (w *WoClient) request(channel string, key string, param, other Json, resp interface{}, retry bool, opts ...RestyOption) ([]byte, error) {
req := w.NewRequest()
req.SetHeaders(map[string]string{
"Origin": "https://pan.wo.cn",
"Referer": "https://pan.wo.cn/",
})
if w.accessToken != "" {
req.SetHeader("Accesstoken", w.accessToken)
}
header := calHeader(channel, key)
body, err := w.NewBody(channel, param, other)
if err != nil {
return nil, err
}
var _resp Resp
req.SetBody(Req[interface{}]{
Header: header,
Body: body,
}).SetResult(&_resp)
for _, opt := range opts {
opt(req)
}
res, err := req.Post(fmt.Sprintf("%s/%s/dispatcher", DefaultBaseURL, channel))
if err != nil {
return nil, err
}
if res.IsError() {
return res.Body(), fmt.Errorf("request failed with status: %s", res.Status())
}
if _resp.Status != "200" {
return res.Body(), fmt.Errorf("request failed with status: %s, msg: %s", _resp.Status, _resp.Msg)
}
if _resp.Rsp.RspCode != "0000" {
if channel != ChannelAPIUser && retry && _resp.Rsp.RspCode == "9999" {
err := w.RefreshToken()
if err != nil {
return res.Body(), err
}
return w.request(channel, key, param, other, resp, false, opts...)
}
return res.Body(), fmt.Errorf("request failed with rsp_code: %s,rep_desc: %s", _resp.Rsp.RspCode, _resp.Rsp.RspDesc)
}
if resp != nil {
data := string(_resp.Rsp.Data)
if strings.HasSuffix(data, "\"") && strings.HasPrefix(data, "\"") {
data, err = w.crypto.Decrypt(data[1:len(data)-1], channel)
if err != nil {
return res.Body(), err
}
}
err = w.jsonUnmarshalFunc([]byte(data), resp)
if err != nil {
return res.Body(), err
}
}
return res.Body(), nil
}
func (w *WoClient) Request(channel string, key string, param, other Json, resp interface{}, opts ...RestyOption) ([]byte, error) {
return w.request(channel, key, param, other, resp, true, opts...)
}
func (w *WoClient) RequestApiUser(key string, param, other Json, resp interface{}, opts ...RestyOption) ([]byte, error) {
return w.Request(ChannelAPIUser, key, param, other, resp, opts...)
}
func (w *WoClient) RequestWoHome(key string, param, other Json, resp interface{}, opts ...RestyOption) ([]byte, error) {
return w.Request(ChannelWoHome, key, param, other, resp, opts...)
}
backend/vendor/github.com/OpenListTeam/wopan-sdk-go/types.go
Generated Vendored
+38
View File
@@ -0,0 +1,38 @@
package wopan
import "encoding/json"
type Json map[string]interface{}
func copyJson(src Json) Json {
dst := make(Json, len(src))
for k, v := range src {
dst[k] = v
}
return dst
}
type Header struct {
Key string `json:"key"`
ResTime int64 `json:"resTime"`
ReqSeq int `json:"reqSeq"`
Channel string `json:"channel"`
Sign string `json:"sign"`
Version string `json:"version"`
}
type Req[T any] struct {
Header `json:"header"`
Body T `json:"body"`
}
type Resp struct {
Status string `json:"STATUS"`
Msg string `json:"MSG"`
LogID string `json:"LOGID"`
Rsp struct {
RspCode string `json:"RSP_CODE"`
RspDesc string `json:"RSP_DESC"`
Data json.RawMessage `json:"DATA"`
} `json:"RSP"`
}
backend/vendor/github.com/OpenListTeam/wopan-sdk-go/upload.go
Generated Vendored
+174
View File
@@ -0,0 +1,174 @@
package wopan
import (
"context"
"fmt"
"io"
"math/rand"
"os"
"strconv"
"time"
)
type Upload2COption struct {
OnProgress func(current, total int64)
Ctx context.Context
RetryTimes int
OnRetry func(err error, fileName string, partIndex int64, finishedSize int64)
}
type Upload2CFile struct {
Name string
Size int64
Content *os.File
ContentType string
}
type Upload2CResp struct {
Code string `json:"code"`
Data struct {
Fid string `json:"fid"`
} `json:"data"`
Msg string `json:"msg"`
}
func (w *WoClient) Upload2C(spaceType string, file Upload2CFile, targetDirId string, familyId string, opt Upload2COption) (string, error) {
zoneURL := DefaultZoneURL
w.zoneURLOnce.Do(func() {
_ = w.InitZoneURL()
})
if w.ZoneURL != "" {
zoneURL = w.ZoneURL
}
batchNo := time.Now().Format("20060102150405")
fileInfo := Json{
"spaceType": spaceType,
"directoryId": targetDirId,
"batchNo": batchNo,
"fileName": file.Name,
"fileSize": file.Size,
"fileType": w.GetFileType(file.Name),
}
if spaceType == SpaceTypeFamily {
fileInfo["familyId"] = familyId
}
if spaceType == SpaceTypePrivate {
if w.psToken == "" {
return "", ErrInvalidPsToken
}
fileInfo["psToken"] = w.psToken
}
fileInfoStr, err := w.EncryptParam(ChannelWoHome, fileInfo)
if err != nil {
return "", err
}
uploadURL := zoneURL + "/openapi/client/" + KeyUpload2C
totalPart := file.Size / DefaultPartSize
if totalPart == 0 {
totalPart = 1
}
formData := map[string]string{
"uniqueId": strconv.FormatInt(time.Now().UnixMilli(), 10) + "_" + randomChars(6),
"accessToken": w.accessToken,
"fileName": file.Name,
"psToken": "undefined",
"fileSize": strconv.FormatInt(file.Size, 10),
"totalPart": strconv.FormatInt(totalPart, 10),
"channel": ChannelWoCloud,
"directoryId": targetDirId,
"fileInfo": fileInfoStr,
//partSize: 8388608
//partIndex: 7
//file: (binary)
}
var resp Upload2CResp
var fid string
var finishedSize int64 = 0
for partIndex := int64(1); partIndex <= totalPart; partIndex++ {
if opt.Ctx != nil {
select {
case <-opt.Ctx.Done():
return "", opt.Ctx.Err()
default:
}
}
partSize := DefaultPartSize
if partIndex == totalPart {
partSize = file.Size - finishedSize
}
formData["partSize"] = strconv.FormatInt(partSize, 10)
formData["partIndex"] = strconv.FormatInt(partIndex, 10)
err := w.uploadPart(opt.Ctx, uploadURL, file, formData, partIndex, partSize, &resp)
if err != nil {
for i := 0; i < opt.RetryTimes; i++ {
if opt.OnRetry != nil {
opt.OnRetry(err, file.Name, partIndex, finishedSize)
}
_, serr := file.Content.Seek(finishedSize, 0)
if serr != nil {
return "", serr
}
err = w.uploadPart(opt.Ctx, uploadURL, file, formData, partIndex, partSize, &resp)
if err == nil {
break
}
}
if err != nil {
return "", err
}
}
if resp.Data.Fid != "" {
fid = resp.Data.Fid
}
finishedSize += partSize
if opt.OnProgress != nil {
opt.OnProgress(finishedSize, file.Size)
}
}
return fid, nil
}
func (w *WoClient) uploadPart(ctx context.Context, uploadURL string, file Upload2CFile, formData map[string]string, partIndex int64, partSize int64, resp *Upload2CResp) error {
req := w.NewRequest().
SetResult(resp).
ForceContentType("application/json;charset=UTF-8").
SetHeaders(map[string]string{
"Origin": "https://pan.wo.cn",
"Referer": "https://pan.wo.cn/",
"User-Agent": w.ua,
}).
SetMultipartFormData(formData).
SetMultipartField("file", file.Name, file.ContentType, io.LimitReader(file.Content, partSize))
if ctx != nil {
req.SetContext(ctx)
}
res, err := req.Post(uploadURL)
if err != nil {
return err
}
if res.IsError() {
return fmt.Errorf("partIndex: %d, failed to upload2C with http status: %d, body: %s", partIndex, res.StatusCode(), res.String())
}
if resp.Code != "0000" {
return fmt.Errorf("partIndex: %d, failed to upload2C with code: %s, msg: %s", partIndex, resp.Code, resp.Msg)
}
return nil
}
func (w *WoClient) Upload2CPersonal(file Upload2CFile, targetDirId string, opt Upload2COption) (string, error) {
return w.Upload2C(SpaceTypePersonal, file, targetDirId, "", opt)
}
func (w *WoClient) Upload2CFamily(file Upload2CFile, targetDirId string, familyId string, opt Upload2COption) (string, error) {
return w.Upload2C(SpaceTypeFamily, file, targetDirId, familyId, opt)
}
func randomChars(length int) string {
const charset = "abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ"
result := make([]byte, length)
for i := range result {
result[i] = charset[rand.Intn(len(charset))]
}
return string(result)
}
backend/vendor/github.com/OpenListTeam/wopan-sdk-go/vars.go
Generated Vendored
+18
View File
@@ -0,0 +1,18 @@
package wopan
import "errors"
var (
JsonClientIDSecret = Json{
"clientId": DefaultClientID,
"secret": true,
}
JsonSecret = Json{
"secret": true,
}
)
var (
ErrInvalidAccessToken = errors.New("invalid access token")
ErrInvalidPsToken = errors.New("invalid psToken")
)
backend/vendor/github.com/SheltonZhu/115driver/LICENSE
Generated Vendored
+45
View File
@@ -0,0 +1,45 @@
**SPECIAL NOTICE - POLICY STATEMENT**
While this software is licensed under MIT License (see below), I, SheltonZhu,
the copyright holder, explicitly oppose and refuse use of this software by:
- **AlistGo organization** (the official Alist maintainers)
- **Any entity controlled by or controlling AlistGo**
**Reason:** Alist's commercialization plans include collecting user hardware data (CPU, GPU,
memory, storage) without transparent consent. This practice violates user privacy and
principles of open-source software. See: https://github.com/AlistGo/alist/issues/9142
I understand that under the MIT License below, I cannot legally prevent the use of this
software. However, this notice serves as my explicit opposition to AlistGo's use of my work.
**I strongly advise against using 115driver in the official Alist maintained by AlistGo.**
**Note: This opposition does NOT apply to:**
- Community-maintained forks (e.g., OpenList)
- Other commercial users and organizations
- Any entity that respects user privacy and open-source principles
---
MIT License
Copyright (c) 2022-2024 SheltonZhu
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
backend/vendor/github.com/SheltonZhu/115driver/pkg/crypto/digest.go
Generated Vendored
+45
View File
@@ -0,0 +1,45 @@
package hash
import (
"crypto/md5"
"crypto/sha1"
"encoding/base64"
"encoding/hex"
"io"
"strings"
)
const (
hashPreSize = 128 * 1024
)
type DigestResult struct {
Size int64
PreID string
QuickID string
MD5 string
}
func Digest(r io.Reader, result *DigestResult) (err error) {
hs, hm := sha1.New(), md5.New()
w := io.MultiWriter(hs, hm)
// Calculate SHA1 hash of first 128K, which is used as PreID
result.Size, err = io.CopyN(w, r, hashPreSize)
if err != nil && err != io.EOF {
return
}
result.PreID = strings.ToUpper(hex.EncodeToString(hs.Sum(nil)))
// Write remain data.
if err == nil {
var n int64
if n, err = io.Copy(w, r); err != nil {
return
}
result.Size += n
result.QuickID = strings.ToUpper(hex.EncodeToString(hs.Sum(nil)))
} else {
result.QuickID = result.PreID
}
result.MD5 = base64.StdEncoding.EncodeToString(hm.Sum(nil))
return nil
}
backend/vendor/github.com/SheltonZhu/115driver/pkg/crypto/ec115/cipher.go
Generated Vendored
+201
View File
@@ -0,0 +1,201 @@
package ec115
import (
"crypto/aes"
"crypto/cipher"
"crypto/elliptic"
"crypto/rand"
"encoding/base64"
"encoding/binary"
"fmt"
"hash/crc32"
"math/big"
"github.com/aead/ecdh"
"github.com/andreburgaud/crypt2go/ecb"
"github.com/andreburgaud/crypt2go/padding"
"github.com/pierrec/lz4/v4"
)
var remotePubKey = []byte{
0x57, 0xA2, 0x92, 0x57, 0xCD, 0x23, 0x20, 0xE5,
0xD6, 0xD1, 0x43, 0x32, 0x2F, 0xA4, 0xBB, 0x8A,
0x3C, 0xF9, 0xD3, 0xCC, 0x62, 0x3E, 0xF5, 0xED,
0xAC, 0x62, 0xB7, 0x67, 0x8A, 0x89, 0xC9, 0x1A,
0x83, 0xBA, 0x80, 0x0D, 0x61, 0x29, 0xF5, 0x22,
0xD0, 0x34, 0xC8, 0x95, 0xDD, 0x24, 0x65, 0x24,
0x3A, 0xDD, 0xC2, 0x50, 0x95, 0x3B, 0xEE, 0xBA,
}
const (
p224BaseLen = 28
crcSalt = "^j>WD3Kr?J2gLFjD4W2y@"
)
// 利用key进行异或操作
func xor(src, key []byte) []byte {
secret := make([]byte, 0, len(src))
pad := len(src) % 4
if pad > 0 {
for i := 0; i < pad; i++ {
secret = append(secret, src[i]^key[i])
}
src = src[pad:]
}
keyLen := len(key)
num := 0
for _, s := range src {
if num >= keyLen {
num = num % keyLen
}
secret = append(secret, s^key[num])
num++
}
return secret
}
// EcdhCipher ECDH加密解密信息
type EcdhCipher struct {
key []byte
iv []byte
pubKey []byte
}
// NewEcdhCipher 新建EcdhCipher
func NewEcdhCipher() (*EcdhCipher, error) {
x := big.NewInt(0).SetBytes(remotePubKey[:p224BaseLen])
y := big.NewInt(0).SetBytes(remotePubKey[p224BaseLen:])
remotePublic := ecdh.Point{X: x, Y: y}
p224 := ecdh.Generic(elliptic.P224())
private, public, err := p224.GenerateKey(rand.Reader)
buf := make([]byte, p224BaseLen)
switch p := public.(type) {
case ecdh.Point:
p.X.FillBytes(buf)
if big.NewInt(0).And(p.Y, big.NewInt(1)).Cmp(big.NewInt(1)) == 0 {
buf = append([]byte{p224BaseLen + 1, 0x03}, buf...)
} else {
buf = append([]byte{p224BaseLen + 1, 0x02}, buf...)
}
default:
return nil, fmt.Errorf("错误的public key类型")
}
if err != nil {
return nil, err
}
secret := p224.ComputeSecret(private, remotePublic)
cipher := new(EcdhCipher)
cipher.key = secret[:aes.BlockSize]
cipher.iv = secret[len(secret)-aes.BlockSize:]
cipher.pubKey = buf
return cipher, nil
}
// Encrypt 加密
func (c *EcdhCipher) Encrypt(plainText []byte) ([]byte, error) {
pad := padding.NewPkcs7Padding(aes.BlockSize)
data, err := pad.Pad(plainText)
if err != nil {
return nil, err
}
cipherText := make([]byte, 0, len(data))
var xorKey []byte
xorKey = append(xorKey, c.iv...)
block, err := aes.NewCipher(c.key)
if err != nil {
return nil, err
}
mode := ecb.NewECBEncrypter(block)
tmp := make([]byte, 0, aes.BlockSize)
for i, b := range data {
tmp = append(tmp, b^xorKey[i%aes.BlockSize])
if i%aes.BlockSize == aes.BlockSize-1 {
mode.CryptBlocks(xorKey, tmp)
cipherText = append(cipherText, xorKey...)
tmp = make([]byte, 0, aes.BlockSize)
}
}
return cipherText, nil
}
// Decrypt 解密
func (c *EcdhCipher) Decrypt(cipherText []byte) (text []byte, e error) {
defer func() {
if err := recover(); err != nil {
e = fmt.Errorf("%v", err)
}
}()
cipherText = cipherText[0 : len(cipherText)-len(cipherText)%aes.BlockSize]
block, err := aes.NewCipher(c.key)
if err != nil {
return nil, err
}
lz4Block := make([]byte, len(cipherText))
mode := cipher.NewCBCDecrypter(block, c.iv)
mode.CryptBlocks(lz4Block, cipherText)
length := int(lz4Block[0]) + int(lz4Block[1])<<8
text = make([]byte, 0x2000)
l, err := lz4.UncompressBlock(lz4Block[2:length+2], text)
if err != nil {
return nil, err
}
return text[:l], nil
}
// EncodeToken 加密token
func (c *EcdhCipher) EncodeToken(timestamp int64) (string, error) {
random, err := rand.Int(rand.Reader, big.NewInt(256))
if err != nil {
return "", err
}
r1 := byte(random.Uint64())
random, err = rand.Int(rand.Reader, big.NewInt(256))
if err != nil {
return "", err
}
r2 := byte(random.Uint64())
tmp := make([]byte, 0, 48)
time := make([]byte, 4)
binary.BigEndian.PutUint32(time, uint32(timestamp))
for i := 0; i < 15; i++ {
tmp = append(tmp, c.pubKey[i]^r1)
}
tmp = append(tmp, []byte{r1, 0x73 ^ r1}...)
for i := 0; i < 3; i++ {
tmp = append(tmp, r1)
}
for i := 0; i < 4; i++ {
tmp = append(tmp, r1^time[3-i])
}
for i := 15; i < len(c.pubKey); i++ {
tmp = append(tmp, c.pubKey[i]^r2)
}
tmp = append(tmp, []byte{r2, 0x01 ^ r2}...)
for i := 0; i < 3; i++ {
tmp = append(tmp, r2)
}
crc := make([]byte, 4)
binary.BigEndian.PutUint32(crc, crc32.ChecksumIEEE(append([]byte(crcSalt), tmp...)))
for i := 0; i < 4; i++ {
tmp = append(tmp, crc[3-i])
}
return base64.StdEncoding.EncodeToString(tmp), nil
}
backend/vendor/github.com/SheltonZhu/115driver/pkg/crypto/m115/public.go
Generated Vendored
+47
View File
@@ -0,0 +1,47 @@
package m115
import (
"crypto/rand"
"encoding/base64"
"io"
)
type Key [16]byte
func GenerateKey() Key {
key := Key{}
_, _ = io.ReadFull(rand.Reader, key[:])
return key
}
func Encode(input []byte, key Key) (output string) {
// Prepare buffer
buf := make([]byte, 16+len(input))
// Copy key and data to buffer
copy(buf, key[:])
copy(buf[16:], input)
// XOR encode
xorTransform(buf[16:], xorDeriveKey(key[:], 4))
reverseBytes(buf[16:])
xorTransform(buf[16:], xorClientKey)
// Encrypt and encode
output = base64.StdEncoding.EncodeToString(rsaEncrypt(buf))
return
}
func Decode(input string, key Key) (output []byte, err error) {
// Base64 decode
data, err := base64.StdEncoding.DecodeString(input)
if err != nil {
return
}
// RSA decrypt
data = rsaDecrypt(data)
// XOR decode
output = make([]byte, len(data)-16)
copy(output, data[16:])
xorTransform(output, xorDeriveKey(data[:16], 12))
reverseBytes(output)
xorTransform(output, xorDeriveKey(key[:], 4))
return
}
backend/vendor/github.com/SheltonZhu/115driver/pkg/crypto/m115/rsa.go
Generated Vendored
+87
View File
@@ -0,0 +1,87 @@
package m115
import (
"bytes"
"crypto/rand"
"io"
"math/big"
)
var (
_N, _ = big.NewInt(0).SetString(
"8686980c0f5a24c4b9d43020cd2c22703ff3f450756529058b1cf88f09b86021"+
"36477198a6e2683149659bd122c33592fdb5ad47944ad1ea4d36c6b172aad633"+
"8c3bb6ac6227502d010993ac967d1aef00f0c8e038de2e4d3bc2ec368af2e9f1"+
"0a6f1eda4f7262f136420c07c331b871bf139f74f3010e3c4fe57df3afb71683", 16)
_E, _ = big.NewInt(0).SetString("10001", 16)
_KeyLength = _N.BitLen() / 8
)
func rsaEncrypt(input []byte) []byte {
buf := &bytes.Buffer{}
for remainSize := len(input); remainSize > 0; {
sliceSize := _KeyLength - 11
if sliceSize > remainSize {
sliceSize = remainSize
}
rsaEncryptSlice(input[:sliceSize], buf)
input = input[sliceSize:]
remainSize -= sliceSize
}
return buf.Bytes()
}
func rsaEncryptSlice(input []byte, w io.Writer) {
// Padding
padSize := _KeyLength - len(input) - 3
padData := make([]byte, padSize)
_, _ = rand.Read(padData)
// Prepare message
buf := make([]byte, _KeyLength)
buf[0], buf[1] = 0, 2
for i, b := range padData {
buf[2+i] = b%0xff + 0x01
}
buf[padSize+2] = 0
copy(buf[padSize+3:], input)
msg := big.NewInt(0).SetBytes(buf)
// RSA Encrypt
ret := big.NewInt(0).Exp(msg, _E, _N).Bytes()
// Fill zeros at beginning
if fillSize := _KeyLength - len(ret); fillSize > 0 {
zeros := make([]byte, fillSize)
_, _ = w.Write(zeros)
}
_, _ = w.Write(ret)
}
func rsaDecrypt(input []byte) []byte {
buf := &bytes.Buffer{}
for remainSize := len(input); remainSize > 0; {
sliceSize := _KeyLength
if sliceSize > remainSize {
sliceSize = remainSize
}
rsaDecryptSlice(input[:sliceSize], buf)
input = input[sliceSize:]
remainSize -= sliceSize
}
return buf.Bytes()
}
func rsaDecryptSlice(input []byte, w io.Writer) {
// RSA Decrypt
msg := big.NewInt(0).SetBytes(input)
ret := big.NewInt(0).Exp(msg, _E, _N).Bytes()
// Un-padding
for i, b := range ret {
// Find the beginning of plaintext
if b == 0 && i != 0 {
_, _ = w.Write(ret[i+1:])
break
}
}
}
backend/vendor/github.com/SheltonZhu/115driver/pkg/crypto/m115/util.go
Generated Vendored
+8
View File
@@ -0,0 +1,8 @@
package m115
// reverseBytes reverses data in place.
func reverseBytes(data []byte) {
for i, j := 0, len(data)-1; i < j; i, j = i+1, j-1 {
data[i], data[j] = data[j], data[i]
}
}
backend/vendor/github.com/SheltonZhu/115driver/pkg/crypto/m115/xor.go
Generated Vendored
+52
View File
@@ -0,0 +1,52 @@
package m115
var (
// Pre-calculated key data
xorKeySeed = []byte{
0xf0, 0xe5, 0x69, 0xae, 0xbf, 0xdc, 0xbf, 0x8a,
0x1a, 0x45, 0xe8, 0xbe, 0x7d, 0xa6, 0x73, 0xb8,
0xde, 0x8f, 0xe7, 0xc4, 0x45, 0xda, 0x86, 0xc4,
0x9b, 0x64, 0x8b, 0x14, 0x6a, 0xb4, 0xf1, 0xaa,
0x38, 0x01, 0x35, 0x9e, 0x26, 0x69, 0x2c, 0x86,
0x00, 0x6b, 0x4f, 0xa5, 0x36, 0x34, 0x62, 0xa6,
0x2a, 0x96, 0x68, 0x18, 0xf2, 0x4a, 0xfd, 0xbd,
0x6b, 0x97, 0x8f, 0x4d, 0x8f, 0x89, 0x13, 0xb7,
0x6c, 0x8e, 0x93, 0xed, 0x0e, 0x0d, 0x48, 0x3e,
0xd7, 0x2f, 0x88, 0xd8, 0xfe, 0xfe, 0x7e, 0x86,
0x50, 0x95, 0x4f, 0xd1, 0xeb, 0x83, 0x26, 0x34,
0xdb, 0x66, 0x7b, 0x9c, 0x7e, 0x9d, 0x7a, 0x81,
0x32, 0xea, 0xb6, 0x33, 0xde, 0x3a, 0xa9, 0x59,
0x34, 0x66, 0x3b, 0xaa, 0xba, 0x81, 0x60, 0x48,
0xb9, 0xd5, 0x81, 0x9c, 0xf8, 0x6c, 0x84, 0x77,
0xff, 0x54, 0x78, 0x26, 0x5f, 0xbe, 0xe8, 0x1e,
0x36, 0x9f, 0x34, 0x80, 0x5c, 0x45, 0x2c, 0x9b,
0x76, 0xd5, 0x1b, 0x8f, 0xcc, 0xc3, 0xb8, 0xf5,
}
xorClientKey = []byte{
0x78, 0x06, 0xad, 0x4c, 0x33, 0x86, 0x5d, 0x18,
0x4c, 0x01, 0x3f, 0x46,
}
)
func xorDeriveKey(seed []byte, size int) []byte {
key := make([]byte, size)
for i := 0; i < size; i++ {
key[i] = (seed[i] + xorKeySeed[size*i]) & 0xff
key[i] ^= xorKeySeed[size*(size-i-1)]
}
return key
}
func xorTransform(data []byte, key []byte) {
dataSize, keySize := len(data), len(key)
mod := dataSize % 4
if mod > 0 {
for i := 0; i < mod; i++ {
data[i] ^= key[i%keySize]
}
}
for i := mod; i < dataSize; i++ {
data[i] ^= key[(i-mod)%keySize]
}
}
backend/vendor/github.com/SheltonZhu/115driver/pkg/driver/api.go
Generated Vendored
+64
View File
@@ -0,0 +1,64 @@
package driver
const (
ApiGetVersion = "https://appversion.115.com/1/web/1.0/api/chrome"
// login
ApiLoginCheck = "https://passportapi.115.com/app/1.0/web/1.0/check/sso"
ApiUserInfo = "https://my.115.com/?ct=ajax&ac=nav"
ApiStatusCheck = "https://my.115.com/?ct=guide&ac=status"
// dir
ApiDirAdd = "https://webapi.115.com/files/add"
ApiDirName2CID = "https://webapi.115.com/files/getid"
// file
ApiFileDelete = "https://webapi.115.com/rb/delete"
ApiFileMove = "https://webapi.115.com/files/move"
ApiFileCopy = "https://webapi.115.com/files/copy"
ApiFileRename = "https://webapi.115.com/files/batch_rename"
ApiFileIndexInfo = "https://webapi.115.com/files/index_info"
ApiFileList = "https://webapi.115.com/files"
ApiFileList1 = "http://web.api.115.com/files"
// ApiFileList2 = "http://anxia.com/webapi/files"
// ApiFileList3 = "http://v.anxia.com/webapi/files"
ApiFileListByName = "https://aps.115.com/natsort/files.php"
ApiFileStat = "https://webapi.115.com/category/get"
ApiFileInfo = "https://webapi.115.com/files/get_info"
ApiFileSearch = "https://webapi.115.com/files/search"
// share
ApiShareSnap = "https://115cdn.com/webapi/share/snap"
// download
ApiDownloadGetUrl = "https://proapi.115.com/app/chrome/downurl"
ApiDownloadGetShareUrl = "https://115cdn.com/webapi/share/downurl"
AndroidApiDownloadGetUrl = "https://proapi.115.com/android/2.0/ufile/download"
// offline download
ApiAddOfflineUrl = "https://lixian.115.com/lixianssp/?ac=add_task_urls"
ApiDelOfflineUrl = "https://lixian.115.com/lixian/?ct=lixian&ac=task_del"
ApiListOfflineUrl = "https://lixian.115.com/lixian/?ct=lixian&ac=task_lists"
ApiClearOfflineUrl = "https://lixian.115.com/lixian/?ct=lixian&ac=task_clear"
// upload
ApiUploadInfo = "https://proapi.115.com/app/uploadinfo"
ApiGetUploadEndpoint = "https://uplb.115.com/3.0/getuploadinfo.php"
ApiUploadInit = "https://uplb.115.com/4.0/initupload.php"
// oss
ApiUploadOSSToken = "https://uplb.115.com/3.0/gettoken.php"
// qrcode
ApiQrcodeToken = "https://qrcodeapi.115.com/api/1.0/web/1.0/token"
ApiQrcodeStatus = "https://qrcodeapi.115.com/get/status/"
ApiQrcodeLogin = "https://passportapi.115.com/app/1.0/web/1.0/login/qrcode"
ApiQrcodeLoginWithApp = "https://passportapi.115.com/app/1.0/%s/1.0/login/qrcode"
ApiQrcodeImage = "https://qrcodeapi.115.com/api/1.0/mac/1.0/qrcode?uid=%s"
// recycle
ApiRecycleList = "https://webapi.115.com/rb"
ApiRecycleClean = "https://webapi.115.com/rb/clean"
ApiRecycleRevert = "https://webapi.115.com/rb/revert"
)
backend/vendor/github.com/SheltonZhu/115driver/pkg/driver/app_version.go
Generated Vendored
+52
View File
@@ -0,0 +1,52 @@
package driver
// GetAppVersion get app version (win, android, mac, mac_arc, etc...)
func (c *Pan115Client) GetAppVersion() ([]AppVersion, error) {
result := VersionResp{}
req := c.NewRequest().
SetResult(&result).
ForceContentType("application/json;charset=UTF-8")
resp, err := req.Get(ApiGetVersion)
err = CheckErr(err, &result, resp)
if err != nil {
return nil, err
}
return result.Data.GetAppVersions(), nil
}
type VersionResp struct {
BasicResp
ErrCode int `json:"err_code,omitempty"`
Data Versions `json:"data"`
}
type Versions map[string]map[string]any
func (v Versions) GetAppVersions() []AppVersion {
vers := make([]AppVersion, len(v))
for app, ver := range v {
vers = append(vers, AppVersion{
AppName: app,
Version: ver["version_code"].(string),
})
}
return vers
}
func (resp *VersionResp) Err(respBody ...string) error {
if resp.State {
return nil
}
if len(respBody) > 0 {
return GetErr(resp.ErrCode, respBody[0])
}
return GetErr(resp.ErrCode)
}
type AppVersion struct {
AppName string
Version string
}
backend/vendor/github.com/SheltonZhu/115driver/pkg/driver/client.go
Generated Vendored
+82
View File
@@ -0,0 +1,82 @@
package driver
import (
"net/http"
"github.com/go-resty/resty/v2"
)
// Pan115Client driver client
type Pan115Client struct {
Client *resty.Client
Request *resty.Request
UserID int64
Userkey string
UploadMetaInfo *UploadMetaInfo
UseInternalUpload bool
}
// New creates Client with customized options.
func New(opts ...Option) *Pan115Client {
c := &Pan115Client{
Client: resty.New(),
}
if len(opts) > 0 {
for _, optFunc := range opts {
optFunc(c)
}
}
return c
}
// Default creates an Client with default settings.
func Default() *Pan115Client {
return New(UA())
}
// Defalut is deprecated: use Default instead. This function exists for backward compatibility.
func Defalut() *Pan115Client {
return Default()
}
func (c *Pan115Client) SetHttpClient(httpClient *http.Client) *Pan115Client {
c.Client = resty.NewWithClient(httpClient)
return c
}
func (c *Pan115Client) SetUserAgent(userAgent string) *Pan115Client {
c.Client.SetHeader("User-Agent", userAgent)
return c
}
func (c *Pan115Client) SetCookies(cs ...*http.Cookie) *Pan115Client {
c.Client.SetCookies(cs)
return c
}
func (c *Pan115Client) SetDebug(d bool) *Pan115Client {
c.Client.SetDebug(d)
return c
}
func (c *Pan115Client) EnableTrace() *Pan115Client {
c.Client.EnableTrace()
return c
}
func (c *Pan115Client) SetProxy(proxy string) *Pan115Client {
c.Client.SetProxy(proxy)
return c
}
func (c *Pan115Client) NewRequest() *resty.Request {
c.Request = c.Client.R()
return c.Request
}
func (c *Pan115Client) GetRequest() *resty.Request {
if c.Request != nil {
return c.Request
}
return c.NewRequest()
}
backend/vendor/github.com/SheltonZhu/115driver/pkg/driver/consts.go
Generated Vendored
+35
View File
@@ -0,0 +1,35 @@
package driver
const (
UADefault = "Mozilla/5.0"
UADefalut = "Mozilla/5.0" // UADefalut is deprecated: use UADefault instead. This constant exists for backward compatibility
UA115Browser = "Mozilla/5.0 115Browser/27.0.5.7"
UA115Disk = "Mozilla/5.0 115disk/30.1.0"
UA115Desktop = "Mozilla/5.0 115Desktop/2.0.3.6"
UAIosApp = "Mozilla/5.0; Darwin/10.0; UDown/30.1.0"
)
const (
CookieDomain115 = ".115.com"
CookieUrl = "https://115.com"
CookieNameUid = "UID"
CookieNameCid = "CID"
CookieNameSeid = "SEID"
CookieNameKid = "KID"
)
const (
OSSRegionID = "oss-cn-shenzhen"
OSSEndpoint = "cn-shenzhen.oss.aliyuncs.com" // 双栈域名
OSSUserAgent = "aliyun-sdk-android/2.9.1"
OssSecurityTokenHeaderName = "X-OSS-Security-Token"
)
const (
KB = 1 << (10 * (iota + 1))
MB
GB
)
backend/vendor/github.com/SheltonZhu/115driver/pkg/driver/dir.go
Generated Vendored
+156
View File
@@ -0,0 +1,156 @@
package driver
import (
"strings"
"github.com/go-resty/resty/v2"
)
// Mkdir make a new directory which name and parent directory id, return directory id
func (c *Pan115Client) Mkdir(parentID string, name string) (string, error) {
result := MkdirResp{}
form := map[string]string{
"pid": parentID,
"cname": name,
}
req := c.NewRequest().
SetFormData(form).
SetResult(&result).
ForceContentType("application/json;charset=UTF-8")
resp, err := req.Post(ApiDirAdd)
err = CheckErr(err, &result, resp)
if err != nil {
return "", err
}
return string(result.CategoryID), nil
}
// List list all files and directories
func (c *Pan115Client) List(dirID string, opts ...ListOption) (*[]File, error) {
return c.ListWithLimit(dirID, FileListLimit, opts...)
}
const MaxDirPageLimit = 1150
// ListWithLimit list all files and directories with limit
func (c *Pan115Client) ListWithLimit(dirID string, limit int64, opts ...ListOption) (*[]File, error) {
if isCalledByAlistV3() {
return nil, ErrorNotSupportAlist
}
if limit > MaxDirPageLimit {
limit = MaxDirPageLimit
}
o := DefaultListOptions()
if len(opts) > 0 {
for _, opt := range opts {
opt(o)
}
}
apiURLs := o.ApiURLs
var files []File
offset := int64(0)
for i := 0; ; i++ {
apiURL := apiURLs[i%len(apiURLs)]
req := c.NewRequest().ForceContentType("application/json;charset=UTF-8")
getFilesOpts := []GetFileOptions{
WithApiURL(apiURL),
WithLimit(limit),
WithOffset(offset),
}
result, err := GetFiles(req, dirID, getFilesOpts...)
if err != nil {
return nil, err
}
for _, fileInfo := range result.Files {
files = append(files, *(&File{}).from(&fileInfo))
}
offset = int64(result.Offset) + limit
if offset >= int64(result.Count) {
break
}
}
return &files, nil
}
// ListPage list files and directories with page
func (c *Pan115Client) ListPage(dirID string, offset, limit int64, opts ...ListOption) (*[]File, error) {
o := DefaultListOptions()
if len(opts) > 0 {
for _, opt := range opts {
opt(o)
}
}
apiURLs := o.ApiURLs
var files []File
req := c.NewRequest().ForceContentType("application/json;charset=UTF-8")
getFilesOpts := []GetFileOptions{
WithApiURL(apiURLs[0]),
WithLimit(limit),
WithOffset(offset),
}
result, err := GetFiles(req, dirID, getFilesOpts...)
if err != nil {
return nil, err
}
if int64(result.Count) <= offset {
return &files, nil
}
for _, fileInfo := range result.Files {
files = append(files, *(&File{}).from(&fileInfo))
}
return &files, nil
}
func GetFiles(req *resty.Request, dirID string, opts ...GetFileOptions) (*FileListResp, error) {
if dirID == "" {
dirID = "0"
}
o := DefaultGetFileOptions()
if len(opts) > 0 {
for _, opt := range opts {
opt(o)
}
}
result := FileListResp{}
params := map[string]string{
"aid": "1",
"cid": dirID,
"o": o.GetOrder(),
"asc": o.GetAsc(),
"offset": o.GetOffset(),
"show_dir": o.GetshowDir(),
"limit": o.GetPageSize(),
"snap": "0",
"natsort": "0",
"record_open_time": "1",
"format": "json",
"fc_mix": "0",
}
req = req.SetQueryParams(params).
SetResult(&result)
resp, err := req.Get(o.GetApiURL())
if err = CheckErr(err, &result, resp); err != nil {
return &FileListResp{}, err
}
if dirID != string(result.CategoryID) {
return &FileListResp{}, err
}
return &result, err
}
func (c *Pan115Client) DirName2CID(dir string) (*APIGetDirIDResp, error) {
result := APIGetDirIDResp{}
dir = strings.TrimPrefix(dir, "/")
req := c.NewRequest().ForceContentType("application/json;charset=UTF-8")
req.SetQueryParam("path", dir).SetResult(&result)
resp, err := req.Get(ApiDirName2CID)
if err = CheckErr(err, &result, resp); err != nil {
return nil, err
}
return &result, err
}
backend/vendor/github.com/SheltonZhu/115driver/pkg/driver/download.go
Generated Vendored
+207
View File
@@ -0,0 +1,207 @@
package driver
import (
"bytes"
"encoding/json"
"io"
"net/http"
"strings"
crypto "github.com/SheltonZhu/115driver/pkg/crypto/m115"
"github.com/go-resty/resty/v2"
)
type FileDownloadUrl struct {
Client float64 `json:"client"`
OSSID string `json:"oss_id"`
Url string `json:"url"`
}
type DownloadInfo struct {
FileName string `json:"file_name"`
FileSize StringInt64 `json:"file_size"`
PickCode string `json:"pick_code"`
Url FileDownloadUrl `json:"url"`
Header http.Header
}
// Get Download file from download info url
func (info *DownloadInfo) Get() (io.ReadSeeker, error) {
req := resty.New().R().SetHeaderMultiValues(info.Header)
resp, err := req.Get(info.Url.Url)
if err != nil {
return nil, err
}
return bytes.NewReader(resp.Body()), nil
}
type DownloadData map[string]*DownloadInfo
// DownloadWithUA get download info with pickcode and user agent
func (c *Pan115Client) DownloadWithUA(pickCode, ua string) (*DownloadInfo, error) {
key := crypto.GenerateKey()
result := DownloadResp{}
params, err := json.Marshal(map[string]string{"pickcode": pickCode})
if err != nil {
return nil, err
}
data := crypto.Encode(params, key)
req := c.NewRequest().
SetQueryParam("t", Now().String()).
SetFormData(map[string]string{"data": data}).
ForceContentType("application/json").
SetResult(&result)
if len(ua) > 0 {
req = req.SetHeader("User-Agent", ua)
}
resp, err := req.Post(ApiDownloadGetUrl)
if err := CheckErr(err, &result, resp); err != nil {
return nil, err
}
bytes, err := crypto.Decode(string(result.EncodedData), key)
if err != nil {
return nil, err
}
downloadInfo := DownloadData{}
if err := json.Unmarshal(bytes, &downloadInfo); err != nil {
return nil, err
}
for _, info := range downloadInfo {
if info.FileSize < 0 {
return nil, ErrDownloadEmpty
}
info.Header = buildDownloadHeaders(resp.Request.Header, resp.Cookies())
return info, nil
}
return nil, ErrUnexpected
}
// DownloadWithUAByAndroidAPI get download info with pickcode and user agent
func (c *Pan115Client) DownloadWithUAByAndroidAPI(pickCode string, ua string) (*DownloadInfo, error) {
key := crypto.GenerateKey()
result := DownloadResp{}
params, err := json.Marshal(map[string]string{"pick_code": pickCode})
if err != nil {
return nil, err
}
data := crypto.Encode(params, key)
req := c.NewRequest().
SetQueryParam("t", Now().String()).
SetFormData(map[string]string{"data": data}).
ForceContentType("application/json").
SetResult(&result)
if len(ua) > 0 {
req = req.SetHeader("User-Agent", ua)
}
resp, err := req.Post(AndroidApiDownloadGetUrl)
if err := CheckErr(err, &result, resp); err != nil {
return nil, err
}
bytes, err := crypto.Decode(string(result.EncodedData), key)
if err != nil {
return nil, err
}
infoResp := struct {
URL string `json:"url"`
}{}
if err := json.Unmarshal(bytes, &infoResp); err != nil {
return nil, err
}
info := DownloadInfo{
Url: FileDownloadUrl{
Url: infoResp.URL,
},
PickCode: pickCode,
Header: buildDownloadHeaders(resp.Request.Header, resp.Cookies()),
}
return &info, nil
}
// Download get download info with pickcode
func (c *Pan115Client) Download(pickCode string) (*DownloadInfo, error) {
return c.DownloadWithUA(pickCode, "")
}
func buildDownloadHeaders(requestHeaders http.Header, responseCookies []*http.Cookie) http.Header {
headers := requestHeaders.Clone()
if len(responseCookies) == 0 {
return headers
}
cookies := make([]string, 0, len(responseCookies)+1)
if existing := strings.TrimSpace(headers.Get("Cookie")); existing != "" {
cookies = append(cookies, existing)
}
for _, cookie := range responseCookies {
if cookie == nil {
continue
}
cookies = append(cookies, cookie.String())
}
if len(cookies) > 0 {
headers.Set("Cookie", strings.Join(cookies, "; "))
}
return headers
}
type SharedDownloadInfo struct {
FileID string `json:"fid"`
FileName string `json:"fn"`
FileSize StringInt64 `json:"fs"`
URL struct {
URL string `json:"url"`
Client int `json:"client"`
Desc any `json:"desc"`
Isp any `json:"isp"`
OSSID string `json:"oss_id"`
OOID string `json:"ooid"`
} `json:"url"`
}
// DownloadByShareCode get download info with share code
func (c *Pan115Client) DownloadByShareCode(shareCode, receiveCode, fileID string) (*SharedDownloadInfo, error) {
return c.DownloadByShareCodeWithUA("", shareCode, receiveCode, fileID)
}
func (c *Pan115Client) DownloadByShareCodeWithUA(ua, shareCode, receiveCode, fileID string) (*SharedDownloadInfo, error) {
if isCalledByAlistV3() {
return nil, ErrorNotSupportAlist
}
result := DownloadShareResp{}
params := map[string]string{
"share_code": shareCode,
"receive_code": receiveCode,
"file_id": fileID,
"dl": "1",
}
req := c.NewRequest().
SetQueryParams(params).
ForceContentType("application/json").
SetHeader("referer", BuildShareReferer(shareCode, receiveCode)).
SetResult(&result)
if len(ua) > 0 {
req = req.SetHeader("User-Agent", ua)
}
resp, err := req.Get(ApiDownloadGetShareUrl)
if err := CheckErr(err, &result, resp); err != nil {
return nil, err
}
downloadInfo := result.Data
return &downloadInfo, nil
}
backend/vendor/github.com/SheltonZhu/115driver/pkg/driver/error.go
Generated Vendored
+163
View File
@@ -0,0 +1,163 @@
package driver
import (
"strconv"
"strings"
"github.com/go-resty/resty/v2"
"github.com/pkg/errors"
)
var (
ErrorNotSupportAlist = errors.New("not support alist due to privacy risk, please use openlist: https://github.com/OpenListTeam/OpenList")
)
// cookie err
var (
ErrBadCookie = errors.New("bad cookie")
)
var (
ErrNotLogin = errors.New("user not login")
ErrOfflineNoTimes = errors.New("offline download quota has been used up, you can purchase a VIP experience or upgrade to VIP service to get more quota")
ErrOfflineInvalidLink = errors.New("invalid download link")
ErrOfflineTaskExisted = errors.New("offline task existed")
ErrOrderNotSupport = errors.New("file order not supported")
ErrPasswordIncorrect = errors.New("password incorrect")
ErrLoginTwoStepVerify = errors.New("requires two-step verification")
ErrAccountNotBindMobile = errors.New("account not binds mobile")
ErrCredentialInvalid = errors.New("credential invalid")
ErrSessionExited = errors.New("session exited")
ErrQrcodeExpired = errors.New("qrcode expired")
// ErrUnexpected is the fall-back error whose code is not handled.
ErrUnexpected = errors.New("unexpected error")
// ErrExist means an item which you want to create is already existed.
ErrExist = errors.New("target already exists")
// ErrNotExist means an item which you find is not existed.
ErrNotExist = errors.New("target does not exist")
ErrInvalidCursor = errors.New("invalid cursor")
ErrUploadTooLarge = errors.New("upload reach the limit")
ErrUploadFailed = errors.New("upload failed")
ErrImportDirectory = errors.New("can not import directory")
ErrDownloadEmpty = errors.New("can not get download URL")
ErrDownloadDirectory = errors.New("can not download directory")
ErrDownloadFileNotExistOrHasDeleted = errors.New("target file does not exist or has deleted")
ErrDownloadFileTooBig = errors.New("target file is too big to download")
ErrCyclicCopy = errors.New("cyclic copy")
ErrCyclicMove = errors.New("cyclic move")
ErrVideoNotReady = errors.New("video is not ready")
ErrWrongParams = errors.New("wrong parameters")
ErrRepeatLogin = errors.New("repeat login")
ErrFailedToLogin = errors.New("failed to login")
ErrDoesLoggedOut = errors.New("you have been kicked out by multi-device login management")
ErrPickCodeNotExist = errors.New("pickcode does not exist")
ErrSharedInvalid = errors.New("shared link invalid")
ErrSharedNotFound = errors.New("shared link not found")
ErrPickCodeIsEmpty = errors.New("empty pickcode")
ErrUploadSH1Invalid = errors.New("userid/filesize/target/pickcode/ invalid")
ErrUploadSigInvalid = errors.New("sig invalid")
errMap = map[int]error{
// Normal errors
99: ErrNotLogin,
990001: ErrNotLogin,
// Offline errors
10010: ErrOfflineNoTimes,
10004: ErrOfflineInvalidLink,
10008: ErrOfflineTaskExisted,
// Dir errors
20004: ErrExist,
// Label errors
21003: ErrExist,
// File errors
20130827: ErrOrderNotSupport,
50028: ErrDownloadFileTooBig,
70005: ErrDownloadFileNotExistOrHasDeleted,
231011: ErrDownloadFileNotExistOrHasDeleted,
91002: ErrCyclicCopy,
800006: ErrCyclicMove,
// Login errors
40101009: ErrPasswordIncorrect,
40101010: ErrLoginTwoStepVerify,
40101017: ErrFailedToLogin,
40100000: ErrWrongParams,
40101030: ErrAccountNotBindMobile,
40101032: ErrCredentialInvalid,
40101033: ErrRepeatLogin,
40101035: ErrDoesLoggedOut,
40101037: ErrSessionExited,
40101038: ErrRepeatLogin,
// QRCode errors
40199002: ErrQrcodeExpired,
// Params errors
1001: ErrWrongParams,
200900: ErrWrongParams,
990002: ErrWrongParams,
// share
4100009: ErrSharedInvalid,
4100026: ErrSharedNotFound,
// pickCode
50003: ErrPickCodeNotExist,
50001: ErrPickCodeIsEmpty,
// upload SH1
402: ErrUploadSH1Invalid,
400: ErrUploadSigInvalid,
}
)
func GetErr(code int, respBody ...string) error {
errWithMsg := ErrUnexpected
if err, found := errMap[code]; found {
errWithMsg = err
}
// if len(respBody) > 0 && errors.Is(ErrUnexpected, errWithMsg) {
if len(respBody) > 0 {
bodyRaw := respBody[0]
readableBody, err := strconv.Unquote(strings.Replace(strconv.Quote(bodyRaw), `\\u`, `\u`, -1))
if err != nil {
return errors.Wrap(errWithMsg, bodyRaw)
}
return errors.Wrap(errWithMsg, readableBody)
}
return errWithMsg
}
type ResultWithErr interface {
Err(respBody ...string) error
}
func CheckErr(err error, result ResultWithErr, restyResp *resty.Response) error {
if err == nil {
err = result.Err(restyResp.String())
}
if err != nil {
return err
}
return nil
}
backend/vendor/github.com/SheltonZhu/115driver/pkg/driver/file.go
Generated Vendored
+109
View File
@@ -0,0 +1,109 @@
package driver
import (
"strconv"
"time"
)
type File struct {
// Marks is the file a directory.
IsDirectory bool
// Unique identifier of the file on the cloud storage.
FileID string
// FileID of the parent directory.
ParentID string
// Base name of the file.
Name string
// Size in bytes of the file.
Size int64
// IDentifier used for downloading or playing the file.
PickCode string
// SHA1 hash of file content, in HEX format.
Sha1 string
// Is file stared
Star bool
// File labels
Labels []*Label
// Create time of the file.
CreateTime time.Time
// Update time of the file.
UpdateTime time.Time
// Thumb URL of the file.
ThumbURL string
}
func (f *File) From(fileInfo *FileInfo) *File {
return f.from(fileInfo)
}
func (f *File) from(fileInfo *FileInfo) *File {
if fileInfo.FileID != "" {
f.FileID = fileInfo.FileID
f.ParentID = string(fileInfo.CategoryID)
f.IsDirectory = false
loc, err := time.LoadLocation("Asia/Shanghai") // updatetime is a string without timezone
if err != nil {
// if missing Asia/Shanghai use CST（UTC+8）
loc = time.FixedZone("UTC+8", 8*3600)
}
localTime, err := time.ParseInLocation("2006-01-02 15:04", fileInfo.UpdateTime, loc)
if err == nil {
f.UpdateTime = time.Unix(localTime.Unix(), 0)
}
f.ThumbURL = fileInfo.ThumbURL
} else {
f.FileID = string(fileInfo.CategoryID)
f.ParentID = fileInfo.ParentID
f.IsDirectory = true
t, err := strconv.ParseInt(fileInfo.UpdateTime, 10, 64)
if err == nil {
f.UpdateTime = time.Unix(t, 0)
}
}
f.Name = fileInfo.Name
f.Size = int64(fileInfo.Size)
f.PickCode = fileInfo.PickCode
f.Sha1 = fileInfo.Sha1
f.Star = fileInfo.IsStar != 0
f.Labels = make([]*Label, len(fileInfo.Labels))
for i, l := range fileInfo.Labels {
f.Labels[i] = &Label{
ID: l.ID,
Name: l.Name,
Color: LabelColor(LabelColorMap[l.Color]),
}
}
f.CreateTime = time.Unix(int64(fileInfo.CreateTime), 0)
return f
}
func (f File) GetPath() string {
return ""
}
func (f File) GetSize() int64 {
return f.Size
}
func (f File) GetName() string {
return f.Name
}
func (f File) ModTime() time.Time {
return f.UpdateTime
}
func (f File) IsDir() bool {
return f.IsDirectory
}
func (f File) GetID() string {
return f.FileID
}
backend/vendor/github.com/SheltonZhu/115driver/pkg/driver/info.go
Generated Vendored
+76
View File
@@ -0,0 +1,76 @@
package driver
// GetInfo get space info and login device info.
func (c *Pan115Client) GetInfo() (InfoData, error) {
result := InfoResponse{}
req := c.NewRequest().
SetResult(&result).
ForceContentType("application/json;charset=UTF-8")
resp, err := req.Get(ApiFileIndexInfo)
if err = CheckErr(err, &result, resp); err != nil {
return InfoData{}, err
}
return result.Data, nil
}
type InfoResponse struct {
BasicResp
Data InfoData `json:"data"`
}
type InfoData struct {
SpaceInfo SpaceInfo `json:"space_info"`
LoginDevicesInfo LoginDevicesInfo `json:"login_devices_info"`
ImeiInfo bool `json:"imei_info"`
}
type TotalSize struct {
Size int64 `json:"size"`
SizeFormat string `json:"size_format"`
}
type RemainSize struct {
Size int64 `json:"size"`
SizeFormat string `json:"size_format"`
}
type UseSize struct {
Size int64 `json:"size"`
SizeFormat string `json:"size_format"`
}
type SpaceInfo struct {
AllTotal TotalSize `json:"all_total"`
AllRemain RemainSize `json:"all_remain"`
AllUse UseSize `json:"all_use"`
}
type LastDevice struct {
IP string `json:"ip"`
Device string `json:"device"`
DeviceID string `json:"device_id"`
Network string `json:"network"`
Os string `json:"os"`
City string `json:"city"`
Utime int `json:"utime"`
}
type Device struct {
IsCurrent int `json:"is_current"`
Ssoent string `json:"ssoent"`
Utime int `json:"utime"`
Device string `json:"device"`
Name string `json:"name"`
Icon string `json:"icon"`
Desc string `json:"desc"`
IP string `json:"ip"`
City string `json:"city"`
IsUnusual int `json:"is_unusual"`
}
type LoginDevicesInfo struct {
Last LastDevice `json:"last"`
List []Device `json:"list"`
}
backend/vendor/github.com/SheltonZhu/115driver/pkg/driver/label.go
Generated Vendored
+41
View File
@@ -0,0 +1,41 @@
package driver
var (
LabelColors = []string{
// No Color
"#000000",
// Red
"#FF4B30",
// Orange
"#F78C26",
// Yellow
"#FFC032",
// Green
"#43BA80",
// Blue
"#2670FC",
// Purple
"#8B69FE",
// Gray
"#CCCCCC",
}
LabelColorMap = map[string]int{
"#000000": 0,
"#FF4B30": 1,
"#F78C26": 2,
"#FFC032": 3,
"#43BA80": 4,
"#2670FC": 5,
"#8B69FE": 6,
"#CCCCCC": 7,
}
)
type Label struct {
ID string
Name string
Color LabelColor
}
type LabelColor int
backend/vendor/github.com/SheltonZhu/115driver/pkg/driver/login.go
Generated Vendored
+134
View File
@@ -0,0 +1,134 @@
package driver
import (
"fmt"
"net/http"
neturl "net/url"
"strings"
"github.com/pkg/errors"
)
// CookieCheck checks the cookie status and will not logout of other devices.
func (c *Pan115Client) CookieCheck() error {
result := struct {
State bool `json:"state"`
}{}
req := c.NewRequest().
SetQueryParam("_", NowMilli().String()).
SetResult(&result)
if _, _ = req.Get(ApiStatusCheck); !result.State {
return ErrBadCookie
}
return nil
}
// LoginCheck checks the login status and will logout of other devices.
func (c *Pan115Client) LoginCheck() error {
result := LoginResp{}
req := c.NewRequest().
SetQueryParam("_", NowMilli().String()).
SetResult(&result)
resp, err := req.Get(ApiLoginCheck)
if err = CheckErr(err, &result, resp); err != nil {
return err
}
c.UserID = result.Data.UserID
return nil
}
// ImportCredential import uid, cid, seid
func (c *Pan115Client) ImportCredential(cr *Credential) *Pan115Client {
cookies := map[string]string{
CookieNameUid: cr.UID,
CookieNameCid: cr.CID,
CookieNameSeid: cr.SEID,
CookieNameKid: cr.KID,
}
c.ImportCookies(cookies, CookieDomain115)
return c
}
func (c *Pan115Client) ImportCookies(cookies map[string]string, domains ...string) {
for _, domain := range domains {
c.importCookies(cookies, domain, "/")
}
}
func (c *Pan115Client) importCookies(cookies map[string]string, domain string, path string) {
// Make a dummy URL for saving cookie
url := &neturl.URL{
Scheme: "https",
Path: "/",
}
if domain[0] == '.' {
url.Host = "www" + domain
} else {
url.Host = domain
}
// Prepare cookies
cks := make([]*http.Cookie, 0, len(cookies))
for name, value := range cookies {
cookie := &http.Cookie{
Name: name,
Value: value,
Domain: domain,
Path: path,
HttpOnly: true,
}
cks = append(cks, cookie)
}
// Save cookies
c.SetCookies(cks...)
}
type Credential struct {
UID string `json:"UID"`
CID string `json:"CID"`
SEID string `json:"SEID"`
KID string `json:"KID"`
}
// FromCookie get uid, cid, seid from cookie string
func (cr *Credential) FromCookie(cookie string) error {
items := strings.Split(cookie, ";")
if len(items) < 3 {
return errors.Wrap(ErrBadCookie, "number of cookie paris < 3")
}
cookieMap := map[string]string{}
for _, item := range items {
pairs := strings.Split(strings.TrimSpace(item), "=")
if len(pairs) != 2 {
return ErrBadCookie
}
key := pairs[0]
value := pairs[1]
cookieMap[strings.ToUpper(key)] = value
}
cr.UID = cookieMap["UID"]
cr.CID = cookieMap["CID"]
cr.SEID = cookieMap["SEID"]
cr.KID = cookieMap["KID"]
// No need to verify the KID for those old cookies that are still available.
if cr.CID == "" || cr.UID == "" || cr.SEID == "" {
return errors.Wrap(ErrBadCookie, "bad cookie, miss UID, CID or SEID")
}
return nil
}
// Cookie return cookie format
func (cr *Credential) Cookie() string {
return fmt.Sprintf("UID=%s;CID=%s;SEID=%s;KID=%s", cr.UID, cr.CID, cr.SEID, cr.KID)
}
// GetUser get user information
func (c *Pan115Client) GetUser() (*UserInfo, error) {
result := UserInfoResp{}
req := c.NewRequest().
SetQueryParam("_", Now().String()).
SetResult(&result)
resp, err := req.Get(ApiUserInfo)
return &result.UserInfo, CheckErr(err, &result, resp)
}
backend/vendor/github.com/SheltonZhu/115driver/pkg/driver/offline.go
Generated Vendored
+213
View File
@@ -0,0 +1,213 @@
package driver
import (
"encoding/json"
"fmt"
"net/url"
"strconv"
crypto "github.com/SheltonZhu/115driver/pkg/crypto/m115"
)
// OfflineTask describe an offline downloading task.
type OfflineTask struct {
InfoHash string `json:"info_hash"`
Name string `json:"name"`
Size int64 `json:"size"`
Url string `json:"url"`
AddTime int64 `json:"add_time"`
Peers int64 `json:"peers"`
RateDownload float64 `json:"rateDownload"`
Status int `json:"status"`
Percent float64 `json:"percentDone"`
UpdateTime int64 `json:"last_update"`
LeftTime int64 `json:"left_time"`
FileId string `json:"file_id"`
DelFileId string `json:"delete_file_id"`
DirId string `json:"wp_path_id"`
Move int `json:"move"`
}
func (t *OfflineTask) IsTodo() bool {
return t.Status == 0
}
func (t *OfflineTask) IsRunning() bool {
return t.Status == 1
}
func (t *OfflineTask) IsDone() bool {
return t.Status == 2
}
func (t *OfflineTask) IsFailed() bool {
return t.Status == -1
}
func (t *OfflineTask) GetStatus() string {
if t.IsTodo() {
return "准备开始离线下载"
}
if t.IsDone() {
return "离线下载完成"
}
if t.IsFailed() {
return "离线下载失败"
}
if t.IsRunning() {
return "离线任务下载中"
}
return fmt.Sprintf("未知状态: %d", t.Status)
}
// ListOfflineTask list tasks
func (c *Pan115Client) ListOfflineTask(page int64) (OfflineTaskResp, error) {
result := OfflineTaskResp{}
if isCalledByAlistV3() {
return result, ErrorNotSupportAlist
}
req := c.NewRequest().
SetQueryParam("page", strconv.FormatInt(page, 10)).
SetResult(&result).
ForceContentType("application/json;charset=UTF-8")
resp, err := req.Post(ApiListOfflineUrl)
if err := CheckErr(err, &result, resp); err != nil {
return OfflineTaskResp{}, err
}
return result, nil
}
// AddOfflineTaskURIs adds offline tasks by download URIs.
// supports http, ed2k, magent
func (c *Pan115Client) AddOfflineTaskURIs(uris []string, saveDirID string, opts ...OfflineOption) (hashes []string, err error) {
if isCalledByAlistV3() {
return nil, ErrorNotSupportAlist
}
opt := DefaultOfflineOptions()
for _, o := range opts {
o(&opt)
}
count := len(uris)
if count == 0 {
return
}
if c.UserID <= 0 {
userInfo, err := c.GetUser()
if err != nil {
return nil, err
}
c.UserID = userInfo.UserID
}
key := crypto.GenerateKey()
result := DownloadResp{}
params := map[string]string{
"ac": "add_task_urls",
"wp_path_id": saveDirID,
"app_ver": opt.appVer,
"uid": strconv.FormatInt(c.UserID, 10),
}
for i, uri := range uris {
key := fmt.Sprintf("url[%d]", i)
params[key] = uri
}
paramsBytes, err := json.Marshal(params)
if err != nil {
return nil, err
}
data := crypto.Encode(paramsBytes, key)
req := c.NewRequest().
SetQueryParam("t", Now().String()).
SetFormData(map[string]string{"data": data}).
ForceContentType("application/json").
SetResult(&result)
resp, err := req.Post(ApiAddOfflineUrl)
if err := CheckErr(err, &result, resp); err != nil {
return nil, err
}
bytes, err := crypto.Decode(string(result.EncodedData), key)
if err != nil {
return nil, err
}
taskInfos := OfflineAddUrlResponse{}
if err := json.Unmarshal(bytes, &taskInfos); err != nil {
return nil, err
}
hashes = make([]string, count)
for i, task := range taskInfos.Result {
hashes[i] = task.InfoHash
}
return hashes, nil
}
// DeleteOfflineTasks deletes tasks.
func (c *Pan115Client) DeleteOfflineTasks(hashes []string, deleteFiles bool) error {
if isCalledByAlistV3() {
return ErrorNotSupportAlist
}
form := url.Values{}
for _, hash := range hashes {
form.Add("hash", hash)
}
form.Set("flag", "0")
if deleteFiles {
form.Set("flag", "1")
}
result := MkdirResp{}
req := c.NewRequest().
SetFormDataFromValues(form).
SetResult(&result).
ForceContentType("application/json;charset=UTF-8")
resp, err := req.Post(ApiDelOfflineUrl)
return CheckErr(err, &result, resp)
}
// ClearOfflineTasks deletes tasks.
func (c *Pan115Client) ClearOfflineTasks(clearFlag int64) error {
form := url.Values{}
form.Set("flag", strconv.FormatInt(int64(clearFlag), 10))
result := MkdirResp{}
req := c.NewRequest().
SetFormDataFromValues(form).
SetResult(&result).
ForceContentType("application/json;charset=UTF-8")
resp, err := req.Post(ApiClearOfflineUrl)
return CheckErr(err, &result, resp)
}
type OfflineAddUrlResponse struct {
BasicResp
Result []OfflineTaskResponse `json:"result"`
}
type OfflineTaskResponse struct {
InfoHash string `json:"info_hash"`
Url string `json:"url"`
}
type OfflineTaskResp struct {
BasicResp
Total int64 `json:"total"`
Count int64 `json:"count"`
PageRow int64 `json:"page_row"`
PageCount int64 `json:"page_count"`
Page int64 `json:"page"`
Quota int64 `json:"quota"`
Tasks []*OfflineTask `json:"tasks"`
}
backend/vendor/github.com/SheltonZhu/115driver/pkg/driver/op.go
Generated Vendored
+184
View File
@@ -0,0 +1,184 @@
package driver
import (
"fmt"
"strconv"
"time"
)
// Delete delete files or directory from file ids
func (c *Pan115Client) Delete(fileIDs ...string) error {
if len(fileIDs) == 0 {
return nil
}
form := map[string]string{}
for i, value := range fileIDs {
key := fmt.Sprintf("%s[%d]", "fid", i)
form[key] = value
}
result := BasicResp{}
req := c.NewRequest().
SetFormData(form).
ForceContentType("application/json;charset=UTF-8").
SetResult(&result)
resp, err := req.Post(ApiFileDelete)
return CheckErr(err, &result, resp)
}
// Rename rename a file or directory with file id and name
func (c *Pan115Client) Rename(fileID, newName string) error {
if isCalledByAlistV3() {
return ErrorNotSupportAlist
}
form := map[string]string{
"fid": fileID,
"file_name": newName,
fmt.Sprintf("files_new_name[%s]", fileID): newName,
}
result := BasicResp{}
req := c.NewRequest().
SetFormData(form).
ForceContentType("application/json;charset=UTF-8").
SetResult(&result)
resp, err := req.Post(ApiFileRename)
return CheckErr(err, &result, resp)
}
// Move move files or directory into another directory with directroy id
func (c *Pan115Client) Move(dirID string, fileIDs ...string) error {
if isCalledByAlistV3() {
return ErrorNotSupportAlist
}
if len(fileIDs) == 0 {
return nil
}
form := map[string]string{
"pid": dirID,
}
for i, value := range fileIDs {
key := fmt.Sprintf("%s[%d]", "fid", i)
form[key] = value
}
result := BasicResp{}
req := c.NewRequest().
SetFormData(form).
ForceContentType("application/json;charset=UTF-8").
SetResult(&result)
resp, err := req.Post(ApiFileMove)
return CheckErr(err, &result, resp)
}
// Copy copy files or directory into another directory with directroy id
func (c *Pan115Client) Copy(dirID string, fileIDs ...string) error {
if isCalledByAlistV3() {
return ErrorNotSupportAlist
}
if len(fileIDs) == 0 {
return nil
}
form := map[string]string{
"pid": dirID,
}
for i, value := range fileIDs {
key := fmt.Sprintf("%s[%d]", "fid", i)
form[key] = value
}
result := BasicResp{}
req := c.NewRequest().
SetFormData(form).
ForceContentType("application/json;charset=UTF-8").
SetResult(&result)
resp, err := req.Post(ApiFileCopy)
return CheckErr(err, &result, resp)
}
type FileStatInfo struct {
// Base name of the file.
Name string
// Identifier used for downloading or playing the file.
PickCode string
// SHA1 hash of file content, in HEX format.
Sha1 string
// Marks is file a directory.
IsDirectory bool
// Files count under this directory.
FileCount int
// Subdirectories count under this directory.
DirCount int
// Create time of the file.
CreateTime time.Time
// Last update time of the file.
UpdateTime time.Time
// Last access time of the file.
// AccessTime time.Time
// Parent directory list.
Parents []*DirInfo
}
// DirInfo only used in FileInfo.
type DirInfo struct {
// Directory ID.
ID string
// Directory Name.
Name string
}
// Stat get statistic information of a file or directory
func (c *Pan115Client) Stat(fileID string) (*FileStatInfo, error) {
result := FileStatResponse{}
req := c.NewRequest().
SetQueryParam("cid", fileID).
ForceContentType("application/json;charset=UTF-8").
SetResult(&result)
resp, err := req.Get(ApiFileStat)
if err := CheckErr(err, &result, resp); err != nil {
return nil, err
}
info := &FileStatInfo{
Name: result.FileName,
PickCode: result.PickCode,
Sha1: result.Sha1,
CreateTime: time.Unix(int64(result.CreateTime), 0),
UpdateTime: time.Unix(int64(result.UpdateTime), 0),
// AccessTime: time.Unix(result.AccessTime, 0),
}
// Fill parents
info.Parents = make([]*DirInfo, len(result.Paths))
for i, path := range result.Paths {
info.Parents[i] = &DirInfo{
ID: strconv.Itoa(path.FileID),
Name: path.FileName,
}
}
// Directory info
info.IsDirectory = result.IsFile == 0
if info.IsDirectory {
info.FileCount = int(result.FileCount)
info.DirCount = int(result.FolderCount)
}
return info, nil
}
// GetFile gets information of a file or directory by its ID.
func (c *Pan115Client) GetFile(fileID string) (*File, error) {
result := GetFileInfoResponse{}
req := c.NewRequest().
SetQueryParam("file_id", fileID).
ForceContentType("application/json;charset=UTF-8").
SetResult(&result)
resp, err := req.Get(ApiFileInfo)
if err := CheckErr(err, &result, resp); err != nil {
return nil, err
}
fileInfo := &FileInfo{}
if len(result.Files) > 0 {
fileInfo = result.Files[0]
}
f := &File{}
f.from(fileInfo)
return f, nil
}
backend/vendor/github.com/SheltonZhu/115driver/pkg/driver/option.go
Generated Vendored
+244
View File
@@ -0,0 +1,244 @@
package driver
import (
"crypto/tls"
"net/http"
"strconv"
"time"
"github.com/go-resty/resty/v2"
)
// Option driver client options
type Option func(c *Pan115Client)
func UA(userAgent ...string) Option {
return func(c *Pan115Client) {
if len(userAgent) > 0 {
c.SetUserAgent(userAgent[0])
} else {
c.SetUserAgent(UADefault)
}
}
}
func WithClient(hc *http.Client) Option {
return func(c *Pan115Client) {
c.SetHttpClient(hc)
}
}
func WithRestyClient(resty *resty.Client) Option {
return func(c *Pan115Client) {
c.Client = resty
}
}
func WithDebug() Option {
return func(c *Pan115Client) {
c.SetDebug(true)
}
}
func WithTrace() Option {
return func(c *Pan115Client) {
c.EnableTrace()
}
}
func WithProxy(proxy string) Option {
return func(c *Pan115Client) {
c.SetProxy(proxy)
}
}
func InsecureSkipVerify(insecureSkipVerify bool) Option {
return func(c *Pan115Client) {
c.Client.SetTLSClientConfig(&tls.Config{InsecureSkipVerify: insecureSkipVerify})
}
}
const (
FileOrderByTime = "user_ptime"
FileOrderByType = "file_type"
FileOrderBySize = "file_size"
FileOrderByName = "file_name"
FileListLimit = int64(56)
)
// GetFileOption get file options
type GetFileOption struct {
order string
asc string
pageSize int64
offset int64
showDir string
apiURL string
}
type GetFileOptions func(o *GetFileOption)
func WithApiURL(url string) GetFileOptions {
return func(o *GetFileOption) {
o.apiURL = url
}
}
func WithLimit(pageSize int64) GetFileOptions {
return func(o *GetFileOption) {
o.pageSize = pageSize
}
}
func WithOffset(offset int64) GetFileOptions {
return func(o *GetFileOption) {
o.offset = offset
}
}
func WithOrder(order string) GetFileOptions {
return func(o *GetFileOption) {
o.order = order
}
}
func WithShowDirEnable(e bool) GetFileOptions {
return func(o *GetFileOption) {
o.showDir = "0"
if e {
o.showDir = "1"
}
}
}
func WithAsc(d bool) GetFileOptions {
return func(o *GetFileOption) {
o.showDir = "0"
if d {
o.showDir = "1"
}
}
}
func (o *GetFileOption) GetApiURL() string {
return o.apiURL
}
func (o *GetFileOption) GetOrder() string {
return o.order
}
func (o *GetFileOption) GetAsc() string {
return o.asc
}
func (o *GetFileOption) GetPageSize() string {
return strconv.FormatInt(o.pageSize, 10)
}
func (o *GetFileOption) GetOffset() string {
return strconv.FormatInt(o.offset, 10)
}
func (o *GetFileOption) GetshowDir() string {
return o.showDir
}
func DefaultGetFileOptions() *GetFileOption {
return &GetFileOption{
order: FileOrderByTime,
asc: "1",
pageSize: int64(56),
offset: int64(0),
showDir: "1",
apiURL: ApiFileList,
}
}
type UploadMultipartOptions struct {
ThreadsNum int
Timeout time.Duration
TokenRefreshTime time.Duration
}
// DefalutUploadMultipartOptions is deprecated: use DefaultUploadMultipartOptions instead. This function exists for backward compatibility.
func DefalutUploadMultipartOptions() *UploadMultipartOptions {
return DefaultUploadMultipartOptions()
}
func DefaultUploadMultipartOptions() *UploadMultipartOptions {
return &UploadMultipartOptions{
// oss 启用Sequential必须按顺序上传
ThreadsNum: 1,
Timeout: time.Hour * 24,
TokenRefreshTime: time.Minute * 50,
}
}
type UploadMultipartOption func(o *UploadMultipartOptions)
func UploadMultipartWithThreadsNum(n int) UploadMultipartOption {
return func(o *UploadMultipartOptions) {
o.ThreadsNum = n
}
}
func UploadMultipartWithTimeout(timeout time.Duration) UploadMultipartOption {
return func(o *UploadMultipartOptions) {
o.Timeout = timeout
}
}
func UploadMultipartWithTokenRefreshTime(refreshTime time.Duration) UploadMultipartOption {
return func(o *UploadMultipartOptions) {
o.TokenRefreshTime = refreshTime
}
}
type ListOptions struct {
ApiURLs []string
}
func DefaultListOptions() *ListOptions {
return &ListOptions{
ApiURLs: []string{ApiFileList},
}
}
type ListOption func(o *ListOptions)
func WithApiURLs(urls ...string) ListOption {
return func(o *ListOptions) {
if len(urls) > 0 {
o.ApiURLs = urls
}
}
}
func WithMultiUrls() ListOption {
return WithApiURLs([]string{
ApiFileList,
ApiFileList1,
// ApiFileList2,
// ApiFileList3,
}...)
}
type OfflineOptions struct {
appVer string
}
func DefaultOfflineOptions() OfflineOptions {
return OfflineOptions{
appVer: appVer,
}
}
type OfflineOption func(o *OfflineOptions)
func WithAppVer(appVer string) OfflineOption {
return func(o *OfflineOptions) {
o.appVer = appVer
}
}
backend/vendor/github.com/SheltonZhu/115driver/pkg/driver/qrcode.go
Generated Vendored
+137
View File
@@ -0,0 +1,137 @@
package driver
import (
"fmt"
"strconv"
"github.com/go-resty/resty/v2"
qrcode "github.com/skip2/go-qrcode"
)
type QRCodeSession struct {
// The raw data of QRCode, caller should use third-party tools/libraries
// to convert it into QRCode matrix or image.
QrcodeContent string `json:"qrcode"`
Sign string `json:"sign"`
Time int64 `json:"time"`
UID string `json:"uid"`
}
// QRCode get QRCode matrix or image.
func (s *QRCodeSession) QRCode() ([]byte, error) {
return qrcode.Encode(s.QrcodeContent, qrcode.Medium, 256)
}
// QRCodeByApi get QRCode matrix or image by api.
func (s *QRCodeSession) QRCodeByApi() ([]byte, error) {
resp, err := resty.New().R().Get(fmt.Sprintf(ApiQrcodeImage, s.UID))
return resp.Body(), err
}
// QRCodeStart starts a QRCode login session.
func (c *Pan115Client) QRCodeStart() (*QRCodeSession, error) {
result := QRCodeTokenResp{}
resp, err := c.NewRequest().
SetResult(&result).
ForceContentType("application/json;charset=UTF-8").
Get(ApiQrcodeToken)
if err = CheckErr(err, &result, resp); err != nil {
return nil, err
}
return &result.Data, nil
}
// QRCodeLogin logins user through QRCode with web app.
// You SHOULD call this method ONLY when `QRCodeStatus.IsAllowed()` is true.
func (c *Pan115Client) QRCodeLogin(s *QRCodeSession) (*Credential, error) {
return c.QRCodeLoginWithApp(s, LoginAppWeb)
}
type LoginApp string
const (
LoginAppWeb LoginApp = "web"
LoginAppAndroid LoginApp = "android"
LoginAppIOS LoginApp = "ios"
// LoginAppLinux LoginApp = "linux" // disabled
// LoginAppMac LoginApp = "mac" // disabled
// LoginAppWindows LoginApp = "windows" // disabled
LoginAppTV LoginApp = "tv"
LoginAppAlipayMini LoginApp = "alipaymini"
LoginAppWechatMini LoginApp = "wechatmini"
LoginQAppAndroid LoginApp = "qandroid"
)
// QRCodeLoginWithApp logins user through QRCode with specified app.
// You SHOULD call this method ONLY when `QRCodeStatus.IsAllowed()` is true.
func (c *Pan115Client) QRCodeLoginWithApp(s *QRCodeSession, app LoginApp) (*Credential, error) {
result := QRCodeLoginResp{}
req := c.NewRequest().
SetFormData(map[string]string{
"account": s.UID,
"app": string(app),
}).
ForceContentType("application/json;charset=UTF-8").
SetResult(&result)
resp, err := req.Post(fmt.Sprintf(ApiQrcodeLoginWithApp, app))
if err = CheckErr(err, &result, resp); err != nil {
return nil, err
}
return &result.Data.Credential, nil
}
type QRCodeStatus struct {
Msg string `json:"msg"`
Status int `json:"status"`
Version string `json:"version"`
}
func (s *QRCodeStatus) IsWaiting() bool {
return s.Status == 0
}
func (s *QRCodeStatus) IsScanned() bool {
return s.Status == 1
}
func (s *QRCodeStatus) IsAllowed() bool {
return s.Status == 2
}
func (s *QRCodeStatus) IsExpired() bool {
return s.Status == -1
}
func (s *QRCodeStatus) IsCanceled() bool {
return s.Status == -2
}
/*
QRCodeStatus represents the status of a QRCode session.
There are 4 possible status values:
- Waiting
- Scanned
- Allowed
- Canceled
*/
func (c *Pan115Client) QRCodeStatus(s *QRCodeSession) (*QRCodeStatus, error) {
result := QRCodeStatusResp{}
req := c.NewRequest().
SetQueryParams(map[string]string{
"uid": s.UID,
"time": strconv.FormatInt(s.Time, 10),
"sign": s.Sign,
"_": Now().String(),
}).
ForceContentType("application/json;charset=UTF-8").
SetResult(&result)
resp, err := req.Get(ApiQrcodeStatus)
if err = CheckErr(err, &result, resp); err != nil {
return nil, err
}
return &result.Data, nil
}
backend/vendor/github.com/SheltonZhu/115driver/pkg/driver/recylcle.go
Generated Vendored
+76
View File
@@ -0,0 +1,76 @@
package driver
import (
"fmt"
"net/url"
"strconv"
)
// CleanRecycleBin clean the recycle bin
func (c *Pan115Client) CleanRecycleBin(password string, rIDs ...string) error {
form := url.Values{}
form.Set("password", password)
for idx, rID := range rIDs {
form.Add(fmt.Sprintf("rid[%d]", idx), rID)
}
result := BasicResp{}
req := c.NewRequest().
SetFormDataFromValues(form).
SetResult(&result).
ForceContentType("application/json;charset=UTF-8")
resp, err := req.Post(ApiRecycleClean)
return CheckErr(err, &result, resp)
}
// ListRecycleBin list the recycle bin
func (c *Pan115Client) ListRecycleBin(offset, limit int) ([]RecycleBinItem, error) {
result := RecycleListResponse{}
req := c.NewRequest().
SetQueryParams(map[string]string{
"aid": "7",
"cid": "0",
"format": "json",
"offset": strconv.Itoa(offset),
"limit": strconv.Itoa(limit),
}).
SetResult(&result).
ForceContentType("application/json;charset=UTF-8")
resp, err := req.Get(ApiRecycleList)
err = CheckErr(err, &result, resp)
if err != nil {
return nil, err
}
return result.Data, nil
}
type RecycleListResponse struct {
BasicResp
Data []RecycleBinItem `json:"data"`
}
type RecycleBinItem struct {
FileId string `json:"id"`
FileName string `json:"file_name"`
FileSize StringInt64 `json:"file_size"`
ParentId IntString `json:"cid"`
ParentName string `json:"parent_name"`
DeleteTime StringInt64 `json:"dtime"`
}
// RevertRecycleBin revert the recycle bin
func (c *Pan115Client) RevertRecycleBin(rIDs ...string) error {
form := url.Values{}
for idx, rID := range rIDs {
form.Add(fmt.Sprintf("rid[%d]", idx), rID)
}
result := BasicResp{}
req := c.NewRequest().
SetFormDataFromValues(form).
SetResult(&result).
ForceContentType("application/json;charset=UTF-8")
resp, err := req.Post(ApiRecycleRevert)
return CheckErr(err, &result, resp)
}
backend/vendor/github.com/SheltonZhu/115driver/pkg/driver/response.go
Generated Vendored
+428
View File
@@ -0,0 +1,428 @@
package driver
import (
"encoding/json"
"time"
)
type LoginResp struct {
Code int `json:"code"`
CheckSsd bool `json:"check_ssd"`
Data struct {
Expire int64 `json:"expire"`
Link string `json:"link"`
UserID int64 `json:"user_id"`
} `json:"data"`
Errno int `json:"errno"`
Error string `json:"error"`
Message string `json:"message"`
State int `json:"state"`
Expire int `json:"expire"`
}
func (resp *LoginResp) Err(respBody ...string) error {
if resp.State == 0 {
return nil
}
if len(respBody) > 0 {
return GetErr(resp.Code, respBody[0])
}
return GetErr(resp.Code)
}
type BasicResp struct {
Errno StringInt `json:"errno,omitempty"`
ErrNo int `json:"errNo,omitempty"`
Error string `json:"error,omitempty"`
State bool `json:"state,omitempty"`
Errtype string `json:"errtype,omitempty"`
Msg string `json:"msg,omitempty"`
}
func (resp *BasicResp) Err(respBody ...string) error {
if resp.State {
return nil
}
nonZeroCode := findNonZero(int(resp.Errno), resp.ErrNo)
if len(respBody) > 0 {
return GetErr(nonZeroCode, respBody[0])
}
return GetErr(nonZeroCode)
}
func findNonZero(code ...int) int {
for _, c := range code {
if c != 0 {
return c
}
}
return 0
}
type MkdirResp struct {
BasicResp
AreaID IntString `json:"aid"`
CategoryID IntString `json:"cid"`
CategoryName string `json:"cname"`
FileID string `json:"file_id"`
FileName string `json:"file_name"`
}
type FileListResp struct {
BasicResp
AreaID string `json:"aid"`
CategoryID IntString `json:"cid"`
Count int `json:"count"`
Order string `json:"order"`
IsAsc int `json:"is_asc"`
Offset int `json:"offset"`
Limit int `json:"limit"`
PageSize int `json:"page_size"`
Files []FileInfo `json:"data"`
}
type FileInfo struct {
AreaID IntString `json:"aid"`
CategoryID IntString `json:"cid"`
FileID string `json:"fid"`
ParentID string `json:"pid"`
Name string `json:"n"`
Type string `json:"ico"`
Size StringInt64 `json:"s"`
Sha1 string `json:"sha"`
PickCode string `json:"pc"`
IsStar StringInt `json:"m"`
Labels []*LabelInfo `json:"fl"`
CreateTime StringInt64 `json:"tp"`
UpdateTime string `json:"t"`
ThumbURL string `json:"u"`
}
type LabelInfo struct {
ID string `json:"id"`
Name string `json:"name"`
Color string `json:"color"`
Sort StringInt `json:"sort"`
CreateTime int64 `json:"create_time"`
UpdateTime int64 `json:"update_time"`
}
type UploadInfoResp struct {
BasicResp
UploadMetaInfo
UserID int64 `json:"user_id"`
Userkey string `json:"userkey"`
}
type UploadEndpointResp struct {
Endpoint string `json:"endpoint"`
GetTokenURL string `json:"gettokenurl"`
}
type UploadMetaInfo struct {
AppID int64 `json:"app_id"`
AppVersion int64 `json:"app_version"`
IspType int64 `json:"isp_type"`
MaxDirLevel int64 `json:"max_dir_level"`
MaxDirLevelYun int64 `json:"max_dir_level_yun"`
MaxFileNum int64 `json:"max_file_num"`
MaxFileNumYun int64 `json:"max_file_num_yun"`
SizeLimit int64 `json:"size_limit"`
SizeLimitYun int64 `json:"size_limit_yun"`
TypeLimit []string `json:"type_limit"`
UploadAllowed bool `json:"upload_allowed"`
UploadAllowedMsg string `json:"upload_allowed_msg"`
}
type UploadInitResp struct {
Request string `json:"request"`
ErrorCode int `json:"statuscode"`
ErrorMsg string `json:"statusmsg"`
Status BoolInt `json:"status"`
PickCode string `json:"pickcode"`
Target string `json:"target"`
Version string `json:"version"`
// OSS upload fields
UploadOSSParams
// Useless fields
FileID int `json:"fileid"`
FileInfo string `json:"fileinfo"`
// New fields in upload v4.0
SignKey string `json:"sign_key"`
SignCheck string `json:"sign_check"`
}
type UploadOSSParams struct {
SHA1 string `json:"-"`
Bucket string `json:"bucket"`
Object string `json:"object"`
Callback struct {
Callback string `json:"callback"`
CallbackVar string `json:"callback_var"`
} `json:"callback"`
}
func (r *UploadInitResp) Err(respBody ...string) error {
if r.ErrorCode == 0 || r.ErrorCode == 701 {
return nil
}
return GetErr(r.ErrorCode, r.ErrorMsg)
}
// Ok if fastupload is successful will return true, otherwise return false
func (r *UploadInitResp) Ok() (bool, error) {
switch r.Status {
case 2:
return true, nil
case 1:
return false, nil
default:
return false, ErrUnexpected
}
}
type UploadOSSTokenResp struct {
AccessKeyID string `json:"AccessKeyID"`
AccessKeySecret string `json:"AccessKeySecret"`
Expiration time.Time `json:"Expiration"`
SecurityToken string `json:"SecurityToken"`
StatusCode string `json:"StatusCode"`
}
func (r *UploadOSSTokenResp) Err(respBody ...string) error {
if r.StatusCode == "200" {
return nil
}
if len(respBody) > 0 {
return GetErr(0, respBody[0])
}
return ErrUnexpected
}
type DownloadResp struct {
BasicResp
EncodedData DataString `json:"data,omitempty"`
}
type DataString string
func (v *DataString) UnmarshalJSON(b []byte) (err error) {
var s string
if b[0] == '"' {
err = json.Unmarshal(b, &s)
}
if err == nil {
*v = DataString(s)
}
return
}
type UserInfoResp struct {
BasicResp
UserInfo UserInfo `json:"data"`
}
type UserInfo struct {
Device int `json:"device"`
Rank int `json:"rank"`
Liang int `json:"liang"`
Mark int `json:"mark"`
Mark1 int `json:"mark1"`
Vip int `json:"vip"`
Expire int `json:"expire"`
Global int `json:"global"`
Forever int `json:"forever"`
IsPrivilege bool `json:"is_privilege"`
Privilege Privilege `json:"privilege"`
UserName string `json:"user_name"`
Face string `json:"face"`
UserID int64 `json:"user_id"`
}
type Privilege struct {
Start int `json:"start"`
Expire int `json:"expire"`
State bool `json:"state"`
Mark StringInt `json:"mark"`
}
type FileStatResponse struct {
FileCount StringInt `json:"count"`
Size string `json:"size"`
FolderCount StringInt `json:"folder_count"`
CreateTime StringInt64 `json:"ptime"`
UpdateTime StringInt64 `json:"utime"`
IsShare StringInt `json:"is_share"`
FileName string `json:"file_name"`
PickCode string `json:"pick_code"`
Sha1 string `json:"sha1"`
IsMark StringInt `json:"is_mark"`
OpenTime int64 `json:"open_time"`
IsFile StringInt `json:"file_category"`
Paths []*FileParentInfo `json:"paths"`
}
type FileParentInfo struct {
FileID int `json:"file_id"`
FileName string `json:"file_name"`
}
func (r *FileStatResponse) Err(respBody ...string) error {
return nil
}
type GetFileInfoResponse struct {
BasicResp
Files []*FileInfo `json:"data"`
}
type QRCodeBasicResp struct {
Code int `json:"code"`
Message string `json:"message"`
State int `json:"state"`
Errno int `json:"errno"`
Error string `json:"error"`
}
func (resp *QRCodeBasicResp) Err(respBody ...string) error {
if resp.State == 1 {
return nil
}
if len(respBody) > 0 {
return GetErr(resp.Code, respBody[0])
}
return GetErr(resp.Code)
}
type QRCodeTokenResp struct {
QRCodeBasicResp
Data QRCodeSession `json:"data"`
}
type QRCodeLoginResp struct {
QRCodeBasicResp
Data struct {
Alert string `json:"alert"`
BindMobile int `json:"bind_mobile"`
Credential Credential `json:"cookie"`
Country string `json:"country"`
Email string `json:"email"`
Face struct {
FaceL string `json:"face_l"`
FaceM string `json:"face_m"`
FaceS string `json:"face_s"`
} `json:"face"`
From string `json:"from"`
IsChangPasswd int `json:"is_chang_passwd"`
IsFirstLogin int `json:"is_first_login"`
IsTrusted interface{} `json:"is_trusted"`
IsVip int64 `json:"is_vip"`
Mark int `json:"mark"`
Mobile string `json:"mobile"`
UserID int `json:"user_id"`
UserName string `json:"user_name"`
} `json:"data"`
}
type QRCodeStatusResp struct {
QRCodeBasicResp
Data QRCodeStatus `json:"data"`
}
type ShareSnapResp struct {
BasicResp
Data struct {
Userinfo struct {
UserID string `json:"user_id"`
UserName string `json:"user_name"`
Face string `json:"face"`
} `json:"userinfo"`
Shareinfo struct {
SnapID string `json:"snap_id"`
FileSize StringInt64 `json:"file_size"`
ShareTitle string `json:"share_title"`
ShareState StringInt64 `json:"share_state"`
ForbidReason string `json:"forbid_reason"`
CreateTime StringInt64 `json:"create_time"`
ReceiveCode string `json:"receive_code"`
ReceiveCount StringInt64 `json:"receive_count"`
ExpireTime int64 `json:"expire_time"`
FileCategory int64 `json:"file_category"`
AutoRenewal StringInt64 `json:"auto_renewal"`
ShareDuration int `json:"share_duration"`
AutoFillRecvcode StringInt64 `json:"auto_fill_recvcode"`
CanReport int `json:"can_report"`
CanNotice int `json:"can_notice"`
HaveVioFile int `json:"have_vio_file"`
SkipLoginState StringInt64 `json:"skip_login_state"`
} `json:"shareinfo"`
Count int `json:"count"`
List []ShareFile `json:"list"`
ShareState StringInt64 `json:"share_state"`
UserAppeal struct {
CanAppeal int `json:"can_appeal"`
CanShareAppeal int `json:"can_share_appeal"`
PopupAppealPage int `json:"popup_appeal_page"`
CanGlobalAppeal int `json:"can_global_appeal"`
} `json:"user_appeal"`
} `json:"data"`
}
type ShareFile struct {
FileID string `json:"fid"`
UID int `json:"uid"`
CategoryID IntString `json:"cid"`
FileName string `json:"n"`
Type string `json:"ico"`
Sha1 string `json:"sha"`
Size StringInt64 `json:"s"`
Labels []*LabelInfo `json:"fl"`
UpdateTime string `json:"t"`
IsFile int `json:"fc"`
ParentID string `json:"pid"`
// Ns string `json:"ns"`
// D int `json:"d"`
// C int `json:"c"`
// E string `json:"e"`
// Ms int `json:"ms"`
IsSkipLogin int `json:"is_skip_login"`
ThumbURL string `json:"u"`
}
type UploadResult struct {
BasicResp
Data struct {
PickCode string `json:"pick_code"`
FileSize int `json:"file_size"`
FileID string `json:"file_id"`
ThumbURL string `json:"thumb_url"`
Sha1 string `json:"sha1"`
Aid int `json:"aid"`
FileName string `json:"file_name"`
Cid string `json:"cid"`
IsVideo int `json:"is_video"`
} `json:"data"`
}
type APIGetDirIDResp struct {
BasicResp
CategoryID IntString `json:"id"`
IsPrivate IntString `json:"is_private"`
}
type DownloadShareResp struct {
BasicResp
Data SharedDownloadInfo `json:"data"`
}
backend/vendor/github.com/SheltonZhu/115driver/pkg/driver/search.go
Generated Vendored
+178
View File
@@ -0,0 +1,178 @@
package driver
import (
"strconv"
)
// SearchOption defines options for search
type SearchOption struct {
// Offset for pagination
Offset int
// Limit number of results
Limit int
// SearchValue search keyword
SearchValue string
// Date filter
Date string
// Aid area ID
Aid string
// Cid category ID
Cid string
// PickCode pickcode
PickCode string
// Type file type filter 0:all 1:folder 2:document 3:image 4:video 5:audio 6:archive
Type int
// CountFolders whether to count folders
CountFolders int
// Source source filter
Source string
// Star star file only
Star string
// Suffix file suffix filter
Suffix string
// Order sort field
Order string
// Asc ascending order 0:descending 1:ascending
Asc int
}
// SearchFile represents a file in search results
type SearchFile struct {
// File ID
FileID string `json:"fid"`
// Category ID
CategoryID IntString `json:"cid"`
// File name
Name string `json:"n"`
// File size
Size StringInt64 `json:"s"`
// SHA1 hash
Sha1 string `json:"sha"`
// PickCode
PickCode string `json:"pc"`
// Is directory
IsDirectory int `json:"fc"`
// Is starred file
IsStar StringInt `json:"m"`
// Update time
UpdateTime string `json:"t"`
// Create time
CreateTime StringInt64 `json:"tp"`
// File type icon
Icon string `json:"ico"`
// Highlighted file name
HighlightName string `json:"ns"`
// File labels
Labels []*LabelInfo `json:"fl"`
// Thumbnail URL
ThumbURL string `json:"u"`
}
// SearchResult represents search results
type SearchResult struct {
// File list
Files []File `json:"data"`
// Total count
Count int `json:"count"`
// File count
FileCount int `json:"file_count"`
// Folder count
FolderCount int `json:"folder_count"`
// Page size
PageSize int `json:"page_size"`
// Offset
Offset int `json:"offset"`
// Sort field
Order string `json:"order"`
// Ascending order
IsAsc int `json:"is_asc"`
}
// Search searches for files using given options
func (c *Pan115Client) Search(opts *SearchOption) (*SearchResult, error) {
result := FileListResp{}
params := map[string]string{
"aid": "7",
"cid": "0",
"format": "json",
"offset": "0",
"limit": "30",
"search_value": "",
"type": "0",
"count_folders": "1",
"o": "file_name",
"asc": "1",
}
// Set search parameters
if opts != nil {
if opts.Offset >= 0 {
params["offset"] = strconv.Itoa(opts.Offset)
}
if opts.Limit > 0 {
params["limit"] = strconv.Itoa(opts.Limit)
}
if opts.SearchValue != "" {
params["search_value"] = opts.SearchValue
}
if opts.Date != "" {
params["date"] = opts.Date
}
if opts.Aid != "" {
params["aid"] = opts.Aid
}
if opts.Cid != "" {
params["cid"] = opts.Cid
}
if opts.PickCode != "" {
params["pick_code"] = opts.PickCode
}
if opts.Type > 0 {
params["type"] = strconv.Itoa(opts.Type)
}
if opts.CountFolders > 0 {
params["count_folders"] = strconv.Itoa(opts.CountFolders)
}
if opts.Source != "" {
params["source"] = opts.Source
}
if opts.Star != "" {
params["star"] = opts.Star
}
if opts.Suffix != "" {
params["suffix"] = opts.Suffix
}
if opts.Order != "" {
params["o"] = opts.Order
}
params["asc"] = strconv.Itoa(opts.Asc)
}
req := c.NewRequest().
SetQueryParams(params).
SetResult(&result).
ForceContentType("application/json;charset=UTF-8")
resp, err := req.Get(ApiFileSearch)
if err = CheckErr(err, &result, resp); err != nil {
return nil, err
}
// Convert results
searchResult := &SearchResult{
Count: result.Count,
FileCount: 0, // Not available in FileListResp
FolderCount: 0, // Not available in FileListResp
PageSize: result.PageSize,
Offset: result.Offset,
Order: result.Order,
IsAsc: result.IsAsc,
Files: make([]File, 0, len(result.Files)),
}
for _, fileInfo := range result.Files {
searchResult.Files = append(searchResult.Files, *(&File{}).from(&fileInfo))
}
return searchResult, nil
}
backend/vendor/github.com/SheltonZhu/115driver/pkg/driver/share.go
Generated Vendored
+67
View File
@@ -0,0 +1,67 @@
package driver
import (
"fmt"
"strconv"
)
type Query func(query *map[string]string)
// QueryLimit set query limit
func QueryLimit(limit int) Query {
return func(query *map[string]string) {
(*query)["limit"] = strconv.FormatInt(int64(limit), 10)
}
}
// QueryOffset set query offset
func QueryOffset(offset int) Query {
return func(query *map[string]string) {
(*query)["offset"] = strconv.FormatInt(int64(offset), 10)
}
}
// GetShareSnapWithUA get share snap info with user agent
func (c *Pan115Client) GetShareSnapWithUA(ua, shareCode, receiveCode, dirID string, Queries ...Query) (*ShareSnapResp, error) {
if isCalledByAlistV3() {
return nil, ErrorNotSupportAlist
}
result := ShareSnapResp{}
query := map[string]string{
"share_code": shareCode,
"receive_code": receiveCode,
"cid": dirID,
"limit": "20",
"asc": "0",
"offset": "0",
"format": "json",
}
for _, q := range Queries {
q(&query)
}
req := c.NewRequest().
SetQueryParams(query).
SetHeader("referer", BuildShareReferer(shareCode, receiveCode)).
ForceContentType("application/json;charset=UTF-8").
SetResult(&result)
if ua != "" {
req.SetHeader("User-Agent", ua)
}
resp, err := req.Get(ApiShareSnap)
if err := CheckErr(err, &result, resp); err != nil {
return nil, err
}
return &result, nil
}
func BuildShareReferer(shareCode, receiveCode string) string {
return fmt.Sprintf("https://115cdn.com/s/%s?password=%s&", shareCode, receiveCode)
}
// GetShareSnap get share snap info
func (c *Pan115Client) GetShareSnap(shareCode, receiveCode, dirID string, Queries ...Query) (*ShareSnapResp, error) {
return c.GetShareSnapWithUA("", shareCode, receiveCode, dirID, Queries...)
}
backend/vendor/github.com/SheltonZhu/115driver/pkg/driver/types.go
Generated Vendored
+121
View File
@@ -0,0 +1,121 @@
package driver
import (
"encoding/json"
"strconv"
"time"
)
// StringInt uses for json field which maybe a string or an int.
type StringInt int64
func (v *StringInt) UnmarshalJSON(b []byte) (err error) {
var i int
if b[0] == '"' {
var s string
if err = json.Unmarshal(b, &s); err == nil {
i, _ = strconv.Atoi(s)
}
} else {
err = json.Unmarshal(b, &i)
}
if err == nil {
*v = StringInt(i)
}
return
}
// StringInt64 uses for json field which maybe a string or an int64.
type StringInt64 int64
func (v *StringInt64) UnmarshalJSON(b []byte) (err error) {
var i int64
if b[0] == '"' {
var s string
if err = json.Unmarshal(b, &s); err == nil {
i, err = strconv.ParseInt(s, 10, 64)
}
} else {
err = json.Unmarshal(b, &i)
}
if err == nil {
*v = StringInt64(i)
}
return
}
// StringFloat64 uses for json field which maybe a string or a float64.
type StringFloat64 float64
func (v *StringFloat64) UnmarshalJSON(b []byte) (err error) {
var f float64
if b[0] == '"' {
var s string
if err = json.Unmarshal(b, &s); err == nil {
f, err = strconv.ParseFloat(s, 64)
}
} else {
err = json.Unmarshal(b, &f)
}
if err == nil {
*v = StringFloat64(f)
}
return
}
type IntString string
func (v *IntString) UnmarshalJSON(b []byte) (err error) {
var s string
if b[0] == '"' {
err = json.Unmarshal(b, &s)
} else {
var i int64
if err = json.Unmarshal(b, &i); err == nil {
s = strconv.FormatInt(i, 10)
}
}
if err == nil {
*v = IntString(s)
}
return
}
type BoolInt int
func (v *BoolInt) UnmarshalJSON(b []byte) (err error) {
if b[0] == 'f' || b[0] == 'F' {
*v = -1
} else {
var i int
if err = json.Unmarshal(b, &i); err == nil {
*v = BoolInt(i)
}
}
return
}
func BoolToInt(b bool) int {
if b {
return 1
}
return 0
}
type StringTime int64
func (v *StringTime) UnmarshalJSON(b []byte) (err error) {
var t time.Time
if b[0] == '"' {
var s string
if err = json.Unmarshal(b, &s); err == nil {
t, err = time.Parse("2006-01-02 15:04", s)
}
} else {
err = json.Unmarshal(b, &t)
}
if err == nil {
*v = StringTime(t.Unix())
}
return
}
backend/vendor/github.com/SheltonZhu/115driver/pkg/driver/upload.go
Generated Vendored
+554
View File
@@ -0,0 +1,554 @@
package driver
import (
"bytes"
"crypto/md5"
"crypto/sha1"
"encoding/base64"
"encoding/hex"
"encoding/json"
"fmt"
"io"
"net/url"
"os"
"strconv"
"strings"
"sync"
"time"
hash "github.com/SheltonZhu/115driver/pkg/crypto"
cipher "github.com/SheltonZhu/115driver/pkg/crypto/ec115"
"github.com/aliyun/aliyun-oss-go-sdk/oss"
"github.com/pkg/errors"
)
// GetDigestResult get digest of file or stream
func (c *Pan115Client) GetDigestResult(r io.Reader) (*hash.DigestResult, error) {
d := hash.DigestResult{}
return &d, hash.Digest(r, &d)
}
// GetUploadEndpoint get upload endPoint
func (c *Pan115Client) GetUploadEndpoint(endpoint *UploadEndpointResp) error {
req := c.NewRequest().
ForceContentType("application/json;charset=UTF-8").
SetResult(&endpoint)
_, err := req.Get(ApiGetUploadEndpoint)
if err != nil {
return err
}
return nil
}
// GetUploadInfo get some info for upload
func (c *Pan115Client) GetUploadInfo() error {
result := UploadInfoResp{}
req := c.NewRequest().
ForceContentType("application/json;charset=UTF-8").
SetResult(&result)
resp, err := req.Post(ApiUploadInfo)
if err = CheckErr(err, &result, resp); err != nil {
return err
}
c.Userkey = result.Userkey
c.UserID = result.UserID
c.UploadMetaInfo = &result.UploadMetaInfo
return nil
}
// UploadAvailable check and prepare to upload
func (c *Pan115Client) UploadAvailable() (bool, error) {
if c.UserID != 0 && len(c.Userkey) > 0 {
return true, nil
}
if err := c.GetUploadInfo(); err != nil {
return false, err
}
return true, nil
}
// UploadFastOrByOSS Upload By OSS when unable to rapid upload file
// Deprecated: As of v1.0.22, this function simply calls [RapidUploadOrByOSS].
func (c *Pan115Client) UploadFastOrByOSS(dirID, fileName string, fileSize int64, r io.ReadSeeker) error {
return c.RapidUploadOrByOSS(dirID, fileName, fileSize, r)
}
// RapidUploadOrByOSS Upload By OSS when unable to rapid upload file
func (c *Pan115Client) RapidUploadOrByOSS(dirID, fileName string, fileSize int64, r io.ReadSeeker) error {
var (
err error
digest *hash.DigestResult
fastInfo *UploadInitResp
)
if ok, err := c.UploadAvailable(); err != nil || !ok {
return err
}
if fileSize > c.UploadMetaInfo.SizeLimit {
return ErrUploadTooLarge
}
if digest, err = c.GetDigestResult(r); err != nil {
return err
}
// 闪传
if fastInfo, err = c.RapidUpload(
digest.Size, fileName, dirID, digest.PreID, digest.QuickID, r,
); err != nil {
return err
}
if ok, err := fastInfo.Ok(); err != nil {
return err
} else if ok {
return nil
}
if _, err = r.Seek(0, io.SeekStart); err != nil {
return err
}
// 闪传失败，普通上传
return c.UploadByOSS(&fastInfo.UploadOSSParams, r, dirID)
}
// getOSSEndpoint get oss endpoint 利用阿里云内网上传文件，需要在阿里云服务器上运行本程序，同时也需要115在服务器的所在地域开通了阿里云OSS
func (c *Pan115Client) getOSSEndpoint(enableInternalUpload bool) string {
if enableInternalUpload {
uploadEndpoint := UploadEndpointResp{}
if err := c.GetUploadEndpoint(&uploadEndpoint); err != nil {
// TODO warn error log
return OSSEndpoint
}
i := strings.Index(uploadEndpoint.Endpoint, ".aliyuncs.com")
if i > -1 {
endpoint := uploadEndpoint.Endpoint[:i] + "-internal" + uploadEndpoint.Endpoint[i:]
return endpoint
}
}
return OSSEndpoint
}
// GetOSSEndpoint get oss endpoint 利用阿里云内网上传文件，需要在阿里云服务器上运行本程序，同时也需要115在服务器的所在地域开通了阿里云OSS
func (c *Pan115Client) GetOSSEndpoint(enableInternalUpload bool) string {
return c.getOSSEndpoint(enableInternalUpload)
}
// UploadByOSS use aliyun sdk to upload
func (c *Pan115Client) UploadByOSS(params *UploadOSSParams, r io.Reader, dirID string) error {
ossToken, err := c.GetOSSToken()
if err != nil {
return err
}
ossClient, err := oss.New(c.getOSSEndpoint(c.UseInternalUpload), ossToken.AccessKeyID, ossToken.AccessKeySecret)
if err != nil {
return err
}
bucket, err := ossClient.Bucket(params.Bucket)
if err != nil {
return err
}
if err = bucket.PutObject(params.Object, r, OssOption(params, ossToken)...); err != nil {
return err
}
return c.checkUploadStatus(dirID, params.SHA1)
}
func (c *Pan115Client) checkUploadStatus(dirID, sha1 string) error {
// 验证上传是否成功
req := c.NewRequest().ForceContentType("application/json;charset=UTF-8")
opts := []GetFileOptions{
WithOrder(FileOrderByTime),
WithShowDirEnable(false),
WithAsc(false),
WithLimit(500),
}
fResp, err := GetFiles(req, dirID, opts...)
if err != nil {
return err
}
for _, fileInfo := range fResp.Files {
if fileInfo.Sha1 == sha1 {
return nil
}
}
return ErrUploadFailed
}
// GetOSSToken get oss token for oss upload
func (c *Pan115Client) GetOSSToken() (*UploadOSSTokenResp, error) {
result := UploadOSSTokenResp{}
req := c.NewRequest().
ForceContentType("application/json;charset=UTF-8").
SetResult(&result)
resp, err := req.Get(ApiUploadOSSToken)
return &result, CheckErr(err, &result, resp)
}
// UploadSHA1 upload a sha1, alias of RapidUpload
// Deprecated: As of v1.0.22, this function simply calls [RapidUpload].
func (c *Pan115Client) UploadSHA1(fileSize int64, fileName, dirID, preID, fileID string, r io.ReadSeeker) (*UploadInitResp, error) {
return c.RapidUpload(fileSize, fileName, dirID, preID, fileID, r)
}
// RapidUpload rapid upload
func (c *Pan115Client) RapidUpload(fileSize int64, fileName, dirID, preID, fileID string, r io.ReadSeeker) (*UploadInitResp, error) {
var (
ecdhCipher *cipher.EcdhCipher
encrypted []byte
decrypted []byte
encodedToken string
err error
target = "U_1_" + dirID
bodyBytes []byte
result = UploadInitResp{}
fileSizeStr = strconv.FormatInt(fileSize, 10)
)
if ecdhCipher, err = cipher.NewEcdhCipher(); err != nil {
return nil, err
}
if ok, err := c.UploadAvailable(); !ok || err != nil {
return nil, err
}
userID := strconv.FormatInt(c.UserID, 10)
form := url.Values{}
form.Set("appid", "0")
form.Set("appversion", appVer)
form.Set("userid", userID)
form.Set("filename", fileName)
form.Set("filesize", fileSizeStr)
form.Set("fileid", fileID)
form.Set("target", target)
form.Set("sig", c.GenerateSignature(fileID, target))
form.Set("topupload", "true")
signKey, signVal := "", ""
for retry := true; retry; {
t := NowMilli()
if encodedToken, err = ecdhCipher.EncodeToken(t.ToInt64()); err != nil {
return nil, err
}
params := map[string]string{
"k_ec": encodedToken,
}
form.Set("t", t.String())
form.Set("token", c.GenerateToken(fileID, preID, t.String(), fileSizeStr, signKey, signVal))
if signKey != "" && signVal != "" {
form.Set("sign_key", signKey)
form.Set("sign_val", signVal)
}
if encrypted, err = ecdhCipher.Encrypt([]byte(form.Encode())); err != nil {
return nil, err
}
req := c.NewRequest().
SetQueryParams(params).
SetBody(encrypted).
SetHeaderVerbatim("Content-Type", "application/x-www-form-urlencoded").
SetDoNotParseResponse(true)
resp, err := req.Post(ApiUploadInit)
if err != nil {
return nil, err
}
data := resp.RawBody()
defer data.Close()
if bodyBytes, err = io.ReadAll(data); err != nil {
return nil, err
}
if decrypted, err = ecdhCipher.Decrypt(bodyBytes); err != nil {
return nil, err
}
if err = CheckErr(json.Unmarshal(decrypted, &result), &result, resp); err != nil {
return nil, err
}
if result.Status == 7 {
// Update signKey & signVal
signKey = result.SignKey
signVal, _ = c.UploadDigestRange(r, result.SignCheck)
} else {
retry = false
}
result.SHA1 = fileID
}
return &result, nil
}
const (
md5Salt = "Qclm8MGWUv59TnrR0XPg"
appVer = "27.0.5.7"
)
func (c *Pan115Client) UploadDigestRange(r io.ReadSeeker, rangeSpec string) (result string, err error) {
var start, end int64
if _, err = fmt.Sscanf(rangeSpec, "%d-%d", &start, &end); err != nil {
return
}
h := sha1.New()
_, err = r.Seek(start, io.SeekStart)
if err != nil {
return
}
if _, err = io.CopyN(h, r, end-start+1); err == nil {
result = strings.ToUpper(hex.EncodeToString(h.Sum(nil)))
}
return
}
func (c *Pan115Client) GenerateSignature(fileID, target string) string {
sh1hash := sha1.Sum([]byte(strconv.FormatInt(c.UserID, 10) + fileID + target + "0"))
sigStr := c.Userkey + hex.EncodeToString(sh1hash[:]) + "000000"
sh1Sig := sha1.Sum([]byte(sigStr))
return strings.ToUpper(hex.EncodeToString(sh1Sig[:]))
}
func (c *Pan115Client) GenerateToken(fileID, preID, timeStamp, fileSize, signKey, signVal string) string {
userID := strconv.FormatInt(c.UserID, 10)
userIDMd5 := md5.Sum([]byte(userID))
tokenMd5 := md5.Sum([]byte(md5Salt + fileID + fileSize + signKey + signVal + userID + timeStamp + hex.EncodeToString(userIDMd5[:]) + appVer))
return hex.EncodeToString(tokenMd5[:])
}
// UploadFastOrByMultipart upload by mutipart blocks when unable to rapid upload
// Deprecated: As of v1.0.22, this function simply calls [RapidUploadOrByMultipart].
func (c *Pan115Client) UploadFastOrByMultipart(dirID, fileName string, fileSize int64, r *os.File, opts ...UploadMultipartOption) error {
return c.RapidUploadOrByMultipart(dirID, fileName, fileSize, r, opts...)
}
// RapidUploadOrByMultipart upload by mutipart blocks when unable to rapid upload
func (c *Pan115Client) RapidUploadOrByMultipart(dirID, fileName string, fileSize int64, r *os.File, opts ...UploadMultipartOption) error {
var (
err error
digest *hash.DigestResult
fastInfo *UploadInitResp
)
if ok, err := c.UploadAvailable(); err != nil || !ok {
return err
}
if fileSize > c.UploadMetaInfo.SizeLimit {
return ErrUploadTooLarge
}
if digest, err = c.GetDigestResult(r); err != nil {
return err
}
// 闪传
if fastInfo, err = c.RapidUpload(
digest.Size, fileName, dirID, digest.PreID, digest.QuickID, r,
); err != nil {
return err
}
if ok, err := fastInfo.Ok(); err != nil {
return err
} else if ok {
return nil
}
if _, err = r.Seek(0, io.SeekStart); err != nil {
return err
}
// 闪传失败，上传
if digest.Size <= KB { // 文件大小小于1KB，改用普通模式上传
return c.UploadByOSS(&fastInfo.UploadOSSParams, r, dirID)
}
// 分片上传
return c.UploadByMultipart(&fastInfo.UploadOSSParams, digest.Size, r, dirID, opts...)
}
// UploadByMultipart upload by mutipart blocks
func (c *Pan115Client) UploadByMultipart(params *UploadOSSParams, fileSize int64, f *os.File, dirID string, opts ...UploadMultipartOption) error {
var (
chunks []oss.FileChunk
parts []oss.UploadPart
imur oss.InitiateMultipartUploadResult
ossClient *oss.Client
bucket *oss.Bucket
ossToken *UploadOSSTokenResp
bodyBytes []byte
err error
)
options := DefaultUploadMultipartOptions()
if len(opts) > 0 {
for _, f := range opts {
f(options)
}
}
options.ThreadsNum = 1
if ossToken, err = c.GetOSSToken(); err != nil {
return err
}
if ossClient, err = oss.New(
c.getOSSEndpoint(c.UseInternalUpload),
ossToken.AccessKeyID,
ossToken.AccessKeySecret,
oss.EnableMD5(true),
oss.EnableCRC(true),
); err != nil {
return err
}
if bucket, err = ossClient.Bucket(params.Bucket); err != nil {
return err
}
// ossToken一小时后就会失效，所以每50分钟重新获取一次
ticker := time.NewTicker(options.TokenRefreshTime)
defer ticker.Stop()
// 设置超时
timeout := time.NewTimer(options.Timeout)
if chunks, err = SplitFile(f.Name(), fileSize); err != nil {
return err
}
if imur, err = bucket.InitiateMultipartUpload(params.Object,
oss.SetHeader(OssSecurityTokenHeaderName, ossToken.SecurityToken),
oss.UserAgentHeader(OSSUserAgent),
oss.EnableSha1(),
oss.Sequential(), // oss 启用Sequential必须按顺序上传, options.ThreadsNum = 1
); err != nil {
return err
}
wg := sync.WaitGroup{}
wg.Add(len(chunks))
chunksCh := make(chan oss.FileChunk)
errCh := make(chan error)
UploadedPartsCh := make(chan oss.UploadPart)
quit := make(chan struct{})
// producter
go chunksProducer(chunksCh, chunks)
go func() {
wg.Wait()
quit <- struct{}{}
}()
// consumers
for i := 0; i < options.ThreadsNum; i++ {
go func(threadId int) {
defer func() {
if r := recover(); r != nil {
errCh <- fmt.Errorf("recovered in %v", r)
}
}()
for chunk := range chunksCh {
var part oss.UploadPart // 出现错误就继续尝试，共尝试3次
for retry := 0; retry < 3; retry++ {
select {
case <-ticker.C:
if ossToken, err = c.GetOSSToken(); err != nil { // 到时重新获取ossToken
errCh <- errors.Wrap(err, "刷新token时出现错误")
}
default:
}
buf := make([]byte, chunk.Size)
if _, err = f.ReadAt(buf, chunk.Offset); err != nil && !errors.Is(err, io.EOF) {
continue
}
if part, err = bucket.UploadPart(
imur,
bytes.NewBuffer(buf),
chunk.Size,
chunk.Number,
OssOption(params, ossToken)...); err == nil {
break
}
}
if err != nil {
errCh <- errors.Wrap(err, fmt.Sprintf("上传 %s 的第%d个分片时出现错误：%v", f.Name(), chunk.Number, err))
}
UploadedPartsCh <- part
}
}(i)
}
go func() {
for part := range UploadedPartsCh {
parts = append(parts, part)
wg.Done()
}
}()
LOOP:
for {
select {
case <-ticker.C:
// 到时重新获取ossToken
if ossToken, err = c.GetOSSToken(); err != nil {
return err
}
case <-quit:
break LOOP
case <-errCh:
return err
case <-timeout.C:
return fmt.Errorf("time out")
}
}
if _, err := bucket.CompleteMultipartUpload(imur, parts,
append(
OssOption(params, ossToken),
oss.CallbackResult(&bodyBytes),
)...); err != nil {
return err
}
var uploadResult UploadResult
if err = json.Unmarshal(bodyBytes, &uploadResult); err != nil {
return err
}
return uploadResult.Err(string(bodyBytes))
}
func chunksProducer(ch chan oss.FileChunk, chunks []oss.FileChunk) {
for _, chunk := range chunks {
ch <- chunk
}
}
// SplitFile pplitFile
func SplitFile(filePath string, fileSize int64) (chunks []oss.FileChunk, err error) {
for i := int64(1); i < 10; i++ {
if fileSize < i*GB { // 文件大小小于iGB时分为i*1000片
if chunks, err = oss.SplitFileByPartNum(filePath, int(i*1000)); err != nil {
return
}
break
}
}
if fileSize > 9*GB { // 文件大小大于9GB时分为10000片
if chunks, err = oss.SplitFileByPartNum(filePath, 10000); err != nil {
return
}
}
// 单个分片大小不能小于100KB
if chunks[0].Size < 100*KB {
if chunks, err = oss.SplitFileByPartSize(filePath, 100*KB); err != nil {
return
}
}
return
}
// OssOption get options
func OssOption(params *UploadOSSParams, ossToken *UploadOSSTokenResp) []oss.Option {
options := []oss.Option{
oss.SetHeader(OssSecurityTokenHeaderName, ossToken.SecurityToken),
oss.Callback(base64.StdEncoding.EncodeToString([]byte(params.Callback.Callback))),
oss.CallbackVar(base64.StdEncoding.EncodeToString([]byte(params.Callback.CallbackVar))),
oss.UserAgentHeader(OSSUserAgent),
}
return options
}
backend/vendor/github.com/SheltonZhu/115driver/pkg/driver/util.go
Generated Vendored
+42
View File
@@ -0,0 +1,42 @@
package driver
import (
"runtime"
"strconv"
"strings"
"time"
)
type Time int64
func Now() Time {
return Time(time.Now().Unix())
}
func NowMilli() Time {
return Time(time.Now().UnixMilli())
}
func (t Time) String() string {
return strconv.FormatInt(t.ToInt64(), 10)
}
func (t Time) ToInt64() int64 {
return int64(t)
}
func Date() string {
GMT, _ := time.LoadLocation("GMT")
now := time.Now().In(GMT)
return now.Format(time.RFC1123)
}
func isCalledByAlistV3() bool {
pc, _, _, ok := runtime.Caller(3)
if !ok {
return false
}
funcName := runtime.FuncForPC(pc).Name()
return strings.Contains(funcName, "alist")
}
backend/vendor/github.com/aead/ecdh/.gitignore
Generated
+25
View File
@@ -0,0 +1,25 @@
# Compiled Object files, Static and Dynamic libs (Shared Objects)
*.o
*.a
*.so
# Folders
_obj
_test
.vscode
# Architecture specific extensions/prefixes
*.[568vq]
[568vq].out
*.cgo1.go
*.cgo2.c
_cgo_defun.c
_cgo_gotypes.go
_cgo_export.*
_testmain.go
*.exe
*.test
*.prof
backend/vendor/github.com/aead/ecdh/.travis.yml
Generated Vendored
+13
View File
@@ -0,0 +1,13 @@
language: go
go:
- 1.11.x
- master
branches:
only:
- master
script:
- diff -au <(gofmt -d .) <(printf "")
- go test -v ./...
backend/vendor/github.com/aead/ecdh/LICENSE
Generated Vendored
+21
View File
@@ -0,0 +1,21 @@
The MIT License (MIT)
Copyright (c) 2016 Andreas Auernhammer
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
backend/vendor/github.com/aead/ecdh/README.md
Generated Vendored
+13
View File
@@ -0,0 +1,13 @@
[![Godoc Reference](https://godoc.org/github.com/aead/ecdh?status.svg)](https://godoc.org/github.com/aead/ecdh)
[![Build Status](https://travis-ci.org/aead/ecdh.svg?branch=master)](https://travis-ci.org/aead/ecdh)
## The ECDH key exchange
Elliptic curve Diffie–Hellman (ECDH) is an anonymous key agreement protocol that allows two parties,
each having an elliptic curve public–private key pair, to establish a shared secret over an insecure channel.
This package implements a generic interface for ECDH and supports the generic [crypto/elliptic](https://godoc.org/crypto/elliptic)
and the [x/crypto/curve25519](https://godoc.org/golang.org/x/crypto/curve25519) out of the box.
### Installation
Install in your GOPATH: `go get -u github.com/aead/ecdh`
backend/vendor/github.com/aead/ecdh/curve25519.go
Generated Vendored
+108
View File
@@ -0,0 +1,108 @@
// Copyright (c) 2016 Andreas Auernhammer. All rights reserved.
// Use of this source code is governed by a license that can be
// found in the LICENSE file.
package ecdh
import (
"crypto"
"crypto/rand"
"errors"
"io"
"golang.org/x/crypto/curve25519"
)
type ecdh25519 struct{}
var curve25519Params = CurveParams{
Name: "Curve25519",
BitSize: 255,
}
// X25519 creates a new ecdh.KeyExchange with
// the elliptic curve Curve25519.
func X25519() KeyExchange {
return ecdh25519{}
}
func (ecdh25519) GenerateKey(random io.Reader) (private crypto.PrivateKey, public crypto.PublicKey, err error) {
if random == nil {
random = rand.Reader
}
var pri, pub [32]byte
_, err = io.ReadFull(random, pri[:])
if err != nil {
return
}
// From https://cr.yp.to/ecdh.html
pri[0] &= 248
pri[31] &= 127
pri[31] |= 64
curve25519.ScalarBaseMult(&pub, &pri)
private = pri
public = pub
return
}
func (ecdh25519) Params() CurveParams { return curve25519Params }
func (ecdh25519) PublicKey(private crypto.PrivateKey) (public crypto.PublicKey) {
var pri, pub [32]byte
if ok := checkType(&pri, private); !ok {
panic("ecdh: unexpected type of private key")
}
curve25519.ScalarBaseMult(&pub, &pri)
public = pub
return
}
func (ecdh25519) Check(peersPublic crypto.PublicKey) (err error) {
if ok := checkType(new([32]byte), peersPublic); !ok {
err = errors.New("unexptected type of peers public key")
}
return
}
func (ecdh25519) ComputeSecret(private crypto.PrivateKey, peersPublic crypto.PublicKey) (secret []byte) {
var sec, pri, pub [32]byte
if ok := checkType(&pri, private); !ok {
panic("ecdh: unexpected type of private key")
}
if ok := checkType(&pub, peersPublic); !ok {
panic("ecdh: unexpected type of peers public key")
}
curve25519.ScalarMult(&sec, &pri, &pub)
secret = sec[:]
return
}
func checkType(key *[32]byte, typeToCheck interface{}) (ok bool) {
switch t := typeToCheck.(type) {
case [32]byte:
copy(key[:], t[:])
ok = true
case *[32]byte:
copy(key[:], t[:])
ok = true
case []byte:
if len(t) == 32 {
copy(key[:], t)
ok = true
}
case *[]byte:
if len(*t) == 32 {
copy(key[:], *t)
ok = true
}
}
return
}
backend/vendor/github.com/aead/ecdh/ecdh.go
Generated Vendored
+46
View File
@@ -0,0 +1,46 @@
// Copyright (c) 2016 Andreas Auernhammer. All rights reserved.
// Use of this source code is governed by a license that can be
// found in the LICENSE file.
// Package ecdh implements the Diffie-Hellman key exchange
// using elliptic curves (ECDH). It directly provides ECDH
// implementations for the NIST curves P224, P256, P384,
// and Bernstein's Cruve25519.
//
// For generic curves this implementation of ECDH
// only uses the x-coordinate as the computed secret.
package ecdh // import "github.com/aead/ecdh"
import (
"crypto"
"io"
)
// KeyExchange is the interface defining all functions
// necessary for ECDH.
type KeyExchange interface {
// GenerateKey generates a private/public key pair using entropy from rand.
// If rand is nil, crypto/rand.Reader will be used.
GenerateKey(rand io.Reader) (private crypto.PrivateKey, public crypto.PublicKey, err error)
// Params returns the curve parameters - like the field size.
Params() CurveParams
// PublicKey returns the public key corresponding to the given private one.
PublicKey(private crypto.PrivateKey) (public crypto.PublicKey)
// Check returns a non-nil error if the peers public key cannot used for the
// key exchange - for instance the public key isn't a point on the elliptic curve.
// It's recommended to check peer's public key before computing the secret.
Check(peersPublic crypto.PublicKey) (err error)
// ComputeSecret returns the secret value computed from the given private key
// and the peers public key.
ComputeSecret(private crypto.PrivateKey, peersPublic crypto.PublicKey) (secret []byte)
}
// CurveParams contains the parameters of an elliptic curve.
type CurveParams struct {
Name string // the canonical name of the curve
BitSize int // the size of the underlying field
}
backend/vendor/github.com/aead/ecdh/generic.go
Generated Vendored
+121
View File
@@ -0,0 +1,121 @@
// Copyright (c) 2016 Andreas Auernhammer. All rights reserved.
// Use of this source code is governed by a license that can be
// found in the LICENSE file.
package ecdh
import (
"crypto"
"crypto/elliptic"
"crypto/rand"
"errors"
"io"
"math/big"
)
// Point represents a generic elliptic curve Point with a
// X and a Y coordinate.
type Point struct {
X, Y *big.Int
}
// Generic creates a new ecdh.KeyExchange with
// generic elliptic.Curve implementations.
func Generic(c elliptic.Curve) KeyExchange {
if c == nil {
panic("ecdh: curve is nil")
}
return genericCurve{curve: c}
}
type genericCurve struct {
curve elliptic.Curve
}
func (g genericCurve) GenerateKey(random io.Reader) (private crypto.PrivateKey, public crypto.PublicKey, err error) {
if random == nil {
random = rand.Reader
}
private, x, y, err := elliptic.GenerateKey(g.curve, random)
if err != nil {
private = nil
return
}
public = Point{X: x, Y: y}
return
}
func (g genericCurve) Params() CurveParams {
p := g.curve.Params()
return CurveParams{
Name: p.Name,
BitSize: p.BitSize,
}
}
func (g genericCurve) PublicKey(private crypto.PrivateKey) (public crypto.PublicKey) {
key, ok := checkPrivateKey(private)
if !ok {
panic("ecdh: unexpected type of private key")
}
N := g.curve.Params().N
if new(big.Int).SetBytes(key).Cmp(N) >= 0 {
panic("ecdh: private key cannot used with given curve")
}
x, y := g.curve.ScalarBaseMult(key)
public = Point{X: x, Y: y}
return
}
func (g genericCurve) Check(peersPublic crypto.PublicKey) (err error) {
key, ok := checkPublicKey(peersPublic)
if !ok {
err = errors.New("unexpected type of peers public key")
}
if !g.curve.IsOnCurve(key.X, key.Y) {
err = errors.New("peer's public key is not on curve")
}
return
}
func (g genericCurve) ComputeSecret(private crypto.PrivateKey, peersPublic crypto.PublicKey) (secret []byte) {
priKey, ok := checkPrivateKey(private)
if !ok {
panic("ecdh: unexpected type of private key")
}
pubKey, ok := checkPublicKey(peersPublic)
if !ok {
panic("ecdh: unexpected type of peers public key")
}
sX, _ := g.curve.ScalarMult(pubKey.X, pubKey.Y, priKey)
secret = sX.Bytes()
return
}
func checkPrivateKey(typeToCheck interface{}) (key []byte, ok bool) {
switch t := typeToCheck.(type) {
case []byte:
key = t
ok = true
case *[]byte:
key = *t
ok = true
}
return
}
func checkPublicKey(typeToCheck interface{}) (key Point, ok bool) {
switch t := typeToCheck.(type) {
case Point:
key = t
ok = true
case *Point:
key = *t
ok = true
}
return
}
backend/vendor/github.com/aliyun/aliyun-oss-go-sdk/LICENSE
Generated Vendored
+14
View File
@@ -0,0 +1,14 @@
Copyright (c) 2015 aliyun.com
Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated
documentation files (the "Software"), to deal in the Software without restriction, including without limitation the
rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to
permit persons to whom the Software is furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all copies or substantial portions of the
Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE
WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR
COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR
OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
backend/vendor/github.com/aliyun/aliyun-oss-go-sdk/oss/auth.go
Generated Vendored
+339
View File
@@ -0,0 +1,339 @@
package oss
import (
"bytes"
"crypto/hmac"
"crypto/sha1"
"crypto/sha256"
"encoding/base64"
"encoding/hex"
"fmt"
"hash"
"io"
"net/http"
"sort"
"strconv"
"strings"
"time"
)
// headerSorter defines the key-value structure for storing the sorted data in signHeader.
type headerSorter struct {
Keys []string
Vals []string
}
// getAdditionalHeaderKeys get exist key in http header
func (conn Conn) getAdditionalHeaderKeys(req *http.Request) ([]string, map[string]string) {
var keysList []string
keysMap := make(map[string]string)
srcKeys := make(map[string]string)
for k := range req.Header {
srcKeys[strings.ToLower(k)] = ""
}
for _, v := range conn.config.AdditionalHeaders {
if _, ok := srcKeys[strings.ToLower(v)]; ok {
keysMap[strings.ToLower(v)] = ""
}
}
for k := range keysMap {
keysList = append(keysList, k)
}
sort.Strings(keysList)
return keysList, keysMap
}
// getAdditionalHeaderKeysV4 get exist key in http header
func (conn Conn) getAdditionalHeaderKeysV4(req *http.Request) ([]string, map[string]string) {
var keysList []string
keysMap := make(map[string]string)
srcKeys := make(map[string]string)
for k := range req.Header {
srcKeys[strings.ToLower(k)] = ""
}
for _, v := range conn.config.AdditionalHeaders {
if _, ok := srcKeys[strings.ToLower(v)]; ok {
if !strings.EqualFold(v, HTTPHeaderContentMD5) && !strings.EqualFold(v, HTTPHeaderContentType) {
keysMap[strings.ToLower(v)] = ""
}
}
}
for k := range keysMap {
keysList = append(keysList, k)
}
sort.Strings(keysList)
return keysList, keysMap
}
// signHeader signs the header and sets it as the authorization header.
func (conn Conn) signHeader(req *http.Request, canonicalizedResource string, credentials Credentials) {
akIf := credentials
authorizationStr := ""
if conn.config.AuthVersion == AuthV4 {
strDay := ""
strDate := req.Header.Get(HttpHeaderOssDate)
if strDate == "" {
strDate = req.Header.Get(HTTPHeaderDate)
t, _ := time.Parse(http.TimeFormat, strDate)
strDay = t.Format("20060102")
} else {
t, _ := time.Parse(timeFormatV4, strDate)
strDay = t.Format("20060102")
}
signHeaderProduct := conn.config.GetSignProduct()
signHeaderRegion := conn.config.GetSignRegion()
additionalList, _ := conn.getAdditionalHeaderKeysV4(req)
if len(additionalList) > 0 {
authorizationFmt := "OSS4-HMAC-SHA256 Credential=%v/%v/%v/" + signHeaderProduct + "/aliyun_v4_request,AdditionalHeaders=%v,Signature=%v"
additionnalHeadersStr := strings.Join(additionalList, ";")
authorizationStr = fmt.Sprintf(authorizationFmt, akIf.GetAccessKeyID(), strDay, signHeaderRegion, additionnalHeadersStr, conn.getSignedStrV4(req, canonicalizedResource, akIf.GetAccessKeySecret(), nil))
} else {
authorizationFmt := "OSS4-HMAC-SHA256 Credential=%v/%v/%v/" + signHeaderProduct + "/aliyun_v4_request,Signature=%v"
authorizationStr = fmt.Sprintf(authorizationFmt, akIf.GetAccessKeyID(), strDay, signHeaderRegion, conn.getSignedStrV4(req, canonicalizedResource, akIf.GetAccessKeySecret(), nil))
}
} else if conn.config.AuthVersion == AuthV2 {
additionalList, _ := conn.getAdditionalHeaderKeys(req)
if len(additionalList) > 0 {
authorizationFmt := "OSS2 AccessKeyId:%v,AdditionalHeaders:%v,Signature:%v"
additionnalHeadersStr := strings.Join(additionalList, ";")
authorizationStr = fmt.Sprintf(authorizationFmt, akIf.GetAccessKeyID(), additionnalHeadersStr, conn.getSignedStr(req, canonicalizedResource, akIf.GetAccessKeySecret()))
} else {
authorizationFmt := "OSS2 AccessKeyId:%v,Signature:%v"
authorizationStr = fmt.Sprintf(authorizationFmt, akIf.GetAccessKeyID(), conn.getSignedStr(req, canonicalizedResource, akIf.GetAccessKeySecret()))
}
} else {
// Get the final authorization string
authorizationStr = "OSS " + akIf.GetAccessKeyID() + ":" + conn.getSignedStr(req, canonicalizedResource, akIf.GetAccessKeySecret())
}
// Give the parameter "Authorization" value
req.Header.Set(HTTPHeaderAuthorization, authorizationStr)
}
func (conn Conn) getSignedStr(req *http.Request, canonicalizedResource string, keySecret string) string {
// Find out the "x-oss-"'s address in header of the request
ossHeadersMap := make(map[string]string)
additionalList, additionalMap := conn.getAdditionalHeaderKeys(req)
for k, v := range req.Header {
if strings.HasPrefix(strings.ToLower(k), "x-oss-") {
ossHeadersMap[strings.ToLower(k)] = v[0]
} else if conn.config.AuthVersion == AuthV2 {
if _, ok := additionalMap[strings.ToLower(k)]; ok {
ossHeadersMap[strings.ToLower(k)] = v[0]
}
}
}
hs := newHeaderSorter(ossHeadersMap)
// Sort the ossHeadersMap by the ascending order
hs.Sort()
// Get the canonicalizedOSSHeaders
canonicalizedOSSHeaders := ""
for i := range hs.Keys {
canonicalizedOSSHeaders += hs.Keys[i] + ":" + hs.Vals[i] + "\n"
}
// Give other parameters values
// when sign URL, date is expires
date := req.Header.Get(HTTPHeaderDate)
contentType := req.Header.Get(HTTPHeaderContentType)
contentMd5 := req.Header.Get(HTTPHeaderContentMD5)
// default is v1 signature
signStr := req.Method + "\n" + contentMd5 + "\n" + contentType + "\n" + date + "\n" + canonicalizedOSSHeaders + canonicalizedResource
h := hmac.New(func() hash.Hash { return sha1.New() }, []byte(keySecret))
// v2 signature
if conn.config.AuthVersion == AuthV2 {
signStr = req.Method + "\n" + contentMd5 + "\n" + contentType + "\n" + date + "\n" + canonicalizedOSSHeaders + strings.Join(additionalList, ";") + "\n" + canonicalizedResource
h = hmac.New(func() hash.Hash { return sha256.New() }, []byte(keySecret))
}
if conn.config.LogLevel >= Debug {
conn.config.WriteLog(Debug, "[Req:%p]signStr:%s\n", req, EscapeLFString(signStr))
}
io.WriteString(h, signStr)
signedStr := base64.StdEncoding.EncodeToString(h.Sum(nil))
return signedStr
}
func (conn Conn) getSignedStrV4(req *http.Request, canonicalizedResource string, keySecret string, signingTime *time.Time) string {
// Find out the "x-oss-"'s address in header of the request
ossHeadersMap := make(map[string]string)
additionalList, additionalMap := conn.getAdditionalHeaderKeysV4(req)
for k, v := range req.Header {
lowKey := strings.ToLower(k)
if strings.EqualFold(lowKey, HTTPHeaderContentMD5) ||
strings.EqualFold(lowKey, HTTPHeaderContentType) ||
strings.HasPrefix(lowKey, "x-oss-") {
ossHeadersMap[lowKey] = strings.Trim(v[0], " ")
} else {
if _, ok := additionalMap[lowKey]; ok {
ossHeadersMap[lowKey] = strings.Trim(v[0], " ")
}
}
}
// get day,eg 20210914
//signingTime
signDate := ""
strDay := ""
if signingTime != nil {
signDate = signingTime.Format(timeFormatV4)
strDay = signingTime.Format(shortTimeFormatV4)
} else {
var t time.Time
// Required parameters
if date := req.Header.Get(HTTPHeaderDate); date != "" {
signDate = date
t, _ = time.Parse(http.TimeFormat, date)
}
if ossDate := req.Header.Get(HttpHeaderOssDate); ossDate != "" {
signDate = ossDate
t, _ = time.Parse(timeFormatV4, ossDate)
}
strDay = t.Format("20060102")
}
hs := newHeaderSorter(ossHeadersMap)
// Sort the ossHeadersMap by the ascending order
hs.Sort()
// Get the canonicalizedOSSHeaders
canonicalizedOSSHeaders := ""
for i := range hs.Keys {
canonicalizedOSSHeaders += hs.Keys[i] + ":" + hs.Vals[i] + "\n"
}
signStr := ""
// v4 signature
hashedPayload := DefaultContentSha256
if val := req.Header.Get(HttpHeaderOssContentSha256); val != "" {
hashedPayload = val
}
// subResource
resource := canonicalizedResource
subResource := ""
subPos := strings.LastIndex(canonicalizedResource, "?")
if subPos != -1 {
subResource = canonicalizedResource[subPos+1:]
resource = canonicalizedResource[0:subPos]
}
// get canonical request
canonicalReuqest := req.Method + "\n" + resource + "\n" + subResource + "\n" + canonicalizedOSSHeaders + "\n" + strings.Join(additionalList, ";") + "\n" + hashedPayload
rh := sha256.New()
io.WriteString(rh, canonicalReuqest)
hashedRequest := hex.EncodeToString(rh.Sum(nil))
if conn.config.LogLevel >= Debug {
conn.config.WriteLog(Debug, "[Req:%p]CanonicalRequest:%s\n", req, EscapeLFString(canonicalReuqest))
}
// Product & Region
signedStrV4Product := conn.config.GetSignProduct()
signedStrV4Region := conn.config.GetSignRegion()
signStr = "OSS4-HMAC-SHA256" + "\n" + signDate + "\n" + strDay + "/" + signedStrV4Region + "/" + signedStrV4Product + "/aliyun_v4_request" + "\n" + hashedRequest
if conn.config.LogLevel >= Debug {
conn.config.WriteLog(Debug, "[Req:%p]signStr:%s\n", req, EscapeLFString(signStr))
}
h1 := hmac.New(func() hash.Hash { return sha256.New() }, []byte("aliyun_v4"+keySecret))
io.WriteString(h1, strDay)
h1Key := h1.Sum(nil)
h2 := hmac.New(func() hash.Hash { return sha256.New() }, h1Key)
io.WriteString(h2, signedStrV4Region)
h2Key := h2.Sum(nil)
h3 := hmac.New(func() hash.Hash { return sha256.New() }, h2Key)
io.WriteString(h3, signedStrV4Product)
h3Key := h3.Sum(nil)
h4 := hmac.New(func() hash.Hash { return sha256.New() }, h3Key)
io.WriteString(h4, "aliyun_v4_request")
h4Key := h4.Sum(nil)
h := hmac.New(func() hash.Hash { return sha256.New() }, h4Key)
io.WriteString(h, signStr)
return fmt.Sprintf("%x", h.Sum(nil))
}
func (conn Conn) getRtmpSignedStr(bucketName, channelName, playlistName string, expiration int64, keySecret string, params map[string]interface{}) string {
if params[HTTPParamAccessKeyID] == nil {
return ""
}
canonResource := fmt.Sprintf("/%s/%s", bucketName, channelName)
canonParamsKeys := []string{}
for key := range params {
if key != HTTPParamAccessKeyID && key != HTTPParamSignature && key != HTTPParamExpires && key != HTTPParamSecurityToken {
canonParamsKeys = append(canonParamsKeys, key)
}
}
sort.Strings(canonParamsKeys)
canonParamsStr := ""
for _, key := range canonParamsKeys {
canonParamsStr = fmt.Sprintf("%s%s:%s\n", canonParamsStr, key, params[key].(string))
}
expireStr := strconv.FormatInt(expiration, 10)
signStr := expireStr + "\n" + canonParamsStr + canonResource
h := hmac.New(func() hash.Hash { return sha1.New() }, []byte(keySecret))
io.WriteString(h, signStr)
signedStr := base64.StdEncoding.EncodeToString(h.Sum(nil))
return signedStr
}
// newHeaderSorter is an additional function for function SignHeader.
func newHeaderSorter(m map[string]string) *headerSorter {
hs := &headerSorter{
Keys: make([]string, 0, len(m)),
Vals: make([]string, 0, len(m)),
}
for k, v := range m {
hs.Keys = append(hs.Keys, k)
hs.Vals = append(hs.Vals, v)
}
return hs
}
// Sort is an additional function for function SignHeader.
func (hs *headerSorter) Sort() {
sort.Sort(hs)
}
// Len is an additional function for function SignHeader.
func (hs *headerSorter) Len() int {
return len(hs.Vals)
}
// Less is an additional function for function SignHeader.
func (hs *headerSorter) Less(i, j int) bool {
return bytes.Compare([]byte(hs.Keys[i]), []byte(hs.Keys[j])) < 0
}
// Swap is an additional function for function SignHeader.
func (hs *headerSorter) Swap(i, j int) {
hs.Vals[i], hs.Vals[j] = hs.Vals[j], hs.Vals[i]
hs.Keys[i], hs.Keys[j] = hs.Keys[j], hs.Keys[i]
}
backend/vendor/github.com/aliyun/aliyun-oss-go-sdk/oss/bucket.go
Generated Vendored
+1321
View File
File diff suppressed because it is too large Load Diff
backend/vendor/github.com/aliyun/aliyun-oss-go-sdk/oss/client.go
Generated Vendored
+2956
View File
File diff suppressed because it is too large Load Diff
backend/vendor/github.com/aliyun/aliyun-oss-go-sdk/oss/conf.go
Generated Vendored
+301
View File
@@ -0,0 +1,301 @@
package oss
import (
"bytes"
"fmt"
"log"
"net"
"os"
"time"
)
// Define the level of the output log
const (
LogOff = iota
Error
Warn
Info
Debug
)
// LogTag Tag for each level of log
var LogTag = []string{"[error]", "[warn]", "[info]", "[debug]"}
// HTTPTimeout defines HTTP timeout.
type HTTPTimeout struct {
ConnectTimeout time.Duration
ReadWriteTimeout time.Duration
HeaderTimeout time.Duration
LongTimeout time.Duration
IdleConnTimeout time.Duration
}
// HTTPMaxConns defines max idle connections and max idle connections per host
type HTTPMaxConns struct {
MaxIdleConns int
MaxIdleConnsPerHost int
MaxConnsPerHost int
}
// Credentials is interface for get AccessKeyID,AccessKeySecret,SecurityToken
type Credentials interface {
GetAccessKeyID() string
GetAccessKeySecret() string
GetSecurityToken() string
}
// CredentialsProvider is interface for get Credential Info
type CredentialsProvider interface {
GetCredentials() Credentials
}
type CredentialsProviderE interface {
CredentialsProvider
GetCredentialsE() (Credentials, error)
}
type defaultCredentials struct {
config *Config
}
func (defCre *defaultCredentials) GetAccessKeyID() string {
return defCre.config.AccessKeyID
}
func (defCre *defaultCredentials) GetAccessKeySecret() string {
return defCre.config.AccessKeySecret
}
func (defCre *defaultCredentials) GetSecurityToken() string {
return defCre.config.SecurityToken
}
type defaultCredentialsProvider struct {
config *Config
}
func (defBuild *defaultCredentialsProvider) GetCredentials() Credentials {
return &defaultCredentials{config: defBuild.config}
}
type envCredentials struct {
AccessKeyId string
AccessKeySecret string
SecurityToken string
}
type EnvironmentVariableCredentialsProvider struct {
cred Credentials
}
func (credentials *envCredentials) GetAccessKeyID() string {
return credentials.AccessKeyId
}
func (credentials *envCredentials) GetAccessKeySecret() string {
return credentials.AccessKeySecret
}
func (credentials *envCredentials) GetSecurityToken() string {
return credentials.SecurityToken
}
func (defBuild *EnvironmentVariableCredentialsProvider) GetCredentials() Credentials {
var accessID, accessKey, token string
if defBuild.cred == nil {
accessID = os.Getenv("OSS_ACCESS_KEY_ID")
accessKey = os.Getenv("OSS_ACCESS_KEY_SECRET")
token = os.Getenv("OSS_SESSION_TOKEN")
} else {
accessID = defBuild.cred.GetAccessKeyID()
accessKey = defBuild.cred.GetAccessKeySecret()
token = defBuild.cred.GetSecurityToken()
}
return &envCredentials{
AccessKeyId: accessID,
AccessKeySecret: accessKey,
SecurityToken: token,
}
}
func NewEnvironmentVariableCredentialsProvider() (EnvironmentVariableCredentialsProvider, error) {
var provider EnvironmentVariableCredentialsProvider
accessID := os.Getenv("OSS_ACCESS_KEY_ID")
if accessID == "" {
return provider, fmt.Errorf("access key id is empty!")
}
accessKey := os.Getenv("OSS_ACCESS_KEY_SECRET")
if accessKey == "" {
return provider, fmt.Errorf("access key secret is empty!")
}
token := os.Getenv("OSS_SESSION_TOKEN")
envCredential := &envCredentials{
AccessKeyId: accessID,
AccessKeySecret: accessKey,
SecurityToken: token,
}
return EnvironmentVariableCredentialsProvider{
cred: envCredential,
}, nil
}
// Config defines oss configuration
type Config struct {
Endpoint string // OSS endpoint
AccessKeyID string // AccessId
AccessKeySecret string // AccessKey
RetryTimes uint // Retry count by default it's 5.
UserAgent string // SDK name/version/system information
IsDebug bool // Enable debug mode. Default is false.
Timeout uint // Timeout in seconds. By default it's 60.
SecurityToken string // STS Token
IsCname bool // If cname is in the endpoint.
IsPathStyle bool // If Path Style is in the endpoint.
HTTPTimeout HTTPTimeout // HTTP timeout
HTTPMaxConns HTTPMaxConns // Http max connections
IsUseProxy bool // Flag of using proxy.
ProxyHost string // Flag of using proxy host.
IsAuthProxy bool // Flag of needing authentication.
ProxyUser string // Proxy user
ProxyPassword string // Proxy password
IsEnableMD5 bool // Flag of enabling MD5 for upload.
MD5Threshold int64 // Memory footprint threshold for each MD5 computation (16MB is the default), in byte. When the data is more than that, temp file is used.
IsEnableCRC bool // Flag of enabling CRC for upload.
LogLevel int // Log level
Logger *log.Logger // For write log
UploadLimitSpeed int // Upload limit speed:KB/s, 0 is unlimited
UploadLimiter *OssLimiter // Bandwidth limit reader for upload
DownloadLimitSpeed int // Download limit speed:KB/s, 0 is unlimited
DownloadLimiter *OssLimiter // Bandwidth limit reader for download
CredentialsProvider CredentialsProvider // User provides interface to get AccessKeyID, AccessKeySecret, SecurityToken
LocalAddr net.Addr // local client host info
UserSetUa bool // UserAgent is set by user or not
AuthVersion AuthVersionType // v1 or v2, v4 signature,default is v1
AdditionalHeaders []string // special http headers needed to be sign
RedirectEnabled bool // only effective from go1.7 onward, enable http redirect or not
InsecureSkipVerify bool // for https, Whether to skip verifying the server certificate file
Region string // such as cn-hangzhou
CloudBoxId string //
Product string // oss or oss-cloudbox, default is oss
VerifyObjectStrict bool // a flag of verifying object name strictly. Default is enable.
}
// LimitUploadSpeed uploadSpeed:KB/s, 0 is unlimited,default is 0
func (config *Config) LimitUploadSpeed(uploadSpeed int) error {
if uploadSpeed < 0 {
return fmt.Errorf("invalid argument, the value of uploadSpeed is less than 0")
} else if uploadSpeed == 0 {
config.UploadLimitSpeed = 0
config.UploadLimiter = nil
return nil
}
var err error
config.UploadLimiter, err = GetOssLimiter(uploadSpeed)
if err == nil {
config.UploadLimitSpeed = uploadSpeed
}
return err
}
// LimitDownLoadSpeed downloadSpeed:KB/s, 0 is unlimited,default is 0
func (config *Config) LimitDownloadSpeed(downloadSpeed int) error {
if downloadSpeed < 0 {
return fmt.Errorf("invalid argument, the value of downloadSpeed is less than 0")
} else if downloadSpeed == 0 {
config.DownloadLimitSpeed = 0
config.DownloadLimiter = nil
return nil
}
var err error
config.DownloadLimiter, err = GetOssLimiter(downloadSpeed)
if err == nil {
config.DownloadLimitSpeed = downloadSpeed
}
return err
}
// WriteLog output log function
func (config *Config) WriteLog(LogLevel int, format string, a ...interface{}) {
if config.LogLevel < LogLevel || config.Logger == nil {
return
}
var logBuffer bytes.Buffer
logBuffer.WriteString(LogTag[LogLevel-1])
logBuffer.WriteString(fmt.Sprintf(format, a...))
config.Logger.Printf("%s", logBuffer.String())
}
// for get Credentials
func (config *Config) GetCredentials() Credentials {
return config.CredentialsProvider.GetCredentials()
}
// for get Sign Product
func (config *Config) GetSignProduct() string {
if config.CloudBoxId != "" {
return "oss-cloudbox"
}
return "oss"
}
// for get Sign Region
func (config *Config) GetSignRegion() string {
if config.CloudBoxId != "" {
return config.CloudBoxId
}
return config.Region
}
// getDefaultOssConfig gets the default configuration.
func getDefaultOssConfig() *Config {
config := Config{}
config.Endpoint = ""
config.AccessKeyID = ""
config.AccessKeySecret = ""
config.RetryTimes = 5
config.IsDebug = false
config.UserAgent = userAgent()
config.Timeout = 60 // Seconds
config.SecurityToken = ""
config.IsCname = false
config.IsPathStyle = false
config.HTTPTimeout.ConnectTimeout = time.Second * 30 // 30s
config.HTTPTimeout.ReadWriteTimeout = time.Second * 60 // 60s
config.HTTPTimeout.HeaderTimeout = time.Second * 60 // 60s
config.HTTPTimeout.LongTimeout = time.Second * 300 // 300s
config.HTTPTimeout.IdleConnTimeout = time.Second * 50 // 50s
config.HTTPMaxConns.MaxIdleConns = 100
config.HTTPMaxConns.MaxIdleConnsPerHost = 100
config.IsUseProxy = false
config.ProxyHost = ""
config.IsAuthProxy = false
config.ProxyUser = ""
config.ProxyPassword = ""
config.MD5Threshold = 16 * 1024 * 1024 // 16MB
config.IsEnableMD5 = false
config.IsEnableCRC = true
config.LogLevel = LogOff
config.Logger = log.New(os.Stdout, "", log.LstdFlags)
provider := &defaultCredentialsProvider{config: &config}
config.CredentialsProvider = provider
config.AuthVersion = AuthV1
config.RedirectEnabled = true
config.InsecureSkipVerify = false
config.Product = "oss"
config.VerifyObjectStrict = true
return &config
}
backend/vendor/github.com/aliyun/aliyun-oss-go-sdk/oss/conn.go
Generated Vendored
+1021
View File
File diff suppressed because it is too large Load Diff
backend/vendor/github.com/aliyun/aliyun-oss-go-sdk/oss/const.go
Generated Vendored
+273
View File
@@ -0,0 +1,273 @@
package oss
import "os"
// ACLType bucket/object ACL
type ACLType string
const (
// ACLPrivate definition : private read and write
ACLPrivate ACLType = "private"
// ACLPublicRead definition : public read and private write
ACLPublicRead ACLType = "public-read"
// ACLPublicReadWrite definition : public read and public write
ACLPublicReadWrite ACLType = "public-read-write"
// ACLDefault Object. It's only applicable for object.
ACLDefault ACLType = "default"
)
// bucket versioning status
type VersioningStatus string
const (
// Versioning Status definition: Enabled
VersionEnabled VersioningStatus = "Enabled"
// Versioning Status definition: Suspended
VersionSuspended VersioningStatus = "Suspended"
)
// MetadataDirectiveType specifying whether use the metadata of source object when copying object.
type MetadataDirectiveType string
const (
// MetaCopy the target object's metadata is copied from the source one
MetaCopy MetadataDirectiveType = "COPY"
// MetaReplace the target object's metadata is created as part of the copy request (not same as the source one)
MetaReplace MetadataDirectiveType = "REPLACE"
)
// TaggingDirectiveType specifying whether use the tagging of source object when copying object.
type TaggingDirectiveType string
const (
// TaggingCopy the target object's tagging is copied from the source one
TaggingCopy TaggingDirectiveType = "COPY"
// TaggingReplace the target object's tagging is created as part of the copy request (not same as the source one)
TaggingReplace TaggingDirectiveType = "REPLACE"
)
// AlgorithmType specifying the server side encryption algorithm name
type AlgorithmType string
const (
KMSAlgorithm AlgorithmType = "KMS"
AESAlgorithm AlgorithmType = "AES256"
SM4Algorithm AlgorithmType = "SM4"
)
// StorageClassType bucket storage type
type StorageClassType string
const (
// StorageStandard standard
StorageStandard StorageClassType = "Standard"
// StorageIA infrequent access
StorageIA StorageClassType = "IA"
// StorageArchive archive
StorageArchive StorageClassType = "Archive"
// StorageColdArchive cold archive
StorageColdArchive StorageClassType = "ColdArchive"
// StorageDeepColdArchive deep cold archive
StorageDeepColdArchive StorageClassType = "DeepColdArchive"
)
//RedundancyType bucket data Redundancy type
type DataRedundancyType string
const (
// RedundancyLRS Local redundancy, default value
RedundancyLRS DataRedundancyType = "LRS"
// RedundancyZRS Same city redundancy
RedundancyZRS DataRedundancyType = "ZRS"
)
//ObjecthashFuncType
type ObjecthashFuncType string
const (
HashFuncSha1 ObjecthashFuncType = "SHA-1"
HashFuncSha256 ObjecthashFuncType = "SHA-256"
)
// PayerType the type of request payer
type PayerType string
const (
// Requester the requester who send the request
Requester PayerType = "Requester"
// BucketOwner the requester who send the request
BucketOwner PayerType = "BucketOwner"
)
//RestoreMode the restore mode for coldArchive object
type RestoreMode string
const (
//RestoreExpedited object will be restored in 1 hour
RestoreExpedited RestoreMode = "Expedited"
//RestoreStandard object will be restored in 2-5 hours
RestoreStandard RestoreMode = "Standard"
//RestoreBulk object will be restored in 5-10 hours
RestoreBulk RestoreMode = "Bulk"
)
// HTTPMethod HTTP request method
type HTTPMethod string
const (
// HTTPGet HTTP GET
HTTPGet HTTPMethod = "GET"
// HTTPPut HTTP PUT
HTTPPut HTTPMethod = "PUT"
// HTTPHead HTTP HEAD
HTTPHead HTTPMethod = "HEAD"
// HTTPPost HTTP POST
HTTPPost HTTPMethod = "POST"
// HTTPDelete HTTP DELETE
HTTPDelete HTTPMethod = "DELETE"
)
// HTTP headers
const (
HTTPHeaderAcceptEncoding string = "Accept-Encoding"
HTTPHeaderAuthorization = "Authorization"
HTTPHeaderCacheControl = "Cache-Control"
HTTPHeaderContentDisposition = "Content-Disposition"
HTTPHeaderContentEncoding = "Content-Encoding"
HTTPHeaderContentLength = "Content-Length"
HTTPHeaderContentMD5 = "Content-MD5"
HTTPHeaderContentType = "Content-Type"
HTTPHeaderContentLanguage = "Content-Language"
HTTPHeaderDate = "Date"
HTTPHeaderEtag = "ETag"
HTTPHeaderExpires = "Expires"
HTTPHeaderHost = "Host"
HTTPHeaderLastModified = "Last-Modified"
HTTPHeaderRange = "Range"
HTTPHeaderLocation = "Location"
HTTPHeaderOrigin = "Origin"
HTTPHeaderServer = "Server"
HTTPHeaderUserAgent = "User-Agent"
HTTPHeaderIfModifiedSince = "If-Modified-Since"
HTTPHeaderIfUnmodifiedSince = "If-Unmodified-Since"
HTTPHeaderIfMatch = "If-Match"
HTTPHeaderIfNoneMatch = "If-None-Match"
HTTPHeaderACReqMethod = "Access-Control-Request-Method"
HTTPHeaderACReqHeaders = "Access-Control-Request-Headers"
HTTPHeaderOssACL = "X-Oss-Acl"
HTTPHeaderOssMetaPrefix = "X-Oss-Meta-"
HTTPHeaderOssObjectACL = "X-Oss-Object-Acl"
HTTPHeaderOssSecurityToken = "X-Oss-Security-Token"
HTTPHeaderOssServerSideEncryption = "X-Oss-Server-Side-Encryption"
HTTPHeaderOssServerSideEncryptionKeyID = "X-Oss-Server-Side-Encryption-Key-Id"
HTTPHeaderOssServerSideDataEncryption = "X-Oss-Server-Side-Data-Encryption"
HTTPHeaderSSECAlgorithm = "X-Oss-Server-Side-Encryption-Customer-Algorithm"
HTTPHeaderSSECKey = "X-Oss-Server-Side-Encryption-Customer-Key"
HTTPHeaderSSECKeyMd5 = "X-Oss-Server-Side-Encryption-Customer-Key-MD5"
HTTPHeaderOssCopySource = "X-Oss-Copy-Source"
HTTPHeaderOssCopySourceRange = "X-Oss-Copy-Source-Range"
HTTPHeaderOssCopySourceIfMatch = "X-Oss-Copy-Source-If-Match"
HTTPHeaderOssCopySourceIfNoneMatch = "X-Oss-Copy-Source-If-None-Match"
HTTPHeaderOssCopySourceIfModifiedSince = "X-Oss-Copy-Source-If-Modified-Since"
HTTPHeaderOssCopySourceIfUnmodifiedSince = "X-Oss-Copy-Source-If-Unmodified-Since"
HTTPHeaderOssMetadataDirective = "X-Oss-Metadata-Directive"
HTTPHeaderOssNextAppendPosition = "X-Oss-Next-Append-Position"
HTTPHeaderOssRequestID = "X-Oss-Request-Id"
HTTPHeaderOssCRC64 = "X-Oss-Hash-Crc64ecma"
HTTPHeaderOssSymlinkTarget = "X-Oss-Symlink-Target"
HTTPHeaderOssStorageClass = "X-Oss-Storage-Class"
HTTPHeaderOssCallback = "X-Oss-Callback"
HTTPHeaderOssCallbackVar = "X-Oss-Callback-Var"
HTTPHeaderOssRequester = "X-Oss-Request-Payer"
HTTPHeaderOssTagging = "X-Oss-Tagging"
HTTPHeaderOssTaggingDirective = "X-Oss-Tagging-Directive"
HTTPHeaderOssTrafficLimit = "X-Oss-Traffic-Limit"
HTTPHeaderOssForbidOverWrite = "X-Oss-Forbid-Overwrite"
HTTPHeaderOssRangeBehavior = "X-Oss-Range-Behavior"
HTTPHeaderOssTaskID = "X-Oss-Task-Id"
HTTPHeaderOssHashCtx = "X-Oss-Hash-Ctx"
HTTPHeaderOssMd5Ctx = "X-Oss-Md5-Ctx"
HTTPHeaderAllowSameActionOverLap = "X-Oss-Allow-Same-Action-Overlap"
HttpHeaderOssDate = "X-Oss-Date"
HttpHeaderOssContentSha256 = "X-Oss-Content-Sha256"
HttpHeaderOssNotification = "X-Oss-Notification"
HTTPHeaderOssEc = "X-Oss-Ec"
HTTPHeaderOssErr = "X-Oss-Err"
)
// HTTP Param
const (
HTTPParamExpires = "Expires"
HTTPParamAccessKeyID = "OSSAccessKeyId"
HTTPParamSignature = "Signature"
HTTPParamSecurityToken = "security-token"
HTTPParamPlaylistName = "playlistName"
HTTPParamSignatureVersion = "x-oss-signature-version"
HTTPParamExpiresV2 = "x-oss-expires"
HTTPParamAccessKeyIDV2 = "x-oss-access-key-id"
HTTPParamSignatureV2 = "x-oss-signature"
HTTPParamAdditionalHeadersV2 = "x-oss-additional-headers"
HTTPParamCredential = "x-oss-credential"
HTTPParamDate = "x-oss-date"
HTTPParamOssSecurityToken = "x-oss-security-token"
)
// Other constants
const (
MaxPartSize = 5 * 1024 * 1024 * 1024 // Max part size, 5GB
MinPartSize = 100 * 1024 // Min part size, 100KB
FilePermMode = os.FileMode(0664) // Default file permission
TempFilePrefix = "oss-go-temp-" // Temp file prefix
TempFileSuffix = ".temp" // Temp file suffix
CheckpointFileSuffix = ".cp" // Checkpoint file suffix
NullVersion = "null"
DefaultContentSha256 = "UNSIGNED-PAYLOAD" // for v4 signature
Version = "v3.0.2" // Go SDK version
)
// FrameType
const (
DataFrameType = 8388609
ContinuousFrameType = 8388612
EndFrameType = 8388613
MetaEndFrameCSVType = 8388614
MetaEndFrameJSONType = 8388615
)
// AuthVersion the version of auth
type AuthVersionType string
const (
// AuthV1 v1
AuthV1 AuthVersionType = "v1"
// AuthV2 v2
AuthV2 AuthVersionType = "v2"
// AuthV4 v4
AuthV4 AuthVersionType = "v4"
)
backend/vendor/github.com/aliyun/aliyun-oss-go-sdk/oss/crc.go
Generated Vendored
+123
View File
@@ -0,0 +1,123 @@
package oss
import (
"hash"
"hash/crc64"
)
// digest represents the partial evaluation of a checksum.
type digest struct {
crc uint64
tab *crc64.Table
}
// NewCRC creates a new hash.Hash64 computing the CRC64 checksum
// using the polynomial represented by the Table.
func NewCRC(tab *crc64.Table, init uint64) hash.Hash64 { return &digest{init, tab} }
// Size returns the number of bytes sum will return.
func (d *digest) Size() int { return crc64.Size }
// BlockSize returns the hash's underlying block size.
// The Write method must be able to accept any amount
// of data, but it may operate more efficiently if all writes
// are a multiple of the block size.
func (d *digest) BlockSize() int { return 1 }
// Reset resets the hash to its initial state.
func (d *digest) Reset() { d.crc = 0 }
// Write (via the embedded io.Writer interface) adds more data to the running hash.
// It never returns an error.
func (d *digest) Write(p []byte) (n int, err error) {
d.crc = crc64.Update(d.crc, d.tab, p)
return len(p), nil
}
// Sum64 returns CRC64 value.
func (d *digest) Sum64() uint64 { return d.crc }
// Sum returns hash value.
func (d *digest) Sum(in []byte) []byte {
s := d.Sum64()
return append(in, byte(s>>56), byte(s>>48), byte(s>>40), byte(s>>32), byte(s>>24), byte(s>>16), byte(s>>8), byte(s))
}
// gf2Dim dimension of GF(2) vectors (length of CRC)
const gf2Dim int = 64
func gf2MatrixTimes(mat []uint64, vec uint64) uint64 {
var sum uint64
for i := 0; vec != 0; i++ {
if vec&1 != 0 {
sum ^= mat[i]
}
vec >>= 1
}
return sum
}
func gf2MatrixSquare(square []uint64, mat []uint64) {
for n := 0; n < gf2Dim; n++ {
square[n] = gf2MatrixTimes(mat, mat[n])
}
}
// CRC64Combine combines CRC64
func CRC64Combine(crc1 uint64, crc2 uint64, len2 uint64) uint64 {
var even [gf2Dim]uint64 // Even-power-of-two zeros operator
var odd [gf2Dim]uint64 // Odd-power-of-two zeros operator
// Degenerate case
if len2 == 0 {
return crc1
}
// Put operator for one zero bit in odd
odd[0] = crc64.ECMA // CRC64 polynomial
var row uint64 = 1
for n := 1; n < gf2Dim; n++ {
odd[n] = row
row <<= 1
}
// Put operator for two zero bits in even
gf2MatrixSquare(even[:], odd[:])
// Put operator for four zero bits in odd
gf2MatrixSquare(odd[:], even[:])
// Apply len2 zeros to crc1, first square will put the operator for one zero byte, eight zero bits, in even
for {
// Apply zeros operator for this bit of len2
gf2MatrixSquare(even[:], odd[:])
if len2&1 != 0 {
crc1 = gf2MatrixTimes(even[:], crc1)
}
len2 >>= 1
// If no more bits set, then done
if len2 == 0 {
break
}
// Another iteration of the loop with odd and even swapped
gf2MatrixSquare(odd[:], even[:])
if len2&1 != 0 {
crc1 = gf2MatrixTimes(odd[:], crc1)
}
len2 >>= 1
// If no more bits set, then done
if len2 == 0 {
break
}
}
// Return combined CRC
crc1 ^= crc2
return crc1
}
backend/vendor/github.com/aliyun/aliyun-oss-go-sdk/oss/download.go
Generated Vendored
+567
View File
@@ -0,0 +1,567 @@
package oss
import (
"crypto/md5"
"encoding/base64"
"encoding/json"
"errors"
"fmt"
"hash"
"hash/crc64"
"io"
"io/ioutil"
"net/http"
"os"
"path/filepath"
"strconv"
"time"
)
// DownloadFile downloads files with multipart download.
//
// objectKey the object key.
// filePath the local file to download from objectKey in OSS.
// partSize the part size in bytes.
// options object's constraints, check out GetObject for the reference.
//
// error it's nil when the call succeeds, otherwise it's an error object.
//
func (bucket Bucket) DownloadFile(objectKey, filePath string, partSize int64, options ...Option) error {
if partSize < 1 {
return errors.New("oss: part size smaller than 1")
}
uRange, err := GetRangeConfig(options)
if err != nil {
return err
}
cpConf := getCpConfig(options)
routines := getRoutines(options)
var strVersionId string
versionId, _ := FindOption(options, "versionId", nil)
if versionId != nil {
strVersionId = versionId.(string)
}
if cpConf != nil && cpConf.IsEnable {
cpFilePath := getDownloadCpFilePath(cpConf, bucket.BucketName, objectKey, strVersionId, filePath)
if cpFilePath != "" {
return bucket.downloadFileWithCp(objectKey, filePath, partSize, options, cpFilePath, routines, uRange)
}
}
return bucket.downloadFile(objectKey, filePath, partSize, options, routines, uRange)
}
func getDownloadCpFilePath(cpConf *cpConfig, srcBucket, srcObject, versionId, destFile string) string {
if cpConf.FilePath == "" && cpConf.DirPath != "" {
src := fmt.Sprintf("oss://%v/%v", srcBucket, srcObject)
absPath, _ := filepath.Abs(destFile)
cpFileName := getCpFileName(src, absPath, versionId)
cpConf.FilePath = cpConf.DirPath + string(os.PathSeparator) + cpFileName
}
return cpConf.FilePath
}
// downloadWorkerArg is download worker's parameters
type downloadWorkerArg struct {
bucket *Bucket
key string
filePath string
options []Option
hook downloadPartHook
enableCRC bool
}
// downloadPartHook is hook for test
type downloadPartHook func(part downloadPart) error
var downloadPartHooker downloadPartHook = defaultDownloadPartHook
func defaultDownloadPartHook(part downloadPart) error {
return nil
}
// defaultDownloadProgressListener defines default ProgressListener, shields the ProgressListener in options of GetObject.
type defaultDownloadProgressListener struct {
}
// ProgressChanged no-ops
func (listener *defaultDownloadProgressListener) ProgressChanged(event *ProgressEvent) {
}
// downloadWorker
func downloadWorker(id int, arg downloadWorkerArg, jobs <-chan downloadPart, results chan<- downloadPart, failed chan<- error, die <-chan bool) {
for part := range jobs {
if err := arg.hook(part); err != nil {
failed <- err
break
}
// Resolve options
r := Range(part.Start, part.End)
p := Progress(&defaultDownloadProgressListener{})
var respHeader http.Header
opts := make([]Option, len(arg.options)+3)
// Append orderly, can not be reversed!
opts = append(opts, arg.options...)
opts = append(opts, r, p, GetResponseHeader(&respHeader))
rd, err := arg.bucket.GetObject(arg.key, opts...)
if err != nil {
failed <- err
break
}
defer rd.Close()
var crcCalc hash.Hash64
if arg.enableCRC {
crcCalc = crc64.New(CrcTable())
contentLen := part.End - part.Start + 1
rd = ioutil.NopCloser(TeeReader(rd, crcCalc, contentLen, nil, nil))
}
defer rd.Close()
select {
case <-die:
return
default:
}
fd, err := os.OpenFile(arg.filePath, os.O_WRONLY, FilePermMode)
if err != nil {
failed <- err
break
}
_, err = fd.Seek(part.Start-part.Offset, os.SEEK_SET)
if err != nil {
fd.Close()
failed <- err
break
}
startT := time.Now().UnixNano() / 1000 / 1000 / 1000
_, err = io.Copy(fd, rd)
endT := time.Now().UnixNano() / 1000 / 1000 / 1000
if err != nil {
arg.bucket.Client.Config.WriteLog(Debug, "download part error,cost:%d second,part number:%d,request id:%s,error:%s.\n", endT-startT, part.Index, GetRequestId(respHeader), err.Error())
fd.Close()
failed <- err
break
}
if arg.enableCRC {
part.CRC64 = crcCalc.Sum64()
}
fd.Close()
results <- part
}
}
// downloadScheduler
func downloadScheduler(jobs chan downloadPart, parts []downloadPart) {
for _, part := range parts {
jobs <- part
}
close(jobs)
}
// downloadPart defines download part
type downloadPart struct {
Index int // Part number, starting from 0
Start int64 // Start index
End int64 // End index
Offset int64 // Offset
CRC64 uint64 // CRC check value of part
}
// getDownloadParts gets download parts
func getDownloadParts(objectSize, partSize int64, uRange *UnpackedRange) []downloadPart {
parts := []downloadPart{}
part := downloadPart{}
i := 0
start, end := AdjustRange(uRange, objectSize)
for offset := start; offset < end; offset += partSize {
part.Index = i
part.Start = offset
part.End = GetPartEnd(offset, end, partSize)
part.Offset = start
part.CRC64 = 0
parts = append(parts, part)
i++
}
return parts
}
// getObjectBytes gets object bytes length
func getObjectBytes(parts []downloadPart) int64 {
var ob int64
for _, part := range parts {
ob += (part.End - part.Start + 1)
}
return ob
}
// combineCRCInParts caculates the total CRC of continuous parts
func combineCRCInParts(dps []downloadPart) uint64 {
if dps == nil || len(dps) == 0 {
return 0
}
crc := dps[0].CRC64
for i := 1; i < len(dps); i++ {
crc = CRC64Combine(crc, dps[i].CRC64, (uint64)(dps[i].End-dps[i].Start+1))
}
return crc
}
// downloadFile downloads file concurrently without checkpoint.
func (bucket Bucket) downloadFile(objectKey, filePath string, partSize int64, options []Option, routines int, uRange *UnpackedRange) error {
tempFilePath := filePath + TempFileSuffix
listener := GetProgressListener(options)
// If the file does not exist, create one. If exists, the download will overwrite it.
fd, err := os.OpenFile(tempFilePath, os.O_WRONLY|os.O_CREATE, FilePermMode)
if err != nil {
return err
}
fd.Close()
// Get the object detailed meta for object whole size
// must delete header:range to get whole object size
skipOptions := DeleteOption(options, HTTPHeaderRange)
meta, err := bucket.GetObjectDetailedMeta(objectKey, skipOptions...)
if err != nil {
return err
}
objectSize, err := strconv.ParseInt(meta.Get(HTTPHeaderContentLength), 10, 64)
if err != nil {
return err
}
enableCRC := false
expectedCRC := (uint64)(0)
if bucket.GetConfig().IsEnableCRC && meta.Get(HTTPHeaderOssCRC64) != "" {
if uRange == nil || (!uRange.HasStart && !uRange.HasEnd) {
enableCRC = true
expectedCRC, _ = strconv.ParseUint(meta.Get(HTTPHeaderOssCRC64), 10, 64)
}
}
// Get the parts of the file
parts := getDownloadParts(objectSize, partSize, uRange)
jobs := make(chan downloadPart, len(parts))
results := make(chan downloadPart, len(parts))
failed := make(chan error)
die := make(chan bool)
var completedBytes int64
totalBytes := getObjectBytes(parts)
event := newProgressEvent(TransferStartedEvent, 0, totalBytes, 0)
publishProgress(listener, event)
// Start the download workers
arg := downloadWorkerArg{&bucket, objectKey, tempFilePath, options, downloadPartHooker, enableCRC}
for w := 1; w <= routines; w++ {
go downloadWorker(w, arg, jobs, results, failed, die)
}
// Download parts concurrently
go downloadScheduler(jobs, parts)
// Waiting for parts download finished
completed := 0
for completed < len(parts) {
select {
case part := <-results:
completed++
downBytes := (part.End - part.Start + 1)
completedBytes += downBytes
parts[part.Index].CRC64 = part.CRC64
event = newProgressEvent(TransferDataEvent, completedBytes, totalBytes, downBytes)
publishProgress(listener, event)
case err := <-failed:
close(die)
event = newProgressEvent(TransferFailedEvent, completedBytes, totalBytes, 0)
publishProgress(listener, event)
return err
}
if completed >= len(parts) {
break
}
}
event = newProgressEvent(TransferCompletedEvent, completedBytes, totalBytes, 0)
publishProgress(listener, event)
if enableCRC {
actualCRC := combineCRCInParts(parts)
err = CheckDownloadCRC(actualCRC, expectedCRC)
if err != nil {
return err
}
}
return os.Rename(tempFilePath, filePath)
}
// ----- Concurrent download with chcekpoint -----
const downloadCpMagic = "92611BED-89E2-46B6-89E5-72F273D4B0A3"
type downloadCheckpoint struct {
Magic string // Magic
MD5 string // Checkpoint content MD5
FilePath string // Local file
Object string // Key
ObjStat objectStat // Object status
Parts []downloadPart // All download parts
PartStat []bool // Parts' download status
Start int64 // Start point of the file
End int64 // End point of the file
enableCRC bool // Whether has CRC check
CRC uint64 // CRC check value
}
type objectStat struct {
Size int64 // Object size
LastModified string // Last modified time
Etag string // Etag
}
// isValid flags of checkpoint data is valid. It returns true when the data is valid and the checkpoint is valid and the object is not updated.
func (cp downloadCheckpoint) isValid(meta http.Header, uRange *UnpackedRange) (bool, error) {
// Compare the CP's Magic and the MD5
cpb := cp
cpb.MD5 = ""
js, _ := json.Marshal(cpb)
sum := md5.Sum(js)
b64 := base64.StdEncoding.EncodeToString(sum[:])
if cp.Magic != downloadCpMagic || b64 != cp.MD5 {
return false, nil
}
objectSize, err := strconv.ParseInt(meta.Get(HTTPHeaderContentLength), 10, 64)
if err != nil {
return false, err
}
// Compare the object size, last modified time and etag
if cp.ObjStat.Size != objectSize ||
cp.ObjStat.LastModified != meta.Get(HTTPHeaderLastModified) ||
cp.ObjStat.Etag != meta.Get(HTTPHeaderEtag) {
return false, nil
}
// Check the download range
if uRange != nil {
start, end := AdjustRange(uRange, objectSize)
if start != cp.Start || end != cp.End {
return false, nil
}
}
return true, nil
}
// load checkpoint from local file
func (cp *downloadCheckpoint) load(filePath string) error {
contents, err := ioutil.ReadFile(filePath)
if err != nil {
return err
}
err = json.Unmarshal(contents, cp)
return err
}
// dump funciton dumps to file
func (cp *downloadCheckpoint) dump(filePath string) error {
bcp := *cp
// Calculate MD5
bcp.MD5 = ""
js, err := json.Marshal(bcp)
if err != nil {
return err
}
sum := md5.Sum(js)
b64 := base64.StdEncoding.EncodeToString(sum[:])
bcp.MD5 = b64
// Serialize
js, err = json.Marshal(bcp)
if err != nil {
return err
}
// Dump
return ioutil.WriteFile(filePath, js, FilePermMode)
}
// todoParts gets unfinished parts
func (cp downloadCheckpoint) todoParts() []downloadPart {
dps := []downloadPart{}
for i, ps := range cp.PartStat {
if !ps {
dps = append(dps, cp.Parts[i])
}
}
return dps
}
// getCompletedBytes gets completed size
func (cp downloadCheckpoint) getCompletedBytes() int64 {
var completedBytes int64
for i, part := range cp.Parts {
if cp.PartStat[i] {
completedBytes += (part.End - part.Start + 1)
}
}
return completedBytes
}
// prepare initiates download tasks
func (cp *downloadCheckpoint) prepare(meta http.Header, bucket *Bucket, objectKey, filePath string, partSize int64, uRange *UnpackedRange) error {
// CP
cp.Magic = downloadCpMagic
cp.FilePath = filePath
cp.Object = objectKey
objectSize, err := strconv.ParseInt(meta.Get(HTTPHeaderContentLength), 10, 64)
if err != nil {
return err
}
cp.ObjStat.Size = objectSize
cp.ObjStat.LastModified = meta.Get(HTTPHeaderLastModified)
cp.ObjStat.Etag = meta.Get(HTTPHeaderEtag)
if bucket.GetConfig().IsEnableCRC && meta.Get(HTTPHeaderOssCRC64) != "" {
if uRange == nil || (!uRange.HasStart && !uRange.HasEnd) {
cp.enableCRC = true
cp.CRC, _ = strconv.ParseUint(meta.Get(HTTPHeaderOssCRC64), 10, 64)
}
}
// Parts
cp.Parts = getDownloadParts(objectSize, partSize, uRange)
cp.PartStat = make([]bool, len(cp.Parts))
for i := range cp.PartStat {
cp.PartStat[i] = false
}
return nil
}
func (cp *downloadCheckpoint) complete(cpFilePath, downFilepath string) error {
err := os.Rename(downFilepath, cp.FilePath)
if err != nil {
return err
}
return os.Remove(cpFilePath)
}
// downloadFileWithCp downloads files with checkpoint.
func (bucket Bucket) downloadFileWithCp(objectKey, filePath string, partSize int64, options []Option, cpFilePath string, routines int, uRange *UnpackedRange) error {
tempFilePath := filePath + TempFileSuffix
listener := GetProgressListener(options)
// Load checkpoint data.
dcp := downloadCheckpoint{}
err := dcp.load(cpFilePath)
if err != nil {
os.Remove(cpFilePath)
}
// Get the object detailed meta for object whole size
// must delete header:range to get whole object size
skipOptions := DeleteOption(options, HTTPHeaderRange)
meta, err := bucket.GetObjectDetailedMeta(objectKey, skipOptions...)
if err != nil {
return err
}
// Load error or data invalid. Re-initialize the download.
valid, err := dcp.isValid(meta, uRange)
if err != nil || !valid {
if err = dcp.prepare(meta, &bucket, objectKey, filePath, partSize, uRange); err != nil {
return err
}
os.Remove(cpFilePath)
}
// Create the file if not exists. Otherwise the parts download will overwrite it.
fd, err := os.OpenFile(tempFilePath, os.O_WRONLY|os.O_CREATE, FilePermMode)
if err != nil {
return err
}
fd.Close()
// Unfinished parts
parts := dcp.todoParts()
jobs := make(chan downloadPart, len(parts))
results := make(chan downloadPart, len(parts))
failed := make(chan error)
die := make(chan bool)
completedBytes := dcp.getCompletedBytes()
event := newProgressEvent(TransferStartedEvent, completedBytes, dcp.ObjStat.Size, 0)
publishProgress(listener, event)
// Start the download workers routine
arg := downloadWorkerArg{&bucket, objectKey, tempFilePath, options, downloadPartHooker, dcp.enableCRC}
for w := 1; w <= routines; w++ {
go downloadWorker(w, arg, jobs, results, failed, die)
}
// Concurrently downloads parts
go downloadScheduler(jobs, parts)
// Wait for the parts download finished
completed := 0
for completed < len(parts) {
select {
case part := <-results:
completed++
dcp.PartStat[part.Index] = true
dcp.Parts[part.Index].CRC64 = part.CRC64
dcp.dump(cpFilePath)
downBytes := (part.End - part.Start + 1)
completedBytes += downBytes
event = newProgressEvent(TransferDataEvent, completedBytes, dcp.ObjStat.Size, downBytes)
publishProgress(listener, event)
case err := <-failed:
close(die)
event = newProgressEvent(TransferFailedEvent, completedBytes, dcp.ObjStat.Size, 0)
publishProgress(listener, event)
return err
}
if completed >= len(parts) {
break
}
}
event = newProgressEvent(TransferCompletedEvent, completedBytes, dcp.ObjStat.Size, 0)
publishProgress(listener, event)
if dcp.enableCRC {
actualCRC := combineCRCInParts(dcp.Parts)
err = CheckDownloadCRC(actualCRC, dcp.CRC)
if err != nil {
return err
}
}
return dcp.complete(cpFilePath, tempFilePath)
}
backend/vendor/github.com/aliyun/aliyun-oss-go-sdk/oss/error.go
Generated Vendored
+136
View File
@@ -0,0 +1,136 @@
package oss
import (
"encoding/xml"
"fmt"
"io/ioutil"
"net/http"
"strconv"
"strings"
)
// ServiceError contains fields of the error response from Oss Service REST API.
type ServiceError struct {
XMLName xml.Name `xml:"Error"`
Code string `xml:"Code"` // The error code returned from OSS to the caller
Message string `xml:"Message"` // The detail error message from OSS
RequestID string `xml:"RequestId"` // The UUID used to uniquely identify the request
HostID string `xml:"HostId"` // The OSS server cluster's Id
Endpoint string `xml:"Endpoint"`
Ec string `xml:"EC"`
RawMessage string // The raw messages from OSS
StatusCode int // HTTP status code
}
// Error implements interface error
func (e ServiceError) Error() string {
errorStr := fmt.Sprintf("oss: service returned error: StatusCode=%d, ErrorCode=%s, ErrorMessage=\"%s\", RequestId=%s", e.StatusCode, e.Code, e.Message, e.RequestID)
if len(e.Endpoint) > 0 {
errorStr = fmt.Sprintf("%s, Endpoint=%s", errorStr, e.Endpoint)
}
if len(e.Ec) > 0 {
errorStr = fmt.Sprintf("%s, Ec=%s", errorStr, e.Ec)
}
return errorStr
}
// UnexpectedStatusCodeError is returned when a storage service responds with neither an error
// nor with an HTTP status code indicating success.
type UnexpectedStatusCodeError struct {
allowed []int // The expected HTTP stats code returned from OSS
got int // The actual HTTP status code from OSS
}
// Error implements interface error
func (e UnexpectedStatusCodeError) Error() string {
s := func(i int) string { return fmt.Sprintf("%d %s", i, http.StatusText(i)) }
got := s(e.got)
expected := []string{}
for _, v := range e.allowed {
expected = append(expected, s(v))
}
return fmt.Sprintf("oss: status code from service response is %s; was expecting %s",
got, strings.Join(expected, " or "))
}
// Got is the actual status code returned by oss.
func (e UnexpectedStatusCodeError) Got() int {
return e.got
}
// CheckRespCode returns UnexpectedStatusError if the given response code is not
// one of the allowed status codes; otherwise nil.
func CheckRespCode(respCode int, allowed []int) error {
for _, v := range allowed {
if respCode == v {
return nil
}
}
return UnexpectedStatusCodeError{allowed, respCode}
}
// CheckCallbackResp return error if the given response code is not 200
func CheckCallbackResp(resp *Response) error {
var err error
contentLengthStr := resp.Headers.Get("Content-Length")
contentLength, _ := strconv.Atoi(contentLengthStr)
var bodyBytes []byte
if contentLength > 0 {
bodyBytes, _ = ioutil.ReadAll(resp.Body)
}
if len(bodyBytes) > 0 {
srvErr, errIn := serviceErrFromXML(bodyBytes, resp.StatusCode,
resp.Headers.Get(HTTPHeaderOssRequestID))
if errIn != nil {
if len(resp.Headers.Get(HTTPHeaderOssEc)) > 0 {
err = fmt.Errorf("unknown response body, status code = %d, RequestId = %s, ec = %s", resp.StatusCode, resp.Headers.Get(HTTPHeaderOssRequestID), resp.Headers.Get(HTTPHeaderOssEc))
} else {
err = fmt.Errorf("unknown response body, status code= %d, RequestId = %s", resp.StatusCode, resp.Headers.Get(HTTPHeaderOssRequestID))
}
} else {
err = srvErr
}
}
return err
}
func tryConvertServiceError(data []byte, resp *Response, def error) (err error) {
err = def
if len(data) > 0 {
srvErr, errIn := serviceErrFromXML(data, resp.StatusCode, resp.Headers.Get(HTTPHeaderOssRequestID))
if errIn == nil {
err = srvErr
}
}
return err
}
// CRCCheckError is returned when crc check is inconsistent between client and server
type CRCCheckError struct {
clientCRC uint64 // Calculated CRC64 in client
serverCRC uint64 // Calculated CRC64 in server
operation string // Upload operations such as PutObject/AppendObject/UploadPart, etc
requestID string // The request id of this operation
}
// Error implements interface error
func (e CRCCheckError) Error() string {
return fmt.Sprintf("oss: the crc of %s is inconsistent, client %d but server %d; request id is %s",
e.operation, e.clientCRC, e.serverCRC, e.requestID)
}
func CheckDownloadCRC(clientCRC, serverCRC uint64) error {
if clientCRC == serverCRC {
return nil
}
return CRCCheckError{clientCRC, serverCRC, "DownloadFile", ""}
}
func CheckCRC(resp *Response, operation string) error {
if resp.Headers.Get(HTTPHeaderOssCRC64) == "" || resp.ClientCRC == resp.ServerCRC {
return nil
}
return CRCCheckError{resp.ClientCRC, resp.ServerCRC, operation, resp.Headers.Get(HTTPHeaderOssRequestID)}
}
backend/vendor/github.com/aliyun/aliyun-oss-go-sdk/oss/limit_reader_1_6.go
Generated Vendored
+29
View File
@@ -0,0 +1,29 @@
//go:build !go1.7
// +build !go1.7
// "golang.org/x/time/rate" is depended on golang context package go1.7 onward
// this file is only for build,not supports limit upload speed
package oss
import (
"fmt"
"io"
)
const (
perTokenBandwidthSize int = 1024
)
type OssLimiter struct {
}
type LimitSpeedReader struct {
io.ReadCloser
reader io.Reader
ossLimiter *OssLimiter
}
func GetOssLimiter(uploadSpeed int) (ossLimiter *OssLimiter, err error) {
err = fmt.Errorf("rate.Limiter is not supported below version go1.7")
return nil, err
}
backend/vendor/github.com/aliyun/aliyun-oss-go-sdk/oss/limit_reader_1_7.go
Generated Vendored
+91
View File
@@ -0,0 +1,91 @@
//go:build go1.7
// +build go1.7
package oss
import (
"fmt"
"io"
"math"
"time"
"golang.org/x/time/rate"
)
const (
perTokenBandwidthSize int = 1024
)
// OssLimiter wrapper rate.Limiter
type OssLimiter struct {
limiter *rate.Limiter
}
// GetOssLimiter create OssLimiter
// uploadSpeed KB/s
func GetOssLimiter(uploadSpeed int) (ossLimiter *OssLimiter, err error) {
limiter := rate.NewLimiter(rate.Limit(uploadSpeed), uploadSpeed)
// first consume the initial full token,the limiter will behave more accurately
limiter.AllowN(time.Now(), uploadSpeed)
return &OssLimiter{
limiter: limiter,
}, nil
}
// LimitSpeedReader for limit bandwidth upload
type LimitSpeedReader struct {
io.ReadCloser
reader io.Reader
ossLimiter *OssLimiter
}
// Read
func (r *LimitSpeedReader) Read(p []byte) (n int, err error) {
n = 0
err = nil
start := 0
burst := r.ossLimiter.limiter.Burst()
var end int
var tmpN int
var tc int
for start < len(p) {
if start+burst*perTokenBandwidthSize < len(p) {
end = start + burst*perTokenBandwidthSize
} else {
end = len(p)
}
tmpN, err = r.reader.Read(p[start:end])
if tmpN > 0 {
n += tmpN
start = n
}
if err != nil {
return
}
tc = int(math.Ceil(float64(tmpN) / float64(perTokenBandwidthSize)))
now := time.Now()
re := r.ossLimiter.limiter.ReserveN(now, tc)
if !re.OK() {
err = fmt.Errorf("LimitSpeedReader.Read() failure,ReserveN error,start:%d,end:%d,burst:%d,perTokenBandwidthSize:%d",
start, end, burst, perTokenBandwidthSize)
return
}
timeDelay := re.Delay()
time.Sleep(timeDelay)
}
return
}
// Close ...
func (r *LimitSpeedReader) Close() error {
rc, ok := r.reader.(io.ReadCloser)
if ok {
return rc.Close()
}
return nil
}
backend/vendor/github.com/aliyun/aliyun-oss-go-sdk/oss/livechannel.go
Generated Vendored
+257
View File
@@ -0,0 +1,257 @@
package oss
import (
"bytes"
"encoding/xml"
"fmt"
"io"
"net/http"
"strconv"
"time"
)
//
// CreateLiveChannel create a live-channel
//
// channelName the name of the channel
// config configuration of the channel
//
// CreateLiveChannelResult the result of create live-channel
// error nil if success, otherwise error
//
func (bucket Bucket) CreateLiveChannel(channelName string, config LiveChannelConfiguration) (CreateLiveChannelResult, error) {
var out CreateLiveChannelResult
bs, err := xml.Marshal(config)
if err != nil {
return out, err
}
buffer := new(bytes.Buffer)
buffer.Write(bs)
params := map[string]interface{}{}
params["live"] = nil
resp, err := bucket.do("PUT", channelName, params, nil, buffer, nil)
if err != nil {
return out, err
}
defer resp.Body.Close()
err = xmlUnmarshal(resp.Body, &out)
return out, err
}
//
// PutLiveChannelStatus Set the status of the live-channel: enabled/disabled
//
// channelName the name of the channel
// status enabled/disabled
//
// error nil if success, otherwise error
//
func (bucket Bucket) PutLiveChannelStatus(channelName, status string) error {
params := map[string]interface{}{}
params["live"] = nil
params["status"] = status
resp, err := bucket.do("PUT", channelName, params, nil, nil, nil)
if err != nil {
return err
}
defer resp.Body.Close()
return CheckRespCode(resp.StatusCode, []int{http.StatusOK})
}
// PostVodPlaylist create an playlist based on the specified playlist name, startTime and endTime
//
// channelName the name of the channel
// playlistName the name of the playlist, must end with ".m3u8"
// startTime the start time of the playlist
// endTime the endtime of the playlist
//
// error nil if success, otherwise error
//
func (bucket Bucket) PostVodPlaylist(channelName, playlistName string, startTime, endTime time.Time) error {
params := map[string]interface{}{}
params["vod"] = nil
params["startTime"] = strconv.FormatInt(startTime.Unix(), 10)
params["endTime"] = strconv.FormatInt(endTime.Unix(), 10)
key := fmt.Sprintf("%s/%s", channelName, playlistName)
resp, err := bucket.do("POST", key, params, nil, nil, nil)
if err != nil {
return err
}
defer resp.Body.Close()
return CheckRespCode(resp.StatusCode, []int{http.StatusOK})
}
// GetVodPlaylist get the playlist based on the specified channelName, startTime and endTime
//
// channelName the name of the channel
// startTime the start time of the playlist
// endTime the endtime of the playlist
//
// io.ReadCloser reader instance for reading data from response. It must be called close() after the usage and only valid when error is nil.
// error nil if success, otherwise error
//
func (bucket Bucket) GetVodPlaylist(channelName string, startTime, endTime time.Time) (io.ReadCloser, error) {
params := map[string]interface{}{}
params["vod"] = nil
params["startTime"] = strconv.FormatInt(startTime.Unix(), 10)
params["endTime"] = strconv.FormatInt(endTime.Unix(), 10)
resp, err := bucket.do("GET", channelName, params, nil, nil, nil)
if err != nil {
return nil, err
}
return resp.Body, nil
}
//
// GetLiveChannelStat Get the state of the live-channel
//
// channelName the name of the channel
//
// LiveChannelStat the state of the live-channel
// error nil if success, otherwise error
//
func (bucket Bucket) GetLiveChannelStat(channelName string) (LiveChannelStat, error) {
var out LiveChannelStat
params := map[string]interface{}{}
params["live"] = nil
params["comp"] = "stat"
resp, err := bucket.do("GET", channelName, params, nil, nil, nil)
if err != nil {
return out, err
}
defer resp.Body.Close()
err = xmlUnmarshal(resp.Body, &out)
return out, err
}
//
// GetLiveChannelInfo Get the configuration info of the live-channel
//
// channelName the name of the channel
//
// LiveChannelConfiguration the configuration info of the live-channel
// error nil if success, otherwise error
//
func (bucket Bucket) GetLiveChannelInfo(channelName string) (LiveChannelConfiguration, error) {
var out LiveChannelConfiguration
params := map[string]interface{}{}
params["live"] = nil
resp, err := bucket.do("GET", channelName, params, nil, nil, nil)
if err != nil {
return out, err
}
defer resp.Body.Close()
err = xmlUnmarshal(resp.Body, &out)
return out, err
}
//
// GetLiveChannelHistory Get push records of live-channel
//
// channelName the name of the channel
//
// LiveChannelHistory push records
// error nil if success, otherwise error
//
func (bucket Bucket) GetLiveChannelHistory(channelName string) (LiveChannelHistory, error) {
var out LiveChannelHistory
params := map[string]interface{}{}
params["live"] = nil
params["comp"] = "history"
resp, err := bucket.do("GET", channelName, params, nil, nil, nil)
if err != nil {
return out, err
}
defer resp.Body.Close()
err = xmlUnmarshal(resp.Body, &out)
return out, err
}
//
// ListLiveChannel list the live-channels
//
// options Prefix: filter by the name start with the value of "Prefix"
// MaxKeys: the maximum count returned
// Marker: cursor from which starting list
//
// ListLiveChannelResult live-channel list
// error nil if success, otherwise error
//
func (bucket Bucket) ListLiveChannel(options ...Option) (ListLiveChannelResult, error) {
var out ListLiveChannelResult
params, err := GetRawParams(options)
if err != nil {
return out, err
}
params["live"] = nil
resp, err := bucket.doInner("GET", "", params, nil, nil, nil)
if err != nil {
return out, err
}
defer resp.Body.Close()
err = xmlUnmarshal(resp.Body, &out)
return out, err
}
//
// DeleteLiveChannel Delete the live-channel. When a client trying to stream the live-channel, the operation will fail. it will only delete the live-channel itself and the object generated by the live-channel will not be deleted.
//
// channelName the name of the channel
//
// error nil if success, otherwise error
//
func (bucket Bucket) DeleteLiveChannel(channelName string) error {
params := map[string]interface{}{}
params["live"] = nil
if channelName == "" {
return fmt.Errorf("invalid argument: channel name is empty")
}
resp, err := bucket.do("DELETE", channelName, params, nil, nil, nil)
if err != nil {
return err
}
defer resp.Body.Close()
return CheckRespCode(resp.StatusCode, []int{http.StatusNoContent})
}
//
// SignRtmpURL Generate a RTMP push-stream signature URL for the trusted user to push the RTMP stream to the live-channel.
//
// channelName the name of the channel
// playlistName the name of the playlist, must end with ".m3u8"
// expires expiration (in seconds)
//
// string singed rtmp push stream url
// error nil if success, otherwise error
//
func (bucket Bucket) SignRtmpURL(channelName, playlistName string, expires int64) (string, error) {
if expires <= 0 {
return "", fmt.Errorf("invalid argument: %d, expires must greater than 0", expires)
}
expiration := time.Now().Unix() + expires
return bucket.Client.Conn.signRtmpURL(bucket.BucketName, channelName, playlistName, expiration), nil
}
backend/vendor/github.com/aliyun/aliyun-oss-go-sdk/oss/mime.go
Generated Vendored
+594
View File
@@ -0,0 +1,594 @@
package oss
import (
"mime"
"path"
"strings"
)
var extToMimeType = map[string]string{
".xlsx": "application/vnd.openxmlformats-officedocument.spreadsheetml.sheet",
".xltx": "application/vnd.openxmlformats-officedocument.spreadsheetml.template",
".potx": "application/vnd.openxmlformats-officedocument.presentationml.template",
".ppsx": "application/vnd.openxmlformats-officedocument.presentationml.slideshow",
".pptx": "application/vnd.openxmlformats-officedocument.presentationml.presentation",
".sldx": "application/vnd.openxmlformats-officedocument.presentationml.slide",
".docx": "application/vnd.openxmlformats-officedocument.wordprocessingml.document",
".dotx": "application/vnd.openxmlformats-officedocument.wordprocessingml.template",
".xlam": "application/vnd.ms-excel.addin.macroEnabled.12",
".xlsb": "application/vnd.ms-excel.sheet.binary.macroEnabled.12",
".apk": "application/vnd.android.package-archive",
".hqx": "application/mac-binhex40",
".cpt": "application/mac-compactpro",
".doc": "application/msword",
".ogg": "application/ogg",
".pdf": "application/pdf",
".rtf": "text/rtf",
".mif": "application/vnd.mif",
".xls": "application/vnd.ms-excel",
".ppt": "application/vnd.ms-powerpoint",
".odc": "application/vnd.oasis.opendocument.chart",
".odb": "application/vnd.oasis.opendocument.database",
".odf": "application/vnd.oasis.opendocument.formula",
".odg": "application/vnd.oasis.opendocument.graphics",
".otg": "application/vnd.oasis.opendocument.graphics-template",
".odi": "application/vnd.oasis.opendocument.image",
".odp": "application/vnd.oasis.opendocument.presentation",
".otp": "application/vnd.oasis.opendocument.presentation-template",
".ods": "application/vnd.oasis.opendocument.spreadsheet",
".ots": "application/vnd.oasis.opendocument.spreadsheet-template",
".odt": "application/vnd.oasis.opendocument.text",
".odm": "application/vnd.oasis.opendocument.text-master",
".ott": "application/vnd.oasis.opendocument.text-template",
".oth": "application/vnd.oasis.opendocument.text-web",
".sxw": "application/vnd.sun.xml.writer",
".stw": "application/vnd.sun.xml.writer.template",
".sxc": "application/vnd.sun.xml.calc",
".stc": "application/vnd.sun.xml.calc.template",
".sxd": "application/vnd.sun.xml.draw",
".std": "application/vnd.sun.xml.draw.template",
".sxi": "application/vnd.sun.xml.impress",
".sti": "application/vnd.sun.xml.impress.template",
".sxg": "application/vnd.sun.xml.writer.global",
".sxm": "application/vnd.sun.xml.math",
".sis": "application/vnd.symbian.install",
".wbxml": "application/vnd.wap.wbxml",
".wmlc": "application/vnd.wap.wmlc",
".wmlsc": "application/vnd.wap.wmlscriptc",
".bcpio": "application/x-bcpio",
".torrent": "application/x-bittorrent",
".bz2": "application/x-bzip2",
".vcd": "application/x-cdlink",
".pgn": "application/x-chess-pgn",
".cpio": "application/x-cpio",
".csh": "application/x-csh",
".dvi": "application/x-dvi",
".spl": "application/x-futuresplash",
".gtar": "application/x-gtar",
".hdf": "application/x-hdf",
".jar": "application/x-java-archive",
".jnlp": "application/x-java-jnlp-file",
".js": "application/x-javascript",
".ksp": "application/x-kspread",
".chrt": "application/x-kchart",
".kil": "application/x-killustrator",
".latex": "application/x-latex",
".rpm": "application/x-rpm",
".sh": "application/x-sh",
".shar": "application/x-shar",
".swf": "application/x-shockwave-flash",
".sit": "application/x-stuffit",
".sv4cpio": "application/x-sv4cpio",
".sv4crc": "application/x-sv4crc",
".tar": "application/x-tar",
".tcl": "application/x-tcl",
".tex": "application/x-tex",
".man": "application/x-troff-man",
".me": "application/x-troff-me",
".ms": "application/x-troff-ms",
".ustar": "application/x-ustar",
".src": "application/x-wais-source",
".zip": "application/zip",
".m3u": "audio/x-mpegurl",
".ra": "audio/x-pn-realaudio",
".wav": "audio/x-wav",
".wma": "audio/x-ms-wma",
".wax": "audio/x-ms-wax",
".pdb": "chemical/x-pdb",
".xyz": "chemical/x-xyz",
".bmp": "image/bmp",
".gif": "image/gif",
".ief": "image/ief",
".png": "image/png",
".wbmp": "image/vnd.wap.wbmp",
".ras": "image/x-cmu-raster",
".pnm": "image/x-portable-anymap",
".pbm": "image/x-portable-bitmap",
".pgm": "image/x-portable-graymap",
".ppm": "image/x-portable-pixmap",
".rgb": "image/x-rgb",
".xbm": "image/x-xbitmap",
".xpm": "image/x-xpixmap",
".xwd": "image/x-xwindowdump",
".css": "text/css",
".rtx": "text/richtext",
".tsv": "text/tab-separated-values",
".jad": "text/vnd.sun.j2me.app-descriptor",
".wml": "text/vnd.wap.wml",
".wmls": "text/vnd.wap.wmlscript",
".etx": "text/x-setext",
".mxu": "video/vnd.mpegurl",
".flv": "video/x-flv",
".wm": "video/x-ms-wm",
".wmv": "video/x-ms-wmv",
".wmx": "video/x-ms-wmx",
".wvx": "video/x-ms-wvx",
".avi": "video/x-msvideo",
".movie": "video/x-sgi-movie",
".ice": "x-conference/x-cooltalk",
".3gp": "video/3gpp",
".ai": "application/postscript",
".aif": "audio/x-aiff",
".aifc": "audio/x-aiff",
".aiff": "audio/x-aiff",
".asc": "text/plain",
".atom": "application/atom+xml",
".au": "audio/basic",
".bin": "application/octet-stream",
".cdf": "application/x-netcdf",
".cgm": "image/cgm",
".class": "application/octet-stream",
".dcr": "application/x-director",
".dif": "video/x-dv",
".dir": "application/x-director",
".djv": "image/vnd.djvu",
".djvu": "image/vnd.djvu",
".dll": "application/octet-stream",
".dmg": "application/octet-stream",
".dms": "application/octet-stream",
".dtd": "application/xml-dtd",
".dv": "video/x-dv",
".dxr": "application/x-director",
".eps": "application/postscript",
".exe": "application/octet-stream",
".ez": "application/andrew-inset",
".gram": "application/srgs",
".grxml": "application/srgs+xml",
".gz": "application/x-gzip",
".htm": "text/html",
".html": "text/html",
".ico": "image/x-icon",
".ics": "text/calendar",
".ifb": "text/calendar",
".iges": "model/iges",
".igs": "model/iges",
".jp2": "image/jp2",
".jpe": "image/jpeg",
".jpeg": "image/jpeg",
".jpg": "image/jpeg",
".kar": "audio/midi",
".lha": "application/octet-stream",
".lzh": "application/octet-stream",
".m4a": "audio/mp4a-latm",
".m4p": "audio/mp4a-latm",
".m4u": "video/vnd.mpegurl",
".m4v": "video/x-m4v",
".mac": "image/x-macpaint",
".mathml": "application/mathml+xml",
".mesh": "model/mesh",
".mid": "audio/midi",
".midi": "audio/midi",
".mov": "video/quicktime",
".mp2": "audio/mpeg",
".mp3": "audio/mpeg",
".mp4": "video/mp4",
".mpe": "video/mpeg",
".mpeg": "video/mpeg",
".mpg": "video/mpeg",
".mpga": "audio/mpeg",
".msh": "model/mesh",
".nc": "application/x-netcdf",
".oda": "application/oda",
".ogv": "video/ogv",
".pct": "image/pict",
".pic": "image/pict",
".pict": "image/pict",
".pnt": "image/x-macpaint",
".pntg": "image/x-macpaint",
".ps": "application/postscript",
".qt": "video/quicktime",
".qti": "image/x-quicktime",
".qtif": "image/x-quicktime",
".ram": "audio/x-pn-realaudio",
".rdf": "application/rdf+xml",
".rm": "application/vnd.rn-realmedia",
".roff": "application/x-troff",
".sgm": "text/sgml",
".sgml": "text/sgml",
".silo": "model/mesh",
".skd": "application/x-koan",
".skm": "application/x-koan",
".skp": "application/x-koan",
".skt": "application/x-koan",
".smi": "application/smil",
".smil": "application/smil",
".snd": "audio/basic",
".so": "application/octet-stream",
".svg": "image/svg+xml",
".t": "application/x-troff",
".texi": "application/x-texinfo",
".texinfo": "application/x-texinfo",
".tif": "image/tiff",
".tiff": "image/tiff",
".tr": "application/x-troff",
".txt": "text/plain",
".vrml": "model/vrml",
".vxml": "application/voicexml+xml",
".webm": "video/webm",
".wrl": "model/vrml",
".xht": "application/xhtml+xml",
".xhtml": "application/xhtml+xml",
".xml": "application/xml",
".xsl": "application/xml",
".xslt": "application/xslt+xml",
".xul": "application/vnd.mozilla.xul+xml",
".webp": "image/webp",
".323": "text/h323",
".aab": "application/x-authoware-bin",
".aam": "application/x-authoware-map",
".aas": "application/x-authoware-seg",
".acx": "application/internet-property-stream",
".als": "audio/X-Alpha5",
".amc": "application/x-mpeg",
".ani": "application/octet-stream",
".asd": "application/astound",
".asf": "video/x-ms-asf",
".asn": "application/astound",
".asp": "application/x-asap",
".asr": "video/x-ms-asf",
".asx": "video/x-ms-asf",
".avb": "application/octet-stream",
".awb": "audio/amr-wb",
".axs": "application/olescript",
".bas": "text/plain",
".bin ": "application/octet-stream",
".bld": "application/bld",
".bld2": "application/bld2",
".bpk": "application/octet-stream",
".c": "text/plain",
".cal": "image/x-cals",
".cat": "application/vnd.ms-pkiseccat",
".ccn": "application/x-cnc",
".cco": "application/x-cocoa",
".cer": "application/x-x509-ca-cert",
".cgi": "magnus-internal/cgi",
".chat": "application/x-chat",
".clp": "application/x-msclip",
".cmx": "image/x-cmx",
".co": "application/x-cult3d-object",
".cod": "image/cis-cod",
".conf": "text/plain",
".cpp": "text/plain",
".crd": "application/x-mscardfile",
".crl": "application/pkix-crl",
".crt": "application/x-x509-ca-cert",
".csm": "chemical/x-csml",
".csml": "chemical/x-csml",
".cur": "application/octet-stream",
".dcm": "x-lml/x-evm",
".dcx": "image/x-dcx",
".der": "application/x-x509-ca-cert",
".dhtml": "text/html",
".dot": "application/msword",
".dwf": "drawing/x-dwf",
".dwg": "application/x-autocad",
".dxf": "application/x-autocad",
".ebk": "application/x-expandedbook",
".emb": "chemical/x-embl-dl-nucleotide",
".embl": "chemical/x-embl-dl-nucleotide",
".epub": "application/epub+zip",
".eri": "image/x-eri",
".es": "audio/echospeech",
".esl": "audio/echospeech",
".etc": "application/x-earthtime",
".evm": "x-lml/x-evm",
".evy": "application/envoy",
".fh4": "image/x-freehand",
".fh5": "image/x-freehand",
".fhc": "image/x-freehand",
".fif": "application/fractals",
".flr": "x-world/x-vrml",
".fm": "application/x-maker",
".fpx": "image/x-fpx",
".fvi": "video/isivideo",
".gau": "chemical/x-gaussian-input",
".gca": "application/x-gca-compressed",
".gdb": "x-lml/x-gdb",
".gps": "application/x-gps",
".h": "text/plain",
".hdm": "text/x-hdml",
".hdml": "text/x-hdml",
".hlp": "application/winhlp",
".hta": "application/hta",
".htc": "text/x-component",
".hts": "text/html",
".htt": "text/webviewhtml",
".ifm": "image/gif",
".ifs": "image/ifs",
".iii": "application/x-iphone",
".imy": "audio/melody",
".ins": "application/x-internet-signup",
".ips": "application/x-ipscript",
".ipx": "application/x-ipix",
".isp": "application/x-internet-signup",
".it": "audio/x-mod",
".itz": "audio/x-mod",
".ivr": "i-world/i-vrml",
".j2k": "image/j2k",
".jam": "application/x-jam",
".java": "text/plain",
".jfif": "image/pipeg",
".jpz": "image/jpeg",
".jwc": "application/jwc",
".kjx": "application/x-kjx",
".lak": "x-lml/x-lak",
".lcc": "application/fastman",
".lcl": "application/x-digitalloca",
".lcr": "application/x-digitalloca",
".lgh": "application/lgh",
".lml": "x-lml/x-lml",
".lmlpack": "x-lml/x-lmlpack",
".log": "text/plain",
".lsf": "video/x-la-asf",
".lsx": "video/x-la-asf",
".m13": "application/x-msmediaview",
".m14": "application/x-msmediaview",
".m15": "audio/x-mod",
".m3url": "audio/x-mpegurl",
".m4b": "audio/mp4a-latm",
".ma1": "audio/ma1",
".ma2": "audio/ma2",
".ma3": "audio/ma3",
".ma5": "audio/ma5",
".map": "magnus-internal/imagemap",
".mbd": "application/mbedlet",
".mct": "application/x-mascot",
".mdb": "application/x-msaccess",
".mdz": "audio/x-mod",
".mel": "text/x-vmel",
".mht": "message/rfc822",
".mhtml": "message/rfc822",
".mi": "application/x-mif",
".mil": "image/x-cals",
".mio": "audio/x-mio",
".mmf": "application/x-skt-lbs",
".mng": "video/x-mng",
".mny": "application/x-msmoney",
".moc": "application/x-mocha",
".mocha": "application/x-mocha",
".mod": "audio/x-mod",
".mof": "application/x-yumekara",
".mol": "chemical/x-mdl-molfile",
".mop": "chemical/x-mopac-input",
".mpa": "video/mpeg",
".mpc": "application/vnd.mpohun.certificate",
".mpg4": "video/mp4",
".mpn": "application/vnd.mophun.application",
".mpp": "application/vnd.ms-project",
".mps": "application/x-mapserver",
".mpv2": "video/mpeg",
".mrl": "text/x-mrml",
".mrm": "application/x-mrm",
".msg": "application/vnd.ms-outlook",
".mts": "application/metastream",
".mtx": "application/metastream",
".mtz": "application/metastream",
".mvb": "application/x-msmediaview",
".mzv": "application/metastream",
".nar": "application/zip",
".nbmp": "image/nbmp",
".ndb": "x-lml/x-ndb",
".ndwn": "application/ndwn",
".nif": "application/x-nif",
".nmz": "application/x-scream",
".nokia-op-logo": "image/vnd.nok-oplogo-color",
".npx": "application/x-netfpx",
".nsnd": "audio/nsnd",
".nva": "application/x-neva1",
".nws": "message/rfc822",
".oom": "application/x-AtlasMate-Plugin",
".p10": "application/pkcs10",
".p12": "application/x-pkcs12",
".p7b": "application/x-pkcs7-certificates",
".p7c": "application/x-pkcs7-mime",
".p7m": "application/x-pkcs7-mime",
".p7r": "application/x-pkcs7-certreqresp",
".p7s": "application/x-pkcs7-signature",
".pac": "audio/x-pac",
".pae": "audio/x-epac",
".pan": "application/x-pan",
".pcx": "image/x-pcx",
".pda": "image/x-pda",
".pfr": "application/font-tdpfr",
".pfx": "application/x-pkcs12",
".pko": "application/ynd.ms-pkipko",
".pm": "application/x-perl",
".pma": "application/x-perfmon",
".pmc": "application/x-perfmon",
".pmd": "application/x-pmd",
".pml": "application/x-perfmon",
".pmr": "application/x-perfmon",
".pmw": "application/x-perfmon",
".pnz": "image/png",
".pot,": "application/vnd.ms-powerpoint",
".pps": "application/vnd.ms-powerpoint",
".pqf": "application/x-cprplayer",
".pqi": "application/cprplayer",
".prc": "application/x-prc",
".prf": "application/pics-rules",
".prop": "text/plain",
".proxy": "application/x-ns-proxy-autoconfig",
".ptlk": "application/listenup",
".pub": "application/x-mspublisher",
".pvx": "video/x-pv-pvx",
".qcp": "audio/vnd.qcelp",
".r3t": "text/vnd.rn-realtext3d",
".rar": "application/octet-stream",
".rc": "text/plain",
".rf": "image/vnd.rn-realflash",
".rlf": "application/x-richlink",
".rmf": "audio/x-rmf",
".rmi": "audio/mid",
".rmm": "audio/x-pn-realaudio",
".rmvb": "audio/x-pn-realaudio",
".rnx": "application/vnd.rn-realplayer",
".rp": "image/vnd.rn-realpix",
".rt": "text/vnd.rn-realtext",
".rte": "x-lml/x-gps",
".rtg": "application/metastream",
".rv": "video/vnd.rn-realvideo",
".rwc": "application/x-rogerwilco",
".s3m": "audio/x-mod",
".s3z": "audio/x-mod",
".sca": "application/x-supercard",
".scd": "application/x-msschedule",
".sct": "text/scriptlet",
".sdf": "application/e-score",
".sea": "application/x-stuffit",
".setpay": "application/set-payment-initiation",
".setreg": "application/set-registration-initiation",
".shtml": "text/html",
".shtm": "text/html",
".shw": "application/presentations",
".si6": "image/si6",
".si7": "image/vnd.stiwap.sis",
".si9": "image/vnd.lgtwap.sis",
".slc": "application/x-salsa",
".smd": "audio/x-smd",
".smp": "application/studiom",
".smz": "audio/x-smd",
".spc": "application/x-pkcs7-certificates",
".spr": "application/x-sprite",
".sprite": "application/x-sprite",
".sdp": "application/sdp",
".spt": "application/x-spt",
".sst": "application/vnd.ms-pkicertstore",
".stk": "application/hyperstudio",
".stl": "application/vnd.ms-pkistl",
".stm": "text/html",
".svf": "image/vnd",
".svh": "image/svh",
".svr": "x-world/x-svr",
".swfl": "application/x-shockwave-flash",
".tad": "application/octet-stream",
".talk": "text/x-speech",
".taz": "application/x-tar",
".tbp": "application/x-timbuktu",
".tbt": "application/x-timbuktu",
".tgz": "application/x-compressed",
".thm": "application/vnd.eri.thm",
".tki": "application/x-tkined",
".tkined": "application/x-tkined",
".toc": "application/toc",
".toy": "image/toy",
".trk": "x-lml/x-gps",
".trm": "application/x-msterminal",
".tsi": "audio/tsplayer",
".tsp": "application/dsptype",
".ttf": "application/octet-stream",
".ttz": "application/t-time",
".uls": "text/iuls",
".ult": "audio/x-mod",
".uu": "application/x-uuencode",
".uue": "application/x-uuencode",
".vcf": "text/x-vcard",
".vdo": "video/vdo",
".vib": "audio/vib",
".viv": "video/vivo",
".vivo": "video/vivo",
".vmd": "application/vocaltec-media-desc",
".vmf": "application/vocaltec-media-file",
".vmi": "application/x-dreamcast-vms-info",
".vms": "application/x-dreamcast-vms",
".vox": "audio/voxware",
".vqe": "audio/x-twinvq-plugin",
".vqf": "audio/x-twinvq",
".vql": "audio/x-twinvq",
".vre": "x-world/x-vream",
".vrt": "x-world/x-vrt",
".vrw": "x-world/x-vream",
".vts": "workbook/formulaone",
".wcm": "application/vnd.ms-works",
".wdb": "application/vnd.ms-works",
".web": "application/vnd.xara",
".wi": "image/wavelet",
".wis": "application/x-InstallShield",
".wks": "application/vnd.ms-works",
".wmd": "application/x-ms-wmd",
".wmf": "application/x-msmetafile",
".wmlscript": "text/vnd.wap.wmlscript",
".wmz": "application/x-ms-wmz",
".wpng": "image/x-up-wpng",
".wps": "application/vnd.ms-works",
".wpt": "x-lml/x-gps",
".wri": "application/x-mswrite",
".wrz": "x-world/x-vrml",
".ws": "text/vnd.wap.wmlscript",
".wsc": "application/vnd.wap.wmlscriptc",
".wv": "video/wavelet",
".wxl": "application/x-wxl",
".x-gzip": "application/x-gzip",
".xaf": "x-world/x-vrml",
".xar": "application/vnd.xara",
".xdm": "application/x-xdma",
".xdma": "application/x-xdma",
".xdw": "application/vnd.fujixerox.docuworks",
".xhtm": "application/xhtml+xml",
".xla": "application/vnd.ms-excel",
".xlc": "application/vnd.ms-excel",
".xll": "application/x-excel",
".xlm": "application/vnd.ms-excel",
".xlt": "application/vnd.ms-excel",
".xlw": "application/vnd.ms-excel",
".xm": "audio/x-mod",
".xmz": "audio/x-mod",
".xof": "x-world/x-vrml",
".xpi": "application/x-xpinstall",
".xsit": "text/xml",
".yz1": "application/x-yz1",
".z": "application/x-compress",
".zac": "application/x-zaurus-zac",
".json": "application/json",
}
// TypeByExtension returns the MIME type associated with the file extension ext.
// gets the file's MIME type for HTTP header Content-Type
func TypeByExtension(filePath string) string {
typ := mime.TypeByExtension(path.Ext(filePath))
if typ == "" {
typ = extToMimeType[strings.ToLower(path.Ext(filePath))]
} else {
if strings.HasPrefix(typ, "text/") && strings.Contains(typ, "charset=") {
typ = removeCharsetInMimeType(typ)
}
}
return typ
}
// Remove charset from mime type
func removeCharsetInMimeType(typ string) (str string) {
temArr := strings.Split(typ, ";")
var builder strings.Builder
for i, s := range temArr {
tmpStr := strings.Trim(s, " ")
if strings.Contains(tmpStr, "charset=") {
continue
}
if i == 0 {
builder.WriteString(s)
} else {
builder.WriteString("; " + s)
}
}
return builder.String()
}
backend/vendor/github.com/aliyun/aliyun-oss-go-sdk/oss/model.go
Generated Vendored
+69
View File
@@ -0,0 +1,69 @@
package oss
import (
"hash"
"io"
"net/http"
)
// Response defines HTTP response from OSS
type Response struct {
StatusCode int
Headers http.Header
Body io.ReadCloser
ClientCRC uint64
ServerCRC uint64
}
func (r *Response) Read(p []byte) (n int, err error) {
return r.Body.Read(p)
}
// Close close http reponse body
func (r *Response) Close() error {
return r.Body.Close()
}
// PutObjectRequest is the request of DoPutObject
type PutObjectRequest struct {
ObjectKey string
Reader io.Reader
}
// GetObjectRequest is the request of DoGetObject
type GetObjectRequest struct {
ObjectKey string
}
// GetObjectResult is the result of DoGetObject
type GetObjectResult struct {
Response *Response
ClientCRC hash.Hash64
ServerCRC uint64
}
// AppendObjectRequest is the requtest of DoAppendObject
type AppendObjectRequest struct {
ObjectKey string
Reader io.Reader
Position int64
}
// AppendObjectResult is the result of DoAppendObject
type AppendObjectResult struct {
NextPosition int64
CRC uint64
}
// UploadPartRequest is the request of DoUploadPart
type UploadPartRequest struct {
InitResult *InitiateMultipartUploadResult
Reader io.Reader
PartSize int64
PartNumber int
}
// UploadPartResult is the result of DoUploadPart
type UploadPartResult struct {
Part UploadPart
}
backend/vendor/github.com/aliyun/aliyun-oss-go-sdk/oss/multicopy.go
Generated Vendored
+474
View File
@@ -0,0 +1,474 @@
package oss
import (
"crypto/md5"
"encoding/base64"
"encoding/json"
"errors"
"fmt"
"io/ioutil"
"net/http"
"os"
"strconv"
)
// CopyFile is multipart copy object
//
// srcBucketName source bucket name
// srcObjectKey source object name
// destObjectKey target object name in the form of bucketname.objectkey
// partSize the part size in byte.
// options object's contraints. Check out function InitiateMultipartUpload.
//
// error it's nil if the operation succeeds, otherwise it's an error object.
//
func (bucket Bucket) CopyFile(srcBucketName, srcObjectKey, destObjectKey string, partSize int64, options ...Option) error {
destBucketName := bucket.BucketName
if partSize < MinPartSize || partSize > MaxPartSize {
return errors.New("oss: part size invalid range (1024KB, 5GB]")
}
cpConf := getCpConfig(options)
routines := getRoutines(options)
var strVersionId string
versionId, _ := FindOption(options, "versionId", nil)
if versionId != nil {
strVersionId = versionId.(string)
}
if cpConf != nil && cpConf.IsEnable {
cpFilePath := getCopyCpFilePath(cpConf, srcBucketName, srcObjectKey, destBucketName, destObjectKey, strVersionId)
if cpFilePath != "" {
return bucket.copyFileWithCp(srcBucketName, srcObjectKey, destBucketName, destObjectKey, partSize, options, cpFilePath, routines)
}
}
return bucket.copyFile(srcBucketName, srcObjectKey, destBucketName, destObjectKey,
partSize, options, routines)
}
func getCopyCpFilePath(cpConf *cpConfig, srcBucket, srcObject, destBucket, destObject, versionId string) string {
if cpConf.FilePath == "" && cpConf.DirPath != "" {
dest := fmt.Sprintf("oss://%v/%v", destBucket, destObject)
src := fmt.Sprintf("oss://%v/%v", srcBucket, srcObject)
cpFileName := getCpFileName(src, dest, versionId)
cpConf.FilePath = cpConf.DirPath + string(os.PathSeparator) + cpFileName
}
return cpConf.FilePath
}
// ----- Concurrently copy without checkpoint ---------
// copyWorkerArg defines the copy worker arguments
type copyWorkerArg struct {
bucket *Bucket
imur InitiateMultipartUploadResult
srcBucketName string
srcObjectKey string
options []Option
hook copyPartHook
}
// copyPartHook is the hook for testing purpose
type copyPartHook func(part copyPart) error
var copyPartHooker copyPartHook = defaultCopyPartHook
func defaultCopyPartHook(part copyPart) error {
return nil
}
// copyWorker copies worker
func copyWorker(id int, arg copyWorkerArg, jobs <-chan copyPart, results chan<- UploadPart, failed chan<- error, die <-chan bool) {
for chunk := range jobs {
if err := arg.hook(chunk); err != nil {
failed <- err
break
}
chunkSize := chunk.End - chunk.Start + 1
part, err := arg.bucket.UploadPartCopy(arg.imur, arg.srcBucketName, arg.srcObjectKey,
chunk.Start, chunkSize, chunk.Number, arg.options...)
if err != nil {
failed <- err
break
}
select {
case <-die:
return
default:
}
results <- part
}
}
// copyScheduler
func copyScheduler(jobs chan copyPart, parts []copyPart) {
for _, part := range parts {
jobs <- part
}
close(jobs)
}
// copyPart structure
type copyPart struct {
Number int // Part number (from 1 to 10,000)
Start int64 // The start index in the source file.
End int64 // The end index in the source file
}
// getCopyParts calculates copy parts
func getCopyParts(objectSize, partSize int64) []copyPart {
parts := []copyPart{}
part := copyPart{}
i := 0
for offset := int64(0); offset < objectSize; offset += partSize {
part.Number = i + 1
part.Start = offset
part.End = GetPartEnd(offset, objectSize, partSize)
parts = append(parts, part)
i++
}
return parts
}
// getSrcObjectBytes gets the source file size
func getSrcObjectBytes(parts []copyPart) int64 {
var ob int64
for _, part := range parts {
ob += (part.End - part.Start + 1)
}
return ob
}
// copyFile is a concurrently copy without checkpoint
func (bucket Bucket) copyFile(srcBucketName, srcObjectKey, destBucketName, destObjectKey string,
partSize int64, options []Option, routines int) error {
descBucket, err := bucket.Client.Bucket(destBucketName)
srcBucket, err := bucket.Client.Bucket(srcBucketName)
listener := GetProgressListener(options)
// choice valid options
headerOptions := ChoiceHeadObjectOption(options)
partOptions := ChoiceTransferPartOption(options)
completeOptions := ChoiceCompletePartOption(options)
abortOptions := ChoiceAbortPartOption(options)
meta, err := srcBucket.GetObjectDetailedMeta(srcObjectKey, headerOptions...)
if err != nil {
return err
}
objectSize, err := strconv.ParseInt(meta.Get(HTTPHeaderContentLength), 10, 0)
if err != nil {
return err
}
// Get copy parts
parts := getCopyParts(objectSize, partSize)
// Initialize the multipart upload
imur, err := descBucket.InitiateMultipartUpload(destObjectKey, options...)
if err != nil {
return err
}
jobs := make(chan copyPart, len(parts))
results := make(chan UploadPart, len(parts))
failed := make(chan error)
die := make(chan bool)
var completedBytes int64
totalBytes := getSrcObjectBytes(parts)
event := newProgressEvent(TransferStartedEvent, 0, totalBytes, 0)
publishProgress(listener, event)
// Start to copy workers
arg := copyWorkerArg{descBucket, imur, srcBucketName, srcObjectKey, partOptions, copyPartHooker}
for w := 1; w <= routines; w++ {
go copyWorker(w, arg, jobs, results, failed, die)
}
// Start the scheduler
go copyScheduler(jobs, parts)
// Wait for the parts finished.
completed := 0
ups := make([]UploadPart, len(parts))
for completed < len(parts) {
select {
case part := <-results:
completed++
ups[part.PartNumber-1] = part
copyBytes := (parts[part.PartNumber-1].End - parts[part.PartNumber-1].Start + 1)
completedBytes += copyBytes
event = newProgressEvent(TransferDataEvent, completedBytes, totalBytes, copyBytes)
publishProgress(listener, event)
case err := <-failed:
close(die)
descBucket.AbortMultipartUpload(imur, abortOptions...)
event = newProgressEvent(TransferFailedEvent, completedBytes, totalBytes, 0)
publishProgress(listener, event)
return err
}
if completed >= len(parts) {
break
}
}
event = newProgressEvent(TransferCompletedEvent, completedBytes, totalBytes, 0)
publishProgress(listener, event)
// Complete the multipart upload
_, err = descBucket.CompleteMultipartUpload(imur, ups, completeOptions...)
if err != nil {
bucket.AbortMultipartUpload(imur, abortOptions...)
return err
}
return nil
}
// ----- Concurrently copy with checkpoint -----
const copyCpMagic = "84F1F18C-FF1D-403B-A1D8-9DEB5F65910A"
type copyCheckpoint struct {
Magic string // Magic
MD5 string // CP content MD5
SrcBucketName string // Source bucket
SrcObjectKey string // Source object
DestBucketName string // Target bucket
DestObjectKey string // Target object
CopyID string // Copy ID
ObjStat objectStat // Object stat
Parts []copyPart // Copy parts
CopyParts []UploadPart // The uploaded parts
PartStat []bool // The part status
}
// isValid checks if the data is valid which means CP is valid and object is not updated.
func (cp copyCheckpoint) isValid(meta http.Header) (bool, error) {
// Compare CP's magic number and the MD5.
cpb := cp
cpb.MD5 = ""
js, _ := json.Marshal(cpb)
sum := md5.Sum(js)
b64 := base64.StdEncoding.EncodeToString(sum[:])
if cp.Magic != downloadCpMagic || b64 != cp.MD5 {
return false, nil
}
objectSize, err := strconv.ParseInt(meta.Get(HTTPHeaderContentLength), 10, 64)
if err != nil {
return false, err
}
// Compare the object size and last modified time and etag.
if cp.ObjStat.Size != objectSize ||
cp.ObjStat.LastModified != meta.Get(HTTPHeaderLastModified) ||
cp.ObjStat.Etag != meta.Get(HTTPHeaderEtag) {
return false, nil
}
return true, nil
}
// load loads from the checkpoint file
func (cp *copyCheckpoint) load(filePath string) error {
contents, err := ioutil.ReadFile(filePath)
if err != nil {
return err
}
err = json.Unmarshal(contents, cp)
return err
}
// update updates the parts status
func (cp *copyCheckpoint) update(part UploadPart) {
cp.CopyParts[part.PartNumber-1] = part
cp.PartStat[part.PartNumber-1] = true
}
// dump dumps the CP to the file
func (cp *copyCheckpoint) dump(filePath string) error {
bcp := *cp
// Calculate MD5
bcp.MD5 = ""
js, err := json.Marshal(bcp)
if err != nil {
return err
}
sum := md5.Sum(js)
b64 := base64.StdEncoding.EncodeToString(sum[:])
bcp.MD5 = b64
// Serialization
js, err = json.Marshal(bcp)
if err != nil {
return err
}
// Dump
return ioutil.WriteFile(filePath, js, FilePermMode)
}
// todoParts returns unfinished parts
func (cp copyCheckpoint) todoParts() []copyPart {
dps := []copyPart{}
for i, ps := range cp.PartStat {
if !ps {
dps = append(dps, cp.Parts[i])
}
}
return dps
}
// getCompletedBytes returns finished bytes count
func (cp copyCheckpoint) getCompletedBytes() int64 {
var completedBytes int64
for i, part := range cp.Parts {
if cp.PartStat[i] {
completedBytes += (part.End - part.Start + 1)
}
}
return completedBytes
}
// prepare initializes the multipart upload
func (cp *copyCheckpoint) prepare(meta http.Header, srcBucket *Bucket, srcObjectKey string, destBucket *Bucket, destObjectKey string,
partSize int64, options []Option) error {
// CP
cp.Magic = copyCpMagic
cp.SrcBucketName = srcBucket.BucketName
cp.SrcObjectKey = srcObjectKey
cp.DestBucketName = destBucket.BucketName
cp.DestObjectKey = destObjectKey
objectSize, err := strconv.ParseInt(meta.Get(HTTPHeaderContentLength), 10, 64)
if err != nil {
return err
}
cp.ObjStat.Size = objectSize
cp.ObjStat.LastModified = meta.Get(HTTPHeaderLastModified)
cp.ObjStat.Etag = meta.Get(HTTPHeaderEtag)
// Parts
cp.Parts = getCopyParts(objectSize, partSize)
cp.PartStat = make([]bool, len(cp.Parts))
for i := range cp.PartStat {
cp.PartStat[i] = false
}
cp.CopyParts = make([]UploadPart, len(cp.Parts))
// Init copy
imur, err := destBucket.InitiateMultipartUpload(destObjectKey, options...)
if err != nil {
return err
}
cp.CopyID = imur.UploadID
return nil
}
func (cp *copyCheckpoint) complete(bucket *Bucket, parts []UploadPart, cpFilePath string, options []Option) error {
imur := InitiateMultipartUploadResult{Bucket: cp.DestBucketName,
Key: cp.DestObjectKey, UploadID: cp.CopyID}
_, err := bucket.CompleteMultipartUpload(imur, parts, options...)
if err != nil {
return err
}
os.Remove(cpFilePath)
return err
}
// copyFileWithCp is concurrently copy with checkpoint
func (bucket Bucket) copyFileWithCp(srcBucketName, srcObjectKey, destBucketName, destObjectKey string,
partSize int64, options []Option, cpFilePath string, routines int) error {
descBucket, err := bucket.Client.Bucket(destBucketName)
srcBucket, err := bucket.Client.Bucket(srcBucketName)
listener := GetProgressListener(options)
// Load CP data
ccp := copyCheckpoint{}
err = ccp.load(cpFilePath)
if err != nil {
os.Remove(cpFilePath)
}
// choice valid options
headerOptions := ChoiceHeadObjectOption(options)
partOptions := ChoiceTransferPartOption(options)
completeOptions := ChoiceCompletePartOption(options)
meta, err := srcBucket.GetObjectDetailedMeta(srcObjectKey, headerOptions...)
if err != nil {
return err
}
// Load error or the CP data is invalid---reinitialize
valid, err := ccp.isValid(meta)
if err != nil || !valid {
if err = ccp.prepare(meta, srcBucket, srcObjectKey, descBucket, destObjectKey, partSize, options); err != nil {
return err
}
os.Remove(cpFilePath)
}
// Unfinished parts
parts := ccp.todoParts()
imur := InitiateMultipartUploadResult{
Bucket: destBucketName,
Key: destObjectKey,
UploadID: ccp.CopyID}
jobs := make(chan copyPart, len(parts))
results := make(chan UploadPart, len(parts))
failed := make(chan error)
die := make(chan bool)
completedBytes := ccp.getCompletedBytes()
event := newProgressEvent(TransferStartedEvent, completedBytes, ccp.ObjStat.Size, 0)
publishProgress(listener, event)
// Start the worker coroutines
arg := copyWorkerArg{descBucket, imur, srcBucketName, srcObjectKey, partOptions, copyPartHooker}
for w := 1; w <= routines; w++ {
go copyWorker(w, arg, jobs, results, failed, die)
}
// Start the scheduler
go copyScheduler(jobs, parts)
// Wait for the parts completed.
completed := 0
for completed < len(parts) {
select {
case part := <-results:
completed++
ccp.update(part)
ccp.dump(cpFilePath)
copyBytes := (parts[part.PartNumber-1].End - parts[part.PartNumber-1].Start + 1)
completedBytes += copyBytes
event = newProgressEvent(TransferDataEvent, completedBytes, ccp.ObjStat.Size, copyBytes)
publishProgress(listener, event)
case err := <-failed:
close(die)
event = newProgressEvent(TransferFailedEvent, completedBytes, ccp.ObjStat.Size, 0)
publishProgress(listener, event)
return err
}
if completed >= len(parts) {
break
}
}
event = newProgressEvent(TransferCompletedEvent, completedBytes, ccp.ObjStat.Size, 0)
publishProgress(listener, event)
return ccp.complete(descBucket, ccp.CopyParts, cpFilePath, completeOptions)
}
backend/vendor/github.com/aliyun/aliyun-oss-go-sdk/oss/multipart.go
Generated Vendored
+320
View File
@@ -0,0 +1,320 @@
package oss
import (
"bytes"
"encoding/xml"
"io"
"io/ioutil"
"net/http"
"net/url"
"os"
"sort"
"strconv"
)
// InitiateMultipartUpload initializes multipart upload
//
// objectKey object name
// options the object constricts for upload. The valid options are CacheControl, ContentDisposition, ContentEncoding, Expires,
//
// ServerSideEncryption, Meta, check out the following link:
// https://www.alibabacloud.com/help/en/object-storage-service/latest/initiatemultipartupload
//
// InitiateMultipartUploadResult the return value of the InitiateMultipartUpload, which is used for calls later on such as UploadPartFromFile,UploadPartCopy.
// error it's nil if the operation succeeds, otherwise it's an error object.
func (bucket Bucket) InitiateMultipartUpload(objectKey string, options ...Option) (InitiateMultipartUploadResult, error) {
var imur InitiateMultipartUploadResult
opts := AddContentType(options, objectKey)
params, _ := GetRawParams(options)
paramKeys := []string{"sequential", "withHashContext", "x-oss-enable-md5", "x-oss-enable-sha1", "x-oss-enable-sha256"}
ConvertEmptyValueToNil(params, paramKeys)
params["uploads"] = nil
resp, err := bucket.do("POST", objectKey, params, opts, nil, nil)
if err != nil {
return imur, err
}
defer resp.Body.Close()
err = xmlUnmarshal(resp.Body, &imur)
return imur, err
}
// UploadPart uploads parts
//
// After initializing a Multipart Upload, the upload Id and object key could be used for uploading the parts.
// Each part has its part number (ranges from 1 to 10,000). And for each upload Id, the part number identifies the position of the part in the whole file.
// And thus with the same part number and upload Id, another part upload will overwrite the data.
// Except the last one, minimal part size is 100KB. There's no limit on the last part size.
//
// imur the returned value of InitiateMultipartUpload.
// reader io.Reader the reader for the part's data.
// size the part size.
// partNumber the part number (ranges from 1 to 10,000). Invalid part number will lead to InvalidArgument error.
//
// UploadPart the return value of the upload part. It consists of PartNumber and ETag. It's valid when error is nil.
// error it's nil if the operation succeeds, otherwise it's an error object.
func (bucket Bucket) UploadPart(imur InitiateMultipartUploadResult, reader io.Reader,
partSize int64, partNumber int, options ...Option) (UploadPart, error) {
request := &UploadPartRequest{
InitResult: &imur,
Reader: reader,
PartSize: partSize,
PartNumber: partNumber,
}
result, err := bucket.DoUploadPart(request, options)
return result.Part, err
}
// UploadPartFromFile uploads part from the file.
//
// imur the return value of a successful InitiateMultipartUpload.
// filePath the local file path to upload.
// startPosition the start position in the local file.
// partSize the part size.
// partNumber the part number (from 1 to 10,000)
//
// UploadPart the return value consists of PartNumber and ETag.
// error it's nil if the operation succeeds, otherwise it's an error object.
func (bucket Bucket) UploadPartFromFile(imur InitiateMultipartUploadResult, filePath string,
startPosition, partSize int64, partNumber int, options ...Option) (UploadPart, error) {
var part = UploadPart{}
fd, err := os.Open(filePath)
if err != nil {
return part, err
}
defer fd.Close()
fd.Seek(startPosition, os.SEEK_SET)
request := &UploadPartRequest{
InitResult: &imur,
Reader: fd,
PartSize: partSize,
PartNumber: partNumber,
}
result, err := bucket.DoUploadPart(request, options)
return result.Part, err
}
// DoUploadPart does the actual part upload.
//
// request part upload request
//
// UploadPartResult the result of uploading part.
// error it's nil if the operation succeeds, otherwise it's an error object.
func (bucket Bucket) DoUploadPart(request *UploadPartRequest, options []Option) (*UploadPartResult, error) {
listener := GetProgressListener(options)
options = append(options, ContentLength(request.PartSize))
params := map[string]interface{}{}
params["partNumber"] = strconv.Itoa(request.PartNumber)
params["uploadId"] = request.InitResult.UploadID
resp, err := bucket.do("PUT", request.InitResult.Key, params, options,
&io.LimitedReader{R: request.Reader, N: request.PartSize}, listener)
if err != nil {
return &UploadPartResult{}, err
}
defer resp.Body.Close()
part := UploadPart{
ETag: resp.Headers.Get(HTTPHeaderEtag),
PartNumber: request.PartNumber,
}
if bucket.GetConfig().IsEnableCRC {
err = CheckCRC(resp, "DoUploadPart")
if err != nil {
return &UploadPartResult{part}, err
}
}
return &UploadPartResult{part}, nil
}
// UploadPartCopy uploads part copy
//
// imur the return value of InitiateMultipartUpload
// copySrc source Object name
// startPosition the part's start index in the source file
// partSize the part size
// partNumber the part number, ranges from 1 to 10,000. If it exceeds the range OSS returns InvalidArgument error.
// options the constraints of source object for the copy. The copy happens only when these contraints are met. Otherwise it returns error.
//
// CopySourceIfNoneMatch, CopySourceIfModifiedSince CopySourceIfUnmodifiedSince, check out the following link for the detail
// https://www.alibabacloud.com/help/en/object-storage-service/latest/uploadpartcopy
//
// UploadPart the return value consists of PartNumber and ETag.
// error it's nil if the operation succeeds, otherwise it's an error object.
func (bucket Bucket) UploadPartCopy(imur InitiateMultipartUploadResult, srcBucketName, srcObjectKey string,
startPosition, partSize int64, partNumber int, options ...Option) (UploadPart, error) {
var out UploadPartCopyResult
var part UploadPart
var opts []Option
//first find version id
versionIdKey := "versionId"
versionId, _ := FindOption(options, versionIdKey, nil)
if versionId == nil {
opts = []Option{CopySource(srcBucketName, url.QueryEscape(srcObjectKey)),
CopySourceRange(startPosition, partSize)}
} else {
opts = []Option{CopySourceVersion(srcBucketName, url.QueryEscape(srcObjectKey), versionId.(string)),
CopySourceRange(startPosition, partSize)}
options = DeleteOption(options, versionIdKey)
}
opts = append(opts, options...)
params := map[string]interface{}{}
params["partNumber"] = strconv.Itoa(partNumber)
params["uploadId"] = imur.UploadID
resp, err := bucket.do("PUT", imur.Key, params, opts, nil, nil)
if err != nil {
return part, err
}
defer resp.Body.Close()
err = xmlUnmarshal(resp.Body, &out)
if err != nil {
return part, err
}
part.ETag = out.ETag
part.PartNumber = partNumber
return part, nil
}
// CompleteMultipartUpload completes the multipart upload.
//
// imur the return value of InitiateMultipartUpload.
// parts the array of return value of UploadPart/UploadPartFromFile/UploadPartCopy.
//
// CompleteMultipartUploadResponse the return value when the call succeeds. Only valid when the error is nil.
// error it's nil if the operation succeeds, otherwise it's an error object.
func (bucket Bucket) CompleteMultipartUpload(imur InitiateMultipartUploadResult,
parts []UploadPart, options ...Option) (CompleteMultipartUploadResult, error) {
var out CompleteMultipartUploadResult
sort.Sort(UploadParts(parts))
cxml := completeMultipartUploadXML{}
cxml.Part = parts
bs, err := xml.Marshal(cxml)
if err != nil {
return out, err
}
buffer := new(bytes.Buffer)
buffer.Write(bs)
params := map[string]interface{}{}
params["uploadId"] = imur.UploadID
resp, err := bucket.do("POST", imur.Key, params, options, buffer, nil)
if err != nil {
return out, err
}
defer resp.Body.Close()
body, err := ioutil.ReadAll(resp.Body)
if err != nil {
return out, err
}
err = CheckRespCode(resp.StatusCode, []int{http.StatusOK})
if len(body) > 0 {
if err != nil {
err = tryConvertServiceError(body, resp, err)
} else {
callback, _ := FindOption(options, HTTPHeaderOssCallback, nil)
if callback == nil {
err = xml.Unmarshal(body, &out)
} else {
rb, _ := FindOption(options, responseBody, nil)
if rb != nil {
if rbody, ok := rb.(*[]byte); ok {
*rbody = body
}
}
}
}
}
return out, err
}
// AbortMultipartUpload aborts the multipart upload.
//
// imur the return value of InitiateMultipartUpload.
//
// error it's nil if the operation succeeds, otherwise it's an error object.
func (bucket Bucket) AbortMultipartUpload(imur InitiateMultipartUploadResult, options ...Option) error {
params := map[string]interface{}{}
params["uploadId"] = imur.UploadID
resp, err := bucket.do("DELETE", imur.Key, params, options, nil, nil)
if err != nil {
return err
}
defer resp.Body.Close()
return CheckRespCode(resp.StatusCode, []int{http.StatusNoContent})
}
// ListUploadedParts lists the uploaded parts.
//
// imur the return value of InitiateMultipartUpload.
//
// ListUploadedPartsResponse the return value if it succeeds, only valid when error is nil.
// error it's nil if the operation succeeds, otherwise it's an error object.
func (bucket Bucket) ListUploadedParts(imur InitiateMultipartUploadResult, options ...Option) (ListUploadedPartsResult, error) {
var out ListUploadedPartsResult
options = append(options, EncodingType("url"))
params := map[string]interface{}{}
params, err := GetRawParams(options)
if err != nil {
return out, err
}
params["uploadId"] = imur.UploadID
resp, err := bucket.do("GET", imur.Key, params, options, nil, nil)
if err != nil {
return out, err
}
defer resp.Body.Close()
err = xmlUnmarshal(resp.Body, &out)
if err != nil {
return out, err
}
err = decodeListUploadedPartsResult(&out)
return out, err
}
// ListMultipartUploads lists all ongoing multipart upload tasks
//
// options listObject's filter. Prefix specifies the returned object's prefix; KeyMarker specifies the returned object's start point in lexicographic order;
//
// MaxKeys specifies the max entries to return; Delimiter is the character for grouping object keys.
//
// ListMultipartUploadResponse the return value if it succeeds, only valid when error is nil.
// error it's nil if the operation succeeds, otherwise it's an error object.
func (bucket Bucket) ListMultipartUploads(options ...Option) (ListMultipartUploadResult, error) {
var out ListMultipartUploadResult
options = append(options, EncodingType("url"))
params, err := GetRawParams(options)
if err != nil {
return out, err
}
params["uploads"] = nil
resp, err := bucket.doInner("GET", "", params, options, nil, nil)
if err != nil {
return out, err
}
defer resp.Body.Close()
err = xmlUnmarshal(resp.Body, &out)
if err != nil {
return out, err
}
err = decodeListMultipartUploadResult(&out)
return out, err
}
backend/vendor/github.com/aliyun/aliyun-oss-go-sdk/oss/option.go
Generated Vendored
+735
View File
@@ -0,0 +1,735 @@
package oss
import (
"context"
"fmt"
"io/ioutil"
"net/http"
"net/url"
"strconv"
"strings"
"time"
)
type optionType string
const (
optionParam optionType = "HTTPParameter" // URL parameter
optionHTTP optionType = "HTTPHeader" // HTTP header
optionContext optionType = "HTTPContext" // context
optionArg optionType = "FuncArgument" // Function argument
)
const (
deleteObjectsQuiet = "delete-objects-quiet"
routineNum = "x-routine-num"
checkpointConfig = "x-cp-config"
initCRC64 = "init-crc64"
progressListener = "x-progress-listener"
storageClass = "storage-class"
responseHeader = "x-response-header"
redundancyType = "redundancy-type"
objectHashFunc = "object-hash-func"
responseBody = "x-response-body"
contextArg = "x-context-arg"
)
type (
optionValue struct {
Value interface{}
Type optionType
}
// Option HTTP option
Option func(map[string]optionValue) error
)
// ACL is an option to set X-Oss-Acl header
func ACL(acl ACLType) Option {
return setHeader(HTTPHeaderOssACL, string(acl))
}
// ContentType is an option to set Content-Type header
func ContentType(value string) Option {
return setHeader(HTTPHeaderContentType, value)
}
// ContentLength is an option to set Content-Length header
func ContentLength(length int64) Option {
return setHeader(HTTPHeaderContentLength, strconv.FormatInt(length, 10))
}
// CacheControl is an option to set Cache-Control header
func CacheControl(value string) Option {
return setHeader(HTTPHeaderCacheControl, value)
}
// ContentDisposition is an option to set Content-Disposition header
func ContentDisposition(value string) Option {
return setHeader(HTTPHeaderContentDisposition, value)
}
// ContentEncoding is an option to set Content-Encoding header
func ContentEncoding(value string) Option {
return setHeader(HTTPHeaderContentEncoding, value)
}
// ContentLanguage is an option to set Content-Language header
func ContentLanguage(value string) Option {
return setHeader(HTTPHeaderContentLanguage, value)
}
// ContentMD5 is an option to set Content-MD5 header
func ContentMD5(value string) Option {
return setHeader(HTTPHeaderContentMD5, value)
}
// Expires is an option to set Expires header
func Expires(t time.Time) Option {
return setHeader(HTTPHeaderExpires, t.Format(http.TimeFormat))
}
// Meta is an option to set Meta header
func Meta(key, value string) Option {
return setHeader(HTTPHeaderOssMetaPrefix+key, value)
}
// Range is an option to set Range header, [start, end]
func Range(start, end int64) Option {
return setHeader(HTTPHeaderRange, fmt.Sprintf("bytes=%d-%d", start, end))
}
// NormalizedRange is an option to set Range header, such as 1024-2048 or 1024- or -2048
func NormalizedRange(nr string) Option {
return setHeader(HTTPHeaderRange, fmt.Sprintf("bytes=%s", strings.TrimSpace(nr)))
}
// AcceptEncoding is an option to set Accept-Encoding header
func AcceptEncoding(value string) Option {
return setHeader(HTTPHeaderAcceptEncoding, value)
}
// IfModifiedSince is an option to set If-Modified-Since header
func IfModifiedSince(t time.Time) Option {
return setHeader(HTTPHeaderIfModifiedSince, t.Format(http.TimeFormat))
}
// IfUnmodifiedSince is an option to set If-Unmodified-Since header
func IfUnmodifiedSince(t time.Time) Option {
return setHeader(HTTPHeaderIfUnmodifiedSince, t.Format(http.TimeFormat))
}
// IfMatch is an option to set If-Match header
func IfMatch(value string) Option {
return setHeader(HTTPHeaderIfMatch, value)
}
// IfNoneMatch is an option to set IfNoneMatch header
func IfNoneMatch(value string) Option {
return setHeader(HTTPHeaderIfNoneMatch, value)
}
// CopySource is an option to set X-Oss-Copy-Source header
func CopySource(sourceBucket, sourceObject string) Option {
return setHeader(HTTPHeaderOssCopySource, "/"+sourceBucket+"/"+sourceObject)
}
// CopySourceVersion is an option to set X-Oss-Copy-Source header,include versionId
func CopySourceVersion(sourceBucket, sourceObject string, versionId string) Option {
return setHeader(HTTPHeaderOssCopySource, "/"+sourceBucket+"/"+sourceObject+"?"+"versionId="+versionId)
}
// CopySourceRange is an option to set X-Oss-Copy-Source header
func CopySourceRange(startPosition, partSize int64) Option {
val := "bytes=" + strconv.FormatInt(startPosition, 10) + "-" +
strconv.FormatInt((startPosition+partSize-1), 10)
return setHeader(HTTPHeaderOssCopySourceRange, val)
}
// CopySourceIfMatch is an option to set X-Oss-Copy-Source-If-Match header
func CopySourceIfMatch(value string) Option {
return setHeader(HTTPHeaderOssCopySourceIfMatch, value)
}
// CopySourceIfNoneMatch is an option to set X-Oss-Copy-Source-If-None-Match header
func CopySourceIfNoneMatch(value string) Option {
return setHeader(HTTPHeaderOssCopySourceIfNoneMatch, value)
}
// CopySourceIfModifiedSince is an option to set X-Oss-CopySource-If-Modified-Since header
func CopySourceIfModifiedSince(t time.Time) Option {
return setHeader(HTTPHeaderOssCopySourceIfModifiedSince, t.Format(http.TimeFormat))
}
// CopySourceIfUnmodifiedSince is an option to set X-Oss-Copy-Source-If-Unmodified-Since header
func CopySourceIfUnmodifiedSince(t time.Time) Option {
return setHeader(HTTPHeaderOssCopySourceIfUnmodifiedSince, t.Format(http.TimeFormat))
}
// MetadataDirective is an option to set X-Oss-Metadata-Directive header
func MetadataDirective(directive MetadataDirectiveType) Option {
return setHeader(HTTPHeaderOssMetadataDirective, string(directive))
}
// ServerSideEncryption is an option to set X-Oss-Server-Side-Encryption header
func ServerSideEncryption(value string) Option {
return setHeader(HTTPHeaderOssServerSideEncryption, value)
}
// ServerSideEncryptionKeyID is an option to set X-Oss-Server-Side-Encryption-Key-Id header
func ServerSideEncryptionKeyID(value string) Option {
return setHeader(HTTPHeaderOssServerSideEncryptionKeyID, value)
}
// ServerSideDataEncryption is an option to set X-Oss-Server-Side-Data-Encryption header
func ServerSideDataEncryption(value string) Option {
return setHeader(HTTPHeaderOssServerSideDataEncryption, value)
}
// SSECAlgorithm is an option to set X-Oss-Server-Side-Encryption-Customer-Algorithm header
func SSECAlgorithm(value string) Option {
return setHeader(HTTPHeaderSSECAlgorithm, value)
}
// SSECKey is an option to set X-Oss-Server-Side-Encryption-Customer-Key header
func SSECKey(value string) Option {
return setHeader(HTTPHeaderSSECKey, value)
}
// SSECKeyMd5 is an option to set X-Oss-Server-Side-Encryption-Customer-Key-Md5 header
func SSECKeyMd5(value string) Option {
return setHeader(HTTPHeaderSSECKeyMd5, value)
}
// ObjectACL is an option to set X-Oss-Object-Acl header
func ObjectACL(acl ACLType) Option {
return setHeader(HTTPHeaderOssObjectACL, string(acl))
}
// symlinkTarget is an option to set X-Oss-Symlink-Target
func symlinkTarget(targetObjectKey string) Option {
return setHeader(HTTPHeaderOssSymlinkTarget, targetObjectKey)
}
// Origin is an option to set Origin header
func Origin(value string) Option {
return setHeader(HTTPHeaderOrigin, value)
}
// ObjectStorageClass is an option to set the storage class of object
func ObjectStorageClass(storageClass StorageClassType) Option {
return setHeader(HTTPHeaderOssStorageClass, string(storageClass))
}
// Callback is an option to set callback values
func Callback(callback string) Option {
return setHeader(HTTPHeaderOssCallback, callback)
}
// CallbackVar is an option to set callback user defined values
func CallbackVar(callbackVar string) Option {
return setHeader(HTTPHeaderOssCallbackVar, callbackVar)
}
// RequestPayer is an option to set payer who pay for the request
func RequestPayer(payerType PayerType) Option {
return setHeader(HTTPHeaderOssRequester, strings.ToLower(string(payerType)))
}
// RequestPayerParam is an option to set payer who pay for the request
func RequestPayerParam(payerType PayerType) Option {
return addParam(strings.ToLower(HTTPHeaderOssRequester), strings.ToLower(string(payerType)))
}
// SetTagging is an option to set object tagging
func SetTagging(tagging Tagging) Option {
if len(tagging.Tags) == 0 {
return nil
}
taggingValue := ""
for index, tag := range tagging.Tags {
if index != 0 {
taggingValue += "&"
}
taggingValue += url.QueryEscape(tag.Key) + "=" + url.QueryEscape(tag.Value)
}
return setHeader(HTTPHeaderOssTagging, taggingValue)
}
// TaggingDirective is an option to set X-Oss-Metadata-Directive header
func TaggingDirective(directive TaggingDirectiveType) Option {
return setHeader(HTTPHeaderOssTaggingDirective, string(directive))
}
// ACReqMethod is an option to set Access-Control-Request-Method header
func ACReqMethod(value string) Option {
return setHeader(HTTPHeaderACReqMethod, value)
}
// ACReqHeaders is an option to set Access-Control-Request-Headers header
func ACReqHeaders(value string) Option {
return setHeader(HTTPHeaderACReqHeaders, value)
}
// TrafficLimitHeader is an option to set X-Oss-Traffic-Limit
func TrafficLimitHeader(value int64) Option {
return setHeader(HTTPHeaderOssTrafficLimit, strconv.FormatInt(value, 10))
}
// UserAgentHeader is an option to set HTTPHeaderUserAgent
func UserAgentHeader(ua string) Option {
return setHeader(HTTPHeaderUserAgent, ua)
}
// ForbidOverWrite is an option to set X-Oss-Forbid-Overwrite
func ForbidOverWrite(forbidWrite bool) Option {
if forbidWrite {
return setHeader(HTTPHeaderOssForbidOverWrite, "true")
} else {
return setHeader(HTTPHeaderOssForbidOverWrite, "false")
}
}
// RangeBehavior is an option to set Range value, such as "standard"
func RangeBehavior(value string) Option {
return setHeader(HTTPHeaderOssRangeBehavior, value)
}
func PartHashCtxHeader(value string) Option {
return setHeader(HTTPHeaderOssHashCtx, value)
}
func PartMd5CtxHeader(value string) Option {
return setHeader(HTTPHeaderOssMd5Ctx, value)
}
func PartHashCtxParam(value string) Option {
return addParam("x-oss-hash-ctx", value)
}
func PartMd5CtxParam(value string) Option {
return addParam("x-oss-md5-ctx", value)
}
// Delimiter is an option to set delimiler parameter
func Delimiter(value string) Option {
return addParam("delimiter", value)
}
// Marker is an option to set marker parameter
func Marker(value string) Option {
return addParam("marker", value)
}
// MaxKeys is an option to set maxkeys parameter
func MaxKeys(value int) Option {
return addParam("max-keys", strconv.Itoa(value))
}
// Prefix is an option to set prefix parameter
func Prefix(value string) Option {
return addParam("prefix", value)
}
// EncodingType is an option to set encoding-type parameter
func EncodingType(value string) Option {
return addParam("encoding-type", value)
}
// MaxUploads is an option to set max-uploads parameter
func MaxUploads(value int) Option {
return addParam("max-uploads", strconv.Itoa(value))
}
// KeyMarker is an option to set key-marker parameter
func KeyMarker(value string) Option {
return addParam("key-marker", value)
}
// VersionIdMarker is an option to set version-id-marker parameter
func VersionIdMarker(value string) Option {
return addParam("version-id-marker", value)
}
// VersionId is an option to set versionId parameter
func VersionId(value string) Option {
return addParam("versionId", value)
}
// TagKey is an option to set tag key parameter
func TagKey(value string) Option {
return addParam("tag-key", value)
}
// TagValue is an option to set tag value parameter
func TagValue(value string) Option {
return addParam("tag-value", value)
}
// UploadIDMarker is an option to set upload-id-marker parameter
func UploadIDMarker(value string) Option {
return addParam("upload-id-marker", value)
}
// MaxParts is an option to set max-parts parameter
func MaxParts(value int) Option {
return addParam("max-parts", strconv.Itoa(value))
}
// PartNumberMarker is an option to set part-number-marker parameter
func PartNumberMarker(value int) Option {
return addParam("part-number-marker", strconv.Itoa(value))
}
// Sequential is an option to set sequential parameter for InitiateMultipartUpload
func Sequential() Option {
return addParam("sequential", "")
}
// WithHashContext is an option to set withHashContext parameter for InitiateMultipartUpload
func WithHashContext() Option {
return addParam("withHashContext", "")
}
// EnableMd5 is an option to set x-oss-enable-md5 parameter for InitiateMultipartUpload
func EnableMd5() Option {
return addParam("x-oss-enable-md5", "")
}
// EnableSha1 is an option to set x-oss-enable-sha1 parameter for InitiateMultipartUpload
func EnableSha1() Option {
return addParam("x-oss-enable-sha1", "")
}
// EnableSha256 is an option to set x-oss-enable-sha256 parameter for InitiateMultipartUpload
func EnableSha256() Option {
return addParam("x-oss-enable-sha256", "")
}
// ListType is an option to set List-type parameter for ListObjectsV2
func ListType(value int) Option {
return addParam("list-type", strconv.Itoa(value))
}
// StartAfter is an option to set start-after parameter for ListObjectsV2
func StartAfter(value string) Option {
return addParam("start-after", value)
}
// ContinuationToken is an option to set Continuation-token parameter for ListObjectsV2
func ContinuationToken(value string) Option {
if value == "" {
return addParam("continuation-token", nil)
}
return addParam("continuation-token", value)
}
// FetchOwner is an option to set Fetch-owner parameter for ListObjectsV2
func FetchOwner(value bool) Option {
if value {
return addParam("fetch-owner", "true")
}
return addParam("fetch-owner", "false")
}
// DeleteObjectsQuiet false:DeleteObjects in verbose mode; true:DeleteObjects in quite mode. Default is false.
func DeleteObjectsQuiet(isQuiet bool) Option {
return addArg(deleteObjectsQuiet, isQuiet)
}
// StorageClass bucket storage class
func StorageClass(value StorageClassType) Option {
return addArg(storageClass, value)
}
// RedundancyType bucket data redundancy type
func RedundancyType(value DataRedundancyType) Option {
return addArg(redundancyType, value)
}
// RedundancyType bucket data redundancy type
func ObjectHashFunc(value ObjecthashFuncType) Option {
return addArg(objectHashFunc, value)
}
// WithContext returns an option that sets the context for requests.
func WithContext(ctx context.Context) Option {
return addArg(contextArg, ctx)
}
// Checkpoint configuration
type cpConfig struct {
IsEnable bool
FilePath string
DirPath string
}
// Checkpoint sets the isEnable flag and checkpoint file path for DownloadFile/UploadFile.
func Checkpoint(isEnable bool, filePath string) Option {
return addArg(checkpointConfig, &cpConfig{IsEnable: isEnable, FilePath: filePath})
}
// CheckpointDir sets the isEnable flag and checkpoint dir path for DownloadFile/UploadFile.
func CheckpointDir(isEnable bool, dirPath string) Option {
return addArg(checkpointConfig, &cpConfig{IsEnable: isEnable, DirPath: dirPath})
}
// Routines DownloadFile/UploadFile routine count
func Routines(n int) Option {
return addArg(routineNum, n)
}
// InitCRC Init AppendObject CRC
func InitCRC(initCRC uint64) Option {
return addArg(initCRC64, initCRC)
}
// Progress set progress listener
func Progress(listener ProgressListener) Option {
return addArg(progressListener, listener)
}
// GetResponseHeader for get response http header
func GetResponseHeader(respHeader *http.Header) Option {
return addArg(responseHeader, respHeader)
}
// CallbackResult for get response of call back
func CallbackResult(body *[]byte) Option {
return addArg(responseBody, body)
}
// ResponseContentType is an option to set response-content-type param
func ResponseContentType(value string) Option {
return addParam("response-content-type", value)
}
// ResponseContentLanguage is an option to set response-content-language param
func ResponseContentLanguage(value string) Option {
return addParam("response-content-language", value)
}
// ResponseExpires is an option to set response-expires param
func ResponseExpires(value string) Option {
return addParam("response-expires", value)
}
// ResponseCacheControl is an option to set response-cache-control param
func ResponseCacheControl(value string) Option {
return addParam("response-cache-control", value)
}
// ResponseContentDisposition is an option to set response-content-disposition param
func ResponseContentDisposition(value string) Option {
return addParam("response-content-disposition", value)
}
// ResponseContentEncoding is an option to set response-content-encoding param
func ResponseContentEncoding(value string) Option {
return addParam("response-content-encoding", value)
}
// Process is an option to set x-oss-process param
func Process(value string) Option {
return addParam("x-oss-process", value)
}
// TrafficLimitParam is a option to set x-oss-traffic-limit
func TrafficLimitParam(value int64) Option {
return addParam("x-oss-traffic-limit", strconv.FormatInt(value, 10))
}
// SetHeader Allow users to set personalized http headers
func SetHeader(key string, value interface{}) Option {
return setHeader(key, value)
}
// AddParam Allow users to set personalized http params
func AddParam(key string, value interface{}) Option {
return addParam(key, value)
}
func setHeader(key string, value interface{}) Option {
return func(params map[string]optionValue) error {
if value == nil {
return nil
}
params[key] = optionValue{value, optionHTTP}
return nil
}
}
func addParam(key string, value interface{}) Option {
return func(params map[string]optionValue) error {
if value == nil {
return nil
}
params[key] = optionValue{value, optionParam}
return nil
}
}
func addArg(key string, value interface{}) Option {
return func(params map[string]optionValue) error {
if value == nil {
return nil
}
params[key] = optionValue{value, optionArg}
return nil
}
}
func handleOptions(headers map[string]string, options []Option) error {
params := map[string]optionValue{}
for _, option := range options {
if option != nil {
if err := option(params); err != nil {
return err
}
}
}
for k, v := range params {
if v.Type == optionHTTP {
headers[k] = v.Value.(string)
}
}
return nil
}
func GetRawParams(options []Option) (map[string]interface{}, error) {
// Option
params := map[string]optionValue{}
for _, option := range options {
if option != nil {
if err := option(params); err != nil {
return nil, err
}
}
}
paramsm := map[string]interface{}{}
// Serialize
for k, v := range params {
if v.Type == optionParam {
vs := params[k]
paramsm[k] = vs.Value.(string)
}
}
return paramsm, nil
}
func FindOption(options []Option, param string, defaultVal interface{}) (interface{}, error) {
params := map[string]optionValue{}
for _, option := range options {
if option != nil {
if err := option(params); err != nil {
return nil, err
}
}
}
if val, ok := params[param]; ok {
return val.Value, nil
}
return defaultVal, nil
}
func IsOptionSet(options []Option, option string) (bool, interface{}, error) {
params := map[string]optionValue{}
for _, option := range options {
if option != nil {
if err := option(params); err != nil {
return false, nil, err
}
}
}
if val, ok := params[option]; ok {
return true, val.Value, nil
}
return false, nil, nil
}
func DeleteOption(options []Option, strKey string) []Option {
var outOption []Option
params := map[string]optionValue{}
for _, option := range options {
if option != nil {
option(params)
_, exist := params[strKey]
if !exist {
outOption = append(outOption, option)
} else {
delete(params, strKey)
}
}
}
return outOption
}
func GetRequestId(header http.Header) string {
return header.Get("x-oss-request-id")
}
func GetVersionId(header http.Header) string {
return header.Get("x-oss-version-id")
}
func GetCopySrcVersionId(header http.Header) string {
return header.Get("x-oss-copy-source-version-id")
}
func GetDeleteMark(header http.Header) bool {
value := header.Get("x-oss-delete-marker")
if strings.ToUpper(value) == "TRUE" {
return true
}
return false
}
func GetQosDelayTime(header http.Header) string {
return header.Get("x-oss-qos-delay-time")
}
// ForbidOverWrite is an option to set X-Oss-Forbid-Overwrite
func AllowSameActionOverLap(enabled bool) Option {
if enabled {
return setHeader(HTTPHeaderAllowSameActionOverLap, "true")
} else {
return setHeader(HTTPHeaderAllowSameActionOverLap, "false")
}
}
func GetCallbackBody(options []Option, resp *Response, callbackSet bool) error {
var err error
// get response body
if callbackSet {
err = setBody(options, resp)
} else {
callback, _ := FindOption(options, HTTPHeaderOssCallback, nil)
if callback != nil {
err = setBody(options, resp)
}
}
return err
}
func setBody(options []Option, resp *Response) error {
respBody, _ := FindOption(options, responseBody, nil)
if respBody != nil && resp != nil {
pRespBody := respBody.(*[]byte)
pBody, err := ioutil.ReadAll(resp.Body)
if err != nil {
return err
}
if pBody != nil {
*pRespBody = pBody
}
}
return nil
}
backend/vendor/github.com/aliyun/aliyun-oss-go-sdk/oss/progress.go
Generated Vendored
+116
View File
@@ -0,0 +1,116 @@
package oss
import (
"io"
)
// ProgressEventType defines transfer progress event type
type ProgressEventType int
const (
// TransferStartedEvent transfer started, set TotalBytes
TransferStartedEvent ProgressEventType = 1 + iota
// TransferDataEvent transfer data, set ConsumedBytes and TotalBytes
TransferDataEvent
// TransferCompletedEvent transfer completed
TransferCompletedEvent
// TransferFailedEvent transfer encounters an error
TransferFailedEvent
)
// ProgressEvent defines progress event
type ProgressEvent struct {
ConsumedBytes int64
TotalBytes int64
RwBytes int64
EventType ProgressEventType
}
// ProgressListener listens progress change
type ProgressListener interface {
ProgressChanged(event *ProgressEvent)
}
// -------------------- Private --------------------
func newProgressEvent(eventType ProgressEventType, consumed, total int64, rwBytes int64) *ProgressEvent {
return &ProgressEvent{
ConsumedBytes: consumed,
TotalBytes: total,
RwBytes: rwBytes,
EventType: eventType}
}
// publishProgress
func publishProgress(listener ProgressListener, event *ProgressEvent) {
if listener != nil && event != nil {
listener.ProgressChanged(event)
}
}
type readerTracker struct {
completedBytes int64
}
type teeReader struct {
reader io.Reader
writer io.Writer
listener ProgressListener
consumedBytes int64
totalBytes int64
tracker *readerTracker
}
// TeeReader returns a Reader that writes to w what it reads from r.
// All reads from r performed through it are matched with
// corresponding writes to w. There is no internal buffering -
// the write must complete before the read completes.
// Any error encountered while writing is reported as a read error.
func TeeReader(reader io.Reader, writer io.Writer, totalBytes int64, listener ProgressListener, tracker *readerTracker) io.ReadCloser {
return &teeReader{
reader: reader,
writer: writer,
listener: listener,
consumedBytes: 0,
totalBytes: totalBytes,
tracker: tracker,
}
}
func (t *teeReader) Read(p []byte) (n int, err error) {
n, err = t.reader.Read(p)
// Read encountered error
if err != nil && err != io.EOF {
event := newProgressEvent(TransferFailedEvent, t.consumedBytes, t.totalBytes, 0)
publishProgress(t.listener, event)
}
if n > 0 {
t.consumedBytes += int64(n)
// CRC
if t.writer != nil {
if n, err := t.writer.Write(p[:n]); err != nil {
return n, err
}
}
// Progress
if t.listener != nil {
event := newProgressEvent(TransferDataEvent, t.consumedBytes, t.totalBytes, int64(n))
publishProgress(t.listener, event)
}
// Track
if t.tracker != nil {
t.tracker.completedBytes = t.consumedBytes
}
}
return
}
func (t *teeReader) Close() error {
if rc, ok := t.reader.(io.ReadCloser); ok {
return rc.Close()
}
return nil
}
backend/vendor/github.com/aliyun/aliyun-oss-go-sdk/oss/redirect_1_6.go
Generated Vendored
+12
View File
@@ -0,0 +1,12 @@
//go:build !go1.7
// +build !go1.7
package oss
import "net/http"
// http.ErrUseLastResponse only is defined go1.7 onward
func disableHTTPRedirect(client *http.Client) {
}
backend/vendor/github.com/aliyun/aliyun-oss-go-sdk/oss/redirect_1_7.go
Generated Vendored
+13
View File
@@ -0,0 +1,13 @@
//go:build go1.7
// +build go1.7
package oss
import "net/http"
// http.ErrUseLastResponse only is defined go1.7 onward
func disableHTTPRedirect(client *http.Client) {
client.CheckRedirect = func(req *http.Request, via []*http.Request) error {
return http.ErrUseLastResponse
}
}
backend/vendor/github.com/aliyun/aliyun-oss-go-sdk/oss/select_object.go
Generated Vendored
+197
View File
@@ -0,0 +1,197 @@
package oss
import (
"bytes"
"encoding/xml"
"hash/crc32"
"io"
"io/ioutil"
"net/http"
"os"
"strings"
)
// CreateSelectCsvObjectMeta is Creating csv object meta
//
// key the object key.
// csvMeta the csv file meta
// options the options for create csv Meta of the object.
//
// MetaEndFrameCSV the csv file meta info
// error it's nil if no error, otherwise it's an error object.
//
func (bucket Bucket) CreateSelectCsvObjectMeta(key string, csvMeta CsvMetaRequest, options ...Option) (MetaEndFrameCSV, error) {
var endFrame MetaEndFrameCSV
params := map[string]interface{}{}
params["x-oss-process"] = "csv/meta"
csvMeta.encodeBase64()
bs, err := xml.Marshal(csvMeta)
if err != nil {
return endFrame, err
}
buffer := new(bytes.Buffer)
buffer.Write(bs)
resp, err := bucket.DoPostSelectObject(key, params, buffer, options...)
if err != nil {
return endFrame, err
}
defer resp.Body.Close()
_, err = ioutil.ReadAll(resp)
return resp.Frame.MetaEndFrameCSV, err
}
// CreateSelectJsonObjectMeta is Creating json object meta
//
// key the object key.
// csvMeta the json file meta
// options the options for create json Meta of the object.
//
// MetaEndFrameJSON the json file meta info
// error it's nil if no error, otherwise it's an error object.
//
func (bucket Bucket) CreateSelectJsonObjectMeta(key string, jsonMeta JsonMetaRequest, options ...Option) (MetaEndFrameJSON, error) {
var endFrame MetaEndFrameJSON
params := map[string]interface{}{}
params["x-oss-process"] = "json/meta"
bs, err := xml.Marshal(jsonMeta)
if err != nil {
return endFrame, err
}
buffer := new(bytes.Buffer)
buffer.Write(bs)
resp, err := bucket.DoPostSelectObject(key, params, buffer, options...)
if err != nil {
return endFrame, err
}
defer resp.Body.Close()
_, err = ioutil.ReadAll(resp)
return resp.Frame.MetaEndFrameJSON, err
}
// SelectObject is the select object api, approve csv and json file.
//
// key the object key.
// selectReq the request data for select object
// options the options for select file of the object.
//
// o.ReadCloser reader instance for reading data from response. It must be called close() after the usage and only valid when error is nil.
// error it's nil if no error, otherwise it's an error object.
//
func (bucket Bucket) SelectObject(key string, selectReq SelectRequest, options ...Option) (io.ReadCloser, error) {
params := map[string]interface{}{}
if selectReq.InputSerializationSelect.JsonBodyInput.JsonIsEmpty() {
params["x-oss-process"] = "csv/select" // default select csv file
} else {
params["x-oss-process"] = "json/select"
}
selectReq.encodeBase64()
bs, err := xml.Marshal(selectReq)
if err != nil {
return nil, err
}
buffer := new(bytes.Buffer)
buffer.Write(bs)
resp, err := bucket.DoPostSelectObject(key, params, buffer, options...)
if err != nil {
return nil, err
}
if selectReq.OutputSerializationSelect.EnablePayloadCrc != nil && *selectReq.OutputSerializationSelect.EnablePayloadCrc == true {
resp.Frame.EnablePayloadCrc = true
}
resp.Frame.OutputRawData = strings.ToUpper(resp.Headers.Get("x-oss-select-output-raw")) == "TRUE"
return resp, err
}
// DoPostSelectObject is the SelectObject/CreateMeta api, approve csv and json file.
//
// key the object key.
// params the resource of oss approve csv/meta, json/meta, csv/select, json/select.
// buf the request data trans to buffer.
// options the options for select file of the object.
//
// SelectObjectResponse the response of select object.
// error it's nil if no error, otherwise it's an error object.
//
func (bucket Bucket) DoPostSelectObject(key string, params map[string]interface{}, buf *bytes.Buffer, options ...Option) (*SelectObjectResponse, error) {
resp, err := bucket.do("POST", key, params, options, buf, nil)
if err != nil {
return nil, err
}
result := &SelectObjectResponse{
Body: resp.Body,
StatusCode: resp.StatusCode,
Frame: SelectObjectResult{},
}
result.Headers = resp.Headers
// result.Frame = SelectObjectResult{}
result.ReadTimeOut = bucket.GetConfig().Timeout
// Progress
listener := GetProgressListener(options)
// CRC32
crcCalc := crc32.NewIEEE()
result.WriterForCheckCrc32 = crcCalc
result.Body = TeeReader(resp.Body, nil, 0, listener, nil)
err = CheckRespCode(resp.StatusCode, []int{http.StatusPartialContent, http.StatusOK})
return result, err
}
// SelectObjectIntoFile is the selectObject to file api
//
// key the object key.
// fileName saving file's name to localstation.
// selectReq the request data for select object
// options the options for select file of the object.
//
// error it's nil if no error, otherwise it's an error object.
//
func (bucket Bucket) SelectObjectIntoFile(key, fileName string, selectReq SelectRequest, options ...Option) error {
tempFilePath := fileName + TempFileSuffix
params := map[string]interface{}{}
if selectReq.InputSerializationSelect.JsonBodyInput.JsonIsEmpty() {
params["x-oss-process"] = "csv/select" // default select csv file
} else {
params["x-oss-process"] = "json/select"
}
selectReq.encodeBase64()
bs, err := xml.Marshal(selectReq)
if err != nil {
return err
}
buffer := new(bytes.Buffer)
buffer.Write(bs)
resp, err := bucket.DoPostSelectObject(key, params, buffer, options...)
if err != nil {
return err
}
defer resp.Close()
// If the local file does not exist, create a new one. If it exists, overwrite it.
fd, err := os.OpenFile(tempFilePath, os.O_CREATE|os.O_TRUNC|os.O_WRONLY, FilePermMode)
if err != nil {
return err
}
// Copy the data to the local file path.
_, err = io.Copy(fd, resp)
fd.Close()
if err != nil {
return err
}
return os.Rename(tempFilePath, fileName)
}
backend/vendor/github.com/aliyun/aliyun-oss-go-sdk/oss/select_object_type.go
Generated Vendored
+365
View File
@@ -0,0 +1,365 @@
package oss
import (
"bytes"
"encoding/binary"
"fmt"
"hash"
"hash/crc32"
"io"
"net/http"
"time"
)
// The adapter class for Select object's response.
// The response consists of frames. Each frame has the following format:
// Type | Payload Length | Header Checksum | Payload | Payload Checksum
// |<4-->| <--4 bytes------><---4 bytes-------><-n/a-----><--4 bytes--------->
// And we have three kind of frames.
// Data Frame:
// Type:8388609
// Payload: Offset | Data
// <-8 bytes>
// Continuous Frame
// Type:8388612
// Payload: Offset (8-bytes)
// End Frame
// Type:8388613
// Payload: Offset | total scanned bytes | http status code | error message
// <-- 8bytes--><-----8 bytes--------><---4 bytes-------><---variabe--->
// SelectObjectResponse defines HTTP response from OSS SelectObject
type SelectObjectResponse struct {
StatusCode int
Headers http.Header
Body io.ReadCloser
Frame SelectObjectResult
ReadTimeOut uint
ClientCRC32 uint32
ServerCRC32 uint32
WriterForCheckCrc32 hash.Hash32
Finish bool
}
func (sr *SelectObjectResponse) Read(p []byte) (n int, err error) {
n, err = sr.readFrames(p)
return
}
// Close http reponse body
func (sr *SelectObjectResponse) Close() error {
return sr.Body.Close()
}
// PostSelectResult is the request of SelectObject
type PostSelectResult struct {
Response *SelectObjectResponse
}
// readFrames is read Frame
func (sr *SelectObjectResponse) readFrames(p []byte) (int, error) {
var nn int
var err error
var checkValid bool
if sr.Frame.OutputRawData == true {
nn, err = sr.Body.Read(p)
return nn, err
}
if sr.Finish {
return 0, io.EOF
}
for {
// if this Frame is Readed, then not reading Header
if sr.Frame.OpenLine != true {
err = sr.analysisHeader()
if err != nil {
return nn, err
}
}
if sr.Frame.FrameType == DataFrameType {
n, err := sr.analysisData(p[nn:])
if err != nil {
return nn, err
}
nn += n
// if this Frame is readed all data, then empty the Frame to read it with next frame
if sr.Frame.ConsumedBytesLength == sr.Frame.PayloadLength-8 {
checkValid, err = sr.checkPayloadSum()
if err != nil || !checkValid {
return nn, fmt.Errorf("%s", err.Error())
}
sr.emptyFrame()
}
if nn == len(p) {
return nn, nil
}
} else if sr.Frame.FrameType == ContinuousFrameType {
checkValid, err = sr.checkPayloadSum()
if err != nil || !checkValid {
return nn, fmt.Errorf("%s", err.Error())
}
sr.Frame.OpenLine = false
} else if sr.Frame.FrameType == EndFrameType {
err = sr.analysisEndFrame()
if err != nil {
return nn, err
}
checkValid, err = sr.checkPayloadSum()
if checkValid {
sr.Finish = true
}
return nn, err
} else if sr.Frame.FrameType == MetaEndFrameCSVType {
err = sr.analysisMetaEndFrameCSV()
if err != nil {
return nn, err
}
checkValid, err = sr.checkPayloadSum()
if checkValid {
sr.Finish = true
}
return nn, err
} else if sr.Frame.FrameType == MetaEndFrameJSONType {
err = sr.analysisMetaEndFrameJSON()
if err != nil {
return nn, err
}
checkValid, err = sr.checkPayloadSum()
if checkValid {
sr.Finish = true
}
return nn, err
}
}
return nn, nil
}
type chanReadIO struct {
readLen int
err error
}
func (sr *SelectObjectResponse) readLen(p []byte, timeOut time.Duration) (int, error) {
r := sr.Body
ch := make(chan chanReadIO, 1)
defer close(ch)
go func(p []byte) {
var needReadLength int
readChan := chanReadIO{}
needReadLength = len(p)
for {
n, err := r.Read(p[readChan.readLen:needReadLength])
readChan.readLen += n
if err != nil {
readChan.err = err
ch <- readChan
return
}
if readChan.readLen == needReadLength {
break
}
}
ch <- readChan
}(p)
select {
case <-time.After(time.Second * timeOut):
return 0, fmt.Errorf("requestId: %s, readLen timeout, timeout is %d(second),need read:%d", sr.Headers.Get(HTTPHeaderOssRequestID), timeOut, len(p))
case result := <-ch:
return result.readLen, result.err
}
}
// analysisHeader is reading selectObject response body's header
func (sr *SelectObjectResponse) analysisHeader() error {
headFrameByte := make([]byte, 20)
_, err := sr.readLen(headFrameByte, time.Duration(sr.ReadTimeOut))
if err != nil {
return fmt.Errorf("requestId: %s, Read response frame header failure,err:%s", sr.Headers.Get(HTTPHeaderOssRequestID), err.Error())
}
frameTypeByte := headFrameByte[0:4]
sr.Frame.Version = frameTypeByte[0]
frameTypeByte[0] = 0
bytesToInt(frameTypeByte, &sr.Frame.FrameType)
if sr.Frame.FrameType != DataFrameType && sr.Frame.FrameType != ContinuousFrameType &&
sr.Frame.FrameType != EndFrameType && sr.Frame.FrameType != MetaEndFrameCSVType && sr.Frame.FrameType != MetaEndFrameJSONType {
return fmt.Errorf("requestId: %s, Unexpected frame type: %d", sr.Headers.Get(HTTPHeaderOssRequestID), sr.Frame.FrameType)
}
payloadLengthByte := headFrameByte[4:8]
bytesToInt(payloadLengthByte, &sr.Frame.PayloadLength)
headCheckSumByte := headFrameByte[8:12]
bytesToInt(headCheckSumByte, &sr.Frame.HeaderCheckSum)
byteOffset := headFrameByte[12:20]
bytesToInt(byteOffset, &sr.Frame.Offset)
sr.Frame.OpenLine = true
err = sr.writerCheckCrc32(byteOffset)
return err
}
// analysisData is reading the DataFrameType data of selectObject response body
func (sr *SelectObjectResponse) analysisData(p []byte) (int, error) {
var needReadLength int32
lenP := int32(len(p))
restByteLength := sr.Frame.PayloadLength - 8 - sr.Frame.ConsumedBytesLength
if lenP <= restByteLength {
needReadLength = lenP
} else {
needReadLength = restByteLength
}
n, err := sr.readLen(p[:needReadLength], time.Duration(sr.ReadTimeOut))
if err != nil {
return n, fmt.Errorf("read frame data error,%s", err.Error())
}
sr.Frame.ConsumedBytesLength += int32(n)
err = sr.writerCheckCrc32(p[:n])
return n, err
}
// analysisEndFrame is reading the EndFrameType data of selectObject response body
func (sr *SelectObjectResponse) analysisEndFrame() error {
var eF EndFrame
payLoadBytes := make([]byte, sr.Frame.PayloadLength-8)
_, err := sr.readLen(payLoadBytes, time.Duration(sr.ReadTimeOut))
if err != nil {
return fmt.Errorf("read end frame error:%s", err.Error())
}
bytesToInt(payLoadBytes[0:8], &eF.TotalScanned)
bytesToInt(payLoadBytes[8:12], &eF.HTTPStatusCode)
errMsgLength := sr.Frame.PayloadLength - 20
eF.ErrorMsg = string(payLoadBytes[12 : errMsgLength+12])
sr.Frame.EndFrame.TotalScanned = eF.TotalScanned
sr.Frame.EndFrame.HTTPStatusCode = eF.HTTPStatusCode
sr.Frame.EndFrame.ErrorMsg = eF.ErrorMsg
err = sr.writerCheckCrc32(payLoadBytes)
return err
}
// analysisMetaEndFrameCSV is reading the MetaEndFrameCSVType data of selectObject response body
func (sr *SelectObjectResponse) analysisMetaEndFrameCSV() error {
var mCF MetaEndFrameCSV
payLoadBytes := make([]byte, sr.Frame.PayloadLength-8)
_, err := sr.readLen(payLoadBytes, time.Duration(sr.ReadTimeOut))
if err != nil {
return fmt.Errorf("read meta end csv frame error:%s", err.Error())
}
bytesToInt(payLoadBytes[0:8], &mCF.TotalScanned)
bytesToInt(payLoadBytes[8:12], &mCF.Status)
bytesToInt(payLoadBytes[12:16], &mCF.SplitsCount)
bytesToInt(payLoadBytes[16:24], &mCF.RowsCount)
bytesToInt(payLoadBytes[24:28], &mCF.ColumnsCount)
errMsgLength := sr.Frame.PayloadLength - 36
mCF.ErrorMsg = string(payLoadBytes[28 : errMsgLength+28])
sr.Frame.MetaEndFrameCSV.ErrorMsg = mCF.ErrorMsg
sr.Frame.MetaEndFrameCSV.TotalScanned = mCF.TotalScanned
sr.Frame.MetaEndFrameCSV.Status = mCF.Status
sr.Frame.MetaEndFrameCSV.SplitsCount = mCF.SplitsCount
sr.Frame.MetaEndFrameCSV.RowsCount = mCF.RowsCount
sr.Frame.MetaEndFrameCSV.ColumnsCount = mCF.ColumnsCount
err = sr.writerCheckCrc32(payLoadBytes)
return err
}
// analysisMetaEndFrameJSON is reading the MetaEndFrameJSONType data of selectObject response body
func (sr *SelectObjectResponse) analysisMetaEndFrameJSON() error {
var mJF MetaEndFrameJSON
payLoadBytes := make([]byte, sr.Frame.PayloadLength-8)
_, err := sr.readLen(payLoadBytes, time.Duration(sr.ReadTimeOut))
if err != nil {
return fmt.Errorf("read meta end json frame error:%s", err.Error())
}
bytesToInt(payLoadBytes[0:8], &mJF.TotalScanned)
bytesToInt(payLoadBytes[8:12], &mJF.Status)
bytesToInt(payLoadBytes[12:16], &mJF.SplitsCount)
bytesToInt(payLoadBytes[16:24], &mJF.RowsCount)
errMsgLength := sr.Frame.PayloadLength - 32
mJF.ErrorMsg = string(payLoadBytes[24 : errMsgLength+24])
sr.Frame.MetaEndFrameJSON.ErrorMsg = mJF.ErrorMsg
sr.Frame.MetaEndFrameJSON.TotalScanned = mJF.TotalScanned
sr.Frame.MetaEndFrameJSON.Status = mJF.Status
sr.Frame.MetaEndFrameJSON.SplitsCount = mJF.SplitsCount
sr.Frame.MetaEndFrameJSON.RowsCount = mJF.RowsCount
err = sr.writerCheckCrc32(payLoadBytes)
return err
}
func (sr *SelectObjectResponse) checkPayloadSum() (bool, error) {
payLoadChecksumByte := make([]byte, 4)
n, err := sr.readLen(payLoadChecksumByte, time.Duration(sr.ReadTimeOut))
if n == 4 {
bytesToInt(payLoadChecksumByte, &sr.Frame.PayloadChecksum)
sr.ServerCRC32 = sr.Frame.PayloadChecksum
sr.ClientCRC32 = sr.WriterForCheckCrc32.Sum32()
if sr.Frame.EnablePayloadCrc == true && sr.ServerCRC32 != 0 && sr.ServerCRC32 != sr.ClientCRC32 {
return false, fmt.Errorf("RequestId: %s, Unexpected frame type: %d, client %d but server %d",
sr.Headers.Get(HTTPHeaderOssRequestID), sr.Frame.FrameType, sr.ClientCRC32, sr.ServerCRC32)
}
return true, err
}
return false, fmt.Errorf("RequestId:%s, read checksum error:%s", sr.Headers.Get(HTTPHeaderOssRequestID), err.Error())
}
func (sr *SelectObjectResponse) writerCheckCrc32(p []byte) (err error) {
err = nil
if sr.Frame.EnablePayloadCrc == true {
_, err = sr.WriterForCheckCrc32.Write(p)
}
return err
}
// emptyFrame is emptying SelectObjectResponse Frame information
func (sr *SelectObjectResponse) emptyFrame() {
crcCalc := crc32.NewIEEE()
sr.WriterForCheckCrc32 = crcCalc
sr.Finish = false
sr.Frame.ConsumedBytesLength = 0
sr.Frame.OpenLine = false
sr.Frame.Version = byte(0)
sr.Frame.FrameType = 0
sr.Frame.PayloadLength = 0
sr.Frame.HeaderCheckSum = 0
sr.Frame.Offset = 0
sr.Frame.Data = ""
sr.Frame.EndFrame.TotalScanned = 0
sr.Frame.EndFrame.HTTPStatusCode = 0
sr.Frame.EndFrame.ErrorMsg = ""
sr.Frame.MetaEndFrameCSV.TotalScanned = 0
sr.Frame.MetaEndFrameCSV.Status = 0
sr.Frame.MetaEndFrameCSV.SplitsCount = 0
sr.Frame.MetaEndFrameCSV.RowsCount = 0
sr.Frame.MetaEndFrameCSV.ColumnsCount = 0
sr.Frame.MetaEndFrameCSV.ErrorMsg = ""
sr.Frame.MetaEndFrameJSON.TotalScanned = 0
sr.Frame.MetaEndFrameJSON.Status = 0
sr.Frame.MetaEndFrameJSON.SplitsCount = 0
sr.Frame.MetaEndFrameJSON.RowsCount = 0
sr.Frame.MetaEndFrameJSON.ErrorMsg = ""
sr.Frame.PayloadChecksum = 0
}
// bytesToInt byte's array trans to int
func bytesToInt(b []byte, ret interface{}) {
binBuf := bytes.NewBuffer(b)
binary.Read(binBuf, binary.BigEndian, ret)
}
backend/vendor/github.com/aliyun/aliyun-oss-go-sdk/oss/transport_1_6.go
Generated Vendored
+42
View File
@@ -0,0 +1,42 @@
//go:build !go1.7
// +build !go1.7
package oss
import (
"crypto/tls"
"net"
"net/http"
"time"
)
func newTransport(conn *Conn, config *Config) *http.Transport {
httpTimeOut := conn.config.HTTPTimeout
httpMaxConns := conn.config.HTTPMaxConns
// New Transport
transport := &http.Transport{
Dial: func(netw, addr string) (net.Conn, error) {
d := net.Dialer{
Timeout: httpTimeOut.ConnectTimeout,
KeepAlive: 30 * time.Second,
}
if config.LocalAddr != nil {
d.LocalAddr = config.LocalAddr
}
conn, err := d.Dial(netw, addr)
if err != nil {
return nil, err
}
return newTimeoutConn(conn, httpTimeOut.ReadWriteTimeout, httpTimeOut.LongTimeout), nil
},
MaxIdleConnsPerHost: httpMaxConns.MaxIdleConnsPerHost,
ResponseHeaderTimeout: httpTimeOut.HeaderTimeout,
}
if config.InsecureSkipVerify {
transport.TLSClientConfig = &tls.Config{
InsecureSkipVerify: true,
}
}
return transport
}

Some files were not shown because too many files have changed in this diff Show More