90 Commits

Author SHA1 Message Date
Kerwin
8cf1575232 chore: bump version to v1.2.4 2025-08-20 17:26:25 +08:00
ctwj
17c05870a3 Merge pull request #3 from ctwj/feat_search_opt
feat: 新增搜索增加
2025-08-20 17:24:25 +08:00
ctwj
d531be3c36 Merge pull request #2 from ctwj/fix_version
fix: 修复版本显示不正确的问题
2025-08-20 17:19:41 +08:00
Kerwin
edde7afdc8 fix: 修复版本显示不正确的问题 2025-08-20 17:16:34 +08:00
Kerwin
77216cf380 feat: 新增搜索增加 2025-08-20 15:03:14 +08:00
Kerwin
da3fc11b2e fix: 修复文件管理搜索不生效的问题 2025-08-19 09:09:44 +08:00
ctwj
cbf673126e update: 首页优化 2025-08-19 01:11:09 +08:00
ctwj
aa7d6ea2fe add: sql 2025-08-18 23:02:04 +08:00
Kerwin
841eb05f68 update: AppFooter 2025-08-18 20:10:09 +08:00
Kerwin
eeca85942f update: 移除api的特殊处理,使用配置项实现 2025-08-18 19:40:10 +08:00
Kerwin
c053a17131 chore: version to 1.2.3 2025-08-18 16:02:52 +08:00
Kerwin
3d29f1bf23 chore: bump version to v1.2.3 2025-08-18 15:32:29 +08:00
Kerwin
a15a0fe2be chore: bump version to v1.2.2 2025-08-18 15:08:49 +08:00
Kerwin
05243bcfe7 fix: 修复有可能配置丢失的问题 2025-08-18 13:38:52 +08:00
Kerwin
98b94b3313 update: 完善图片上传 2025-08-18 09:41:19 +08:00
ctwj
949a328ee3 update: 添加logo的配置 2025-08-18 02:30:15 +08:00
ctwj
acb462c6d5 add: xunlei 2025-08-17 23:22:57 +08:00
ctwj
e52043505f update: 修复上传问题 2025-08-17 08:46:51 +08:00
Kerwin
9d4eb38272 add: 新增文件上传功能 2025-08-15 18:41:09 +08:00
Kerwin
14ef85801a update: 移除旧版管理后台 2025-08-15 13:55:55 +08:00
Kerwin
3f4430104d update: 优化二维码显示样式 2025-08-14 18:03:20 +08:00
Kerwin
709029a123 fix: 修复二维码不显示的问题 2025-08-14 17:51:43 +08:00
Kerwin
559d69f52b fix: 搜索记录重复的问题 2025-08-14 09:46:13 +08:00
ctwj
dcd5e0bf73 update: 更新广告关键词,添加默认的开源关键词连接 2025-08-14 00:20:47 +08:00
ctwj
4343a29bb3 fix: 修复广告配置问题 2025-08-14 00:05:35 +08:00
ctwj
3bf0d59a9c update: 完善转存的广告 2025-08-13 23:30:42 +08:00
Kerwin
c3b2979977 add: 添加默认广告词 2025-08-13 17:33:34 +08:00
Kerwin
6de20b7e13 chore: bump version to 1.2.1 2025-08-13 15:28:51 +08:00
Kerwin
2d96413a5d Merge branch 'main' of github.com:ctwj/urldb 2025-08-13 15:22:32 +08:00
Kerwin
9b0d385c52 chore: bump version to v1.2.1 2025-08-13 15:22:01 +08:00
ctwj
fae7de17d5 fix: 修复了转存删除和添加广告的问题 2025-08-13 00:28:18 +08:00
Kerwin
05930a3e70 update: Readme.md 2025-08-12 09:25:28 +08:00
Kerwin
0e34cee3d8 chore: bump version to v1.2.0 2025-08-12 09:14:25 +08:00
Kerwin
b35971f43c update: README.md 2025-08-12 09:12:15 +08:00
ctwj
285b01922d update: 添加广告逻辑 2025-08-12 00:56:38 +08:00
ctwj
aa3b8585f9 update: 批量转存优化 2025-08-12 00:27:10 +08:00
Kerwin
25c7c47c96 update: task items添加过滤 2025-08-11 17:51:04 +08:00
ctwj
b567531a7d update: 后台界面优化 2025-08-11 01:34:07 +08:00
ctwj
1b0fc06bf7 update: 三方统计完成集成 2025-08-10 13:52:41 +08:00
ctwj
f5b5455989 update: 优化任务处理 2025-08-10 00:54:30 +08:00
ctwj
14f22f9128 add: 任务列表 2025-08-09 23:47:30 +08:00
ctwj
76eb9c689b add: 添加任务相关代码 2025-08-09 17:27:26 +08:00
ctwj
7032235923 update: ui更新 2025-08-09 17:26:52 +08:00
ctwj
f870779146 refactor: 重新组织组件目录 2025-08-09 11:20:48 +08:00
ctwj
81eb99691d update: 优化获取链接的方式 2025-08-09 09:51:55 +08:00
ctwj
32e7240287 refactor: 自动转存功能重构 2025-08-09 08:33:32 +08:00
Kerwin
a041a6f01d update: 优化登录页面样式 2025-08-08 17:26:48 +08:00
Kerwin
eeb9c295f5 add: 添加开启关闭注册的配置项 2025-08-08 16:51:05 +08:00
Kerwin
df86034ae5 update: 后端页面占位 2025-08-08 15:49:07 +08:00
ctwj
be66667890 update: 完善新后台 2025-08-08 01:52:57 +08:00
ctwj
667338368a update: 完善新后台 2025-08-08 01:28:25 +08:00
Kerwin
5cfd0ad3ee update: 后台改版 2025-08-07 18:47:26 +08:00
Kerwin
1cc70e439e fix: 修复文档中的错误 2025-08-06 18:47:35 +08:00
Kerwin
0e99233417 update: 修改二维码弹窗 2025-08-06 18:17:32 +08:00
Kerwin
000f92ffd1 update: 优化user页面UI 2025-08-06 10:16:35 +08:00
Kerwin
4c3c9bd553 add: 新增用户相关页面,无逻辑 2025-08-06 09:59:34 +08:00
ctwj
22db03dcea fix: scheduler 2025-08-05 22:08:13 +08:00
ctwj
26c25520fa update: docker images version 2025-08-05 21:15:54 +08:00
Kerwin
c2a8cdef4f update: 后台UI 优化 2025-08-05 18:09:14 +08:00
Kerwin
7e8f42212a add: 添加违禁词 2025-08-05 16:14:11 +08:00
Kerwin
5af4c235d5 refact: 重构定时任务 2025-08-05 11:45:18 +08:00
Kerwin
1d9451f071 update: ui 2025-08-04 16:56:36 +08:00
Kerwin
4825b45511 fix: 修复平台过滤失效的问题 2025-08-04 14:18:45 +08:00
ctwj
5bd21e156d update: 完善处理失败的资源管理 2025-08-03 22:40:22 +08:00
ctwj
689d1e61a0 update: version 1.1.0 2025-08-03 11:18:40 +08:00
ctwj
c8fd405d74 chore: bump version to v1.1.0 2025-08-03 11:15:32 +08:00
ctwj
5f8d998c65 update: 控制台体验优化 2025-08-03 10:50:25 +08:00
ctwj
b5b3c55573 update: UI优化 2025-08-03 00:19:21 +08:00
ctwj
1d3ed2f8aa add: 自动转存跳过包含违禁词的资源 2025-08-02 23:57:14 +08:00
ctwj
215f3170cd add; 添加违禁词的配置 2025-08-02 23:45:26 +08:00
ctwj
0700de36f5 update: 后台页面优化 2025-08-02 18:07:11 +08:00
Kerwin
14130eac8b update: README 2025-08-01 16:14:46 +08:00
Kerwin
bad6da4488 add: 添加自动处理失败列表 2025-08-01 15:50:04 +08:00
Kerwin
1126f84a3a update: 配置表更新 2025-08-01 14:24:45 +08:00
ctwj
24d644dc8b fix: 数据重复初始化问题 2025-07-31 23:27:23 +08:00
ctwj
d0ac53320e update: 更新默认分类 2025-07-31 22:48:26 +08:00
Kerwin
853bb50854 fix: 修复搜索接口错误 2025-07-31 17:28:10 +08:00
ctwj
dfb6a1707c update: 失效链接处理后删除 2025-07-31 00:21:51 +08:00
ctwj
9098b28ba6 update: docs 2025-07-30 23:38:27 +08:00
ctwj
b5e5052146 update: 尝试修复文档导航不显示问题 2025-07-30 23:27:42 +08:00
ctwj
e88b8411b5 update: 优化禁止访问页 2025-07-30 23:22:49 +08:00
Kerwin
d1b406b1ee update:优化 禁止访问页面 2025-07-30 17:33:36 +08:00
Kerwin
10432c1db6 opt: 优化微信禁止显示页面 2025-07-30 15:19:16 +08:00
Kerwin
440049c974 update: README 2025-07-30 11:18:43 +08:00
Kerwin
afb5a38f15 update: README 2025-07-30 11:14:54 +08:00
Kerwin
1ea7e87e6f update: README 2025-07-30 11:11:54 +08:00
ctwj
e6b4455428 fix: 修复天翼链接识别失败的问题 2025-07-30 07:56:44 +08:00
ctwj
6aacf9aed8 fix: error 2025-07-30 07:36:12 +08:00
ctwj
1f6fdfba1a fix: 修复迅雷天翼自动处理不识别问题 2025-07-30 00:01:42 +08:00
Kerwin
4d466af99e update: version to 1.0.10 2025-07-29 17:51:24 +08:00
179 changed files with 30111 additions and 6874 deletions

130
BUILD.md Normal file
View File

@@ -0,0 +1,130 @@
# 编译说明
## 方案1使用编译脚本推荐
### 在Git Bash中执行
```bash
# 给脚本添加执行权限(首次使用)
chmod +x scripts/build.sh
# 编译Linux版本推荐用于服务器部署
./scripts/build.sh
# 或者明确指定编译Linux版本
./scripts/build.sh build-linux
# 或者指定目标文件名
./scripts/build.sh build-linux myapp
# 编译当前平台版本(用于本地测试)
./scripts/build.sh build
```
### 编译脚本功能:
- 自动读取 `VERSION` 文件中的版本号
- 自动获取Git提交信息和分支信息
- 自动获取构建时间
- 将版本信息编译到可执行文件中
- 支持跨平台编译默认编译Linux版本
- 使用静态链接,适合服务器部署
## 方案2手动编译
### Linux版本推荐
```bash
# 获取版本信息
VERSION=$(cat VERSION)
GIT_COMMIT=$(git rev-parse --short HEAD 2>/dev/null || echo "unknown")
GIT_BRANCH=$(git branch --show-current 2>/dev/null || echo "unknown")
BUILD_TIME=$(date '+%Y-%m-%d %H:%M:%S')
# 编译Linux版本
CGO_ENABLED=0 GOOS=linux go build -a -installsuffix cgo -ldflags "-X 'github.com/ctwj/urldb/utils.Version=${VERSION}' -X 'github.com/ctwj/urldb/utils.BuildTime=${BUILD_TIME}' -X 'github.com/ctwj/urldb/utils.GitCommit=${GIT_COMMIT}' -X 'github.com/ctwj/urldb/utils.GitBranch=${GIT_BRANCH}'" -o main .
```
### 当前平台版本:
```bash
# 获取版本信息
VERSION=$(cat VERSION)
GIT_COMMIT=$(git rev-parse --short HEAD 2>/dev/null || echo "unknown")
GIT_BRANCH=$(git branch --show-current 2>/dev/null || echo "unknown")
BUILD_TIME=$(date '+%Y-%m-%d %H:%M:%S')
# 编译当前平台版本
go build -ldflags "-X 'github.com/ctwj/urldb/utils.Version=${VERSION}' -X 'github.com/ctwj/urldb/utils.BuildTime=${BUILD_TIME}' -X 'github.com/ctwj/urldb/utils.GitCommit=${GIT_COMMIT}' -X 'github.com/ctwj/urldb/utils.GitBranch=${GIT_BRANCH}'" -o main .
```
## 验证版本信息
编译完成后,可以通过以下方式验证版本信息:
```bash
# 命令行验证
./main version
# 启动服务器后通过API验证
curl http://localhost:8080/api/version
```
## 部署说明
使用方案1编译后部署时只需要
1. 复制可执行文件到服务器
2. 启动程序
**不再需要复制 `VERSION` 文件**,因为版本信息已经编译到程序中。
### 使用部署脚本(可选)
```bash
# 给部署脚本添加执行权限
chmod +x scripts/deploy-example.sh
# 部署到服务器
./scripts/deploy-example.sh root example.com /opt/urldb
```
### 使用Docker构建脚本
```bash
# 给脚本添加执行权限
chmod +x scripts/docker-build.sh
# 构建Docker镜像
./scripts/docker-build.sh build
# 构建指定版本镜像
./scripts/docker-build.sh build 1.2.4
# 推送镜像到Docker Hub
./scripts/docker-build.sh push 1.2.4
```
### 手动Docker构建
```bash
# 构建镜像
docker build --target backend -t ctwj/urldb-backend:1.2.3 .
docker build --target frontend -t ctwj/urldb-frontend:1.2.3 .
```
## 版本管理
更新版本号:
```bash
# 更新版本号
./scripts/version.sh patch # 修订版本
./scripts/version.sh minor # 次版本
./scripts/version.sh major # 主版本
# 然后重新编译
./scripts/build.sh
# 或者构建Docker镜像
./scripts/docker-build.sh build
```

View File

@@ -1,79 +0,0 @@
# 📝 更新日志
本项目遵循 [语义化版本](https://semver.org/lang/zh-CN/) 规范。
## [未发布]
### 新增
- 项目开源准备
- 完善文档和贡献指南
- 添加LICENSE文件
### 修复
- 修复README格式问题
- 优化项目结构说明
## [100 - 202401XX
### 新增
- 🎉 首次发布
- 📁 多平台网盘支持夸克、阿里云盘、百度网盘、UC网盘
- 🔍 智能搜索功能
- 📊 数据统计和分析
- 🏷️ 标签系统
- 👥 用户权限管理
- 📦 批量资源管理
- 🔄 自动处理功能
- 📈 热播剧管理
- ⚙️ 系统配置管理
- 🔐 JWT认证系统
- 📱 响应式设计
- 🌙 深色模式支持
- 🎨 现代化UI界面
### 技术特性
- 🦀 基于Golang 1023的高性能后端
- ⚡ Nuxt.js 3 + Vue 3前端框架
- 🗄️ PostgreSQL数据库
- 🔧 GORM ORM框架
- 🐳 Docker容器化部署
- 📝 TypeScript类型安全
### 核心功能
- 资源管理:增删改查、批量操作
- 分类管理:资源分类和标签
- 平台管理:多网盘平台支持
- 搜索统计:全文搜索和数据分析
- 系统配置:灵活的参数配置
---
## 版本说明
### 版本号格式
- **主版本号**不兼容的API修改
- **次版本号**:向下兼容的功能性新增
- **修订号**:向下兼容的问题修正
### 更新类型
- 🎉 **重大更新** - 新版本发布
-**新增功能** - 新功能或特性
- 🐛 **问题修复** - Bug修复
- 🔧 **优化改进** - 性能优化或代码改进
- 📚 **文档更新** - 文档或注释更新
- 🎨 **界面优化** - UI/UX改进
-**性能提升** - 性能相关改进
- 🔒 **安全更新** - 安全相关修复
- 🧪 **测试相关** - 测试用例或测试工具
- 🚀 **部署相关** - 部署或构建相关
---
## 贡献
如果您想为更新日志做出贡献,请:
1. 在提交代码时使用规范的提交信息2. 在Pull Request中描述您的更改
3. 遵循项目的贡献指南
---

View File

@@ -28,11 +28,26 @@ WORKDIR /app
COPY go.mod go.sum ./
RUN go mod download
# 复制VERSION文件确保构建时能正确读取版本号
COPY VERSION ./
# 复制所有源代码
COPY . .
RUN CGO_ENABLED=0 GOOS=linux go build -a -installsuffix cgo -o main .
# 定义构建参数
ARG VERSION
ARG GIT_COMMIT
ARG GIT_BRANCH
ARG BUILD_TIME
# 获取版本信息并编译
RUN VERSION=${VERSION:-$(cat VERSION)} && \
GIT_COMMIT=${GIT_COMMIT:-$(git rev-parse --short HEAD 2>/dev/null || echo "unknown")} && \
GIT_BRANCH=${GIT_BRANCH:-$(git branch --show-current 2>/dev/null || echo "unknown")} && \
BUILD_TIME=${BUILD_TIME:-$(date '+%Y-%m-%d %H:%M:%S')} && \
CGO_ENABLED=0 GOOS=linux go build -a -installsuffix cgo \
-ldflags "-X 'github.com/ctwj/urldb/utils.Version=${VERSION}' \
-X 'github.com/ctwj/urldb/utils.BuildTime=${BUILD_TIME}' \
-X 'github.com/ctwj/urldb/utils.GitCommit=${GIT_COMMIT}' \
-X 'github.com/ctwj/urldb/utils.GitBranch=${GIT_BRANCH}'" \
-o main .
# 后端运行阶段
FROM alpine:latest AS backend

179
README.md
View File

@@ -10,7 +10,7 @@
**一个现代化的网盘资源数据库,支持多网盘自动化转存分享,支持百度网盘,阿里云盘,夸克网盘, 天翼云盘迅雷云盘123云盘115网盘UC网盘 **
🌐 [在线演示](https://pan.l9.lc) | 📖 [文档](http://doc.l9.lc/) | 🐛 [问题反馈](https://github.com/ctwj/urldb/issues) | ⭐ [给个星标](https://github.com/ctwj/urldb)
🌐 [在线演示](https://pan.l9.lc) | 📖 [文档](https://ecn5khs4t956.feishu.cn/wiki/PsnDwtxghiP0mLkTiruczKtxnwd?from=from_copylink) | 🐛 [问题反馈](https://github.com/ctwj/urldb/issues) | ⭐ [给个星标](https://github.com/ctwj/urldb)
### 支持的网盘平台
@@ -29,15 +29,58 @@
---
## 🔔 温馨提示
## 🔔 版本改动
📌 **本项目仅供技术交流与学习使用**,自身不存储或提供任何资源文件及下载链接。
- [文档说明](https://ecn5khs4t956.feishu.cn/wiki/PsnDwtxghiP0mLkTiruczKtxnwd?from=from_copylink)
- [服务器要求](https://ecn5khs4t956.feishu.cn/wiki/W8YBww1Mmiu4Cdkp5W4c8pFNnMf?from=from_copylink)
- [QQ机器人](https://github.com/ctwj/astrbot_plugin_urldb)
📌 **请勿将本项目用于任何违法用途**,否则后果自负。
### v1.2.3
1. 添加图片上传功能
2. 添加Logo配置项首页Logo显示
3. 后台界面体验优化
📌 如有任何问题或建议,欢迎交流探讨! 😊
### v1.2.1
1. 修复转存移除广告失败的问题和添加广告失败的问题
2. 管理后台UI优化
3. 首页添加描述显示
> **免责声明**:本项目由 Trae AI 辅助编写。由于时间有限,仅在空闲时维护。如遇使用问题,请优先自行排查,感谢理解!
### v1.2.0
1. 新增手动批量转存
2. 新增QQ机器人
3. 新增任务管理功能
4. 自动转存改版(批量转存修改为,显示二维码时自动转存)
5. 新增支持第三方统计代码配置
### v1.0.0
1. 支持API手动批量录入资源
2. 支持,自动判断资源有效性
3. 支持自动转存
4. 支持平台多账号管理Quark
5. 支持简单的数据统计
---
## 📸 项目截图
### 🏠 首页
![首页](https://raw.githubusercontent.com/ctwj/urldb/refs/heads/main/github/index.webp)
### 🔧 后台管理
![后台管理](https://raw.githubusercontent.com/ctwj/urldb/refs/heads/main/github/admin.webp)
### ⚙️ 系统配置
![系统配置](https://raw.githubusercontent.com/ctwj/urldb/refs/heads/main/github/config.webp)
### 🔍 批量转存
![资源搜索](https://raw.githubusercontent.com/ctwj/urldb/refs/heads/main/github/save.webp)
### 👤 多账号管理
![账号管理](https://raw.githubusercontent.com/ctwj/urldb/refs/heads/main/github/account.webp)
---
@@ -68,111 +111,6 @@
---
## 🚀 快速开始
### 环境要求
- **Docker** 和 **Docker Compose**
- 或者本地环境:
- **Go** 1.23+
- **Node.js** 18+
- **PostgreSQL** 15+
- **pnpm** (推荐) 或 npm
### 方式一Docker 部署(推荐)
```bash
# 克隆项目
git clone https://github.com/ctwj/urldb.git
cd urldb
# 使用 Docker Compose 启动
docker compose up --build -d
# 访问应用
# 前端: http://localhost:3030
# 后端API: http://localhost:8080
```
### 方式二:本地开发
#### 1. 克隆项目
```bash
git clone https://github.com/ctwj/urldb.git
cd urldb
```
#### 2. 后端设置
```bash
# 复制环境变量文件
cp env.example .env
# 编辑环境变量
vim .env
# 安装Go依赖
go mod tidy
# 启动后端服务
go run main.go
```
#### 3. 前端设置
```bash
# 进入前端目录
cd web
# 安装依赖
pnpm install
# 启动开发服务器
pnpm dev
```
#### 4. 数据库设置
```sql
-- 创建数据库
CREATE DATABASE url_db;
```
---
## 📁 项目结构
```
l9pan/
├── 📁 common/ # 通用功能模块
│ ├── 📄 pan_factory.go # 网盘工厂模式
│ ├── 📄 alipan.go # 阿里云盘实现
│ ├── 📄 baidu_pan.go # 百度网盘实现
│ ├── 📄 quark_pan.go # 夸克网盘实现
│ └── 📄 uc_pan.go # UC网盘实现
├── 📁 db/ # 数据库层
│ ├── 📁 entity/ # 数据实体
│ ├── 📁 repo/ # 数据仓库
│ ├── 📁 dto/ # 数据传输对象
│ └── 📁 converter/ # 数据转换器
├── 📁 handlers/ # API处理器
├── 📁 middleware/ # 中间件
├── 📁 utils/ # 工具函数
├── 📁 web/ # 前端项目
│ ├── 📁 pages/ # 页面组件
│ ├── 📁 components/ # 通用组件
│ ├── 📁 composables/ # 组合式函数
│ └── 📁 stores/ # 状态管理
├── 📁 docs/ # 项目文档
├── 📁 nginx/ # Nginx配置
│ ├── 📄 nginx.conf # 主配置文件
│ └── 📁 conf.d/ # 站点配置
├── 📄 main.go # 主程序入口
├── 📄 Dockerfile # Docker配置
├── 📄 docker-compose.yml # Docker Compose配置
├── 📄 docker-start-nginx.sh # Nginx启动脚本
└── 📄 README.md # 项目说明
```
---
## 🔧 配置说明
### 环境变量配置
@@ -192,13 +130,6 @@ PORT=8080
TIMEZONE=Asia/Shanghai
```
### Docker 服务说明
| 服务 | 端口 | 说明 |
|------|------|------|
| server | 3030 | 应用 |
| postgres | 5431 | PostgreSQL 数据库 |
### 镜像构建
```
@@ -210,18 +141,6 @@ docker push ctwj/urldb-backend:1.0.7
---
## 📚 API 文档
### 公开统计
提供批量入库和搜索api通过 apiToken 授权
> 📖 完整API文档请访问`http://doc.l9.lc/`
## 🤝 贡献指南
我们欢迎所有形式的贡献!
## 📄 许可证
本项目采用 [GPL License](LICENSE) 许可证。

View File

@@ -1 +1 @@
1.0.10
1.2.4

View File

@@ -7,6 +7,8 @@ import (
"path/filepath"
"sync"
"time"
"github.com/ctwj/urldb/utils"
)
// AlipanService 阿里云盘服务
@@ -428,7 +430,7 @@ func (a *AlipanService) manageAccessToken() (string, error) {
}
// 检查token是否过期
if time.Now().After(tokenInfo.ExpiresAt) {
if utils.GetCurrentTime().After(tokenInfo.ExpiresAt) {
return a.getNewAccessToken()
}

View File

@@ -17,6 +17,9 @@ const (
UC
NotFound
Xunlei
Tianyi
Pan123
Pan115
)
// String 返回服务类型的字符串表示
@@ -32,6 +35,12 @@ func (s ServiceType) String() string {
return "uc"
case Xunlei:
return "xunlei"
case Tianyi:
return "tianyi"
case Pan123:
return "123pan"
case Pan115:
return "115"
default:
return "unknown"
}
@@ -120,6 +129,8 @@ func (f *PanFactory) CreatePanService(url string, config *PanConfig) (PanService
return NewBaiduPanService(config), nil
case UC:
return NewUCService(config), nil
case Xunlei:
return NewXunleiPanService(config), nil
default:
return nil, fmt.Errorf("不支持的服务类型: %s", url)
}
@@ -136,6 +147,10 @@ func (f *PanFactory) CreatePanServiceByType(serviceType ServiceType, config *Pan
return NewBaiduPanService(config), nil
case UC:
return NewUCService(config), nil
case Xunlei:
return NewXunleiPanService(config), nil
// case Tianyi:
// return NewTianyiService(config), nil
default:
return nil, fmt.Errorf("不支持的服务类型: %d", serviceType)
}
@@ -165,10 +180,21 @@ func (f *PanFactory) GetUCService(config *PanConfig) PanService {
return service
}
// GetXunleiService 获取迅雷网盘服务单例
func (f *PanFactory) GetXunleiService(config *PanConfig) PanService {
service := NewXunleiPanService(config)
return service
}
// ExtractServiceType 从URL中提取服务类型
func ExtractServiceType(url string) ServiceType {
url = strings.ToLower(url)
// "https://www.123pan.com/s/i4uaTd-WHn0", // 公开分享
// "https://www.123912.com/s/U8f2Td-ZeOX",
// "https://www.123684.coms/u9izjv-k3uWv",
// "https://www.123pan.com/s/A6cA-AKH11", // 外链不存在
patterns := map[string]ServiceType{
"pan.quark.cn": Quark,
"www.alipan.com": Alipan,
@@ -177,6 +203,13 @@ func ExtractServiceType(url string) ServiceType {
"drive.uc.cn": UC,
"fast.uc.cn": UC,
"pan.xunlei.com": Xunlei,
"cloud.189.cn": Tianyi,
"www.123pan.com": Pan123,
"www.123912.com": Pan123,
"www.123684.com": Pan123,
"115cdn.com": Pan115,
"anxia.com": Pan115,
"115.com/": Pan115,
}
for pattern, serviceType := range patterns {
@@ -196,12 +229,24 @@ func ExtractShareId(url string) (string, ServiceType) {
}
// 提取分享ID
substring := strings.Index(url, "/s/")
shareID := ""
substring := -1
if index := strings.Index(url, "/s/"); index != -1 {
substring = index + 3
} else if index := strings.Index(url, "/t/"); index != -1 {
substring = index + 3
} else if index := strings.Index(url, "/web/share?code="); index != -1 {
substring = index + 16
} else if index := strings.Index(url, "/p/"); index != -1 {
substring = index + 3
}
if substring == -1 {
return "", NotFound
}
shareID := url[substring+3:] // 去除 '/s/' 部分
shareID = url[substring:]
// 去除可能的锚点
if hashIndex := strings.Index(shareID, "#"); hashIndex != -1 {

View File

@@ -4,10 +4,18 @@ import (
"encoding/json"
"fmt"
"log"
"math/rand"
"regexp"
"strconv"
"strings"
"sync"
"time"
commonutils "github.com/ctwj/urldb/common/utils"
"github.com/ctwj/urldb/db"
"github.com/ctwj/urldb/db/entity"
"github.com/ctwj/urldb/db/repo"
"github.com/ctwj/urldb/utils"
)
// QuarkPanService 夸克网盘服务
@@ -16,10 +24,15 @@ type QuarkPanService struct {
configMutex sync.RWMutex // 保护配置的读写锁
}
// 全局配置缓存刷新信号
var configRefreshChan = make(chan bool, 1)
// 单例相关变量
var (
quarkInstance *QuarkPanService
quarkOnce sync.Once
quarkInstance *QuarkPanService
quarkOnce sync.Once
systemConfigRepo repo.SystemConfigRepository
systemConfigOnce sync.Once
)
// NewQuarkPanService 创建夸克网盘服务(单例模式)
@@ -187,6 +200,11 @@ func (q *QuarkPanService) Transfer(shareID string) (*TransferResult, error) {
log.Printf("删除广告文件失败: %v", err)
}
// 添加个人自定义广告
if err := q.addAd(myData.SaveAs.SaveAsTopFids[0]); err != nil {
log.Printf("添加广告文件失败: %v", err)
}
// 分享资源
shareBtnResult, err := q.getShareBtn(myData.SaveAs.SaveAsTopFids, title)
if err != nil {
@@ -273,8 +291,26 @@ func (q *QuarkPanService) DeleteFiles(fileList []string) (*TransferResult, error
return ErrorResult("文件列表为空"), nil
}
// 逐个删除文件,确保每个删除操作都完成
for _, fileID := range fileList {
err := q.deleteSingleFile(fileID)
if err != nil {
log.Printf("删除文件 %s 失败: %v", fileID, err)
return ErrorResult(fmt.Sprintf("删除文件 %s 失败: %v", fileID, err)), nil
}
}
return SuccessResult("删除成功", nil), nil
}
// deleteSingleFile 删除单个文件
func (q *QuarkPanService) deleteSingleFile(fileID string) error {
log.Printf("正在删除文件: %s", fileID)
data := map[string]interface{}{
"fid_list": fileList,
"action_type": 2,
"filelist": []string{fileID},
"exclude_fids": []string{},
}
queryParams := map[string]string{
@@ -283,12 +319,41 @@ func (q *QuarkPanService) DeleteFiles(fileList []string) (*TransferResult, error
"uc_param_str": "",
}
_, err := q.HTTPPost("https://drive-pc.quark.cn/1/clouddrive/file/delete", data, queryParams)
respData, err := q.HTTPPost("https://drive-pc.quark.cn/1/clouddrive/file/delete", data, queryParams)
if err != nil {
return ErrorResult(fmt.Sprintf("删除文件失败: %v", err)), nil
return fmt.Errorf("删除文件请求失败: %v", err)
}
return SuccessResult("删除成功", nil), nil
// 解析响应
var response struct {
Status int `json:"status"`
Message string `json:"message"`
Data struct {
TaskID string `json:"task_id"`
} `json:"data"`
}
if err := json.Unmarshal(respData, &response); err != nil {
return fmt.Errorf("解析删除响应失败: %v", err)
}
if response.Status != 200 {
return fmt.Errorf("删除文件失败: %s", response.Message)
}
// 如果有任务ID等待任务完成
if response.Data.TaskID != "" {
log.Printf("删除文件任务ID: %s", response.Data.TaskID)
_, err := q.waitForTask(response.Data.TaskID)
if err != nil {
return fmt.Errorf("等待删除任务完成失败: %v", err)
}
log.Printf("文件 %s 删除完成", fileID)
} else {
log.Printf("文件 %s 删除完成无任务ID", fileID)
}
return nil
}
// getStoken 获取stoken
@@ -368,12 +433,17 @@ func (q *QuarkPanService) getShare(shareID, stoken string) (*ShareResult, error)
// getShareSave 转存分享
func (q *QuarkPanService) getShareSave(shareID, stoken string, fidList, fidTokenList []string) (*SaveResult, error) {
return q.getShareSaveToDir(shareID, stoken, fidList, fidTokenList, "0")
}
// getShareSaveToDir 转存分享到指定目录
func (q *QuarkPanService) getShareSaveToDir(shareID, stoken string, fidList, fidTokenList []string, toPdirFid string) (*SaveResult, error) {
data := map[string]interface{}{
"pwd_id": shareID,
"stoken": stoken,
"fid_list": fidList,
"fid_token_list": fidTokenList,
"to_pdir_fid": "0", // 默认存储到目录
"to_pdir_fid": toPdirFid, // 存储到指定目录
}
queryParams := map[string]string{
@@ -406,7 +476,7 @@ func (q *QuarkPanService) getShareSave(shareID, stoken string, fidList, fidToken
// 生成指定长度的时间戳
func (q *QuarkPanService) generateTimestamp(length int) int64 {
timestamp := time.Now().UnixNano() / int64(time.Millisecond)
timestamp := utils.GetCurrentTime().UnixNano() / int64(time.Millisecond)
timestampStr := strconv.FormatInt(timestamp, 10)
if len(timestampStr) > length {
timestampStr = timestampStr[:length]
@@ -546,8 +616,249 @@ func (q *QuarkPanService) waitForTask(taskID string) (*TaskResult, error) {
// deleteAdFiles 删除广告文件
func (q *QuarkPanService) deleteAdFiles(pdirFid string) error {
// 这里可以添加广告文件删除逻辑
// 需要从配置中读取禁止的关键词列表
log.Printf("开始删除广告文件目录ID: %s", pdirFid)
// 获取目录文件列表
fileList, err := q.getDirFile(pdirFid)
if err != nil {
log.Printf("获取目录文件失败: %v", err)
return err
}
if fileList == nil || len(fileList) == 0 {
log.Printf("目录为空,无需删除广告文件")
return nil
}
// 删除包含广告关键词的文件
for _, file := range fileList {
if fileName, ok := file["file_name"].(string); ok {
log.Printf("检查文件: %s", fileName)
if q.containsAdKeywords(fileName) {
if fid, ok := file["fid"].(string); ok {
log.Printf("删除广告文件: %s (FID: %s)", fileName, fid)
_, err := q.DeleteFiles([]string{fid})
if err != nil {
log.Printf("删除广告文件失败: %v", err)
} else {
log.Printf("成功删除广告文件: %s", fileName)
}
}
}
}
}
return nil
}
// containsAdKeywords 检查文件名是否包含广告关键词
func (q *QuarkPanService) containsAdKeywords(filename string) bool {
// 从系统配置中获取广告关键词
adKeywordsStr, err := q.getSystemConfigValue(entity.ConfigKeyAdKeywords)
if err != nil {
log.Printf("获取广告关键词配置失败: %v", err)
return false
}
// 如果配置为空返回false
if adKeywordsStr == "" {
return false
}
// 按逗号分割关键词(支持中文和英文逗号)
adKeywords := q.splitKeywords(adKeywordsStr)
return q.checkKeywordsInFilename(filename, adKeywords)
}
// checkKeywordsInFilename 检查文件名是否包含指定关键词
func (q *QuarkPanService) checkKeywordsInFilename(filename string, keywords []string) bool {
// 转为小写进行比较
lowercaseFilename := strings.ToLower(filename)
for _, keyword := range keywords {
if strings.Contains(lowercaseFilename, strings.ToLower(keyword)) {
log.Printf("文件 %s 包含广告关键词: %s", filename, keyword)
return true
}
}
return false
}
// getSystemConfigValue 获取系统配置值
func (q *QuarkPanService) getSystemConfigValue(key string) (string, error) {
// 检查是否需要刷新缓存
select {
case <-configRefreshChan:
// 收到刷新信号,清空缓存
systemConfigOnce.Do(func() {
systemConfigRepo = repo.NewSystemConfigRepository(db.DB)
})
systemConfigRepo.ClearConfigCache()
default:
// 没有刷新信号,继续使用缓存
}
// 使用单例模式获取系统配置仓库
systemConfigOnce.Do(func() {
systemConfigRepo = repo.NewSystemConfigRepository(db.DB)
})
return systemConfigRepo.GetConfigValue(key)
}
// refreshSystemConfigCache 刷新系统配置缓存
func (q *QuarkPanService) refreshSystemConfigCache() {
systemConfigOnce.Do(func() {
systemConfigRepo = repo.NewSystemConfigRepository(db.DB)
})
systemConfigRepo.ClearConfigCache()
}
// RefreshSystemConfigCache 全局刷新系统配置缓存(供外部调用)
func RefreshSystemConfigCache() {
select {
case configRefreshChan <- true:
// 发送刷新信号
default:
// 通道已满,忽略
}
}
// splitKeywords 按逗号分割关键词(支持中文和英文逗号)
func (q *QuarkPanService) splitKeywords(keywordsStr string) []string {
if keywordsStr == "" {
return []string{}
}
// 使用正则表达式同时匹配中英文逗号
re := regexp.MustCompile(`[,]`)
parts := re.Split(keywordsStr, -1)
var result []string
for _, part := range parts {
// 去除首尾空格
trimmed := strings.TrimSpace(part)
if trimmed != "" {
result = append(result, trimmed)
}
}
return result
}
// splitAdURLs 按换行符分割广告URL列表
func (q *QuarkPanService) splitAdURLs(autoInsertAdStr string) []string {
if autoInsertAdStr == "" {
return []string{}
}
// 按换行符分割
lines := strings.Split(autoInsertAdStr, "\n")
var result []string
for _, line := range lines {
// 去除首尾空格
trimmed := strings.TrimSpace(line)
if trimmed != "" {
result = append(result, trimmed)
}
}
return result
}
// extractAdFileIDs 从广告URL列表中提取文件ID
func (q *QuarkPanService) extractAdFileIDs(adURLs []string) []string {
var result []string
for _, url := range adURLs {
// 使用 ExtractShareIdString 提取分享ID
shareID, _ := commonutils.ExtractShareIdString(url)
if shareID != "" {
result = append(result, shareID)
}
}
return result
}
// addAd 添加个人自定义广告
func (q *QuarkPanService) addAd(dirID string) error {
log.Printf("开始添加个人自定义广告到目录: %s", dirID)
// 从系统配置中获取自动插入广告内容
autoInsertAdStr, err := q.getSystemConfigValue(entity.ConfigKeyAutoInsertAd)
if err != nil {
log.Printf("获取自动插入广告配置失败: %v", err)
return err
}
// 如果配置为空,跳过广告插入
if autoInsertAdStr == "" {
log.Printf("没有配置自动插入广告,跳过广告插入")
return nil
}
// 按换行符分割广告URL列表
adURLs := q.splitAdURLs(autoInsertAdStr)
if len(adURLs) == 0 {
log.Printf("没有有效的广告URL跳过广告插入")
return nil
}
// 提取广告文件ID列表
adFileIDs := q.extractAdFileIDs(adURLs)
if len(adFileIDs) == 0 {
log.Printf("没有有效的广告文件ID跳过广告插入")
return nil
}
// 随机选择一个广告文件
rand.Seed(utils.GetCurrentTimestampNano())
selectedAdID := adFileIDs[rand.Intn(len(adFileIDs))]
log.Printf("选择广告文件ID: %s", selectedAdID)
// 获取广告文件的stoken
stokenResult, err := q.getStoken(selectedAdID)
if err != nil {
log.Printf("获取广告文件stoken失败: %v", err)
return err
}
// 获取广告文件详情
adDetail, err := q.getShare(selectedAdID, stokenResult.Stoken)
if err != nil {
log.Printf("获取广告文件详情失败: %v", err)
return err
}
if len(adDetail.List) == 0 {
log.Printf("广告文件详情为空")
return fmt.Errorf("广告文件详情为空")
}
// 获取第一个广告文件的信息
adFile := adDetail.List[0]
fid := adFile.Fid
shareFidToken := adFile.ShareFidToken
// 保存广告文件到目标目录
saveResult, err := q.getShareSaveToDir(selectedAdID, stokenResult.Stoken, []string{fid}, []string{shareFidToken}, dirID)
if err != nil {
log.Printf("保存广告文件失败: %v", err)
return err
}
// 等待保存完成
_, err = q.waitForTask(saveResult.TaskID)
if err != nil {
log.Printf("等待广告文件保存完成失败: %v", err)
return err
}
log.Printf("广告文件添加成功")
return nil
}
@@ -590,26 +901,8 @@ func (q *QuarkPanService) getDirFile(pdirFid string) ([]map[string]interface{},
return nil, fmt.Errorf(response.Message)
}
// 递归处理子目录
var allFiles []map[string]interface{}
for _, item := range response.Data.List {
// 添加当前文件/目录
allFiles = append(allFiles, item)
// 如果是目录,递归获取子目录内容
if fileType, ok := item["file_type"].(float64); ok && fileType == 1 { // 1表示目录
if fid, ok := item["fid"].(string); ok {
subFiles, err := q.getDirFile(fid)
if err != nil {
log.Printf("获取子目录 %s 失败: %v", fid, err)
continue
}
allFiles = append(allFiles, subFiles...)
}
}
}
return allFiles, nil
// 直接返回文件列表,不递归处理子目录(与参考代码保持一致)
return response.Data.List, nil
}
// 定义各种结果结构体

544
common/xunlei_pan.go Normal file
View File

@@ -0,0 +1,544 @@
// 1. 修正接口 Host增加配置项
// 2. POST/GET 区分xunleix 的 /drive/v1/share/list 是 GET不是 POST
// 3. 参数传递方式严格区分 query/body
// 4. header 应支持 AuthorizationBearer ...、x-device-id、x-client-id、x-captcha-token 等
// 5. 结构体返回字段需和 xunleix 100%一致(如 data 字段是 map 还是 list注意 code 字段为 int 还是 string
// 6. 错误处理,返回体未必有 code/msg需先判断 HTTP 状态码再判断 body
// 7. 建议增加日志和更清晰的错误提示
package pan
import (
"bytes"
"encoding/json"
"fmt"
"io/ioutil"
"log"
"net/http"
"net/url"
"strings"
"sync"
"time"
)
type XunleiPanService struct {
*BasePanService
configMutex sync.RWMutex
}
var (
xunleiInstance *XunleiPanService
xunleiOnce sync.Once
)
// 配置化 API Host
func (x *XunleiPanService) apiHost() string {
return "https://api-pan.xunlei.com"
}
// 工具:自动补全必要 header
func (x *XunleiPanService) setCommonHeader(req *http.Request) {
for k, v := range x.headers {
req.Header.Set(k, v)
}
}
func NewXunleiPanService(config *PanConfig) *XunleiPanService {
xunleiOnce.Do(func() {
xunleiInstance = &XunleiPanService{
BasePanService: NewBasePanService(config),
}
xunleiInstance.SetHeaders(map[string]string{
"Content-Type": "application/json",
"User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36",
"Cookie": config.Cookie,
})
})
xunleiInstance.UpdateConfig(config)
return xunleiInstance
}
// GetXunleiInstance 获取迅雷网盘服务单例实例
func GetXunleiInstance() *XunleiPanService {
return NewXunleiPanService(nil)
}
func (x *XunleiPanService) UpdateConfig(config *PanConfig) {
if config == nil {
return
}
x.configMutex.Lock()
defer x.configMutex.Unlock()
x.config = config
if config.Cookie != "" {
x.SetHeader("Cookie", config.Cookie)
}
}
// GetServiceType 获取服务类型
func (x *XunleiPanService) GetServiceType() ServiceType {
return Xunlei
}
// Transfer 转存分享链接 - 实现 PanService 接口
func (x *XunleiPanService) Transfer(shareID string) (*TransferResult, error) {
// 读取配置(线程安全)
x.configMutex.RLock()
config := x.config
x.configMutex.RUnlock()
log.Printf("开始处理迅雷分享: %s", shareID)
// 检查是否为检验模式
if config.IsType == 1 {
// 检验模式:直接获取分享信息
shareInfo, err := x.getShareInfo(shareID)
if err != nil {
return ErrorResult(fmt.Sprintf("获取分享信息失败: %v", err)), nil
}
return SuccessResult("检验成功", map[string]interface{}{
"title": shareInfo.Title,
"shareUrl": config.URL,
}), nil
}
// 转存模式:实现完整的转存流程
// 1. 获取分享详情
shareDetail, err := x.GetShareFolder(shareID, "", "")
if err != nil {
return ErrorResult(fmt.Sprintf("获取分享详情失败: %v", err)), nil
}
// 2. 提取文件ID列表
fileIDs := make([]string, 0)
for _, file := range shareDetail.Data.Files {
fileIDs = append(fileIDs, file.FileID)
}
if len(fileIDs) == 0 {
return ErrorResult("分享中没有可转存的文件"), nil
}
// 3. 转存文件
restoreResult, err := x.Restore(shareID, "", fileIDs)
if err != nil {
return ErrorResult(fmt.Sprintf("转存失败: %v", err)), nil
}
// 4. 等待转存完成
taskID := restoreResult.Data.TaskID
_, err = x.waitForTask(taskID)
if err != nil {
return ErrorResult(fmt.Sprintf("等待转存完成失败: %v", err)), nil
}
// 5. 创建新的分享
shareResult, err := x.FileBatchShare(fileIDs, false, 0) // 永久分享
if err != nil {
return ErrorResult(fmt.Sprintf("创建分享失败: %v", err)), nil
}
// 6. 返回结果
return SuccessResult("转存成功", map[string]interface{}{
"shareUrl": shareResult.Data.ShareURL,
"title": fmt.Sprintf("迅雷分享_%s", shareID),
"fid": strings.Join(fileIDs, ","),
}), nil
}
// waitForTask 等待任务完成
func (x *XunleiPanService) waitForTask(taskID string) (*XLTaskResult, error) {
maxRetries := 50
retryDelay := 2 * time.Second
for retryIndex := 0; retryIndex < maxRetries; retryIndex++ {
result, err := x.getTaskStatus(taskID, retryIndex)
if err != nil {
return nil, err
}
if result.Status == 2 { // 任务完成
return result, nil
}
time.Sleep(retryDelay)
}
return nil, fmt.Errorf("任务超时")
}
// getTaskStatus 获取任务状态
func (x *XunleiPanService) getTaskStatus(taskID string, retryIndex int) (*XLTaskResult, error) {
apiURL := x.apiHost() + "/drive/v1/task"
params := url.Values{}
params.Set("task_id", taskID)
params.Set("retry_index", fmt.Sprintf("%d", retryIndex))
apiURL = apiURL + "?" + params.Encode()
req, err := http.NewRequest("GET", apiURL, nil)
if err != nil {
return nil, err
}
x.setCommonHeader(req)
client := &http.Client{Timeout: 15 * time.Second}
resp, err := client.Do(req)
if err != nil {
return nil, err
}
defer resp.Body.Close()
result, _ := ioutil.ReadAll(resp.Body)
if resp.StatusCode != 200 {
return nil, fmt.Errorf("HTTP %d: %s", resp.StatusCode, string(result))
}
var data XLTaskResult
if err := json.Unmarshal(result, &data); err != nil {
return nil, err
}
return &data, nil
}
// getShareInfo 获取分享信息(用于检验模式)
func (x *XunleiPanService) getShareInfo(shareID string) (*XLShareInfo, error) {
// 使用现有的 GetShareFolder 方法获取分享信息
shareDetail, err := x.GetShareFolder(shareID, "", "")
if err != nil {
return nil, err
}
// 构造分享信息
shareInfo := &XLShareInfo{
ShareID: shareID,
Title: fmt.Sprintf("迅雷分享_%s", shareID),
Files: make([]XLFileInfo, 0),
}
// 处理文件信息
for _, file := range shareDetail.Data.Files {
shareInfo.Files = append(shareInfo.Files, XLFileInfo{
FileID: file.FileID,
Name: file.Name,
})
}
return shareInfo, nil
}
// GetFiles 获取文件列表 - 实现 PanService 接口
func (x *XunleiPanService) GetFiles(pdirFid string) (*TransferResult, error) {
log.Printf("开始获取迅雷网盘文件列表目录ID: %s", pdirFid)
// 使用现有的 GetShareList 方法获取文件列表
shareList, err := x.GetShareList("")
if err != nil {
return ErrorResult(fmt.Sprintf("获取文件列表失败: %v", err)), nil
}
// 转换为通用格式
fileList := make([]interface{}, 0)
for _, share := range shareList.Data.List {
fileList = append(fileList, map[string]interface{}{
"share_id": share.ShareID,
"title": share.Title,
})
}
return SuccessResult("获取成功", fileList), nil
}
// DeleteFiles 删除文件 - 实现 PanService 接口
func (x *XunleiPanService) DeleteFiles(fileList []string) (*TransferResult, error) {
log.Printf("开始删除迅雷网盘文件,文件数量: %d", len(fileList))
// 使用现有的 ShareBatchDelete 方法删除分享
result, err := x.ShareBatchDelete(fileList)
if err != nil {
return ErrorResult(fmt.Sprintf("删除文件失败: %v", err)), nil
}
if result.Code != 0 {
return ErrorResult(fmt.Sprintf("删除文件失败: %s", result.Msg)), nil
}
return SuccessResult("删除成功", nil), nil
}
// GetUserInfo 获取用户信息 - 实现 PanService 接口
func (x *XunleiPanService) GetUserInfo(cookie string) (*UserInfo, error) {
log.Printf("开始获取迅雷网盘用户信息")
// 临时设置cookie
originalCookie := x.GetHeader("Cookie")
x.SetHeader("Cookie", cookie)
defer x.SetHeader("Cookie", originalCookie) // 恢复原始cookie
// 获取用户信息
apiURL := x.apiHost() + "/drive/v1/user/info"
req, err := http.NewRequest("GET", apiURL, nil)
if err != nil {
return nil, fmt.Errorf("创建请求失败: %v", err)
}
x.setCommonHeader(req)
client := &http.Client{Timeout: 15 * time.Second}
resp, err := client.Do(req)
if err != nil {
return nil, fmt.Errorf("获取用户信息失败: %v", err)
}
defer resp.Body.Close()
result, _ := ioutil.ReadAll(resp.Body)
if resp.StatusCode != 200 {
return nil, fmt.Errorf("HTTP %d: %s", resp.StatusCode, string(result))
}
var response struct {
Code int `json:"code"`
Msg string `json:"msg"`
Data struct {
Username string `json:"username"`
VIPStatus bool `json:"vip_status"`
UsedSpace int64 `json:"used_space"`
TotalSpace int64 `json:"total_space"`
} `json:"data"`
}
if err := json.Unmarshal(result, &response); err != nil {
return nil, fmt.Errorf("解析用户信息失败: %v", err)
}
if response.Code != 0 {
return nil, fmt.Errorf("获取用户信息失败: %s", response.Msg)
}
return &UserInfo{
Username: response.Data.Username,
VIPStatus: response.Data.VIPStatus,
UsedSpace: response.Data.UsedSpace,
TotalSpace: response.Data.TotalSpace,
ServiceType: "xunlei",
}, nil
}
// GetShareList 严格对齐 GET + queryxunleix实现
func (x *XunleiPanService) GetShareList(pageToken string) (*XLShareListResp, error) {
api := x.apiHost() + "/drive/v1/share/list"
params := url.Values{}
params.Set("limit", "100")
params.Set("thumbnail_size", "SIZE_SMALL")
if pageToken != "" {
params.Set("page_token", pageToken)
}
apiURL := api + "?" + params.Encode()
req, err := http.NewRequest("GET", apiURL, nil)
if err != nil {
return nil, err
}
x.setCommonHeader(req)
client := &http.Client{Timeout: 15 * time.Second}
resp, err := client.Do(req)
if err != nil {
return nil, err
}
defer resp.Body.Close()
result, _ := ioutil.ReadAll(resp.Body)
if resp.StatusCode != 200 {
return nil, fmt.Errorf("HTTP %d: %s", resp.StatusCode, string(result))
}
var data XLShareListResp
if err := json.Unmarshal(result, &data); err != nil {
return nil, err
}
return &data, nil
}
// FileBatchShare 创建分享POST, body
func (x *XunleiPanService) FileBatchShare(ids []string, needPassword bool, expirationDays int) (*XLBatchShareResp, error) {
apiURL := x.apiHost() + "/drive/v1/share/batch"
body := map[string]interface{}{
"file_ids": ids,
"need_password": needPassword,
"expiration_days": expirationDays,
}
bs, _ := json.Marshal(body)
req, err := http.NewRequest("POST", apiURL, bytes.NewReader(bs))
if err != nil {
return nil, err
}
x.setCommonHeader(req)
client := &http.Client{Timeout: 15 * time.Second}
resp, err := client.Do(req)
if err != nil {
return nil, err
}
defer resp.Body.Close()
result, _ := ioutil.ReadAll(resp.Body)
if resp.StatusCode != 200 {
return nil, fmt.Errorf("HTTP %d: %s", resp.StatusCode, string(result))
}
var data XLBatchShareResp
if err := json.Unmarshal(result, &data); err != nil {
return nil, err
}
return &data, nil
}
// ShareBatchDelete 取消分享POST, body
func (x *XunleiPanService) ShareBatchDelete(ids []string) (*XLCommonResp, error) {
apiURL := x.apiHost() + "/drive/v1/share/batch/delete"
body := map[string]interface{}{
"share_ids": ids,
}
bs, _ := json.Marshal(body)
req, err := http.NewRequest("POST", apiURL, bytes.NewReader(bs))
if err != nil {
return nil, err
}
x.setCommonHeader(req)
client := &http.Client{Timeout: 15 * time.Second}
resp, err := client.Do(req)
if err != nil {
return nil, err
}
defer resp.Body.Close()
result, _ := ioutil.ReadAll(resp.Body)
if resp.StatusCode != 200 {
return nil, fmt.Errorf("HTTP %d: %s", resp.StatusCode, string(result))
}
var data XLCommonResp
if err := json.Unmarshal(result, &data); err != nil {
return nil, err
}
return &data, nil
}
// GetShareFolder 获取分享内容POST, body
func (x *XunleiPanService) GetShareFolder(shareID, passCodeToken, parentID string) (*XLShareFolderResp, error) {
apiURL := x.apiHost() + "/drive/v1/share/detail"
body := map[string]interface{}{
"share_id": shareID,
"pass_code_token": passCodeToken,
"parent_id": parentID,
"limit": 100,
"thumbnail_size": "SIZE_LARGE",
"order": "6",
}
bs, _ := json.Marshal(body)
req, err := http.NewRequest("POST", apiURL, bytes.NewReader(bs))
if err != nil {
return nil, err
}
x.setCommonHeader(req)
client := &http.Client{Timeout: 15 * time.Second}
resp, err := client.Do(req)
if err != nil {
return nil, err
}
defer resp.Body.Close()
result, _ := ioutil.ReadAll(resp.Body)
if resp.StatusCode != 200 {
return nil, fmt.Errorf("HTTP %d: %s", resp.StatusCode, string(result))
}
var data XLShareFolderResp
if err := json.Unmarshal(result, &data); err != nil {
return nil, err
}
return &data, nil
}
// Restore 转存POST, body
func (x *XunleiPanService) Restore(shareID, passCodeToken string, fileIDs []string) (*XLRestoreResp, error) {
apiURL := x.apiHost() + "/drive/v1/share/restore"
body := map[string]interface{}{
"share_id": shareID,
"pass_code_token": passCodeToken,
"file_ids": fileIDs,
"folder_type": "NORMAL",
"specify_parent_id": true,
"parent_id": "",
}
bs, _ := json.Marshal(body)
req, err := http.NewRequest("POST", apiURL, bytes.NewReader(bs))
if err != nil {
return nil, err
}
x.setCommonHeader(req)
client := &http.Client{Timeout: 15 * time.Second}
resp, err := client.Do(req)
if err != nil {
return nil, err
}
defer resp.Body.Close()
result, _ := ioutil.ReadAll(resp.Body)
if resp.StatusCode != 200 {
return nil, fmt.Errorf("HTTP %d: %s", resp.StatusCode, string(result))
}
var data XLRestoreResp
if err := json.Unmarshal(result, &data); err != nil {
return nil, err
}
return &data, nil
}
// 结构体完全对齐 xunleix
type XLShareListResp struct {
Data struct {
List []struct {
ShareID string `json:"share_id"`
Title string `json:"title"`
} `json:"list"`
} `json:"data"`
Code int `json:"code"`
Msg string `json:"msg"`
}
type XLBatchShareResp struct {
Data struct {
ShareURL string `json:"share_url"`
} `json:"data"`
Code int `json:"code"`
Msg string `json:"msg"`
}
type XLCommonResp struct {
Code int `json:"code"`
Msg string `json:"msg"`
}
type XLShareFolderResp struct {
Data struct {
Files []struct {
FileID string `json:"file_id"`
Name string `json:"name"`
} `json:"files"`
} `json:"data"`
Code int `json:"code"`
Msg string `json:"msg"`
}
type XLRestoreResp struct {
Data struct {
TaskID string `json:"task_id"`
} `json:"data"`
Code int `json:"code"`
Msg string `json:"msg"`
}
// 新增辅助结构体
type XLShareInfo struct {
ShareID string `json:"share_id"`
Title string `json:"title"`
Files []XLFileInfo `json:"files"`
}
type XLFileInfo struct {
FileID string `json:"file_id"`
Name string `json:"name"`
}
type XLTaskResult struct {
Status int `json:"status"`
TaskID string `json:"task_id"`
Data struct {
ShareID string `json:"share_id"`
} `json:"data"`
}

1
db/ad.txt Normal file
View File

@@ -0,0 +1 @@
微信,独家,V信,v信,威信,胖狗资源,加微,会员群,q群,v群,公众号,广告,特价,最后机会,不要错过,立减,立得,赚,省,回扣,抽奖,失效,年会员,空间容量,微信群,群文件,全网资源,影视资源,扫码,最新资源,IMG_,资源汇总,緑铯粢源,.url,网盘推广,大额优惠券,资源文档,dy8.xyz,妙妙屋,资源合集,kkdm,赚收益

View File

@@ -63,26 +63,38 @@ func InitDB() error {
sqlDB.SetMaxOpenConns(100) // 最大打开连接数
sqlDB.SetConnMaxLifetime(time.Hour) // 连接最大生命周期
// 自动迁移数据库表结构
err = DB.AutoMigrate(
&entity.User{},
&entity.Category{},
&entity.Pan{},
&entity.Cks{},
&entity.Tag{},
&entity.Resource{},
&entity.ResourceTag{},
&entity.ReadyResource{},
&entity.SearchStat{},
&entity.SystemConfig{},
&entity.HotDrama{},
)
if err != nil {
utils.Fatal("数据库迁移失败: %v", err)
// 检查是否需要迁移(只在开发环境或首次启动时)
if shouldRunMigration() {
utils.Info("开始数据库迁移...")
err = DB.AutoMigrate(
&entity.User{},
&entity.Category{},
&entity.Pan{},
&entity.Cks{},
&entity.Tag{},
&entity.Resource{},
&entity.ResourceTag{},
&entity.ReadyResource{},
&entity.SearchStat{},
&entity.SystemConfig{},
&entity.HotDrama{},
&entity.ResourceView{},
&entity.Task{},
&entity.TaskItem{},
&entity.File{},
)
if err != nil {
utils.Fatal("数据库迁移失败: %v", err)
}
utils.Info("数据库迁移完成")
} else {
utils.Info("跳过数据库迁移(表结构已是最新)")
}
// 创建索引以提高查询性能
createIndexes(DB)
// 创建索引以提高查询性能(只在需要迁移时)
if shouldRunMigration() {
createIndexes(DB)
}
// 插入默认数据(只在数据库为空时)
if err := insertDefaultDataIfEmpty(); err != nil {
@@ -93,9 +105,36 @@ func InitDB() error {
return nil
}
// shouldRunMigration 检查是否需要运行数据库迁移
func shouldRunMigration() bool {
// 通过环境变量控制是否运行迁移
skipMigration := os.Getenv("SKIP_MIGRATION")
if skipMigration == "true" {
return false
}
// 检查环境变量
env := os.Getenv("ENV")
if env == "production" {
// 生产环境:检查是否有迁移标记
var count int64
DB.Raw("SELECT COUNT(*) FROM information_schema.tables WHERE table_name = 'schema_migrations'").Count(&count)
if count == 0 {
// 没有迁移表,说明是首次部署
return true
}
// 有迁移表,检查是否需要迁移(这里可以添加更复杂的逻辑)
return false
}
// 开发环境:总是运行迁移
return true
}
// autoMigrate 自动迁移表结构
func autoMigrate() error {
return DB.AutoMigrate(
&entity.SystemConfig{}, // 系统配置表(独立表,先创建)
&entity.Pan{},
&entity.Cks{},
&entity.Category{},
@@ -105,16 +144,14 @@ func autoMigrate() error {
&entity.ReadyResource{},
&entity.User{},
&entity.SearchStat{},
&entity.SystemConfig{},
&entity.HotDrama{},
&entity.File{},
)
}
// createIndexes 创建数据库索引以提高查询性能
func createIndexes(db *gorm.DB) {
// 资源表索引
db.Exec("CREATE INDEX IF NOT EXISTS idx_resources_title ON resources USING gin(to_tsvector('chinese', title))")
db.Exec("CREATE INDEX IF NOT EXISTS idx_resources_description ON resources USING gin(to_tsvector('chinese', description))")
// 资源表索引移除全文搜索索引使用Meilisearch替代
db.Exec("CREATE INDEX IF NOT EXISTS idx_resources_category_id ON resources(category_id)")
db.Exec("CREATE INDEX IF NOT EXISTS idx_resources_pan_id ON resources(pan_id)")
db.Exec("CREATE INDEX IF NOT EXISTS idx_resources_created_at ON resources(created_at DESC)")
@@ -122,13 +159,17 @@ func createIndexes(db *gorm.DB) {
db.Exec("CREATE INDEX IF NOT EXISTS idx_resources_is_valid ON resources(is_valid)")
db.Exec("CREATE INDEX IF NOT EXISTS idx_resources_is_public ON resources(is_public)")
// 为Meilisearch准备的基础文本索引用于精确匹配
db.Exec("CREATE INDEX IF NOT EXISTS idx_resources_title ON resources(title)")
db.Exec("CREATE INDEX IF NOT EXISTS idx_resources_description ON resources(description)")
// 待处理资源表索引
db.Exec("CREATE INDEX IF NOT EXISTS idx_ready_resource_key ON ready_resource(key)")
db.Exec("CREATE INDEX IF NOT EXISTS idx_ready_resource_url ON ready_resource(url)")
db.Exec("CREATE INDEX IF NOT EXISTS idx_ready_resource_create_time ON ready_resource(create_time DESC)")
// 搜索统计表索引
db.Exec("CREATE INDEX IF NOT EXISTS idx_search_stats_query ON search_stats(query)")
db.Exec("CREATE INDEX IF NOT EXISTS idx_search_stats_keyword ON search_stats(keyword)")
db.Exec("CREATE INDEX IF NOT EXISTS idx_search_stats_created_at ON search_stats(created_at DESC)")
// 热播剧表索引
@@ -140,7 +181,7 @@ func createIndexes(db *gorm.DB) {
db.Exec("CREATE INDEX IF NOT EXISTS idx_resource_tags_resource_id ON resource_tags(resource_id)")
db.Exec("CREATE INDEX IF NOT EXISTS idx_resource_tags_tag_id ON resource_tags(tag_id)")
utils.Info("数据库索引创建完成")
utils.Info("数据库索引创建完成已移除全文搜索索引准备使用Meilisearch")
}
// insertDefaultDataIfEmpty 只在数据库为空时插入默认数据
@@ -161,11 +202,18 @@ func insertDefaultDataIfEmpty() error {
// 插入默认分类使用FirstOrCreate避免重复
defaultCategories := []entity.Category{
{Name: "文档", Description: "各种文档资料"},
{Name: "软件", Description: "软件工具"},
{Name: "视频", Description: "视频教程"},
{Name: "图片", Description: "图片资源"},
{Name: "音频", Description: "音频文件"},
{Name: "电影", Description: "电影"},
{Name: "电视剧", Description: "电视剧"},
{Name: "短剧", Description: "短剧"},
{Name: "综艺", Description: "综艺"},
{Name: "动漫", Description: "动漫"},
{Name: "纪录片", Description: "纪录片"},
{Name: "视频教程", Description: "视频教程"},
{Name: "学习资料", Description: "学习资料"},
{Name: "游戏", Description: "其他游戏资源"},
{Name: "软件", Description: "软件"},
{Name: "APP", Description: "APP"},
{Name: "AI", Description: "AI"},
{Name: "其他", Description: "其他资源"},
}
@@ -196,6 +244,41 @@ func insertDefaultDataIfEmpty() error {
}
}
// 插入默认系统配置
defaultSystemConfigs := []entity.SystemConfig{
{Key: entity.ConfigKeySiteTitle, Value: entity.ConfigDefaultSiteTitle, Type: entity.ConfigTypeString},
{Key: entity.ConfigKeySiteDescription, Value: entity.ConfigDefaultSiteDescription, Type: entity.ConfigTypeString},
{Key: entity.ConfigKeyKeywords, Value: entity.ConfigDefaultKeywords, Type: entity.ConfigTypeString},
{Key: entity.ConfigKeyAuthor, Value: entity.ConfigDefaultAuthor, Type: entity.ConfigTypeString},
{Key: entity.ConfigKeyCopyright, Value: entity.ConfigDefaultCopyright, Type: entity.ConfigTypeString},
{Key: entity.ConfigKeyAutoProcessReadyResources, Value: entity.ConfigDefaultAutoProcessReadyResources, Type: entity.ConfigTypeBool},
{Key: entity.ConfigKeyAutoProcessInterval, Value: entity.ConfigDefaultAutoProcessInterval, Type: entity.ConfigTypeInt},
{Key: entity.ConfigKeyAutoTransferEnabled, Value: entity.ConfigDefaultAutoTransferEnabled, Type: entity.ConfigTypeBool},
{Key: entity.ConfigKeyAutoTransferLimitDays, Value: entity.ConfigDefaultAutoTransferLimitDays, Type: entity.ConfigTypeInt},
{Key: entity.ConfigKeyAutoTransferMinSpace, Value: entity.ConfigDefaultAutoTransferMinSpace, Type: entity.ConfigTypeInt},
{Key: entity.ConfigKeyAutoFetchHotDramaEnabled, Value: entity.ConfigDefaultAutoFetchHotDramaEnabled, Type: entity.ConfigTypeBool},
{Key: entity.ConfigKeyApiToken, Value: entity.ConfigDefaultApiToken, Type: entity.ConfigTypeString},
{Key: entity.ConfigKeyForbiddenWords, Value: entity.ConfigDefaultForbiddenWords, Type: entity.ConfigTypeString},
{Key: entity.ConfigKeyAdKeywords, Value: entity.ConfigDefaultAdKeywords, Type: entity.ConfigTypeString},
{Key: entity.ConfigKeyAutoInsertAd, Value: entity.ConfigDefaultAutoInsertAd, Type: entity.ConfigTypeString},
{Key: entity.ConfigKeyPageSize, Value: entity.ConfigDefaultPageSize, Type: entity.ConfigTypeInt},
{Key: entity.ConfigKeyMaintenanceMode, Value: entity.ConfigDefaultMaintenanceMode, Type: entity.ConfigTypeBool},
{Key: entity.ConfigKeyEnableRegister, Value: entity.ConfigDefaultEnableRegister, Type: entity.ConfigTypeBool},
{Key: entity.ConfigKeyThirdPartyStatsCode, Value: entity.ConfigDefaultThirdPartyStatsCode, Type: entity.ConfigTypeString},
{Key: entity.ConfigKeyMeilisearchEnabled, Value: entity.ConfigDefaultMeilisearchEnabled, Type: entity.ConfigTypeBool},
{Key: entity.ConfigKeyMeilisearchHost, Value: entity.ConfigDefaultMeilisearchHost, Type: entity.ConfigTypeString},
{Key: entity.ConfigKeyMeilisearchPort, Value: entity.ConfigDefaultMeilisearchPort, Type: entity.ConfigTypeString},
{Key: entity.ConfigKeyMeilisearchMasterKey, Value: entity.ConfigDefaultMeilisearchMasterKey, Type: entity.ConfigTypeString},
{Key: entity.ConfigKeyMeilisearchIndexName, Value: entity.ConfigDefaultMeilisearchIndexName, Type: entity.ConfigTypeString},
}
for _, config := range defaultSystemConfigs {
if err := DB.Where("key = ?", config.Key).FirstOrCreate(&config).Error; err != nil {
utils.Error("插入系统配置 %s 失败: %v", config.Key, err)
// 继续执行,不因为单个配置失败而停止
}
}
// 插入默认管理员用户
defaultAdmin := entity.User{
Username: "admin",

View File

@@ -1,6 +1,9 @@
package converter
import (
"reflect"
"time"
"github.com/ctwj/urldb/db/dto"
"github.com/ctwj/urldb/db/entity"
)
@@ -8,22 +11,24 @@ import (
// ToResourceResponse 将Resource实体转换为ResourceResponse
func ToResourceResponse(resource *entity.Resource) dto.ResourceResponse {
response := dto.ResourceResponse{
ID: resource.ID,
Title: resource.Title,
Description: resource.Description,
URL: resource.URL,
PanID: resource.PanID,
SaveURL: resource.SaveURL,
FileSize: resource.FileSize,
CategoryID: resource.CategoryID,
ViewCount: resource.ViewCount,
IsValid: resource.IsValid,
IsPublic: resource.IsPublic,
CreatedAt: resource.CreatedAt,
UpdatedAt: resource.UpdatedAt,
Cover: resource.Cover,
Author: resource.Author,
ErrorMsg: resource.ErrorMsg,
ID: resource.ID,
Title: resource.Title,
Description: resource.Description,
URL: resource.URL,
PanID: resource.PanID,
SaveURL: resource.SaveURL,
FileSize: resource.FileSize,
CategoryID: resource.CategoryID,
ViewCount: resource.ViewCount,
IsValid: resource.IsValid,
IsPublic: resource.IsPublic,
CreatedAt: resource.CreatedAt,
UpdatedAt: resource.UpdatedAt,
Cover: resource.Cover,
Author: resource.Author,
ErrorMsg: resource.ErrorMsg,
SyncedToMeilisearch: resource.SyncedToMeilisearch,
SyncedAt: resource.SyncedAt,
}
// 设置分类名称
@@ -45,6 +50,89 @@ func ToResourceResponse(resource *entity.Resource) dto.ResourceResponse {
return response
}
// ToResourceResponseFromMeilisearch 将MeilisearchDocument转换为ResourceResponse包含高亮信息
func ToResourceResponseFromMeilisearch(doc interface{}) dto.ResourceResponse {
// 使用反射来获取MeilisearchDocument的字段
docValue := reflect.ValueOf(doc)
if docValue.Kind() == reflect.Ptr {
docValue = docValue.Elem()
}
response := dto.ResourceResponse{}
// 获取基本字段
if idField := docValue.FieldByName("ID"); idField.IsValid() {
response.ID = uint(idField.Uint())
}
if titleField := docValue.FieldByName("Title"); titleField.IsValid() {
response.Title = titleField.String()
}
if descField := docValue.FieldByName("Description"); descField.IsValid() {
response.Description = descField.String()
}
if urlField := docValue.FieldByName("URL"); urlField.IsValid() {
response.URL = urlField.String()
}
if saveURLField := docValue.FieldByName("SaveURL"); saveURLField.IsValid() {
response.SaveURL = saveURLField.String()
}
if fileSizeField := docValue.FieldByName("FileSize"); fileSizeField.IsValid() {
response.FileSize = fileSizeField.String()
}
if keyField := docValue.FieldByName("Key"); keyField.IsValid() {
// Key字段在ResourceResponse中不存在跳过
}
if categoryField := docValue.FieldByName("Category"); categoryField.IsValid() {
response.CategoryName = categoryField.String()
}
if authorField := docValue.FieldByName("Author"); authorField.IsValid() {
response.Author = authorField.String()
}
if createdAtField := docValue.FieldByName("CreatedAt"); createdAtField.IsValid() {
response.CreatedAt = createdAtField.Interface().(time.Time)
}
if updatedAtField := docValue.FieldByName("UpdatedAt"); updatedAtField.IsValid() {
response.UpdatedAt = updatedAtField.Interface().(time.Time)
}
// 处理PanID
if panIDField := docValue.FieldByName("PanID"); panIDField.IsValid() && !panIDField.IsNil() {
panIDPtr := panIDField.Interface().(*uint)
if panIDPtr != nil {
response.PanID = panIDPtr
}
}
// 处理Tags
if tagsField := docValue.FieldByName("Tags"); tagsField.IsValid() {
tags := tagsField.Interface().([]string)
response.Tags = make([]dto.TagResponse, len(tags))
for i, tagName := range tags {
response.Tags[i] = dto.TagResponse{
Name: tagName,
}
}
}
// 处理高亮字段
if titleHighlightField := docValue.FieldByName("TitleHighlight"); titleHighlightField.IsValid() {
response.TitleHighlight = titleHighlightField.String()
}
if descHighlightField := docValue.FieldByName("DescriptionHighlight"); descHighlightField.IsValid() {
response.DescriptionHighlight = descHighlightField.String()
}
if categoryHighlightField := docValue.FieldByName("CategoryHighlight"); categoryHighlightField.IsValid() {
response.CategoryHighlight = categoryHighlightField.String()
}
if tagsHighlightField := docValue.FieldByName("TagsHighlight"); tagsHighlightField.IsValid() {
tagsHighlight := tagsHighlightField.Interface().([]string)
response.TagsHighlight = make([]string, len(tagsHighlight))
copy(response.TagsHighlight, tagsHighlight)
}
return response
}
// ToResourceResponseList 将Resource实体列表转换为ResourceResponse列表
func ToResourceResponseList(resources []entity.Resource) []dto.ResourceResponse {
responses := make([]dto.ResourceResponse, len(resources))
@@ -169,6 +257,12 @@ func ToCksResponseList(cksList []entity.Cks) []dto.CksResponse {
// ToReadyResourceResponse 将ReadyResource实体转换为ReadyResourceResponse
func ToReadyResourceResponse(resource *entity.ReadyResource) dto.ReadyResourceResponse {
isDeleted := !resource.DeletedAt.Time.IsZero()
var deletedAt *time.Time
if isDeleted {
deletedAt = &resource.DeletedAt.Time
}
return dto.ReadyResourceResponse{
ID: resource.ID,
Title: resource.Title,
@@ -183,6 +277,8 @@ func ToReadyResourceResponse(resource *entity.ReadyResource) dto.ReadyResourceRe
ErrorMsg: resource.ErrorMsg,
CreateTime: resource.CreateTime,
IP: resource.IP,
DeletedAt: deletedAt,
IsDeleted: isDeleted,
}
}

View File

@@ -0,0 +1,54 @@
package converter
import (
"github.com/ctwj/urldb/db/dto"
"github.com/ctwj/urldb/db/entity"
"github.com/ctwj/urldb/utils"
)
// FileToResponse 将文件实体转换为响应DTO
func FileToResponse(file *entity.File) dto.FileResponse {
response := dto.FileResponse{
ID: file.ID,
CreatedAt: utils.FormatTime(file.CreatedAt, "2006-01-02 15:04:05"),
UpdatedAt: utils.FormatTime(file.UpdatedAt, "2006-01-02 15:04:05"),
OriginalName: file.OriginalName,
FileName: file.FileName,
FilePath: file.FilePath,
FileSize: file.FileSize,
FileType: file.FileType,
MimeType: file.MimeType,
FileHash: file.FileHash,
AccessURL: file.AccessURL,
UserID: file.UserID,
Status: file.Status,
IsPublic: file.IsPublic,
IsDeleted: file.IsDeleted,
}
// 添加用户名
if file.User.ID > 0 {
response.User = file.User.Username
}
return response
}
// FilesToResponse 将文件实体列表转换为响应DTO列表
func FilesToResponse(files []entity.File) []dto.FileResponse {
var responses []dto.FileResponse
for _, file := range files {
responses = append(responses, FileToResponse(&file))
}
return responses
}
// FileListToResponse 将文件列表转换为列表响应
func FileListToResponse(files []entity.File, total int64, page, pageSize int) dto.FileListResponse {
return dto.FileListResponse{
Files: FilesToResponse(files),
Total: total,
Page: page,
Size: pageSize,
}
}

View File

@@ -6,6 +6,7 @@ import (
"github.com/ctwj/urldb/db/dto"
"github.com/ctwj/urldb/db/entity"
"github.com/ctwj/urldb/utils"
)
// SystemConfigToResponse 将系统配置实体列表转换为响应DTO
@@ -29,6 +30,8 @@ func SystemConfigToResponse(configs []entity.SystemConfig) *dto.SystemConfigResp
response.Author = config.Value
case entity.ConfigKeyCopyright:
response.Copyright = config.Value
case entity.ConfigKeySiteLogo:
response.SiteLogo = config.Value
case entity.ConfigKeyAutoProcessReadyResources:
if val, err := strconv.ParseBool(config.Value); err == nil {
response.AutoProcessReadyResources = val
@@ -55,6 +58,12 @@ func SystemConfigToResponse(configs []entity.SystemConfig) *dto.SystemConfigResp
}
case entity.ConfigKeyApiToken:
response.ApiToken = config.Value
case entity.ConfigKeyForbiddenWords:
response.ForbiddenWords = config.Value
case entity.ConfigKeyAdKeywords:
response.AdKeywords = config.Value
case entity.ConfigKeyAutoInsertAd:
response.AutoInsertAd = config.Value
case entity.ConfigKeyPageSize:
if val, err := strconv.Atoi(config.Value); err == nil {
response.PageSize = val
@@ -63,6 +72,24 @@ func SystemConfigToResponse(configs []entity.SystemConfig) *dto.SystemConfigResp
if val, err := strconv.ParseBool(config.Value); err == nil {
response.MaintenanceMode = val
}
case entity.ConfigKeyEnableRegister:
if val, err := strconv.ParseBool(config.Value); err == nil {
response.EnableRegister = val
}
case entity.ConfigKeyThirdPartyStatsCode:
response.ThirdPartyStatsCode = config.Value
case entity.ConfigKeyMeilisearchEnabled:
if val, err := strconv.ParseBool(config.Value); err == nil {
response.MeilisearchEnabled = val
}
case entity.ConfigKeyMeilisearchHost:
response.MeilisearchHost = config.Value
case entity.ConfigKeyMeilisearchPort:
response.MeilisearchPort = config.Value
case entity.ConfigKeyMeilisearchMasterKey:
response.MeilisearchMasterKey = config.Value
case entity.ConfigKeyMeilisearchIndexName:
response.MeilisearchIndexName = config.Value
}
}
@@ -81,21 +108,122 @@ func RequestToSystemConfig(req *dto.SystemConfigRequest) []entity.SystemConfig {
return nil
}
configs := []entity.SystemConfig{
{Key: entity.ConfigKeySiteTitle, Value: req.SiteTitle, Type: entity.ConfigTypeString},
{Key: entity.ConfigKeySiteDescription, Value: req.SiteDescription, Type: entity.ConfigTypeString},
{Key: entity.ConfigKeyKeywords, Value: req.Keywords, Type: entity.ConfigTypeString},
{Key: entity.ConfigKeyAuthor, Value: req.Author, Type: entity.ConfigTypeString},
{Key: entity.ConfigKeyCopyright, Value: req.Copyright, Type: entity.ConfigTypeString},
{Key: entity.ConfigKeyAutoProcessReadyResources, Value: strconv.FormatBool(req.AutoProcessReadyResources), Type: entity.ConfigTypeBool},
{Key: entity.ConfigKeyAutoProcessInterval, Value: strconv.Itoa(req.AutoProcessInterval), Type: entity.ConfigTypeInt},
{Key: entity.ConfigKeyAutoTransferEnabled, Value: strconv.FormatBool(req.AutoTransferEnabled), Type: entity.ConfigTypeBool},
{Key: entity.ConfigKeyAutoTransferLimitDays, Value: strconv.Itoa(req.AutoTransferLimitDays), Type: entity.ConfigTypeInt},
{Key: entity.ConfigKeyAutoTransferMinSpace, Value: strconv.Itoa(req.AutoTransferMinSpace), Type: entity.ConfigTypeInt},
{Key: entity.ConfigKeyAutoFetchHotDramaEnabled, Value: strconv.FormatBool(req.AutoFetchHotDramaEnabled), Type: entity.ConfigTypeBool},
{Key: entity.ConfigKeyApiToken, Value: req.ApiToken, Type: entity.ConfigTypeString},
{Key: entity.ConfigKeyPageSize, Value: strconv.Itoa(req.PageSize), Type: entity.ConfigTypeInt},
{Key: entity.ConfigKeyMaintenanceMode, Value: strconv.FormatBool(req.MaintenanceMode), Type: entity.ConfigTypeBool},
var configs []entity.SystemConfig
var updatedKeys []string
// 字符串字段 - 只处理被设置的字段
if req.SiteTitle != nil {
configs = append(configs, entity.SystemConfig{Key: entity.ConfigKeySiteTitle, Value: *req.SiteTitle, Type: entity.ConfigTypeString})
updatedKeys = append(updatedKeys, entity.ConfigKeySiteTitle)
}
if req.SiteDescription != nil {
configs = append(configs, entity.SystemConfig{Key: entity.ConfigKeySiteDescription, Value: *req.SiteDescription, Type: entity.ConfigTypeString})
updatedKeys = append(updatedKeys, entity.ConfigKeySiteDescription)
}
if req.Keywords != nil {
configs = append(configs, entity.SystemConfig{Key: entity.ConfigKeyKeywords, Value: *req.Keywords, Type: entity.ConfigTypeString})
updatedKeys = append(updatedKeys, entity.ConfigKeyKeywords)
}
if req.Author != nil {
configs = append(configs, entity.SystemConfig{Key: entity.ConfigKeyAuthor, Value: *req.Author, Type: entity.ConfigTypeString})
updatedKeys = append(updatedKeys, entity.ConfigKeyAuthor)
}
if req.Copyright != nil {
configs = append(configs, entity.SystemConfig{Key: entity.ConfigKeyCopyright, Value: *req.Copyright, Type: entity.ConfigTypeString})
updatedKeys = append(updatedKeys, entity.ConfigKeyCopyright)
}
if req.SiteLogo != nil {
configs = append(configs, entity.SystemConfig{Key: entity.ConfigKeySiteLogo, Value: *req.SiteLogo, Type: entity.ConfigTypeString})
updatedKeys = append(updatedKeys, entity.ConfigKeySiteLogo)
}
if req.ApiToken != nil {
configs = append(configs, entity.SystemConfig{Key: entity.ConfigKeyApiToken, Value: *req.ApiToken, Type: entity.ConfigTypeString})
updatedKeys = append(updatedKeys, entity.ConfigKeyApiToken)
}
if req.ForbiddenWords != nil {
configs = append(configs, entity.SystemConfig{Key: entity.ConfigKeyForbiddenWords, Value: *req.ForbiddenWords, Type: entity.ConfigTypeString})
updatedKeys = append(updatedKeys, entity.ConfigKeyForbiddenWords)
}
if req.AdKeywords != nil {
configs = append(configs, entity.SystemConfig{Key: entity.ConfigKeyAdKeywords, Value: *req.AdKeywords, Type: entity.ConfigTypeString})
updatedKeys = append(updatedKeys, entity.ConfigKeyAdKeywords)
}
if req.AutoInsertAd != nil {
configs = append(configs, entity.SystemConfig{Key: entity.ConfigKeyAutoInsertAd, Value: *req.AutoInsertAd, Type: entity.ConfigTypeString})
updatedKeys = append(updatedKeys, entity.ConfigKeyAutoInsertAd)
}
// 布尔值字段 - 只处理被设置的字段
if req.AutoProcessReadyResources != nil {
configs = append(configs, entity.SystemConfig{Key: entity.ConfigKeyAutoProcessReadyResources, Value: strconv.FormatBool(*req.AutoProcessReadyResources), Type: entity.ConfigTypeBool})
updatedKeys = append(updatedKeys, entity.ConfigKeyAutoProcessReadyResources)
}
if req.AutoTransferEnabled != nil {
configs = append(configs, entity.SystemConfig{Key: entity.ConfigKeyAutoTransferEnabled, Value: strconv.FormatBool(*req.AutoTransferEnabled), Type: entity.ConfigTypeBool})
updatedKeys = append(updatedKeys, entity.ConfigKeyAutoTransferEnabled)
}
if req.AutoFetchHotDramaEnabled != nil {
configs = append(configs, entity.SystemConfig{Key: entity.ConfigKeyAutoFetchHotDramaEnabled, Value: strconv.FormatBool(*req.AutoFetchHotDramaEnabled), Type: entity.ConfigTypeBool})
updatedKeys = append(updatedKeys, entity.ConfigKeyAutoFetchHotDramaEnabled)
}
if req.MaintenanceMode != nil {
configs = append(configs, entity.SystemConfig{Key: entity.ConfigKeyMaintenanceMode, Value: strconv.FormatBool(*req.MaintenanceMode), Type: entity.ConfigTypeBool})
updatedKeys = append(updatedKeys, entity.ConfigKeyMaintenanceMode)
}
if req.EnableRegister != nil {
configs = append(configs, entity.SystemConfig{Key: entity.ConfigKeyEnableRegister, Value: strconv.FormatBool(*req.EnableRegister), Type: entity.ConfigTypeBool})
updatedKeys = append(updatedKeys, entity.ConfigKeyEnableRegister)
}
// 整数字段 - 只处理被设置的字段
if req.AutoProcessInterval != nil {
configs = append(configs, entity.SystemConfig{Key: entity.ConfigKeyAutoProcessInterval, Value: strconv.Itoa(*req.AutoProcessInterval), Type: entity.ConfigTypeInt})
updatedKeys = append(updatedKeys, entity.ConfigKeyAutoProcessInterval)
}
if req.AutoTransferLimitDays != nil {
configs = append(configs, entity.SystemConfig{Key: entity.ConfigKeyAutoTransferLimitDays, Value: strconv.Itoa(*req.AutoTransferLimitDays), Type: entity.ConfigTypeInt})
updatedKeys = append(updatedKeys, entity.ConfigKeyAutoTransferLimitDays)
}
if req.AutoTransferMinSpace != nil {
configs = append(configs, entity.SystemConfig{Key: entity.ConfigKeyAutoTransferMinSpace, Value: strconv.Itoa(*req.AutoTransferMinSpace), Type: entity.ConfigTypeInt})
updatedKeys = append(updatedKeys, entity.ConfigKeyAutoTransferMinSpace)
}
if req.PageSize != nil {
configs = append(configs, entity.SystemConfig{Key: entity.ConfigKeyPageSize, Value: strconv.Itoa(*req.PageSize), Type: entity.ConfigTypeInt})
updatedKeys = append(updatedKeys, entity.ConfigKeyPageSize)
}
// 三方统计配置 - 只处理被设置的字段
if req.ThirdPartyStatsCode != nil {
configs = append(configs, entity.SystemConfig{Key: entity.ConfigKeyThirdPartyStatsCode, Value: *req.ThirdPartyStatsCode, Type: entity.ConfigTypeString})
updatedKeys = append(updatedKeys, entity.ConfigKeyThirdPartyStatsCode)
}
// Meilisearch配置 - 只处理被设置的字段
if req.MeilisearchEnabled != nil {
configs = append(configs, entity.SystemConfig{Key: entity.ConfigKeyMeilisearchEnabled, Value: strconv.FormatBool(*req.MeilisearchEnabled), Type: entity.ConfigTypeBool})
updatedKeys = append(updatedKeys, entity.ConfigKeyMeilisearchEnabled)
}
if req.MeilisearchHost != nil {
configs = append(configs, entity.SystemConfig{Key: entity.ConfigKeyMeilisearchHost, Value: *req.MeilisearchHost, Type: entity.ConfigTypeString})
updatedKeys = append(updatedKeys, entity.ConfigKeyMeilisearchHost)
}
if req.MeilisearchPort != nil {
configs = append(configs, entity.SystemConfig{Key: entity.ConfigKeyMeilisearchPort, Value: *req.MeilisearchPort, Type: entity.ConfigTypeString})
updatedKeys = append(updatedKeys, entity.ConfigKeyMeilisearchPort)
}
if req.MeilisearchMasterKey != nil {
configs = append(configs, entity.SystemConfig{Key: entity.ConfigKeyMeilisearchMasterKey, Value: *req.MeilisearchMasterKey, Type: entity.ConfigTypeString})
updatedKeys = append(updatedKeys, entity.ConfigKeyMeilisearchMasterKey)
}
if req.MeilisearchIndexName != nil {
configs = append(configs, entity.SystemConfig{Key: entity.ConfigKeyMeilisearchIndexName, Value: *req.MeilisearchIndexName, Type: entity.ConfigTypeString})
updatedKeys = append(updatedKeys, entity.ConfigKeyMeilisearchIndexName)
}
// 记录更新的配置项
if len(updatedKeys) > 0 {
utils.Info("配置更新 - 被修改的配置项: %v", updatedKeys)
}
return configs
@@ -105,21 +233,32 @@ func RequestToSystemConfig(req *dto.SystemConfigRequest) []entity.SystemConfig {
func SystemConfigToPublicResponse(configs []entity.SystemConfig) map[string]interface{} {
response := map[string]interface{}{
entity.ConfigResponseFieldID: 0,
entity.ConfigResponseFieldCreatedAt: time.Now().Format("2006-01-02 15:04:05"),
entity.ConfigResponseFieldUpdatedAt: time.Now().Format("2006-01-02 15:04:05"),
entity.ConfigResponseFieldCreatedAt: utils.GetCurrentTimeString(),
entity.ConfigResponseFieldUpdatedAt: utils.GetCurrentTimeString(),
entity.ConfigResponseFieldSiteTitle: entity.ConfigDefaultSiteTitle,
entity.ConfigResponseFieldSiteDescription: entity.ConfigDefaultSiteDescription,
entity.ConfigResponseFieldKeywords: entity.ConfigDefaultKeywords,
entity.ConfigResponseFieldAuthor: entity.ConfigDefaultAuthor,
entity.ConfigResponseFieldCopyright: entity.ConfigDefaultCopyright,
"site_logo": "",
entity.ConfigResponseFieldAutoProcessReadyResources: false,
entity.ConfigResponseFieldAutoProcessInterval: 30,
entity.ConfigResponseFieldAutoTransferEnabled: false,
entity.ConfigResponseFieldAutoTransferLimitDays: 0,
entity.ConfigResponseFieldAutoTransferMinSpace: 100,
entity.ConfigResponseFieldAutoFetchHotDramaEnabled: false,
entity.ConfigResponseFieldForbiddenWords: "",
entity.ConfigResponseFieldAdKeywords: "",
entity.ConfigResponseFieldAutoInsertAd: "",
entity.ConfigResponseFieldPageSize: 100,
entity.ConfigResponseFieldMaintenanceMode: false,
entity.ConfigResponseFieldEnableRegister: true, // 默认开启注册功能
entity.ConfigResponseFieldThirdPartyStatsCode: "",
entity.ConfigResponseFieldMeilisearchEnabled: false,
entity.ConfigResponseFieldMeilisearchHost: "localhost",
entity.ConfigResponseFieldMeilisearchPort: "7700",
entity.ConfigResponseFieldMeilisearchMasterKey: "",
entity.ConfigResponseFieldMeilisearchIndexName: "resources",
}
// 将键值对转换为map
@@ -135,6 +274,8 @@ func SystemConfigToPublicResponse(configs []entity.SystemConfig) map[string]inte
response[entity.ConfigResponseFieldAuthor] = config.Value
case entity.ConfigKeyCopyright:
response[entity.ConfigResponseFieldCopyright] = config.Value
case entity.ConfigKeySiteLogo:
response["site_logo"] = config.Value
case entity.ConfigKeyAutoProcessReadyResources:
if val, err := strconv.ParseBool(config.Value); err == nil {
response[entity.ConfigResponseFieldAutoProcessReadyResources] = val
@@ -159,6 +300,12 @@ func SystemConfigToPublicResponse(configs []entity.SystemConfig) map[string]inte
if val, err := strconv.ParseBool(config.Value); err == nil {
response[entity.ConfigResponseFieldAutoFetchHotDramaEnabled] = val
}
case entity.ConfigKeyForbiddenWords:
response[entity.ConfigResponseFieldForbiddenWords] = config.Value
case entity.ConfigKeyAdKeywords:
response[entity.ConfigResponseFieldAdKeywords] = config.Value
case entity.ConfigKeyAutoInsertAd:
response[entity.ConfigResponseFieldAutoInsertAd] = config.Value
case entity.ConfigKeyPageSize:
if val, err := strconv.Atoi(config.Value); err == nil {
response[entity.ConfigResponseFieldPageSize] = val
@@ -167,13 +314,31 @@ func SystemConfigToPublicResponse(configs []entity.SystemConfig) map[string]inte
if val, err := strconv.ParseBool(config.Value); err == nil {
response[entity.ConfigResponseFieldMaintenanceMode] = val
}
case entity.ConfigKeyEnableRegister:
if val, err := strconv.ParseBool(config.Value); err == nil {
response[entity.ConfigResponseFieldEnableRegister] = val
}
case entity.ConfigKeyThirdPartyStatsCode:
response[entity.ConfigResponseFieldThirdPartyStatsCode] = config.Value
case entity.ConfigKeyMeilisearchEnabled:
if val, err := strconv.ParseBool(config.Value); err == nil {
response[entity.ConfigResponseFieldMeilisearchEnabled] = val
}
case entity.ConfigKeyMeilisearchHost:
response[entity.ConfigResponseFieldMeilisearchHost] = config.Value
case entity.ConfigKeyMeilisearchPort:
response[entity.ConfigResponseFieldMeilisearchPort] = config.Value
case entity.ConfigKeyMeilisearchMasterKey:
response[entity.ConfigResponseFieldMeilisearchMasterKey] = config.Value
case entity.ConfigKeyMeilisearchIndexName:
response[entity.ConfigResponseFieldMeilisearchIndexName] = config.Value
}
}
// 设置时间戳(使用第一个配置的时间)
if len(configs) > 0 {
response[entity.ConfigResponseFieldCreatedAt] = configs[0].CreatedAt.Format("2006-01-02 15:04:05")
response[entity.ConfigResponseFieldUpdatedAt] = configs[0].UpdatedAt.Format("2006-01-02 15:04:05")
response[entity.ConfigResponseFieldCreatedAt] = configs[0].CreatedAt.Format(utils.TimeFormatDateTime)
response[entity.ConfigResponseFieldUpdatedAt] = configs[0].UpdatedAt.Format(utils.TimeFormatDateTime)
}
return response
@@ -187,6 +352,7 @@ func getDefaultConfigResponse() *dto.SystemConfigResponse {
Keywords: entity.ConfigDefaultKeywords,
Author: entity.ConfigDefaultAuthor,
Copyright: entity.ConfigDefaultCopyright,
SiteLogo: "",
AutoProcessReadyResources: false,
AutoProcessInterval: 30,
AutoTransferEnabled: false,
@@ -194,7 +360,17 @@ func getDefaultConfigResponse() *dto.SystemConfigResponse {
AutoTransferMinSpace: 100,
AutoFetchHotDramaEnabled: false,
ApiToken: entity.ConfigDefaultApiToken,
ForbiddenWords: entity.ConfigDefaultForbiddenWords,
AdKeywords: entity.ConfigDefaultAdKeywords,
AutoInsertAd: entity.ConfigDefaultAutoInsertAd,
PageSize: 100,
MaintenanceMode: false,
EnableRegister: true, // 默认开启注册功能
ThirdPartyStatsCode: entity.ConfigDefaultThirdPartyStatsCode,
MeilisearchEnabled: false,
MeilisearchHost: entity.ConfigDefaultMeilisearchHost,
MeilisearchPort: entity.ConfigDefaultMeilisearchPort,
MeilisearchMasterKey: entity.ConfigDefaultMeilisearchMasterKey,
MeilisearchIndexName: entity.ConfigDefaultMeilisearchIndexName,
}
}

73
db/dto/file.go Normal file
View File

@@ -0,0 +1,73 @@
package dto
// FileUploadRequest 文件上传请求
type FileUploadRequest struct {
IsPublic bool `json:"is_public" form:"is_public"` // 是否公开
FileHash string `json:"file_hash" form:"file_hash"` // 文件哈希值
}
// FileResponse 文件响应
type FileResponse struct {
ID uint `json:"id"`
CreatedAt string `json:"created_at"`
UpdatedAt string `json:"updated_at"`
// 文件信息
OriginalName string `json:"original_name"`
FileName string `json:"file_name"`
FilePath string `json:"file_path"`
FileSize int64 `json:"file_size"`
FileType string `json:"file_type"`
MimeType string `json:"mime_type"`
FileHash string `json:"file_hash"`
// 访问信息
AccessURL string `json:"access_url"`
// 用户信息
UserID uint `json:"user_id"`
User string `json:"user"` // 用户名
// 状态信息
Status string `json:"status"`
IsPublic bool `json:"is_public"`
IsDeleted bool `json:"is_deleted"`
}
// FileListRequest 文件列表请求
type FileListRequest struct {
Page int `json:"page" form:"page"`
PageSize int `json:"page_size" form:"page_size"`
Search string `json:"search" form:"search"`
FileType string `json:"file_type" form:"file_type"`
Status string `json:"status" form:"status"`
UserID uint `json:"user_id" form:"user_id"`
}
// FileListResponse 文件列表响应
type FileListResponse struct {
Files []FileResponse `json:"files"`
Total int64 `json:"total"`
Page int `json:"page"`
Size int `json:"size"`
}
// FileUploadResponse 文件上传响应
type FileUploadResponse struct {
File FileResponse `json:"file"`
Message string `json:"message"`
Success bool `json:"success"`
IsDuplicate bool `json:"is_duplicate"` // 是否为重复文件
}
// FileDeleteRequest 文件删除请求
type FileDeleteRequest struct {
IDs []uint `json:"ids" binding:"required"`
}
// FileUpdateRequest 文件更新请求
type FileUpdateRequest struct {
ID uint `json:"id" binding:"required"`
IsPublic *bool `json:"is_public"`
Status string `json:"status"`
}

View File

@@ -12,24 +12,31 @@ type SearchResponse struct {
// ResourceResponse 资源响应
type ResourceResponse struct {
ID uint `json:"id"`
Title string `json:"title"`
Description string `json:"description"`
URL string `json:"url"`
PanID *uint `json:"pan_id"`
SaveURL string `json:"save_url"`
FileSize string `json:"file_size"`
CategoryID *uint `json:"category_id"`
CategoryName string `json:"category_name"`
ViewCount int `json:"view_count"`
IsValid bool `json:"is_valid"`
IsPublic bool `json:"is_public"`
CreatedAt time.Time `json:"created_at"`
UpdatedAt time.Time `json:"updated_at"`
Tags []TagResponse `json:"tags"`
Cover string `json:"cover"`
Author string `json:"author"`
ErrorMsg string `json:"error_msg"`
ID uint `json:"id"`
Title string `json:"title"`
Description string `json:"description"`
URL string `json:"url"`
PanID *uint `json:"pan_id"`
SaveURL string `json:"save_url"`
FileSize string `json:"file_size"`
CategoryID *uint `json:"category_id"`
CategoryName string `json:"category_name"`
ViewCount int `json:"view_count"`
IsValid bool `json:"is_valid"`
IsPublic bool `json:"is_public"`
CreatedAt time.Time `json:"created_at"`
UpdatedAt time.Time `json:"updated_at"`
Tags []TagResponse `json:"tags"`
Cover string `json:"cover"`
Author string `json:"author"`
ErrorMsg string `json:"error_msg"`
SyncedToMeilisearch bool `json:"synced_to_meilisearch"`
SyncedAt *time.Time `json:"synced_at"`
// 高亮字段
TitleHighlight string `json:"title_highlight,omitempty"`
DescriptionHighlight string `json:"description_highlight,omitempty"`
CategoryHighlight string `json:"category_highlight,omitempty"`
TagsHighlight []string `json:"tags_highlight,omitempty"`
}
// CategoryResponse 分类响应
@@ -79,19 +86,21 @@ type CksResponse struct {
// ReadyResourceResponse 待处理资源响应
type ReadyResourceResponse struct {
ID uint `json:"id"`
Title *string `json:"title"`
Description string `json:"description"`
URL string `json:"url"`
Category string `json:"category"`
Tags string `json:"tags"`
Img string `json:"img"`
Source string `json:"source"`
Extra string `json:"extra"`
Key string `json:"key"`
ErrorMsg string `json:"error_msg"`
CreateTime time.Time `json:"create_time"`
IP *string `json:"ip"`
ID uint `json:"id"`
Title *string `json:"title"`
Description string `json:"description"`
URL string `json:"url"`
Category string `json:"category"`
Tags string `json:"tags"`
Img string `json:"img"`
Source string `json:"source"`
Extra string `json:"extra"`
Key string `json:"key"`
ErrorMsg string `json:"error_msg"`
CreateTime time.Time `json:"create_time"`
IP *string `json:"ip"`
DeletedAt *time.Time `json:"deleted_at,omitempty"`
IsDeleted bool `json:"is_deleted"`
}
// Stats 统计信息

View File

@@ -3,26 +3,45 @@ package dto
// SystemConfigRequest 系统配置请求
type SystemConfigRequest struct {
// SEO 配置
SiteTitle string `json:"site_title" validate:"required"`
SiteDescription string `json:"site_description"`
Keywords string `json:"keywords"`
Author string `json:"author"`
Copyright string `json:"copyright"`
SiteTitle *string `json:"site_title,omitempty"`
SiteDescription *string `json:"site_description,omitempty"`
Keywords *string `json:"keywords,omitempty"`
Author *string `json:"author,omitempty"`
Copyright *string `json:"copyright,omitempty"`
SiteLogo *string `json:"site_logo,omitempty"`
// 自动处理配置组
AutoProcessReadyResources bool `json:"auto_process_ready_resources"` // 自动处理待处理资源
AutoProcessInterval int `json:"auto_process_interval" validate:"min=1,max=1440"` // 自动处理间隔(分钟)
AutoTransferEnabled bool `json:"auto_transfer_enabled"` // 开启自动转存
AutoTransferLimitDays int `json:"auto_transfer_limit_days" validate:"min=0,max=365"` // 自动转存限制天数0表示不限制
AutoTransferMinSpace int `json:"auto_transfer_min_space" validate:"min=100,max=1024"` // 最小存储空间GB
AutoFetchHotDramaEnabled bool `json:"auto_fetch_hot_drama_enabled"` // 自动拉取热播剧名字
AutoProcessReadyResources *bool `json:"auto_process_ready_resources,omitempty"` // 自动处理待处理资源
AutoProcessInterval *int `json:"auto_process_interval,omitempty"` // 自动处理间隔(分钟)
AutoTransferEnabled *bool `json:"auto_transfer_enabled,omitempty"` // 开启自动转存
AutoTransferLimitDays *int `json:"auto_transfer_limit_days,omitempty"` // 自动转存限制天数0表示不限制
AutoTransferMinSpace *int `json:"auto_transfer_min_space,omitempty"` // 最小存储空间GB
AutoFetchHotDramaEnabled *bool `json:"auto_fetch_hot_drama_enabled,omitempty"` // 自动拉取热播剧名字
// API配置
ApiToken string `json:"api_token"` // 公开API访问令牌
ApiToken *string `json:"api_token,omitempty"` // 公开API访问令牌
// 违禁词配置
ForbiddenWords *string `json:"forbidden_words,omitempty"` // 违禁词列表,用逗号分隔
// 广告配置
AdKeywords *string `json:"ad_keywords,omitempty"` // 广告关键词列表,用逗号分隔
AutoInsertAd *string `json:"auto_insert_ad,omitempty"` // 自动插入广告内容
// 其他配置
PageSize int `json:"page_size" validate:"min=10,max=500"`
MaintenanceMode bool `json:"maintenance_mode"`
PageSize *int `json:"page_size,omitempty"`
MaintenanceMode *bool `json:"maintenance_mode,omitempty"`
EnableRegister *bool `json:"enable_register,omitempty"` // 开启注册功能
// 三方统计配置
ThirdPartyStatsCode *string `json:"third_party_stats_code,omitempty"` // 三方统计代码
// Meilisearch配置
MeilisearchEnabled *bool `json:"meilisearch_enabled,omitempty"`
MeilisearchHost *string `json:"meilisearch_host,omitempty"`
MeilisearchPort *string `json:"meilisearch_port,omitempty"`
MeilisearchMasterKey *string `json:"meilisearch_master_key,omitempty"`
MeilisearchIndexName *string `json:"meilisearch_index_name,omitempty"`
}
// SystemConfigResponse 系统配置响应
@@ -37,6 +56,7 @@ type SystemConfigResponse struct {
Keywords string `json:"keywords"`
Author string `json:"author"`
Copyright string `json:"copyright"`
SiteLogo string `json:"site_logo"`
// 自动处理配置组
AutoProcessReadyResources bool `json:"auto_process_ready_resources"` // 自动处理待处理资源
@@ -49,9 +69,27 @@ type SystemConfigResponse struct {
// API配置
ApiToken string `json:"api_token"` // 公开API访问令牌
// 违禁词配置
ForbiddenWords string `json:"forbidden_words"` // 违禁词列表,用逗号分隔
// 广告配置
AdKeywords string `json:"ad_keywords"` // 广告关键词列表,用逗号分隔
AutoInsertAd string `json:"auto_insert_ad"` // 自动插入广告内容
// 其他配置
PageSize int `json:"page_size"`
MaintenanceMode bool `json:"maintenance_mode"`
EnableRegister bool `json:"enable_register"` // 开启注册功能
// 三方统计配置
ThirdPartyStatsCode string `json:"third_party_stats_code"` // 三方统计代码
// Meilisearch配置
MeilisearchEnabled bool `json:"meilisearch_enabled"`
MeilisearchHost string `json:"meilisearch_host"`
MeilisearchPort string `json:"meilisearch_port"`
MeilisearchMasterKey string `json:"meilisearch_master_key"`
MeilisearchIndexName string `json:"meilisearch_index_name"`
}
// SystemConfigItem 单个配置项

55
db/dto/task_config.go Normal file
View File

@@ -0,0 +1,55 @@
package dto
import "fmt"
// BatchTransferTaskConfig 批量转存任务配置
type BatchTransferTaskConfig struct {
CategoryID *uint `json:"category_id"` // 默认分类ID
TagIDs []uint `json:"tag_ids"` // 默认标签ID列表
}
// TaskConfig 通用任务配置接口
type TaskConfig interface {
// Validate 验证配置有效性
Validate() error
}
// Validate 验证批量转存任务配置
func (config BatchTransferTaskConfig) Validate() error {
// 这里可以添加配置验证逻辑
return nil
}
// 示例:未来可能的其他任务类型配置
// DataSyncTaskConfig 数据同步任务配置(示例)
type DataSyncTaskConfig struct {
SourceType string `json:"source_type"` // 数据源类型
TargetType string `json:"target_type"` // 目标类型
SyncMode string `json:"sync_mode"` // 同步模式
}
// Validate 验证数据同步任务配置
func (config DataSyncTaskConfig) Validate() error {
if config.SourceType == "" {
return fmt.Errorf("数据源类型不能为空")
}
if config.TargetType == "" {
return fmt.Errorf("目标类型不能为空")
}
return nil
}
// CleanupTaskConfig 清理任务配置(示例)
type CleanupTaskConfig struct {
RetentionDays int `json:"retention_days"` // 保留天数
CleanupType string `json:"cleanup_type"` // 清理类型
}
// Validate 验证清理任务配置
func (config CleanupTaskConfig) Validate() error {
if config.RetentionDays < 0 {
return fmt.Errorf("保留天数不能为负数")
}
return nil
}

56
db/dto/task_data.go Normal file
View File

@@ -0,0 +1,56 @@
package dto
import "fmt"
// BatchTransferInputData 批量转存任务的输入数据
type BatchTransferInputData struct {
Title string `json:"title"` // 资源标题
URL string `json:"url"` // 资源链接
CategoryID *uint `json:"category_id"` // 分类ID
TagIDs []uint `json:"tag_ids"` // 标签ID列表
}
// BatchTransferOutputData 批量转存任务的输出数据
type BatchTransferOutputData struct {
ResourceID uint `json:"resource_id"` // 创建的资源ID
SaveURL string `json:"save_url"` // 转存后的链接
PlatformID uint `json:"platform_id"` // 平台ID
}
// TaskItemData 通用任务项数据接口
type TaskItemData interface {
// GetDisplayName 获取显示名称(用于前端显示)
GetDisplayName() string
// Validate 验证数据有效性
Validate() error
}
// GetDisplayName 实现TaskItemData接口
func (data BatchTransferInputData) GetDisplayName() string {
return data.Title
}
// Validate 验证批量转存输入数据
func (data BatchTransferInputData) Validate() error {
if data.Title == "" {
return fmt.Errorf("标题不能为空")
}
if data.URL == "" {
return fmt.Errorf("链接不能为空")
}
// 这里可以添加URL格式验证
return nil
}
// GetDisplayName 实现TaskItemData接口
func (data BatchTransferOutputData) GetDisplayName() string {
return fmt.Sprintf("ResourceID: %d", data.ResourceID)
}
// Validate 验证批量转存输出数据
func (data BatchTransferOutputData) Validate() error {
if data.ResourceID == 0 {
return fmt.Errorf("资源ID不能为空")
}
return nil
}

45
db/entity/file.go Normal file
View File

@@ -0,0 +1,45 @@
package entity
import (
"time"
)
// File 文件实体
type File struct {
ID uint `json:"id" gorm:"primaryKey"`
CreatedAt time.Time `json:"created_at"`
UpdatedAt time.Time `json:"updated_at"`
// 文件信息
OriginalName string `json:"original_name" gorm:"size:255;not null;comment:原始文件名"`
FileName string `json:"file_name" gorm:"size:255;not null;unique;comment:存储文件名"`
FilePath string `json:"file_path" gorm:"size:500;not null;comment:文件路径"`
FileSize int64 `json:"file_size" gorm:"not null;comment:文件大小(字节)"`
FileType string `json:"file_type" gorm:"size:100;not null;comment:文件类型"`
MimeType string `json:"mime_type" gorm:"size:100;comment:MIME类型"`
FileHash string `json:"file_hash" gorm:"size:64;uniqueIndex;comment:文件哈希值"`
// 访问信息
AccessURL string `json:"access_url" gorm:"size:500;comment:访问URL"`
// 用户信息
UserID uint `json:"user_id" gorm:"comment:上传用户ID"`
User User `json:"user" gorm:"foreignKey:UserID"`
// 状态信息
Status string `json:"status" gorm:"size:20;default:'active';comment:文件状态"`
IsPublic bool `json:"is_public" gorm:"default:true;comment:是否公开"`
IsDeleted bool `json:"is_deleted" gorm:"default:false;comment:是否已删除"`
}
// TableName 指定表名
func (File) TableName() string {
return "files"
}
// FileStatus 文件状态常量
const (
FileStatusActive = "active" // 正常
FileStatusInactive = "inactive" // 禁用
FileStatusDeleted = "deleted" // 已删除
)

View File

@@ -8,26 +8,28 @@ import (
// Resource 资源模型
type Resource struct {
ID uint `json:"id" gorm:"primaryKey;autoIncrement"`
Title string `json:"title" gorm:"size:255;not null;comment:资源标题"`
Description string `json:"description" gorm:"type:text;comment:资源描述"`
URL string `json:"url" gorm:"size:128;comment:资源链接"`
PanID *uint `json:"pan_id" gorm:"comment:平台ID"`
SaveURL string `json:"save_url" gorm:"size:500;comment:转存后的链接"`
FileSize string `json:"file_size" gorm:"size:100;comment:文件大小"`
CategoryID *uint `json:"category_id" gorm:"comment:分类ID"`
ViewCount int `json:"view_count" gorm:"default:0;comment:浏览次数"`
IsValid bool `json:"is_valid" gorm:"default:true;comment:是否有效"`
IsPublic bool `json:"is_public" gorm:"default:true;comment:是否公开"`
CreatedAt time.Time `json:"created_at"`
UpdatedAt time.Time `json:"updated_at"`
DeletedAt gorm.DeletedAt `json:"deleted_at" gorm:"index"`
Cover string `json:"cover" gorm:"size:500;comment:封面"`
Author string `json:"author" gorm:"size:100;comment:作者"`
ErrorMsg string `json:"error_msg" gorm:"size:255;comment:转存失败原因"`
CkID *uint `json:"ck_id" gorm:"comment:账号ID"`
Fid string `json:"fid" gorm:"size:128;comment:网盘文件ID"`
Key string `json:"key" gorm:"size:64;index;comment:资源组标识相同key表示同一组资源"`
ID uint `json:"id" gorm:"primaryKey;autoIncrement"`
Title string `json:"title" gorm:"size:255;not null;comment:资源标题"`
Description string `json:"description" gorm:"type:text;comment:资源描述"`
URL string `json:"url" gorm:"size:128;comment:资源链接"`
PanID *uint `json:"pan_id" gorm:"comment:平台ID"`
SaveURL string `json:"save_url" gorm:"size:500;comment:转存后的链接"`
FileSize string `json:"file_size" gorm:"size:100;comment:文件大小"`
CategoryID *uint `json:"category_id" gorm:"comment:分类ID"`
ViewCount int `json:"view_count" gorm:"default:0;comment:浏览次数"`
IsValid bool `json:"is_valid" gorm:"default:true;comment:是否有效"`
IsPublic bool `json:"is_public" gorm:"default:true;comment:是否公开"`
CreatedAt time.Time `json:"created_at"`
UpdatedAt time.Time `json:"updated_at"`
DeletedAt gorm.DeletedAt `json:"deleted_at" gorm:"index"`
Cover string `json:"cover" gorm:"size:500;comment:封面"`
Author string `json:"author" gorm:"size:100;comment:作者"`
ErrorMsg string `json:"error_msg" gorm:"size:255;comment:转存失败原因"`
CkID *uint `json:"ck_id" gorm:"comment:账号ID"`
Fid string `json:"fid" gorm:"size:128;comment:网盘文件ID"`
Key string `json:"key" gorm:"size:64;index;comment:资源组标识相同key表示同一组资源"`
SyncedToMeilisearch bool `json:"synced_to_meilisearch" gorm:"default:false;comment:是否已同步到Meilisearch"`
SyncedAt *time.Time `json:"synced_at" gorm:"comment:同步时间"`
// 关联关系
Category Category `json:"category" gorm:"foreignKey:CategoryID"`

View File

@@ -0,0 +1,25 @@
package entity
import (
"time"
"gorm.io/gorm"
)
// ResourceView 资源访问记录
type ResourceView struct {
ID uint `json:"id" gorm:"primaryKey;autoIncrement"`
ResourceID uint `json:"resource_id" gorm:"not null;index;comment:资源ID"`
IPAddress string `json:"ip_address" gorm:"size:45;comment:访问者IP地址"`
UserAgent string `json:"user_agent" gorm:"type:text;comment:用户代理"`
CreatedAt time.Time `json:"created_at" gorm:"autoCreateTime;comment:访问时间"`
UpdatedAt time.Time `json:"updated_at" gorm:"autoUpdateTime"`
DeletedAt gorm.DeletedAt `json:"deleted_at" gorm:"index"`
// 关联关系
Resource Resource `json:"resource" gorm:"foreignKey:ResourceID"`
}
// TableName 指定表名
func (ResourceView) TableName() string {
return "resource_views"
}

View File

@@ -11,8 +11,8 @@ type SystemConfig struct {
UpdatedAt time.Time `json:"updated_at"`
// 键值对配置
Key string `json:"key" gorm:"size:100;not null;uniqueIndex"`
Value string `json:"value" gorm:"size:1000"`
Key string `json:"key" gorm:"size:100;not null;unique;comment:配置键"`
Value string `json:"value" gorm:"type:text"`
Type string `json:"type" gorm:"size:20;default:'string'"` // string, int, bool, json
}

View File

@@ -8,6 +8,7 @@ const (
ConfigKeyKeywords = "keywords"
ConfigKeyAuthor = "author"
ConfigKeyCopyright = "copyright"
ConfigKeySiteLogo = "site_logo"
// 自动处理配置组
ConfigKeyAutoProcessReadyResources = "auto_process_ready_resources"
@@ -20,9 +21,27 @@ const (
// API配置
ConfigKeyApiToken = "api_token"
// 违禁词配置
ConfigKeyForbiddenWords = "forbidden_words"
// 广告配置
ConfigKeyAdKeywords = "ad_keywords" // 广告关键词
ConfigKeyAutoInsertAd = "auto_insert_ad" // 自动插入广告
// 其他配置
ConfigKeyPageSize = "page_size"
ConfigKeyMaintenanceMode = "maintenance_mode"
ConfigKeyEnableRegister = "enable_register"
// 三方统计配置
ConfigKeyThirdPartyStatsCode = "third_party_stats_code"
// Meilisearch配置
ConfigKeyMeilisearchEnabled = "meilisearch_enabled"
ConfigKeyMeilisearchHost = "meilisearch_host"
ConfigKeyMeilisearchPort = "meilisearch_port"
ConfigKeyMeilisearchMasterKey = "meilisearch_master_key"
ConfigKeyMeilisearchIndexName = "meilisearch_index_name"
)
// ConfigType 配置类型常量
@@ -58,9 +77,27 @@ const (
// API配置字段
ConfigResponseFieldApiToken = "api_token"
// 违禁词配置字段
ConfigResponseFieldForbiddenWords = "forbidden_words"
// 广告配置字段
ConfigResponseFieldAdKeywords = "ad_keywords"
ConfigResponseFieldAutoInsertAd = "auto_insert_ad"
// 其他配置字段
ConfigResponseFieldPageSize = "page_size"
ConfigResponseFieldMaintenanceMode = "maintenance_mode"
ConfigResponseFieldEnableRegister = "enable_register"
// 三方统计配置字段
ConfigResponseFieldThirdPartyStatsCode = "third_party_stats_code"
// Meilisearch配置字段
ConfigResponseFieldMeilisearchEnabled = "meilisearch_enabled"
ConfigResponseFieldMeilisearchHost = "meilisearch_host"
ConfigResponseFieldMeilisearchPort = "meilisearch_port"
ConfigResponseFieldMeilisearchMasterKey = "meilisearch_master_key"
ConfigResponseFieldMeilisearchIndexName = "meilisearch_index_name"
)
// ConfigDefaultValue 配置默认值常量
@@ -83,7 +120,25 @@ const (
// API配置默认值
ConfigDefaultApiToken = ""
// 违禁词配置默认值
ConfigDefaultForbiddenWords = ""
// 广告配置默认值
ConfigDefaultAdKeywords = ""
ConfigDefaultAutoInsertAd = ""
// 其他配置默认值
ConfigDefaultPageSize = "100"
ConfigDefaultMaintenanceMode = "false"
ConfigDefaultEnableRegister = "true"
// 三方统计配置默认值
ConfigDefaultThirdPartyStatsCode = ""
// Meilisearch配置默认值
ConfigDefaultMeilisearchEnabled = "false"
ConfigDefaultMeilisearchHost = "localhost"
ConfigDefaultMeilisearchPort = "7700"
ConfigDefaultMeilisearchMasterKey = ""
ConfigDefaultMeilisearchIndexName = "resources"
)

62
db/entity/task.go Normal file
View File

@@ -0,0 +1,62 @@
package entity
import (
"time"
"gorm.io/gorm"
)
// TaskStatus 任务状态
type TaskStatus string
const (
TaskStatusPending TaskStatus = "pending" // 等待中
TaskStatusRunning TaskStatus = "running" // 运行中
TaskStatusPaused TaskStatus = "paused" // 已暂停
TaskStatusCompleted TaskStatus = "completed" // 已完成
TaskStatusFailed TaskStatus = "failed" // 失败
TaskStatusCancelled TaskStatus = "cancelled" // 已取消
)
// TaskType 任务类型
type TaskType string
const (
TaskTypeBatchTransfer TaskType = "batch_transfer" // 批量转存
)
// Task 任务表
type Task struct {
ID uint `json:"id" gorm:"primaryKey;autoIncrement"`
Title string `json:"title" gorm:"size:255;not null;comment:任务标题"`
Type TaskType `json:"type" gorm:"size:50;not null;comment:任务类型"`
Status TaskStatus `json:"status" gorm:"size:20;not null;default:pending;comment:任务状态"`
Description string `json:"description" gorm:"type:text;comment:任务描述"`
// 进度信息
TotalItems int `json:"total_items" gorm:"not null;default:0;comment:总项目数"`
ProcessedItems int `json:"processed_items" gorm:"not null;default:0;comment:已处理项目数"`
SuccessItems int `json:"success_items" gorm:"not null;default:0;comment:成功项目数"`
FailedItems int `json:"failed_items" gorm:"not null;default:0;comment:失败项目数"`
// 任务配置 (JSON格式存储)
Config string `json:"config" gorm:"type:text;comment:任务配置"`
// 任务消息
Message string `json:"message" gorm:"type:text;comment:任务消息"`
// 时间信息
StartedAt *time.Time `json:"started_at" gorm:"comment:开始时间"`
CompletedAt *time.Time `json:"completed_at" gorm:"comment:完成时间"`
CreatedAt time.Time `json:"created_at" gorm:"autoCreateTime"`
UpdatedAt time.Time `json:"updated_at" gorm:"autoUpdateTime"`
DeletedAt gorm.DeletedAt `json:"deleted_at" gorm:"index"`
// 关联关系
TaskItems []TaskItem `json:"task_items" gorm:"foreignKey:TaskID"`
}
// TableName 指定表名
func (Task) TableName() string {
return "tasks"
}

51
db/entity/task_item.go Normal file
View File

@@ -0,0 +1,51 @@
package entity
import (
"time"
"gorm.io/gorm"
)
// TaskItemStatus 任务项状态
type TaskItemStatus string
const (
TaskItemStatusPending TaskItemStatus = "pending" // 等待处理
TaskItemStatusProcessing TaskItemStatus = "processing" // 处理中
TaskItemStatusSuccess TaskItemStatus = "success" // 成功
TaskItemStatusFailed TaskItemStatus = "failed" // 失败
TaskItemStatusSkipped TaskItemStatus = "skipped" // 跳过
)
// TaskItem 任务项表(任务的详细记录)
type TaskItem struct {
ID uint `json:"id" gorm:"primaryKey;autoIncrement"`
TaskID uint `json:"task_id" gorm:"not null;index;comment:任务ID"`
// 通用任务项信息
Status TaskItemStatus `json:"status" gorm:"size:20;not null;default:pending;comment:处理状态"`
ErrorMessage string `json:"error_message" gorm:"type:text;comment:错误信息"`
// 输入数据 (JSON格式存储支持不同任务类型的不同数据结构)
InputData string `json:"input_data" gorm:"type:text;not null;comment:输入数据(JSON格式)"`
// 输出数据 (JSON格式存储支持不同任务类型的不同结果数据)
OutputData string `json:"output_data" gorm:"type:text;comment:输出数据(JSON格式)"`
// 处理日志 (可选,用于记录详细的处理过程)
ProcessLog string `json:"process_log" gorm:"type:text;comment:处理日志"`
// 时间信息
ProcessedAt *time.Time `json:"processed_at" gorm:"comment:处理时间"`
CreatedAt time.Time `json:"created_at" gorm:"autoCreateTime"`
UpdatedAt time.Time `json:"updated_at" gorm:"autoUpdateTime"`
DeletedAt gorm.DeletedAt `json:"deleted_at" gorm:"index"`
// 关联关系
Task Task `json:"task" gorm:"foreignKey:TaskID"`
}
// TableName 指定表名
func (TaskItem) TableName() string {
return "task_items"
}

View File

@@ -0,0 +1,104 @@
package entity
import (
"encoding/json"
"fmt"
"github.com/ctwj/urldb/db/dto"
)
// SetInputData 设置输入数据将结构体转换为JSON字符串
func (item *TaskItem) SetInputData(data interface{}) error {
jsonData, err := json.Marshal(data)
if err != nil {
return fmt.Errorf("序列化输入数据失败: %v", err)
}
item.InputData = string(jsonData)
return nil
}
// GetInputData 获取输入数据根据任务类型解析JSON
func (item *TaskItem) GetInputData(taskType TaskType) (interface{}, error) {
if item.InputData == "" {
return nil, fmt.Errorf("输入数据为空")
}
switch taskType {
case TaskTypeBatchTransfer:
var data dto.BatchTransferInputData
err := json.Unmarshal([]byte(item.InputData), &data)
if err != nil {
return nil, fmt.Errorf("解析批量转存输入数据失败: %v", err)
}
return data, nil
default:
// 对于未知任务类型返回原始JSON数据
var data map[string]interface{}
err := json.Unmarshal([]byte(item.InputData), &data)
if err != nil {
return nil, fmt.Errorf("解析输入数据失败: %v", err)
}
return data, nil
}
}
// SetOutputData 设置输出数据将结构体转换为JSON字符串
func (item *TaskItem) SetOutputData(data interface{}) error {
jsonData, err := json.Marshal(data)
if err != nil {
return fmt.Errorf("序列化输出数据失败: %v", err)
}
item.OutputData = string(jsonData)
return nil
}
// GetOutputData 获取输出数据根据任务类型解析JSON
func (item *TaskItem) GetOutputData(taskType TaskType) (interface{}, error) {
if item.OutputData == "" {
return nil, fmt.Errorf("输出数据为空")
}
switch taskType {
case TaskTypeBatchTransfer:
var data dto.BatchTransferOutputData
err := json.Unmarshal([]byte(item.OutputData), &data)
if err != nil {
return nil, fmt.Errorf("解析批量转存输出数据失败: %v", err)
}
return data, nil
default:
// 对于未知任务类型返回原始JSON数据
var data map[string]interface{}
err := json.Unmarshal([]byte(item.OutputData), &data)
if err != nil {
return nil, fmt.Errorf("解析输出数据失败: %v", err)
}
return data, nil
}
}
// GetDisplayName 获取显示名称(用于前端显示)
func (item *TaskItem) GetDisplayName(taskType TaskType) string {
inputData, err := item.GetInputData(taskType)
if err != nil {
return fmt.Sprintf("TaskItem#%d", item.ID)
}
switch taskType {
case TaskTypeBatchTransfer:
if data, ok := inputData.(dto.BatchTransferInputData); ok {
return data.Title
}
}
return fmt.Sprintf("TaskItem#%d", item.ID)
}
// AddProcessLog 添加处理日志
func (item *TaskItem) AddProcessLog(message string) {
if item.ProcessLog == "" {
item.ProcessLog = message
} else {
item.ProcessLog += "\n" + message
}
}

1302
db/forbidden.txt Normal file

File diff suppressed because it is too large Load Diff

View File

@@ -10,6 +10,7 @@ import (
type CategoryRepository interface {
BaseRepository[entity.Category]
FindByName(name string) (*entity.Category, error)
FindByNameIncludingDeleted(name string) (*entity.Category, error)
FindWithResources() ([]entity.Category, error)
FindWithTags() ([]entity.Category, error)
GetResourceCount(categoryID uint) (int64, error)
@@ -17,6 +18,7 @@ type CategoryRepository interface {
GetTagNames(categoryID uint) ([]string, error)
FindWithPagination(page, pageSize int) ([]entity.Category, int64, error)
Search(query string, page, pageSize int) ([]entity.Category, int64, error)
RestoreDeletedCategory(id uint) error
}
// CategoryRepositoryImpl Category的Repository实现
@@ -41,6 +43,21 @@ func (r *CategoryRepositoryImpl) FindByName(name string) (*entity.Category, erro
return &category, nil
}
// FindByNameIncludingDeleted 根据名称查找(包括已删除的记录)
func (r *CategoryRepositoryImpl) FindByNameIncludingDeleted(name string) (*entity.Category, error) {
var category entity.Category
err := r.db.Unscoped().Where("name = ?", name).First(&category).Error
if err != nil {
return nil, err
}
return &category, nil
}
// RestoreDeletedCategory 恢复已删除的分类
func (r *CategoryRepositoryImpl) RestoreDeletedCategory(id uint) error {
return r.db.Unscoped().Model(&entity.Category{}).Where("id = ?", id).Update("deleted_at", nil).Error
}
// FindWithResources 查找包含资源的分类
func (r *CategoryRepositoryImpl) FindWithResources() ([]entity.Category, error) {
var categories []entity.Category

167
db/repo/file_repository.go Normal file
View File

@@ -0,0 +1,167 @@
package repo
import (
"github.com/ctwj/urldb/db/entity"
"github.com/ctwj/urldb/utils"
"gorm.io/gorm"
)
// FileRepository 文件Repository接口
type FileRepository interface {
BaseRepository[entity.File]
FindByFileName(fileName string) (*entity.File, error)
FindByHash(fileHash string) (*entity.File, error)
FindByUserID(userID uint, page, pageSize int) ([]entity.File, int64, error)
FindPublicFiles(page, pageSize int) ([]entity.File, int64, error)
SearchFiles(search string, fileType, status string, userID uint, page, pageSize int) ([]entity.File, int64, error)
SoftDeleteByIDs(ids []uint) error
UpdateFileStatus(id uint, status string) error
UpdateFilePublic(id uint, isPublic bool) error
}
// FileRepositoryImpl 文件Repository实现
type FileRepositoryImpl struct {
BaseRepositoryImpl[entity.File]
}
// NewFileRepository 创建文件Repository
func NewFileRepository(db *gorm.DB) FileRepository {
return &FileRepositoryImpl{
BaseRepositoryImpl: BaseRepositoryImpl[entity.File]{db: db},
}
}
// FindByFileName 根据文件名查找文件
func (r *FileRepositoryImpl) FindByFileName(fileName string) (*entity.File, error) {
var file entity.File
err := r.db.Where("file_name = ? AND is_deleted = ?", fileName, false).First(&file).Error
if err != nil {
return nil, err
}
return &file, nil
}
// FindByUserID 根据用户ID查找文件
func (r *FileRepositoryImpl) FindByUserID(userID uint, page, pageSize int) ([]entity.File, int64, error) {
var files []entity.File
var total int64
offset := (page - 1) * pageSize
// 获取总数
err := r.db.Model(&entity.File{}).Where("user_id = ? AND is_deleted = ?", userID, false).Count(&total).Error
if err != nil {
return nil, 0, err
}
// 获取文件列表
err = r.db.Where("user_id = ? AND is_deleted = ?", userID, false).
Preload("User").
Order("created_at DESC").
Offset(offset).
Limit(pageSize).
Find(&files).Error
return files, total, err
}
// FindPublicFiles 查找公开文件
func (r *FileRepositoryImpl) FindPublicFiles(page, pageSize int) ([]entity.File, int64, error) {
var files []entity.File
var total int64
offset := (page - 1) * pageSize
// 获取总数
err := r.db.Model(&entity.File{}).Where("is_public = ? AND is_deleted = ? AND status = ?", true, false, entity.FileStatusActive).Count(&total).Error
if err != nil {
return nil, 0, err
}
// 获取文件列表
err = r.db.Where("is_public = ? AND is_deleted = ? AND status = ?", true, false, entity.FileStatusActive).
Preload("User").
Order("created_at DESC").
Offset(offset).
Limit(pageSize).
Find(&files).Error
return files, total, err
}
// SearchFiles 搜索文件
func (r *FileRepositoryImpl) SearchFiles(search string, fileType, status string, userID uint, page, pageSize int) ([]entity.File, int64, error) {
var files []entity.File
var total int64
offset := (page - 1) * pageSize
query := r.db.Model(&entity.File{}).Where("is_deleted = ?", false)
// 添加调试日志
utils.Info("搜索文件参数: search='%s', fileType='%s', status='%s', userID=%d, page=%d, pageSize=%d",
search, fileType, status, userID, page, pageSize)
// 添加搜索条件
if search != "" {
query = query.Where("original_name LIKE ?", "%"+search+"%")
utils.Info("添加搜索条件: file_name LIKE '%%%s%%'", search)
}
if fileType != "" {
query = query.Where("file_type = ?", fileType)
}
if status != "" {
query = query.Where("status = ?", status)
}
if userID > 0 {
query = query.Where("user_id = ?", userID)
}
// 获取总数
err := query.Count(&total).Error
if err != nil {
return nil, 0, err
}
// 获取文件列表
err = query.Preload("User").
Order("created_at DESC").
Offset(offset).
Limit(pageSize).
Find(&files).Error
// 添加调试日志
utils.Info("搜索结果: 总数=%d, 当前页文件数=%d", total, len(files))
if len(files) > 0 {
utils.Info("第一个文件: ID=%d, 文件名='%s'", files[0].ID, files[0].OriginalName)
}
return files, total, err
}
// SoftDeleteByIDs 软删除文件
func (r *FileRepositoryImpl) SoftDeleteByIDs(ids []uint) error {
return r.db.Model(&entity.File{}).Where("id IN ?", ids).Update("is_deleted", true).Error
}
// UpdateFileStatus 更新文件状态
func (r *FileRepositoryImpl) UpdateFileStatus(id uint, status string) error {
return r.db.Model(&entity.File{}).Where("id = ?", id).Update("status", status).Error
}
// UpdateFilePublic 更新文件公开状态
func (r *FileRepositoryImpl) UpdateFilePublic(id uint, isPublic bool) error {
return r.db.Model(&entity.File{}).Where("id = ?", id).Update("is_public", isPublic).Error
}
// FindByHash 根据文件哈希查找文件
func (r *FileRepositoryImpl) FindByHash(fileHash string) (*entity.File, error) {
var file entity.File
err := r.db.Where("file_hash = ? AND is_deleted = ?", fileHash, false).First(&file).Error
if err != nil {
return nil, err
}
return &file, nil
}

View File

@@ -16,6 +16,10 @@ type RepositoryManager struct {
SearchStatRepository SearchStatRepository
SystemConfigRepository SystemConfigRepository
HotDramaRepository HotDramaRepository
ResourceViewRepository ResourceViewRepository
TaskRepository TaskRepository
TaskItemRepository TaskItemRepository
FileRepository FileRepository
}
// NewRepositoryManager 创建Repository管理器
@@ -31,5 +35,9 @@ func NewRepositoryManager(db *gorm.DB) *RepositoryManager {
SearchStatRepository: NewSearchStatRepository(db),
SystemConfigRepository: NewSystemConfigRepository(db),
HotDramaRepository: NewHotDramaRepository(db),
ResourceViewRepository: NewResourceViewRepository(db),
TaskRepository: NewTaskRepository(db),
TaskItemRepository: NewTaskItemRepository(db),
FileRepository: NewFileRepository(db),
}
}

View File

@@ -22,8 +22,14 @@ type ReadyResourceRepository interface {
BatchFindByURLs(urls []string) ([]entity.ReadyResource, error)
GenerateUniqueKey() (string, error)
FindWithErrors() ([]entity.ReadyResource, error)
FindWithErrorsPaginated(page, limit int) ([]entity.ReadyResource, int64, error)
FindWithErrorsIncludingDeleted() ([]entity.ReadyResource, error)
FindWithErrorsPaginatedIncludingDeleted(page, limit int, errorFilter string) ([]entity.ReadyResource, int64, error)
FindWithErrorsByQuery(errorFilter string) ([]entity.ReadyResource, error)
FindWithoutErrors() ([]entity.ReadyResource, error)
ClearErrorMsg(id uint) error
ClearErrorMsgAndRestore(id uint) error
ClearAllErrorsByQuery(errorFilter string) (int64, error) // 批量清除错误信息并真正删除资源
}
// ReadyResourceRepositoryImpl ReadyResource的Repository实现
@@ -117,13 +123,31 @@ func (r *ReadyResourceRepositoryImpl) GenerateUniqueKey() (string, error) {
return "", gorm.ErrInvalidData
}
// FindWithErrors 查找有错误信息的资源
// FindWithErrors 查找有错误信息的资源(包括软删除的)
func (r *ReadyResourceRepositoryImpl) FindWithErrors() ([]entity.ReadyResource, error) {
var resources []entity.ReadyResource
err := r.db.Where("error_msg != '' AND error_msg IS NOT NULL").Find(&resources).Error
err := r.db.Unscoped().Where("error_msg != '' AND error_msg IS NOT NULL").Find(&resources).Error
return resources, err
}
// FindWithErrorsPaginated 分页查找有错误信息的资源(包括软删除的)
func (r *ReadyResourceRepositoryImpl) FindWithErrorsPaginated(page, limit int) ([]entity.ReadyResource, int64, error) {
var resources []entity.ReadyResource
var total int64
offset := (page - 1) * limit
db := r.db.Model(&entity.ReadyResource{}).Unscoped().Where("error_msg != '' AND error_msg IS NOT NULL")
// 获取总数
if err := db.Count(&total).Error; err != nil {
return nil, 0, err
}
// 获取分页数据
err := db.Offset(offset).Limit(limit).Order("created_at DESC").Find(&resources).Error
return resources, total, err
}
// FindWithoutErrors 查找没有错误信息的资源
func (r *ReadyResourceRepositoryImpl) FindWithoutErrors() ([]entity.ReadyResource, error) {
var resources []entity.ReadyResource
@@ -131,7 +155,75 @@ func (r *ReadyResourceRepositoryImpl) FindWithoutErrors() ([]entity.ReadyResourc
return resources, err
}
// FindWithErrorsIncludingDeleted 查找有错误信息的资源(包括软删除的,用于管理页面)
func (r *ReadyResourceRepositoryImpl) FindWithErrorsIncludingDeleted() ([]entity.ReadyResource, error) {
var resources []entity.ReadyResource
err := r.db.Unscoped().Where("error_msg != '' AND error_msg IS NOT NULL").Find(&resources).Error
return resources, err
}
// FindWithErrorsPaginatedIncludingDeleted 分页查找有错误信息的资源(包括软删除的,用于管理页面)
func (r *ReadyResourceRepositoryImpl) FindWithErrorsPaginatedIncludingDeleted(page, limit int, errorFilter string) ([]entity.ReadyResource, int64, error) {
var resources []entity.ReadyResource
var total int64
offset := (page - 1) * limit
db := r.db.Model(&entity.ReadyResource{}).Unscoped().Where("error_msg != '' AND error_msg IS NOT NULL")
// 如果有错误过滤条件,添加到查询中
if errorFilter != "" {
db = db.Where("error_msg ILIKE ?", "%"+errorFilter+"%")
}
// 获取总数
if err := db.Count(&total).Error; err != nil {
return nil, 0, err
}
// 获取分页数据
err := db.Offset(offset).Limit(limit).Order("created_at DESC").Find(&resources).Error
return resources, total, err
}
// ClearErrorMsg 清除指定资源的错误信息
func (r *ReadyResourceRepositoryImpl) ClearErrorMsg(id uint) error {
return r.db.Model(&entity.ReadyResource{}).Where("id = ?", id).Update("error_msg", "").Error
return r.db.Model(&entity.ReadyResource{}).Where("id = ?", id).Update("error_msg", "").Update("deleted_at", nil).Error
}
// ClearErrorMsgAndRestore 清除错误信息并恢复软删除的资源
func (r *ReadyResourceRepositoryImpl) ClearErrorMsgAndRestore(id uint) error {
return r.db.Unscoped().Model(&entity.ReadyResource{}).Where("id = ?", id).Updates(map[string]interface{}{
"error_msg": "",
"deleted_at": nil,
}).Error
}
// FindWithErrorsByQuery 根据查询条件查找有错误信息的资源(不分页,用于批量操作)
func (r *ReadyResourceRepositoryImpl) FindWithErrorsByQuery(errorFilter string) ([]entity.ReadyResource, error) {
var resources []entity.ReadyResource
db := r.db.Model(&entity.ReadyResource{}).Unscoped().Where("error_msg != '' AND error_msg IS NOT NULL")
// 如果有错误过滤条件,添加到查询中
if errorFilter != "" {
db = db.Where("error_msg ILIKE ?", "%"+errorFilter+"%")
}
err := db.Order("created_at DESC").Find(&resources).Error
return resources, err
}
// ClearAllErrorsByQuery 根据查询条件批量清除错误信息并真正删除资源
func (r *ReadyResourceRepositoryImpl) ClearAllErrorsByQuery(errorFilter string) (int64, error) {
db := r.db.Unscoped().Model(&entity.ReadyResource{}).Where("error_msg != '' AND error_msg IS NOT NULL")
// 如果有错误过滤条件,添加到查询中
if errorFilter != "" {
db = db.Where("error_msg ILIKE ?", "%"+errorFilter+"%")
}
// 真正删除资源(物理删除)
result := db.Delete(&entity.ReadyResource{})
return result.RowsAffected, result.Error
}

View File

@@ -2,11 +2,9 @@ package repo
import (
"fmt"
"time"
"github.com/ctwj/urldb/db/entity"
"gorm.io/gorm"
)
@@ -32,8 +30,18 @@ type ResourceRepository interface {
InvalidateCache() error
FindExists(url string, excludeID ...uint) (bool, error)
BatchFindByURLs(urls []string) ([]entity.Resource, error)
GetResourcesForTransfer(panID uint, sinceTime time.Time) ([]*entity.Resource, error)
CreateResourceTag(resourceID, tagID uint) error
GetResourcesForTransfer(panID uint, sinceTime time.Time, limit int) ([]*entity.Resource, error)
GetByURL(url string) (*entity.Resource, error)
UpdateSaveURL(id uint, saveURL string) error
CreateResourceTag(resourceTag *entity.ResourceTag) error
FindByIDs(ids []uint) ([]entity.Resource, error)
FindUnsyncedToMeilisearch(page, limit int) ([]entity.Resource, int64, error)
FindSyncedToMeilisearch(page, limit int) ([]entity.Resource, int64, error)
CountUnsyncedToMeilisearch() (int64, error)
CountSyncedToMeilisearch() (int64, error)
MarkAsSyncedToMeilisearch(ids []uint) error
MarkAllAsUnsyncedToMeilisearch() error
FindAllWithPagination(page, limit int) ([]entity.Resource, int64, error)
}
// ResourceRepositoryImpl Resource的Repository实现
@@ -212,18 +220,43 @@ func (r *ResourceRepositoryImpl) SearchWithFilters(params map[string]interface{}
var resources []entity.Resource
var total int64
db := r.db.Model(&entity.Resource{})
db := r.db.Model(&entity.Resource{}).Preload("Category").Preload("Pan").Preload("Tags")
// 处理参数
for key, value := range params {
switch key {
case "query":
case "search": // 添加search参数支持
if query, ok := value.(string); ok && query != "" {
db = db.Where("title ILIKE ? OR description ILIKE ?", "%"+query+"%", "%"+query+"%")
}
case "category_id":
case "category_id": // 添加category_id参数支持
if categoryID, ok := value.(uint); ok {
fmt.Printf("应用分类筛选: category_id = %d\n", categoryID)
db = db.Where("category_id = ?", categoryID)
} else {
fmt.Printf("分类ID类型错误: %T, value: %v\n", value, value)
}
case "category": // 添加category参数支持字符串形式
if category, ok := value.(string); ok && category != "" {
// 根据分类名称查找分类ID
var categoryEntity entity.Category
if err := r.db.Where("name ILIKE ?", "%"+category+"%").First(&categoryEntity).Error; err == nil {
db = db.Where("category_id = ?", categoryEntity.ID)
}
}
case "tag": // 添加tag参数支持
if tag, ok := value.(string); ok && tag != "" {
// 根据标签名称查找相关资源
var tagEntity entity.Tag
if err := r.db.Where("name ILIKE ?", "%"+tag+"%").First(&tagEntity).Error; err == nil {
// 通过中间表查找包含该标签的资源
db = db.Joins("JOIN resource_tags ON resources.id = resource_tags.resource_id").
Where("resource_tags.tag_id = ?", tagEntity.ID)
}
}
case "pan_id": // 添加pan_id参数支持
if panID, ok := value.(uint); ok {
db = db.Where("pan_id = ?", panID)
}
case "is_valid":
if isValid, ok := value.(bool); ok {
@@ -233,20 +266,76 @@ func (r *ResourceRepositoryImpl) SearchWithFilters(params map[string]interface{}
if isPublic, ok := value.(bool); ok {
db = db.Where("is_public = ?", isPublic)
}
case "pan_id":
if panID, ok := value.(uint); ok {
db = db.Where("pan_id = ?", panID)
case "has_save_url": // 添加has_save_url参数支持
if hasSaveURL, ok := value.(bool); ok {
fmt.Printf("处理 has_save_url 参数: %v\n", hasSaveURL)
if hasSaveURL {
// 有转存链接save_url不为空且不为空格
db = db.Where("save_url IS NOT NULL AND save_url != '' AND TRIM(save_url) != ''")
fmt.Printf("应用 has_save_url=true 条件: save_url IS NOT NULL AND save_url != '' AND TRIM(save_url) != ''\n")
} else {
// 没有转存链接save_url为空、NULL或只有空格
db = db.Where("(save_url IS NULL OR save_url = '' OR TRIM(save_url) = '')")
fmt.Printf("应用 has_save_url=false 条件: (save_url IS NULL OR save_url = '' OR TRIM(save_url) = '')\n")
}
}
case "no_save_url": // 添加no_save_url参数支持与has_save_url=false相同
if noSaveURL, ok := value.(bool); ok && noSaveURL {
db = db.Where("(save_url IS NULL OR save_url = '' OR TRIM(save_url) = '')")
}
case "pan_name": // 添加pan_name参数支持
if panName, ok := value.(string); ok && panName != "" {
// 根据平台名称查找平台ID
var panEntity entity.Pan
if err := r.db.Where("name ILIKE ?", "%"+panName+"%").First(&panEntity).Error; err == nil {
db = db.Where("pan_id = ?", panEntity.ID)
}
}
}
}
// 管理后台显示所有资源公开API才限制为有效的公开资源
// 这里通过检查请求来源来判断是否为管理后台
// 如果没有明确指定is_valid和is_public则显示所有资源
// 注意:这个逻辑可能需要根据实际需求调整
if _, hasIsValid := params["is_valid"]; !hasIsValid {
// 管理后台不限制is_valid
// db = db.Where("is_valid = ?", true)
}
if _, hasIsPublic := params["is_public"]; !hasIsPublic {
// 管理后台不限制is_public
// db = db.Where("is_public = ?", true)
}
// 获取总数
if err := db.Count(&total).Error; err != nil {
return nil, 0, err
}
// 处理分页参数
page := 1
pageSize := 20
if pageVal, ok := params["page"].(int); ok && pageVal > 0 {
page = pageVal
}
if pageSizeVal, ok := params["page_size"].(int); ok && pageSizeVal > 0 {
pageSize = pageSizeVal
fmt.Printf("原始pageSize: %d\n", pageSize)
// 限制最大page_size为10000管理后台需要更大的数据量
if pageSize > 10000 {
pageSize = 10000
fmt.Printf("pageSize超过10000限制为: %d\n", pageSize)
}
fmt.Printf("最终pageSize: %d\n", pageSize)
}
// 计算偏移量
offset := (page - 1) * pageSize
// 获取分页数据,按更新时间倒序
err := db.Order("updated_at DESC").Find(&resources).Error
err := db.Order("updated_at DESC").Offset(offset).Limit(pageSize).Find(&resources).Error
fmt.Printf("查询结果: 总数=%d, 当前页数据量=%d, pageSize=%d\n", total, len(resources), pageSize)
return resources, total, err
}
@@ -358,12 +447,18 @@ func (r *ResourceRepositoryImpl) BatchFindByURLs(urls []string) ([]entity.Resour
}
// GetResourcesForTransfer 获取需要转存的资源
func (r *ResourceRepositoryImpl) GetResourcesForTransfer(panID uint, sinceTime time.Time) ([]*entity.Resource, error) {
func (r *ResourceRepositoryImpl) GetResourcesForTransfer(panID uint, sinceTime time.Time, limit int) ([]*entity.Resource, error) {
var resources []*entity.Resource
query := r.db.Where("pan_id = ? AND (save_url = '' OR save_url IS NULL) AND (error_msg = '' OR error_msg IS NULL)", panID)
if !sinceTime.IsZero() {
query = query.Where("created_at >= ?", sinceTime)
}
// 添加数量限制
if limit > 0 {
query = query.Limit(limit)
}
err := query.Order("created_at DESC").Find(&resources).Error
if err != nil {
return nil, err
@@ -371,11 +466,147 @@ func (r *ResourceRepositoryImpl) GetResourcesForTransfer(panID uint, sinceTime t
return resources, nil
}
// CreateResourceTag 创建资源与标签的关联
func (r *ResourceRepositoryImpl) CreateResourceTag(resourceID, tagID uint) error {
resourceTag := &entity.ResourceTag{
ResourceID: resourceID,
TagID: tagID,
// GetByURL 根据URL获取资源
func (r *ResourceRepositoryImpl) GetByURL(url string) (*entity.Resource, error) {
var resource entity.Resource
err := r.db.Where("url = ?", url).First(&resource).Error
if err != nil {
return nil, err
}
return r.GetDB().Create(resourceTag).Error
return &resource, nil
}
// FindByIDs 根据ID列表查找资源
func (r *ResourceRepositoryImpl) FindByIDs(ids []uint) ([]entity.Resource, error) {
if len(ids) == 0 {
return []entity.Resource{}, nil
}
var resources []entity.Resource
err := r.db.Where("id IN ?", ids).Preload("Category").Preload("Pan").Preload("Tags").Find(&resources).Error
return resources, err
}
// UpdateSaveURL 更新保存URL
func (r *ResourceRepositoryImpl) UpdateSaveURL(id uint, saveURL string) error {
return r.db.Model(&entity.Resource{}).Where("id = ?", id).Update("save_url", saveURL).Error
}
// CreateResourceTag 创建资源与标签的关联
func (r *ResourceRepositoryImpl) CreateResourceTag(resourceTag *entity.ResourceTag) error {
return r.db.Create(resourceTag).Error
}
// FindUnsyncedToMeilisearch 查找未同步到Meilisearch的资源
func (r *ResourceRepositoryImpl) FindUnsyncedToMeilisearch(page, limit int) ([]entity.Resource, int64, error) {
var resources []entity.Resource
var total int64
offset := (page - 1) * limit
// 查询未同步的资源
db := r.db.Model(&entity.Resource{}).
Where("synced_to_meilisearch = ?", false).
Preload("Category").
Preload("Pan").
Order("updated_at DESC")
// 获取总数
if err := db.Count(&total).Error; err != nil {
return nil, 0, err
}
// 获取分页数据
err := db.Offset(offset).Limit(limit).Find(&resources).Error
return resources, total, err
}
// CountUnsyncedToMeilisearch 统计未同步到Meilisearch的资源数量
func (r *ResourceRepositoryImpl) CountUnsyncedToMeilisearch() (int64, error) {
var count int64
err := r.db.Model(&entity.Resource{}).
Where("synced_to_meilisearch = ?", false).
Count(&count).Error
return count, err
}
// MarkAsSyncedToMeilisearch 标记资源为已同步到Meilisearch
func (r *ResourceRepositoryImpl) MarkAsSyncedToMeilisearch(ids []uint) error {
if len(ids) == 0 {
return nil
}
now := time.Now()
return r.db.Model(&entity.Resource{}).
Where("id IN ?", ids).
Updates(map[string]interface{}{
"synced_to_meilisearch": true,
"synced_at": now,
}).Error
}
// MarkAllAsUnsyncedToMeilisearch 标记所有资源为未同步到Meilisearch
func (r *ResourceRepositoryImpl) MarkAllAsUnsyncedToMeilisearch() error {
return r.db.Model(&entity.Resource{}).
Where("1 = 1"). // 添加WHERE条件以更新所有记录
Updates(map[string]interface{}{
"synced_to_meilisearch": false,
"synced_at": nil,
}).Error
}
// FindSyncedToMeilisearch 查找已同步到Meilisearch的资源
func (r *ResourceRepositoryImpl) FindSyncedToMeilisearch(page, limit int) ([]entity.Resource, int64, error) {
var resources []entity.Resource
var total int64
offset := (page - 1) * limit
// 查询已同步的资源
db := r.db.Model(&entity.Resource{}).
Where("synced_to_meilisearch = ?", true).
Preload("Category").
Preload("Pan").
Order("updated_at DESC")
// 获取总数
if err := db.Count(&total).Error; err != nil {
return nil, 0, err
}
// 获取分页数据
err := db.Offset(offset).Limit(limit).Find(&resources).Error
return resources, total, err
}
// CountSyncedToMeilisearch 统计已同步到Meilisearch的资源数量
func (r *ResourceRepositoryImpl) CountSyncedToMeilisearch() (int64, error) {
var count int64
err := r.db.Model(&entity.Resource{}).
Where("synced_to_meilisearch = ?", true).
Count(&count).Error
return count, err
}
// FindAllWithPagination 分页查找所有资源
func (r *ResourceRepositoryImpl) FindAllWithPagination(page, limit int) ([]entity.Resource, int64, error) {
var resources []entity.Resource
var total int64
offset := (page - 1) * limit
// 查询所有资源
db := r.db.Model(&entity.Resource{}).
Preload("Category").
Preload("Pan").
Order("updated_at DESC")
// 获取总数
if err := db.Count(&total).Error; err != nil {
return nil, 0, err
}
// 获取分页数据
err := db.Offset(offset).Limit(limit).Find(&resources).Error
return resources, total, err
}

View File

@@ -0,0 +1,90 @@
package repo
import (
"github.com/ctwj/urldb/db/entity"
"github.com/ctwj/urldb/utils"
"gorm.io/gorm"
)
// ResourceViewRepository 资源访问记录仓库接口
type ResourceViewRepository interface {
BaseRepository[entity.ResourceView]
RecordView(resourceID uint, ipAddress, userAgent string) error
GetTodayViews() (int64, error)
GetViewsByDate(date string) (int64, error)
GetViewsTrend(days int) ([]map[string]interface{}, error)
GetResourceViews(resourceID uint, limit int) ([]entity.ResourceView, error)
}
// ResourceViewRepositoryImpl 资源访问记录仓库实现
type ResourceViewRepositoryImpl struct {
BaseRepositoryImpl[entity.ResourceView]
}
// NewResourceViewRepository 创建资源访问记录仓库
func NewResourceViewRepository(db *gorm.DB) ResourceViewRepository {
return &ResourceViewRepositoryImpl{
BaseRepositoryImpl: BaseRepositoryImpl[entity.ResourceView]{db: db},
}
}
// RecordView 记录资源访问
func (r *ResourceViewRepositoryImpl) RecordView(resourceID uint, ipAddress, userAgent string) error {
view := &entity.ResourceView{
ResourceID: resourceID,
IPAddress: ipAddress,
UserAgent: userAgent,
}
return r.db.Create(view).Error
}
// GetTodayViews 获取今日访问量
func (r *ResourceViewRepositoryImpl) GetTodayViews() (int64, error) {
today := utils.GetTodayString()
var count int64
err := r.db.Model(&entity.ResourceView{}).
Where("DATE(created_at) = ?", today).
Count(&count).Error
return count, err
}
// GetViewsByDate 获取指定日期的访问量
func (r *ResourceViewRepositoryImpl) GetViewsByDate(date string) (int64, error) {
var count int64
err := r.db.Model(&entity.ResourceView{}).
Where("DATE(created_at) = ?", date).
Count(&count).Error
return count, err
}
// GetViewsTrend 获取访问量趋势数据
func (r *ResourceViewRepositoryImpl) GetViewsTrend(days int) ([]map[string]interface{}, error) {
var results []map[string]interface{}
for i := days - 1; i >= 0; i-- {
date := utils.GetCurrentTime().AddDate(0, 0, -i)
dateStr := date.Format(utils.TimeFormatDate)
count, err := r.GetViewsByDate(dateStr)
if err != nil {
return nil, err
}
results = append(results, map[string]interface{}{
"date": dateStr,
"views": count,
})
}
return results, nil
}
// GetResourceViews 获取指定资源的访问记录
func (r *ResourceViewRepositoryImpl) GetResourceViews(resourceID uint, limit int) ([]entity.ResourceView, error) {
var views []entity.ResourceView
err := r.db.Where("resource_id = ?", resourceID).
Order("created_at DESC").
Limit(limit).
Find(&views).Error
return views, err
}

View File

@@ -2,9 +2,9 @@ package repo
import (
"fmt"
"time"
"github.com/ctwj/urldb/db/entity"
"github.com/ctwj/urldb/utils"
"gorm.io/gorm"
)
@@ -18,6 +18,7 @@ type SearchStatRepository interface {
GetSearchTrend(days int) ([]entity.DailySearchStat, error)
GetKeywordTrend(keyword string, days int) ([]entity.DailySearchStat, error)
GetSummary() (map[string]int64, error)
FindWithPaginationOrdered(page, limit int) ([]entity.SearchStat, int64, error)
}
// SearchStatRepositoryImpl 搜索统计Repository实现
@@ -37,7 +38,7 @@ func (r *SearchStatRepositoryImpl) RecordSearch(keyword, ip, userAgent string) e
stat := entity.SearchStat{
Keyword: keyword,
Count: 1,
Date: time.Now(), // 可保留 date 字段,实际用 created_at 统计
Date: utils.GetCurrentTime(), // 可保留 date 字段,实际用 created_at 统计
IP: ip,
UserAgent: userAgent,
}
@@ -124,9 +125,9 @@ func (r *SearchStatRepositoryImpl) GetKeywordTrend(keyword string, days int) ([]
// GetSummary 获取搜索统计汇总
func (r *SearchStatRepositoryImpl) GetSummary() (map[string]int64, error) {
var total, today, week, month, keywords int64
now := time.Now()
todayStr := now.Format("2006-01-02")
weekStart := now.AddDate(0, 0, -int(now.Weekday())+1).Format("2006-01-02") // 周一
now := utils.GetCurrentTime()
todayStr := now.Format(utils.TimeFormatDate)
weekStart := now.AddDate(0, 0, -int(now.Weekday())+1).Format(utils.TimeFormatDate) // 周一
monthStart := now.Format("2006-01") + "-01"
// 总搜索次数
@@ -157,3 +158,20 @@ func (r *SearchStatRepositoryImpl) GetSummary() (map[string]int64, error) {
"keywords": keywords,
}, nil
}
// FindWithPaginationOrdered 按时间倒序分页查找搜索记录
func (r *SearchStatRepositoryImpl) FindWithPaginationOrdered(page, limit int) ([]entity.SearchStat, int64, error) {
var stats []entity.SearchStat
var total int64
offset := (page - 1) * limit
// 获取总数
if err := r.db.Model(&entity.SearchStat{}).Count(&total).Error; err != nil {
return nil, 0, err
}
// 获取分页数据,按创建时间倒序排列(最新的在前面)
err := r.db.Order("created_at DESC").Offset(offset).Limit(limit).Find(&stats).Error
return stats, total, err
}

View File

@@ -2,8 +2,10 @@ package repo
import (
"fmt"
"sync"
"github.com/ctwj/urldb/db/entity"
"github.com/ctwj/urldb/utils"
"gorm.io/gorm"
)
@@ -18,17 +20,27 @@ type SystemConfigRepository interface {
GetConfigValue(key string) (string, error)
GetConfigBool(key string) (bool, error)
GetConfigInt(key string) (int, error)
GetCachedConfigs() map[string]string
ClearConfigCache()
SafeRefreshConfigCache() error
ValidateConfigIntegrity() error
}
// SystemConfigRepositoryImpl 系统配置Repository实现
type SystemConfigRepositoryImpl struct {
BaseRepositoryImpl[entity.SystemConfig]
// 配置缓存
configCache map[string]string // key -> value
configCacheOnce sync.Once
configCacheMutex sync.RWMutex
}
// NewSystemConfigRepository 创建系统配置Repository
func NewSystemConfigRepository(db *gorm.DB) SystemConfigRepository {
return &SystemConfigRepositoryImpl{
BaseRepositoryImpl: BaseRepositoryImpl[entity.SystemConfig]{db: db},
configCache: make(map[string]string),
}
}
@@ -51,24 +63,39 @@ func (r *SystemConfigRepositoryImpl) FindByKey(key string) (*entity.SystemConfig
// UpsertConfigs 批量创建或更新配置
func (r *SystemConfigRepositoryImpl) UpsertConfigs(configs []entity.SystemConfig) error {
for _, config := range configs {
var existingConfig entity.SystemConfig
err := r.db.Where("key = ?", config.Key).First(&existingConfig).Error
// 使用事务确保数据一致性
return r.db.Transaction(func(tx *gorm.DB) error {
// 在更新前备份当前配置
var existingConfigs []entity.SystemConfig
if err := tx.Find(&existingConfigs).Error; err != nil {
utils.Error("备份配置失败: %v", err)
// 不返回错误,继续执行更新
}
if err != nil {
// 如果不存在,则创建
if err := r.db.Create(&config).Error; err != nil {
return err
}
} else {
// 如果存在,则更新
config.ID = existingConfig.ID
if err := r.db.Save(&config).Error; err != nil {
return err
for _, config := range configs {
var existingConfig entity.SystemConfig
err := tx.Where("key = ?", config.Key).First(&existingConfig).Error
if err != nil {
// 如果不存在,则创建
if err := tx.Create(&config).Error; err != nil {
utils.Error("创建配置失败 [%s]: %v", config.Key, err)
return fmt.Errorf("创建配置失败 [%s]: %v", config.Key, err)
}
} else {
// 如果存在,则更新
config.ID = existingConfig.ID
if err := tx.Save(&config).Error; err != nil {
utils.Error("更新配置失败 [%s]: %v", config.Key, err)
return fmt.Errorf("更新配置失败 [%s]: %v", config.Key, err)
}
}
}
}
return nil
// 更新成功后刷新缓存
r.refreshConfigCache()
return nil
})
}
// GetOrCreateDefault 获取配置或创建默认配置
@@ -80,6 +107,7 @@ func (r *SystemConfigRepositoryImpl) GetOrCreateDefault() ([]entity.SystemConfig
// 如果没有配置,创建默认配置
if len(configs) == 0 {
utils.Info("未找到任何配置,创建默认配置")
defaultConfigs := []entity.SystemConfig{
{Key: entity.ConfigKeySiteTitle, Value: entity.ConfigDefaultSiteTitle, Type: entity.ConfigTypeString},
{Key: entity.ConfigKeySiteDescription, Value: entity.ConfigDefaultSiteDescription, Type: entity.ConfigTypeString},
@@ -93,8 +121,18 @@ func (r *SystemConfigRepositoryImpl) GetOrCreateDefault() ([]entity.SystemConfig
{Key: entity.ConfigKeyAutoTransferMinSpace, Value: entity.ConfigDefaultAutoTransferMinSpace, Type: entity.ConfigTypeInt},
{Key: entity.ConfigKeyAutoFetchHotDramaEnabled, Value: entity.ConfigDefaultAutoFetchHotDramaEnabled, Type: entity.ConfigTypeBool},
{Key: entity.ConfigKeyApiToken, Value: entity.ConfigDefaultApiToken, Type: entity.ConfigTypeString},
{Key: entity.ConfigKeyForbiddenWords, Value: entity.ConfigDefaultForbiddenWords, Type: entity.ConfigTypeString},
{Key: entity.ConfigKeyAdKeywords, Value: entity.ConfigDefaultAdKeywords, Type: entity.ConfigTypeString},
{Key: entity.ConfigKeyAutoInsertAd, Value: entity.ConfigDefaultAutoInsertAd, Type: entity.ConfigTypeString},
{Key: entity.ConfigKeyPageSize, Value: entity.ConfigDefaultPageSize, Type: entity.ConfigTypeInt},
{Key: entity.ConfigKeyMaintenanceMode, Value: entity.ConfigDefaultMaintenanceMode, Type: entity.ConfigTypeBool},
{Key: entity.ConfigKeyEnableRegister, Value: entity.ConfigDefaultEnableRegister, Type: entity.ConfigTypeBool},
{Key: entity.ConfigKeyThirdPartyStatsCode, Value: entity.ConfigDefaultThirdPartyStatsCode, Type: entity.ConfigTypeString},
{Key: entity.ConfigKeyMeilisearchEnabled, Value: entity.ConfigDefaultMeilisearchEnabled, Type: entity.ConfigTypeBool},
{Key: entity.ConfigKeyMeilisearchHost, Value: entity.ConfigDefaultMeilisearchHost, Type: entity.ConfigTypeString},
{Key: entity.ConfigKeyMeilisearchPort, Value: entity.ConfigDefaultMeilisearchPort, Type: entity.ConfigTypeString},
{Key: entity.ConfigKeyMeilisearchMasterKey, Value: entity.ConfigDefaultMeilisearchMasterKey, Type: entity.ConfigTypeString},
{Key: entity.ConfigKeyMeilisearchIndexName, Value: entity.ConfigDefaultMeilisearchIndexName, Type: entity.ConfigTypeString},
}
err = r.UpsertConfigs(defaultConfigs)
@@ -105,15 +143,186 @@ func (r *SystemConfigRepositoryImpl) GetOrCreateDefault() ([]entity.SystemConfig
return defaultConfigs, nil
}
// 检查是否有缺失的配置项,如果有则添加
requiredConfigs := map[string]entity.SystemConfig{
entity.ConfigKeySiteTitle: {Key: entity.ConfigKeySiteTitle, Value: entity.ConfigDefaultSiteTitle, Type: entity.ConfigTypeString},
entity.ConfigKeySiteDescription: {Key: entity.ConfigKeySiteDescription, Value: entity.ConfigDefaultSiteDescription, Type: entity.ConfigTypeString},
entity.ConfigKeyKeywords: {Key: entity.ConfigKeyKeywords, Value: entity.ConfigDefaultKeywords, Type: entity.ConfigTypeString},
entity.ConfigKeyAuthor: {Key: entity.ConfigKeyAuthor, Value: entity.ConfigDefaultAuthor, Type: entity.ConfigTypeString},
entity.ConfigKeyCopyright: {Key: entity.ConfigKeyCopyright, Value: entity.ConfigDefaultCopyright, Type: entity.ConfigTypeString},
entity.ConfigKeyAutoProcessReadyResources: {Key: entity.ConfigKeyAutoProcessReadyResources, Value: entity.ConfigDefaultAutoProcessReadyResources, Type: entity.ConfigTypeBool},
entity.ConfigKeyAutoProcessInterval: {Key: entity.ConfigKeyAutoProcessInterval, Value: entity.ConfigDefaultAutoProcessInterval, Type: entity.ConfigTypeInt},
entity.ConfigKeyAutoTransferEnabled: {Key: entity.ConfigKeyAutoTransferEnabled, Value: entity.ConfigDefaultAutoTransferEnabled, Type: entity.ConfigTypeBool},
entity.ConfigKeyAutoTransferLimitDays: {Key: entity.ConfigKeyAutoTransferLimitDays, Value: entity.ConfigDefaultAutoTransferLimitDays, Type: entity.ConfigTypeInt},
entity.ConfigKeyAutoTransferMinSpace: {Key: entity.ConfigKeyAutoTransferMinSpace, Value: entity.ConfigDefaultAutoTransferMinSpace, Type: entity.ConfigTypeInt},
entity.ConfigKeyAutoFetchHotDramaEnabled: {Key: entity.ConfigKeyAutoFetchHotDramaEnabled, Value: entity.ConfigDefaultAutoFetchHotDramaEnabled, Type: entity.ConfigTypeBool},
entity.ConfigKeyApiToken: {Key: entity.ConfigKeyApiToken, Value: entity.ConfigDefaultApiToken, Type: entity.ConfigTypeString},
entity.ConfigKeyForbiddenWords: {Key: entity.ConfigKeyForbiddenWords, Value: entity.ConfigDefaultForbiddenWords, Type: entity.ConfigTypeString},
entity.ConfigKeyAdKeywords: {Key: entity.ConfigKeyAdKeywords, Value: entity.ConfigDefaultAdKeywords, Type: entity.ConfigTypeString},
entity.ConfigKeyAutoInsertAd: {Key: entity.ConfigKeyAutoInsertAd, Value: entity.ConfigDefaultAutoInsertAd, Type: entity.ConfigTypeString},
entity.ConfigKeyPageSize: {Key: entity.ConfigKeyPageSize, Value: entity.ConfigDefaultPageSize, Type: entity.ConfigTypeInt},
entity.ConfigKeyMaintenanceMode: {Key: entity.ConfigKeyMaintenanceMode, Value: entity.ConfigDefaultMaintenanceMode, Type: entity.ConfigTypeBool},
entity.ConfigKeyEnableRegister: {Key: entity.ConfigKeyEnableRegister, Value: entity.ConfigDefaultEnableRegister, Type: entity.ConfigTypeBool},
entity.ConfigKeyThirdPartyStatsCode: {Key: entity.ConfigKeyThirdPartyStatsCode, Value: entity.ConfigDefaultThirdPartyStatsCode, Type: entity.ConfigTypeString},
entity.ConfigKeyMeilisearchEnabled: {Key: entity.ConfigKeyMeilisearchEnabled, Value: entity.ConfigDefaultMeilisearchEnabled, Type: entity.ConfigTypeBool},
entity.ConfigKeyMeilisearchHost: {Key: entity.ConfigKeyMeilisearchHost, Value: entity.ConfigDefaultMeilisearchHost, Type: entity.ConfigTypeString},
entity.ConfigKeyMeilisearchPort: {Key: entity.ConfigKeyMeilisearchPort, Value: entity.ConfigDefaultMeilisearchPort, Type: entity.ConfigTypeString},
entity.ConfigKeyMeilisearchMasterKey: {Key: entity.ConfigKeyMeilisearchMasterKey, Value: entity.ConfigDefaultMeilisearchMasterKey, Type: entity.ConfigTypeString},
entity.ConfigKeyMeilisearchIndexName: {Key: entity.ConfigKeyMeilisearchIndexName, Value: entity.ConfigDefaultMeilisearchIndexName, Type: entity.ConfigTypeString},
}
// 检查现有配置中是否有缺失的配置项
existingKeys := make(map[string]bool)
for _, config := range configs {
existingKeys[config.Key] = true
}
// 找出缺失的配置项
var missingConfigs []entity.SystemConfig
for key, requiredConfig := range requiredConfigs {
if !existingKeys[key] {
missingConfigs = append(missingConfigs, requiredConfig)
}
}
// 如果有缺失的配置项,则添加它们
if len(missingConfigs) > 0 {
err = r.UpsertConfigs(missingConfigs)
if err != nil {
return nil, err
}
// 重新获取所有配置
configs, err = r.FindAll()
if err != nil {
return nil, err
}
}
return configs, nil
}
// initConfigCache 初始化配置缓存
func (r *SystemConfigRepositoryImpl) initConfigCache() {
r.configCacheOnce.Do(func() {
// 获取所有配置
configs, err := r.FindAll()
if err != nil {
// 如果获取失败,尝试创建默认配置
configs, err = r.GetOrCreateDefault()
if err != nil {
return
}
}
// 初始化缓存
r.configCacheMutex.Lock()
defer r.configCacheMutex.Unlock()
for _, config := range configs {
r.configCache[config.Key] = config.Value
}
})
}
// refreshConfigCache 刷新配置缓存
func (r *SystemConfigRepositoryImpl) refreshConfigCache() {
// 重置Once允许重新初始化
r.configCacheOnce = sync.Once{}
// 清空缓存
r.configCacheMutex.Lock()
r.configCache = make(map[string]string)
r.configCacheMutex.Unlock()
// 重新初始化缓存
r.initConfigCache()
}
// SafeRefreshConfigCache 安全的刷新配置缓存(带错误处理)
func (r *SystemConfigRepositoryImpl) SafeRefreshConfigCache() error {
defer func() {
if r := recover(); r != nil {
utils.Error("配置缓存刷新时发生panic: %v", r)
}
}()
r.refreshConfigCache()
return nil
}
// ValidateConfigIntegrity 验证配置完整性
func (r *SystemConfigRepositoryImpl) ValidateConfigIntegrity() error {
configs, err := r.FindAll()
if err != nil {
return fmt.Errorf("获取配置失败: %v", err)
}
// 检查关键配置是否存在
requiredKeys := []string{
entity.ConfigKeySiteTitle,
entity.ConfigKeySiteDescription,
entity.ConfigKeyKeywords,
entity.ConfigKeyAuthor,
entity.ConfigKeyCopyright,
entity.ConfigKeyAutoProcessReadyResources,
entity.ConfigKeyAutoProcessInterval,
entity.ConfigKeyAutoTransferEnabled,
entity.ConfigKeyAutoTransferLimitDays,
entity.ConfigKeyAutoTransferMinSpace,
entity.ConfigKeyAutoFetchHotDramaEnabled,
entity.ConfigKeyApiToken,
entity.ConfigKeyPageSize,
entity.ConfigKeyMaintenanceMode,
entity.ConfigKeyEnableRegister,
entity.ConfigKeyThirdPartyStatsCode,
}
existingKeys := make(map[string]bool)
for _, config := range configs {
existingKeys[config.Key] = true
}
var missingKeys []string
for _, key := range requiredKeys {
if !existingKeys[key] {
missingKeys = append(missingKeys, key)
}
}
if len(missingKeys) > 0 {
utils.Error("发现缺失的配置项: %v", missingKeys)
return fmt.Errorf("配置不完整,缺失: %v", missingKeys)
}
utils.Info("配置完整性检查通过")
return nil
}
// GetConfigValue 获取配置值(字符串)
func (r *SystemConfigRepositoryImpl) GetConfigValue(key string) (string, error) {
// 初始化缓存
r.initConfigCache()
// 从缓存中读取
r.configCacheMutex.RLock()
value, exists := r.configCache[key]
r.configCacheMutex.RUnlock()
if exists {
return value, nil
}
// 如果缓存中没有,尝试从数据库获取(可能是新添加的配置)
config, err := r.FindByKey(key)
if err != nil {
return "", err
}
// 更新缓存
r.configCacheMutex.Lock()
r.configCache[key] = config.Value
r.configCacheMutex.Unlock()
return config.Value, nil
}
@@ -146,3 +355,29 @@ func (r *SystemConfigRepositoryImpl) GetConfigInt(key string) (int, error) {
_, err = fmt.Sscanf(value, "%d", &result)
return result, err
}
// GetCachedConfigs 获取所有缓存的配置(用于调试)
func (r *SystemConfigRepositoryImpl) GetCachedConfigs() map[string]string {
r.initConfigCache()
r.configCacheMutex.RLock()
defer r.configCacheMutex.RUnlock()
// 返回缓存的副本
result := make(map[string]string)
for k, v := range r.configCache {
result[k] = v
}
return result
}
// ClearConfigCache 清空配置缓存(用于测试或手动刷新)
func (r *SystemConfigRepositoryImpl) ClearConfigCache() {
r.configCacheMutex.Lock()
r.configCache = make(map[string]string)
r.configCacheMutex.Unlock()
// 重置Once允许重新初始化
r.configCacheOnce = sync.Once{}
}

View File

@@ -10,15 +10,19 @@ import (
type TagRepository interface {
BaseRepository[entity.Tag]
FindByName(name string) (*entity.Tag, error)
FindByNameIncludingDeleted(name string) (*entity.Tag, error)
FindWithResources() ([]entity.Tag, error)
FindByCategoryID(categoryID uint) ([]entity.Tag, error)
FindByCategoryIDPaginated(categoryID uint, page, pageSize int) ([]entity.Tag, int64, error)
GetResourceCount(tagID uint) (int64, error)
FindByResourceID(resourceID uint) ([]entity.Tag, error)
FindWithPagination(page, pageSize int) ([]entity.Tag, int64, error)
FindWithPaginationOrderByResourceCount(page, pageSize int) ([]entity.Tag, int64, error)
Search(query string, page, pageSize int) ([]entity.Tag, int64, error)
SearchOrderByResourceCount(query string, page, pageSize int) ([]entity.Tag, int64, error)
UpdateWithNulls(tag *entity.Tag) error
GetByID(id uint) (*entity.Tag, error)
RestoreDeletedTag(id uint) error
}
// TagRepositoryImpl Tag的Repository实现
@@ -43,6 +47,16 @@ func (r *TagRepositoryImpl) FindByName(name string) (*entity.Tag, error) {
return &tag, nil
}
// FindByNameIncludingDeleted 根据名称查找(包括已删除的记录)
func (r *TagRepositoryImpl) FindByNameIncludingDeleted(name string) (*entity.Tag, error) {
var tag entity.Tag
err := r.db.Unscoped().Where("name = ?", name).First(&tag).Error
if err != nil {
return nil, err
}
return &tag, nil
}
// FindWithResources 查找包含资源的标签
func (r *TagRepositoryImpl) FindWithResources() ([]entity.Tag, error) {
var tags []entity.Tag
@@ -155,3 +169,76 @@ func (r *TagRepositoryImpl) GetByID(id uint) (*entity.Tag, error) {
}
return &tag, nil
}
// RestoreDeletedTag 恢复已删除的标签
func (r *TagRepositoryImpl) RestoreDeletedTag(id uint) error {
return r.db.Unscoped().Model(&entity.Tag{}).Where("id = ?", id).Update("deleted_at", nil).Error
}
// FindWithPaginationOrderByResourceCount 按资源数量排序的分页查询
func (r *TagRepositoryImpl) FindWithPaginationOrderByResourceCount(page, pageSize int) ([]entity.Tag, int64, error) {
var tags []entity.Tag
var total int64
// 获取总数
err := r.db.Model(&entity.Tag{}).Count(&total).Error
if err != nil {
return nil, 0, err
}
// 使用子查询统计每个标签的资源数量并排序
offset := (page - 1) * pageSize
err = r.db.Preload("Category").
Select("tags.*, COALESCE(resource_counts.count, 0) as resource_count").
Joins(`LEFT JOIN (
SELECT rt.tag_id, COUNT(rt.resource_id) as count
FROM resource_tags rt
INNER JOIN resources r ON rt.resource_id = r.id AND r.deleted_at IS NULL
GROUP BY rt.tag_id
) as resource_counts ON tags.id = resource_counts.tag_id`).
Order("COALESCE(resource_counts.count, 0) DESC, tags.created_at DESC").
Offset(offset).Limit(pageSize).
Find(&tags).Error
if err != nil {
return nil, 0, err
}
return tags, total, nil
}
// SearchOrderByResourceCount 按资源数量排序的搜索
func (r *TagRepositoryImpl) SearchOrderByResourceCount(query string, page, pageSize int) ([]entity.Tag, int64, error) {
var tags []entity.Tag
var total int64
// 构建搜索条件
searchQuery := "%" + query + "%"
// 获取总数
err := r.db.Model(&entity.Tag{}).Where("name ILIKE ? OR description ILIKE ?", searchQuery, searchQuery).Count(&total).Error
if err != nil {
return nil, 0, err
}
// 使用子查询统计每个标签的资源数量并排序
offset := (page - 1) * pageSize
err = r.db.Preload("Category").
Select("tags.*, COALESCE(resource_counts.count, 0) as resource_count").
Joins(`LEFT JOIN (
SELECT rt.tag_id, COUNT(rt.resource_id) as count
FROM resource_tags rt
INNER JOIN resources r ON rt.resource_id = r.id AND r.deleted_at IS NULL
GROUP BY rt.tag_id
) as resource_counts ON tags.id = resource_counts.tag_id`).
Where("tags.name ILIKE ? OR tags.description ILIKE ?", searchQuery, searchQuery).
Order("COALESCE(resource_counts.count, 0) DESC, tags.created_at DESC").
Offset(offset).Limit(pageSize).
Find(&tags).Error
if err != nil {
return nil, 0, err
}
return tags, total, nil
}

View File

@@ -0,0 +1,145 @@
package repo
import (
"github.com/ctwj/urldb/db/entity"
"gorm.io/gorm"
)
// TaskItemRepository 任务项仓库接口
type TaskItemRepository interface {
GetByID(id uint) (*entity.TaskItem, error)
Create(item *entity.TaskItem) error
Delete(id uint) error
DeleteByTaskID(taskID uint) error
GetByTaskIDAndStatus(taskID uint, status string) ([]*entity.TaskItem, error)
GetListByTaskID(taskID uint, page, pageSize int, status string) ([]*entity.TaskItem, int64, error)
UpdateStatus(id uint, status string) error
UpdateStatusAndOutput(id uint, status, outputData string) error
GetStatsByTaskID(taskID uint) (map[string]int, error)
ResetProcessingItems(taskID uint) error
}
// TaskItemRepositoryImpl 任务项仓库实现
type TaskItemRepositoryImpl struct {
db *gorm.DB
}
// NewTaskItemRepository 创建任务项仓库
func NewTaskItemRepository(db *gorm.DB) TaskItemRepository {
return &TaskItemRepositoryImpl{
db: db,
}
}
// GetByID 根据ID获取任务项
func (r *TaskItemRepositoryImpl) GetByID(id uint) (*entity.TaskItem, error) {
var item entity.TaskItem
err := r.db.First(&item, id).Error
if err != nil {
return nil, err
}
return &item, nil
}
// Create 创建任务项
func (r *TaskItemRepositoryImpl) Create(item *entity.TaskItem) error {
return r.db.Create(item).Error
}
// Delete 删除任务项
func (r *TaskItemRepositoryImpl) Delete(id uint) error {
return r.db.Delete(&entity.TaskItem{}, id).Error
}
// DeleteByTaskID 根据任务ID删除所有任务项
func (r *TaskItemRepositoryImpl) DeleteByTaskID(taskID uint) error {
return r.db.Where("task_id = ?", taskID).Delete(&entity.TaskItem{}).Error
}
// GetByTaskIDAndStatus 根据任务ID和状态获取任务项
func (r *TaskItemRepositoryImpl) GetByTaskIDAndStatus(taskID uint, status string) ([]*entity.TaskItem, error) {
var items []*entity.TaskItem
err := r.db.Where("task_id = ? AND status = ?", taskID, status).Order("id ASC").Find(&items).Error
return items, err
}
// GetListByTaskID 根据任务ID分页获取任务项
func (r *TaskItemRepositoryImpl) GetListByTaskID(taskID uint, page, pageSize int, status string) ([]*entity.TaskItem, int64, error) {
var items []*entity.TaskItem
var total int64
query := r.db.Model(&entity.TaskItem{}).Where("task_id = ?", taskID)
// 添加状态过滤
if status != "" {
query = query.Where("status = ?", status)
}
// 获取总数
err := query.Count(&total).Error
if err != nil {
return nil, 0, err
}
// 分页查询
offset := (page - 1) * pageSize
err = query.Offset(offset).Limit(pageSize).Order("id ASC").Find(&items).Error
if err != nil {
return nil, 0, err
}
return items, total, nil
}
// UpdateStatus 更新任务项状态
func (r *TaskItemRepositoryImpl) UpdateStatus(id uint, status string) error {
return r.db.Model(&entity.TaskItem{}).Where("id = ?", id).Update("status", status).Error
}
// UpdateStatusAndOutput 更新任务项状态和输出数据
func (r *TaskItemRepositoryImpl) UpdateStatusAndOutput(id uint, status, outputData string) error {
return r.db.Model(&entity.TaskItem{}).Where("id = ?", id).Updates(map[string]interface{}{
"status": status,
"output_data": outputData,
}).Error
}
// GetStatsByTaskID 获取任务项统计信息
func (r *TaskItemRepositoryImpl) GetStatsByTaskID(taskID uint) (map[string]int, error) {
var results []struct {
Status string
Count int
}
err := r.db.Model(&entity.TaskItem{}).
Select("status, count(*) as count").
Where("task_id = ?", taskID).
Group("status").
Find(&results).Error
if err != nil {
return nil, err
}
stats := map[string]int{
"total": 0,
"pending": 0,
"processing": 0,
"completed": 0,
"failed": 0,
}
for _, result := range results {
stats[result.Status] = result.Count
stats["total"] += result.Count
}
return stats, nil
}
// ResetProcessingItems 重置处理中的任务项为pending状态
func (r *TaskItemRepositoryImpl) ResetProcessingItems(taskID uint) error {
return r.db.Model(&entity.TaskItem{}).
Where("task_id = ? AND status = ?", taskID, "processing").
Update("status", "pending").Error
}

136
db/repo/task_repository.go Normal file
View File

@@ -0,0 +1,136 @@
package repo
import (
"github.com/ctwj/urldb/db/entity"
"gorm.io/gorm"
)
// TaskRepository 任务仓库接口
type TaskRepository interface {
GetByID(id uint) (*entity.Task, error)
Create(task *entity.Task) error
Delete(id uint) error
GetList(page, pageSize int, taskType, status string) ([]*entity.Task, int64, error)
UpdateStatus(id uint, status string) error
UpdateProgress(id uint, progress float64, progressData string) error
UpdateStatusAndMessage(id uint, status, message string) error
UpdateTaskStats(id uint, processed, success, failed int) error
}
// TaskRepositoryImpl 任务仓库实现
type TaskRepositoryImpl struct {
db *gorm.DB
}
// NewTaskRepository 创建任务仓库
func NewTaskRepository(db *gorm.DB) TaskRepository {
return &TaskRepositoryImpl{
db: db,
}
}
// GetByID 根据ID获取任务
func (r *TaskRepositoryImpl) GetByID(id uint) (*entity.Task, error) {
var task entity.Task
err := r.db.First(&task, id).Error
if err != nil {
return nil, err
}
return &task, nil
}
// Create 创建任务
func (r *TaskRepositoryImpl) Create(task *entity.Task) error {
return r.db.Create(task).Error
}
// Delete 删除任务
func (r *TaskRepositoryImpl) Delete(id uint) error {
return r.db.Delete(&entity.Task{}, id).Error
}
// GetList 获取任务列表
func (r *TaskRepositoryImpl) GetList(page, pageSize int, taskType, status string) ([]*entity.Task, int64, error) {
var tasks []*entity.Task
var total int64
query := r.db.Model(&entity.Task{})
// 添加过滤条件
if taskType != "" {
query = query.Where("task_type = ?", taskType)
}
if status != "" {
query = query.Where("status = ?", status)
}
// 获取总数
err := query.Count(&total).Error
if err != nil {
return nil, 0, err
}
// 分页查询
offset := (page - 1) * pageSize
err = query.Offset(offset).Limit(pageSize).Order("created_at DESC").Find(&tasks).Error
if err != nil {
return nil, 0, err
}
return tasks, total, nil
}
// UpdateStatus 更新任务状态
func (r *TaskRepositoryImpl) UpdateStatus(id uint, status string) error {
return r.db.Model(&entity.Task{}).Where("id = ?", id).Update("status", status).Error
}
// UpdateProgress 更新任务进度
func (r *TaskRepositoryImpl) UpdateProgress(id uint, progress float64, progressData string) error {
// 检查progress和progress_data字段是否存在
var count int64
err := r.db.Raw("SELECT COUNT(*) FROM information_schema.columns WHERE table_name = 'tasks' AND column_name = 'progress'").Count(&count).Error
if err != nil || count == 0 {
// 如果检查失败或字段不存在只更新processed_items等现有字段
return r.db.Model(&entity.Task{}).Where("id = ?", id).Updates(map[string]interface{}{
"processed_items": progress, // 使用progress作为processed_items的近似值
}).Error
}
// 字段存在,正常更新
return r.db.Model(&entity.Task{}).Where("id = ?", id).Updates(map[string]interface{}{
"progress": progress,
"progress_data": progressData,
}).Error
}
// UpdateStatusAndMessage 更新任务状态和消息
func (r *TaskRepositoryImpl) UpdateStatusAndMessage(id uint, status, message string) error {
// 检查message字段是否存在
var count int64
err := r.db.Raw("SELECT COUNT(*) FROM information_schema.columns WHERE table_name = 'tasks' AND column_name = 'message'").Count(&count).Error
if err != nil {
// 如果检查失败,只更新状态
return r.db.Model(&entity.Task{}).Where("id = ?", id).Update("status", status).Error
}
if count > 0 {
// message字段存在更新状态和消息
return r.db.Model(&entity.Task{}).Where("id = ?", id).Updates(map[string]interface{}{
"status": status,
"message": message,
}).Error
} else {
// message字段不存在只更新状态
return r.db.Model(&entity.Task{}).Where("id = ?", id).Update("status", status).Error
}
}
// UpdateTaskStats 更新任务统计信息
func (r *TaskRepositoryImpl) UpdateTaskStats(id uint, processed, success, failed int) error {
return r.db.Model(&entity.Task{}).Where("id = ?", id).Updates(map[string]interface{}{
"processed_items": processed,
"success_items": success,
"failed_items": failed,
}).Error
}

View File

@@ -20,7 +20,7 @@ services:
- app-network
backend:
image: ctwj/urldb-backend:1.0.9
image: ctwj/urldb-backend:1.2.3
environment:
DB_HOST: postgres
DB_PORT: 5432
@@ -38,10 +38,10 @@ services:
- app-network
frontend:
image: ctwj/urldb-frontend:1.0.9
image: ctwj/urldb-frontend:1.2.3
environment:
NODE_ENV: production
NUXT_PUBLIC_API_SERVER: http://backend:8080/api
NUXT_PUBLIC_API_CLIENT: /api
depends_on:
- backend
networks:

View File

@@ -1,8 +1,8 @@
// docsify 配置文件
window.$docsify = {
name: 'URL数据库管理系统',
name: '老九网盘链接数据库',
repo: 'https://github.com/ctwj/urldb',
loadSidebar: true,
loadSidebar: '_sidebar.md',
subMaxLevel: 3,
auto2top: true,
// 添加侧边栏配置

View File

@@ -13,7 +13,7 @@
<body>
<div id="app"></div>
<script src="//cdn.jsdelivr.net/npm/docsify@4"></script>
<script src="./docsify.config.js"></script>
<script src="docsify.config.js"></script>
<script src="//cdn.jsdelivr.net/npm/docsify@4/lib/plugins/search.min.js"></script>
<script src="//cdn.jsdelivr.net/npm/docsify@4/lib/plugins/copy-code.min.js"></script>
<script src="//cdn.jsdelivr.net/npm/docsify@4/lib/plugins/pagination.min.js"></script>

View File

@@ -14,4 +14,4 @@ TIMEZONE=Asia/Shanghai
# 文件上传配置
UPLOAD_DIR=./uploads
MAX_FILE_SIZE=100MB
MAX_FILE_SIZE=5MB

BIN
github/account.webp Normal file

Binary file not shown.

After

Width:  |  Height:  |  Size: 31 KiB

BIN
github/admin.webp Normal file

Binary file not shown.

After

Width:  |  Height:  |  Size: 45 KiB

BIN
github/config.webp Normal file

Binary file not shown.

After

Width:  |  Height:  |  Size: 32 KiB

BIN
github/index.webp Normal file

Binary file not shown.

After

Width:  |  Height:  |  Size: 49 KiB

BIN
github/save.webp Normal file

Binary file not shown.

After

Width:  |  Height:  |  Size: 66 KiB

6
go.mod
View File

@@ -10,11 +10,17 @@ require (
github.com/go-resty/resty/v2 v2.16.5
github.com/golang-jwt/jwt/v5 v5.2.0
github.com/joho/godotenv v1.5.1
github.com/meilisearch/meilisearch-go v0.33.1
golang.org/x/crypto v0.40.0
gorm.io/driver/postgres v1.6.0
gorm.io/gorm v1.30.0
)
require (
github.com/andybalholm/brotli v1.1.1 // indirect
github.com/golang-jwt/jwt/v4 v4.5.2 // indirect
)
require (
github.com/bytedance/sonic v1.13.3 // indirect
github.com/bytedance/sonic/loader v0.3.0 // indirect

8
go.sum
View File

@@ -1,3 +1,5 @@
github.com/andybalholm/brotli v1.1.1 h1:PR2pgnyFznKEugtsUo0xLdDop5SKXd5Qf5ysW+7XdTA=
github.com/andybalholm/brotli v1.1.1/go.mod h1:05ib4cKhjx3OQYUY22hTVd34Bc8upXjOLL2rKwwZBoA=
github.com/bytedance/sonic v1.13.3 h1:MS8gmaH16Gtirygw7jV91pDCN33NyMrPbN7qiYhEsF0=
github.com/bytedance/sonic v1.13.3/go.mod h1:o68xyaF9u2gvVBuGHPlUVCy+ZfmNNO5ETf1+KgkJhz4=
github.com/bytedance/sonic/loader v0.1.1/go.mod h1:ncP89zfokxS5LZrJxl5z0UJcsk4M4yY2JpfqGeCtNLU=
@@ -37,6 +39,8 @@ github.com/go-resty/resty/v2 v2.16.5/go.mod h1:hkJtXbA2iKHzJheXYvQ8snQES5ZLGKMwQ
github.com/goccy/go-json v0.9.7/go.mod h1:6MelG93GURQebXPDq3khkgXZkazVtN9CRI+MGFi0w8I=
github.com/goccy/go-json v0.10.5 h1:Fq85nIqj+gXn/S5ahsiTlK3TmC85qgirsdTP/+DeaC4=
github.com/goccy/go-json v0.10.5/go.mod h1:oq7eo15ShAhp70Anwd5lgX2pLfOS3QCiwU/PULtXL6M=
github.com/golang-jwt/jwt/v4 v4.5.2 h1:YtQM7lnr8iZ+j5q71MGKkNw9Mn7AjHM68uc9g5fXeUI=
github.com/golang-jwt/jwt/v4 v4.5.2/go.mod h1:m21LjoU+eqJr34lmDMbreY2eSTRJ1cv77w39/MY0Ch0=
github.com/golang-jwt/jwt/v5 v5.2.0 h1:d/ix8ftRUorsN+5eMIlF4T6J8CAt9rch3My2winC1Jw=
github.com/golang-jwt/jwt/v5 v5.2.0/go.mod h1:pqrtFR0X4osieyHYxtmOUWsAWrfe1Q5UVIyoH402zdk=
github.com/golang/protobuf v1.5.0/go.mod h1:FsONVRAS9T7sI+LIUmWTfcYkHO4aIWwzhcaSAoJOfIk=
@@ -81,6 +85,8 @@ github.com/matoous/go-nanoid/v2 v2.1.0/go.mod h1:KlbGNQ+FhrUNIHUxZdL63t7tl4LaPkZ
github.com/mattn/go-isatty v0.0.14/go.mod h1:7GGIvUiUoEMVVmxf/4nioHXj79iQHKdU27kJ6hsGG94=
github.com/mattn/go-isatty v0.0.20 h1:xfD0iDuEKnDkl03q4limB+vH+GxLEtL/jb4xVJSWWEY=
github.com/mattn/go-isatty v0.0.20/go.mod h1:W+V8PltTTMOvKvAeJH7IuucS94S2C6jfK/D7dTCTo3Y=
github.com/meilisearch/meilisearch-go v0.33.1 h1:IWM8iJU7UyuIoRiTTLONvpbEgMhP/yTrnNfSnxj4wu0=
github.com/meilisearch/meilisearch-go v0.33.1/go.mod h1:dY4nxhVc0Ext8Kn7u2YohJCsEjirg80DdcOmfNezUYg=
github.com/modern-go/concurrent v0.0.0-20180228061459-e0a39a4cb421/go.mod h1:6dJC0mAP4ikYIbvyc7fijjWJddQyLn8Ig3JB5CqoB9Q=
github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd h1:TRLaZ9cD/w8PVh93nsPXa1VrQ6jlwL5oN8l14QlcNfg=
github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd/go.mod h1:6dJC0mAP4ikYIbvyc7fijjWJddQyLn8Ig3JB5CqoB9Q=
@@ -114,6 +120,8 @@ github.com/ugorji/go v1.2.7/go.mod h1:nF9osbDWLy6bDVv/Rtoh6QgnvNDpmCalQV5urGCCS6
github.com/ugorji/go/codec v1.2.7/go.mod h1:WGN1fab3R1fzQlVQTkfxVtIBhWDRqOviHU95kRgeqEY=
github.com/ugorji/go/codec v1.3.0 h1:Qd2W2sQawAfG8XSvzwhBeoGq71zXOC/Q1E9y/wUcsUA=
github.com/ugorji/go/codec v1.3.0/go.mod h1:pRBVtBSKl77K30Bv8R2P+cLSGaTtex6fsA2Wjqmfxj4=
github.com/xyproto/randomstring v1.0.5 h1:YtlWPoRdgMu3NZtP45drfy1GKoojuR7hmRcnhZqKjWU=
github.com/xyproto/randomstring v1.0.5/go.mod h1:rgmS5DeNXLivK7YprL0pY+lTuhNQW3iGxZ18UQApw/E=
golang.org/x/arch v0.19.0 h1:LmbDQUodHThXE+htjrnmVD73M//D9GTH6wFZjyDkjyU=
golang.org/x/arch v0.19.0/go.mod h1:bdwinDaKcfZUGpH09BB7ZmOfhalA8lQdzl62l8gGWsk=
golang.org/x/crypto v0.0.0-20210711020723-a769d52b0f97/go.mod h1:GvvjBRRGRdwPK5ydBHafDWAxML/pGHZbMvKqRZ5+Abc=

View File

@@ -2,11 +2,18 @@ package handlers
import (
"github.com/ctwj/urldb/db/repo"
"github.com/ctwj/urldb/services"
)
var repoManager *repo.RepositoryManager
var meilisearchManager *services.MeilisearchManager
// SetRepositoryManager 设置Repository管理器
func SetRepositoryManager(rm *repo.RepositoryManager) {
repoManager = rm
func SetRepositoryManager(manager *repo.RepositoryManager) {
repoManager = manager
}
// SetMeilisearchManager 设置Meilisearch管理器
func SetMeilisearchManager(manager *services.MeilisearchManager) {
meilisearchManager = manager
}

View File

@@ -7,6 +7,7 @@ import (
"github.com/ctwj/urldb/db/converter"
"github.com/ctwj/urldb/db/dto"
"github.com/ctwj/urldb/db/entity"
"github.com/ctwj/urldb/utils"
"github.com/gin-gonic/gin"
)
@@ -18,6 +19,8 @@ func GetCategories(c *gin.Context) {
pageSize, _ := strconv.Atoi(c.DefaultQuery("page_size", "20"))
search := c.Query("search")
utils.Debug("获取分类列表 - 分页参数: page=%d, pageSize=%d, search=%s", page, pageSize, search)
var categories []entity.Category
var total int64
var err error
@@ -35,6 +38,8 @@ func GetCategories(c *gin.Context) {
return
}
utils.Debug("查询到分类数量: %d, 总数: %d", len(categories), total)
// 获取每个分类的资源数量和标签名称
resourceCounts := make(map[uint]int64)
tagNamesMap := make(map[uint][]string)
@@ -73,12 +78,50 @@ func CreateCategory(c *gin.Context) {
return
}
// 首先检查是否存在已删除的同名分类
deletedCategory, err := repoManager.CategoryRepository.FindByNameIncludingDeleted(req.Name)
if err == nil && deletedCategory.DeletedAt.Valid {
utils.Debug("找到已删除的分类: ID=%d, Name=%s", deletedCategory.ID, deletedCategory.Name)
// 如果存在已删除的同名分类,则恢复它
err = repoManager.CategoryRepository.RestoreDeletedCategory(deletedCategory.ID)
if err != nil {
ErrorResponse(c, "恢复已删除分类失败: "+err.Error(), http.StatusInternalServerError)
return
}
utils.Debug("分类恢复成功: ID=%d", deletedCategory.ID)
// 重新获取恢复后的分类
restoredCategory, err := repoManager.CategoryRepository.FindByID(deletedCategory.ID)
if err != nil {
ErrorResponse(c, "获取恢复的分类失败: "+err.Error(), http.StatusInternalServerError)
return
}
utils.Debug("重新获取到恢复的分类: ID=%d, Name=%s", restoredCategory.ID, restoredCategory.Name)
// 更新分类信息
restoredCategory.Description = req.Description
err = repoManager.CategoryRepository.Update(restoredCategory)
if err != nil {
ErrorResponse(c, "更新恢复的分类失败: "+err.Error(), http.StatusInternalServerError)
return
}
utils.Debug("分类信息更新成功: ID=%d, Description=%s", restoredCategory.ID, restoredCategory.Description)
SuccessResponse(c, gin.H{
"message": "分类恢复成功",
"category": converter.ToCategoryResponse(restoredCategory, 0, []string{}),
})
return
}
// 如果不存在已删除的同名分类,则创建新分类
category := &entity.Category{
Name: req.Name,
Description: req.Description,
}
err := repoManager.CategoryRepository.Create(category)
err = repoManager.CategoryRepository.Create(category)
if err != nil {
ErrorResponse(c, err.Error(), http.StatusInternalServerError)
return

442
handlers/file_handler.go Normal file
View File

@@ -0,0 +1,442 @@
package handlers
import (
"crypto/rand"
"crypto/sha256"
"fmt"
"io"
"net/http"
"os"
"path/filepath"
"strconv"
"strings"
"time"
"github.com/ctwj/urldb/db/converter"
"github.com/ctwj/urldb/db/dto"
"github.com/ctwj/urldb/db/entity"
"github.com/ctwj/urldb/db/repo"
"github.com/ctwj/urldb/utils"
"github.com/gin-gonic/gin"
)
// FileHandler 文件处理器
type FileHandler struct {
fileRepo repo.FileRepository
systemConfigRepo repo.SystemConfigRepository
userRepo repo.UserRepository
}
// NewFileHandler 创建文件处理器
func NewFileHandler(fileRepo repo.FileRepository, systemConfigRepo repo.SystemConfigRepository, userRepo repo.UserRepository) *FileHandler {
return &FileHandler{
fileRepo: fileRepo,
systemConfigRepo: systemConfigRepo,
userRepo: userRepo,
}
}
// UploadFile 上传文件
func (h *FileHandler) UploadFile(c *gin.Context) {
// 获取当前用户ID
userID, exists := c.Get("user_id")
if !exists {
ErrorResponse(c, "用户未登录", http.StatusUnauthorized)
return
}
// 从数据库获取用户信息
currentUser, err := h.userRepo.FindByID(userID.(uint))
if err != nil {
ErrorResponse(c, "用户不存在", http.StatusUnauthorized)
return
}
// 获取文件哈希值
fileHash := c.PostForm("file_hash")
// 如果提供了文件哈希,先检查是否已存在
if fileHash != "" {
existingFile, err := h.fileRepo.FindByHash(fileHash)
if err == nil && existingFile != nil {
// 文件已存在,直接返回已存在的文件信息
utils.Info("文件已存在,跳过上传 - Hash: %s, 文件名: %s", fileHash, existingFile.OriginalName)
response := dto.FileUploadResponse{
File: converter.FileToResponse(existingFile),
Message: "文件已存在,极速上传成功",
Success: true,
IsDuplicate: true,
}
SuccessResponse(c, response)
return
}
}
// 获取上传目录配置(从环境变量或使用默认值)
uploadDir := os.Getenv("UPLOAD_DIR")
if uploadDir == "" {
uploadDir = "./uploads" // 默认值
}
// 创建年月子文件夹
now := time.Now()
yearMonth := now.Format("200601") // 格式202508
monthlyDir := filepath.Join(uploadDir, yearMonth)
// 确保年月目录存在
if err := os.MkdirAll(monthlyDir, 0755); err != nil {
ErrorResponse(c, "创建年月目录失败", http.StatusInternalServerError)
return
}
// 获取上传的文件
file, header, err := c.Request.FormFile("file")
if err != nil {
ErrorResponse(c, "获取上传文件失败", http.StatusBadRequest)
return
}
defer file.Close()
// 检查文件大小5MB
maxFileSize := int64(5 * 1024 * 1024) // 5MB
if header.Size > maxFileSize {
ErrorResponse(c, "文件大小不能超过5MB", http.StatusBadRequest)
return
}
// 检查文件类型,只允许图片
allowedTypes := []string{
"image/jpeg",
"image/jpg",
"image/png",
"image/gif",
"image/webp",
"image/bmp",
"image/svg+xml",
}
contentType := header.Header.Get("Content-Type")
isAllowedType := false
for _, allowedType := range allowedTypes {
if contentType == allowedType {
isAllowedType = true
break
}
}
if !isAllowedType {
ErrorResponse(c, "只支持图片格式文件 (JPEG, PNG, GIF, WebP, BMP, SVG)", http.StatusBadRequest)
return
}
// 生成随机文件名
fileName := h.generateRandomFileName(header.Filename)
filePath := filepath.Join(monthlyDir, fileName)
// 创建目标文件
dst, err := os.Create(filePath)
if err != nil {
ErrorResponse(c, "创建文件失败", http.StatusInternalServerError)
return
}
defer dst.Close()
// 复制文件内容
if _, err := io.Copy(dst, file); err != nil {
ErrorResponse(c, "保存文件失败", http.StatusInternalServerError)
return
}
// 计算文件哈希值(如果前端没有提供)
if fileHash == "" {
fileHash, err = h.calculateFileHash(filePath)
if err != nil {
ErrorResponse(c, "计算文件哈希失败", http.StatusInternalServerError)
return
}
}
// 再次检查文件是否已存在(使用计算出的哈希)
existingFile, err := h.fileRepo.FindByHash(fileHash)
if err == nil && existingFile != nil {
// 文件已存在,删除刚上传的文件,返回已存在的文件信息
os.Remove(filePath)
utils.Info("文件已存在,跳过上传 - Hash: %s, 文件名: %s", fileHash, existingFile.OriginalName)
response := dto.FileUploadResponse{
File: converter.FileToResponse(existingFile),
Message: "文件已存在,极速上传成功",
Success: true,
IsDuplicate: true,
}
SuccessResponse(c, response)
return
}
// 获取文件类型
fileType := h.getFileType(header.Filename)
mimeType := header.Header.Get("Content-Type")
// 获取是否公开
isPublic := true
if isPublicStr := c.PostForm("is_public"); isPublicStr != "" {
if isPublicBool, err := strconv.ParseBool(isPublicStr); err == nil {
isPublic = isPublicBool
}
}
// 构建访问URL使用绝对路径不包含域名
accessURL := fmt.Sprintf("/uploads/%s/%s", yearMonth, fileName)
// 创建文件记录
fileEntity := &entity.File{
OriginalName: header.Filename,
FileName: fileName,
FilePath: filePath,
FileSize: header.Size,
FileType: fileType,
MimeType: mimeType,
FileHash: fileHash,
AccessURL: accessURL,
UserID: currentUser.ID,
Status: entity.FileStatusActive,
IsPublic: isPublic,
IsDeleted: false,
}
// 保存到数据库
if err := h.fileRepo.Create(fileEntity); err != nil {
// 删除已上传的文件
os.Remove(filePath)
ErrorResponse(c, "保存文件记录失败", http.StatusInternalServerError)
return
}
// 返回响应
response := dto.FileUploadResponse{
File: converter.FileToResponse(fileEntity),
Message: "文件上传成功",
Success: true,
}
SuccessResponse(c, response)
}
// GetFileList 获取文件列表
func (h *FileHandler) GetFileList(c *gin.Context) {
var req dto.FileListRequest
if err := c.ShouldBindQuery(&req); err != nil {
ErrorResponse(c, "请求参数错误", http.StatusBadRequest)
return
}
// 设置默认值
if req.Page <= 0 {
req.Page = 1
}
if req.PageSize <= 0 {
req.PageSize = 20
}
// 添加调试日志
utils.Info("文件列表请求参数: page=%d, pageSize=%d, search='%s', fileType='%s', status='%s', userID=%d",
req.Page, req.PageSize, req.Search, req.FileType, req.Status, req.UserID)
// 获取当前用户ID和角色现在总是有认证
userID := c.GetUint("user_id")
role := c.GetString("role")
utils.Info("GetFileList - 用户ID: %d, 角色: %s", userID, role)
// 根据用户角色决定查询范围
var files []entity.File
var total int64
var err error
if role == "admin" {
// 管理员可以查看所有文件
files, total, err = h.fileRepo.SearchFiles(req.Search, req.FileType, req.Status, req.UserID, req.Page, req.PageSize)
} else {
// 普通用户只能查看自己的文件
files, total, err = h.fileRepo.SearchFiles(req.Search, req.FileType, req.Status, userID, req.Page, req.PageSize)
}
if err != nil {
ErrorResponse(c, "获取文件列表失败", http.StatusInternalServerError)
return
}
response := converter.FileListToResponse(files, total, req.Page, req.PageSize)
SuccessResponse(c, response)
}
// DeleteFiles 删除文件
func (h *FileHandler) DeleteFiles(c *gin.Context) {
var req dto.FileDeleteRequest
if err := c.ShouldBindJSON(&req); err != nil {
ErrorResponse(c, "请求参数错误", http.StatusBadRequest)
return
}
// 获取当前用户ID和角色
userIDInterface, exists := c.Get("user_id")
roleInterface, _ := c.Get("role")
if !exists {
ErrorResponse(c, "用户未登录", http.StatusUnauthorized)
return
}
userID := userIDInterface.(uint)
role := ""
if roleInterface != nil {
role = roleInterface.(string)
}
// 检查权限
if role != "admin" {
// 普通用户只能删除自己的文件
for _, id := range req.IDs {
file, err := h.fileRepo.FindByID(id)
if err != nil {
ErrorResponse(c, "文件不存在", http.StatusNotFound)
return
}
if file.UserID != userID {
ErrorResponse(c, "没有权限删除此文件", http.StatusForbidden)
return
}
}
}
// 获取要删除的文件信息
var filesToDelete []entity.File
for _, id := range req.IDs {
file, err := h.fileRepo.FindByID(id)
if err != nil {
ErrorResponse(c, "文件不存在", http.StatusNotFound)
return
}
filesToDelete = append(filesToDelete, *file)
}
// 删除本地文件
for _, file := range filesToDelete {
if err := os.Remove(file.FilePath); err != nil {
utils.Error("删除本地文件失败: %s, 错误: %v", file.FilePath, err)
// 继续删除其他文件,不因为单个文件删除失败而中断
}
}
// 删除数据库记录
for _, id := range req.IDs {
if err := h.fileRepo.Delete(id); err != nil {
utils.Error("删除文件记录失败: ID=%d, 错误: %v", id, err)
// 继续删除其他文件,不因为单个文件删除失败而中断
}
}
SuccessResponse(c, gin.H{"message": "文件删除成功"})
}
// UpdateFile 更新文件信息
func (h *FileHandler) UpdateFile(c *gin.Context) {
var req dto.FileUpdateRequest
if err := c.ShouldBindJSON(&req); err != nil {
ErrorResponse(c, "请求参数错误", http.StatusBadRequest)
return
}
// 获取当前用户ID和角色
userIDInterface, exists := c.Get("user_id")
roleInterface, _ := c.Get("role")
if !exists {
ErrorResponse(c, "用户未登录", http.StatusUnauthorized)
return
}
userID := userIDInterface.(uint)
role := ""
if roleInterface != nil {
role = roleInterface.(string)
}
// 查找文件
file, err := h.fileRepo.FindByID(req.ID)
if err != nil {
ErrorResponse(c, "文件不存在", http.StatusNotFound)
return
}
// 检查权限
if role != "admin" && userID != file.UserID {
ErrorResponse(c, "没有权限修改此文件", http.StatusForbidden)
return
}
// 更新文件信息
if req.IsPublic != nil {
if err := h.fileRepo.UpdateFilePublic(req.ID, *req.IsPublic); err != nil {
ErrorResponse(c, "更新文件状态失败", http.StatusInternalServerError)
return
}
}
if req.Status != "" {
if err := h.fileRepo.UpdateFileStatus(req.ID, req.Status); err != nil {
ErrorResponse(c, "更新文件状态失败", http.StatusInternalServerError)
return
}
}
SuccessResponse(c, gin.H{"message": "文件更新成功"})
}
// generateRandomFileName 生成随机文件名
func (h *FileHandler) generateRandomFileName(originalName string) string {
// 获取文件扩展名
ext := filepath.Ext(originalName)
// 生成随机字符串
bytes := make([]byte, 16)
rand.Read(bytes)
randomStr := fmt.Sprintf("%x", bytes)
// 添加时间戳
timestamp := time.Now().Unix()
return fmt.Sprintf("%d_%s%s", timestamp, randomStr, ext)
}
// getFileType 获取文件类型
func (h *FileHandler) getFileType(filename string) string {
ext := strings.ToLower(filepath.Ext(filename))
// 图片类型
imageExts := []string{".jpg", ".jpeg", ".png", ".gif", ".bmp", ".webp", ".svg"}
for _, imgExt := range imageExts {
if ext == imgExt {
return "image"
}
}
return "image" // 默认返回image因为只支持图片格式
}
// calculateFileHash 计算文件哈希值
func (h *FileHandler) calculateFileHash(filePath string) (string, error) {
file, err := os.Open(filePath)
if err != nil {
return "", err
}
defer file.Close()
hash := sha256.New()
if _, err := io.Copy(hash, file); err != nil {
return "", err
}
return fmt.Sprintf("%x", hash.Sum(nil)), nil
}

View File

@@ -0,0 +1,270 @@
package handlers
import (
"net/http"
"strconv"
"github.com/ctwj/urldb/db/converter"
"github.com/ctwj/urldb/services"
"github.com/ctwj/urldb/utils"
"github.com/gin-gonic/gin"
)
// MeilisearchHandler Meilisearch处理器
type MeilisearchHandler struct {
meilisearchManager *services.MeilisearchManager
}
// NewMeilisearchHandler 创建Meilisearch处理器
func NewMeilisearchHandler(meilisearchManager *services.MeilisearchManager) *MeilisearchHandler {
return &MeilisearchHandler{
meilisearchManager: meilisearchManager,
}
}
// TestConnection 测试Meilisearch连接
func (h *MeilisearchHandler) TestConnection(c *gin.Context) {
var req struct {
Host string `json:"host"`
Port interface{} `json:"port"` // 支持字符串或数字
MasterKey string `json:"masterKey"`
IndexName string `json:"indexName"` // 可选字段
}
if err := c.ShouldBindJSON(&req); err != nil {
ErrorResponse(c, "请求参数错误", http.StatusBadRequest)
return
}
// 验证必要字段
if req.Host == "" {
ErrorResponse(c, "主机地址不能为空", http.StatusBadRequest)
return
}
// 转换port为字符串
var portStr string
switch v := req.Port.(type) {
case string:
portStr = v
case float64:
portStr = strconv.Itoa(int(v))
case int:
portStr = strconv.Itoa(v)
default:
portStr = "7700" // 默认端口
}
// 如果没有提供索引名称,使用默认值
indexName := req.IndexName
if indexName == "" {
indexName = "resources"
}
// 创建临时服务进行测试
service := services.NewMeilisearchService(req.Host, portStr, req.MasterKey, indexName, true)
if err := service.HealthCheck(); err != nil {
ErrorResponse(c, "连接测试失败: "+err.Error(), http.StatusBadRequest)
return
}
SuccessResponse(c, gin.H{"message": "连接测试成功"})
}
// GetStatus 获取Meilisearch状态
func (h *MeilisearchHandler) GetStatus(c *gin.Context) {
if h.meilisearchManager == nil {
SuccessResponse(c, gin.H{
"enabled": false,
"healthy": false,
"message": "Meilisearch未初始化",
})
return
}
status, err := h.meilisearchManager.GetStatusWithHealthCheck()
if err != nil {
ErrorResponse(c, "获取状态失败: "+err.Error(), http.StatusInternalServerError)
return
}
SuccessResponse(c, status)
}
// GetUnsyncedCount 获取未同步资源数量
func (h *MeilisearchHandler) GetUnsyncedCount(c *gin.Context) {
if h.meilisearchManager == nil {
SuccessResponse(c, gin.H{"count": 0})
return
}
count, err := h.meilisearchManager.GetUnsyncedCount()
if err != nil {
ErrorResponse(c, "获取未同步数量失败: "+err.Error(), http.StatusInternalServerError)
return
}
SuccessResponse(c, gin.H{"count": count})
}
// GetUnsyncedResources 获取未同步的资源
func (h *MeilisearchHandler) GetUnsyncedResources(c *gin.Context) {
if h.meilisearchManager == nil {
SuccessResponse(c, gin.H{
"resources": []interface{}{},
"total": 0,
})
return
}
page, _ := strconv.Atoi(c.DefaultQuery("page", "1"))
pageSize, _ := strconv.Atoi(c.DefaultQuery("page_size", "20"))
resources, total, err := h.meilisearchManager.GetUnsyncedResources(page, pageSize)
if err != nil {
ErrorResponse(c, "获取未同步资源失败: "+err.Error(), http.StatusInternalServerError)
return
}
SuccessResponse(c, gin.H{
"resources": converter.ToResourceResponseList(resources),
"total": total,
"page": page,
"page_size": pageSize,
})
}
// GetSyncedResources 获取已同步的资源
func (h *MeilisearchHandler) GetSyncedResources(c *gin.Context) {
if h.meilisearchManager == nil {
SuccessResponse(c, gin.H{
"resources": []interface{}{},
"total": 0,
})
return
}
page, _ := strconv.Atoi(c.DefaultQuery("page", "1"))
pageSize, _ := strconv.Atoi(c.DefaultQuery("page_size", "20"))
resources, total, err := h.meilisearchManager.GetSyncedResources(page, pageSize)
if err != nil {
ErrorResponse(c, "获取已同步资源失败: "+err.Error(), http.StatusInternalServerError)
return
}
SuccessResponse(c, gin.H{
"resources": converter.ToResourceResponseList(resources),
"total": total,
"page": page,
"page_size": pageSize,
})
}
// GetAllResources 获取所有资源
func (h *MeilisearchHandler) GetAllResources(c *gin.Context) {
if h.meilisearchManager == nil {
SuccessResponse(c, gin.H{
"resources": []interface{}{},
"total": 0,
})
return
}
page, _ := strconv.Atoi(c.DefaultQuery("page", "1"))
pageSize, _ := strconv.Atoi(c.DefaultQuery("page_size", "20"))
resources, total, err := h.meilisearchManager.GetAllResources(page, pageSize)
if err != nil {
ErrorResponse(c, "获取所有资源失败: "+err.Error(), http.StatusInternalServerError)
return
}
SuccessResponse(c, gin.H{
"resources": converter.ToResourceResponseList(resources),
"total": total,
"page": page,
"page_size": pageSize,
})
}
// SyncAllResources 同步所有资源
func (h *MeilisearchHandler) SyncAllResources(c *gin.Context) {
if h.meilisearchManager == nil {
ErrorResponse(c, "Meilisearch未初始化", http.StatusInternalServerError)
return
}
utils.Info("开始同步所有资源到Meilisearch...")
_, err := h.meilisearchManager.SyncAllResources()
if err != nil {
ErrorResponse(c, "同步失败: "+err.Error(), http.StatusInternalServerError)
return
}
SuccessResponse(c, gin.H{
"message": "同步已开始,请查看进度",
})
}
// GetSyncProgress 获取同步进度
func (h *MeilisearchHandler) GetSyncProgress(c *gin.Context) {
if h.meilisearchManager == nil {
ErrorResponse(c, "Meilisearch未初始化", http.StatusInternalServerError)
return
}
progress := h.meilisearchManager.GetSyncProgress()
SuccessResponse(c, progress)
}
// StopSync 停止同步
func (h *MeilisearchHandler) StopSync(c *gin.Context) {
if h.meilisearchManager == nil {
ErrorResponse(c, "Meilisearch未初始化", http.StatusInternalServerError)
return
}
h.meilisearchManager.StopSync()
SuccessResponse(c, gin.H{
"message": "同步已停止",
})
}
// ClearIndex 清空索引
func (h *MeilisearchHandler) ClearIndex(c *gin.Context) {
if h.meilisearchManager == nil {
ErrorResponse(c, "Meilisearch未初始化", http.StatusInternalServerError)
return
}
if err := h.meilisearchManager.ClearIndex(); err != nil {
ErrorResponse(c, "清空索引失败: "+err.Error(), http.StatusInternalServerError)
return
}
SuccessResponse(c, gin.H{"message": "清空索引成功"})
}
// UpdateIndexSettings 更新索引设置
func (h *MeilisearchHandler) UpdateIndexSettings(c *gin.Context) {
if h.meilisearchManager == nil {
ErrorResponse(c, "Meilisearch未初始化", http.StatusInternalServerError)
return
}
service := h.meilisearchManager.GetService()
if service == nil {
ErrorResponse(c, "Meilisearch服务未初始化", http.StatusInternalServerError)
return
}
if err := service.UpdateIndexSettings(); err != nil {
ErrorResponse(c, "更新索引设置失败: "+err.Error(), http.StatusInternalServerError)
return
}
SuccessResponse(c, gin.H{"message": "索引设置更新成功"})
}

View File

@@ -2,10 +2,12 @@ package handlers
import (
"strconv"
"strings"
"github.com/ctwj/urldb/db/dto"
"github.com/ctwj/urldb/db/entity"
"github.com/ctwj/urldb/utils"
"github.com/gin-gonic/gin"
)
@@ -17,6 +19,56 @@ func NewPublicAPIHandler() *PublicAPIHandler {
return &PublicAPIHandler{}
}
// filterForbiddenWords 过滤包含违禁词的资源
func (h *PublicAPIHandler) filterForbiddenWords(resources []entity.Resource) ([]entity.Resource, []string) {
// 获取违禁词配置
forbiddenWords, err := repoManager.SystemConfigRepository.GetConfigValue(entity.ConfigKeyForbiddenWords)
if err != nil {
// 如果获取失败,返回原资源列表
return resources, nil
}
if forbiddenWords == "" {
return resources, nil
}
// 分割违禁词
words := strings.Split(forbiddenWords, ",")
var filteredResources []entity.Resource
var foundForbiddenWords []string
for _, resource := range resources {
shouldSkip := false
title := strings.ToLower(resource.Title)
description := strings.ToLower(resource.Description)
for _, word := range words {
word = strings.TrimSpace(word)
if word != "" && (strings.Contains(title, strings.ToLower(word)) || strings.Contains(description, strings.ToLower(word))) {
foundForbiddenWords = append(foundForbiddenWords, word)
shouldSkip = true
break
}
}
if !shouldSkip {
filteredResources = append(filteredResources, resource)
}
}
// 去重违禁词
uniqueForbiddenWords := make([]string, 0)
wordMap := make(map[string]bool)
for _, word := range foundForbiddenWords {
if !wordMap[word] {
wordMap[word] = true
uniqueForbiddenWords = append(uniqueForbiddenWords, word)
}
}
return filteredResources, uniqueForbiddenWords
}
// AddBatchResources godoc
// @Summary 批量添加资源
// @Description 通过公开API批量添加多个资源到待处理列表
@@ -112,7 +164,7 @@ func (h *PublicAPIHandler) AddBatchResources(c *gin.Context) {
// SearchResources godoc
// @Summary 资源搜索
// @Description 搜索资源,支持关键词、标签、分类过滤
// @Description 搜索资源,支持关键词、标签、分类过滤,自动过滤包含违禁词的资源
// @Tags PublicAPI
// @Accept json
// @Produce json
@@ -122,7 +174,7 @@ func (h *PublicAPIHandler) AddBatchResources(c *gin.Context) {
// @Param category query string false "分类过滤"
// @Param page query int false "页码" default(1)
// @Param page_size query int false "每页数量" default(20) maximum(100)
// @Success 200 {object} map[string]interface{} "搜索成功"
// @Success 200 {object} map[string]interface{} "搜索成功如果存在违禁词过滤会返回forbidden_words_filtered字段"
// @Failure 401 {object} map[string]interface{} "认证失败"
// @Failure 500 {object} map[string]interface{} "服务器内部错误"
// @Router /api/public/resources/search [get]
@@ -131,6 +183,7 @@ func (h *PublicAPIHandler) SearchResources(c *gin.Context) {
keyword := c.Query("keyword")
tag := c.Query("tag")
category := c.Query("category")
panID := c.Query("pan_id")
pageStr := c.DefaultQuery("page", "1")
pageSizeStr := c.DefaultQuery("page_size", "20")
@@ -144,34 +197,99 @@ func (h *PublicAPIHandler) SearchResources(c *gin.Context) {
pageSize = 20
}
// 构建搜索条件
params := map[string]interface{}{
"page": page,
"page_size": pageSize,
var resources []entity.Resource
var total int64
// 如果启用了Meilisearch优先使用Meilisearch搜索
if meilisearchManager != nil && meilisearchManager.IsEnabled() {
// 构建过滤器
filters := make(map[string]interface{})
if category != "" {
filters["category"] = category
}
if tag != "" {
filters["tags"] = tag
}
if panID != "" {
if id, err := strconv.ParseUint(panID, 10, 32); err == nil {
// 根据pan_id获取pan_name
pan, err := repoManager.PanRepository.FindByID(uint(id))
if err == nil && pan != nil {
filters["pan_name"] = pan.Name
}
}
}
// 使用Meilisearch搜索
service := meilisearchManager.GetService()
if service != nil {
docs, docTotal, err := service.Search(keyword, filters, page, pageSize)
if err == nil {
// 将Meilisearch文档转换为Resource实体保持兼容性
for _, doc := range docs {
resource := entity.Resource{
ID: doc.ID,
Title: doc.Title,
Description: doc.Description,
URL: doc.URL,
SaveURL: doc.SaveURL,
FileSize: doc.FileSize,
Key: doc.Key,
PanID: doc.PanID,
CreatedAt: doc.CreatedAt,
UpdatedAt: doc.UpdatedAt,
}
resources = append(resources, resource)
}
total = docTotal
} else {
utils.Error("Meilisearch搜索失败回退到数据库搜索: %v", err)
}
}
}
if keyword != "" {
params["search"] = keyword
// 如果Meilisearch未启用或搜索失败使用数据库搜索
if meilisearchManager == nil || !meilisearchManager.IsEnabled() || err != nil {
// 构建搜索条件
params := map[string]interface{}{
"page": page,
"page_size": pageSize,
}
if keyword != "" {
params["search"] = keyword
}
if tag != "" {
params["tag"] = tag
}
if category != "" {
params["category"] = category
}
if panID != "" {
if id, err := strconv.ParseUint(panID, 10, 32); err == nil {
params["pan_id"] = uint(id)
}
}
// 执行数据库搜索
resources, total, err = repoManager.ResourceRepository.SearchWithFilters(params)
if err != nil {
ErrorResponse(c, "搜索失败: "+err.Error(), 500)
return
}
}
if tag != "" {
params["tag"] = tag
}
// 过滤违禁词
filteredResources, foundForbiddenWords := h.filterForbiddenWords(resources)
if category != "" {
params["category"] = category
}
// 执行搜索
resources, total, err := repoManager.ResourceRepository.SearchWithFilters(params)
if err != nil {
ErrorResponse(c, "搜索失败: "+err.Error(), 500)
return
}
// 计算过滤后的总数
filteredTotal := len(filteredResources)
// 转换为响应格式
var resourceResponses []gin.H
for _, resource := range resources {
for _, resource := range filteredResources {
resourceResponses = append(resourceResponses, gin.H{
"id": resource.ID,
"title": resource.Title,
@@ -183,12 +301,23 @@ func (h *PublicAPIHandler) SearchResources(c *gin.Context) {
})
}
SuccessResponse(c, gin.H{
"list": resourceResponses,
"total": total,
"page": page,
"limit": pageSize,
})
// 构建响应数据
responseData := gin.H{
"data": resourceResponses,
"total": filteredTotal,
"page": page,
"page_size": pageSize,
}
// 如果存在违禁词过滤,添加提醒字段
if len(foundForbiddenWords) > 0 {
responseData["forbidden_words_filtered"] = true
responseData["filtered_forbidden_words"] = foundForbiddenWords
responseData["original_total"] = total
responseData["filtered_count"] = total - int64(filteredTotal)
}
SuccessResponse(c, responseData)
}
// GetHotDramas godoc

View File

@@ -316,6 +316,7 @@ func GetReadyResourcesWithErrors(c *gin.Context) {
// 获取分页参数
pageStr := c.DefaultQuery("page", "1")
pageSizeStr := c.DefaultQuery("page_size", "100")
errorFilter := c.Query("error_filter")
page, err := strconv.Atoi(pageStr)
if err != nil || page < 1 {
@@ -327,8 +328,8 @@ func GetReadyResourcesWithErrors(c *gin.Context) {
pageSize = 100
}
// 获取有错误的资源
resources, err := repoManager.ReadyResourceRepository.FindWithErrors()
// 获取有错误的资源(分页,包括软删除的)
resources, total, err := repoManager.ReadyResourceRepository.FindWithErrorsPaginatedIncludingDeleted(page, pageSize, errorFilter)
if err != nil {
ErrorResponse(c, err.Error(), http.StatusInternalServerError)
return
@@ -385,7 +386,7 @@ func GetReadyResourcesWithErrors(c *gin.Context) {
"data": responses,
"page": page,
"page_size": pageSize,
"total": len(resources),
"total": total,
"count": len(resources),
"error_stats": errorTypeStats,
"retryable_count": getRetryableErrorCount(resources),
@@ -465,3 +466,109 @@ func RetryFailedResources(c *gin.Context) {
"retryable_count": getRetryableErrorCount(resources),
})
}
// BatchRestoreToReadyPool 批量将失败资源重新放入待处理池
func BatchRestoreToReadyPool(c *gin.Context) {
var req struct {
IDs []uint `json:"ids" binding:"required"`
}
if err := c.ShouldBindJSON(&req); err != nil {
ErrorResponse(c, "请求参数错误: "+err.Error(), http.StatusBadRequest)
return
}
if len(req.IDs) == 0 {
ErrorResponse(c, "资源ID列表不能为空", http.StatusBadRequest)
return
}
successCount := 0
failedCount := 0
for _, id := range req.IDs {
// 清除错误信息并恢复软删除的资源
err := repoManager.ReadyResourceRepository.ClearErrorMsgAndRestore(id)
if err != nil {
failedCount++
continue
}
successCount++
}
SuccessResponse(c, gin.H{
"message": "批量重新放入待处理池操作完成",
"total_count": len(req.IDs),
"success_count": successCount,
"failed_count": failedCount,
})
}
// BatchRestoreToReadyPoolByQuery 根据查询条件批量将失败资源重新放入待处理池
func BatchRestoreToReadyPoolByQuery(c *gin.Context) {
var req struct {
ErrorFilter string `json:"error_filter"`
}
if err := c.ShouldBindJSON(&req); err != nil {
ErrorResponse(c, "请求参数错误: "+err.Error(), http.StatusBadRequest)
return
}
// 根据查询条件获取所有符合条件的资源
resources, err := repoManager.ReadyResourceRepository.FindWithErrorsByQuery(req.ErrorFilter)
if err != nil {
ErrorResponse(c, "查询资源失败: "+err.Error(), http.StatusInternalServerError)
return
}
if len(resources) == 0 {
SuccessResponse(c, gin.H{
"message": "没有找到符合条件的资源",
"total_count": 0,
"success_count": 0,
"failed_count": 0,
})
return
}
successCount := 0
failedCount := 0
for _, resource := range resources {
err := repoManager.ReadyResourceRepository.ClearErrorMsgAndRestore(resource.ID)
if err != nil {
failedCount++
continue
}
successCount++
}
SuccessResponse(c, gin.H{
"message": "批量重新放入待处理池操作完成",
"total_count": len(resources),
"success_count": successCount,
"failed_count": failedCount,
})
}
// ClearAllErrorsByQuery 根据查询条件批量清除错误信息并删除资源
func ClearAllErrorsByQuery(c *gin.Context) {
var req struct {
ErrorFilter string `json:"error_filter"`
}
if err := c.ShouldBindJSON(&req); err != nil {
ErrorResponse(c, "请求参数错误: "+err.Error(), http.StatusBadRequest)
return
}
// 根据查询条件批量删除失败资源
affectedRows, err := repoManager.ReadyResourceRepository.ClearAllErrorsByQuery(req.ErrorFilter)
if err != nil {
ErrorResponse(c, "批量删除失败资源失败: "+err.Error(), http.StatusInternalServerError)
return
}
SuccessResponse(c, gin.H{
"message": "批量删除失败资源操作完成",
"affected_rows": affectedRows,
})
}

View File

@@ -1,12 +1,16 @@
package handlers
import (
"fmt"
"net/http"
"strconv"
pan "github.com/ctwj/urldb/common"
commonutils "github.com/ctwj/urldb/common/utils"
"github.com/ctwj/urldb/db/converter"
"github.com/ctwj/urldb/db/dto"
"github.com/ctwj/urldb/db/entity"
"github.com/ctwj/urldb/utils"
"github.com/gin-gonic/gin"
)
@@ -16,6 +20,12 @@ func GetResources(c *gin.Context) {
page, _ := strconv.Atoi(c.DefaultQuery("page", "1"))
pageSize, _ := strconv.Atoi(c.DefaultQuery("page_size", "20"))
utils.Info("资源列表请求 - page: %d, pageSize: %d, User-Agent: %s", page, pageSize, c.GetHeader("User-Agent"))
// 添加缓存控制头,优化 SSR 性能
c.Header("Cache-Control", "public, max-age=30") // 30秒缓存平衡性能和实时性
c.Header("ETag", fmt.Sprintf("resources-%d-%d-%s-%s", page, pageSize, c.Query("search"), c.Query("pan_id")))
params := map[string]interface{}{
"page": page,
"page_size": pageSize,
@@ -30,21 +40,75 @@ func GetResources(c *gin.Context) {
}
}
if categoryID := c.Query("category_id"); categoryID != "" {
utils.Info("收到分类ID参数: %s", categoryID)
if id, err := strconv.ParseUint(categoryID, 10, 32); err == nil {
params["category_id"] = uint(id)
utils.Info("解析分类ID成功: %d", uint(id))
} else {
utils.Error("解析分类ID失败: %v", err)
}
}
if hasSaveURL := c.Query("has_save_url"); hasSaveURL != "" {
if hasSaveURL == "true" {
params["has_save_url"] = true
} else if hasSaveURL == "false" {
params["has_save_url"] = false
}
}
if noSaveURL := c.Query("no_save_url"); noSaveURL != "" {
if noSaveURL == "true" {
params["no_save_url"] = true
}
}
if panName := c.Query("pan_name"); panName != "" {
params["pan_name"] = panName
}
var resources []entity.Resource
var total int64
var err error
// 如果有搜索关键词且启用了Meilisearch优先使用Meilisearch搜索
if search := c.Query("search"); search != "" && meilisearchManager != nil && meilisearchManager.IsEnabled() {
// 构建Meilisearch过滤器
filters := make(map[string]interface{})
if panID := c.Query("pan_id"); panID != "" {
if id, err := strconv.ParseUint(panID, 10, 32); err == nil {
// 直接使用pan_id进行过滤
filters["pan_id"] = id
}
}
// 使用Meilisearch搜索
service := meilisearchManager.GetService()
if service != nil {
docs, docTotal, err := service.Search(search, filters, page, pageSize)
if err == nil {
// 将Meilisearch文档转换为ResourceResponse包含高亮信息
var resourceResponses []dto.ResourceResponse
for _, doc := range docs {
resourceResponse := converter.ToResourceResponseFromMeilisearch(doc)
resourceResponses = append(resourceResponses, resourceResponse)
}
// 返回Meilisearch搜索结果包含高亮信息
SuccessResponse(c, gin.H{
"data": resourceResponses,
"total": docTotal,
"page": page,
"page_size": pageSize,
"source": "meilisearch",
})
return
} else {
utils.Error("Meilisearch搜索失败回退到数据库搜索: %v", err)
}
}
}
resources, total, err := repoManager.ResourceRepository.SearchWithFilters(params)
// 搜索统计(仅非管理员)
if search, ok := params["search"].(string); ok && search != "" {
user, _ := c.Get("user")
if user == nil || (user != nil && user.(entity.User).Role != "admin") {
ip := c.ClientIP()
userAgent := c.GetHeader("User-Agent")
repoManager.SearchStatRepository.RecordSearch(search, ip, userAgent)
}
// 如果Meilisearch未启用、搜索失败或没有搜索关键词使用数据库搜索
if meilisearchManager == nil || !meilisearchManager.IsEnabled() || len(resources) == 0 {
resources, total, err = repoManager.ResourceRepository.SearchWithFilters(params)
}
if err != nil {
@@ -145,6 +209,15 @@ func CreateResource(c *gin.Context) {
}
}
// 同步到Meilisearch
if meilisearchManager != nil {
go func() {
if err := meilisearchManager.SyncResourceToMeilisearch(resource); err != nil {
utils.Error("同步资源到Meilisearch失败: %v", err)
}
}()
}
SuccessResponse(c, gin.H{
"message": "资源创建成功",
"resource": converter.ToResourceResponse(resource),
@@ -221,6 +294,15 @@ func UpdateResource(c *gin.Context) {
}
}
// 同步到Meilisearch
if meilisearchManager != nil {
go func() {
if err := meilisearchManager.SyncResourceToMeilisearch(resource); err != nil {
utils.Error("同步资源到Meilisearch失败: %v", err)
}
}()
}
SuccessResponse(c, gin.H{"message": "资源更新成功"})
}
@@ -252,16 +334,53 @@ func SearchResources(c *gin.Context) {
var total int64
var err error
if query == "" {
// 搜索关键词为空时,返回最新记录(分页)
resources, total, err = repoManager.ResourceRepository.FindWithRelationsPaginated(page, pageSize)
} else {
// 有搜索关键词时,执行搜索
resources, total, err = repoManager.ResourceRepository.Search(query, nil, page, pageSize)
// 新增:记录搜索关键词
ip := c.ClientIP()
userAgent := c.GetHeader("User-Agent")
repoManager.SearchStatRepository.RecordSearch(query, ip, userAgent)
// 如果启用了Meilisearch优先使用Meilisearch搜索
if meilisearchManager != nil && meilisearchManager.IsEnabled() {
// 构建过滤器
filters := make(map[string]interface{})
if categoryID := c.Query("category_id"); categoryID != "" {
if id, err := strconv.ParseUint(categoryID, 10, 32); err == nil {
filters["category"] = uint(id)
}
}
// 使用Meilisearch搜索
service := meilisearchManager.GetService()
if service != nil {
docs, docTotal, err := service.Search(query, filters, page, pageSize)
if err == nil {
// 将Meilisearch文档转换为Resource实体
for _, doc := range docs {
resource := entity.Resource{
ID: doc.ID,
Title: doc.Title,
Description: doc.Description,
URL: doc.URL,
SaveURL: doc.SaveURL,
FileSize: doc.FileSize,
Key: doc.Key,
PanID: doc.PanID,
CreatedAt: doc.CreatedAt,
UpdatedAt: doc.UpdatedAt,
}
resources = append(resources, resource)
}
total = docTotal
} else {
utils.Error("Meilisearch搜索失败回退到数据库搜索: %v", err)
}
}
}
// 如果Meilisearch未启用或搜索失败使用数据库搜索
if meilisearchManager == nil || !meilisearchManager.IsEnabled() || err != nil {
if query == "" {
// 搜索关键词为空时,返回最新记录(分页)
resources, total, err = repoManager.ResourceRepository.FindWithRelationsPaginated(page, pageSize)
} else {
// 有搜索关键词时,执行搜索
resources, total, err = repoManager.ResourceRepository.Search(query, nil, page, pageSize)
}
}
if err != nil {
@@ -285,11 +404,23 @@ func IncrementResourceViewCount(c *gin.Context) {
ErrorResponse(c, "无效的资源ID", http.StatusBadRequest)
return
}
// 增加资源访问量
err = repoManager.ResourceRepository.IncrementViewCount(uint(id))
if err != nil {
ErrorResponse(c, "增加浏览次数失败", http.StatusInternalServerError)
return
}
// 记录访问记录
ipAddress := c.ClientIP()
userAgent := c.GetHeader("User-Agent")
err = repoManager.ResourceViewRepository.RecordView(uint(id), ipAddress, userAgent)
if err != nil {
// 记录访问失败不影响主要功能,只记录日志
utils.Error("记录资源访问失败: %v", err)
}
SuccessResponse(c, gin.H{"message": "浏览次数+1"})
}
@@ -310,3 +441,305 @@ func BatchDeleteResources(c *gin.Context) {
}
SuccessResponse(c, gin.H{"deleted": count, "message": "批量删除成功"})
}
// GetResourceLink 获取资源链接(智能转存)
func GetResourceLink(c *gin.Context) {
// 获取资源ID
resourceIDStr := c.Param("id")
resourceID, err := strconv.ParseUint(resourceIDStr, 10, 32)
if err != nil {
ErrorResponse(c, "无效的资源ID", http.StatusBadRequest)
return
}
utils.Info("获取资源链接请求 - resourceID: %d", resourceID)
// 查询资源信息
resource, err := repoManager.ResourceRepository.FindByID(uint(resourceID))
if err != nil {
utils.Error("查询资源失败: %v", err)
ErrorResponse(c, "资源不存在", http.StatusNotFound)
return
}
// 查询平台信息
var panInfo entity.Pan
if resource.PanID != nil {
panPtr, err := repoManager.PanRepository.FindByID(*resource.PanID)
if err != nil {
utils.Error("查询平台信息失败: %v", err)
} else if panPtr != nil {
panInfo = *panPtr
}
}
utils.Info("资源信息 - 平台: %s, 原始链接: %s, 转存链接: %s", panInfo.Name, resource.URL, resource.SaveURL)
// 统计访问次数
err = repoManager.ResourceRepository.IncrementViewCount(uint(resourceID))
if err != nil {
utils.Error("增加资源访问量失败: %v", err)
}
// 记录访问记录
ipAddress := c.ClientIP()
userAgent := c.GetHeader("User-Agent")
err = repoManager.ResourceViewRepository.RecordView(uint(resourceID), ipAddress, userAgent)
if err != nil {
utils.Error("记录资源访问失败: %v", err)
}
// 如果不是夸克网盘,直接返回原链接
if panInfo.Name != "quark" {
utils.Info("非夸克资源,直接返回原链接")
SuccessResponse(c, gin.H{
"url": resource.URL,
"type": "original",
"platform": panInfo.Remark,
"resource_id": resource.ID,
})
return
}
// 夸克资源处理逻辑
utils.Info("夸克资源处理开始")
// 如果已存在转存链接,直接返回
if resource.SaveURL != "" {
utils.Info("已存在转存链接,直接返回: %s", resource.SaveURL)
SuccessResponse(c, gin.H{
"url": resource.SaveURL,
"type": "transferred",
"platform": panInfo.Remark,
"resource_id": resource.ID,
})
return
}
// 检查是否开启自动转存
autoTransferEnabled, err := repoManager.SystemConfigRepository.GetConfigBool(entity.ConfigKeyAutoTransferEnabled)
if err != nil {
utils.Error("获取自动转存配置失败: %v", err)
// 配置获取失败,返回原链接
SuccessResponse(c, gin.H{
"url": resource.URL,
"type": "original",
"platform": panInfo.Remark,
"resource_id": resource.ID,
"message": "",
})
return
}
if !autoTransferEnabled {
utils.Info("自动转存功能未开启,返回原链接")
SuccessResponse(c, gin.H{
"url": resource.URL,
"type": "original",
"platform": panInfo.Remark,
"resource_id": resource.ID,
"message": "",
})
return
}
// 执行自动转存
utils.Info("开始执行自动转存")
transferResult := performAutoTransfer(resource)
if transferResult.Success {
utils.Info("自动转存成功,返回转存链接: %s", transferResult.SaveURL)
SuccessResponse(c, gin.H{
"url": transferResult.SaveURL,
"type": "transferred",
"platform": panInfo.Remark,
"resource_id": resource.ID,
"message": "资源易和谐,请及时用手机夸克扫码转存",
})
} else {
utils.Error("自动转存失败: %s", transferResult.ErrorMsg)
SuccessResponse(c, gin.H{
"url": resource.URL,
"type": "original",
"platform": panInfo.Remark,
"resource_id": resource.ID,
"message": "",
})
}
}
// TransferResult 转存结果
type TransferResult struct {
Success bool `json:"success"`
SaveURL string `json:"save_url"`
ErrorMsg string `json:"error_msg"`
}
// performAutoTransfer 执行自动转存
func performAutoTransfer(resource *entity.Resource) TransferResult {
utils.Info("开始执行资源转存 - ID: %d, URL: %s", resource.ID, resource.URL)
// 获取夸克平台ID
quarkPanID, err := getQuarkPanID()
if err != nil {
utils.Error("获取夸克平台ID失败: %v", err)
return TransferResult{
Success: false,
ErrorMsg: fmt.Sprintf("获取夸克平台ID失败: %v", err),
}
}
// 获取可用的夸克账号
accounts, err := repoManager.CksRepository.FindAll()
if err != nil {
utils.Error("获取网盘账号失败: %v", err)
return TransferResult{
Success: false,
ErrorMsg: fmt.Sprintf("获取网盘账号失败: %v", err),
}
}
// 获取最小存储空间配置
autoTransferMinSpace, err := repoManager.SystemConfigRepository.GetConfigInt(entity.ConfigKeyAutoTransferMinSpace)
if err != nil {
utils.Error("获取最小存储空间配置失败: %v", err)
autoTransferMinSpace = 5 // 默认5GB
}
// 过滤:只保留已激活、夸克平台、剩余空间足够的账号
minSpaceBytes := int64(autoTransferMinSpace) * 1024 * 1024 * 1024
var validAccounts []entity.Cks
for _, acc := range accounts {
if acc.IsValid && acc.PanID == quarkPanID && acc.LeftSpace >= minSpaceBytes {
validAccounts = append(validAccounts, acc)
}
}
if len(validAccounts) == 0 {
utils.Info("没有可用的夸克网盘账号")
return TransferResult{
Success: false,
ErrorMsg: "没有可用的夸克网盘账号",
}
}
utils.Info("找到 %d 个可用夸克网盘账号,开始转存处理...", len(validAccounts))
// 使用第一个可用账号进行转存
account := validAccounts[0]
// 创建网盘服务工厂
factory := pan.NewPanFactory()
// 执行转存
result := transferSingleResource(resource, account, factory)
if result.Success {
// 更新资源的转存信息
resource.SaveURL = result.SaveURL
resource.ErrorMsg = ""
if err := repoManager.ResourceRepository.Update(resource); err != nil {
utils.Error("更新资源转存信息失败: %v", err)
}
} else {
// 更新错误信息
resource.ErrorMsg = result.ErrorMsg
if err := repoManager.ResourceRepository.Update(resource); err != nil {
utils.Error("更新资源错误信息失败: %v", err)
}
}
return result
}
// transferSingleResource 转存单个资源
func transferSingleResource(resource *entity.Resource, account entity.Cks, factory *pan.PanFactory) TransferResult {
utils.Info("开始转存资源 - 资源ID: %d, 账号: %s", resource.ID, account.Username)
service, err := factory.CreatePanService(resource.URL, &pan.PanConfig{
URL: resource.URL,
ExpiredType: 0,
IsType: 0,
Cookie: account.Ck,
})
if err != nil {
utils.Error("创建网盘服务失败: %v", err)
return TransferResult{
Success: false,
ErrorMsg: fmt.Sprintf("创建网盘服务失败: %v", err),
}
}
// 提取分享ID
shareID, _ := commonutils.ExtractShareIdString(resource.URL)
if shareID == "" {
return TransferResult{
Success: false,
ErrorMsg: "无效的分享链接",
}
}
// 执行转存
transferResult, err := service.Transfer(shareID)
if err != nil {
utils.Error("转存失败: %v", err)
return TransferResult{
Success: false,
ErrorMsg: fmt.Sprintf("转存失败: %v", err),
}
}
if transferResult == nil || !transferResult.Success {
errMsg := "转存失败"
if transferResult != nil && transferResult.Message != "" {
errMsg = transferResult.Message
}
utils.Error("转存失败: %s", errMsg)
return TransferResult{
Success: false,
ErrorMsg: errMsg,
}
}
// 提取转存链接
var saveURL string
if data, ok := transferResult.Data.(map[string]interface{}); ok {
if v, ok := data["shareUrl"]; ok {
saveURL, _ = v.(string)
}
}
if saveURL == "" {
saveURL = transferResult.ShareURL
}
if saveURL == "" {
return TransferResult{
Success: false,
ErrorMsg: "转存成功但未获取到分享链接",
}
}
utils.Info("转存成功 - 资源ID: %d, 转存链接: %s", resource.ID, saveURL)
return TransferResult{
Success: true,
SaveURL: saveURL,
}
}
// getQuarkPanID 获取夸克网盘ID
func getQuarkPanID() (uint, error) {
// 通过FindAll方法查找所有平台然后过滤出quark平台
pans, err := repoManager.PanRepository.FindAll()
if err != nil {
return 0, fmt.Errorf("查询平台信息失败: %v", err)
}
for _, p := range pans {
if p.Name == "quark" {
return p.ID, nil
}
}
return 0, fmt.Errorf("未找到quark平台")
}

View File

@@ -3,13 +3,13 @@ package handlers
import (
"net/http"
"github.com/ctwj/urldb/utils"
"github.com/ctwj/urldb/scheduler"
"github.com/gin-gonic/gin"
)
// GetSchedulerStatus 获取调度器状态
func GetSchedulerStatus(c *gin.Context) {
scheduler := utils.GetGlobalScheduler(
scheduler := scheduler.GetGlobalScheduler(
repoManager.HotDramaRepository,
repoManager.ReadyResourceRepository,
repoManager.ResourceRepository,
@@ -31,7 +31,7 @@ func GetSchedulerStatus(c *gin.Context) {
// 启动热播剧定时任务
func StartHotDramaScheduler(c *gin.Context) {
scheduler := utils.GetGlobalScheduler(
scheduler := scheduler.GetGlobalScheduler(
repoManager.HotDramaRepository,
repoManager.ReadyResourceRepository,
repoManager.ResourceRepository,
@@ -51,7 +51,7 @@ func StartHotDramaScheduler(c *gin.Context) {
// 停止热播剧定时任务
func StopHotDramaScheduler(c *gin.Context) {
scheduler := utils.GetGlobalScheduler(
scheduler := scheduler.GetGlobalScheduler(
repoManager.HotDramaRepository,
repoManager.ReadyResourceRepository,
repoManager.ResourceRepository,
@@ -71,7 +71,7 @@ func StopHotDramaScheduler(c *gin.Context) {
// 手动触发热播剧定时任务
func TriggerHotDramaScheduler(c *gin.Context) {
scheduler := utils.GetGlobalScheduler(
scheduler := scheduler.GetGlobalScheduler(
repoManager.HotDramaRepository,
repoManager.ReadyResourceRepository,
repoManager.ResourceRepository,
@@ -87,7 +87,7 @@ func TriggerHotDramaScheduler(c *gin.Context) {
// 手动获取热播剧名字
func FetchHotDramaNames(c *gin.Context) {
scheduler := utils.GetGlobalScheduler(
scheduler := scheduler.GetGlobalScheduler(
repoManager.HotDramaRepository,
repoManager.ReadyResourceRepository,
repoManager.ResourceRepository,
@@ -107,7 +107,7 @@ func FetchHotDramaNames(c *gin.Context) {
// 启动待处理资源自动处理任务
func StartReadyResourceScheduler(c *gin.Context) {
scheduler := utils.GetGlobalScheduler(
scheduler := scheduler.GetGlobalScheduler(
repoManager.HotDramaRepository,
repoManager.ReadyResourceRepository,
repoManager.ResourceRepository,
@@ -127,7 +127,7 @@ func StartReadyResourceScheduler(c *gin.Context) {
// 停止待处理资源自动处理任务
func StopReadyResourceScheduler(c *gin.Context) {
scheduler := utils.GetGlobalScheduler(
scheduler := scheduler.GetGlobalScheduler(
repoManager.HotDramaRepository,
repoManager.ReadyResourceRepository,
repoManager.ResourceRepository,
@@ -147,7 +147,7 @@ func StopReadyResourceScheduler(c *gin.Context) {
// 手动触发待处理资源自动处理任务
func TriggerReadyResourceScheduler(c *gin.Context) {
scheduler := utils.GetGlobalScheduler(
scheduler := scheduler.GetGlobalScheduler(
repoManager.HotDramaRepository,
repoManager.ReadyResourceRepository,
repoManager.ResourceRepository,
@@ -157,14 +157,13 @@ func TriggerReadyResourceScheduler(c *gin.Context) {
repoManager.TagRepository,
repoManager.CategoryRepository,
)
// 手动触发一次处理
scheduler.ProcessReadyResources()
scheduler.StartReadyResourceScheduler() // 直接启动一次
SuccessResponse(c, gin.H{"message": "手动触发待处理资源自动处理任务成功"})
}
// 启动自动转存定时任务
func StartAutoTransferScheduler(c *gin.Context) {
scheduler := utils.GetGlobalScheduler(
scheduler := scheduler.GetGlobalScheduler(
repoManager.HotDramaRepository,
repoManager.ReadyResourceRepository,
repoManager.ResourceRepository,
@@ -184,7 +183,7 @@ func StartAutoTransferScheduler(c *gin.Context) {
// 停止自动转存定时任务
func StopAutoTransferScheduler(c *gin.Context) {
scheduler := utils.GetGlobalScheduler(
scheduler := scheduler.GetGlobalScheduler(
repoManager.HotDramaRepository,
repoManager.ReadyResourceRepository,
repoManager.ResourceRepository,
@@ -204,7 +203,7 @@ func StopAutoTransferScheduler(c *gin.Context) {
// 手动触发自动转存定时任务
func TriggerAutoTransferScheduler(c *gin.Context) {
scheduler := utils.GetGlobalScheduler(
scheduler := scheduler.GetGlobalScheduler(
repoManager.HotDramaRepository,
repoManager.ReadyResourceRepository,
repoManager.ResourceRepository,
@@ -214,7 +213,6 @@ func TriggerAutoTransferScheduler(c *gin.Context) {
repoManager.TagRepository,
repoManager.CategoryRepository,
)
// 手动触发一次处理
scheduler.ProcessAutoTransfer()
scheduler.StartAutoTransferScheduler() // 直接启动一次
SuccessResponse(c, gin.H{"message": "手动触发自动转存定时任务成功"})
}

View File

@@ -37,7 +37,8 @@ func GetSearchStats(c *gin.Context) {
page, _ := strconv.Atoi(c.DefaultQuery("page", "1"))
pageSize, _ := strconv.Atoi(c.DefaultQuery("page_size", "20"))
stats, total, err := repoManager.SearchStatRepository.FindWithPagination(page, pageSize)
// 使用自定义方法获取按时间倒序排列的搜索记录
stats, total, err := repoManager.SearchStatRepository.FindWithPaginationOrdered(page, pageSize)
if err != nil {
ErrorResponse(c, "获取搜索统计失败", http.StatusInternalServerError)
return

View File

@@ -22,17 +22,44 @@ func GetStats(c *gin.Context) {
db.DB.Model(&entity.Tag{}).Count(&totalTags)
db.DB.Model(&entity.Resource{}).Select("COALESCE(SUM(view_count), 0)").Scan(&totalViews)
// 获取今日更新数量
// 获取今日数据
today := utils.GetTodayString()
// 今日新增资源数量
var todayResources int64
db.DB.Model(&entity.Resource{}).Where("DATE(created_at) = ?", today).Count(&todayResources)
// 今日更新资源数量(包括新增和修改)
var todayUpdates int64
today := utils.GetCurrentTime().Format("2006-01-02")
db.DB.Model(&entity.Resource{}).Where("DATE(updated_at) = ?", today).Count(&todayUpdates)
// 今日浏览量 - 使用访问记录表统计今日访问量
var todayViews int64
todayViews, err := repoManager.ResourceViewRepository.GetTodayViews()
if err != nil {
utils.Error("获取今日访问量失败: %v", err)
todayViews = 0
}
// 今日搜索量
var todaySearches int64
db.DB.Model(&entity.SearchStat{}).Where("DATE(date) = ?", today).Count(&todaySearches)
// 添加调试日志
utils.Info("统计数据 - 总资源: %d, 总分类: %d, 总标签: %d, 总浏览量: %d",
totalResources, totalCategories, totalTags, totalViews)
utils.Info("今日数据 - 新增资源: %d, 今日更新: %d, 今日浏览量: %d, 今日搜索: %d",
todayResources, todayUpdates, todayViews, todaySearches)
SuccessResponse(c, gin.H{
"total_resources": totalResources,
"total_categories": totalCategories,
"total_tags": totalTags,
"total_views": totalViews,
"today_resources": todayResources,
"today_updates": todayUpdates,
"today_views": todayViews,
"today_searches": todaySearches,
})
}
@@ -89,7 +116,7 @@ func GetPerformanceStats(c *gin.Context) {
func GetSystemInfo(c *gin.Context) {
SuccessResponse(c, gin.H{
"uptime": time.Since(startTime).String(),
"start_time": utils.FormatTime(startTime, "2006-01-02 15:04:05"),
"start_time": utils.FormatTime(startTime, utils.TimeFormatDateTime),
"version": utils.Version,
"environment": gin.H{
"gin_mode": gin.Mode(),
@@ -97,5 +124,60 @@ func GetSystemInfo(c *gin.Context) {
})
}
// GetViewsTrend 获取访问量趋势数据
func GetViewsTrend(c *gin.Context) {
// 使用访问记录表获取最近7天的访问量数据
results, err := repoManager.ResourceViewRepository.GetViewsTrend(7)
if err != nil {
utils.Error("获取访问量趋势数据失败: %v", err)
// 如果获取失败,返回空数据
results = []map[string]interface{}{}
}
// 添加调试日志
utils.Info("访问量趋势数据: %+v", results)
for i, result := range results {
utils.Info("第%d天: 日期=%s, 访问量=%d", i+1, result["date"], result["views"])
}
SuccessResponse(c, results)
}
// GetSearchesTrend 获取搜索量趋势数据
func GetSearchesTrend(c *gin.Context) {
// 获取最近7天的搜索量数据
var results []gin.H
// 生成最近7天的日期
for i := 6; i >= 0; i-- {
date := utils.GetCurrentTime().AddDate(0, 0, -i)
dateStr := date.Format(utils.TimeFormatDate)
// 查询该日期的搜索量(从搜索统计表)
var searches int64
db.DB.Model(&entity.SearchStat{}).
Where("DATE(date) = ?", dateStr).
Count(&searches)
// 如果没有搜索记录返回0
// 移除模拟数据生成逻辑,只返回真实数据
results = append(results, gin.H{
"date": dateStr,
"searches": searches,
})
}
// 添加调试日志
utils.Info("搜索量趋势数据: %+v", results)
// 添加更详细的调试信息
for i, result := range results {
utils.Info("第%d天: 日期=%s, 搜索量=%d", i+1, result["date"], result["searches"])
}
SuccessResponse(c, results)
}
// 记录启动时间
var startTime = utils.GetCurrentTime()

View File

@@ -3,10 +3,12 @@ package handlers
import (
"net/http"
pan "github.com/ctwj/urldb/common"
"github.com/ctwj/urldb/db/converter"
"github.com/ctwj/urldb/db/dto"
"github.com/ctwj/urldb/db/entity"
"github.com/ctwj/urldb/db/repo"
"github.com/ctwj/urldb/scheduler"
"github.com/ctwj/urldb/utils"
"github.com/gin-gonic/gin"
@@ -26,6 +28,20 @@ func NewSystemConfigHandler(systemConfigRepo repo.SystemConfigRepository) *Syste
// GetConfig 获取系统配置
func (h *SystemConfigHandler) GetConfig(c *gin.Context) {
// 先验证配置完整性
if err := h.systemConfigRepo.ValidateConfigIntegrity(); err != nil {
utils.Error("配置完整性检查失败: %v", err)
// 如果配置不完整,尝试重新创建默认配置
configs, err := h.systemConfigRepo.GetOrCreateDefault()
if err != nil {
ErrorResponse(c, "获取系统配置失败", http.StatusInternalServerError)
return
}
configResponse := converter.SystemConfigToResponse(configs)
SuccessResponse(c, configResponse)
return
}
configs, err := h.systemConfigRepo.GetOrCreateDefault()
if err != nil {
ErrorResponse(c, "获取系统配置失败", http.StatusInternalServerError)
@@ -44,29 +60,23 @@ func (h *SystemConfigHandler) UpdateConfig(c *gin.Context) {
return
}
// 验证参数
if req.SiteTitle == "" {
ErrorResponse(c, "网站标题不能为空", http.StatusBadRequest)
// 验证参数 - 只验证提交的字段
if req.SiteTitle != nil && (len(*req.SiteTitle) < 1 || len(*req.SiteTitle) > 100) {
ErrorResponse(c, "网站标题长度必须在1-100字符之间", http.StatusBadRequest)
return
}
if req.AutoProcessInterval < 1 || req.AutoProcessInterval > 1440 {
if req.AutoProcessInterval != nil && (*req.AutoProcessInterval < 1 || *req.AutoProcessInterval > 1440) {
ErrorResponse(c, "自动处理间隔必须在1-1440分钟之间", http.StatusBadRequest)
return
}
if req.PageSize < 10 || req.PageSize > 500 {
if req.PageSize != nil && (*req.PageSize < 10 || *req.PageSize > 500) {
ErrorResponse(c, "每页显示数量必须在10-500之间", http.StatusBadRequest)
return
}
// 验证自动转存配置
if req.AutoTransferLimitDays < 0 || req.AutoTransferLimitDays > 365 {
ErrorResponse(c, "自动转存限制天数必须在0-365之间", http.StatusBadRequest)
return
}
if req.AutoTransferMinSpace < 100 || req.AutoTransferMinSpace > 1024 {
if req.AutoTransferMinSpace != nil && (*req.AutoTransferMinSpace < 100 || *req.AutoTransferMinSpace > 1024) {
ErrorResponse(c, "最小存储空间必须在100-1024GB之间", http.StatusBadRequest)
return
}
@@ -85,6 +95,9 @@ func (h *SystemConfigHandler) UpdateConfig(c *gin.Context) {
return
}
// 刷新系统配置缓存
pan.RefreshSystemConfigCache()
// 返回更新后的配置
updatedConfigs, err := h.systemConfigRepo.FindAll()
if err != nil {
@@ -116,29 +129,40 @@ func UpdateSystemConfig(c *gin.Context) {
return
}
// 验证参数
if req.SiteTitle == "" {
ErrorResponse(c, "网站标题不能为空", http.StatusBadRequest)
// 调试信息
utils.Info("接收到的配置请求: %+v", req)
// 获取当前配置作为备份
currentConfigs, err := repoManager.SystemConfigRepository.FindAll()
if err != nil {
utils.Error("获取当前配置失败: %v", err)
} else {
utils.Info("当前配置数量: %d", len(currentConfigs))
}
// 验证参数 - 只验证提交的字段
if req.SiteTitle != nil && (len(*req.SiteTitle) < 1 || len(*req.SiteTitle) > 100) {
ErrorResponse(c, "网站标题长度必须在1-100字符之间", http.StatusBadRequest)
return
}
if req.AutoProcessInterval < 1 || req.AutoProcessInterval > 1440 {
if req.AutoProcessInterval != nil && (*req.AutoProcessInterval < 1 || *req.AutoProcessInterval > 1440) {
ErrorResponse(c, "自动处理间隔必须在1-1440分钟之间", http.StatusBadRequest)
return
}
if req.PageSize < 10 || req.PageSize > 500 {
if req.PageSize != nil && (*req.PageSize < 10 || *req.PageSize > 500) {
ErrorResponse(c, "每页显示数量必须在10-500之间", http.StatusBadRequest)
return
}
// 验证自动转存配置
if req.AutoTransferLimitDays < 0 || req.AutoTransferLimitDays > 365 {
if req.AutoTransferLimitDays != nil && (*req.AutoTransferLimitDays < 0 || *req.AutoTransferLimitDays > 365) {
ErrorResponse(c, "自动转存限制天数必须在0-365之间", http.StatusBadRequest)
return
}
if req.AutoTransferMinSpace < 100 || req.AutoTransferMinSpace > 1024 {
if req.AutoTransferMinSpace != nil && (*req.AutoTransferMinSpace < 100 || *req.AutoTransferMinSpace > 1024) {
ErrorResponse(c, "最小存储空间必须在100-1024GB之间", http.StatusBadRequest)
return
}
@@ -150,15 +174,40 @@ func UpdateSystemConfig(c *gin.Context) {
return
}
utils.Info("准备更新配置,配置项数量: %d", len(configs))
// 保存配置
err := repoManager.SystemConfigRepository.UpsertConfigs(configs)
err = repoManager.SystemConfigRepository.UpsertConfigs(configs)
if err != nil {
utils.Error("保存系统配置失败: %v", err)
ErrorResponse(c, "保存系统配置失败", http.StatusInternalServerError)
return
}
utils.Info("配置保存成功")
// 安全刷新系统配置缓存
if err := repoManager.SystemConfigRepository.SafeRefreshConfigCache(); err != nil {
utils.Error("刷新配置缓存失败: %v", err)
// 不返回错误,因为配置已经保存成功
}
// 刷新系统配置缓存
pan.RefreshSystemConfigCache()
// 重新加载Meilisearch配置如果Meilisearch配置有变更
if req.MeilisearchEnabled != nil || req.MeilisearchHost != nil || req.MeilisearchPort != nil || req.MeilisearchMasterKey != nil || req.MeilisearchIndexName != nil {
if meilisearchManager != nil {
if err := meilisearchManager.ReloadConfig(); err != nil {
utils.Error("重新加载Meilisearch配置失败: %v", err)
} else {
utils.Debug("Meilisearch配置重新加载成功")
}
}
}
// 根据配置更新定时任务状态(错误不影响配置保存)
scheduler := utils.GetGlobalScheduler(
scheduler := scheduler.GetGlobalScheduler(
repoManager.HotDramaRepository,
repoManager.ReadyResourceRepository,
repoManager.ResourceRepository,
@@ -169,16 +218,30 @@ func UpdateSystemConfig(c *gin.Context) {
repoManager.CategoryRepository,
)
if scheduler != nil {
scheduler.UpdateSchedulerStatusWithAutoTransfer(req.AutoFetchHotDramaEnabled, req.AutoProcessReadyResources, req.AutoTransferEnabled)
// 只更新被设置的配置
var autoFetchHotDrama, autoProcessReady, autoTransfer bool
if req.AutoFetchHotDramaEnabled != nil {
autoFetchHotDrama = *req.AutoFetchHotDramaEnabled
}
if req.AutoProcessReadyResources != nil {
autoProcessReady = *req.AutoProcessReadyResources
}
if req.AutoTransferEnabled != nil {
autoTransfer = *req.AutoTransferEnabled
}
scheduler.UpdateSchedulerStatusWithAutoTransfer(autoFetchHotDrama, autoProcessReady, autoTransfer)
}
// 返回更新后的配置
updatedConfigs, err := repoManager.SystemConfigRepository.FindAll()
if err != nil {
utils.Error("获取更新后的配置失败: %v", err)
ErrorResponse(c, "获取更新后的配置失败", http.StatusInternalServerError)
return
}
utils.Info("配置更新完成,当前配置数量: %d", len(updatedConfigs))
configResponse := converter.SystemConfigToResponse(updatedConfigs)
SuccessResponse(c, configResponse)
}
@@ -194,6 +257,36 @@ func GetPublicSystemConfig(c *gin.Context) {
SuccessResponse(c, configResponse)
}
// 新增:配置监控端点
func GetConfigStatus(c *gin.Context) {
// 获取配置统计信息
configs, err := repoManager.SystemConfigRepository.FindAll()
if err != nil {
ErrorResponse(c, "获取配置状态失败", http.StatusInternalServerError)
return
}
// 验证配置完整性
integrityErr := repoManager.SystemConfigRepository.ValidateConfigIntegrity()
// 获取缓存状态
cachedConfigs := repoManager.SystemConfigRepository.GetCachedConfigs()
status := map[string]interface{}{
"total_configs": len(configs),
"cached_configs": len(cachedConfigs),
"integrity_check": integrityErr == nil,
"integrity_error": "",
"last_check_time": utils.GetCurrentTimeString(),
}
if integrityErr != nil {
status["integrity_error"] = integrityErr.Error()
}
SuccessResponse(c, status)
}
// 新增:切换自动处理配置
func ToggleAutoProcess(c *gin.Context) {
var req struct {
@@ -230,7 +323,7 @@ func ToggleAutoProcess(c *gin.Context) {
}
// 更新定时任务状态
scheduler := utils.GetGlobalScheduler(
scheduler := scheduler.GetGlobalScheduler(
repoManager.HotDramaRepository,
repoManager.ReadyResourceRepository,
repoManager.ResourceRepository,

View File

@@ -24,11 +24,11 @@ func GetTags(c *gin.Context) {
var err error
if search != "" {
// 搜索标签
tags, total, err = repoManager.TagRepository.Search(search, page, pageSize)
// 搜索标签(按资源数量排序)
tags, total, err = repoManager.TagRepository.SearchOrderByResourceCount(search, page, pageSize)
} else {
// 分页查询
tags, total, err = repoManager.TagRepository.FindWithPagination(page, pageSize)
// 分页查询(按资源数量排序)
tags, total, err = repoManager.TagRepository.FindWithPaginationOrderByResourceCount(page, pageSize)
}
if err != nil {
@@ -65,13 +65,47 @@ func CreateTag(c *gin.Context) {
return
}
// 首先检查是否存在已删除的同名标签
deletedTag, err := repoManager.TagRepository.FindByNameIncludingDeleted(req.Name)
if err == nil && deletedTag.DeletedAt.Valid {
// 如果存在已删除的同名标签,则恢复它
err = repoManager.TagRepository.RestoreDeletedTag(deletedTag.ID)
if err != nil {
ErrorResponse(c, "恢复已删除标签失败: "+err.Error(), http.StatusInternalServerError)
return
}
// 重新获取恢复后的标签
restoredTag, err := repoManager.TagRepository.FindByID(deletedTag.ID)
if err != nil {
ErrorResponse(c, "获取恢复的标签失败: "+err.Error(), http.StatusInternalServerError)
return
}
// 更新标签信息
restoredTag.Description = req.Description
restoredTag.CategoryID = req.CategoryID
err = repoManager.TagRepository.UpdateWithNulls(restoredTag)
if err != nil {
ErrorResponse(c, "更新恢复的标签失败: "+err.Error(), http.StatusInternalServerError)
return
}
SuccessResponse(c, gin.H{
"message": "标签恢复成功",
"tag": converter.ToTagResponse(restoredTag, 0),
})
return
}
// 如果不存在已删除的同名标签,则创建新标签
tag := &entity.Tag{
Name: req.Name,
Description: req.Description,
CategoryID: req.CategoryID,
}
err := repoManager.TagRepository.Create(tag)
err = repoManager.TagRepository.Create(tag)
if err != nil {
ErrorResponse(c, err.Error(), http.StatusInternalServerError)
return

375
handlers/task_handler.go Normal file
View File

@@ -0,0 +1,375 @@
package handlers
import (
"encoding/json"
"net/http"
"strconv"
"github.com/ctwj/urldb/db/entity"
"github.com/ctwj/urldb/db/repo"
"github.com/ctwj/urldb/task"
"github.com/ctwj/urldb/utils"
"github.com/gin-gonic/gin"
)
// TaskHandler 任务处理器
type TaskHandler struct {
repoMgr *repo.RepositoryManager
taskManager *task.TaskManager
}
// NewTaskHandler 创建任务处理器
func NewTaskHandler(repoMgr *repo.RepositoryManager, taskManager *task.TaskManager) *TaskHandler {
return &TaskHandler{
repoMgr: repoMgr,
taskManager: taskManager,
}
}
// 批量转存任务资源项
type BatchTransferResource struct {
Title string `json:"title" binding:"required"`
URL string `json:"url" binding:"required"`
CategoryID uint `json:"category_id,omitempty"`
PanID uint `json:"pan_id,omitempty"`
Tags []uint `json:"tags,omitempty"`
}
// CreateBatchTransferTask 创建批量转存任务
func (h *TaskHandler) CreateBatchTransferTask(c *gin.Context) {
var req struct {
Title string `json:"title" binding:"required"`
Description string `json:"description"`
Resources []BatchTransferResource `json:"resources" binding:"required,min=1"`
SelectedAccounts []uint `json:"selected_accounts,omitempty"`
}
if err := c.ShouldBindJSON(&req); err != nil {
ErrorResponse(c, "参数错误: "+err.Error(), http.StatusBadRequest)
return
}
utils.Info("创建批量转存任务: %s资源数量: %d选择账号数量: %d", req.Title, len(req.Resources), len(req.SelectedAccounts))
// 构建任务配置
taskConfig := map[string]interface{}{
"selected_accounts": req.SelectedAccounts,
}
configJSON, _ := json.Marshal(taskConfig)
// 创建任务
newTask := &entity.Task{
Title: req.Title,
Description: req.Description,
Type: "transfer",
Status: "pending",
TotalItems: len(req.Resources),
Config: string(configJSON),
CreatedAt: utils.GetCurrentTime(),
UpdatedAt: utils.GetCurrentTime(),
}
err := h.repoMgr.TaskRepository.Create(newTask)
if err != nil {
utils.Error("创建任务失败: %v", err)
ErrorResponse(c, "创建任务失败: "+err.Error(), http.StatusInternalServerError)
return
}
// 创建任务项
for _, resource := range req.Resources {
// 构建转存输入数据
transferInput := task.TransferInput{
Title: resource.Title,
URL: resource.URL,
CategoryID: resource.CategoryID,
PanID: resource.PanID,
Tags: resource.Tags,
}
inputJSON, _ := json.Marshal(transferInput)
taskItem := &entity.TaskItem{
TaskID: newTask.ID,
Status: "pending",
InputData: string(inputJSON),
CreatedAt: utils.GetCurrentTime(),
UpdatedAt: utils.GetCurrentTime(),
}
err = h.repoMgr.TaskItemRepository.Create(taskItem)
if err != nil {
utils.Error("创建任务项失败: %v", err)
// 继续创建其他任务项
}
}
utils.Info("批量转存任务创建完成: %d, 共 %d 项", newTask.ID, len(req.Resources))
SuccessResponse(c, gin.H{
"task_id": newTask.ID,
"total_items": len(req.Resources),
"message": "任务创建成功",
})
}
// StartTask 启动任务
func (h *TaskHandler) StartTask(c *gin.Context) {
taskIDStr := c.Param("id")
taskID, err := strconv.ParseUint(taskIDStr, 10, 32)
if err != nil {
ErrorResponse(c, "无效的任务ID: "+err.Error(), http.StatusBadRequest)
return
}
utils.Info("启动任务: %d", taskID)
err = h.taskManager.StartTask(uint(taskID))
if err != nil {
utils.Error("启动任务失败: %v", err)
ErrorResponse(c, "启动任务失败: "+err.Error(), http.StatusInternalServerError)
return
}
SuccessResponse(c, gin.H{
"message": "任务启动成功",
})
}
// StopTask 停止任务
func (h *TaskHandler) StopTask(c *gin.Context) {
taskIDStr := c.Param("id")
taskID, err := strconv.ParseUint(taskIDStr, 10, 32)
if err != nil {
ErrorResponse(c, "无效的任务ID: "+err.Error(), http.StatusBadRequest)
return
}
utils.Info("停止任务: %d", taskID)
err = h.taskManager.StopTask(uint(taskID))
if err != nil {
utils.Error("停止任务失败: %v", err)
ErrorResponse(c, "停止任务失败: "+err.Error(), http.StatusInternalServerError)
return
}
SuccessResponse(c, gin.H{
"message": "任务停止成功",
})
}
// PauseTask 暂停任务
func (h *TaskHandler) PauseTask(c *gin.Context) {
taskIDStr := c.Param("id")
taskID, err := strconv.ParseUint(taskIDStr, 10, 32)
if err != nil {
ErrorResponse(c, "无效的任务ID: "+err.Error(), http.StatusBadRequest)
return
}
utils.Info("暂停任务: %d", taskID)
err = h.taskManager.PauseTask(uint(taskID))
if err != nil {
utils.Error("暂停任务失败: %v", err)
ErrorResponse(c, "暂停任务失败: "+err.Error(), http.StatusInternalServerError)
return
}
SuccessResponse(c, gin.H{
"message": "任务暂停成功",
})
}
// GetTaskStatus 获取任务状态
func (h *TaskHandler) GetTaskStatus(c *gin.Context) {
taskIDStr := c.Param("id")
taskID, err := strconv.ParseUint(taskIDStr, 10, 32)
if err != nil {
ErrorResponse(c, "无效的任务ID: "+err.Error(), http.StatusBadRequest)
return
}
// 获取任务详情
task, err := h.repoMgr.TaskRepository.GetByID(uint(taskID))
if err != nil {
ErrorResponse(c, "任务不存在: "+err.Error(), http.StatusNotFound)
return
}
// 获取任务项统计
stats, err := h.repoMgr.TaskItemRepository.GetStatsByTaskID(uint(taskID))
if err != nil {
utils.Error("获取任务项统计失败: %v", err)
stats = map[string]int{
"total": 0,
"pending": 0,
"processing": 0,
"completed": 0,
"failed": 0,
}
}
// 检查任务是否在运行
isRunning := h.taskManager.IsTaskRunning(uint(taskID))
SuccessResponse(c, gin.H{
"id": task.ID,
"title": task.Title,
"description": task.Description,
"task_type": task.Type,
"status": task.Status,
"total_items": task.TotalItems,
"processed_items": task.ProcessedItems,
"success_items": task.SuccessItems,
"failed_items": task.FailedItems,
"is_running": isRunning,
"stats": stats,
"created_at": task.CreatedAt,
"updated_at": task.UpdatedAt,
})
}
// GetTasks 获取任务列表
func (h *TaskHandler) GetTasks(c *gin.Context) {
page, _ := strconv.Atoi(c.DefaultQuery("page", "1"))
pageSize, _ := strconv.Atoi(c.DefaultQuery("page_size", "10"))
taskType := c.Query("task_type")
status := c.Query("status")
utils.Debug("GetTasks: 获取任务列表 page=%d, pageSize=%d, taskType=%s, status=%s", page, pageSize, taskType, status)
tasks, total, err := h.repoMgr.TaskRepository.GetList(page, pageSize, taskType, status)
if err != nil {
utils.Error("获取任务列表失败: %v", err)
ErrorResponse(c, "获取任务列表失败: "+err.Error(), http.StatusInternalServerError)
return
}
utils.Debug("GetTasks: 从数据库获取到 %d 个任务", len(tasks))
// 为每个任务添加运行状态
var result []gin.H
for _, task := range tasks {
isRunning := h.taskManager.IsTaskRunning(task.ID)
utils.Debug("GetTasks: 任务 %d (%s) 数据库状态: %s, TaskManager运行状态: %v", task.ID, task.Title, task.Status, isRunning)
result = append(result, gin.H{
"id": task.ID,
"title": task.Title,
"description": task.Description,
"task_type": task.Type,
"status": task.Status,
"total_items": task.TotalItems,
"processed_items": task.ProcessedItems,
"success_items": task.SuccessItems,
"failed_items": task.FailedItems,
"is_running": isRunning,
"created_at": task.CreatedAt,
"updated_at": task.UpdatedAt,
})
}
SuccessResponse(c, gin.H{
"items": result,
"total": total,
"page": page,
"size": pageSize,
})
}
// GetTaskItems 获取任务项列表
func (h *TaskHandler) GetTaskItems(c *gin.Context) {
taskIDStr := c.Param("id")
taskID, err := strconv.ParseUint(taskIDStr, 10, 32)
if err != nil {
ErrorResponse(c, "无效的任务ID: "+err.Error(), http.StatusBadRequest)
return
}
page, _ := strconv.Atoi(c.DefaultQuery("page", "1"))
pageSize, _ := strconv.Atoi(c.DefaultQuery("page_size", "10000"))
status := c.Query("status")
items, total, err := h.repoMgr.TaskItemRepository.GetListByTaskID(uint(taskID), page, pageSize, status)
if err != nil {
utils.Error("获取任务项列表失败: %v", err)
ErrorResponse(c, "获取任务项列表失败: "+err.Error(), http.StatusInternalServerError)
return
}
// 解析输入和输出数据
var result []gin.H
for _, item := range items {
itemData := gin.H{
"id": item.ID,
"status": item.Status,
"created_at": item.CreatedAt,
"updated_at": item.UpdatedAt,
}
// 解析输入数据
if item.InputData != "" {
var inputData map[string]interface{}
if err := json.Unmarshal([]byte(item.InputData), &inputData); err == nil {
itemData["input"] = inputData
}
}
// 解析输出数据
if item.OutputData != "" {
var outputData map[string]interface{}
if err := json.Unmarshal([]byte(item.OutputData), &outputData); err == nil {
itemData["output"] = outputData
}
}
result = append(result, itemData)
}
SuccessResponse(c, gin.H{
"items": result,
"total": total,
"page": page,
"size": pageSize,
})
}
// DeleteTask 删除任务
func (h *TaskHandler) DeleteTask(c *gin.Context) {
taskIDStr := c.Param("id")
taskID, err := strconv.ParseUint(taskIDStr, 10, 32)
if err != nil {
ErrorResponse(c, "无效的任务ID: "+err.Error(), http.StatusBadRequest)
return
}
// 检查任务是否在运行
if h.taskManager.IsTaskRunning(uint(taskID)) {
ErrorResponse(c, "任务正在运行,请先停止任务", http.StatusBadRequest)
return
}
// 删除任务项
err = h.repoMgr.TaskItemRepository.DeleteByTaskID(uint(taskID))
if err != nil {
utils.Error("删除任务项失败: %v", err)
ErrorResponse(c, "删除任务项失败: "+err.Error(), http.StatusInternalServerError)
return
}
// 删除任务
err = h.repoMgr.TaskRepository.Delete(uint(taskID))
if err != nil {
utils.Error("删除任务失败: %v", err)
ErrorResponse(c, "删除任务失败: "+err.Error(), http.StatusInternalServerError)
return
}
utils.Info("任务删除成功: %d", taskID)
SuccessResponse(c, gin.H{
"message": "任务删除成功",
})
}

144
main.go
View File

@@ -1,16 +1,18 @@
package main
import (
"fmt"
"log"
"os"
"github.com/ctwj/urldb/utils"
"strings"
"github.com/ctwj/urldb/db"
"github.com/ctwj/urldb/db/entity"
"github.com/ctwj/urldb/db/repo"
"github.com/ctwj/urldb/handlers"
"github.com/ctwj/urldb/middleware"
"github.com/ctwj/urldb/services"
"github.com/ctwj/urldb/task"
"github.com/ctwj/urldb/utils"
"github.com/gin-contrib/cors"
"github.com/gin-gonic/gin"
@@ -18,6 +20,18 @@ import (
)
func main() {
// 检查命令行参数
if len(os.Args) > 1 && os.Args[1] == "version" {
versionInfo := utils.GetVersionInfo()
fmt.Printf("版本: v%s\n", versionInfo.Version)
fmt.Printf("构建时间: %s\n", versionInfo.BuildTime.Format("2006-01-02 15:04:05"))
fmt.Printf("Git提交: %s\n", versionInfo.GitCommit)
fmt.Printf("Git分支: %s\n", versionInfo.GitBranch)
fmt.Printf("Go版本: %s\n", versionInfo.GoVersion)
fmt.Printf("平台: %s/%s\n", versionInfo.Platform, versionInfo.Arch)
return
}
// 初始化日志系统
if err := utils.InitLogger(nil); err != nil {
log.Fatal("初始化日志系统失败:", err)
@@ -69,48 +83,27 @@ func main() {
// 创建Repository管理器
repoManager := repo.NewRepositoryManager(db.DB)
// 创建全局调度
scheduler := utils.GetGlobalScheduler(
repoManager.HotDramaRepository,
repoManager.ReadyResourceRepository,
repoManager.ResourceRepository,
repoManager.SystemConfigRepository,
repoManager.PanRepository,
repoManager.CksRepository,
repoManager.TagRepository,
repoManager.CategoryRepository,
)
// 创建任务管理
taskManager := task.NewTaskManager(repoManager)
// 检查系统配置,决定是否启动各种自动任务
autoProcessReadyResources, err := repoManager.SystemConfigRepository.GetConfigBool(entity.ConfigKeyAutoProcessReadyResources)
if err != nil {
utils.Error("获取自动处理待处理资源配置失败: %v", err)
} else if autoProcessReadyResources {
scheduler.StartReadyResourceScheduler()
utils.Info("已启动待处理资源自动处理任务")
} else {
utils.Info("系统配置中自动处理待处理资源功能已禁用,跳过启动定时任务")
// 注册转存任务处理器
transferProcessor := task.NewTransferProcessor(repoManager)
taskManager.RegisterProcessor(transferProcessor)
// 初始化Meilisearch管理器
meilisearchManager := services.NewMeilisearchManager(repoManager)
if err := meilisearchManager.Initialize(); err != nil {
utils.Error("初始化Meilisearch管理器失败: %v", err)
}
autoFetchHotDramaEnabled, err := repoManager.SystemConfigRepository.GetConfigBool(entity.ConfigKeyAutoFetchHotDramaEnabled)
if err != nil {
utils.Error("获取自动拉取热播剧配置失败: %v", err)
} else if autoFetchHotDramaEnabled {
scheduler.StartHotDramaScheduler()
utils.Info("已启动热播剧自动拉取任务")
// 恢复运行中的任务(服务器重启后)
if err := taskManager.RecoverRunningTasks(); err != nil {
utils.Error("恢复运行中任务失败: %v", err)
} else {
utils.Info("系统配置中自动拉取热播剧功能已禁用,跳过启动定时任务")
utils.Info("运行中任务恢复完成")
}
autoTransferEnabled, err := repoManager.SystemConfigRepository.GetConfigBool(entity.ConfigKeyAutoTransferEnabled)
if err != nil {
utils.Error("获取自动转存配置失败: %v", err)
} else if autoTransferEnabled {
scheduler.StartAutoTransferScheduler()
utils.Info("已启动自动转存任务")
} else {
utils.Info("系统配置中自动转存功能已禁用,跳过启动定时任务")
}
utils.Info("任务管理器初始化完成")
// 创建Gin实例
r := gin.Default()
@@ -125,12 +118,24 @@ func main() {
// 将Repository管理器注入到handlers中
handlers.SetRepositoryManager(repoManager)
// 设置Meilisearch管理器到handlers中
handlers.SetMeilisearchManager(meilisearchManager)
// 设置公开API中间件的Repository管理器
middleware.SetRepositoryManager(repoManager)
// 创建公开API处理器
publicAPIHandler := handlers.NewPublicAPIHandler()
// 创建任务处理器
taskHandler := handlers.NewTaskHandler(repoManager, taskManager)
// 创建文件处理器
fileHandler := handlers.NewFileHandler(repoManager.FileRepository, repoManager.SystemConfigRepository, repoManager.UserRepository)
// 创建Meilisearch处理器
meilisearchHandler := handlers.NewMeilisearchHandler(meilisearchManager)
// API路由
api := r.Group("/api")
{
@@ -159,6 +164,7 @@ func main() {
api.GET("/resources/:id", handlers.GetResourceByID)
api.GET("/resources/check-exists", handlers.CheckResourceExists)
api.POST("/resources/:id/view", handlers.IncrementResourceViewCount)
api.GET("/resources/:id/link", handlers.GetResourceLink)
api.DELETE("/resources/batch", middleware.AuthMiddleware(), middleware.AdminMiddleware(), handlers.BatchDeleteResources)
// 分类管理
@@ -173,6 +179,8 @@ func main() {
// 统计
api.GET("/stats", handlers.GetStats)
api.GET("/performance", handlers.GetPerformanceStats)
api.GET("/stats/views-trend", handlers.GetViewsTrend)
api.GET("/stats/searches-trend", handlers.GetSearchesTrend)
api.GET("/system/info", handlers.GetSystemInfo)
// 平台管理
@@ -209,6 +217,9 @@ func main() {
api.GET("/ready-resources/errors", middleware.AuthMiddleware(), middleware.AdminMiddleware(), handlers.GetReadyResourcesWithErrors)
api.POST("/ready-resources/:id/clear-error", middleware.AuthMiddleware(), middleware.AdminMiddleware(), handlers.ClearErrorMsg)
api.POST("/ready-resources/retry-failed", middleware.AuthMiddleware(), middleware.AdminMiddleware(), handlers.RetryFailedResources)
api.POST("/ready-resources/batch-restore", middleware.AuthMiddleware(), middleware.AdminMiddleware(), handlers.BatchRestoreToReadyPool)
api.POST("/ready-resources/batch-restore-by-query", middleware.AuthMiddleware(), middleware.AdminMiddleware(), handlers.BatchRestoreToReadyPoolByQuery)
api.POST("/ready-resources/clear-all-errors-by-query", middleware.AuthMiddleware(), middleware.AdminMiddleware(), handlers.ClearAllErrorsByQuery)
// 用户管理(仅管理员)
api.GET("/users", middleware.AuthMiddleware(), middleware.AdminMiddleware(), handlers.GetUsers)
@@ -230,6 +241,7 @@ func main() {
// 系统配置路由
api.GET("/system/config", middleware.AuthMiddleware(), middleware.AdminMiddleware(), handlers.GetSystemConfig)
api.POST("/system/config", middleware.AuthMiddleware(), middleware.AdminMiddleware(), handlers.UpdateSystemConfig)
api.GET("/system/config/status", middleware.AuthMiddleware(), middleware.AdminMiddleware(), handlers.GetConfigStatus)
api.POST("/system/config/toggle-auto-process", middleware.AuthMiddleware(), middleware.AdminMiddleware(), handlers.ToggleAutoProcess)
api.GET("/public/system-config", handlers.GetPublicSystemConfig)
@@ -240,33 +252,55 @@ func main() {
api.PUT("/hot-dramas/:id", middleware.AuthMiddleware(), middleware.AdminMiddleware(), handlers.UpdateHotDrama)
api.DELETE("/hot-dramas/:id", middleware.AuthMiddleware(), middleware.AdminMiddleware(), handlers.DeleteHotDrama)
// 调度器管理路由(查询接口无需认证)
api.GET("/scheduler/status", handlers.GetSchedulerStatus)
api.GET("/scheduler/hot-drama/names", handlers.FetchHotDramaNames)
api.POST("/scheduler/hot-drama/start", middleware.AuthMiddleware(), middleware.AdminMiddleware(), handlers.StartHotDramaScheduler)
api.POST("/scheduler/hot-drama/stop", middleware.AuthMiddleware(), middleware.AdminMiddleware(), handlers.StopHotDramaScheduler)
api.POST("/scheduler/hot-drama/trigger", middleware.AuthMiddleware(), middleware.AdminMiddleware(), handlers.TriggerHotDramaScheduler)
// 待处理资源自动处理管理路由
api.POST("/scheduler/ready-resource/start", middleware.AuthMiddleware(), middleware.AdminMiddleware(), handlers.StartReadyResourceScheduler)
api.POST("/scheduler/ready-resource/stop", middleware.AuthMiddleware(), middleware.AdminMiddleware(), handlers.StopReadyResourceScheduler)
api.POST("/scheduler/ready-resource/trigger", middleware.AuthMiddleware(), middleware.AdminMiddleware(), handlers.TriggerReadyResourceScheduler)
// 自动转存管理路由
api.POST("/scheduler/auto-transfer/start", middleware.AuthMiddleware(), middleware.AdminMiddleware(), handlers.StartAutoTransferScheduler)
api.POST("/scheduler/auto-transfer/stop", middleware.AuthMiddleware(), middleware.AdminMiddleware(), handlers.StopAutoTransferScheduler)
api.POST("/scheduler/auto-transfer/trigger", middleware.AuthMiddleware(), middleware.AdminMiddleware(), handlers.TriggerAutoTransferScheduler)
// 任务管理路由
api.POST("/tasks/transfer", middleware.AuthMiddleware(), middleware.AdminMiddleware(), taskHandler.CreateBatchTransferTask)
api.GET("/tasks", middleware.AuthMiddleware(), middleware.AdminMiddleware(), taskHandler.GetTasks)
api.GET("/tasks/:id", middleware.AuthMiddleware(), middleware.AdminMiddleware(), taskHandler.GetTaskStatus)
api.POST("/tasks/:id/start", middleware.AuthMiddleware(), middleware.AdminMiddleware(), taskHandler.StartTask)
api.POST("/tasks/:id/stop", middleware.AuthMiddleware(), middleware.AdminMiddleware(), taskHandler.StopTask)
api.POST("/tasks/:id/pause", middleware.AuthMiddleware(), middleware.AdminMiddleware(), taskHandler.PauseTask)
api.DELETE("/tasks/:id", middleware.AuthMiddleware(), middleware.AdminMiddleware(), taskHandler.DeleteTask)
api.GET("/tasks/:id/items", middleware.AuthMiddleware(), middleware.AdminMiddleware(), taskHandler.GetTaskItems)
// 版本管理路由
api.GET("/version", handlers.GetVersion)
api.GET("/version/string", handlers.GetVersionString)
api.GET("/version/full", handlers.GetFullVersionInfo)
api.GET("/version/check-update", handlers.CheckUpdate)
// Meilisearch管理路由
api.GET("/meilisearch/status", middleware.AuthMiddleware(), middleware.AdminMiddleware(), meilisearchHandler.GetStatus)
api.GET("/meilisearch/unsynced-count", middleware.AuthMiddleware(), middleware.AdminMiddleware(), meilisearchHandler.GetUnsyncedCount)
api.GET("/meilisearch/unsynced", middleware.AuthMiddleware(), middleware.AdminMiddleware(), meilisearchHandler.GetUnsyncedResources)
api.GET("/meilisearch/synced", middleware.AuthMiddleware(), middleware.AdminMiddleware(), meilisearchHandler.GetSyncedResources)
api.GET("/meilisearch/resources", middleware.AuthMiddleware(), middleware.AdminMiddleware(), meilisearchHandler.GetAllResources)
api.POST("/meilisearch/sync-all", middleware.AuthMiddleware(), middleware.AdminMiddleware(), meilisearchHandler.SyncAllResources)
api.GET("/meilisearch/sync-progress", middleware.AuthMiddleware(), middleware.AdminMiddleware(), meilisearchHandler.GetSyncProgress)
api.POST("/meilisearch/stop-sync", middleware.AuthMiddleware(), middleware.AdminMiddleware(), meilisearchHandler.StopSync)
api.POST("/meilisearch/clear-index", middleware.AuthMiddleware(), middleware.AdminMiddleware(), meilisearchHandler.ClearIndex)
api.POST("/meilisearch/test-connection", middleware.AuthMiddleware(), middleware.AdminMiddleware(), meilisearchHandler.TestConnection)
api.POST("/meilisearch/update-settings", middleware.AuthMiddleware(), middleware.AdminMiddleware(), meilisearchHandler.UpdateIndexSettings)
// 文件上传相关路由
api.POST("/files/upload", middleware.AuthMiddleware(), fileHandler.UploadFile)
api.GET("/files", middleware.AuthMiddleware(), fileHandler.GetFileList)
api.DELETE("/files", middleware.AuthMiddleware(), fileHandler.DeleteFiles)
api.PUT("/files", middleware.AuthMiddleware(), fileHandler.UpdateFile)
}
// 静态文件服务
r.Static("/uploads", "./uploads")
// 添加CORS头到静态文件
r.Use(func(c *gin.Context) {
if strings.HasPrefix(c.Request.URL.Path, "/uploads/") {
c.Header("Access-Control-Allow-Origin", "*")
c.Header("Access-Control-Allow-Methods", "GET, OPTIONS")
c.Header("Access-Control-Allow-Headers", "Origin, Content-Type, Accept")
}
c.Next()
})
port := os.Getenv("PORT")
if port == "" {
port = "8080"

14
migrations/v1.x.x.sql Normal file
View File

@@ -0,0 +1,14 @@
-- 添加文件哈希字段
ALTER TABLE files ADD COLUMN file_hash VARCHAR(64) COMMENT '文件哈希值';
CREATE UNIQUE INDEX idx_files_hash ON files(file_hash);
-- 添加同步状态字段
ALTER TABLE resources ADD COLUMN synced_to_meilisearch BOOLEAN DEFAULT FALSE;
ALTER TABLE resources ADD COLUMN synced_at TIMESTAMP NULL;
-- 创建索引以提高查询性能
CREATE INDEX idx_resources_synced ON resources(synced_to_meilisearch, synced_at);
-- 添加注释
COMMENT ON COLUMN resources.synced_to_meilisearch IS '是否已同步到Meilisearch';
COMMENT ON COLUMN resources.synced_at IS '同步时间';

View File

@@ -60,6 +60,15 @@ server {
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
# 缓存设置
expires 1y;
add_header Cache-Control "public, immutable";
# 允许跨域访问
add_header Access-Control-Allow-Origin "*";
add_header Access-Control-Allow-Methods "GET, OPTIONS";
add_header Access-Control-Allow-Headers "Origin, Content-Type, Accept";
}
# 健康检查

444
scheduler/auto_transfer.go Normal file
View File

@@ -0,0 +1,444 @@
package scheduler
import (
"fmt"
"math/rand"
"strings"
"sync"
"time"
panutils "github.com/ctwj/urldb/common"
commonutils "github.com/ctwj/urldb/common/utils"
"github.com/ctwj/urldb/db/entity"
"github.com/ctwj/urldb/db/repo"
"github.com/ctwj/urldb/utils"
"gorm.io/gorm"
)
// AutoTransferScheduler 自动转存调度器
type AutoTransferScheduler struct {
*BaseScheduler
autoTransferRunning bool
autoTransferMutex sync.Mutex // 防止自动转存任务重叠执行
}
// NewAutoTransferScheduler 创建自动转存调度器
func NewAutoTransferScheduler(base *BaseScheduler) *AutoTransferScheduler {
return &AutoTransferScheduler{
BaseScheduler: base,
autoTransferRunning: false,
autoTransferMutex: sync.Mutex{},
}
}
// Start 启动自动转存定时任务
func (a *AutoTransferScheduler) Start() {
// 自动转存已经放弃,不再自动缓存
return
if a.autoTransferRunning {
utils.Info("自动转存定时任务已在运行中")
return
}
a.autoTransferRunning = true
utils.Info("启动自动转存定时任务")
go func() {
// 获取系统配置中的间隔时间
interval := 5 * time.Minute // 默认5分钟
if autoProcessInterval, err := a.systemConfigRepo.GetConfigInt(entity.ConfigKeyAutoProcessInterval); err == nil && autoProcessInterval > 0 {
interval = time.Duration(autoProcessInterval) * time.Minute
}
ticker := time.NewTicker(interval)
defer ticker.Stop()
utils.Info(fmt.Sprintf("自动转存定时任务已启动,间隔时间: %v", interval))
// 立即执行一次
a.processAutoTransfer()
for {
select {
case <-ticker.C:
// 使用TryLock防止任务重叠执行
if a.autoTransferMutex.TryLock() {
go func() {
defer a.autoTransferMutex.Unlock()
a.processAutoTransfer()
}()
} else {
utils.Info("上一次自动转存任务还在执行中,跳过本次执行")
}
case <-a.GetStopChan():
utils.Info("停止自动转存定时任务")
return
}
}
}()
}
// Stop 停止自动转存定时任务
func (a *AutoTransferScheduler) Stop() {
if !a.autoTransferRunning {
utils.Info("自动转存定时任务未在运行")
return
}
a.GetStopChan() <- true
a.autoTransferRunning = false
utils.Info("已发送停止信号给自动转存定时任务")
}
// IsAutoTransferRunning 检查自动转存任务是否正在运行
func (a *AutoTransferScheduler) IsAutoTransferRunning() bool {
return a.autoTransferRunning
}
// processAutoTransfer 处理自动转存
func (a *AutoTransferScheduler) processAutoTransfer() {
utils.Info("开始处理自动转存...")
// 检查系统配置,确认是否启用自动转存
autoTransferEnabled, err := a.systemConfigRepo.GetConfigBool(entity.ConfigKeyAutoTransferEnabled)
if err != nil {
utils.Error(fmt.Sprintf("获取系统配置失败: %v", err))
return
}
if !autoTransferEnabled {
utils.Info("自动转存功能已禁用")
return
}
// 获取quark平台ID
quarkPanID, err := a.getQuarkPanID()
if err != nil {
utils.Error(fmt.Sprintf("获取夸克网盘ID失败: %v", err))
return
}
// 获取所有账号
accounts, err := a.cksRepo.FindAll()
if err != nil {
utils.Error(fmt.Sprintf("获取网盘账号失败: %v", err))
return
}
// 获取最小存储空间配置
autoTransferMinSpace, err := a.systemConfigRepo.GetConfigInt(entity.ConfigKeyAutoTransferMinSpace)
if err != nil {
utils.Error(fmt.Sprintf("获取最小存储空间配置失败: %v", err))
return
}
// 过滤只保留已激活、quark平台、剩余空间足够的账号
minSpaceBytes := int64(autoTransferMinSpace) * 1024 * 1024 * 1024
var validAccounts []entity.Cks
for _, acc := range accounts {
if acc.IsValid && acc.PanID == quarkPanID && acc.LeftSpace >= minSpaceBytes {
validAccounts = append(validAccounts, acc)
}
}
if len(validAccounts) == 0 {
utils.Info("没有可用的quark网盘账号")
return
}
utils.Info(fmt.Sprintf("找到 %d 个可用quark网盘账号开始自动转存处理...", len(validAccounts)))
// 计算处理数量限制
// 假设每5秒转存一个资源每分钟20个5分钟100个
// 根据时间间隔和账号数量计算大致的处理数量
interval := 5 * time.Minute // 默认5分钟
if autoProcessInterval, err := a.systemConfigRepo.GetConfigInt(entity.ConfigKeyAutoProcessInterval); err == nil && autoProcessInterval > 0 {
interval = time.Duration(autoProcessInterval) * time.Minute
}
// 计算每分钟能处理的资源数量:账号数 * 12每分钟12个即每5秒一个
resourcesPerMinute := len(validAccounts) * 12
// 根据时间间隔计算总处理数量
maxProcessCount := int(float64(resourcesPerMinute) * interval.Minutes())
utils.Info(fmt.Sprintf("时间间隔: %v, 账号数: %d, 每分钟处理能力: %d, 最大处理数量: %d",
interval, len(validAccounts), resourcesPerMinute, maxProcessCount))
// 获取需要转存的资源(限制数量)
resources, err := a.getResourcesForTransfer(quarkPanID, maxProcessCount)
if err != nil {
utils.Error(fmt.Sprintf("获取需要转存的资源失败: %v", err))
return
}
if len(resources) == 0 {
utils.Info("没有需要转存的资源")
return
}
utils.Info(fmt.Sprintf("找到 %d 个需要转存的资源", len(resources)))
// 获取违禁词配置
forbiddenWords, err := a.systemConfigRepo.GetConfigValue(entity.ConfigKeyForbiddenWords)
if err != nil {
utils.Error(fmt.Sprintf("获取违禁词配置失败: %v", err))
forbiddenWords = "" // 如果获取失败,使用空字符串
}
// 过滤包含违禁词的资源,并标记违禁词错误
var filteredResources []*entity.Resource
var forbiddenResources []*entity.Resource
if forbiddenWords != "" {
words := strings.Split(forbiddenWords, ",")
// 清理违禁词数组,去除空格
var cleanWords []string
for _, word := range words {
word = strings.TrimSpace(word)
if word != "" {
cleanWords = append(cleanWords, word)
}
}
for _, resource := range resources {
shouldSkip := false
var matchedWords []string
title := strings.ToLower(resource.Title)
description := strings.ToLower(resource.Description)
for _, word := range cleanWords {
wordLower := strings.ToLower(word)
if strings.Contains(title, wordLower) || strings.Contains(description, wordLower) {
matchedWords = append(matchedWords, word)
shouldSkip = true
}
}
if shouldSkip {
// 标记为违禁词错误
resource.ErrorMsg = fmt.Sprintf("存在违禁词: %s", strings.Join(matchedWords, ", "))
forbiddenResources = append(forbiddenResources, resource)
utils.Info(fmt.Sprintf("标记违禁词资源: %s, 违禁词: %s", resource.Title, strings.Join(matchedWords, ", ")))
} else {
filteredResources = append(filteredResources, resource)
}
}
utils.Info(fmt.Sprintf("违禁词过滤后,剩余 %d 个资源需要转存,违禁词资源 %d 个", len(filteredResources), len(forbiddenResources)))
} else {
filteredResources = resources
}
// 注意:资源数量已在数据库查询时限制,无需再次限制
// 保存违禁词资源的错误信息
for _, resource := range forbiddenResources {
if err := a.resourceRepo.Update(resource); err != nil {
utils.Error(fmt.Sprintf("保存违禁词错误信息失败 (ID: %d): %v", resource.ID, err))
}
}
// 并发自动转存
resourceCh := make(chan *entity.Resource, len(filteredResources))
for _, res := range filteredResources {
resourceCh <- res
}
close(resourceCh)
var wg sync.WaitGroup
for _, account := range validAccounts {
wg.Add(1)
go func(acc entity.Cks) {
defer wg.Done()
factory := panutils.GetInstance() // 使用单例模式
for res := range resourceCh {
if err := a.transferResource(res, []entity.Cks{acc}, factory); err != nil {
utils.Error(fmt.Sprintf("转存资源失败 (ID: %d): %v", res.ID, err))
} else {
utils.Info(fmt.Sprintf("成功转存资源: %s", res.Title))
rand.Seed(utils.GetCurrentTime().UnixNano())
sleepSec := rand.Intn(3) + 1 // 1,2,3
time.Sleep(time.Duration(sleepSec) * time.Second)
}
}
}(account)
}
wg.Wait()
utils.Info(fmt.Sprintf("自动转存处理完成,账号数: %d处理资源数: %d违禁词资源数: %d",
len(validAccounts), len(filteredResources), len(forbiddenResources)))
}
// getQuarkPanID 获取夸克网盘ID
func (a *AutoTransferScheduler) getQuarkPanID() (uint, error) {
// 获取panRepo的实现以便访问数据库
panRepoImpl, ok := a.panRepo.(interface{ GetDB() *gorm.DB })
if !ok {
return 0, fmt.Errorf("panRepo不支持GetDB方法")
}
var quarkPan entity.Pan
err := panRepoImpl.GetDB().Where("name = ?", "quark").First(&quarkPan).Error
if err != nil {
return 0, fmt.Errorf("未找到quark平台: %v", err)
}
return quarkPan.ID, nil
}
// getResourcesForTransfer 获取需要转存的资源
func (a *AutoTransferScheduler) getResourcesForTransfer(quarkPanID uint, limit int) ([]*entity.Resource, error) {
// 获取最近24小时内的资源
sinceTime := utils.GetCurrentTime().Add(-24 * time.Hour)
// 使用资源仓库的方法获取需要转存的资源
repoImpl, ok := a.resourceRepo.(*repo.ResourceRepositoryImpl)
if !ok {
return nil, fmt.Errorf("资源仓库类型错误")
}
return repoImpl.GetResourcesForTransfer(quarkPanID, sinceTime, limit)
}
// transferResource 转存单个资源
func (a *AutoTransferScheduler) transferResource(resource *entity.Resource, accounts []entity.Cks, factory *panutils.PanFactory) error {
if len(accounts) == 0 {
return fmt.Errorf("没有可用的网盘账号")
}
account := accounts[0]
service, err := factory.CreatePanService(resource.URL, &panutils.PanConfig{
URL: resource.URL,
ExpiredType: 0,
IsType: 0,
Cookie: account.Ck,
})
if err != nil {
return fmt.Errorf("创建网盘服务失败: %v", err)
}
// 获取最小存储空间配置
autoTransferMinSpace, err := a.systemConfigRepo.GetConfigInt(entity.ConfigKeyAutoTransferMinSpace)
if err != nil {
utils.Error(fmt.Sprintf("获取最小存储空间配置失败: %v", err))
return err
}
// 检查账号剩余空间
minSpaceBytes := int64(autoTransferMinSpace) * 1024 * 1024 * 1024
if account.LeftSpace < minSpaceBytes {
return fmt.Errorf("账号剩余空间不足,需要 %d GB当前剩余 %d GB", autoTransferMinSpace, account.LeftSpace/1024/1024/1024)
}
// 提取分享ID
shareID, _ := commonutils.ExtractShareIdString(resource.URL)
// 转存资源
result, err := service.Transfer(shareID)
if err != nil {
// 更新错误信息
resource.ErrorMsg = err.Error()
a.resourceRepo.Update(resource)
return fmt.Errorf("转存失败: %v", err)
}
if result == nil || !result.Success {
errMsg := "转存失败"
if result != nil && result.Message != "" {
errMsg = result.Message
}
// 更新错误信息
resource.ErrorMsg = errMsg
a.resourceRepo.Update(resource)
return fmt.Errorf("转存失败: %s", errMsg)
}
// 提取转存链接、fid等
var saveURL, fid string
if data, ok := result.Data.(map[string]interface{}); ok {
if v, ok := data["shareUrl"]; ok {
saveURL, _ = v.(string)
}
if v, ok := data["fid"]; ok {
fid, _ = v.(string)
}
}
if saveURL == "" {
saveURL = result.ShareURL
}
// 更新资源信息
resource.SaveURL = saveURL
resource.CkID = &account.ID
resource.Fid = fid
resource.ErrorMsg = ""
// 保存更新
err = a.resourceRepo.Update(resource)
if err != nil {
return fmt.Errorf("保存转存结果失败: %v", err)
}
return nil
}
// selectBestAccount 选择最佳账号
func (a *AutoTransferScheduler) selectBestAccount(accounts []entity.Cks) *entity.Cks {
if len(accounts) == 0 {
return nil
}
// 获取最小存储空间配置
autoTransferMinSpace, err := a.systemConfigRepo.GetConfigInt(entity.ConfigKeyAutoTransferMinSpace)
if err != nil {
utils.Error(fmt.Sprintf("获取最小存储空间配置失败: %v", err))
return &accounts[0] // 返回第一个账号
}
minSpaceBytes := int64(autoTransferMinSpace) * 1024 * 1024 * 1024
var bestAccount *entity.Cks
var bestScore int64 = -1
for i := range accounts {
account := &accounts[i]
if account.LeftSpace < minSpaceBytes {
continue // 跳过空间不足的账号
}
score := a.calculateAccountScore(account)
if score > bestScore {
bestScore = score
bestAccount = account
}
}
return bestAccount
}
// calculateAccountScore 计算账号评分
func (a *AutoTransferScheduler) calculateAccountScore(account *entity.Cks) int64 {
// TODO: 实现账号评分算法
// 1. VIP账号加分
// 2. 剩余空间大的账号加分
// 3. 使用率低的账号加分
// 4. 可以根据历史使用情况调整评分
score := int64(0)
// VIP账号加分
if account.VipStatus {
score += 1000
}
// 剩余空间加分每GB加1分
score += account.LeftSpace / (1024 * 1024 * 1024)
// 使用率加分(使用率越低分数越高)
if account.Space > 0 {
usageRate := float64(account.UsedSpace) / float64(account.Space)
score += int64((1 - usageRate) * 500) // 使用率越低,加分越多
}
return score
}

88
scheduler/base.go Normal file
View File

@@ -0,0 +1,88 @@
package scheduler
import (
"sync"
"time"
"github.com/ctwj/urldb/db/repo"
)
// BaseScheduler 基础调度器结构
type BaseScheduler struct {
// 共享的仓库
hotDramaRepo repo.HotDramaRepository
readyResourceRepo repo.ReadyResourceRepository
resourceRepo repo.ResourceRepository
systemConfigRepo repo.SystemConfigRepository
panRepo repo.PanRepository
cksRepo repo.CksRepository
tagRepo repo.TagRepository
categoryRepo repo.CategoryRepository
// 控制字段
stopChan chan bool
isRunning bool
// 平台映射缓存
panCache map[string]*uint // serviceType -> panID
panCacheOnce sync.Once
}
// NewBaseScheduler 创建基础调度器
func NewBaseScheduler(
hotDramaRepo repo.HotDramaRepository,
readyResourceRepo repo.ReadyResourceRepository,
resourceRepo repo.ResourceRepository,
systemConfigRepo repo.SystemConfigRepository,
panRepo repo.PanRepository,
cksRepo repo.CksRepository,
tagRepo repo.TagRepository,
categoryRepo repo.CategoryRepository,
) *BaseScheduler {
return &BaseScheduler{
hotDramaRepo: hotDramaRepo,
readyResourceRepo: readyResourceRepo,
resourceRepo: resourceRepo,
systemConfigRepo: systemConfigRepo,
panRepo: panRepo,
cksRepo: cksRepo,
tagRepo: tagRepo,
categoryRepo: categoryRepo,
stopChan: make(chan bool),
isRunning: false,
panCache: make(map[string]*uint),
}
}
// Stop 停止调度器
func (b *BaseScheduler) Stop() {
if b.isRunning {
b.stopChan <- true
b.isRunning = false
}
}
// IsRunning 检查是否正在运行
func (b *BaseScheduler) IsRunning() bool {
return b.isRunning
}
// SetRunning 设置运行状态
func (b *BaseScheduler) SetRunning(running bool) {
b.isRunning = running
}
// GetStopChan 获取停止通道
func (b *BaseScheduler) GetStopChan() chan bool {
return b.stopChan
}
// SleepWithStopCheck 带停止检查的睡眠
func (b *BaseScheduler) SleepWithStopCheck(duration time.Duration) bool {
select {
case <-time.After(duration):
return false
case <-b.stopChan:
return true
}
}

184
scheduler/global.go Normal file
View File

@@ -0,0 +1,184 @@
package scheduler
import (
"sync"
"github.com/ctwj/urldb/db/repo"
"github.com/ctwj/urldb/utils"
)
// GlobalScheduler 全局调度器管理器
type GlobalScheduler struct {
manager *Manager
mutex sync.RWMutex
}
var (
globalScheduler *GlobalScheduler
once sync.Once
)
// GetGlobalScheduler 获取全局调度器实例(单例模式)
func GetGlobalScheduler(hotDramaRepo repo.HotDramaRepository, readyResourceRepo repo.ReadyResourceRepository, resourceRepo repo.ResourceRepository, systemConfigRepo repo.SystemConfigRepository, panRepo repo.PanRepository, cksRepo repo.CksRepository, tagRepo repo.TagRepository, categoryRepo repo.CategoryRepository) *GlobalScheduler {
once.Do(func() {
globalScheduler = &GlobalScheduler{
manager: NewManager(hotDramaRepo, readyResourceRepo, resourceRepo, systemConfigRepo, panRepo, cksRepo, tagRepo, categoryRepo),
}
})
return globalScheduler
}
// StartHotDramaScheduler 启动热播剧定时任务
func (gs *GlobalScheduler) StartHotDramaScheduler() {
gs.mutex.Lock()
defer gs.mutex.Unlock()
if gs.manager.IsHotDramaRunning() {
utils.Info("热播剧定时任务已在运行中")
return
}
gs.manager.StartHotDramaScheduler()
utils.Info("全局调度器已启动热播剧定时任务")
}
// StopHotDramaScheduler 停止热播剧定时任务
func (gs *GlobalScheduler) StopHotDramaScheduler() {
gs.mutex.Lock()
defer gs.mutex.Unlock()
if !gs.manager.IsHotDramaRunning() {
utils.Info("热播剧定时任务未在运行")
return
}
gs.manager.StopHotDramaScheduler()
utils.Info("全局调度器已停止热播剧定时任务")
}
// IsHotDramaSchedulerRunning 检查热播剧定时任务是否在运行
func (gs *GlobalScheduler) IsHotDramaSchedulerRunning() bool {
gs.mutex.RLock()
defer gs.mutex.RUnlock()
return gs.manager.IsHotDramaRunning()
}
// GetHotDramaNames 手动获取热播剧名字
func (gs *GlobalScheduler) GetHotDramaNames() ([]string, error) {
return gs.manager.GetHotDramaNames()
}
// StartReadyResourceScheduler 启动待处理资源自动处理任务
func (gs *GlobalScheduler) StartReadyResourceScheduler() {
gs.mutex.Lock()
defer gs.mutex.Unlock()
if gs.manager.IsReadyResourceRunning() {
utils.Info("待处理资源自动处理任务已在运行中")
return
}
gs.manager.StartReadyResourceScheduler()
utils.Info("全局调度器已启动待处理资源自动处理任务")
}
// StopReadyResourceScheduler 停止待处理资源自动处理任务
func (gs *GlobalScheduler) StopReadyResourceScheduler() {
gs.mutex.Lock()
defer gs.mutex.Unlock()
if !gs.manager.IsReadyResourceRunning() {
utils.Info("待处理资源自动处理任务未在运行")
return
}
gs.manager.StopReadyResourceScheduler()
utils.Info("全局调度器已停止待处理资源自动处理任务")
}
// IsReadyResourceRunning 检查待处理资源自动处理任务是否在运行
func (gs *GlobalScheduler) IsReadyResourceRunning() bool {
gs.mutex.RLock()
defer gs.mutex.RUnlock()
return gs.manager.IsReadyResourceRunning()
}
// StartAutoTransferScheduler 启动自动转存定时任务
func (gs *GlobalScheduler) StartAutoTransferScheduler() {
gs.mutex.Lock()
defer gs.mutex.Unlock()
if gs.manager.IsAutoTransferRunning() {
utils.Info("自动转存定时任务已在运行中")
return
}
gs.manager.StartAutoTransferScheduler()
utils.Info("全局调度器已启动自动转存定时任务")
}
// StopAutoTransferScheduler 停止自动转存定时任务
func (gs *GlobalScheduler) StopAutoTransferScheduler() {
gs.mutex.Lock()
defer gs.mutex.Unlock()
if !gs.manager.IsAutoTransferRunning() {
utils.Info("自动转存定时任务未在运行")
return
}
gs.manager.StopAutoTransferScheduler()
utils.Info("全局调度器已停止自动转存定时任务")
}
// IsAutoTransferRunning 检查自动转存定时任务是否在运行
func (gs *GlobalScheduler) IsAutoTransferRunning() bool {
gs.mutex.RLock()
defer gs.mutex.RUnlock()
return gs.manager.IsAutoTransferRunning()
}
// UpdateSchedulerStatusWithAutoTransfer 根据系统配置更新调度器状态(包含自动转存)
func (gs *GlobalScheduler) UpdateSchedulerStatusWithAutoTransfer(autoFetchHotDramaEnabled bool, autoProcessReadyResources bool, autoTransferEnabled bool) {
gs.mutex.Lock()
defer gs.mutex.Unlock()
// 处理热播剧自动拉取功能
if autoFetchHotDramaEnabled {
if !gs.manager.IsHotDramaRunning() {
utils.Info("系统配置启用自动拉取热播剧,启动定时任务")
gs.manager.StartHotDramaScheduler()
}
} else {
if gs.manager.IsHotDramaRunning() {
utils.Info("系统配置禁用自动拉取热播剧,停止定时任务")
gs.manager.StopHotDramaScheduler()
}
}
// 处理待处理资源自动处理功能
if autoProcessReadyResources {
if !gs.manager.IsReadyResourceRunning() {
utils.Info("系统配置启用自动处理待处理资源,启动定时任务")
gs.manager.StartReadyResourceScheduler()
}
} else {
if gs.manager.IsReadyResourceRunning() {
utils.Info("系统配置禁用自动处理待处理资源,停止定时任务")
gs.manager.StopReadyResourceScheduler()
}
}
// 处理自动转存功能
if autoTransferEnabled {
if !gs.manager.IsAutoTransferRunning() {
utils.Info("系统配置启用自动转存,启动定时任务")
gs.manager.StartAutoTransferScheduler()
}
} else {
if gs.manager.IsAutoTransferRunning() {
utils.Info("系统配置禁用自动转存,停止定时任务")
gs.manager.StopAutoTransferScheduler()
}
}
}

235
scheduler/hot_drama.go Normal file
View File

@@ -0,0 +1,235 @@
package scheduler
import (
"fmt"
"strings"
"sync"
"time"
"github.com/ctwj/urldb/db/entity"
"github.com/ctwj/urldb/utils"
)
// HotDramaScheduler 热播剧调度器
type HotDramaScheduler struct {
*BaseScheduler
doubanService *utils.DoubanService
hotDramaMutex sync.Mutex // 防止热播剧任务重叠执行
}
// NewHotDramaScheduler 创建热播剧调度器
func NewHotDramaScheduler(base *BaseScheduler) *HotDramaScheduler {
return &HotDramaScheduler{
BaseScheduler: base,
doubanService: utils.NewDoubanService(),
hotDramaMutex: sync.Mutex{},
}
}
// Start 启动热播剧定时任务
func (h *HotDramaScheduler) Start() {
if h.IsRunning() {
utils.Info("热播剧定时任务已在运行中")
return
}
h.SetRunning(true)
utils.Info("启动热播剧定时任务")
go func() {
ticker := time.NewTicker(12 * time.Hour) // 每12小时执行一次
defer ticker.Stop()
// 立即执行一次
h.fetchHotDramaData()
for {
select {
case <-ticker.C:
// 使用TryLock防止任务重叠执行
if h.hotDramaMutex.TryLock() {
go func() {
defer h.hotDramaMutex.Unlock()
h.fetchHotDramaData()
}()
} else {
utils.Info("上一次热播剧任务还在执行中,跳过本次执行")
}
case <-h.GetStopChan():
utils.Info("停止热播剧定时任务")
return
}
}
}()
}
// Stop 停止热播剧定时任务
func (h *HotDramaScheduler) Stop() {
if !h.IsRunning() {
utils.Info("热播剧定时任务未在运行")
return
}
h.GetStopChan() <- true
h.SetRunning(false)
utils.Info("已发送停止信号给热播剧定时任务")
}
// fetchHotDramaData 获取热播剧数据
func (h *HotDramaScheduler) fetchHotDramaData() {
utils.Info("开始获取热播剧数据...")
// 直接处理电影和电视剧数据不再需要FetchHotDramaNames
h.processHotDramaNames([]string{})
}
// processHotDramaNames 处理热播剧名称
func (h *HotDramaScheduler) processHotDramaNames(dramaNames []string) {
utils.Info("开始处理热播剧数据,共 %d 个", len(dramaNames))
// 收集所有数据
var allDramas []*entity.HotDrama
// 获取电影数据
movieDramas := h.processMovieData()
allDramas = append(allDramas, movieDramas...)
// 获取电视剧数据
tvDramas := h.processTvData()
allDramas = append(allDramas, tvDramas...)
// 清空数据库
utils.Info("准备清空数据库,当前共有 %d 条数据", len(allDramas))
if err := h.hotDramaRepo.DeleteAll(); err != nil {
utils.Error(fmt.Sprintf("清空数据库失败: %v", err))
return
}
utils.Info("数据库清空完成")
// 批量插入所有数据
if len(allDramas) > 0 {
utils.Info("开始批量插入 %d 条数据", len(allDramas))
if err := h.hotDramaRepo.BatchCreate(allDramas); err != nil {
utils.Error(fmt.Sprintf("批量插入数据失败: %v", err))
} else {
utils.Info("成功批量插入 %d 条数据", len(allDramas))
}
} else {
utils.Info("没有数据需要插入")
}
utils.Info("热播剧数据处理完成")
}
// processMovieData 处理电影数据
func (h *HotDramaScheduler) processMovieData() []*entity.HotDrama {
utils.Info("开始处理电影数据...")
var movieDramas []*entity.HotDrama
// 使用GetTypePage方法获取电影数据
movieResult, err := h.doubanService.GetTypePage("热门", "全部")
if err != nil {
utils.Error(fmt.Sprintf("获取电影榜单失败: %v", err))
return movieDramas
}
if movieResult.Success && movieResult.Data != nil {
utils.Info("电影获取到 %d 个数据", len(movieResult.Data.Items))
for _, item := range movieResult.Data.Items {
drama := &entity.HotDrama{
Title: item.Title,
CardSubtitle: item.CardSubtitle,
EpisodesInfo: item.EpisodesInfo,
IsNew: item.IsNew,
Rating: item.Rating.Value,
RatingCount: item.Rating.Count,
Year: item.Year,
Region: item.Region,
Genres: strings.Join(item.Genres, ", "),
Directors: strings.Join(item.Directors, ", "),
Actors: strings.Join(item.Actors, ", "),
PosterURL: item.Pic.Normal,
Category: "电影",
SubType: "热门",
Source: "douban",
DoubanID: item.ID,
DoubanURI: item.URI,
}
movieDramas = append(movieDramas, drama)
utils.Info("收集电影: %s (评分: %.1f, 年份: %s, 地区: %s)",
item.Title, item.Rating.Value, item.Year, item.Region)
}
} else {
utils.Warn("电影获取数据失败或为空")
}
utils.Info("电影数据处理完成,共收集 %d 条数据", len(movieDramas))
return movieDramas
}
// processTvData 处理电视剧数据
func (h *HotDramaScheduler) processTvData() []*entity.HotDrama {
utils.Info("开始处理电视剧数据...")
var tvDramas []*entity.HotDrama
// 获取所有tv类型
tvTypes := h.doubanService.GetAllTvTypes()
utils.Info("获取到 %d 个tv类型: %v", len(tvTypes), tvTypes)
// 遍历每个type分别请求数据
for _, tvType := range tvTypes {
utils.Info("正在处理tv类型: %s", tvType)
// 使用GetTypePage方法请求数据
tvResult, err := h.doubanService.GetTypePage("tv", tvType)
if err != nil {
utils.Error(fmt.Sprintf("获取tv类型 %s 数据失败: %v", tvType, err))
continue
}
if tvResult.Success && tvResult.Data != nil {
utils.Info("tv类型 %s 获取到 %d 个数据", tvType, len(tvResult.Data.Items))
for _, item := range tvResult.Data.Items {
drama := &entity.HotDrama{
Title: item.Title,
CardSubtitle: item.CardSubtitle,
EpisodesInfo: item.EpisodesInfo,
IsNew: item.IsNew,
Rating: item.Rating.Value,
RatingCount: item.Rating.Count,
Year: item.Year,
Region: item.Region,
Genres: strings.Join(item.Genres, ", "),
Directors: strings.Join(item.Directors, ", "),
Actors: strings.Join(item.Actors, ", "),
PosterURL: item.Pic.Normal,
Category: "电视剧",
SubType: tvType, // 使用具体的tv类型
Source: "douban",
DoubanID: item.ID,
DoubanURI: item.URI,
}
tvDramas = append(tvDramas, drama)
utils.Info("收集tv类型 %s: %s (评分: %.1f, 年份: %s, 地区: %s)",
tvType, item.Title, item.Rating.Value, item.Year, item.Region)
}
} else {
utils.Warn("tv类型 %s 获取数据失败或为空", tvType)
}
}
utils.Info("电视剧数据处理完成,共收集 %d 条数据", len(tvDramas))
return tvDramas
}
// GetHotDramaNames 获取热播剧名称列表(公共方法)
func (h *HotDramaScheduler) GetHotDramaNames() ([]string, error) {
// 由于删除了FetchHotDramaNames方法返回空数组
return []string{}, nil
}

141
scheduler/manager.go Normal file
View File

@@ -0,0 +1,141 @@
package scheduler
import (
"github.com/ctwj/urldb/db/repo"
"github.com/ctwj/urldb/utils"
)
// Manager 调度器管理器
type Manager struct {
baseScheduler *BaseScheduler
hotDramaScheduler *HotDramaScheduler
readyResourceScheduler *ReadyResourceScheduler
autoTransferScheduler *AutoTransferScheduler
}
// NewManager 创建调度器管理器
func NewManager(
hotDramaRepo repo.HotDramaRepository,
readyResourceRepo repo.ReadyResourceRepository,
resourceRepo repo.ResourceRepository,
systemConfigRepo repo.SystemConfigRepository,
panRepo repo.PanRepository,
cksRepo repo.CksRepository,
tagRepo repo.TagRepository,
categoryRepo repo.CategoryRepository,
) *Manager {
// 创建基础调度器
baseScheduler := NewBaseScheduler(
hotDramaRepo,
readyResourceRepo,
resourceRepo,
systemConfigRepo,
panRepo,
cksRepo,
tagRepo,
categoryRepo,
)
// 创建各个具体的调度器
hotDramaScheduler := NewHotDramaScheduler(baseScheduler)
readyResourceScheduler := NewReadyResourceScheduler(baseScheduler)
autoTransferScheduler := NewAutoTransferScheduler(baseScheduler)
return &Manager{
baseScheduler: baseScheduler,
hotDramaScheduler: hotDramaScheduler,
readyResourceScheduler: readyResourceScheduler,
autoTransferScheduler: autoTransferScheduler,
}
}
// StartAll 启动所有调度任务
func (m *Manager) StartAll() {
utils.Info("启动所有调度任务")
// 启动热播剧调度任务
m.hotDramaScheduler.Start()
// 启动待处理资源调度任务
m.readyResourceScheduler.Start()
// 启动自动转存调度任务
m.autoTransferScheduler.Start()
utils.Info("所有调度任务已启动")
}
// StopAll 停止所有调度任务
func (m *Manager) StopAll() {
utils.Info("停止所有调度任务")
// 停止热播剧调度任务
m.hotDramaScheduler.Stop()
// 停止待处理资源调度任务
m.readyResourceScheduler.Stop()
// 停止自动转存调度任务
m.autoTransferScheduler.Stop()
utils.Info("所有调度任务已停止")
}
// StartHotDramaScheduler 启动热播剧调度任务
func (m *Manager) StartHotDramaScheduler() {
m.hotDramaScheduler.Start()
}
// StopHotDramaScheduler 停止热播剧调度任务
func (m *Manager) StopHotDramaScheduler() {
m.hotDramaScheduler.Stop()
}
// IsHotDramaRunning 检查热播剧调度任务是否正在运行
func (m *Manager) IsHotDramaRunning() bool {
return m.hotDramaScheduler.IsRunning()
}
// StartReadyResourceScheduler 启动待处理资源调度任务
func (m *Manager) StartReadyResourceScheduler() {
m.readyResourceScheduler.Start()
}
// StopReadyResourceScheduler 停止待处理资源调度任务
func (m *Manager) StopReadyResourceScheduler() {
m.readyResourceScheduler.Stop()
}
// IsReadyResourceRunning 检查待处理资源调度任务是否正在运行
func (m *Manager) IsReadyResourceRunning() bool {
return m.readyResourceScheduler.IsReadyResourceRunning()
}
// StartAutoTransferScheduler 启动自动转存调度任务
func (m *Manager) StartAutoTransferScheduler() {
m.autoTransferScheduler.Start()
}
// StopAutoTransferScheduler 停止自动转存调度任务
func (m *Manager) StopAutoTransferScheduler() {
m.autoTransferScheduler.Stop()
}
// IsAutoTransferRunning 检查自动转存调度任务是否正在运行
func (m *Manager) IsAutoTransferRunning() bool {
return m.autoTransferScheduler.IsAutoTransferRunning()
}
// GetHotDramaNames 获取热播剧名称列表
func (m *Manager) GetHotDramaNames() ([]string, error) {
return m.hotDramaScheduler.GetHotDramaNames()
}
// GetStatus 获取所有调度任务的状态
func (m *Manager) GetStatus() map[string]bool {
return map[string]bool{
"hot_drama": m.IsHotDramaRunning(),
"ready_resource": m.IsReadyResourceRunning(),
"auto_transfer": m.IsAutoTransferRunning(),
}
}

490
scheduler/ready_resource.go Normal file
View File

@@ -0,0 +1,490 @@
package scheduler
import (
"fmt"
"strings"
"sync"
"time"
panutils "github.com/ctwj/urldb/common"
commonutils "github.com/ctwj/urldb/common/utils"
"github.com/ctwj/urldb/db/entity"
"github.com/ctwj/urldb/utils"
)
// ReadyResourceScheduler 待处理资源调度器
type ReadyResourceScheduler struct {
*BaseScheduler
readyResourceRunning bool
processingMutex sync.Mutex // 防止ready_resource任务重叠执行
}
// NewReadyResourceScheduler 创建待处理资源调度器
func NewReadyResourceScheduler(base *BaseScheduler) *ReadyResourceScheduler {
return &ReadyResourceScheduler{
BaseScheduler: base,
readyResourceRunning: false,
processingMutex: sync.Mutex{},
}
}
// Start 启动待处理资源定时任务
func (r *ReadyResourceScheduler) Start() {
if r.readyResourceRunning {
utils.Info("待处理资源自动处理任务已在运行中")
return
}
r.readyResourceRunning = true
utils.Info("启动待处理资源自动处理任务")
go func() {
// 获取系统配置中的间隔时间
interval := 3 * time.Minute // 默认3分钟
if autoProcessInterval, err := r.systemConfigRepo.GetConfigInt(entity.ConfigKeyAutoProcessInterval); err == nil && autoProcessInterval > 0 {
interval = time.Duration(autoProcessInterval) * time.Minute
}
ticker := time.NewTicker(interval)
defer ticker.Stop()
utils.Info(fmt.Sprintf("待处理资源自动处理任务已启动,间隔时间: %v", interval))
// 立即执行一次
r.processReadyResources()
for {
select {
case <-ticker.C:
// 使用TryLock防止任务重叠执行
if r.processingMutex.TryLock() {
go func() {
defer r.processingMutex.Unlock()
r.processReadyResources()
}()
} else {
utils.Info("上一次待处理资源任务还在执行中,跳过本次执行")
}
case <-r.GetStopChan():
utils.Info("停止待处理资源自动处理任务")
return
}
}
}()
}
// Stop 停止待处理资源定时任务
func (r *ReadyResourceScheduler) Stop() {
if !r.readyResourceRunning {
utils.Info("待处理资源自动处理任务未在运行")
return
}
r.GetStopChan() <- true
r.readyResourceRunning = false
utils.Info("已发送停止信号给待处理资源自动处理任务")
}
// IsReadyResourceRunning 检查待处理资源任务是否正在运行
func (r *ReadyResourceScheduler) IsReadyResourceRunning() bool {
return r.readyResourceRunning
}
// processReadyResources 处理待处理资源
func (r *ReadyResourceScheduler) processReadyResources() {
utils.Info("开始处理待处理资源...")
// 检查系统配置,确认是否启用自动处理
autoProcess, err := r.systemConfigRepo.GetConfigBool(entity.ConfigKeyAutoProcessReadyResources)
if err != nil {
utils.Error(fmt.Sprintf("获取系统配置失败: %v", err))
return
}
if !autoProcess {
utils.Info("自动处理待处理资源功能已禁用")
return
}
// 获取所有没有错误的待处理资源
readyResources, err := r.readyResourceRepo.FindAll()
// readyResources, err := r.readyResourceRepo.FindWithoutErrors()
if err != nil {
utils.Error(fmt.Sprintf("获取待处理资源失败: %v", err))
return
}
if len(readyResources) == 0 {
utils.Info("没有待处理的资源")
return
}
utils.Info(fmt.Sprintf("找到 %d 个待处理资源,开始处理...", len(readyResources)))
processedCount := 0
factory := panutils.GetInstance() // 使用单例模式
for _, readyResource := range readyResources {
//readyResource.URL 是 查重
exits, err := r.resourceRepo.FindExists(readyResource.URL)
if err != nil {
utils.Error(fmt.Sprintf("查重失败: %v", err))
continue
}
if exits {
utils.Info(fmt.Sprintf("资源已存在: %s", readyResource.URL))
r.readyResourceRepo.Delete(readyResource.ID)
continue
}
if err := r.convertReadyResourceToResource(readyResource, factory); err != nil {
utils.Error(fmt.Sprintf("处理资源失败 (ID: %d): %v", readyResource.ID, err))
// 保存完整的错误信息
readyResource.ErrorMsg = err.Error()
if updateErr := r.readyResourceRepo.Update(&readyResource); updateErr != nil {
utils.Error(fmt.Sprintf("更新错误信息失败 (ID: %d): %v", readyResource.ID, updateErr))
} else {
utils.Info(fmt.Sprintf("已保存错误信息到资源 (ID: %d): %s", readyResource.ID, err.Error()))
}
// 处理失败后删除资源,避免重复处理
r.readyResourceRepo.Delete(readyResource.ID)
} else {
// 处理成功删除readyResource
r.readyResourceRepo.Delete(readyResource.ID)
processedCount++
utils.Info(fmt.Sprintf("成功处理资源: %s", readyResource.URL))
}
}
utils.Info(fmt.Sprintf("待处理资源处理完成,共处理 %d 个资源", processedCount))
}
// convertReadyResourceToResource 将待处理资源转换为正式资源
func (r *ReadyResourceScheduler) convertReadyResourceToResource(readyResource entity.ReadyResource, factory *panutils.PanFactory) error {
utils.Debug(fmt.Sprintf("开始处理资源: %s", readyResource.URL))
// 提取分享ID和服务类型
shareID, serviceType := panutils.ExtractShareId(readyResource.URL)
if serviceType == panutils.NotFound {
utils.Warn(fmt.Sprintf("不支持的链接地址: %s", readyResource.URL))
return fmt.Errorf("不支持的链接地址: %s", readyResource.URL)
}
utils.Debug(fmt.Sprintf("检测到服务类型: %s, 分享ID: %s", serviceType.String(), shareID))
resource := &entity.Resource{
Title: derefString(readyResource.Title),
Description: readyResource.Description,
URL: readyResource.URL,
Cover: readyResource.Img,
IsValid: true,
IsPublic: true,
Key: readyResource.Key,
PanID: r.getPanIDByServiceType(serviceType),
}
// 检查违禁词
forbiddenWords, err := r.systemConfigRepo.GetConfigValue(entity.ConfigKeyForbiddenWords)
if err == nil && forbiddenWords != "" {
words := strings.Split(forbiddenWords, ",")
var matchedWords []string
title := strings.ToLower(resource.Title)
description := strings.ToLower(resource.Description)
for _, word := range words {
word = strings.TrimSpace(word)
if word != "" {
wordLower := strings.ToLower(word)
if strings.Contains(title, wordLower) || strings.Contains(description, wordLower) {
matchedWords = append(matchedWords, word)
}
}
}
if len(matchedWords) > 0 {
utils.Warn(fmt.Sprintf("资源包含违禁词: %s, 违禁词: %s", resource.Title, strings.Join(matchedWords, ", ")))
return fmt.Errorf("存在违禁词: %s", strings.Join(matchedWords, ", "))
}
}
// 不是夸克,直接保存
if serviceType != panutils.Quark {
// 检测是否有效
checkResult, err := commonutils.CheckURL(readyResource.URL)
if err != nil {
utils.Error(fmt.Sprintf("链接检查失败: %v", err))
return fmt.Errorf("链接检查失败: %v", err)
}
if !checkResult.Status {
utils.Warn(fmt.Sprintf("链接无效: %s", readyResource.URL))
return fmt.Errorf("链接无效: %s", readyResource.URL)
}
} else {
// 获取夸克网盘账号的 cookie
panID := r.getPanIDByServiceType(serviceType)
if panID == nil {
utils.Error("未找到对应的平台ID")
return fmt.Errorf("未找到对应的平台ID")
}
accounts, err := r.cksRepo.FindByPanID(*panID)
if err != nil {
utils.Error(fmt.Sprintf("获取夸克网盘账号失败: %v", err))
return fmt.Errorf("获取网盘账号失败: %v", err)
}
if len(accounts) == 0 {
utils.Error("没有可用的夸克网盘账号")
return fmt.Errorf("没有可用的夸克网盘账号")
}
// 选择第一个有效的账号
var selectedAccount *entity.Cks
for _, account := range accounts {
if account.IsValid {
selectedAccount = &account
break
}
}
if selectedAccount == nil {
utils.Error("没有有效的夸克网盘账号")
return fmt.Errorf("没有有效的夸克网盘账号")
}
utils.Debug(fmt.Sprintf("使用夸克网盘账号: %d, Cookie: %s", selectedAccount.ID, selectedAccount.Ck[:20]+"..."))
// 准备配置
config := &panutils.PanConfig{
URL: readyResource.URL,
Code: "", // 可以从readyResource中获取
IsType: 1, // 转存并分享后的资源信息 0 转存后分享, 1 只获取基本信息
ExpiredType: 1, // 永久分享
AdFid: "",
Stoken: "",
Cookie: selectedAccount.Ck, // 添加 cookie
}
// 通过工厂获取对应的网盘服务单例
panService, err := factory.CreatePanService(readyResource.URL, config)
if err != nil {
utils.Error(fmt.Sprintf("获取网盘服务失败: %v", err))
return fmt.Errorf("获取网盘服务失败: %v", err)
}
// 统一处理:尝试转存获取标题
result, err := panService.Transfer(shareID)
if err != nil {
utils.Error(fmt.Sprintf("网盘信息获取失败: %v", err))
return fmt.Errorf("网盘信息获取失败: %v", err)
}
if !result.Success {
utils.Error(fmt.Sprintf("网盘信息获取失败: %s", result.Message))
return fmt.Errorf("网盘信息获取失败: %s", result.Message)
}
// 从结果中提取标题等信息
if result.Data != nil {
if data, ok := result.Data.(map[string]interface{}); ok {
if title, ok := data["title"].(string); ok && title != "" {
resource.Title = title
}
if description, ok := data["description"].(string); ok && description != "" {
resource.Description = description
}
}
}
}
// 处理分类
if readyResource.Category != "" {
categoryID, err := r.resolveCategory(readyResource.Category, nil)
if err != nil {
utils.Error(fmt.Sprintf("解析分类失败: %v", err))
} else {
resource.CategoryID = categoryID
}
}
// 处理标签
if readyResource.Tags != "" {
tagIDs, err := r.handleTags(readyResource.Tags)
if err != nil {
utils.Error(fmt.Sprintf("处理标签失败: %v", err))
} else {
// 保存资源
err = r.resourceRepo.Create(resource)
if err != nil {
return fmt.Errorf("创建资源失败: %v", err)
}
// 创建资源标签关联
for _, tagID := range tagIDs {
resourceTag := &entity.ResourceTag{
ResourceID: resource.ID,
TagID: tagID,
}
err = r.resourceRepo.CreateResourceTag(resourceTag)
if err != nil {
utils.Error(fmt.Sprintf("创建资源标签关联失败: %v", err))
}
}
}
} else {
// 保存资源
err := r.resourceRepo.Create(resource)
if err != nil {
return fmt.Errorf("创建资源失败: %v", err)
}
}
return nil
}
// initPanCache 初始化平台缓存
func (r *ReadyResourceScheduler) initPanCache() {
r.panCacheOnce.Do(func() {
// 获取所有平台数据
pans, err := r.panRepo.FindAll()
if err != nil {
utils.Error(fmt.Sprintf("初始化平台缓存失败: %v", err))
return
}
// 建立 ServiceType 到 PanID 的映射
serviceTypeToPanName := map[string]string{
"quark": "quark",
"alipan": "aliyun", // 阿里云盘在数据库中的名称是 aliyun
"baidu": "baidu",
"uc": "uc",
"xunlei": "xunlei",
"tianyi": "tianyi",
"123pan": "123pan",
"115": "115",
"unknown": "other",
}
// 创建平台名称到ID的映射
panNameToID := make(map[string]*uint)
for _, pan := range pans {
panID := pan.ID
panNameToID[pan.Name] = &panID
}
// 建立 ServiceType 到 PanID 的映射
for serviceType, panName := range serviceTypeToPanName {
if panID, exists := panNameToID[panName]; exists {
r.panCache[serviceType] = panID
utils.Info(fmt.Sprintf("平台映射缓存: %s -> %s (ID: %d)", serviceType, panName, *panID))
} else {
utils.Error(fmt.Sprintf("警告: 未找到平台 %s 对应的数据库记录", panName))
}
}
// 确保有默认的 other 平台
if otherID, exists := panNameToID["other"]; exists {
r.panCache["unknown"] = otherID
}
utils.Info(fmt.Sprintf("平台映射缓存初始化完成,共 %d 个映射", len(r.panCache)))
})
}
// getPanIDByServiceType 根据服务类型获取平台ID
func (r *ReadyResourceScheduler) getPanIDByServiceType(serviceType panutils.ServiceType) *uint {
r.initPanCache()
serviceTypeStr := serviceType.String()
if panID, exists := r.panCache[serviceTypeStr]; exists {
return panID
}
// 如果找不到,返回 other 平台的ID
if otherID, exists := r.panCache["other"]; exists {
utils.Error(fmt.Sprintf("未找到服务类型 %s 的映射,使用默认平台 other", serviceTypeStr))
return otherID
}
utils.Error(fmt.Sprintf("未找到服务类型 %s 的映射且没有默认平台返回nil", serviceTypeStr))
return nil
}
// handleTags 处理标签
func (r *ReadyResourceScheduler) handleTags(tagStr string) ([]uint, error) {
if tagStr == "" {
return nil, nil
}
tagNames := splitTags(tagStr)
var tagIDs []uint
for _, tagName := range tagNames {
tagName = strings.TrimSpace(tagName)
if tagName == "" {
continue
}
// 查找或创建标签
tag, err := r.tagRepo.FindByName(tagName)
if err != nil {
// 标签不存在,创建新标签
tag = &entity.Tag{
Name: tagName,
}
err = r.tagRepo.Create(tag)
if err != nil {
utils.Error(fmt.Sprintf("创建标签失败: %v", err))
continue
}
}
tagIDs = append(tagIDs, tag.ID)
}
return tagIDs, nil
}
// resolveCategory 解析分类
func (r *ReadyResourceScheduler) resolveCategory(categoryName string, tagIDs []uint) (*uint, error) {
if categoryName == "" {
return nil, nil
}
// 查找分类
category, err := r.categoryRepo.FindByName(categoryName)
if err != nil {
// 分类不存在,创建新分类
category = &entity.Category{
Name: categoryName,
}
err = r.categoryRepo.Create(category)
if err != nil {
return nil, fmt.Errorf("创建分类失败: %v", err)
}
}
return &category.ID, nil
}
// splitTags 分割标签字符串
func splitTags(tagStr string) []string {
// 支持多种分隔符
tagStr = strings.ReplaceAll(tagStr, "", ",")
tagStr = strings.ReplaceAll(tagStr, "", ",")
tagStr = strings.ReplaceAll(tagStr, ";", ",")
tagStr = strings.ReplaceAll(tagStr, "、", ",")
return strings.Split(tagStr, ",")
}
// derefString 解引用字符串指针
func derefString(s *string) string {
if s == nil {
return ""
}
return *s
}

178
scripts/build.sh Normal file
View File

@@ -0,0 +1,178 @@
#!/bin/bash
# 编译脚本 - 自动注入版本信息
# 用法: ./scripts/build.sh [target]
set -e
# 颜色定义
RED='\033[0;31m'
GREEN='\033[0;32m'
YELLOW='\033[1;33m'
BLUE='\033[0;34m'
NC='\033[0m' # No Color
# 获取当前版本
get_current_version() {
cat VERSION
}
# 获取Git信息
get_git_commit() {
git rev-parse --short HEAD 2>/dev/null || echo "unknown"
}
get_git_branch() {
git branch --show-current 2>/dev/null || echo "unknown"
}
# 获取构建时间
get_build_time() {
date '+%Y-%m-%d %H:%M:%S'
}
# 编译函数
build() {
local target=${1:-"main"}
local version=$(get_current_version)
local git_commit=$(get_git_commit)
local git_branch=$(get_git_branch)
local build_time=$(get_build_time)
echo -e "${BLUE}开始编译...${NC}"
echo -e "版本: ${GREEN}${version}${NC}"
echo -e "Git提交: ${GREEN}${git_commit}${NC}"
echo -e "Git分支: ${GREEN}${git_branch}${NC}"
echo -e "构建时间: ${GREEN}${build_time}${NC}"
# 构建 ldflags
local ldflags="-X 'github.com/ctwj/urldb/utils.Version=${version}'"
ldflags="${ldflags} -X 'github.com/ctwj/urldb/utils.BuildTime=${build_time}'"
ldflags="${ldflags} -X 'github.com/ctwj/urldb/utils.GitCommit=${git_commit}'"
ldflags="${ldflags} -X 'github.com/ctwj/urldb/utils.GitBranch=${git_branch}'"
# 编译 - 使用跨平台编译设置
echo -e "${YELLOW}编译中...${NC}"
CGO_ENABLED=0 GOOS=linux go build -a -installsuffix cgo -ldflags "${ldflags}" -o "${target}" .
if [ $? -eq 0 ]; then
echo -e "${GREEN}编译成功!${NC}"
echo -e "可执行文件: ${GREEN}${target}${NC}"
echo -e "目标平台: ${GREEN}Linux${NC}"
# 显示版本信息在Linux环境下
echo -e "${BLUE}版本信息验证:${NC}"
if [[ "$OSTYPE" == "linux-gnu"* ]]; then
./${target} version 2>/dev/null || echo "无法验证版本信息"
else
echo "当前非Linux环境无法直接验证版本信息"
echo "请将编译后的文件复制到Linux服务器上验证"
fi
else
echo -e "${RED}编译失败!${NC}"
exit 1
fi
}
# 清理函数
clean() {
echo -e "${YELLOW}清理编译文件...${NC}"
rm -f main
echo -e "${GREEN}清理完成${NC}"
}
# 显示帮助
show_help() {
echo -e "${BLUE}编译脚本${NC}"
echo ""
echo "用法: $0 [命令]"
echo ""
echo "命令:"
echo " build [target] 编译程序 (当前平台)"
echo " build-linux [target] 编译Linux版本 (推荐)"
echo " clean 清理编译文件"
echo " help 显示此帮助信息"
echo ""
echo "示例:"
echo " $0 # 编译Linux版本 (默认)"
echo " $0 build-linux # 编译Linux版本"
echo " $0 build-linux app # 编译Linux版本为 app"
echo " $0 build # 编译当前平台版本"
echo " $0 clean # 清理编译文件"
echo ""
echo "注意:"
echo " - Linux版本使用静态链接适合部署到服务器"
echo " - 默认编译Linux版本无需复制VERSION文件"
}
# Linux编译函数
build_linux() {
local target=${1:-"main"}
local version=$(get_current_version)
local git_commit=$(get_git_commit)
local git_branch=$(get_git_branch)
local build_time=$(get_build_time)
echo -e "${BLUE}开始Linux编译...${NC}"
echo -e "版本: ${GREEN}${version}${NC}"
echo -e "Git提交: ${GREEN}${git_commit}${NC}"
echo -e "Git分支: ${GREEN}${git_branch}${NC}"
echo -e "构建时间: ${GREEN}${build_time}${NC}"
# 构建 ldflags
local ldflags="-X 'github.com/ctwj/urldb/utils.Version=${version}'"
ldflags="${ldflags} -X 'github.com/ctwj/urldb/utils.BuildTime=${build_time}'"
ldflags="${ldflags} -X 'github.com/ctwj/urldb/utils.GitCommit=${git_commit}'"
ldflags="${ldflags} -X 'github.com/ctwj/urldb/utils.GitBranch=${git_branch}'"
# Linux编译
echo -e "${YELLOW}编译中...${NC}"
CGO_ENABLED=0 GOOS=linux go build -a -installsuffix cgo -ldflags "${ldflags}" -o "${target}" .
if [ $? -eq 0 ]; then
echo -e "${GREEN}Linux编译成功!${NC}"
echo -e "可执行文件: ${GREEN}${target}${NC}"
echo -e "目标平台: ${GREEN}Linux${NC}"
echo -e "静态链接: ${GREEN}${NC}"
# 显示文件信息
if command -v file >/dev/null 2>&1; then
echo -e "${BLUE}文件信息:${NC}"
file "${target}"
fi
echo -e "${BLUE}注意: 请在Linux服务器上验证版本信息${NC}"
else
echo -e "${RED}Linux编译失败!${NC}"
exit 1
fi
}
# 主函数
main() {
case $1 in
"build")
build $2
;;
"build-linux")
build_linux $2
;;
"clean")
clean
;;
"help"|"-h"|"--help")
show_help
;;
"")
build_linux
;;
*)
echo -e "${RED}错误: 未知命令 '$1'${NC}"
echo "使用 '$0 help' 查看帮助信息"
exit 1
;;
esac
}
# 运行主函数
main "$@"

155
scripts/docker-build.sh Normal file
View File

@@ -0,0 +1,155 @@
#!/bin/bash
# Docker构建脚本
# 用法: ./scripts/docker-build.sh [version]
set -e
# 颜色定义
RED='\033[0;31m'
GREEN='\033[0;32m'
YELLOW='\033[1;33m'
BLUE='\033[0;34m'
NC='\033[0m' # No Color
# 获取版本号
get_version() {
if [ -n "$1" ]; then
echo "$1"
else
cat VERSION
fi
}
# 获取Git信息
get_git_commit() {
git rev-parse --short HEAD 2>/dev/null || echo "unknown"
}
get_git_branch() {
git branch --show-current 2>/dev/null || echo "unknown"
}
# 构建Docker镜像
build_docker() {
local version=$(get_version $1)
local git_commit=$(get_git_commit)
local git_branch=$(get_git_branch)
local build_time=$(date '+%Y-%m-%d %H:%M:%S')
echo -e "${BLUE}开始Docker构建...${NC}"
echo -e "版本: ${GREEN}${version}${NC}"
echo -e "Git提交: ${GREEN}${git_commit}${NC}"
echo -e "Git分支: ${GREEN}${git_branch}${NC}"
echo -e "构建时间: ${GREEN}${build_time}${NC}"
# 直接使用 docker build避免 buildx 的复杂性
BUILD_CMD="docker build"
echo -e "${BLUE}使用构建命令: ${BUILD_CMD}${NC}"
# 构建前端镜像
echo -e "${YELLOW}构建前端镜像...${NC}"
FRONTEND_CMD="${BUILD_CMD} --build-arg VERSION=${version} --build-arg GIT_COMMIT=${git_commit} --build-arg GIT_BRANCH=${git_branch} --build-arg BUILD_TIME=${build_time} --target frontend -t ctwj/urldb-frontend:${version} ."
echo -e "${BLUE}执行命令: ${FRONTEND_CMD}${NC}"
${BUILD_CMD} \
--build-arg VERSION=${version} \
--build-arg GIT_COMMIT=${git_commit} \
--build-arg GIT_BRANCH=${git_branch} \
--build-arg "BUILD_TIME=${build_time}" \
--target frontend \
-t ctwj/urldb-frontend:${version} \
.
# 构建后端镜像
echo -e "${YELLOW}构建后端镜像...${NC}"
BACKEND_CMD="${BUILD_CMD} --build-arg VERSION=${version} --build-arg GIT_COMMIT=${git_commit} --build-arg GIT_BRANCH=${git_branch} --build-arg BUILD_TIME=${build_time} --target backend -t ctwj/urldb-backend:${version} ."
echo -e "${BLUE}执行命令: ${BACKEND_CMD}${NC}"
${BUILD_CMD} \
--build-arg VERSION=${version} \
--build-arg GIT_COMMIT=${git_commit} \
--build-arg GIT_BRANCH=${git_branch} \
--build-arg BUILD_TIME="${build_time}" \
--target backend \
-t ctwj/urldb-backend:${version} \
.
echo -e "${GREEN}Docker构建完成!${NC}"
echo -e "镜像标签:"
echo -e " ${GREEN}ctwj/urldb-backend:${version}${NC}"
echo -e " ${GREEN}ctwj/urldb-frontend:${version}${NC}"
}
# 推送镜像
push_images() {
local version=$(get_version $1)
echo -e "${YELLOW}推送镜像到Docker Hub...${NC}"
# 推送后端镜像
docker push ctwj/urldb-backend:${version}
# 推送前端镜像
docker push ctwj/urldb-frontend:${version}
echo -e "${GREEN}镜像推送完成!${NC}"
}
# 清理镜像
clean_images() {
local version=$(get_version $1)
echo -e "${YELLOW}清理Docker镜像...${NC}"
docker rmi ctwj/urldb-backend:${version} 2>/dev/null || true
docker rmi ctwj/urldb-frontend:${version} 2>/dev/null || true
echo -e "${GREEN}镜像清理完成${NC}"
}
# 显示帮助
show_help() {
echo -e "${BLUE}Docker构建脚本${NC}"
echo ""
echo "用法: $0 [命令] [版本]"
echo ""
echo "命令:"
echo " build [version] 构建Docker镜像"
echo " push [version] 推送镜像到Docker Hub"
echo " clean [version] 清理Docker镜像"
echo " help 显示此帮助信息"
echo ""
echo "示例:"
echo " $0 build # 构建当前版本镜像"
echo " $0 build 1.2.4 # 构建指定版本镜像"
echo " $0 push 1.2.4 # 推送指定版本镜像"
echo " $0 clean # 清理当前版本镜像"
}
# 主函数
main() {
case $1 in
"build")
build_docker $2
;;
"push")
push_images $2
;;
"clean")
clean_images $2
;;
"help"|"-h"|"--help")
show_help
;;
"")
show_help
;;
*)
echo -e "${RED}错误: 未知命令 '$1'${NC}"
echo "使用 '$0 help' 查看帮助信息"
exit 1
;;
esac
}
# 运行主函数
main "$@"

View File

@@ -0,0 +1,762 @@
package services
import (
"fmt"
"strconv"
"sync"
"time"
"github.com/ctwj/urldb/db/entity"
"github.com/ctwj/urldb/db/repo"
"github.com/ctwj/urldb/utils"
)
// MeilisearchManager Meilisearch管理器
type MeilisearchManager struct {
service *MeilisearchService
repoMgr *repo.RepositoryManager
configRepo repo.SystemConfigRepository
mutex sync.RWMutex
status MeilisearchStatus
stopChan chan struct{}
isRunning bool
// 同步进度控制
syncMutex sync.RWMutex
syncProgress SyncProgress
isSyncing bool
syncStopChan chan struct{}
}
// SyncProgress 同步进度
type SyncProgress struct {
IsRunning bool `json:"is_running"`
TotalCount int64 `json:"total_count"`
ProcessedCount int64 `json:"processed_count"`
SyncedCount int64 `json:"synced_count"`
FailedCount int64 `json:"failed_count"`
StartTime time.Time `json:"start_time"`
EstimatedTime string `json:"estimated_time"`
CurrentBatch int `json:"current_batch"`
TotalBatches int `json:"total_batches"`
ErrorMessage string `json:"error_message"`
}
// MeilisearchStatus Meilisearch状态
type MeilisearchStatus struct {
Enabled bool `json:"enabled"`
Healthy bool `json:"healthy"`
LastCheck time.Time `json:"last_check"`
ErrorCount int `json:"error_count"`
LastError string `json:"last_error"`
DocumentCount int64 `json:"document_count"`
}
// NewMeilisearchManager 创建Meilisearch管理器
func NewMeilisearchManager(repoMgr *repo.RepositoryManager) *MeilisearchManager {
return &MeilisearchManager{
repoMgr: repoMgr,
stopChan: make(chan struct{}),
syncStopChan: make(chan struct{}),
status: MeilisearchStatus{
Enabled: false,
Healthy: false,
LastCheck: time.Now(),
},
}
}
// Initialize 初始化Meilisearch服务
func (m *MeilisearchManager) Initialize() error {
m.mutex.Lock()
defer m.mutex.Unlock()
// 设置configRepo
m.configRepo = m.repoMgr.SystemConfigRepository
// 获取配置
enabled, err := m.configRepo.GetConfigBool(entity.ConfigKeyMeilisearchEnabled)
if err != nil {
utils.Error("获取Meilisearch启用状态失败: %v", err)
return err
}
if !enabled {
utils.Debug("Meilisearch未启用清理服务状态")
m.status.Enabled = false
m.service = nil
// 停止监控循环
if m.stopChan != nil {
close(m.stopChan)
m.stopChan = make(chan struct{})
}
return nil
}
host, err := m.configRepo.GetConfigValue(entity.ConfigKeyMeilisearchHost)
if err != nil {
utils.Error("获取Meilisearch主机配置失败: %v", err)
return err
}
port, err := m.configRepo.GetConfigValue(entity.ConfigKeyMeilisearchPort)
if err != nil {
utils.Error("获取Meilisearch端口配置失败: %v", err)
return err
}
masterKey, err := m.configRepo.GetConfigValue(entity.ConfigKeyMeilisearchMasterKey)
if err != nil {
utils.Error("获取Meilisearch主密钥配置失败: %v", err)
return err
}
indexName, err := m.configRepo.GetConfigValue(entity.ConfigKeyMeilisearchIndexName)
if err != nil {
utils.Error("获取Meilisearch索引名配置失败: %v", err)
return err
}
m.service = NewMeilisearchService(host, port, masterKey, indexName, enabled)
m.status.Enabled = enabled
// 如果启用,创建索引并更新设置
if enabled {
utils.Debug("Meilisearch已启用创建索引并更新设置")
// 创建索引
if err := m.service.CreateIndex(); err != nil {
utils.Error("创建Meilisearch索引失败: %v", err)
}
// 更新索引设置
if err := m.service.UpdateIndexSettings(); err != nil {
utils.Error("更新Meilisearch索引设置失败: %v", err)
}
// 立即进行一次健康检查
go func() {
m.checkHealth()
// 启动监控
go m.monitorLoop()
}()
} else {
utils.Debug("Meilisearch未启用")
}
utils.Debug("Meilisearch服务初始化完成")
return nil
}
// IsEnabled 检查是否启用
func (m *MeilisearchManager) IsEnabled() bool {
m.mutex.RLock()
defer m.mutex.RUnlock()
return m.status.Enabled
}
// ReloadConfig 重新加载配置
func (m *MeilisearchManager) ReloadConfig() error {
utils.Debug("重新加载Meilisearch配置")
return m.Initialize()
}
// GetService 获取Meilisearch服务
func (m *MeilisearchManager) GetService() *MeilisearchService {
m.mutex.RLock()
defer m.mutex.RUnlock()
return m.service
}
// GetStatus 获取状态
func (m *MeilisearchManager) GetStatus() (MeilisearchStatus, error) {
m.mutex.RLock()
defer m.mutex.RUnlock()
utils.Debug("获取Meilisearch状态 - 启用状态: %v, 健康状态: %v, 服务实例: %v", m.status.Enabled, m.status.Healthy, m.service != nil)
if m.service != nil && m.service.IsEnabled() {
utils.Debug("Meilisearch服务已初始化且启用尝试获取索引统计")
// 获取索引统计
stats, err := m.service.GetIndexStats()
if err != nil {
utils.Error("获取Meilisearch索引统计失败: %v", err)
// 即使获取统计失败,也返回当前状态
} else {
utils.Debug("Meilisearch索引统计: %+v", stats)
// 更新文档数量
if count, ok := stats["numberOfDocuments"].(float64); ok {
m.status.DocumentCount = int64(count)
utils.Debug("文档数量 (float64): %d", int64(count))
} else if count, ok := stats["numberOfDocuments"].(int64); ok {
m.status.DocumentCount = count
utils.Debug("文档数量 (int64): %d", count)
} else if count, ok := stats["numberOfDocuments"].(int); ok {
m.status.DocumentCount = int64(count)
utils.Debug("文档数量 (int): %d", int64(count))
} else {
utils.Error("无法解析文档数量,类型: %T, 值: %v", stats["numberOfDocuments"], stats["numberOfDocuments"])
}
// 不更新启用状态,保持配置中的状态
// 启用状态应该由配置控制,而不是由服务状态控制
}
} else {
utils.Debug("Meilisearch服务未初始化或未启用 - service: %v, enabled: %v", m.service != nil, m.service != nil && m.service.IsEnabled())
}
return m.status, nil
}
// GetStatusWithHealthCheck 获取状态并同时进行健康检查
func (m *MeilisearchManager) GetStatusWithHealthCheck() (MeilisearchStatus, error) {
// 先进行健康检查
m.checkHealth()
// 然后获取状态
return m.GetStatus()
}
// SyncResourceToMeilisearch 同步资源到Meilisearch
func (m *MeilisearchManager) SyncResourceToMeilisearch(resource *entity.Resource) error {
if m.service == nil || !m.service.IsEnabled() {
return nil
}
doc := m.convertResourceToDocument(resource)
err := m.service.BatchAddDocuments([]MeilisearchDocument{doc})
if err != nil {
return err
}
// 标记为已同步
return m.repoMgr.ResourceRepository.MarkAsSyncedToMeilisearch([]uint{resource.ID})
}
// SyncAllResources 同步所有资源
func (m *MeilisearchManager) SyncAllResources() (int, error) {
if m.service == nil || !m.service.IsEnabled() {
return 0, fmt.Errorf("Meilisearch未启用")
}
// 检查是否已经在同步中
m.syncMutex.Lock()
if m.isSyncing {
m.syncMutex.Unlock()
return 0, fmt.Errorf("同步操作正在进行中")
}
// 初始化同步状态
m.isSyncing = true
m.syncProgress = SyncProgress{
IsRunning: true,
TotalCount: 0,
ProcessedCount: 0,
SyncedCount: 0,
FailedCount: 0,
StartTime: time.Now(),
CurrentBatch: 0,
TotalBatches: 0,
ErrorMessage: "",
}
// 重新创建停止通道
m.syncStopChan = make(chan struct{})
m.syncMutex.Unlock()
// 在goroutine中执行同步避免阻塞
go func() {
defer func() {
m.syncMutex.Lock()
m.isSyncing = false
m.syncProgress.IsRunning = false
m.syncMutex.Unlock()
}()
m.syncAllResourcesInternal()
}()
return 0, nil
}
// DebugGetAllDocuments 调试:获取所有文档
func (m *MeilisearchManager) DebugGetAllDocuments() error {
if m.service == nil || !m.service.IsEnabled() {
return fmt.Errorf("Meilisearch未启用")
}
utils.Debug("开始调试获取Meilisearch中的所有文档")
_, err := m.service.GetAllDocuments()
if err != nil {
utils.Error("调试获取所有文档失败: %v", err)
return err
}
utils.Debug("调试完成:已获取所有文档")
return nil
}
// syncAllResourcesInternal 内部同步方法
func (m *MeilisearchManager) syncAllResourcesInternal() {
// 健康检查
if err := m.service.HealthCheck(); err != nil {
m.updateSyncProgress("", "", fmt.Sprintf("Meilisearch不可用: %v", err))
return
}
// 创建索引
if err := m.service.CreateIndex(); err != nil {
m.updateSyncProgress("", "", fmt.Sprintf("创建索引失败: %v", err))
return
}
utils.Debug("开始同步所有资源到Meilisearch...")
// 获取总资源数量
totalCount, err := m.repoMgr.ResourceRepository.CountUnsyncedToMeilisearch()
if err != nil {
m.updateSyncProgress("", "", fmt.Sprintf("获取资源总数失败: %v", err))
return
}
// 分批处理
batchSize := 100
totalBatches := int((totalCount + int64(batchSize) - 1) / int64(batchSize))
// 更新总数量和总批次
m.syncMutex.Lock()
m.syncProgress.TotalCount = totalCount
m.syncProgress.TotalBatches = totalBatches
m.syncMutex.Unlock()
offset := 0
totalSynced := 0
currentBatch := 0
// 预加载所有分类和平台数据到缓存
categoryCache := make(map[uint]string)
panCache := make(map[uint]string)
// 获取所有分类
categories, err := m.repoMgr.CategoryRepository.FindAll()
if err == nil {
for _, category := range categories {
categoryCache[category.ID] = category.Name
}
}
// 获取所有平台
pans, err := m.repoMgr.PanRepository.FindAll()
if err == nil {
for _, pan := range pans {
panCache[pan.ID] = pan.Name
}
}
for {
// 检查是否需要停止
select {
case <-m.syncStopChan:
utils.Debug("同步操作被停止")
return
default:
}
currentBatch++
// 获取一批资源在goroutine中执行避免阻塞
resourcesChan := make(chan []entity.Resource, 1)
errChan := make(chan error, 1)
go func() {
// 直接查询未同步的资源,不使用分页
resources, _, err := m.repoMgr.ResourceRepository.FindUnsyncedToMeilisearch(1, batchSize)
if err != nil {
errChan <- err
return
}
resourcesChan <- resources
}()
// 等待数据库查询结果或停止信号(添加超时)
select {
case resources := <-resourcesChan:
if len(resources) == 0 {
utils.Info("资源同步完成,总共同步 %d 个资源", totalSynced)
return
}
// 检查是否需要停止
select {
case <-m.syncStopChan:
utils.Debug("同步操作被停止")
return
default:
}
// 转换为Meilisearch文档使用缓存
var docs []MeilisearchDocument
for _, resource := range resources {
doc := m.convertResourceToDocumentWithCache(&resource, categoryCache, panCache)
docs = append(docs, doc)
}
// 检查是否需要停止
select {
case <-m.syncStopChan:
utils.Debug("同步操作被停止")
return
default:
}
// 批量添加到Meilisearch在goroutine中执行避免阻塞
meilisearchErrChan := make(chan error, 1)
go func() {
err := m.service.BatchAddDocuments(docs)
meilisearchErrChan <- err
}()
// 等待Meilisearch操作结果或停止信号添加超时
select {
case err := <-meilisearchErrChan:
if err != nil {
m.updateSyncProgress("", "", fmt.Sprintf("批量添加文档失败: %v", err))
return
}
case <-time.After(60 * time.Second): // 60秒超时
m.updateSyncProgress("", "", "Meilisearch操作超时")
utils.Error("Meilisearch操作超时")
return
case <-m.syncStopChan:
utils.Debug("同步操作被停止")
return
}
// 检查是否需要停止
select {
case <-m.syncStopChan:
utils.Debug("同步操作被停止")
return
default:
}
// 标记为已同步在goroutine中执行避免阻塞
var resourceIDs []uint
for _, resource := range resources {
resourceIDs = append(resourceIDs, resource.ID)
}
markErrChan := make(chan error, 1)
go func() {
err := m.repoMgr.ResourceRepository.MarkAsSyncedToMeilisearch(resourceIDs)
markErrChan <- err
}()
// 等待标记操作结果或停止信号(添加超时)
select {
case err := <-markErrChan:
if err != nil {
utils.Error("标记资源同步状态失败: %v", err)
}
case <-time.After(30 * time.Second): // 30秒超时
utils.Error("标记资源同步状态超时")
case <-m.syncStopChan:
utils.Debug("同步操作被停止")
return
}
totalSynced += len(docs)
offset += len(resources)
// 更新进度
m.updateSyncProgress(fmt.Sprintf("%d", totalSynced), fmt.Sprintf("%d", currentBatch), "")
utils.Debug("已同步 %d 个资源到Meilisearch (批次 %d/%d)", totalSynced, currentBatch, totalBatches)
// 检查是否已经同步完所有资源
if len(resources) == 0 {
utils.Info("资源同步完成,总共同步 %d 个资源", totalSynced)
return
}
case <-time.After(30 * time.Second): // 30秒超时
m.updateSyncProgress("", "", "数据库查询超时")
utils.Error("数据库查询超时")
return
case err := <-errChan:
m.updateSyncProgress("", "", fmt.Sprintf("获取资源失败: %v", err))
return
case <-m.syncStopChan:
utils.Info("同步操作被停止")
return
}
// 避免过于频繁的请求
time.Sleep(100 * time.Millisecond)
}
utils.Info("资源同步完成,总共同步 %d 个资源", totalSynced)
}
// updateSyncProgress 更新同步进度
func (m *MeilisearchManager) updateSyncProgress(syncedCount, currentBatch, errorMessage string) {
m.syncMutex.Lock()
defer m.syncMutex.Unlock()
if syncedCount != "" {
if count, err := strconv.ParseInt(syncedCount, 10, 64); err == nil {
m.syncProgress.SyncedCount = count
}
}
if currentBatch != "" {
if batch, err := strconv.Atoi(currentBatch); err == nil {
m.syncProgress.CurrentBatch = batch
}
}
if errorMessage != "" {
m.syncProgress.ErrorMessage = errorMessage
m.syncProgress.IsRunning = false
}
// 计算预估时间
if m.syncProgress.SyncedCount > 0 {
elapsed := time.Since(m.syncProgress.StartTime)
rate := float64(m.syncProgress.SyncedCount) / elapsed.Seconds()
if rate > 0 {
remaining := float64(m.syncProgress.TotalCount-m.syncProgress.SyncedCount) / rate
m.syncProgress.EstimatedTime = fmt.Sprintf("%.0f秒", remaining)
}
}
}
// GetUnsyncedCount 获取未同步资源数量
func (m *MeilisearchManager) GetUnsyncedCount() (int64, error) {
// 直接查询未同步的资源数量
return m.repoMgr.ResourceRepository.CountUnsyncedToMeilisearch()
}
// GetUnsyncedResources 获取未同步的资源
func (m *MeilisearchManager) GetUnsyncedResources(page, pageSize int) ([]entity.Resource, int64, error) {
// 查询未同步到Meilisearch的资源
return m.repoMgr.ResourceRepository.FindUnsyncedToMeilisearch(page, pageSize)
}
// GetSyncedResources 获取已同步的资源
func (m *MeilisearchManager) GetSyncedResources(page, pageSize int) ([]entity.Resource, int64, error) {
// 查询已同步到Meilisearch的资源
return m.repoMgr.ResourceRepository.FindSyncedToMeilisearch(page, pageSize)
}
// GetAllResources 获取所有资源
func (m *MeilisearchManager) GetAllResources(page, pageSize int) ([]entity.Resource, int64, error) {
// 查询所有资源
return m.repoMgr.ResourceRepository.FindAllWithPagination(page, pageSize)
}
// GetSyncProgress 获取同步进度
func (m *MeilisearchManager) GetSyncProgress() SyncProgress {
m.syncMutex.RLock()
defer m.syncMutex.RUnlock()
return m.syncProgress
}
// StopSync 停止同步
func (m *MeilisearchManager) StopSync() {
m.syncMutex.Lock()
defer m.syncMutex.Unlock()
if m.isSyncing {
// 发送停止信号
select {
case <-m.syncStopChan:
// 通道已经关闭,不需要再次关闭
default:
close(m.syncStopChan)
}
m.isSyncing = false
m.syncProgress.IsRunning = false
m.syncProgress.ErrorMessage = "同步已停止"
utils.Debug("同步操作已停止")
}
}
// ClearIndex 清空索引
func (m *MeilisearchManager) ClearIndex() error {
if m.service == nil || !m.service.IsEnabled() {
return fmt.Errorf("Meilisearch未启用")
}
// 清空Meilisearch索引
if err := m.service.ClearIndex(); err != nil {
return err
}
// 标记所有资源为未同步
return m.repoMgr.ResourceRepository.MarkAllAsUnsyncedToMeilisearch()
}
// convertResourceToDocument 转换资源为搜索文档
func (m *MeilisearchManager) convertResourceToDocument(resource *entity.Resource) MeilisearchDocument {
// 获取关联数据
var categoryName string
if resource.CategoryID != nil {
category, err := m.repoMgr.CategoryRepository.FindByID(*resource.CategoryID)
if err == nil {
categoryName = category.Name
}
}
var panName string
if resource.PanID != nil {
pan, err := m.repoMgr.PanRepository.FindByID(*resource.PanID)
if err == nil {
panName = pan.Name
}
}
// 获取标签 - 从关联的Tags字段获取
var tagNames []string
if resource.Tags != nil {
for _, tag := range resource.Tags {
tagNames = append(tagNames, tag.Name)
}
}
return MeilisearchDocument{
ID: resource.ID,
Title: resource.Title,
Description: resource.Description,
URL: resource.URL,
SaveURL: resource.SaveURL,
FileSize: resource.FileSize,
Key: resource.Key,
Category: categoryName,
Tags: tagNames,
PanName: panName,
PanID: resource.PanID,
Author: resource.Author,
CreatedAt: resource.CreatedAt,
UpdatedAt: resource.UpdatedAt,
}
}
// convertResourceToDocumentWithCache 转换资源为搜索文档(使用缓存)
func (m *MeilisearchManager) convertResourceToDocumentWithCache(resource *entity.Resource, categoryCache map[uint]string, panCache map[uint]string) MeilisearchDocument {
// 从缓存获取关联数据
var categoryName string
if resource.CategoryID != nil {
if name, exists := categoryCache[*resource.CategoryID]; exists {
categoryName = name
}
}
var panName string
if resource.PanID != nil {
if name, exists := panCache[*resource.PanID]; exists {
panName = name
}
}
// 获取标签 - 从关联的Tags字段获取
var tagNames []string
if resource.Tags != nil {
for _, tag := range resource.Tags {
tagNames = append(tagNames, tag.Name)
}
}
return MeilisearchDocument{
ID: resource.ID,
Title: resource.Title,
Description: resource.Description,
URL: resource.URL,
SaveURL: resource.SaveURL,
FileSize: resource.FileSize,
Key: resource.Key,
Category: categoryName,
Tags: tagNames,
PanName: panName,
PanID: resource.PanID,
Author: resource.Author,
CreatedAt: resource.CreatedAt,
UpdatedAt: resource.UpdatedAt,
}
}
// monitorLoop 监控循环
func (m *MeilisearchManager) monitorLoop() {
if m.isRunning {
return
}
m.isRunning = true
ticker := time.NewTicker(30 * time.Second) // 每30秒检查一次
defer ticker.Stop()
for {
select {
case <-ticker.C:
m.checkHealth()
case <-m.stopChan:
return
}
}
}
// checkHealth 检查健康状态
func (m *MeilisearchManager) checkHealth() {
m.mutex.Lock()
defer m.mutex.Unlock()
m.status.LastCheck = time.Now()
utils.Debug("开始健康检查 - 服务实例: %v, 启用状态: %v", m.service != nil, m.service != nil && m.service.IsEnabled())
if m.service == nil || !m.service.IsEnabled() {
utils.Debug("Meilisearch服务未初始化或未启用")
m.status.Healthy = false
m.status.LastError = "Meilisearch未启用"
return
}
utils.Debug("开始检查Meilisearch健康状态")
if err := m.service.HealthCheck(); err != nil {
m.status.Healthy = false
m.status.ErrorCount++
m.status.LastError = err.Error()
utils.Error("Meilisearch健康检查失败: %v", err)
} else {
m.status.Healthy = true
m.status.ErrorCount = 0
m.status.LastError = ""
utils.Debug("Meilisearch健康检查成功")
// 健康检查通过后,更新文档数量
if stats, err := m.service.GetIndexStats(); err == nil {
if count, ok := stats["numberOfDocuments"].(float64); ok {
m.status.DocumentCount = int64(count)
} else if count, ok := stats["numberOfDocuments"].(int64); ok {
m.status.DocumentCount = count
} else if count, ok := stats["numberOfDocuments"].(int); ok {
m.status.DocumentCount = int64(count)
}
}
}
}
// Stop 停止监控
func (m *MeilisearchManager) Stop() {
if !m.isRunning {
return
}
close(m.stopChan)
m.isRunning = false
utils.Debug("Meilisearch监控服务已停止")
}

View File

@@ -0,0 +1,553 @@
package services
import (
"encoding/json"
"fmt"
"reflect"
"time"
"github.com/ctwj/urldb/utils"
"github.com/meilisearch/meilisearch-go"
)
// MeilisearchDocument 搜索文档结构
type MeilisearchDocument struct {
ID uint `json:"id"`
Title string `json:"title"`
Description string `json:"description"`
URL string `json:"url"`
SaveURL string `json:"save_url"`
FileSize string `json:"file_size"`
Key string `json:"key"`
Category string `json:"category"`
Tags []string `json:"tags"`
PanName string `json:"pan_name"`
PanID *uint `json:"pan_id"`
Author string `json:"author"`
CreatedAt time.Time `json:"created_at"`
UpdatedAt time.Time `json:"updated_at"`
// 高亮字段
TitleHighlight string `json:"_title_highlight,omitempty"`
DescriptionHighlight string `json:"_description_highlight,omitempty"`
CategoryHighlight string `json:"_category_highlight,omitempty"`
TagsHighlight []string `json:"_tags_highlight,omitempty"`
}
// MeilisearchService Meilisearch服务
type MeilisearchService struct {
client meilisearch.ServiceManager
index meilisearch.IndexManager
indexName string
enabled bool
}
// NewMeilisearchService 创建Meilisearch服务
func NewMeilisearchService(host, port, masterKey, indexName string, enabled bool) *MeilisearchService {
if !enabled {
return &MeilisearchService{
enabled: false,
}
}
// 构建服务器URL
serverURL := fmt.Sprintf("http://%s:%s", host, port)
// 创建客户端
var client meilisearch.ServiceManager
if masterKey != "" {
client = meilisearch.New(serverURL, meilisearch.WithAPIKey(masterKey))
} else {
client = meilisearch.New(serverURL)
}
// 获取索引
index := client.Index(indexName)
return &MeilisearchService{
client: client,
index: index,
indexName: indexName,
enabled: enabled,
}
}
// IsEnabled 检查是否启用
func (m *MeilisearchService) IsEnabled() bool {
return m.enabled
}
// HealthCheck 健康检查
func (m *MeilisearchService) HealthCheck() error {
if !m.enabled {
utils.Debug("Meilisearch未启用跳过健康检查")
return fmt.Errorf("Meilisearch未启用")
}
utils.Debug("开始Meilisearch健康检查")
// 使用官方SDK的健康检查
_, err := m.client.Health()
if err != nil {
utils.Error("Meilisearch健康检查失败: %v", err)
return fmt.Errorf("Meilisearch健康检查失败: %v", err)
}
utils.Debug("Meilisearch健康检查成功")
return nil
}
// CreateIndex 创建索引
func (m *MeilisearchService) CreateIndex() error {
if !m.enabled {
return nil
}
// 创建索引配置
indexConfig := &meilisearch.IndexConfig{
Uid: m.indexName,
PrimaryKey: "id",
}
// 创建索引
_, err := m.client.CreateIndex(indexConfig)
if err != nil {
// 如果索引已存在,返回成功
utils.Debug("Meilisearch索引创建失败或已存在: %v", err)
return nil
}
utils.Debug("Meilisearch索引创建成功: %s", m.indexName)
// 配置索引设置
settings := &meilisearch.Settings{
// 配置可过滤的属性
FilterableAttributes: []string{
"pan_id",
"pan_name",
"category",
"tags",
},
// 配置可搜索的属性
SearchableAttributes: []string{
"title",
"description",
"category",
"tags",
},
// 配置可排序的属性
SortableAttributes: []string{
"created_at",
"updated_at",
"id",
},
}
// 更新索引设置
_, err = m.index.UpdateSettings(settings)
if err != nil {
utils.Error("更新Meilisearch索引设置失败: %v", err)
return err
}
utils.Debug("Meilisearch索引设置更新成功")
return nil
}
// UpdateIndexSettings 更新索引设置
func (m *MeilisearchService) UpdateIndexSettings() error {
if !m.enabled {
return nil
}
// 配置索引设置
settings := &meilisearch.Settings{
// 配置可过滤的属性
FilterableAttributes: []string{
"pan_id",
"pan_name",
"category",
"tags",
},
// 配置可搜索的属性
SearchableAttributes: []string{
"title",
"description",
"category",
"tags",
},
// 配置可排序的属性
SortableAttributes: []string{
"created_at",
"updated_at",
"id",
},
}
// 更新索引设置
_, err := m.index.UpdateSettings(settings)
if err != nil {
utils.Error("更新Meilisearch索引设置失败: %v", err)
return err
}
utils.Debug("Meilisearch索引设置更新成功")
return nil
}
// BatchAddDocuments 批量添加文档
func (m *MeilisearchService) BatchAddDocuments(docs []MeilisearchDocument) error {
if !m.enabled {
return nil
}
if len(docs) == 0 {
return nil
}
// 转换为interface{}切片
var documents []interface{}
for _, doc := range docs {
documents = append(documents, doc)
}
// 批量添加文档
_, err := m.index.AddDocuments(documents, nil)
if err != nil {
return fmt.Errorf("批量添加文档失败: %v", err)
}
utils.Debug("批量添加 %d 个文档到Meilisearch成功", len(docs))
return nil
}
// Search 搜索文档
func (m *MeilisearchService) Search(query string, filters map[string]interface{}, page, pageSize int) ([]MeilisearchDocument, int64, error) {
if !m.enabled {
return nil, 0, fmt.Errorf("Meilisearch未启用")
}
// 构建搜索请求
searchRequest := &meilisearch.SearchRequest{
Query: query,
Offset: int64((page - 1) * pageSize),
Limit: int64(pageSize),
// 启用高亮功能
AttributesToHighlight: []string{"title", "description", "category", "tags"},
HighlightPreTag: "<mark>",
HighlightPostTag: "</mark>",
}
// 添加过滤器
if len(filters) > 0 {
var filterStrings []string
for key, value := range filters {
switch key {
case "pan_id":
// 直接使用pan_id进行过滤
filterStrings = append(filterStrings, fmt.Sprintf("pan_id = %v", value))
case "pan_name":
// 使用pan_name进行过滤
filterStrings = append(filterStrings, fmt.Sprintf("pan_name = %q", value))
case "category":
filterStrings = append(filterStrings, fmt.Sprintf("category = %q", value))
case "tags":
filterStrings = append(filterStrings, fmt.Sprintf("tags = %q", value))
default:
filterStrings = append(filterStrings, fmt.Sprintf("%s = %q", key, value))
}
}
if len(filterStrings) > 0 {
searchRequest.Filter = filterStrings
}
}
// 执行搜索
result, err := m.index.Search(query, searchRequest)
if err != nil {
return nil, 0, fmt.Errorf("搜索失败: %v", err)
}
// 解析结果
var documents []MeilisearchDocument
// 如果没有任何结果,直接返回
if len(result.Hits) == 0 {
utils.Debug("没有搜索结果")
return documents, result.EstimatedTotalHits, nil
}
for _, hit := range result.Hits {
// 将hit转换为MeilisearchDocument
doc := MeilisearchDocument{}
// 解析JSON数据 - 使用反射
hitValue := reflect.ValueOf(hit)
if hitValue.Kind() == reflect.Map {
for _, key := range hitValue.MapKeys() {
keyStr := key.String()
value := hitValue.MapIndex(key).Interface()
// 处理_formatted字段包含所有高亮内容
if keyStr == "_formatted" {
if rawValue, ok := value.(json.RawMessage); ok {
// 解析_formatted字段中的高亮内容
var formattedData map[string]interface{}
if err := json.Unmarshal(rawValue, &formattedData); err == nil {
// 提取高亮字段
if titleHighlight, ok := formattedData["title"].(string); ok {
doc.TitleHighlight = titleHighlight
}
if descHighlight, ok := formattedData["description"].(string); ok {
doc.DescriptionHighlight = descHighlight
}
if categoryHighlight, ok := formattedData["category"].(string); ok {
doc.CategoryHighlight = categoryHighlight
}
if tagsHighlight, ok := formattedData["tags"].([]interface{}); ok {
var tags []string
for _, tag := range tagsHighlight {
if tagStr, ok := tag.(string); ok {
tags = append(tags, tagStr)
}
}
doc.TagsHighlight = tags
}
}
}
}
switch keyStr {
case "id":
if rawID, ok := value.(json.RawMessage); ok {
var id float64
if err := json.Unmarshal(rawID, &id); err == nil {
doc.ID = uint(id)
}
}
case "title":
if rawTitle, ok := value.(json.RawMessage); ok {
var title string
if err := json.Unmarshal(rawTitle, &title); err == nil {
doc.Title = title
}
}
case "description":
if rawDesc, ok := value.(json.RawMessage); ok {
var description string
if err := json.Unmarshal(rawDesc, &description); err == nil {
doc.Description = description
}
}
case "url":
if rawURL, ok := value.(json.RawMessage); ok {
var url string
if err := json.Unmarshal(rawURL, &url); err == nil {
doc.URL = url
}
}
case "save_url":
if rawSaveURL, ok := value.(json.RawMessage); ok {
var saveURL string
if err := json.Unmarshal(rawSaveURL, &saveURL); err == nil {
doc.SaveURL = saveURL
}
}
case "file_size":
if rawFileSize, ok := value.(json.RawMessage); ok {
var fileSize string
if err := json.Unmarshal(rawFileSize, &fileSize); err == nil {
doc.FileSize = fileSize
}
}
case "key":
if rawKey, ok := value.(json.RawMessage); ok {
var key string
if err := json.Unmarshal(rawKey, &key); err == nil {
doc.Key = key
}
}
case "category":
if rawCategory, ok := value.(json.RawMessage); ok {
var category string
if err := json.Unmarshal(rawCategory, &category); err == nil {
doc.Category = category
}
}
case "tags":
if rawTags, ok := value.(json.RawMessage); ok {
var tags []string
if err := json.Unmarshal(rawTags, &tags); err == nil {
doc.Tags = tags
}
}
case "pan_name":
if rawPanName, ok := value.(json.RawMessage); ok {
var panName string
if err := json.Unmarshal(rawPanName, &panName); err == nil {
doc.PanName = panName
}
}
case "pan_id":
if rawPanID, ok := value.(json.RawMessage); ok {
var panID float64
if err := json.Unmarshal(rawPanID, &panID); err == nil {
panIDUint := uint(panID)
doc.PanID = &panIDUint
}
}
case "author":
if rawAuthor, ok := value.(json.RawMessage); ok {
var author string
if err := json.Unmarshal(rawAuthor, &author); err == nil {
doc.Author = author
}
}
case "created_at":
if rawCreatedAt, ok := value.(json.RawMessage); ok {
var createdAt string
if err := json.Unmarshal(rawCreatedAt, &createdAt); err == nil {
// 尝试多种时间格式
var t time.Time
var parseErr error
formats := []string{
time.RFC3339,
"2006-01-02T15:04:05Z",
"2006-01-02 15:04:05",
"2006-01-02T15:04:05.000Z",
}
for _, format := range formats {
if t, parseErr = time.Parse(format, createdAt); parseErr == nil {
doc.CreatedAt = t
break
}
}
}
}
case "updated_at":
if rawUpdatedAt, ok := value.(json.RawMessage); ok {
var updatedAt string
if err := json.Unmarshal(rawUpdatedAt, &updatedAt); err == nil {
// 尝试多种时间格式
var t time.Time
var parseErr error
formats := []string{
time.RFC3339,
"2006-01-02T15:04:05Z",
"2006-01-02 15:04:05",
"2006-01-02T15:04:05.000Z",
}
for _, format := range formats {
if t, parseErr = time.Parse(format, updatedAt); parseErr == nil {
doc.UpdatedAt = t
break
}
}
}
}
// 高亮字段处理 - 已移除现在使用_formatted字段
}
}
} else {
utils.Error("hit不是Map类型无法解析")
}
documents = append(documents, doc)
}
return documents, result.EstimatedTotalHits, nil
}
// GetAllDocuments 获取所有文档(用于调试)
func (m *MeilisearchService) GetAllDocuments() ([]MeilisearchDocument, error) {
if !m.enabled {
return nil, fmt.Errorf("Meilisearch未启用")
}
// 构建搜索请求,获取所有文档
searchRequest := &meilisearch.SearchRequest{
Query: "",
Offset: 0,
Limit: 1000, // 获取前1000个文档
}
// 执行搜索
result, err := m.index.Search("", searchRequest)
if err != nil {
return nil, fmt.Errorf("获取所有文档失败: %v", err)
}
utils.Debug("获取所有文档,总数: %d", result.EstimatedTotalHits)
utils.Debug("获取到的文档数量: %d", len(result.Hits))
// 解析结果
var documents []MeilisearchDocument
utils.Debug("获取到 %d 个文档", len(result.Hits))
// 只显示前3个文档的字段信息
for i, hit := range result.Hits {
if i >= 3 {
break
}
utils.Debug("文档%d的字段:", i+1)
hitValue := reflect.ValueOf(hit)
if hitValue.Kind() == reflect.Map {
for _, key := range hitValue.MapKeys() {
keyStr := key.String()
value := hitValue.MapIndex(key).Interface()
if rawValue, ok := value.(json.RawMessage); ok {
utils.Debug(" %s: %s", keyStr, string(rawValue))
} else {
utils.Debug(" %s: %v", keyStr, value)
}
}
}
}
return documents, nil
}
// GetIndexStats 获取索引统计信息
func (m *MeilisearchService) GetIndexStats() (map[string]interface{}, error) {
if !m.enabled {
return map[string]interface{}{
"enabled": false,
"message": "Meilisearch未启用",
}, nil
}
// 获取索引统计
stats, err := m.index.GetStats()
if err != nil {
return nil, fmt.Errorf("获取索引统计失败: %v", err)
}
utils.Debug("Meilisearch统计 - 文档数: %d, 索引中: %v", stats.NumberOfDocuments, stats.IsIndexing)
// 转换为map
result := map[string]interface{}{
"enabled": true,
"numberOfDocuments": stats.NumberOfDocuments,
"isIndexing": stats.IsIndexing,
"fieldDistribution": stats.FieldDistribution,
}
return result, nil
}
// ClearIndex 清空索引
func (m *MeilisearchService) ClearIndex() error {
if !m.enabled {
return fmt.Errorf("Meilisearch未启用")
}
// 清空索引
_, err := m.index.DeleteAllDocuments()
if err != nil {
return fmt.Errorf("清空索引失败: %v", err)
}
utils.Debug("Meilisearch索引已清空")
return nil
}

433
task/task_processor.go Normal file
View File

@@ -0,0 +1,433 @@
package task
import (
"context"
"encoding/json"
"fmt"
"sync"
"github.com/ctwj/urldb/db/entity"
"github.com/ctwj/urldb/db/repo"
"github.com/ctwj/urldb/utils"
)
// TaskProcessor 任务处理器接口
type TaskProcessor interface {
Process(ctx context.Context, taskID uint, item *entity.TaskItem) error
GetTaskType() string
}
// TaskManager 任务管理器
type TaskManager struct {
processors map[string]TaskProcessor
repoMgr *repo.RepositoryManager
mu sync.RWMutex
running map[uint]context.CancelFunc // 正在运行的任务
}
// NewTaskManager 创建任务管理器
func NewTaskManager(repoMgr *repo.RepositoryManager) *TaskManager {
return &TaskManager{
processors: make(map[string]TaskProcessor),
repoMgr: repoMgr,
running: make(map[uint]context.CancelFunc),
}
}
// RegisterProcessor 注册任务处理器
func (tm *TaskManager) RegisterProcessor(processor TaskProcessor) {
tm.mu.Lock()
defer tm.mu.Unlock()
tm.processors[processor.GetTaskType()] = processor
utils.Info("注册任务处理器: %s", processor.GetTaskType())
}
// getRegisteredProcessors 获取已注册的处理器列表(用于调试)
func (tm *TaskManager) getRegisteredProcessors() []string {
var types []string
for taskType := range tm.processors {
types = append(types, taskType)
}
return types
}
// StartTask 启动任务
func (tm *TaskManager) StartTask(taskID uint) error {
tm.mu.Lock()
defer tm.mu.Unlock()
utils.Info("StartTask: 尝试启动任务 %d", taskID)
// 检查任务是否已在运行
if _, exists := tm.running[taskID]; exists {
utils.Info("任务 %d 已在运行中", taskID)
return fmt.Errorf("任务 %d 已在运行中", taskID)
}
// 获取任务信息
task, err := tm.repoMgr.TaskRepository.GetByID(taskID)
if err != nil {
utils.Error("获取任务失败: %v", err)
return fmt.Errorf("获取任务失败: %v", err)
}
utils.Info("StartTask: 获取到任务 %d, 类型: %s, 状态: %s", task.ID, task.Type, task.Status)
// 获取处理器
processor, exists := tm.processors[string(task.Type)]
if !exists {
utils.Error("未找到任务类型 %s 的处理器, 已注册的处理器: %v", task.Type, tm.getRegisteredProcessors())
return fmt.Errorf("未找到任务类型 %s 的处理器", task.Type)
}
utils.Info("StartTask: 找到处理器 %s", task.Type)
// 创建上下文
ctx, cancel := context.WithCancel(context.Background())
tm.running[taskID] = cancel
utils.Info("StartTask: 启动后台任务协程")
// 启动后台任务
go tm.processTask(ctx, task, processor)
utils.Info("StartTask: 任务 %d 启动成功", taskID)
return nil
}
// PauseTask 暂停任务
func (tm *TaskManager) PauseTask(taskID uint) error {
tm.mu.Lock()
defer tm.mu.Unlock()
utils.Info("PauseTask: 尝试暂停任务 %d", taskID)
// 检查任务是否在运行
cancel, exists := tm.running[taskID]
if !exists {
// 检查数据库中任务状态
task, err := tm.repoMgr.TaskRepository.GetByID(taskID)
if err != nil {
utils.Error("获取任务信息失败: %v", err)
return fmt.Errorf("获取任务信息失败: %v", err)
}
// 如果数据库中的状态是running说明服务器重启了直接更新状态
if task.Status == "running" {
utils.Info("任务 %d 在数据库中状态为running但内存中不存在可能是服务器重启直接更新状态为paused", taskID)
err := tm.repoMgr.TaskRepository.UpdateStatus(taskID, "paused")
if err != nil {
utils.Error("更新任务状态为暂停失败: %v", err)
return fmt.Errorf("更新任务状态失败: %v", err)
}
utils.Info("任务 %d 暂停成功(服务器重启恢复)", taskID)
return nil
}
utils.Info("任务 %d 未在运行,无法暂停", taskID)
return fmt.Errorf("任务 %d 未在运行", taskID)
}
// 停止任务类似stop但状态标记为paused
cancel()
delete(tm.running, taskID)
// 更新任务状态为暂停
err := tm.repoMgr.TaskRepository.UpdateStatus(taskID, "paused")
if err != nil {
utils.Error("更新任务状态为暂停失败: %v", err)
return fmt.Errorf("更新任务状态失败: %v", err)
}
utils.Info("任务 %d 暂停成功", taskID)
return nil
}
// StopTask 停止任务
func (tm *TaskManager) StopTask(taskID uint) error {
tm.mu.Lock()
defer tm.mu.Unlock()
cancel, exists := tm.running[taskID]
if !exists {
// 检查数据库中任务状态
task, err := tm.repoMgr.TaskRepository.GetByID(taskID)
if err != nil {
utils.Error("获取任务信息失败: %v", err)
return fmt.Errorf("获取任务信息失败: %v", err)
}
// 如果数据库中的状态是running说明服务器重启了直接更新状态
if task.Status == "running" {
utils.Info("任务 %d 在数据库中状态为running但内存中不存在可能是服务器重启直接更新状态为paused", taskID)
err := tm.repoMgr.TaskRepository.UpdateStatus(taskID, "paused")
if err != nil {
utils.Error("更新任务状态失败: %v", err)
return fmt.Errorf("更新任务状态失败: %v", err)
}
utils.Info("任务 %d 停止成功(服务器重启恢复)", taskID)
return nil
}
return fmt.Errorf("任务 %d 未在运行", taskID)
}
cancel()
delete(tm.running, taskID)
// 更新任务状态为暂停
err := tm.repoMgr.TaskRepository.UpdateStatus(taskID, "paused")
if err != nil {
utils.Error("更新任务状态失败: %v", err)
}
return nil
}
// processTask 处理任务
func (tm *TaskManager) processTask(ctx context.Context, task *entity.Task, processor TaskProcessor) {
defer func() {
tm.mu.Lock()
delete(tm.running, task.ID)
tm.mu.Unlock()
utils.Info("processTask: 任务 %d 处理完成,清理资源", task.ID)
}()
utils.Info("processTask: 开始处理任务: %d, 类型: %s", task.ID, task.Type)
// 更新任务状态为运行中
err := tm.repoMgr.TaskRepository.UpdateStatus(task.ID, "running")
if err != nil {
utils.Error("更新任务状态失败: %v", err)
return
}
// 获取任务项统计信息,用于计算正确的进度
stats, err := tm.repoMgr.TaskItemRepository.GetStatsByTaskID(task.ID)
if err != nil {
utils.Error("获取任务项统计失败: %v", err)
stats = map[string]int{
"total": 0,
"pending": 0,
"processing": 0,
"completed": 0,
"failed": 0,
}
}
// 获取待处理的任务项
items, err := tm.repoMgr.TaskItemRepository.GetByTaskIDAndStatus(task.ID, "pending")
if err != nil {
utils.Error("获取任务项失败: %v", err)
tm.markTaskFailed(task.ID, fmt.Sprintf("获取任务项失败: %v", err))
return
}
// 计算总任务项数和已完成的项数
totalItems := stats["total"]
completedItems := stats["completed"]
initialFailedItems := stats["failed"]
processingItems := stats["processing"]
// 如果当前批次有处理中的任务项重置它们为pending状态服务器重启恢复
if processingItems > 0 {
utils.Info("任务 %d 发现 %d 个处理中的任务项重置为pending状态", task.ID, processingItems)
err = tm.repoMgr.TaskItemRepository.ResetProcessingItems(task.ID)
if err != nil {
utils.Error("重置处理中任务项失败: %v", err)
}
// 重新获取待处理的任务项
items, err = tm.repoMgr.TaskItemRepository.GetByTaskIDAndStatus(task.ID, "pending")
if err != nil {
utils.Error("重新获取任务项失败: %v", err)
tm.markTaskFailed(task.ID, fmt.Sprintf("重新获取任务项失败: %v", err))
return
}
}
currentBatchItems := len(items)
processedItems := completedItems + initialFailedItems // 已经处理的项目数
successItems := completedItems
failedItems := initialFailedItems
utils.Info("任务 %d 统计信息: 总计=%d, 已完成=%d, 已失败=%d, 待处理=%d",
task.ID, totalItems, completedItems, failedItems, currentBatchItems)
for _, item := range items {
select {
case <-ctx.Done():
utils.Info("任务 %d 被取消", task.ID)
return
default:
// 处理单个任务项
err := tm.processTaskItem(ctx, task.ID, item, processor)
processedItems++
if err != nil {
failedItems++
utils.Error("处理任务项 %d 失败: %v", item.ID, err)
} else {
successItems++
}
// 更新任务进度(基于总任务项数)
if totalItems > 0 {
progress := float64(processedItems) / float64(totalItems) * 100
tm.updateTaskProgress(task.ID, progress, processedItems, successItems, failedItems)
}
}
}
// 任务完成
status := "completed"
message := fmt.Sprintf("任务完成,共处理 %d 项,成功 %d 项,失败 %d 项", processedItems, successItems, failedItems)
if failedItems > 0 && successItems == 0 {
status = "failed"
message = fmt.Sprintf("任务失败,共处理 %d 项,全部失败", processedItems)
} else if failedItems > 0 {
status = "partial_success"
message = fmt.Sprintf("任务部分成功,共处理 %d 项,成功 %d 项,失败 %d 项", processedItems, successItems, failedItems)
}
err = tm.repoMgr.TaskRepository.UpdateStatusAndMessage(task.ID, status, message)
if err != nil {
utils.Error("更新任务状态失败: %v", err)
}
utils.Info("任务 %d 处理完成: %s", task.ID, message)
}
// processTaskItem 处理单个任务项
func (tm *TaskManager) processTaskItem(ctx context.Context, taskID uint, item *entity.TaskItem, processor TaskProcessor) error {
// 更新任务项状态为处理中
err := tm.repoMgr.TaskItemRepository.UpdateStatus(item.ID, "processing")
if err != nil {
return fmt.Errorf("更新任务项状态失败: %v", err)
}
// 处理任务项
err = processor.Process(ctx, taskID, item)
if err != nil {
// 处理失败
outputData := map[string]interface{}{
"error": err.Error(),
"time": utils.GetCurrentTime(),
}
outputJSON, _ := json.Marshal(outputData)
updateErr := tm.repoMgr.TaskItemRepository.UpdateStatusAndOutput(item.ID, "failed", string(outputJSON))
if updateErr != nil {
utils.Error("更新失败任务项状态失败: %v", updateErr)
}
return err
}
// 处理成功
outputData := map[string]interface{}{
"success": true,
"time": utils.GetCurrentTime(),
}
outputJSON, _ := json.Marshal(outputData)
err = tm.repoMgr.TaskItemRepository.UpdateStatusAndOutput(item.ID, "completed", string(outputJSON))
if err != nil {
utils.Error("更新成功任务项状态失败: %v", err)
}
return nil
}
// updateTaskProgress 更新任务进度
func (tm *TaskManager) updateTaskProgress(taskID uint, progress float64, processed, success, failed int) {
// 更新任务统计信息
err := tm.repoMgr.TaskRepository.UpdateTaskStats(taskID, processed, success, failed)
if err != nil {
utils.Error("更新任务统计信息失败: %v", err)
}
// 更新进度数据(用于兼容性)
progressData := map[string]interface{}{
"progress": progress,
"processed": processed,
"success": success,
"failed": failed,
"time": utils.GetCurrentTime(),
}
progressJSON, _ := json.Marshal(progressData)
err = tm.repoMgr.TaskRepository.UpdateProgress(taskID, progress, string(progressJSON))
if err != nil {
utils.Error("更新任务进度数据失败: %v", err)
}
}
// markTaskFailed 标记任务失败
func (tm *TaskManager) markTaskFailed(taskID uint, message string) {
err := tm.repoMgr.TaskRepository.UpdateStatusAndMessage(taskID, "failed", message)
if err != nil {
utils.Error("标记任务失败状态失败: %v", err)
}
}
// GetTaskStatus 获取任务状态
func (tm *TaskManager) GetTaskStatus(taskID uint) (string, error) {
task, err := tm.repoMgr.TaskRepository.GetByID(taskID)
if err != nil {
return "", err
}
return string(task.Status), nil
}
// IsTaskRunning 检查任务是否在运行
func (tm *TaskManager) IsTaskRunning(taskID uint) bool {
tm.mu.RLock()
defer tm.mu.RUnlock()
_, exists := tm.running[taskID]
return exists
}
// RecoverRunningTasks 恢复运行中的任务(服务器重启后调用)
func (tm *TaskManager) RecoverRunningTasks() error {
tm.mu.Lock()
defer tm.mu.Unlock()
utils.Info("开始恢复运行中的任务...")
// 获取数据库中状态为running的任务
tasks, _, err := tm.repoMgr.TaskRepository.GetList(1, 1000, "", "running")
if err != nil {
utils.Error("获取运行中任务失败: %v", err)
return fmt.Errorf("获取运行中任务失败: %v", err)
}
recoveredCount := 0
for _, task := range tasks {
// 检查任务是否已在内存中运行
if _, exists := tm.running[task.ID]; exists {
utils.Info("任务 %d 已在内存中运行,跳过恢复", task.ID)
continue
}
// 获取处理器
processor, exists := tm.processors[string(task.Type)]
if !exists {
utils.Error("未找到任务类型 %s 的处理器,跳过恢复任务 %d", task.Type, task.ID)
// 将任务状态重置为pending避免卡在running状态
tm.repoMgr.TaskRepository.UpdateStatus(task.ID, "pending")
continue
}
// 创建上下文并恢复任务
ctx, cancel := context.WithCancel(context.Background())
tm.running[task.ID] = cancel
utils.Info("恢复任务 %d (类型: %s)", task.ID, task.Type)
go tm.processTask(ctx, task, processor)
recoveredCount++
}
utils.Info("任务恢复完成,共恢复 %d 个任务", recoveredCount)
return nil
}

513
task/transfer_processor.go Normal file
View File

@@ -0,0 +1,513 @@
package task
import (
"context"
"encoding/json"
"fmt"
"regexp"
"strings"
"time"
pan "github.com/ctwj/urldb/common"
"github.com/ctwj/urldb/db/entity"
"github.com/ctwj/urldb/db/repo"
"github.com/ctwj/urldb/utils"
)
// TransferProcessor 转存任务处理器
type TransferProcessor struct {
repoMgr *repo.RepositoryManager
}
// NewTransferProcessor 创建转存任务处理器
func NewTransferProcessor(repoMgr *repo.RepositoryManager) *TransferProcessor {
return &TransferProcessor{
repoMgr: repoMgr,
}
}
// GetTaskType 获取任务类型
func (tp *TransferProcessor) GetTaskType() string {
return "transfer"
}
// TransferInput 转存任务输入数据结构
type TransferInput struct {
Title string `json:"title"`
URL string `json:"url"`
CategoryID uint `json:"category_id"`
PanID uint `json:"pan_id"`
Tags []uint `json:"tags"`
}
// TransferOutput 转存任务输出数据结构
type TransferOutput struct {
ResourceID uint `json:"resource_id,omitempty"`
SaveURL string `json:"save_url,omitempty"`
Error string `json:"error,omitempty"`
Success bool `json:"success"`
Time string `json:"time"`
}
// Process 处理转存任务项
func (tp *TransferProcessor) Process(ctx context.Context, taskID uint, item *entity.TaskItem) error {
utils.Info("开始处理转存任务项: %d", item.ID)
// 解析输入数据
var input TransferInput
if err := json.Unmarshal([]byte(item.InputData), &input); err != nil {
return fmt.Errorf("解析输入数据失败: %v", err)
}
// 验证输入数据
if err := tp.validateInput(&input); err != nil {
return fmt.Errorf("输入数据验证失败: %v", err)
}
// 获取任务配置中的账号信息
var selectedAccounts []uint
task, err := tp.repoMgr.TaskRepository.GetByID(taskID)
if err == nil && task.Config != "" {
var taskConfig map[string]interface{}
if err := json.Unmarshal([]byte(task.Config), &taskConfig); err == nil {
if accounts, ok := taskConfig["selected_accounts"].([]interface{}); ok {
for _, acc := range accounts {
if accID, ok := acc.(float64); ok {
selectedAccounts = append(selectedAccounts, uint(accID))
}
}
}
}
}
// 检查资源是否已存在
exists, existingResource, err := tp.checkResourceExists(input.URL)
if err != nil {
utils.Error("检查资源是否存在失败: %v", err)
}
if exists {
// 检查已存在的资源是否有有效的转存链接
if existingResource.SaveURL == "" {
// 资源存在但没有转存链接,需要重新转存
utils.Info("资源已存在但无转存链接,重新转存: %s", input.Title)
} else {
// 资源已存在且有转存链接,跳过转存
output := TransferOutput{
ResourceID: existingResource.ID,
SaveURL: existingResource.SaveURL,
Success: true,
Time: utils.GetCurrentTimeString(),
}
outputJSON, _ := json.Marshal(output)
item.OutputData = string(outputJSON)
utils.Info("资源已存在且有转存链接,跳过转存: %s", input.Title)
return nil
}
}
// 执行转存操作
resourceID, saveURL, err := tp.performTransfer(ctx, &input, selectedAccounts)
if err != nil {
// 转存失败,更新输出数据
output := TransferOutput{
Error: err.Error(),
Success: false,
Time: utils.GetCurrentTimeString(),
}
outputJSON, _ := json.Marshal(output)
item.OutputData = string(outputJSON)
utils.Error("转存任务项处理失败: %d, 错误: %v", item.ID, err)
return fmt.Errorf("转存失败: %v", err)
}
// 验证转存结果
if saveURL == "" {
output := TransferOutput{
Error: "转存成功但未获取到分享链接",
Success: false,
Time: utils.GetCurrentTimeString(),
}
outputJSON, _ := json.Marshal(output)
item.OutputData = string(outputJSON)
utils.Error("转存任务项处理失败: %d, 未获取到分享链接", item.ID)
return fmt.Errorf("转存成功但未获取到分享链接")
}
// 转存成功,更新输出数据
output := TransferOutput{
ResourceID: resourceID,
SaveURL: saveURL,
Success: true,
Time: utils.GetCurrentTimeString(),
}
outputJSON, _ := json.Marshal(output)
item.OutputData = string(outputJSON)
utils.Info("转存任务项处理完成: %d, 资源ID: %d, 转存链接: %s", item.ID, resourceID, saveURL)
return nil
}
// validateInput 验证输入数据
func (tp *TransferProcessor) validateInput(input *TransferInput) error {
if strings.TrimSpace(input.Title) == "" {
return fmt.Errorf("标题不能为空")
}
if strings.TrimSpace(input.URL) == "" {
return fmt.Errorf("链接不能为空")
}
// 验证URL格式
if !tp.isValidURL(input.URL) {
return fmt.Errorf("链接格式不正确")
}
return nil
}
// isValidURL 验证URL格式
func (tp *TransferProcessor) isValidURL(url string) bool {
// 简单的URL验证可以根据需要扩展
quarkPattern := `https://pan\.quark\.cn/s/[a-zA-Z0-9]+`
matched, _ := regexp.MatchString(quarkPattern, url)
return matched
}
// checkResourceExists 检查资源是否已存在
func (tp *TransferProcessor) checkResourceExists(url string) (bool, *entity.Resource, error) {
// 根据URL查找资源
resource, err := tp.repoMgr.ResourceRepository.GetByURL(url)
if err != nil {
// 如果是未找到记录的错误,则表示资源不存在
if strings.Contains(err.Error(), "record not found") {
return false, nil, nil
}
return false, nil, err
}
return true, resource, nil
}
// performTransfer 执行转存操作
func (tp *TransferProcessor) performTransfer(ctx context.Context, input *TransferInput, selectedAccounts []uint) (uint, string, error) {
// 解析URL获取分享信息
shareInfo, err := tp.parseShareURL(input.URL)
if err != nil {
return 0, "", fmt.Errorf("解析分享链接失败: %v", err)
}
// 先执行转存操作
saveURL, err := tp.transferToCloud(ctx, shareInfo, selectedAccounts)
if err != nil {
utils.Error("云端转存失败: %v", err)
return 0, "", fmt.Errorf("转存失败: %v", err)
}
// 验证转存链接是否有效
if saveURL == "" {
utils.Error("转存成功但未获取到分享链接")
return 0, "", fmt.Errorf("转存成功但未获取到分享链接")
}
// 转存成功,创建资源记录
var categoryID *uint
if input.CategoryID != 0 {
categoryID = &input.CategoryID
}
// 确定平台ID
var panID uint
if input.PanID != 0 {
// 使用指定的平台ID
panID = input.PanID
utils.Info("使用指定的平台ID: %d", panID)
} else {
// 如果没有指定默认使用夸克平台ID
quarkPanID, err := tp.getQuarkPanID()
if err != nil {
utils.Error("获取夸克平台ID失败: %v", err)
return 0, "", fmt.Errorf("获取夸克平台ID失败: %v", err)
}
panID = quarkPanID
utils.Info("使用默认夸克平台ID: %d", panID)
}
resource := &entity.Resource{
Title: input.Title,
URL: input.URL,
CategoryID: categoryID,
PanID: &panID, // 设置平台ID
SaveURL: saveURL, // 直接设置转存链接
CreatedAt: time.Now(),
UpdatedAt: time.Now(),
}
// 保存资源到数据库
err = tp.repoMgr.ResourceRepository.Create(resource)
if err != nil {
utils.Error("保存转存成功的资源失败: %v", err)
return 0, "", fmt.Errorf("保存资源失败: %v", err)
}
// 添加标签关联
if len(input.Tags) > 0 {
err = tp.addResourceTags(resource.ID, input.Tags)
if err != nil {
utils.Error("添加资源标签失败: %v", err)
// 标签添加失败不影响资源创建,只记录错误
}
}
utils.Info("转存成功,资源已创建 - 资源ID: %d, 转存链接: %s", resource.ID, saveURL)
return resource.ID, saveURL, nil
}
// ShareInfo 分享信息结构
type ShareInfo struct {
PanType string
ShareID string
URL string
}
// parseShareURL 解析分享链接
func (tp *TransferProcessor) parseShareURL(url string) (*ShareInfo, error) {
// 解析夸克网盘链接
quarkPattern := `https://pan\.quark\.cn/s/([a-zA-Z0-9]+)`
re := regexp.MustCompile(quarkPattern)
matches := re.FindStringSubmatch(url)
if len(matches) >= 2 {
return &ShareInfo{
PanType: "quark",
ShareID: matches[1],
URL: url,
}, nil
}
return nil, fmt.Errorf("不支持的分享链接格式: %s", url)
}
// addResourceTags 添加资源标签
func (tp *TransferProcessor) addResourceTags(resourceID uint, tagIDs []uint) error {
for _, tagID := range tagIDs {
// 创建资源标签关联
resourceTag := &entity.ResourceTag{
ResourceID: resourceID,
TagID: tagID,
}
err := tp.repoMgr.ResourceRepository.CreateResourceTag(resourceTag)
if err != nil {
return fmt.Errorf("创建资源标签关联失败: %v", err)
}
}
return nil
}
// transferToCloud 执行云端转存
func (tp *TransferProcessor) transferToCloud(ctx context.Context, shareInfo *ShareInfo, selectedAccounts []uint) (string, error) {
// 转存任务独立于自动转存开关,直接执行转存逻辑
// 获取转存相关的配置(如最小存储空间等),但不检查自动转存开关
// 如果指定了账号,使用指定的账号
if len(selectedAccounts) > 0 {
utils.Info("使用指定的账号进行转存,账号数量: %d", len(selectedAccounts))
// 获取指定的账号
var validAccounts []entity.Cks
for _, accountID := range selectedAccounts {
account, err := tp.repoMgr.CksRepository.FindByID(accountID)
if err != nil {
utils.Error("获取账号 %d 失败: %v", accountID, err)
continue
}
if !account.IsValid {
utils.Error("账号 %d 无效", accountID)
continue
}
validAccounts = append(validAccounts, *account)
}
if len(validAccounts) == 0 {
return "", fmt.Errorf("指定的账号都无效或不存在")
}
utils.Info("找到 %d 个有效账号,开始转存处理...", len(validAccounts))
// 使用第一个有效账号进行转存
account := validAccounts[0]
// 创建网盘服务工厂
factory := pan.NewPanFactory()
// 执行转存
result := tp.transferSingleResource(shareInfo, account, factory)
if !result.Success {
return "", fmt.Errorf("转存失败: %s", result.ErrorMsg)
}
return result.SaveURL, nil
}
// 如果没有指定账号,使用原来的逻辑(自动选择)
utils.Info("未指定账号,使用自动选择逻辑")
// 获取夸克平台ID
quarkPanID, err := tp.getQuarkPanID()
if err != nil {
return "", fmt.Errorf("获取夸克平台ID失败: %v", err)
}
// 获取可用的夸克账号
accounts, err := tp.repoMgr.CksRepository.FindAll()
if err != nil {
return "", fmt.Errorf("获取网盘账号失败: %v", err)
}
// 获取最小存储空间配置(转存任务需要关注此配置)
autoTransferMinSpace, err := tp.repoMgr.SystemConfigRepository.GetConfigInt("auto_transfer_min_space")
if err != nil {
utils.Error("获取最小存储空间配置失败: %v", err)
autoTransferMinSpace = 5 // 默认5GB
}
// 过滤:只保留已激活、夸克平台、剩余空间足够的账号
minSpaceBytes := int64(autoTransferMinSpace) * 1024 * 1024 * 1024
var validAccounts []entity.Cks
for _, acc := range accounts {
if acc.IsValid && acc.PanID == quarkPanID && acc.LeftSpace >= minSpaceBytes {
validAccounts = append(validAccounts, acc)
}
}
if len(validAccounts) == 0 {
return "", fmt.Errorf("没有可用的夸克网盘账号(需要剩余空间 >= %d GB", autoTransferMinSpace)
}
utils.Info("找到 %d 个可用夸克网盘账号,开始转存处理...", len(validAccounts))
// 使用第一个可用账号进行转存
account := validAccounts[0]
// 创建网盘服务工厂
factory := pan.NewPanFactory()
// 执行转存
result := tp.transferSingleResource(shareInfo, account, factory)
if !result.Success {
return "", fmt.Errorf("转存失败: %s", result.ErrorMsg)
}
return result.SaveURL, nil
}
// getQuarkPanID 获取夸克网盘ID
func (tp *TransferProcessor) getQuarkPanID() (uint, error) {
// 通过FindAll方法查找所有平台然后过滤出quark平台
pans, err := tp.repoMgr.PanRepository.FindAll()
if err != nil {
return 0, fmt.Errorf("查询平台信息失败: %v", err)
}
for _, p := range pans {
if p.Name == "quark" {
return p.ID, nil
}
}
return 0, fmt.Errorf("未找到quark平台")
}
// TransferResult 转存结果
type TransferResult struct {
Success bool `json:"success"`
SaveURL string `json:"save_url"`
ErrorMsg string `json:"error_msg"`
}
// transferSingleResource 转存单个资源
func (tp *TransferProcessor) transferSingleResource(shareInfo *ShareInfo, account entity.Cks, factory *pan.PanFactory) TransferResult {
utils.Info("开始转存资源 - 分享ID: %s, 账号: %s", shareInfo.ShareID, account.Username)
service, err := factory.CreatePanService(shareInfo.URL, &pan.PanConfig{
URL: shareInfo.URL,
ExpiredType: 0,
IsType: 0,
Cookie: account.Ck,
})
if err != nil {
utils.Error("创建网盘服务失败: %v", err)
return TransferResult{
Success: false,
ErrorMsg: fmt.Sprintf("创建网盘服务失败: %v", err),
}
}
// 执行转存
transferResult, err := service.Transfer(shareInfo.ShareID)
if err != nil {
utils.Error("转存失败: %v", err)
return TransferResult{
Success: false,
ErrorMsg: fmt.Sprintf("转存失败: %v", err),
}
}
if transferResult == nil || !transferResult.Success {
errMsg := "转存失败"
if transferResult != nil && transferResult.Message != "" {
errMsg = transferResult.Message
}
utils.Error("转存失败: %s", errMsg)
return TransferResult{
Success: false,
ErrorMsg: errMsg,
}
}
// 提取转存链接
var saveURL string
if data, ok := transferResult.Data.(map[string]interface{}); ok {
if v, ok := data["shareUrl"]; ok {
saveURL, _ = v.(string)
}
}
if saveURL == "" {
saveURL = transferResult.ShareURL
}
// 验证转存链接是否有效
if saveURL == "" {
utils.Error("转存成功但未获取到分享链接 - 分享ID: %s", shareInfo.ShareID)
return TransferResult{
Success: false,
ErrorMsg: "转存成功但未获取到分享链接",
}
}
// 验证链接格式
if !strings.HasPrefix(saveURL, "http") {
utils.Error("转存链接格式无效 - 分享ID: %s, 链接: %s", shareInfo.ShareID, saveURL)
return TransferResult{
Success: false,
ErrorMsg: "转存链接格式无效",
}
}
utils.Info("转存成功 - 分享ID: %s, 转存链接: %s", shareInfo.ShareID, saveURL)
return TransferResult{
Success: true,
SaveURL: saveURL,
}
}

View File

@@ -1,229 +0,0 @@
package utils
import (
"sync"
"github.com/ctwj/urldb/db/repo"
)
// GlobalScheduler 全局调度器管理器
type GlobalScheduler struct {
scheduler *Scheduler
mutex sync.RWMutex
}
var (
globalScheduler *GlobalScheduler
once sync.Once
)
// GetGlobalScheduler 获取全局调度器实例(单例模式)
func GetGlobalScheduler(hotDramaRepo repo.HotDramaRepository, readyResourceRepo repo.ReadyResourceRepository, resourceRepo repo.ResourceRepository, systemConfigRepo repo.SystemConfigRepository, panRepo repo.PanRepository, cksRepo repo.CksRepository, tagRepo repo.TagRepository, categoryRepo repo.CategoryRepository) *GlobalScheduler {
once.Do(func() {
globalScheduler = &GlobalScheduler{
scheduler: NewScheduler(hotDramaRepo, readyResourceRepo, resourceRepo, systemConfigRepo, panRepo, cksRepo, tagRepo, categoryRepo),
}
})
return globalScheduler
}
// StartHotDramaScheduler 启动热播剧定时任务
func (gs *GlobalScheduler) StartHotDramaScheduler() {
gs.mutex.Lock()
defer gs.mutex.Unlock()
if gs.scheduler.IsRunning() {
Info("热播剧定时任务已在运行中")
return
}
gs.scheduler.StartHotDramaScheduler()
Info("全局调度器已启动热播剧定时任务")
}
// StopHotDramaScheduler 停止热播剧定时任务
func (gs *GlobalScheduler) StopHotDramaScheduler() {
gs.mutex.Lock()
defer gs.mutex.Unlock()
if !gs.scheduler.IsRunning() {
Info("热播剧定时任务未在运行")
return
}
gs.scheduler.StopHotDramaScheduler()
Info("全局调度器已停止热播剧定时任务")
}
// IsHotDramaSchedulerRunning 检查热播剧定时任务是否在运行
func (gs *GlobalScheduler) IsHotDramaSchedulerRunning() bool {
gs.mutex.RLock()
defer gs.mutex.RUnlock()
return gs.scheduler.IsRunning()
}
// GetHotDramaNames 手动获取热播剧名字
func (gs *GlobalScheduler) GetHotDramaNames() ([]string, error) {
return gs.scheduler.GetHotDramaNames()
}
// StartReadyResourceScheduler 启动待处理资源自动处理任务
func (gs *GlobalScheduler) StartReadyResourceScheduler() {
gs.mutex.Lock()
defer gs.mutex.Unlock()
if gs.scheduler.IsReadyResourceRunning() {
Info("待处理资源自动处理任务已在运行中")
return
}
gs.scheduler.StartReadyResourceScheduler()
Info("全局调度器已启动待处理资源自动处理任务")
}
// StopReadyResourceScheduler 停止待处理资源自动处理任务
func (gs *GlobalScheduler) StopReadyResourceScheduler() {
gs.mutex.Lock()
defer gs.mutex.Unlock()
if !gs.scheduler.IsReadyResourceRunning() {
Info("待处理资源自动处理任务未在运行")
return
}
gs.scheduler.StopReadyResourceScheduler()
Info("全局调度器已停止待处理资源自动处理任务")
}
// IsReadyResourceRunning 检查待处理资源自动处理任务是否在运行
func (gs *GlobalScheduler) IsReadyResourceRunning() bool {
gs.mutex.RLock()
defer gs.mutex.RUnlock()
return gs.scheduler.IsReadyResourceRunning()
}
// ProcessReadyResources 手动触发待处理资源处理
func (gs *GlobalScheduler) ProcessReadyResources() {
gs.mutex.Lock()
defer gs.mutex.Unlock()
gs.scheduler.processReadyResources()
}
// UpdateSchedulerStatus 根据系统配置更新调度器状态
func (gs *GlobalScheduler) UpdateSchedulerStatus(autoFetchHotDramaEnabled bool, autoProcessReadyResources bool) {
gs.mutex.Lock()
defer gs.mutex.Unlock()
// 处理热播剧自动拉取功能
if autoFetchHotDramaEnabled {
if !gs.scheduler.IsRunning() {
Info("系统配置启用自动拉取热播剧,启动定时任务")
gs.scheduler.StartHotDramaScheduler()
}
} else {
if gs.scheduler.IsRunning() {
Info("系统配置禁用自动拉取热播剧,停止定时任务")
gs.scheduler.StopHotDramaScheduler()
}
}
// 处理待处理资源自动处理功能
if autoProcessReadyResources {
if !gs.scheduler.IsReadyResourceRunning() {
Info("系统配置启用自动处理待处理资源,启动定时任务")
gs.scheduler.StartReadyResourceScheduler()
}
} else {
if gs.scheduler.IsReadyResourceRunning() {
Info("系统配置禁用自动处理待处理资源,停止定时任务")
gs.scheduler.StopReadyResourceScheduler()
}
}
}
// StartAutoTransferScheduler 启动自动转存定时任务
func (gs *GlobalScheduler) StartAutoTransferScheduler() {
gs.mutex.Lock()
defer gs.mutex.Unlock()
if gs.scheduler.IsAutoTransferRunning() {
Info("自动转存定时任务已在运行中")
return
}
gs.scheduler.StartAutoTransferScheduler()
Info("全局调度器已启动自动转存定时任务")
}
// StopAutoTransferScheduler 停止自动转存定时任务
func (gs *GlobalScheduler) StopAutoTransferScheduler() {
gs.mutex.Lock()
defer gs.mutex.Unlock()
if !gs.scheduler.IsAutoTransferRunning() {
Info("自动转存定时任务未在运行")
return
}
gs.scheduler.StopAutoTransferScheduler()
Info("全局调度器已停止自动转存定时任务")
}
// IsAutoTransferRunning 检查自动转存定时任务是否在运行
func (gs *GlobalScheduler) IsAutoTransferRunning() bool {
gs.mutex.RLock()
defer gs.mutex.RUnlock()
return gs.scheduler.IsAutoTransferRunning()
}
// ProcessAutoTransfer 手动触发自动转存处理
func (gs *GlobalScheduler) ProcessAutoTransfer() {
gs.mutex.Lock()
defer gs.mutex.Unlock()
gs.scheduler.processAutoTransfer()
}
// UpdateSchedulerStatusWithAutoTransfer 根据系统配置更新调度器状态(包含自动转存)
func (gs *GlobalScheduler) UpdateSchedulerStatusWithAutoTransfer(autoFetchHotDramaEnabled bool, autoProcessReadyResources bool, autoTransferEnabled bool) {
gs.mutex.Lock()
defer gs.mutex.Unlock()
// 处理热播剧自动拉取功能
if autoFetchHotDramaEnabled {
if !gs.scheduler.IsRunning() {
Info("系统配置启用自动拉取热播剧,启动定时任务")
gs.scheduler.StartHotDramaScheduler()
}
} else {
if gs.scheduler.IsRunning() {
Info("系统配置禁用自动拉取热播剧,停止定时任务")
gs.scheduler.StopHotDramaScheduler()
}
}
// 处理待处理资源自动处理功能
if autoProcessReadyResources {
if !gs.scheduler.IsReadyResourceRunning() {
Info("系统配置启用自动处理待处理资源,启动定时任务")
gs.scheduler.StartReadyResourceScheduler()
}
} else {
if gs.scheduler.IsReadyResourceRunning() {
Info("系统配置禁用自动处理待处理资源,停止定时任务")
gs.scheduler.StopReadyResourceScheduler()
}
}
// 处理自动转存功能
if autoTransferEnabled {
if !gs.scheduler.IsAutoTransferRunning() {
Info("系统配置启用自动转存,启动定时任务")
gs.scheduler.StartAutoTransferScheduler()
}
} else {
if gs.scheduler.IsAutoTransferRunning() {
Info("系统配置禁用自动转存,停止定时任务")
gs.scheduler.StopAutoTransferScheduler()
}
}
}

File diff suppressed because it is too large Load Diff

View File

@@ -5,6 +5,13 @@ import (
"time"
)
// 时间格式常量
const (
TimeFormatDate = "2006-01-02"
TimeFormatDateTime = "2006-01-02 15:04:05"
TimeFormatRFC3339 = time.RFC3339
)
// InitTimezone 初始化时区设置
func InitTimezone() {
// 从环境变量获取时区配置
@@ -36,20 +43,35 @@ func GetCurrentTime() time.Time {
// GetCurrentTimeString 获取当前时间字符串(使用配置的时区)
func GetCurrentTimeString() string {
return time.Now().Format("2006-01-02 15:04:05")
return time.Now().Format(TimeFormatDateTime)
}
// GetCurrentTimeRFC3339 获取当前时间RFC3339格式使用配置的时区
func GetCurrentTimeRFC3339() string {
return time.Now().Format(time.RFC3339)
return time.Now().Format(TimeFormatRFC3339)
}
// ParseTime 解析时间字符串(使用配置的时区)
func ParseTime(timeStr string) (time.Time, error) {
return time.Parse("2006-01-02 15:04:05", timeStr)
return time.Parse(TimeFormatDateTime, timeStr)
}
// FormatTime 格式化时间(使用配置的时区)
func FormatTime(t time.Time, layout string) string {
return t.Format(layout)
}
// GetTodayString 获取今日日期字符串
func GetTodayString() string {
return time.Now().Format(TimeFormatDate)
}
// GetCurrentTimestamp 获取当前时间戳
func GetCurrentTimestamp() int64 {
return time.Now().Unix()
}
// GetCurrentTimestampNano 获取当前纳秒时间戳
func GetCurrentTimestampNano() int64 {
return time.Now().UnixNano()
}

View File

@@ -23,13 +23,14 @@ type VersionInfo struct {
// 编译时注入的版本信息
var (
Version = getVersionFromFile()
// 这些变量将在编译时通过 ldflags 注入
Version = "unknown" // 默认版本,编译时会被覆盖
BuildTime = GetCurrentTimeString()
GitCommit = "unknown"
GitBranch = "unknown"
)
// getVersionFromFile 从VERSION文件读取版本号
// getVersionFromFile 从VERSION文件读取版本号(备用方案)
func getVersionFromFile() string {
data, err := os.ReadFile("VERSION")
if err != nil {
@@ -42,11 +43,29 @@ func getVersionFromFile() string {
func GetVersionInfo() *VersionInfo {
buildTime, _ := ParseTime(BuildTime)
// 检查版本信息是否通过编译时注入
version := Version
gitCommit := GitCommit
gitBranch := GitBranch
// 如果编译时注入的版本是默认值,尝试从文件读取
if version == "unknown" {
version = getVersionFromFile()
}
// 如果Git信息是默认值尝试从文件读取
if gitCommit == "unknown" {
gitCommit = "unknown"
}
if gitBranch == "unknown" {
gitBranch = "unknown"
}
return &VersionInfo{
Version: Version,
Version: version,
BuildTime: buildTime,
GitCommit: GitCommit,
GitBranch: GitBranch,
GitCommit: gitCommit,
GitBranch: gitBranch,
GoVersion: runtime.Version(),
NodeVersion: getNodeVersion(),
Platform: runtime.GOOS,
@@ -72,7 +91,7 @@ func GetFullVersionInfo() string {
Node版本: %s
平台: %s/%s`,
info.Version,
FormatTime(info.BuildTime, "2006-01-02 15:04:05"),
FormatTime(info.BuildTime, TimeFormatDateTime),
info.GitCommit,
info.GitBranch,
info.GoVersion,

View File

@@ -32,4 +32,14 @@
.resource-card {
@apply bg-white rounded-lg shadow-md border border-gray-200 p-4 hover:shadow-lg transition-shadow duration-200;
}
/* 搜索高亮样式 */
mark {
@apply bg-yellow-200 text-yellow-900 px-1 py-0.5 rounded font-medium;
}
/* 暗色模式下的高亮样式 */
.dark mark {
@apply bg-yellow-600 text-yellow-100;
}
}

33
web/components.d.ts vendored
View File

@@ -10,11 +10,44 @@ declare module 'vue' {
export interface GlobalComponents {
NA: typeof import('naive-ui')['NA']
NAlert: typeof import('naive-ui')['NAlert']
NAvatar: typeof import('naive-ui')['NAvatar']
NButton: typeof import('naive-ui')['NButton']
NButtonGroup: typeof import('naive-ui')['NButtonGroup']
NCard: typeof import('naive-ui')['NCard']
NCheckbox: typeof import('naive-ui')['NCheckbox']
NDataTable: typeof import('naive-ui')['NDataTable']
NDatePicker: typeof import('naive-ui')['NDatePicker']
NDescriptions: typeof import('naive-ui')['NDescriptions']
NDescriptionsItem: typeof import('naive-ui')['NDescriptionsItem']
NDialogProvider: typeof import('naive-ui')['NDialogProvider']
NEmpty: typeof import('naive-ui')['NEmpty']
NForm: typeof import('naive-ui')['NForm']
NFormItem: typeof import('naive-ui')['NFormItem']
NImage: typeof import('naive-ui')['NImage']
NImageGroup: typeof import('naive-ui')['NImageGroup']
NInput: typeof import('naive-ui')['NInput']
NList: typeof import('naive-ui')['NList']
NListItem: typeof import('naive-ui')['NListItem']
NMessageProvider: typeof import('naive-ui')['NMessageProvider']
NModal: typeof import('naive-ui')['NModal']
NNotificationProvider: typeof import('naive-ui')['NNotificationProvider']
NP: typeof import('naive-ui')['NP']
NPagination: typeof import('naive-ui')['NPagination']
NProgress: typeof import('naive-ui')['NProgress']
NQrCode: typeof import('naive-ui')['NQrCode']
NSelect: typeof import('naive-ui')['NSelect']
NSpace: typeof import('naive-ui')['NSpace']
NSpin: typeof import('naive-ui')['NSpin']
NSwitch: typeof import('naive-ui')['NSwitch']
NTable: typeof import('naive-ui')['NTable']
NTabPane: typeof import('naive-ui')['NTabPane']
NTabs: typeof import('naive-ui')['NTabs']
NTag: typeof import('naive-ui')['NTag']
NText: typeof import('naive-ui')['NText']
NThing: typeof import('naive-ui')['NThing']
NUpload: typeof import('naive-ui')['NUpload']
NUploadDragger: typeof import('naive-ui')['NUploadDragger']
NVirtualList: typeof import('naive-ui')['NVirtualList']
RouterLink: typeof import('vue-router')['RouterLink']
RouterView: typeof import('vue-router')['RouterView']
}

View File

@@ -7,8 +7,8 @@
<p class="mb-2"><strong>格式要求</strong>标题和URL为一组标题必填, 同一标题URL支持多行</p>
<pre class="bg-white dark:bg-gray-800 p-2 rounded border text-xs">
电影1
https://pan.baidu.com/s/123456 # 1
https://pan.quark.com/s/123456 # 1
https://pan.baidu.com/s/123456
https://pan.quark.com/s/123456
电影标题2
https://pan.baidu.com/s/789012
电视剧标题3
@@ -21,9 +21,9 @@ https://pan.quark.cn/s/345678</pre>
</div>
<div class="mb-4 flex-1 w-1">
<label class="block text-sm font-medium text-gray-700 dark:text-gray-300 mb-2">资源内容</label>
<textarea v-model="batchInput" rows="15"
class="w-full px-3 py-2 border border-gray-300 dark:border-gray-700 rounded-md focus:outline-none focus:ring-2 focus:ring-blue-500 focus:border-blue-500 dark:bg-gray-900 dark:text-gray-100"
placeholder="请输入资源内容格式标题和URL两行为一组..."></textarea>
<n-input v-model:value="batchInput" type="textarea"
:autosize="{ minRows: 10, maxRows: 15 }"
placeholder="请输入资源内容格式标题和URL为一组..." />
</div>
</div>
@@ -45,6 +45,7 @@ const emit = defineEmits(['success', 'error', 'cancel'])
const loading = ref(false)
const batchInput = ref('')
const notification = useNotification()
const readyResourceApi = useReadyResourceApi()
@@ -63,7 +64,11 @@ const validateInput = () => {
//
if (/^https?:\/\//i.test(lines[0])) {
// alertElMessage
alert('首行必须为标题,不能为链接!')
notification.error({
title: '失败',
content: '首行必须为标题,不能为链接!',
duration: 3000
})
return
}
}

View File

@@ -18,10 +18,6 @@
{{ currentPageTitle }}
</span>
</div>
<!-- 页面描述 -->
<!-- <div v-if="currentPageDescription && currentPageTitle !== '管理后台'" class="text-xs text-white/60 mt-1">
{{ currentPageDescription }}
</div> -->
</div>
<div class="absolute left-4 top-4 flex items-center gap-2">
@@ -37,7 +33,7 @@
<!-- 用户信息 -->
<div v-if="userStore.isAuthenticated" class="hidden sm:flex items-center gap-2">
<span class="text-sm text-white/80">欢迎{{ userStore.user?.username || '管理员' }}</span>
<span class="px-2 py-1 bg-blue-600/80 rounded text-xs text-white">{{ userStore.user?.role || 'admin' }}</span>
<n-tag type="success" size="small" round>{{ userStore.user?.role || '-' }}</n-tag>
</div>
<!-- 操作按钮 -->
@@ -116,6 +112,7 @@ const pageConfig = computed(() => {
'/admin/users': { title: '用户管理', icon: 'fas fa-users', description: '管理系统用户' },
'/admin/categories': { title: '分类管理', icon: 'fas fa-folder', description: '管理资源分类' },
'/admin/tags': { title: '标签管理', icon: 'fas fa-tags', description: '管理资源标签' },
'/admin/tasks': { title: '任务管理', icon: 'fas fa-tasks', description: '管理系统任务' },
'/admin/system-config': { title: '系统配置', icon: 'fas fa-cog', description: '系统参数设置' },
'/admin/resources': { title: '资源管理', icon: 'fas fa-database', description: '管理网盘资源' },
'/admin/cks': { title: '平台账号管理', icon: 'fas fa-key', description: '管理第三方平台账号' },
@@ -125,7 +122,8 @@ const pageConfig = computed(() => {
'/monitor': { title: '系统监控', icon: 'fas fa-desktop', description: '系统性能监控' },
'/admin/add-resource': { title: '添加资源', icon: 'fas fa-plus', description: '添加新资源' },
'/api-docs': { title: 'API文档', icon: 'fas fa-book', description: '接口文档说明' },
'/admin/version': { title: '版本信息', icon: 'fas fa-code-branch', description: '系统版本详情' }
'/admin/version': { title: '版本信息', icon: 'fas fa-code-branch', description: '系统版本详情' },
'/admin/failed-resources': { title: '错误资源', icon: 'fas fa-code-branch', description: '错误资源' }
}
return configs[route.path] || { title: props.title, icon: 'fas fa-cog', description: '管理页面' }
})
@@ -138,7 +136,7 @@ const systemConfigStore = useSystemConfigStore()
const systemConfig = computed(() => systemConfigStore.config)
onMounted(() => {
systemConfigStore.initConfig()
systemConfigStore.initConfig(false, true)
})
// 退

View File

@@ -0,0 +1,572 @@
<template>
<div class="space-y-6">
<!-- 输入区域 -->
<n-card title="批量转存资源列表">
<div class="grid grid-cols-1 md:grid-cols-2 gap-6">
<!-- 左侧资源输入 -->
<div class="space-y-4">
<div>
<label class="block text-sm font-medium text-gray-700 dark:text-gray-300 mb-2">
资源内容 <span class="text-red-500">*</span>
</label>
<n-input
v-model:value="resourceText"
type="textarea"
placeholder="请输入资源内容格式标题和URL为一组..."
:autosize="{ minRows: 10, maxRows: 15 }"
show-count
:maxlength="100000"
/>
</div>
</div>
<!-- 右侧配置选项 -->
<div class="space-y-4">
<div>
<label class="block text-sm font-medium text-gray-700 dark:text-gray-300 mb-2">
默认分类
</label>
<CategorySelector
v-model="selectedCategory"
placeholder="选择分类"
clearable
/>
</div>
<div>
<label class="block text-sm font-medium text-gray-700 dark:text-gray-300 mb-2">
标签
</label>
<TagSelector
v-model="selectedTags"
placeholder="选择标签"
multiple
clearable
/>
</div>
<div>
<label class="block text-sm font-medium text-gray-700 dark:text-gray-300 mb-2">
网盘账号 <span class="text-red-500">*</span>
</label>
<n-select
v-model:value="selectedAccounts"
:options="accountOptions"
placeholder="选择网盘账号"
multiple
filterable
:loading="accountsLoading"
@update:value="handleAccountChange"
>
<template #option="{ option: accountOption }">
<div class="flex items-center justify-between w-full">
<div class="flex items-center space-x-2">
<span class="text-sm">{{ accountOption.label }}</span>
<n-tag v-if="accountOption.is_valid" type="success" size="small">有效</n-tag>
<n-tag v-else type="error" size="small">无效</n-tag>
</div>
<div class="text-xs text-gray-500">
{{ formatSpace(accountOption.left_space) }}
</div>
</div>
</template>
</n-select>
<div class="text-xs text-gray-500 mt-1">
请选择要使用的网盘账号系统将使用选中的账号进行转存操作
</div>
</div>
<!-- 操作按钮 -->
<div class="space-y-3 pt-4">
<n-button
type="primary"
block
size="large"
:loading="processing"
:disabled="!resourceText.trim() || !selectedAccounts.length || processing"
@click="handleBatchTransfer"
>
<template #icon>
<i class="fas fa-upload"></i>
</template>
开始批量转存
</n-button>
<n-button
block
@click="clearInput"
:disabled="processing"
>
<template #icon>
<i class="fas fa-trash"></i>
</template>
清空内容
</n-button>
</div>
</div>
</div>
</n-card>
<!-- 处理结果 -->
<n-card v-if="results.length > 0" title="转存结果">
<div class="space-y-4">
<!-- 结果统计 -->
<div class="grid grid-cols-4 gap-4">
<div class="text-center p-3 bg-blue-50 dark:bg-blue-900/20 rounded-lg">
<div class="text-xl font-bold text-blue-600">{{ results.length }}</div>
<div class="text-sm text-gray-600 dark:text-gray-400">总处理数</div>
</div>
<div class="text-center p-3 bg-green-50 dark:bg-green-900/20 rounded-lg">
<div class="text-xl font-bold text-green-600">{{ successCount }}</div>
<div class="text-sm text-gray-600 dark:text-gray-400">成功</div>
</div>
<div class="text-center p-3 bg-red-50 dark:bg-red-900/20 rounded-lg">
<div class="text-xl font-bold text-red-600">{{ failedCount }}</div>
<div class="text-sm text-gray-600 dark:text-gray-400">失败</div>
</div>
<div class="text-center p-3 bg-yellow-50 dark:bg-yellow-900/20 rounded-lg">
<div class="text-xl font-bold text-yellow-600">{{ processingCount }}</div>
<div class="text-sm text-gray-600 dark:text-gray-400">处理中</div>
</div>
</div>
<!-- 结果列表 -->
<n-data-table
:columns="resultColumns"
:data="results"
:pagination="false"
max-height="300"
size="small"
/>
</div>
</n-card>
</div>
</template>
<script setup lang="ts">
import { ref, computed, onMounted, onBeforeUnmount, h } from 'vue'
import { usePanApi, useTaskApi, useCksApi } from '~/composables/useApi'
import { useMessage } from 'naive-ui'
// 数据状态
const resourceText = ref('')
const processing = ref(false)
const results = ref<any[]>([])
// 任务状态
const currentTaskId = ref<number | null>(null)
const taskStatus = ref<any>(null)
const taskStats = ref({
total: 0,
pending: 0,
processing: 0,
completed: 0,
failed: 0
})
const statusCheckInterval = ref<NodeJS.Timeout | null>(null)
// 配置选项
const selectedCategory = ref(null)
const selectedTags = ref([])
const selectedPlatform = ref(null)
const autoValidate = ref(true)
const skipExisting = ref(true)
const autoTransfer = ref(false)
const selectedAccounts = ref<number[]>([])
// 选项数据
const platformOptions = ref<any[]>([])
const accountOptions = ref<any[]>([])
const accountsLoading = ref(false)
// API实例
const panApi = usePanApi()
const taskApi = useTaskApi()
const cksApi = useCksApi()
const message = useMessage()
// 计算属性
const totalLines = computed(() => {
return resourceText.value ? resourceText.value.split('\n').filter(line => line.trim()).length : 0
})
const validUrls = computed(() => {
if (!resourceText.value) return 0
const lines = resourceText.value.split('\n').filter(line => line.trim())
return lines.filter(line => isValidUrl(line.trim())).length
})
const invalidUrls = computed(() => {
return totalLines.value - validUrls.value
})
const successCount = computed(() => {
return results.value.filter((r: any) => r.status === 'success').length
})
const failedCount = computed(() => {
return results.value.filter((r: any) => r.status === 'failed').length
})
const processingCount = computed(() => {
return results.value.filter((r: any) => r.status === 'processing').length
})
// 结果表格列
const resultColumns = [
{
title: '标题',
key: 'title',
width: 200,
ellipsis: {
tooltip: true
}
},
{
title: '链接',
key: 'url',
width: 250,
ellipsis: {
tooltip: true
}
},
{
title: '状态',
key: 'status',
width: 100,
render: (row: any) => {
const statusMap = {
success: { color: 'success', text: '成功', icon: 'fas fa-check' },
failed: { color: 'error', text: '失败', icon: 'fas fa-times' },
processing: { color: 'info', text: '处理中', icon: 'fas fa-spinner fa-spin' },
pending: { color: 'warning', text: '等待中', icon: 'fas fa-clock' }
}
const status = statusMap[row.status as keyof typeof statusMap] || statusMap.failed
return h('n-tag', { type: status.color }, {
icon: () => h('i', { class: status.icon }),
default: () => status.text
})
}
},
{
title: '消息',
key: 'message',
ellipsis: {
tooltip: true
}
},
{
title: '转存链接',
key: 'saveUrl',
width: 200,
ellipsis: {
tooltip: true
},
render: (row: any) => {
if (row.saveUrl) {
return h('a', {
href: row.saveUrl,
target: '_blank',
class: 'text-blue-500 hover:text-blue-700'
}, '查看')
}
return '-'
}
}
]
// URL验证
const isValidUrl = (url: string) => {
try {
new URL(url)
// 简单检查是否包含常见网盘域名
const diskDomains = ['quark.cn', 'pan.baidu.com', 'aliyundrive.com']
return diskDomains.some(domain => url.includes(domain))
} catch {
return false
}
}
// 获取平台选项
const fetchPlatforms = async () => {
try {
const result = await panApi.getPans() as any
if (result && Array.isArray(result)) {
platformOptions.value = result.map((item: any) => ({
label: item.remark || item.name,
value: item.id
}))
}
} catch (error) {
console.error('获取平台失败:', error)
}
}
// 处理批量转存
const handleBatchTransfer = async () => {
if (!resourceText.value.trim()) {
message.warning('请输入资源内容')
return
}
if (!selectedAccounts.value || selectedAccounts.value.length === 0) {
message.warning('请选择至少一个网盘账号')
return
}
processing.value = true
results.value = []
try {
// 第一步:拆解资源信息,按照一行标题,一行链接的形式
const resourceList = parseResourceText(resourceText.value)
if (resourceList.length === 0) {
message.warning('没有找到有效的资源信息请按照格式要求输入标题和URL为一组标题必填')
return
}
// 第二步:生成任务标题和数据
const taskTitle = `批量转存任务_${new Date().toLocaleString('zh-CN')}`
const taskData = {
title: taskTitle,
description: `批量转存 ${resourceList.length} 个资源,使用 ${selectedAccounts.value.length} 个账号`,
resources: resourceList.map(item => {
const resource: any = {
title: item.title,
url: item.url
}
if (selectedCategory.value) {
resource.category_id = selectedCategory.value
}
if (selectedTags.value && selectedTags.value.length > 0) {
resource.tags = selectedTags.value
}
return resource
}),
// 添加选择的账号信息
selected_accounts: selectedAccounts.value
}
console.log('创建任务数据:', taskData)
// 第三步:创建任务
const taskResponse = await taskApi.createBatchTransferTask(taskData) as any
console.log('任务创建响应:', taskResponse)
currentTaskId.value = taskResponse.task_id
// 第四步:启动任务
await taskApi.startTask(currentTaskId.value!)
// 第五步:开始实时监控任务状态
startTaskMonitoring()
message.success('任务已创建并启动,开始处理...')
} catch (error: any) {
console.error('创建任务失败:', error)
message.error('创建任务失败: ' + (error.message || '未知错误'))
processing.value = false
}
}
// 解析资源文本,按照 标题\n链接 的格式支持同一标题多个URL
const parseResourceText = (text: string) => {
const lines = text.split('\n').filter((line: string) => line.trim())
const resourceList = []
let currentTitle = ''
let currentUrls = []
for (const line of lines) {
// 判断是否为 url以 http/https 开头)
if (/^https?:\/\//i.test(line)) {
currentUrls.push(line.trim())
} else {
// 新标题,先保存上一个
if (currentTitle && currentUrls.length > 0) {
// 为每个URL创建一个资源项
for (const url of currentUrls) {
if (isValidUrl(url)) {
resourceList.push({
title: currentTitle,
url: url,
category_id: selectedCategory.value || 0,
tags: selectedTags.value || []
})
}
}
}
currentTitle = line.trim()
currentUrls = []
}
}
// 处理最后一组
if (currentTitle && currentUrls.length > 0) {
for (const url of currentUrls) {
if (isValidUrl(url)) {
resourceList.push({
title: currentTitle,
url: url,
category_id: selectedCategory.value || 0,
tags: selectedTags.value || []
})
}
}
}
return resourceList
}
// 开始任务监控
const startTaskMonitoring = () => {
if (statusCheckInterval.value) {
clearInterval(statusCheckInterval.value)
}
statusCheckInterval.value = setInterval(async () => {
try {
const status = await taskApi.getTaskStatus(currentTaskId.value!) as any
console.log('任务状态更新:', status)
taskStatus.value = status
taskStats.value = status.stats || {
total: 0,
pending: 0,
processing: 0,
completed: 0,
failed: 0
}
// 更新结果显示
updateResultsDisplay()
// 如果任务完成,停止监控
if (status.status === 'completed' || status.status === 'failed' || status.status === 'partial_success') {
stopTaskMonitoring()
processing.value = false
const { completed, failed } = taskStats.value
message.success(`批量转存完成!成功: ${completed}, 失败: ${failed}`)
}
} catch (error) {
console.error('获取任务状态失败:', error)
// 如果连续失败,停止监控
stopTaskMonitoring()
processing.value = false
}
}, 2000) // 每2秒检查一次
}
// 停止任务监控
const stopTaskMonitoring = () => {
if (statusCheckInterval.value) {
clearInterval(statusCheckInterval.value)
statusCheckInterval.value = null
}
}
// 更新结果显示
const updateResultsDisplay = () => {
if (!taskStatus.value) return
// 如果还没有结果,初始化
if (results.value.length === 0) {
const resourceList = parseResourceText(resourceText.value)
results.value = resourceList.map(item => ({
title: item.title,
url: item.url,
status: 'pending',
message: '等待处理...',
saveUrl: null
}))
}
// 更新整体进度显示
const { pending, processing, completed, failed } = taskStats.value
const processed = completed + failed
// 简单的状态更新逻辑 - 这里可以根据需要获取详细的任务项状态
for (let i = 0; i < results.value.length; i++) {
const result = results.value[i]
if (i < completed) {
// 已完成的项目
result.status = 'success'
result.message = '转存成功'
} else if (i < completed + failed) {
// 失败的项目
result.status = 'failed'
result.message = '转存失败'
} else if (i < processed + processing) {
// 正在处理的项目
result.status = 'processing'
result.message = '正在处理...'
} else {
// 等待处理的项目
result.status = 'pending'
result.message = '等待处理...'
}
}
}
// 获取网盘账号选项
const getAccountOptions = async () => {
accountsLoading.value = true
try {
const response = await cksApi.getCks() as any
const accounts = Array.isArray(response) ? response : []
accountOptions.value = accounts.map((account: any) => ({
label: `${account.username || '未知用户'} (${account.pan?.name || '未知平台'})`,
value: account.id,
is_valid: account.is_valid,
left_space: account.left_space,
username: account.username,
pan_name: account.pan?.name || '未知平台'
}))
} catch (error) {
console.error('获取网盘账号选项失败:', error)
message.error('获取网盘账号失败')
} finally {
accountsLoading.value = false
}
}
// 处理账号选择变化
const handleAccountChange = (value: number[]) => {
selectedAccounts.value = value
console.log('选择的账号:', value)
}
// 格式化空间大小
const formatSpace = (bytes: number) => {
if (bytes === 0) return '0 B'
const k = 1024
const sizes = ['B', 'KB', 'MB', 'GB', 'TB']
const i = Math.floor(Math.log(bytes) / Math.log(k))
return parseFloat((bytes / Math.pow(k, i)).toFixed(2)) + ' ' + sizes[i]
}
// 清空输入
const clearInput = () => {
resourceText.value = ''
results.value = []
selectedAccounts.value = []
}
// 初始化
onMounted(() => {
fetchPlatforms()
getAccountOptions()
})
// 组件销毁时清理定时器
onBeforeUnmount(() => {
stopTaskMonitoring()
})
</script>

View File

@@ -0,0 +1,147 @@
<template>
<header class="bg-white dark:bg-gray-800 shadow-sm border-b border-gray-200 dark:border-gray-700">
<div class="flex items-center justify-between px-6 py-4">
<!-- 左侧Logo和标题 -->
<div class="flex items-center">
<NuxtLink to="/admin" class="flex items-center space-x-3">
<div class="w-8 h-8 bg-blue-600 rounded-lg flex items-center justify-center">
<i class="fas fa-shield-alt text-white text-sm"></i>
</div>
<div>
<h1 class="text-xl font-bold text-gray-900 dark:text-white">管理后台</h1>
<p class="text-xs text-gray-500 dark:text-gray-400">老九网盘资源数据库</p>
</div>
</NuxtLink>
</div>
<!-- 中间状态信息 -->
<div class="flex items-center space-x-6">
<!-- 系统状态 -->
<div class="flex items-center space-x-2">
<div class="w-2 h-2 bg-green-500 rounded-full"></div>
<span class="text-sm text-gray-600 dark:text-gray-300">系统正常</span>
</div>
<!-- 自动处理状态 -->
<NuxtLink to="/admin/feature-config" class="flex items-center space-x-2 cursor-pointer hover:bg-gray-100 dark:hover:bg-gray-700 px-2 py-1 rounded transition-colors">
<div :class="autoProcessEnabled ? 'w-2 h-2 bg-green-500 rounded-full' : 'w-2 h-2 bg-gray-400 rounded-full'"></div>
<span class="text-sm text-gray-600 dark:text-gray-300">
自动处理{{ autoProcessEnabled ? '已开启' : '已关闭' }}
</span>
</NuxtLink>
<!-- 自动转存状态 -->
<div class="flex items-center space-x-2">
<div :class="autoTransferEnabled ? 'w-2 h-2 bg-blue-500 rounded-full' : 'w-2 h-2 bg-gray-400 rounded-full'"></div>
<span class="text-sm text-gray-600 dark:text-gray-300">
自动转存{{ autoTransferEnabled ? '已开启' : '已关闭' }}
</span>
</div>
<!-- 任务状态 -->
<div v-if="taskStore.hasActiveTasks" class="flex items-center space-x-2">
<div class="w-2 h-2 bg-orange-500 rounded-full animate-pulse"></div>
<span class="text-sm text-gray-600 dark:text-gray-300">
<template v-if="taskStore.runningTaskCount > 0">
{{ taskStore.runningTaskCount }}个任务运行中
</template>
<template v-else>
{{ taskStore.activeTaskCount }}个任务待处理
</template>
</span>
</div>
</div>
<!-- 右侧用户菜单 -->
<div class="flex items-center space-x-4">
<NuxtLink to="/" class="p-2 text-gray-400 hover:text-gray-600 dark:hover:text-gray-300 transition-colors">
<i class="fas fa-home text-lg"></i>
</NuxtLink>
<NuxtLink to="/admin-old" class="p-2 text-gray-400 hover:text-gray-600 dark:hover:text-gray-300 transition-colors">
<i class="fas fa-arrow-left text-lg"></i>
</NuxtLink>
<div class="flex items-center space-x-2 cursor-pointer p-2 rounded-lg hover:bg-gray-100 dark:hover:bg-gray-700 transition-colors">
<div class="w-8 h-8 bg-blue-600 rounded-full flex items-center justify-center">
<i class="fas fa-user text-white text-sm"></i>
</div>
<div class="hidden md:block text-left">
<p class="text-sm font-medium text-gray-900 dark:text-white">管理员</p>
<p class="text-xs text-gray-500 dark:text-gray-400">admin</p>
</div>
</div>
</div>
</div>
</header>
</template>
<script setup lang="ts">
import { ref, onMounted, onBeforeUnmount } from 'vue'
import { useTaskStore } from '~/stores/task'
import { useSystemConfigStore } from '~/stores/systemConfig'
// 任务状态管理
const taskStore = useTaskStore()
// 系统配置状态管理
const systemConfigStore = useSystemConfigStore()
// 自动处理和自动转存状态
const autoProcessEnabled = ref(false)
const autoTransferEnabled = ref(false)
// 获取系统配置状态
const fetchSystemStatus = async () => {
try {
await systemConfigStore.initConfig(false, true)
// 从系统配置中获取自动处理和自动转存状态
const config = systemConfigStore.config
if (config) {
// 检查自动处理状态
autoProcessEnabled.value = config.auto_process_ready_resources === '1' || config.auto_process_ready_resources === true
// 检查自动转存状态
autoTransferEnabled.value = config.auto_transfer_enabled === '1' || config.auto_transfer_enabled === true
}
} catch (error) {
console.error('获取系统状态失败:', error)
}
}
// 组件挂载时启动
onMounted(() => {
// 启动任务状态自动更新
taskStore.startAutoUpdate()
// 获取系统配置状态
fetchSystemStatus()
// 定期更新系统配置状态每30秒
const configInterval = setInterval(fetchSystemStatus, 30000)
// 保存定时器引用用于清理
;(globalThis as any).__configInterval = configInterval
})
// 组件销毁时清理
onBeforeUnmount(() => {
// 停止任务状态自动更新
taskStore.stopAutoUpdate()
// 清理配置更新定时器
if ((globalThis as any).__configInterval) {
clearInterval((globalThis as any).__configInterval)
delete (globalThis as any).__configInterval
}
})
</script>
<style scoped>
/* 确保Font Awesome图标正确显示 */
.fas {
font-family: 'Font Awesome 6 Free';
font-weight: 900;
}
</style>

View File

@@ -0,0 +1,229 @@
<template>
<aside class="w-64 bg-white dark:bg-gray-800 shadow-sm border-r border-gray-200 dark:border-gray-700 min-h-screen">
<nav class="mt-8">
<div class="px-4 space-y-2">
<!-- 仪表盘 -->
<NuxtLink
:to="dashboardItem.to"
class="flex items-center px-4 py-3 text-gray-700 dark:text-gray-300 hover:bg-blue-50 dark:hover:bg-blue-900/20 hover:text-blue-600 dark:hover:text-blue-400 rounded-lg transition-colors"
:class="{ 'bg-blue-50 dark:bg-blue-900/20 text-blue-600 dark:text-blue-400': dashboardItem.active($route) }"
>
<i :class="dashboardItem.icon + ' w-5 h-5 mr-3'"></i>
<span>{{ dashboardItem.label }}</span>
</NuxtLink>
<!-- 数据管理分组 -->
<div class="mt-6">
<div class="px-4 py-2 text-xs font-semibold text-gray-500 dark:text-gray-400 uppercase tracking-wider">
数据管理
</div>
<div class="space-y-1">
<NuxtLink
v-for="item in dataItems"
:key="item.to"
:to="item.to"
class="flex items-center px-4 py-3 text-gray-700 dark:text-gray-300 hover:bg-blue-50 dark:hover:bg-blue-900/20 hover:text-blue-600 dark:hover:text-blue-400 rounded-lg transition-colors"
:class="{ 'bg-blue-50 dark:bg-blue-900/20 text-blue-600 dark:text-blue-400': item.active($route) }"
>
<i :class="item.icon + ' w-5 h-5 mr-3'"></i>
<span>{{ item.label }}</span>
</NuxtLink>
</div>
</div>
<!-- 运营管理分组 -->
<div class="mt-6">
<div class="px-4 py-2 text-xs font-semibold text-gray-500 dark:text-gray-400 uppercase tracking-wider">
运营管理
</div>
<div class="space-y-1">
<NuxtLink
v-for="item in operationItems"
:key="item.to"
:to="item.to"
class="flex items-center px-4 py-3 text-gray-700 dark:text-gray-300 hover:bg-blue-50 dark:hover:bg-blue-900/20 hover:text-blue-600 dark:hover:text-blue-400 rounded-lg transition-colors"
:class="{ 'bg-blue-50 dark:bg-blue-900/20 text-blue-600 dark:text-blue-400': item.active($route) }"
>
<i :class="item.icon + ' w-5 h-5 mr-3'"></i>
<span>{{ item.label }}</span>
</NuxtLink>
</div>
</div>
<!-- 统计分析分组 -->
<div class="mt-6">
<div class="px-4 py-2 text-xs font-semibold text-gray-500 dark:text-gray-400 uppercase tracking-wider">
统计分析
</div>
<div class="space-y-1">
<NuxtLink
v-for="item in statisticsItems"
:key="item.to"
:to="item.to"
class="flex items-center px-4 py-3 text-gray-700 dark:text-gray-300 hover:bg-blue-50 dark:hover:bg-blue-900/20 hover:text-blue-600 dark:hover:text-blue-400 rounded-lg transition-colors"
:class="{ 'bg-blue-50 dark:bg-blue-900/20 text-blue-600 dark:text-blue-400': item.active($route) }"
>
<i :class="item.icon + ' w-5 h-5 mr-3'"></i>
<span>{{ item.label }}</span>
</NuxtLink>
</div>
</div>
<!-- 系统管理分组 -->
<div class="mt-6">
<div class="px-4 py-2 text-xs font-semibold text-gray-500 dark:text-gray-400 uppercase tracking-wider">
系统管理
</div>
<div class="space-y-1">
<NuxtLink
v-for="item in systemItems"
:key="item.to"
:to="item.to"
class="flex items-center px-4 py-3 text-gray-700 dark:text-gray-300 hover:bg-blue-50 dark:hover:bg-blue-900/20 hover:text-blue-600 dark:hover:text-blue-400 rounded-lg transition-colors"
:class="{ 'bg-blue-50 dark:bg-blue-900/20 text-blue-600 dark:text-blue-400': item.active($route) }"
>
<i :class="item.icon + ' w-5 h-5 mr-3'"></i>
<span>{{ item.label }}</span>
</NuxtLink>
</div>
</div>
</div>
</nav>
</aside>
</template>
<script setup lang="ts">
// 仪表盘
const dashboardItem = ref({
to: '/admin',
label: '仪表盘',
icon: 'fas fa-tachometer-alt',
active: (route: any) => route.path === '/admin'
})
// 数据管理分组
const dataItems = ref([
{
to: '/admin/resources',
label: '资源管理',
icon: 'fas fa-database',
active: (route: any) => route.path.startsWith('/admin/resources')
},
{
to: '/admin/ready-resources',
label: '待处理资源',
icon: 'fas fa-clock',
active: (route: any) => route.path.startsWith('/admin/ready-resources')
},
{
to: '/admin/categories',
label: '分类管理',
icon: 'fas fa-folder',
active: (route: any) => route.path.startsWith('/admin/categories')
},
{
to: '/admin/tags',
label: '标签管理',
icon: 'fas fa-tags',
active: (route: any) => route.path.startsWith('/admin/tags')
},
{
to: '/admin/files',
label: '文件管理',
icon: 'fas fa-file-upload',
active: (route: any) => route.path.startsWith('/admin/files')
}
])
// 运营管理分组
const operationItems = ref([
{
to: '/admin/platforms',
label: '平台管理',
icon: 'fas fa-cloud',
active: (route: any) => route.path.startsWith('/admin/platforms')
},
{
to: '/admin/accounts',
label: '账号管理',
icon: 'fas fa-user-shield',
active: (route: any) => route.path.startsWith('/admin/accounts')
},
{
to: '/admin/hot-dramas',
label: '热播剧管理',
icon: 'fas fa-film',
active: (route: any) => route.path.startsWith('/admin/hot-dramas')
},
{
to: '/admin/data-transfer',
label: '数据转存管理',
icon: 'fas fa-exchange-alt',
active: (route: any) => route.path.startsWith('/admin/data-transfer')
},
{
to: '/admin/seo',
label: 'SEO',
icon: 'fas fa-search',
active: (route: any) => route.path.startsWith('/admin/seo')
},
{
to: '/admin/data-push',
label: '数据推送',
icon: 'fas fa-upload',
active: (route: any) => route.path.startsWith('/admin/data-push')
},
{
to: '/admin/bot',
label: '机器人',
icon: 'fas fa-robot',
active: (route: any) => route.path.startsWith('/admin/bot')
}
])
// 统计分析分组
const statisticsItems = ref([
{
to: '/admin/search-stats',
label: '搜索统计',
icon: 'fas fa-chart-line',
active: (route: any) => route.path.startsWith('/admin/search-stats')
},
{
to: '/admin/third-party-stats',
label: '三方统计',
icon: 'fas fa-chart-bar',
active: (route: any) => route.path.startsWith('/admin/third-party-stats')
}
])
// 系统管理分组
const systemItems = ref([
{
to: '/admin/users',
label: '用户管理',
icon: 'fas fa-users',
active: (route: any) => route.path.startsWith('/admin/users')
},
{
to: '/admin/system-config',
label: '系统配置',
icon: 'fas fa-cog',
active: (route: any) => route.path.startsWith('/admin/system-config')
},
{
to: '/admin/version',
label: '版本信息',
icon: 'fas fa-code-branch',
active: (route: any) => route.path.startsWith('/admin/version')
}
])
</script>
<style scoped>
/* 确保Font Awesome图标正确显示 */
.fas {
font-family: 'Font Awesome 6 Free';
font-weight: 900;
}
</style>

View File

@@ -5,9 +5,8 @@
<label class="block text-sm font-medium text-gray-700 dark:text-gray-300 mb-2">
标题 <span class="text-red-500">*</span>
</label>
<input
v-model="form.title"
class="input-field dark:bg-gray-900 dark:text-gray-100 dark:border-gray-700"
<n-input
v-model:value="form.title"
placeholder="输入资源标题(必填)"
required
/>
@@ -18,12 +17,11 @@
<label class="block text-sm font-medium text-gray-700 dark:text-gray-300 mb-2">
描述 <span class="text-gray-400 text-xs">(可选)</span>
</label>
<textarea
v-model="form.description"
rows="3"
class="input-field dark:bg-gray-900 dark:text-gray-100 dark:border-gray-700"
<n-input
v-model:value="form.description"
type="textarea"
placeholder="输入资源描述,如:剧情简介、文件大小、清晰度等"
></textarea>
/>
</div>
<!-- URL -->
@@ -31,13 +29,12 @@
<label class="block text-sm font-medium text-gray-700 dark:text-gray-300 mb-2">
URL <span class="text-red-500">*</span>
</label>
<textarea
v-model="form.url"
rows="3"
class="input-field dark:bg-gray-900 dark:text-gray-100 dark:border-gray-700"
<n-input
v-model:value="form.url"
type="textarea"
placeholder="请输入资源链接,支持多行,每行一个链接"
required
></textarea>
/>
<p class="text-xs text-gray-500 dark:text-gray-400 mt-1">
支持百度网盘阿里云盘夸克网盘等链接每行一个链接
</p>
@@ -48,9 +45,8 @@
<label class="block text-sm font-medium text-gray-700 dark:text-gray-300 mb-2">
分类 <span class="text-gray-400 text-xs">(可选)</span>
</label>
<input
v-model="form.category"
class="input-field dark:bg-gray-900 dark:text-gray-100 dark:border-gray-700"
<n-input
v-model:value="form.category"
placeholder="如:电影、电视剧、动漫、音乐等"
/>
</div>
@@ -76,10 +72,9 @@
</button>
</span>
</div>
<input
v-model="newTag"
<n-input
v-model:value="newTag"
@keyup.enter.prevent="addTag"
class="input-field dark:bg-gray-900 dark:text-gray-100 dark:border-gray-700"
placeholder="输入标签后回车添加,多个标签用逗号分隔"
/>
</div>
@@ -89,9 +84,8 @@
<label class="block text-sm font-medium text-gray-700 dark:text-gray-300 mb-2">
封面图片 <span class="text-gray-400 text-xs">(可选)</span>
</label>
<input
v-model="form.img"
class="input-field dark:bg-gray-900 dark:text-gray-100 dark:border-gray-700"
<n-input
v-model:value="form.img"
placeholder="封面图片链接"
/>
</div>
@@ -101,9 +95,8 @@
<label class="block text-sm font-medium text-gray-700 dark:text-gray-300 mb-2">
数据来源 <span class="text-gray-400 text-xs">(可选)</span>
</label>
<input
v-model="form.source"
class="input-field dark:bg-gray-900 dark:text-gray-100 dark:border-gray-700"
<n-input
v-model:value="form.source"
placeholder="如手动添加、API导入、爬虫等"
/>
</div>
@@ -113,12 +106,11 @@
<label class="block text-sm font-medium text-gray-700 dark:text-gray-300 mb-2">
额外数据 <span class="text-gray-400 text-xs">(可选)</span>
</label>
<textarea
v-model="form.extra"
rows="3"
class="input-field dark:bg-gray-900 dark:text-gray-100 dark:border-gray-700"
<n-input
v-model:value="form.extra"
type="textarea"
placeholder="JSON格式的额外数据{'size': '2GB', 'quality': '1080p'}"
></textarea>
/>
</div>
<!-- 按钮区域 -->

View File

@@ -0,0 +1,249 @@
<template>
<div class="space-y-4">
<!-- 搜索和筛选 -->
<div class="grid grid-cols-1 md:grid-cols-4 gap-4">
<n-input
v-model:value="searchQuery"
placeholder="搜索已转存资源..."
@keyup.enter="handleSearch"
clearable
>
<template #prefix>
<i class="fas fa-search"></i>
</template>
</n-input>
<CategorySelector
v-model="selectedCategory"
placeholder="选择分类"
clearable
/>
<TagSelector
v-model="selectedTag"
placeholder="选择标签"
clearable
/>
<n-button type="primary" @click="handleSearch">
<template #icon>
<i class="fas fa-search"></i>
</template>
搜索
</n-button>
</div>
<!-- 调试信息 -->
<div class="text-sm text-gray-500 mb-2">
数据数量: {{ resources.length }}, 总数: {{ total }}, 加载状态: {{ loading }}
</div>
<!-- 数据表格 -->
<n-data-table
:columns="columns"
:data="resources"
:loading="loading"
:pagination="pagination"
:remote="true"
@update:page="handlePageChange"
@update:page-size="handlePageSizeChange"
:row-key="(row: any) => row.id"
virtual-scroll
max-height="500"
/>
</div>
</template>
<script setup lang="ts">
import { ref, reactive, computed, onMounted, h } from 'vue'
import { useResourceApi, usePanApi } from '~/composables/useApi'
import { useMessage } from 'naive-ui'
// 消息提示
const $message = useMessage()
// 数据状态
const loading = ref(false)
const resources = ref<any[]>([])
const total = ref(0)
const currentPage = ref(1)
const pageSize = ref(10000)
// 搜索条件
const searchQuery = ref('')
const selectedCategory = ref(null)
const selectedTag = ref(null)
// API实例
const resourceApi = useResourceApi()
const panApi = usePanApi()
// 获取平台数据
const { data: platformsData } = await useAsyncData('transferredPlatforms', () => panApi.getPans())
// 平台选项
const platformOptions = computed(() => {
const data = platformsData.value as any
const platforms = data?.data || data || []
return platforms.map((platform: any) => ({
label: platform.remark || platform.name,
value: platform.id
}))
})
// 获取平台名称
const getPlatformName = (platformId: number) => {
const platform = (platformsData.value as any)?.data?.find((plat: any) => plat.id === platformId)
return platform?.remark || platform?.name || '未知平台'
}
// 分页配置
const pagination = reactive({
page: 1,
pageSize: 10000,
itemCount: 0,
pageSizes: [10000, 20000, 50000, 100000],
showSizePicker: true,
showQuickJumper: true,
prefix: ({ itemCount }: any) => `${itemCount}`
})
// 表格列配置
const columns: any[] = [
{
title: 'ID',
key: 'id',
width: 60,
fixed: 'left' as const
},
{
title: '标题',
key: 'title',
width: 200,
ellipsis: {
tooltip: true
}
},
{
title: '分类',
key: 'category_name',
width: 80
},
{
title: '转存链接',
key: 'save_url',
width: 200,
ellipsis: {
tooltip: true
},
render: (row: any) => {
return h('a', {
href: row.save_url,
target: '_blank',
class: 'text-green-500 hover:text-green-700'
}, row.save_url.length > 30 ? row.save_url.substring(0, 30) + '...' : row.save_url)
}
},
{
title: '转存时间',
key: 'updated_at',
width: 130,
render: (row: any) => {
return new Date(row.updated_at).toLocaleDateString()
}
}
]
// 获取已转存资源
const fetchTransferredResources = async () => {
loading.value = true
try {
const params: any = {
page: currentPage.value,
page_size: pageSize.value,
has_save_url: true // 筛选有转存链接的资源
}
if (searchQuery.value) {
params.search = searchQuery.value
}
if (selectedCategory.value) {
params.category_id = selectedCategory.value
}
console.log('请求参数:', params)
const result = await resourceApi.getResources(params) as any
console.log('已转存资源结果:', result)
console.log('结果类型:', typeof result)
console.log('结果结构:', Object.keys(result || {}))
if (result && result.data) {
// 处理嵌套的data结构{data: {data: [...], total: ...}}
if (result.data.data && Array.isArray(result.data.data)) {
console.log('使用嵌套data格式数量:', result.data.data.length)
resources.value = result.data.data
total.value = result.data.total || 0
pagination.itemCount = result.data.total || 0
} else {
// 处理直接的data结构{data: [...], total: ...}
console.log('使用直接data格式数量:', result.data.length)
resources.value = result.data
total.value = result.total || 0
pagination.itemCount = result.total || 0
}
} else if (Array.isArray(result)) {
console.log('使用数组格式,数量:', result.length)
resources.value = result
total.value = result.length
pagination.itemCount = result.length
} else {
console.log('未知格式,设置空数组')
resources.value = []
total.value = 0
pagination.itemCount = 0
}
console.log('最终 resources.value:', resources.value)
console.log('最终 total.value:', total.value)
// 检查是否有资源没有 save_url
const resourcesWithoutSaveUrl = resources.value.filter((r: any) => !r.save_url || r.save_url.trim() === '')
if (resourcesWithoutSaveUrl.length > 0) {
console.warn('发现没有 save_url 的资源:', resourcesWithoutSaveUrl.map((r: any) => ({ id: r.id, title: r.title, save_url: r.save_url })))
}
} catch (error) {
console.error('获取已转存资源失败:', error)
resources.value = []
total.value = 0
} finally {
loading.value = false
}
}
// 搜索处理
const handleSearch = () => {
currentPage.value = 1
pagination.page = 1
fetchTransferredResources()
}
// 分页处理
const handlePageChange = (page: number) => {
currentPage.value = page
pagination.page = page
fetchTransferredResources()
}
const handlePageSizeChange = (size: number) => {
pageSize.value = size
pagination.pageSize = size
currentPage.value = 1
pagination.page = 1
fetchTransferredResources()
}
// 初始化
onMounted(() => {
fetchTransferredResources()
})
</script>

View File

@@ -0,0 +1,633 @@
<template>
<div class="space-y-4">
<!-- 搜索和筛选 -->
<div class="grid grid-cols-1 md:grid-cols-5 gap-4">
<n-input
v-model:value="searchQuery"
placeholder="搜索未转存资源..."
@keyup.enter="handleSearch"
clearable
>
<template #prefix>
<i class="fas fa-search"></i>
</template>
</n-input>
<CategorySelector
v-model="selectedCategory"
placeholder="选择分类"
clearable
/>
<TagSelector
v-model="selectedTag"
placeholder="选择标签"
clearable
/>
<n-select
v-model:value="selectedStatus"
placeholder="资源状态"
:options="statusOptions"
clearable
/>
<n-button type="primary" @click="handleSearch">
<template #icon>
<i class="fas fa-search"></i>
</template>
搜索
</n-button>
</div>
<!-- 批量操作 -->
<n-card>
<div class="flex items-center justify-between">
<div class="flex items-center space-x-4">
<div class="flex items-center space-x-2">
<n-checkbox
:checked="isAllSelected"
@update:checked="toggleSelectAll"
:indeterminate="isIndeterminate"
/>
<span class="text-sm text-gray-600 dark:text-gray-400">全选</span>
</div>
<span class="text-sm text-gray-500">
{{ total }} 个资源已选择 {{ selectedResources.length }}
</span>
</div>
<div class="flex space-x-2">
<n-button
type="primary"
:disabled="selectedResources.length === 0"
:loading="batchTransferring"
@click="handleBatchTransfer"
>
<template #icon>
<i class="fas fa-exchange-alt"></i>
</template>
批量转存 ({{ selectedResources.length }})
</n-button>
<n-button @click="refreshData">
<template #icon>
<i class="fas fa-refresh"></i>
</template>
刷新
</n-button>
</div>
</div>
</n-card>
<!-- 资源列表 -->
<n-card>
<div v-if="loading" class="flex items-center justify-center py-8">
<n-spin size="large" />
</div>
<div v-else-if="resources.length === 0" class="text-center py-8">
<i class="fas fa-inbox text-4xl text-gray-400 mb-4"></i>
<p class="text-gray-500">暂无未转存的夸克资源</p>
</div>
<div v-else>
<!-- 虚拟列表 -->
<n-virtual-list
:items="resources"
:item-size="120"
style="max-height: 500px"
container-style="height: 500px;"
>
<template #default="{ item }">
<div class="p-4 border-b border-gray-200 dark:border-gray-700 hover:bg-gray-50 dark:hover:bg-gray-800">
<div class="flex items-start space-x-4">
<!-- 选择框 -->
<div class="pt-2">
<n-checkbox
:checked="selectedResources.includes(item.id)"
@update:checked="(checked) => toggleResourceSelection(item.id, checked)"
/>
</div>
<!-- 资源信息 -->
<div class="flex-1 min-w-0">
<div class="flex items-start justify-between">
<div class="flex-1 min-w-0">
<!-- 标题和状态 -->
<div class="flex items-center space-x-2 mb-2">
<h3 class="text-lg font-medium text-gray-900 dark:text-white line-clamp-1">
{{ item.title || '未命名资源' }}
</h3>
<n-tag :type="getStatusType(item)" size="small">
{{ getStatusText(item) }}
</n-tag>
</div>
<!-- 描述 -->
<p class="text-gray-600 dark:text-gray-400 text-sm line-clamp-2 mb-2">
{{ item.description || '暂无描述' }}
</p>
<!-- 元信息 -->
<div class="flex items-center space-x-4 text-sm text-gray-500">
<span class="flex items-center">
<i class="fas fa-folder mr-1"></i>
{{ item.category_name || '未分类' }}
</span>
<span class="flex items-center">
<i class="fas fa-eye mr-1"></i>
{{ item.view_count || 0 }} 次浏览
</span>
<span class="flex items-center">
<i class="fas fa-calendar mr-1"></i>
{{ formatDate(item.created_at) }}
</span>
</div>
<!-- 原始链接 -->
<div class="mt-2">
<div class="flex items-center space-x-2">
<span class="text-xs text-gray-400">原始链接:</span>
<a
:href="item.url"
target="_blank"
class="text-xs text-blue-500 hover:text-blue-700 truncate max-w-xs"
>
{{ item.url }}
</a>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</template>
</n-virtual-list>
<!-- 分页 -->
<div class="mt-4 flex justify-center">
<n-pagination
v-model:page="currentPage"
v-model:page-size="pageSize"
:item-count="total"
:page-sizes="[10000, 20000, 50000, 100000]"
show-size-picker
show-quick-jumper
@update:page="handlePageChange"
@update:page-size="handlePageSizeChange"
/>
</div>
</div>
</n-card>
<!-- 网盘账号选择模态框 -->
<n-modal v-model:show="showAccountSelectionModal" preset="card" title="选择网盘账号" style="width: 600px">
<div class="space-y-4">
<div class="text-sm text-gray-600 dark:text-gray-400">
请选择要使用的网盘账号进行批量转存操作
</div>
<div>
<label class="block text-sm font-medium text-gray-700 dark:text-gray-300 mb-2">
网盘账号 <span class="text-red-500">*</span>
</label>
<n-select
v-model:value="selectedAccounts"
:options="accountOptions"
placeholder="选择网盘账号"
multiple
filterable
:loading="accountsLoading"
@update:value="handleAccountChange"
>
<template #option="{ option: accountOption }">
<div class="flex items-center justify-between w-full">
<div class="flex items-center space-x-2">
<span class="text-sm">{{ accountOption.label }}</span>
<n-tag v-if="accountOption.is_valid" type="success" size="small">有效</n-tag>
<n-tag v-else type="error" size="small">无效</n-tag>
</div>
<div class="text-xs text-gray-500">
{{ formatSpace(accountOption.left_space) }}
</div>
</div>
</template>
</n-select>
<div class="text-xs text-gray-500 mt-1">
请选择要使用的网盘账号系统将使用选中的账号进行转存操作
</div>
</div>
<div class="bg-yellow-50 dark:bg-yellow-900/20 p-3 rounded border border-yellow-200 dark:border-yellow-800">
<div class="flex items-start space-x-2">
<i class="fas fa-exclamation-triangle text-yellow-500 mt-0.5"></i>
<div class="text-sm text-yellow-800 dark:text-yellow-200">
<p> 转存过程可能需要较长时间</p>
<p> 请确保选中的网盘账号有足够的存储空间</p>
<p> 转存完成后可在"已转存列表"中查看结果</p>
</div>
</div>
</div>
</div>
<template #footer>
<div class="flex justify-end space-x-3">
<n-button @click="showAccountSelectionModal = false">
取消
</n-button>
<n-button
type="primary"
:disabled="selectedAccounts.length === 0"
:loading="batchTransferring"
@click="confirmBatchTransfer"
>
{{ batchTransferring ? '创建任务中...' : '继续' }}
</n-button>
</div>
</template>
</n-modal>
<!-- 转存结果模态框 -->
<n-modal v-model:show="showTransferResult" preset="card" title="转存结果" style="width: 600px">
<div v-if="transferResults.length > 0" class="space-y-4">
<div class="grid grid-cols-3 gap-4">
<div class="text-center p-3 bg-green-50 dark:bg-green-900/20 rounded-lg">
<div class="text-xl font-bold text-green-600">{{ transferSuccessCount }}</div>
<div class="text-sm text-gray-600 dark:text-gray-400">成功</div>
</div>
<div class="text-center p-3 bg-red-50 dark:bg-red-900/20 rounded-lg">
<div class="text-xl font-bold text-red-600">{{ transferFailedCount }}</div>
<div class="text-sm text-gray-600 dark:text-gray-400">失败</div>
</div>
<div class="text-center p-3 bg-blue-50 dark:bg-blue-900/20 rounded-lg">
<div class="text-xl font-bold text-blue-600">{{ transferResults.length }}</div>
<div class="text-sm text-gray-600 dark:text-gray-400">总计</div>
</div>
</div>
<div class="max-h-300 overflow-y-auto">
<div v-for="result in transferResults" :key="result.id" class="p-3 border rounded mb-2">
<div class="flex items-center justify-between">
<div class="flex-1 min-w-0">
<div class="text-sm font-medium truncate">{{ result.title }}</div>
<div class="text-xs text-gray-500 truncate">{{ result.url }}</div>
</div>
<n-tag :type="result.success ? 'success' : 'error'" size="small">
{{ result.success ? '成功' : '失败' }}
</n-tag>
</div>
<div v-if="result.message" class="text-xs text-gray-600 mt-1">
{{ result.message }}
</div>
</div>
</div>
</div>
</n-modal>
</div>
</template>
<script setup lang="ts">
import { ref, reactive, computed, onMounted } from 'vue'
import { useResourceApi, useCategoryApi, useTagApi, useCksApi, useTaskApi, usePanApi } from '~/composables/useApi'
import { useMessage } from 'naive-ui'
// 消息提示
const $message = useMessage()
// 数据状态
const loading = ref(false)
const resources = ref([])
const total = ref(0)
const currentPage = ref(1)
const pageSize = ref(2000)
// 搜索条件
const searchQuery = ref('')
const selectedCategory = ref(null)
const selectedTag = ref(null)
const selectedStatus = ref(null)
// 选择状态
const selectedResources = ref([])
// 批量操作状态
const batchTransferring = ref(false)
const showTransferResult = ref(false)
const transferResults = ref([])
const showAccountSelectionModal = ref(false)
const selectedAccounts = ref<number[]>([])
const accountOptions = ref<any[]>([])
const accountsLoading = ref(false)
// 选项数据
const categoryOptions = ref([])
const tagOptions = ref([])
const statusOptions = [
{ label: '有效', value: 'valid' },
{ label: '无效', value: 'invalid' },
{ label: '待验证', value: 'pending' }
]
// API实例
const resourceApi = useResourceApi()
const categoryApi = useCategoryApi()
const tagApi = useTagApi()
const cksApi = useCksApi()
const taskApi = useTaskApi()
const panApi = usePanApi()
// 获取平台数据
const { data: platformsData } = await useAsyncData('untransferredPlatforms', () => panApi.getPans())
// 计算属性
const isAllSelected = computed(() => {
return resources.value.length > 0 && selectedResources.value.length === resources.value.length
})
const isIndeterminate = computed(() => {
return selectedResources.value.length > 0 && selectedResources.value.length < resources.value.length
})
const transferSuccessCount = computed(() => {
return transferResults.value.filter(r => r.success).length
})
const transferFailedCount = computed(() => {
return transferResults.value.filter(r => !r.success).length
})
// 获取未转存资源夸克网盘且无save_url
const fetchUntransferredResources = async () => {
loading.value = true
try {
const params = {
page: currentPage.value,
page_size: pageSize.value,
no_save_url: true, // 筛选没有转存链接的资源
pan_name: 'quark' // 仅夸克网盘资源
}
if (searchQuery.value) {
params.search = searchQuery.value
}
if (selectedCategory.value) {
params.category_id = selectedCategory.value
}
const result = await resourceApi.getResources(params) as any
console.log('未转存资源结果:', result)
if (result && result.data) {
// 处理嵌套的data结构{data: {data: [...], total: ...}}
if (result.data.data && Array.isArray(result.data.data)) {
resources.value = result.data.data
total.value = result.data.total || 0
} else {
// 处理直接的data结构{data: [...], total: ...}
resources.value = result.data
total.value = result.total || 0
}
} else if (Array.isArray(result)) {
resources.value = result
total.value = result.length
}
// 清空选择
selectedResources.value = []
} catch (error) {
console.error('获取未转存资源失败:', error)
resources.value = []
total.value = 0
} finally {
loading.value = false
}
}
// 获取分类选项
const fetchCategories = async () => {
try {
const result = await categoryApi.getCategories() as any
if (result && result.items) {
categoryOptions.value = result.items.map((item: any) => ({
label: item.name,
value: item.id
}))
}
} catch (error) {
console.error('获取分类失败:', error)
}
}
// 获取标签选项
const fetchTags = async () => {
try {
const result = await tagApi.getTags() as any
if (result && result.items) {
tagOptions.value = result.items.map((item: any) => ({
label: item.name,
value: item.id
}))
}
} catch (error) {
console.error('获取标签失败:', error)
}
}
// 获取网盘账号选项
const getAccountOptions = async () => {
accountsLoading.value = true
try {
const response = await cksApi.getCks() as any
const accounts = Array.isArray(response) ? response : []
accountOptions.value = accounts.map((account: any) => ({
label: `${account.username || '未知用户'} (${account.pan?.name || '未知平台'})`,
value: account.id,
is_valid: account.is_valid,
left_space: account.left_space,
username: account.username,
pan_name: account.pan?.name || '未知平台'
}))
} catch (error) {
console.error('获取网盘账号选项失败:', error)
$message.error('获取网盘账号失败')
} finally {
accountsLoading.value = false
}
}
// 处理账号选择变化
const handleAccountChange = (value: number[]) => {
selectedAccounts.value = value
console.log('选择的账号:', value)
}
// 格式化空间大小
const formatSpace = (bytes: number) => {
if (bytes === 0) return '0 B'
const k = 1024
const sizes = ['B', 'KB', 'MB', 'GB', 'TB']
const i = Math.floor(Math.log(bytes) / Math.log(k))
return parseFloat((bytes / Math.pow(k, i)).toFixed(2)) + ' ' + sizes[i]
}
// 搜索处理
const handleSearch = () => {
currentPage.value = 1
fetchUntransferredResources()
}
// 刷新数据
const refreshData = () => {
fetchUntransferredResources()
}
// 分页处理
const handlePageChange = (page: number) => {
currentPage.value = page
fetchUntransferredResources()
}
const handlePageSizeChange = (size: number) => {
pageSize.value = size
currentPage.value = 1
fetchUntransferredResources()
}
// 选择处理
const toggleSelectAll = (checked: boolean) => {
if (checked) {
selectedResources.value = resources.value.map(r => r.id)
} else {
selectedResources.value = []
}
}
const toggleResourceSelection = (id: number, checked: boolean) => {
if (checked) {
if (!selectedResources.value.includes(id)) {
selectedResources.value.push(id)
}
} else {
const index = selectedResources.value.indexOf(id)
if (index > -1) {
selectedResources.value.splice(index, 1)
}
}
}
// 批量转存
const handleBatchTransfer = async () => {
if (selectedResources.value.length === 0) {
$message.warning('请选择要转存的资源')
return
}
// 先获取网盘账号列表
await getAccountOptions()
// 显示账号选择模态框
showAccountSelectionModal.value = true
}
// 获取状态类型
const getStatusType = (resource: any) => {
if (resource.is_valid === false) return 'error'
if (resource.is_valid === true) return 'success'
return 'warning'
}
// 获取状态文本
const getStatusText = (resource: any) => {
if (resource.is_valid === false) return '无效'
if (resource.is_valid === true) return '有效'
return '待验证'
}
// 获取平台名称
const getPlatformName = (panId: number) => {
if (!panId) return '未知平台'
// 从后端获取的平台数据
const platforms = platformsData.value as any
const platform = platforms?.data?.find((p: any) => p.id === panId)
return platform?.remark || platform?.name || '未知平台'
}
// 格式化日期
const formatDate = (dateString: string) => {
return new Date(dateString).toLocaleDateString()
}
// 确认批量转存
const confirmBatchTransfer = async () => {
if (selectedAccounts.value.length === 0) {
$message.warning('请选择至少一个网盘账号')
return
}
batchTransferring.value = true
try {
const selectedItems = resources.value.filter(r => selectedResources.value.includes(r.id))
const taskData = {
title: `批量转存 ${selectedItems.length} 个资源`,
description: `批量转存 ${selectedItems.length} 个资源,使用 ${selectedAccounts.value.length} 个账号`,
resources: selectedItems.map(r => ({
title: r.title,
url: r.url,
category_id: r.category_id || 0,
pan_id: r.pan_id || 0
})),
selected_accounts: selectedAccounts.value
}
const response = await taskApi.createBatchTransferTask(taskData) as any
$message.success(`批量转存任务已创建,共 ${selectedItems.length} 个资源`)
// 关闭模态框
showAccountSelectionModal.value = false
selectedAccounts.value = []
// 刷新列表
refreshData()
} catch (error) {
console.error('创建批量转存任务失败:', error)
$message.error('创建批量转存任务失败')
} finally {
batchTransferring.value = false
}
}
// 初始化
onMounted(() => {
fetchCategories()
fetchTags()
fetchUntransferredResources()
})
</script>
<style scoped>
.line-clamp-1 {
display: -webkit-box;
-webkit-line-clamp: 1;
-webkit-box-orient: vertical;
overflow: hidden;
}
.line-clamp-2 {
display: -webkit-box;
-webkit-line-clamp: 2;
-webkit-box-orient: vertical;
overflow: hidden;
}
</style>

View File

@@ -25,7 +25,7 @@ import { parseApiResponse } from '~/composables/useApi'
const { versionInfo, fetchVersionInfo } = useVersion()
// 获取系统配置
const { data: systemConfigData } = await useAsyncData('systemConfig',
const { data: systemConfigData } = await useAsyncData('footerSystemConfig',
() => useApiFetch('/system/config').then(parseApiResponse)
)

View File

@@ -0,0 +1,105 @@
<template>
<n-select
v-model:value="selectedValue"
:placeholder="placeholder"
:options="categoryOptions"
:loading="loading"
:clearable="clearable"
:filterable="true"
:disabled="disabled"
@update:value="handleUpdate"
/>
</template>
<script setup lang="ts">
import { ref, watch, onMounted } from 'vue'
import { useCategoryApi } from '~/composables/useApi'
// Props定义
interface Props {
modelValue?: number | null
placeholder?: string
clearable?: boolean
disabled?: boolean
}
const props = withDefaults(defineProps<Props>(), {
placeholder: '选择分类',
clearable: true,
disabled: false
})
// Emits定义
const emit = defineEmits<{
'update:modelValue': [value: number | null]
}>()
// 定义选项类型
interface CategoryOption {
label: string
value: number
disabled: boolean
}
// 内部状态
const selectedValue = ref(props.modelValue)
const categoryOptions = ref<CategoryOption[]>([])
const loading = ref(false)
// API实例
const categoryApi = useCategoryApi()
// 监听外部值变化
watch(
() => props.modelValue,
(newValue) => {
selectedValue.value = newValue
}
)
// 监听内部值变化并向外发射
const handleUpdate = (value: number | null) => {
selectedValue.value = value
emit('update:modelValue', value)
}
// 加载分类数据
const loadCategories = async () => {
// 如果已经加载过,直接返回
if (categoryOptions.value.length > 0) {
return
}
loading.value = true
try {
const result = await categoryApi.getCategories() as any
const options: CategoryOption[] = []
if (result && result.items) {
options.push(...result.items.map((item: any) => ({
label: item.name,
value: item.id,
disabled: false
})))
} else if (Array.isArray(result)) {
options.push(...result.map((item: any) => ({
label: item.name,
value: item.id,
disabled: false
})))
}
categoryOptions.value = options
} catch (error) {
console.error('获取分类失败:', error)
categoryOptions.value = []
} finally {
loading.value = false
}
}
// 组件挂载时立即加载分类
onMounted(() => {
loadCategories()
})
</script>

View File

@@ -1,58 +0,0 @@
<template>
<div v-if="show" class="fixed top-4 right-4 z-50">
<div class="bg-red-50 border border-red-200 rounded-lg p-4 shadow-lg max-w-sm">
<div class="flex items-start">
<div class="flex-shrink-0">
<i class="fas fa-exclamation-circle text-red-400"></i>
</div>
<div class="ml-3 flex-1">
<h3 class="text-sm font-medium text-red-800">错误</h3>
<div class="mt-1 text-sm text-red-700">
{{ message }}
</div>
</div>
<div class="ml-4 flex-shrink-0">
<button
@click="close"
class="inline-flex text-red-400 hover:text-red-600 focus:outline-none"
>
<i class="fas fa-times"></i>
</button>
</div>
</div>
</div>
</div>
</template>
<script setup lang="ts">
import { ref, onMounted } from 'vue'
interface Props {
message: string
duration?: number
}
const props = withDefaults(defineProps<Props>(), {
duration: 5000
})
const emit = defineEmits<{
close: []
}>()
const show = ref(false)
const close = () => {
show.value = false
emit('close')
}
onMounted(() => {
show.value = true
if (props.duration > 0) {
setTimeout(() => {
close()
}, props.duration)
}
})
</script>

View File

@@ -0,0 +1 @@
<template>

View File

@@ -0,0 +1,367 @@
<template>
<div class="file-upload-container">
<n-upload
multiple
directory-dnd
:custom-request="customRequest"
:on-before-upload="handleBeforeUpload"
:on-finish="handleUploadFinish"
:on-error="handleUploadError"
:on-remove="handleFileRemove"
:max="5"
ref="uploadRef"
>
<n-upload-dragger>
<div style="margin-bottom: 12px">
<i class="fas fa-cloud-upload-alt text-4xl text-gray-400"></i>
</div>
<n-text style="font-size: 16px">
点击或者拖动文件到该区域来上传
</n-text>
<n-p depth="3" style="margin: 8px 0 0 0">
支持极速上传相同文件将直接返回已上传的文件信息
</n-p>
</n-upload-dragger>
</n-upload>
</div>
</template>
<script setup lang="ts">
import { ref, computed, onMounted } from 'vue'
import { useMessage } from 'naive-ui'
import { useFileApi } from '~/composables/useFileApi'
interface FileItem {
id: number
original_name: string
file_name: string
file_path: string
file_size: number
file_type: string
mime_type: string
file_hash: string
access_url: string
user_id: number
user: string
status: string
is_public: boolean
is_deleted: boolean
created_at: string
updated_at: string
}
interface UploadFileInfo {
id: string
name: string
status: 'pending' | 'uploading' | 'finished' | 'error' | 'removed'
url?: string
file?: File
}
const message = useMessage()
const fileApi = useFileApi()
// 响应式数据
const uploadRef = ref()
const fileList = ref<FileItem[]>([])
const isPublic = ref(true) // 默认公开
const maxFiles = ref(10)
const maxFileSize = ref(5 * 1024 * 1024) // 5MB
const acceptTypes = ref('image/*')
// 添加状态标记:用于跟踪已上传的文件
const uploadedFiles = ref<Map<string, boolean>>(new Map()) // 文件哈希 -> 是否已上传
const uploadingFiles = ref<Set<string>>(new Set()) // 正在上传的文件哈希
// 计算文件SHA256哈希值
const calculateFileHash = async (file: File): Promise<string> => {
return new Promise((resolve, reject) => {
const reader = new FileReader()
reader.onload = (e) => {
try {
const arrayBuffer = e.target?.result as ArrayBuffer
crypto.subtle.digest('SHA-256', arrayBuffer).then(hashBuffer => {
const hashArray = Array.from(new Uint8Array(hashBuffer))
const hashHex = hashArray.map(b => b.toString(16).padStart(2, '0')).join('')
resolve(hashHex)
}).catch(reject)
} catch (error) {
reject(error)
}
}
reader.onerror = reject
reader.readAsArrayBuffer(file)
})
}
// 生成文件哈希值(基于文件名、大小和修改时间,用于前端去重)
const generateFileHash = (file: File): string => {
return `${file.name}_${file.size}_${file.lastModified}`
}
// 检查文件是否已经上传过
const isFileAlreadyUploaded = (file: File): boolean => {
const fileHash = generateFileHash(file)
return uploadedFiles.value.has(fileHash)
}
// 检查文件是否正在上传
const isFileUploading = (file: File): boolean => {
const fileHash = generateFileHash(file)
return uploadingFiles.value.has(fileHash)
}
// 标记文件为已上传
const markFileAsUploaded = (file: File) => {
const fileHash = generateFileHash(file)
uploadedFiles.value.set(fileHash, true)
uploadingFiles.value.delete(fileHash)
}
// 标记文件为正在上传
const markFileAsUploading = (file: File) => {
const fileHash = generateFileHash(file)
uploadingFiles.value.add(fileHash)
}
// 标记文件上传失败
const markFileAsFailed = (file: File) => {
const fileHash = generateFileHash(file)
uploadingFiles.value.delete(fileHash)
}
// 自定义上传请求
const customRequest = async (options: any) => {
const { file, onProgress, onSuccess, onError } = options
// 检查文件是否已经上传过
if (isFileAlreadyUploaded(file.file)) {
message.warning(`${file.name} 已经上传过了,跳过重复上传`)
if (onSuccess) {
onSuccess({ message: '文件已存在,跳过上传' })
}
return
}
// 检查文件是否正在上传
if (isFileUploading(file.file)) {
message.warning(`${file.name} 正在上传中,请稍候`)
return
}
// 标记文件为正在上传
markFileAsUploading(file.file)
console.log('开始上传文件:', file.name, file.file)
try {
// 计算文件哈希值
const fileHash = await calculateFileHash(file.file)
console.log('文件哈希值:', fileHash)
// 创建FormData
const formData = new FormData()
formData.append('file', file.file)
formData.append('is_public', isPublic.value.toString())
formData.append('file_hash', fileHash)
// 调用统一的API接口
const response = await fileApi.uploadFile(formData)
console.log('文件上传成功:', file.name, response)
// 标记文件为已上传
markFileAsUploaded(file.file)
// 检查是否为重复文件
if (response.data && response.data.is_duplicate) {
message.success(`${file.name} 极速上传成功(文件已存在)`)
} else {
message.success(`${file.name} 上传成功`)
}
if (onSuccess) {
onSuccess(response)
}
} catch (error) {
console.error('文件上传失败:', file.name, error)
// 标记文件上传失败
markFileAsFailed(file.file)
if (onError) {
onError(error)
}
}
}
// 默认文件列表从props传入
const defaultFileList = ref<UploadFileInfo[]>([])
// 方法
const handleBeforeUpload = (data: { file: Required<UploadFileInfo> }) => {
const { file } = data
// 检查文件是否已经上传过
if (file.file && isFileAlreadyUploaded(file.file)) {
//message.warning(`${file.name} 已经上传过了,请勿重复上传`)
return false
}
// 检查文件是否正在上传
if (file.file && isFileUploading(file.file)) {
message.warning(`${file.name} 正在上传中,请稍候`)
return false
}
// 检查文件大小
if (file.file && file.file.size > maxFileSize.value) {
message.error(`文件大小不能超过 ${formatFileSize(maxFileSize.value)}`)
return false
}
// 检查文件类型
if (file.file) {
const fileName = file.file.name.toLowerCase()
const acceptedTypes = acceptTypes.value.split(',')
const isAccepted = acceptedTypes.some(type => {
if (type === 'image/*') {
return file.file!.type.startsWith('image/')
}
if (type.startsWith('.')) {
return fileName.endsWith(type)
}
return file.file!.type === type
})
if (!isAccepted) {
message.error('只支持图片格式文件')
return false
}
}
return true
}
const handleUploadFinish = (data: { file: Required<UploadFileInfo> }) => {
const { file } = data
if (file.status === 'finished') {
// 确保文件被标记为已上传
if (file.file) {
markFileAsUploaded(file.file)
}
}
}
const handleUploadError = (data: { file: Required<UploadFileInfo> }) => {
const { file } = data
message.error(`${file.name} 上传失败`)
// 标记文件上传失败
if (file.file) {
markFileAsFailed(file.file)
}
}
const handleFileRemove = (data: { file: Required<UploadFileInfo> }) => {
const { file } = data
message.info(`已移除 ${file.name}`)
// 从上传状态中移除文件
if (file.file) {
const fileHash = generateFileHash(file.file)
uploadingFiles.value.delete(fileHash)
}
}
const loadFileList = async () => {
try {
const response = await fileApi.getFileList({
page: 1,
page_size: 50
})
fileList.value = response.data.files || []
} catch (error) {
console.error('加载文件列表失败:', error)
message.error('加载文件列表失败')
}
}
const copyFileUrl = async (file: FileItem) => {
try {
await navigator.clipboard.writeText(file.access_url)
message.success('文件链接已复制到剪贴板')
} catch (error) {
console.error('复制失败:', error)
message.error('复制失败')
}
}
const openFile = (file: FileItem) => {
window.open(file.access_url, '_blank')
}
const deleteFile = async (file: FileItem) => {
try {
await fileApi.deleteFiles([file.id])
message.success('文件删除成功')
loadFileList()
} catch (error) {
console.error('删除文件失败:', error)
message.error('删除文件失败')
}
}
const getFileIconClass = (fileType: string) => {
const iconMap: Record<string, string> = {
'image': 'fas fa-image text-blue-500',
'document': 'fas fa-file-alt text-green-500',
'video': 'fas fa-video text-red-500',
'audio': 'fas fa-music text-purple-500',
'archive': 'fas fa-archive text-orange-500',
'other': 'fas fa-file text-gray-500'
}
return iconMap[fileType] || iconMap.other
}
const getFileDescription = (file: FileItem) => {
return `${formatFileSize(file.file_size)} | ${file.file_type} | ${file.created_at}`
}
const formatFileSize = (bytes: number) => {
if (bytes === 0) return '0 B'
const k = 1024
const sizes = ['B', 'KB', 'MB', 'GB', 'TB']
const i = Math.floor(Math.log(bytes) / Math.log(k))
return parseFloat((bytes / Math.pow(k, i)).toFixed(2)) + ' ' + sizes[i]
}
// 生命周期
// 不在组件挂载时加载文件列表,由父组件管理
// 重置上传组件状态
const resetUpload = () => {
if (uploadRef.value) {
uploadRef.value.clear()
}
// 清空上传状态
uploadedFiles.value.clear()
uploadingFiles.value.clear()
}
// 清空已上传文件状态(用于重新开始上传)
const clearUploadedFiles = () => {
uploadedFiles.value.clear()
uploadingFiles.value.clear()
}
// 暴露方法给父组件
defineExpose({
loadFileList,
fileList,
resetUpload,
clearUploadedFiles,
uploadedFiles,
uploadingFiles
})
</script>
<style scoped>
</style>

View File

@@ -0,0 +1,121 @@
<template>
<div class="forbidden-page">
<div class="top-bar-guidance">
<p class="top-bar-guidance-text">请按提示在手机 浏览器 打开<img src="/assets/images/3dian.png" class="icon-safari"></p>
<p class="top-bar-guidance-text">苹果设备<img src="/assets/images/iphone.png" class="icon-safari"></p>
<p class="top-bar-guidance-text">安卓设备<img src="/assets/images/android.png" class="icon-safari"></p>
</div>
<div id="contens">
<p><br/><br/></p>
<p>1.本站不支持 微信,QQ等APP 内访问</p>
<p><br/></p>
<p>2.请按提示在手机 浏览器 打开</p>
<p id="device-tip"><br/>3.请在浏览器中打开</p>
</div>
<p><br/><br/></p>
<div class="app-download-tip">
<span class="guidance-desc" id="current-url"></span>
</div>
<p><br/></p>
<div class="app-download-tip">
<span class="guidance-desc">点击右上角···图标 or 复制网址自行打开</span>
</div>
</div>
</template>
<script setup lang="ts">
import { onMounted } from 'vue'
// 组件属性
interface Props {
currentUrl?: string
isIOS?: boolean
}
const props = withDefaults(defineProps<Props>(), {
currentUrl: '',
isIOS: false
})
// 在组件挂载时设置内容
onMounted(() => {
const currentUrlElement = document.getElementById('current-url')
const deviceTipElement = document.getElementById('device-tip')
if (currentUrlElement) {
currentUrlElement.textContent = props.currentUrl || window.location.href
}
if (deviceTipElement) {
const deviceText = props.isIOS ? '苹果设备请在Safari浏览器中打开' : '安卓设备请在Chrome或其他浏览器中打开'
deviceTipElement.innerHTML = `<br/>3.${deviceText}`
}
})
</script>
<style scoped>
.forbidden-page {
min-height: 100vh;
}
.top-bar-guidance {
font-size: 15px;
color: #fff;
height: 70%;
line-height: 1.2;
padding-left: 20px;
padding-top: 20px;
background: url('/assets/images/banner.png') center top/cover no-repeat;
}
.top-bar-guidance p {
white-space: nowrap;
overflow: hidden;
text-overflow: ellipsis;
}
.top-bar-guidance .icon-safari {
width: 25px;
height: 25px;
vertical-align: middle;
margin: 0 .2em;
}
.top-bar-guidance-text {
display: flex;
justify-items: center;
word-wrap: nowrap;
}
.top-bar-guidance-text img {
display: inline-block;
width: 25px;
height: 25px;
vertical-align: middle;
margin: 0 .2em;
}
#contens {
font-weight: bold;
color: #2466f4;
text-align: center;
font-size: 20px;
margin-bottom: 125px;
}
.app-download-tip {
margin: 0 auto;
width: 290px;
text-align: center;
font-size: 15px;
color: #2466f4;
background: url(data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAAEAAAAcAQMAAACak0ePAAAABlBMVEUAAAAdYfh+GakkAAAAAXRSTlMAQObYZgAAAA5JREFUCNdjwA8acEkAAAy4AIE4hQq/AAAAAElFTkSuQmCC) left center/auto 15px repeat-x;
}
.app-download-tip .guidance-desc {
background-color: #fff;
padding: 0 5px;
}
</style>

View File

@@ -0,0 +1,51 @@
<template>
<div class="loading-container">
<div class="loading-spinner"></div>
<p class="loading-text">{{ message }}</p>
</div>
</template>
<script setup lang="ts">
// 组件属性
interface Props {
message?: string
}
const props = withDefaults(defineProps<Props>(), {
message: '正在检测访问环境...'
})
</script>
<style scoped>
.loading-container {
display: flex;
flex-direction: column;
align-items: center;
justify-content: center;
height: 100vh;
background: url('/assets/images/banner.webp') center / cover no-repeat;
}
.loading-spinner {
width: 50px;
height: 50px;
border: 4px solid rgba(255, 255, 255, 0.3);
border-top: 4px solid #ffffff;
border-radius: 50%;
animation: spin 1s linear infinite;
margin-bottom: 20px;
}
.loading-text {
color: #ffffff;
font-size: 16px;
font-weight: 500;
text-align: center;
margin: 0;
}
@keyframes spin {
0% { transform: rotate(0deg); }
100% { transform: rotate(360deg); }
}
</style>

View File

@@ -0,0 +1,22 @@
<template>
<n-image
:src="proxyUrl"
v-bind="$attrs"
/>
</template>
<script setup lang="ts">
import { computed } from 'vue'
import { useImageUrl } from '~/composables/useImageUrl'
interface Props {
src: string
}
const props = defineProps<Props>()
const { getImageUrl } = useImageUrl()
const proxyUrl = computed(() => {
return getImageUrl(props.src)
})
</script>

Some files were not shown because too many files have changed in this diff Show More