Compare commits

...

157 Commits

Author SHA1 Message Date
Junyan Qin
31a7503df3 chore: release v3.4.0 2024-11-17 20:48:45 +08:00
Junyan Qin
11db8d8d17 Merge pull request #904 from RockChinQ/version/3.4.0
Version/3.4.0
2024-11-17 20:47:25 +08:00
Junyan Qin
93ee8d51bc Merge branch 'master' into version/3.4.0 2024-11-17 20:45:24 +08:00
Junyan Qin
83e80f324e perf: webui 文件存在性检查 2024-11-17 20:43:40 +08:00
Junyan Qin
c51eac717e doc: 修复 README 的死链 2024-11-17 20:37:42 +08:00
Junyan Qin
db7d5dcce3 chore: 替换多处 qchatgpt.rockchin.top 2024-11-17 20:35:39 +08:00
Junyan Qin
0d25578e22 perf: 配置文件页放到单独组件 2024-11-17 19:54:07 +08:00
Junyan Qin
1a457be823 Merge pull request #921 from RockChinQ/feat/authenticating
Feat: 用户鉴权
2024-11-17 19:13:11 +08:00
Junyan Qin
20e3edba8f feat: 用户账户系统 2024-11-17 19:11:44 +08:00
Junyan Qin
036c2182a5 chore: 修改aiocqhttp适配器默认端口为2280 2024-11-17 10:18:56 +08:00
Junyan Qin
6238f430e8 Merge pull request #900 from RockChinQ/feat/webui
Feat: webui
2024-11-16 19:27:35 +08:00
Junyan Qin
9fc891ec01 chore: Hello LangBot ! 2024-11-16 17:57:39 +08:00
Junyan Qin
491d977d9e ci: fix 2024-11-16 17:47:50 +08:00
Junyan Qin
9a4bcda9bc ci: 添加 build-artifacts 工作流在 release 分布时执行 2024-11-16 17:44:02 +08:00
Junyan Qin
2c2374a763 ci: fix 2024-11-16 17:39:34 +08:00
Junyan Qin
a76e0b287e ci: typo 2024-11-16 17:36:33 +08:00
Junyan Qin
1d6f1e3c7c fix: chore 2024-11-16 17:34:15 +08:00
Junyan Qin
896fd982a1 ci: upload artifacts 2024-11-16 17:33:24 +08:00
Junyan Qin
c031ab20da Merge pull request #920 from RockChinQ/feat/lifetime-controlling
Feat: 生命周期和热重载
2024-11-16 17:19:42 +08:00
Junyan Qin
318b6e6bf1 typo: minor fix 2024-11-16 17:16:40 +08:00
Junyan Qin
ca3999d251 feat: 插件文件更改热重载 2024-11-16 16:45:13 +08:00
Junyan Qin
658eb278c4 refactor: 重构部分插件管理逻辑 2024-11-16 16:13:02 +08:00
Junyan Qin
bb219889e5 feat: 消息平台热重载 2024-11-16 12:40:57 +08:00
Junyan Qin
3239c9ec3f feat: 彻底移除 yirimirai 2024-11-15 20:03:49 +08:00
Junyan Qin
16153dc573 perf: 设置页标题改为小写 2024-11-12 20:04:03 +08:00
Junyan Qin
e0d9a295ab perf: 优化部分 UI 显示 2024-11-12 18:57:43 +08:00
Junyan Qin
eabdda5eb1 feat: 改为 WebHashHistory 2024-11-12 18:29:37 +08:00
Junyan Qin
43f45f9184 feat: 修改 apibase 2024-11-12 18:14:53 +08:00
Junyan Qin
7c19785a17 fix: http_proxy 环境变量为空检查 2024-11-12 17:56:59 +08:00
Junyan Qin
78005f8b4e ci: 删除 refs-heads 2024-11-12 17:49:00 +08:00
Junyan Qin
0d4784d098 feat: 代理设置同步到环境变量 2024-11-11 19:12:30 +08:00
Junyan Qin
805454e037 chore: 更新 docker-compose.yaml 2024-11-10 16:37:44 +08:00
Junyan Qin
bf383bbf9c ci: fix 2024-11-10 16:29:40 +08:00
Junyan Qin
73ffd67792 ci: 添加构建 ci 2024-11-10 16:27:50 +08:00
Junyan Qin
54bbfc8eda perf: dashboard 添加图表更新提示 2024-11-10 15:38:06 +08:00
Junyan Qin
a3e234c979 perf: debug模式改为绿色 2024-11-10 12:03:34 +08:00
Junyan Qin
9336abff8b perf: 使用 json-editor-vue 作为json编辑器 2024-11-10 11:46:41 +08:00
Junyan Qin
0fe161cd7f pref: 优化日志显示 2024-11-10 11:04:29 +08:00
Junyan Qin
7cc55eab3e feat: 仪表盘基本数据 2024-11-10 00:05:47 +08:00
Junyan Qin
15482e398b feat: 插件删除功能 2024-11-09 23:25:26 +08:00
Junyan Qin
601fa0ac7f feat: 关于 LangBot 对话框 2024-11-09 22:44:56 +08:00
Junyan Qin
2819da5f2f fix: github下载未使用环境变量代理 2024-11-09 18:09:39 +08:00
Junyan Qin
3cb3562477 doc(README): fix deadlinks 2024-11-08 23:14:52 +08:00
Junyan Qin
cee205994f doc: update logo 2024-11-05 15:42:48 +08:00
Junyan Qin
e44df0a3dd feat: dashboard 基本组件 2024-11-04 21:54:02 +08:00
Junyan Qin
84a51cb26d feat: 插件安装支持 2024-11-04 00:01:07 +08:00
Junyan Qin
db02d9c126 feat: 完成任务列表功能 2024-11-03 23:22:33 +08:00
Junyan Qin
709b86b724 refactor: 使插件更新过程全异步 2024-11-03 22:27:31 +08:00
Junyan Qin
68184b0e47 Merge pull request #911 from RockChinQ/feat/trackable-async-tasks
Feat: 用户级任务系统
2024-11-01 22:42:11 +08:00
Junyan Qin
6d2a4c038d feat: 完成异步任务跟踪架构基础 2024-11-01 22:41:26 +08:00
Junyan Qin
2f05f5b456 feat: 添加任务列表框架 2024-10-24 18:28:57 +08:00
Junyan Qin
d5e3120350 chore: 确保 pydantic<2.0 2024-10-24 14:26:18 +08:00
Junyan Qin
a4589327a6 feat: 添加 python 版本检查 2024-10-22 18:17:09 +08:00
Junyan Qin
c151665419 feat: 添加任务管理模块 2024-10-22 18:09:18 +08:00
Junyan Qin
947790e8d1 Update README.md 2024-10-22 13:51:05 +08:00
Junyan Qin
26770439bb fix: 关闭编排对话框时错误的插件顺序 2024-10-21 19:18:40 +08:00
Junyan Qin
7da9171dde feat: 插件优先级更改功能 2024-10-20 22:20:35 +08:00
Junyan Qin
16b386eaf7 feat: 插件页展示功能 2024-10-19 18:38:01 +08:00
Junyan Qin
c330aab48b Merge pull request #902 from RockChinQ/feat/settings-form-render
Feat: 设置项可视化编辑器
2024-10-16 22:32:40 +08:00
Junyan Qin
5f998a0852 perf: settings 页面的一些提示 2024-10-16 22:24:15 +08:00
Junyan Qin
c3dfbb64a6 feat: 异常处理 2024-10-16 21:55:55 +08:00
Junyan Qin
3db52282b8 fix: 修复子字段值为空时导致字段丢失的问题 2024-10-16 16:08:58 +08:00
Junyan Qin
a313ae5f97 feat: 添加多个可视化编辑schema 2024-10-16 15:34:30 +08:00
Junyan Qin
18cce189a4 feat: 完成 pipeline 的可视化配置 2024-10-16 13:57:41 +08:00
Junyan Qin
fb308d576b fix(settings): 切换tab时的异步问题 2024-10-16 12:58:52 +08:00
Junyan Qin
8c976303a4 feat: system.json 的可视化编辑 2024-10-15 21:42:05 +08:00
Junyan Qin
12f1f3609d feat: 引入 vjsf 渲染表单 2024-10-15 16:16:39 +08:00
Junyan Qin
661fdeb6a1 perf: 重新切换到 settings tab 时加载之前编辑的内容 2024-10-15 14:28:06 +08:00
Junyan Qin
d52f9b9543 feat(settings): json 编辑器 2024-10-15 14:23:56 +08:00
Junyan Qin
7174742886 feat: settings 基础组件 2024-10-15 00:07:40 +08:00
Junyan Qin
cd0a8fb24b perf: 使内容背景稍微灰一些 2024-10-14 21:30:10 +08:00
Junyan Qin
1fbc92bc6d perf: 首页展示版本信息 2024-10-14 21:18:36 +08:00
Junyan Qin
231dca956d feat: 日志页面 2024-10-14 18:52:28 +08:00
RockChinQ
0dd74c825b feat: 前端基础框架 2024-10-13 22:34:35 +08:00
RockChinQ
9703fc0366 perf: 优化日志增量获取逻辑 2024-10-13 22:33:51 +08:00
RockChinQ
7c3557e943 feat: 持久化和 web 接口基础架构 2024-10-11 22:27:53 +08:00
RockChinQ
21f153e5c3 chore: webui 前端模板 2024-10-11 22:23:08 +08:00
Junyan Qin
ea6a0af5a7 Merge pull request #890 from RockChinQ/feat/more-platforms
Refactor: 移除 YiriMirai 组件
2024-09-26 14:41:03 +08:00
RockChinQ
c53ffaca6c fix: 处理插件 import mirai 时的兼容性问题 2024-09-26 14:38:18 +08:00
RockChinQ
3469515e04 feat: 删除代码中对 mirai 的引用 2024-09-26 13:01:45 +08:00
RockChinQ
e8da26cb8a fix: missing break 2024-09-26 11:23:37 +08:00
RockChinQ
1235fc1339 chore: release v3.3.1.1 2024-09-26 10:39:35 +08:00
Junyan Qin
47e308b99d Merge pull request #889 from YunZLu/add-check-role
Fix: Add Role Check to Prevent Validation Error
2024-09-26 09:31:25 +08:00
RockChinQ
fdba470e9a perf: 将 platform 的 组件导入包 __init__ 中 2024-09-26 00:28:57 +08:00
Junyan Qin
a1ccceefd2 Merge branch 'master' into feat/more-platforms 2024-09-26 00:26:17 +08:00
RockChinQ
1c4a700d92 refactor: 将 yirimirai 的组件集成进 platform 包 2024-09-26 00:23:03 +08:00
YunZL
81c2c3c0e5 Add Role Check to Prevent Validation Error 2024-09-23 23:25:54 +08:00
Junyan Qin
3c2db5097a Merge pull request #888 from Tigrex-Dai/master
fix: 添加了针对报错内容对event.sender中'role'的存在性检查
2024-09-22 16:50:55 +08:00
Tigrex Dai
ce56f79687 Update aiocqhttp.py
针对报错对"role"做存在性检查
2024-09-22 15:39:48 +08:00
RockChinQ
ee0d6dcdae chore: release v3.3.1.0 2024-09-08 15:14:24 +08:00
Junyan Qin
bcf1d92f73 Merge pull request #881 from RockChinQ/version/3.3.1.0
Version/3.3.1.0
2024-09-08 15:13:39 +08:00
RockChinQ
ffdec16ce6 docs: wiki 所有页面加上已弃用说明 2024-09-08 14:52:35 +08:00
RockChinQ
b2f6e84adc typo: 优化插件执行日志信息 2024-09-08 14:51:39 +08:00
Junyan Qin
f76c457e1f Update README.md 2024-09-03 20:07:41 +08:00
RockChinQ
80bd0a20df doc: 修复 README 中的logo图片 2024-08-30 14:48:23 +08:00
RockChinQ
efeaf73339 doc: 修改README图片链接 2024-08-30 11:13:04 +08:00
Junyan Qin
91b5100a24 Merge pull request #872 from RockChinQ/feat/config-file-api
Feat: 添加yaml配置文件的支持
2024-08-24 20:55:19 +08:00
RockChinQ
d1a06f4730 feat: 添加yaml配置文件的支持 2024-08-24 20:54:36 +08:00
Junyan Qin
b0b186e951 Merge pull request #871 from RockChinQ/feat/qq-c2c
Feat: 添加对 QQ 官方 API 私聊场景的支持
2024-08-24 17:04:41 +08:00
RockChinQ
4c8fedef6e feat: QQ官方api群聊和私聊支持图片 2024-08-24 17:01:35 +08:00
RockChinQ
718c221d01 feat: 支持官方机器人私信接口 2024-08-24 16:26:47 +08:00
Junyan Qin
077e77eee5 Merge pull request #869 from ligen131/lg/fix_image_format
fix: 发送正确的图片格式而不是默认的 `image/jpeg`
2024-08-24 15:47:55 +08:00
ligen131
b51ca06c7c fix: 发送正确的图片格式而不是默认的 image/jpeg 2024-08-19 00:00:29 +08:00
RockChinQ
2f092f4a87 chore: release v3.3.0.2 2024-08-01 23:14:07 +08:00
Junyan Qin
f1ff9c05c4 Merge pull request #864 from RockChinQ/version/3.3.0.2
fix: 消息忽略规则失效 (#854)
2024-08-01 23:12:33 +08:00
RockChinQ
c9c8603ccc fix: 消息忽略规则失效 (#854) 2024-08-01 23:01:28 +08:00
RockChinQ
47e281fb61 chore: release v3.3.0.1 2024-07-28 22:47:49 +08:00
RockChinQ
dc625647eb fix: ollama 依赖检查 2024-07-28 22:47:19 +08:00
RockChinQ
66cf1b05be chore: 优化issue和pr模板 2024-07-28 21:32:22 +08:00
RockChinQ
622cc89414 chore: release v3.3.0 2024-07-28 20:58:29 +08:00
Junyan Qin
78d98c40b1 Merge pull request #847 from RockChinQ/version/3.3
Release: 3.3
2024-07-28 20:57:26 +08:00
RockChinQ
1c5f06d9a9 feat: 添加 reply 和 send_message 两个插件api方法 2024-07-28 20:23:52 +08:00
Junyan Qin
998fe5a980 Merge pull request #857 from RockChinQ/feat/runner-abstraction
Feat: Runner 组件抽象
2024-07-28 18:47:38 +08:00
RockChinQ
8cad4089a7 feat: runner 层抽象 (#839) 2024-07-28 18:45:27 +08:00
RockChinQ
48cc3656bd feat: 允许自定义命令前缀 2024-07-28 16:01:58 +08:00
RockChinQ
68ddb3a6e1 feat: 添加 model 命令 2024-07-28 15:46:09 +08:00
ElvisChenML
70583f5ba0 Fixed aiocqhttp mirai.Voice类型无法正确传递url及base64的异常 2024-07-28 15:08:33 +08:00
Junyan Qin
5bebe01dd0 Update README.md 2024-07-28 15:08:33 +08:00
Junyan Qin
4dd976c9c5 Merge pull request #856 from ElvisChenML/pr
Fixed aiocqhttp mirai.Voice类型无法正确传递url及base64的异常
2024-07-28 13:05:06 +08:00
ElvisChenML
221b310485 Fixed aiocqhttp mirai.Voice类型无法正确传递url及base64的异常 2024-07-25 16:14:24 +08:00
Junyan Qin
dd1cec70c0 Update README.md 2024-07-13 09:15:18 +08:00
Junyan Qin
7656443b28 Merge pull request #845 from ElvisChenML/pr
fixed pkg\provider\entities.py\get_content_mirai_message_chain中ce.type图片类型不正确的异常
2024-07-10 00:13:48 +08:00
Junyan Qin
9d91c13b12 Merge pull request #844 from canyuan0801/pr
Feat: Ollama平台集成
2024-07-10 00:09:48 +08:00
RockChinQ
7c06141ce2 perf(ollama): 优化命令显示细节 2024-07-10 00:07:32 +08:00
RockChinQ
3dc413638b feat(ollama): 配置文件迁移 2024-07-09 23:37:34 +08:00
RockChinQ
bdb8baeddd perf(ollama): 修改请求器名称以适配请求路径 2024-07-09 23:37:19 +08:00
ElvisChenML
21966bfb69 fixed pkg\provider\entities.py\get_content_mirai_message_chain中ce.type图片类型不正确的异常 2024-07-09 17:04:11 +08:00
canyuan
e78c82e999 mod: merge ollama cmd 2024-07-09 16:19:09 +08:00
canyuan
2bdc3468d1 add ollama cmd 2024-07-09 14:57:39 +08:00
canyuan
987b3dc4ef add ollama chat 2024-07-09 14:57:28 +08:00
RockChinQ
45a10b4ac7 chore: release v3.2.4 2024-07-05 18:19:10 +08:00
RockChinQ
b5d33ef629 perf: 优化 pipeline 处理时的报错 2024-07-04 13:03:58 +08:00
RockChinQ
d3629916bf fix: user_notice 处理时为对齐为 MessageChain (#809) 2024-07-04 12:47:55 +08:00
RockChinQ
c5cb26d295 fix: GroupNormalMessageReceived事件设置 alter 无效 (#803) 2024-07-03 23:16:16 +08:00
RockChinQ
4b2785c5eb fix: QQ 官方 API 图片识别功能不正常 (#825) 2024-07-03 22:36:35 +08:00
RockChinQ
7ed190e6d2 doc: 删除广告 2024-07-03 17:50:58 +08:00
Junyan Qin
eac041cdd2 Merge pull request #834 from RockChinQ/feat/env-reminder
Feat: 添加启动信息阶段
2024-07-03 17:45:56 +08:00
RockChinQ
05527cfc01 feat: 添加 windows 下针对选择模式的提示 2024-07-03 17:44:10 +08:00
RockChinQ
61e2af4a14 feat: 添加启动信息阶段 2024-07-03 17:34:23 +08:00
RockChinQ
79804b6ecd chore: release 3.2.3 2024-06-26 10:55:21 +08:00
Junyan Qin
76434b2f4e Merge pull request #829 from RockChinQ/version/3.2.3
Release 3.2.3
2024-06-26 10:54:42 +08:00
RockChinQ
ec8bd4922e fix: 错误地resprule选择逻辑 (#810) 2024-06-26 10:37:08 +08:00
RockChinQ
4ffa773fac fix: 前缀响应时图片被错误地转换为文字 (#820) 2024-06-26 10:15:21 +08:00
Junyan Qin
ea8b7bc8aa Merge pull request #818 from Huoyuuu/master
fix: ensure content is string in chatcmpl call method
2024-06-24 17:12:46 +08:00
RockChinQ
39ce5646f6 perf: content元素拼接时使用换行符间隔 2024-06-24 17:04:50 +08:00
Huoyuuu
5092a82739 Update chatcmpl.py 2024-06-19 19:13:00 +08:00
Huoyuuu
3bba0b6d9a Merge pull request #1 from Huoyuuu/fix/issue-817-ensure-content-string
fix: ensure content is string in chatcmpl call method
2024-06-19 17:32:30 +08:00
Huoyuuu
7a19dd503d fix: ensure content is string in chatcmpl call method
fix: ensure content is string in chatcmpl call method

- Ensure user message content is a string instead of an array
- Updated `call` method in `chatcmpl.py` to guarantee content is a string
- Resolves compatibility issue with the yi-large model
2024-06-19 17:26:06 +08:00
RockChinQ
9e6a01fefd chore: release v3.2.2 2024-05-31 19:20:34 +08:00
RockChinQ
933471b4d9 perf: 启动失败时输出完整traceback (#799) 2024-05-31 15:37:56 +08:00
RockChinQ
f81808d239 perf: 添加JSON配置文件语法检查 (#796) 2024-05-29 21:11:21 +08:00
RockChinQ
96832b6f7d perf: 忽略空的 assistant content 消息 (#795) 2024-05-29 21:00:48 +08:00
Junyan Qin
e2eb0a84b0 Merge pull request #797 from RockChinQ/feat/context-truncater
Feat: 消息截断器
2024-05-29 20:38:14 +08:00
RockChinQ
c8eb2e3376 feat: 消息截断器 2024-05-29 20:34:49 +08:00
Junyan Qin
21fe5822f9 Merge pull request #794 from RockChinQ/perf/advanced-fixwin
Feat: fixwin限速支持设置窗口大小
2024-05-26 10:33:49 +08:00
RockChinQ
d49cc9a7a3 feat: fixwin限速支持设置窗口大小 (#791) 2024-05-26 10:29:10 +08:00
Junyan Qin
910d0bfae1 Update README.md 2024-05-25 12:27:27 +08:00
203 changed files with 14493 additions and 2242 deletions

View File

@@ -3,61 +3,37 @@ description: 报错或漏洞请使用这个模板创建,不使用此模板创
title: "[Bug]: "
labels: ["bug?"]
body:
- type: dropdown
attributes:
label: 部署方式
description: "主程序使用的部署方式"
options:
- 手动部署
- 安装器部署
- 一键安装包部署
- Docker部署
validations:
required: true
- type: dropdown
attributes:
label: 消息平台适配器
description: "连接QQ使用的框架"
options:
- yiri-miraiMirai
- Nakurugo-cqhttp
- aiocqhttp使用 OneBot 协议接入的)
- qq-botpyQQ官方API
- yiri-miraiMirai
validations:
required: false
- type: input
attributes:
label: 系统环境
description: 操作系统、系统架构、**主机地理位置**,地理位置最好写清楚,涉及网络问题排查。
placeholder: 例如: CentOS x64 中国大陆、Windows11 美国
label: 运行环境
description: 操作系统、系统架构、**Python版本**、**主机地理位置**
placeholder: 例如: CentOS x64 Python 3.10.3、Docker 的直接写 Docker 就行
validations:
required: true
- type: input
attributes:
label: Python环境
description: 运行程序的Python版本
placeholder: 例如: Python 3.10
validations:
required: true
- type: input
attributes:
label: QChatGPT版本
description: QChatGPT版本号
placeholder: 例如: v2.6.0,可以使用`!version`命令查看
label: LangBot 版本
description: LangBot (QChatGPT) 版本
placeholder: 例如:v3.3.0,可以使用`!version`命令查看,或者到 pkg/utils/constants.py 查看
validations:
required: true
- type: textarea
attributes:
label: 异常情况
description: 完整描述异常情况,什么时候发生的、发生了什么,尽可能详细
description: 完整描述异常情况,什么时候发生的、发生了什么。**请附带日志信息。**
validations:
required: true
- type: textarea
attributes:
label: 日志信息
description: 请提供完整的 **登录框架 和 QChatGPT控制台**的相关日志信息(若有),不提供日志信息**无法**为您排查问题,请尽可能详细
validations:
required: false
- type: textarea
attributes:
label: 启用的插件

View File

@@ -10,5 +10,4 @@ updates:
schedule:
interval: "weekly"
allow:
- dependency-name: "yiri-mirai-rc"
- dependency-name: "openai"

View File

@@ -2,24 +2,19 @@
实现/解决/优化的内容:
### 事务
## 检查清单
- [ ] 已阅读仓库[贡献指引](https://github.com/RockChinQ/QChatGPT/blob/master/CONTRIBUTING.md)
- [ ] 已与维护者在issues或其他平台沟通此PR大致内容
### PR 作者完成
## 以下内容可在起草PR后、合并PR前逐步完成
*请在方括号间写`x`以打勾
### 功能
- [ ] 阅读仓库[贡献指引](https://github.com/RockChinQ/LangBot/blob/master/CONTRIBUTING.md)了吗?
- [ ] 与项目所有者沟通过了吗?
- [ ] 我确定已自行测试所作的更改,确保功能符合预期。
- [ ] 已编写完善的配置文件字段说明(若有新增)
- [ ] 已编写面向用户的新功能说明(若有必要)
- [ ] 已测试新功能或更改
### 项目所有者完成
### 兼容性
- [ ] 已处理版本兼容性
- [ ] 已处理插件兼容问题
### 风险
可能导致或已知的问题:
- [ ] 相关 issues 链接了吗?
- [ ] 配置项写好了吗?迁移写好了吗?生效了吗?
- [ ] 依赖写到 requirements.txt 和 core/bootutils/deps.py 了吗
- [ ] 文档编写了吗?

24
.github/workflows/build-dev-image.yaml vendored Normal file
View File

@@ -0,0 +1,24 @@
name: Build Dev Image
on:
push:
workflow_dispatch:
jobs:
build-dev-image:
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v2
- name: Generate Tag
id: generate_tag
run: |
# 获取分支名称,把/替换为-
echo ${{ github.ref }} | sed 's/refs\/heads\///g' | sed 's/\//-/g'
echo ::set-output name=tag::$(echo ${{ github.ref }} | sed 's/refs\/heads\///g' | sed 's/\//-/g')
- name: Login to Registry
run: docker login --username=${{ secrets.DOCKER_USERNAME }} --password ${{ secrets.DOCKER_PASSWORD }}
- name: Build Docker Image
run: |
docker buildx create --name mybuilder --use
docker build -t rockchin/langbot:${{ steps.generate_tag.outputs.tag }} . --push

View File

@@ -19,12 +19,6 @@ jobs:
export GITHUB_REF=${{ github.ref }}
echo $GITHUB_REF
fi
# - name: Check GITHUB_REF env
# run: echo $GITHUB_REF
# - name: Get version # 在 GitHub Actions 运行环境
# id: get_version
# if: (startsWith(env.GITHUB_REF, 'refs/tags/')||startsWith(github.ref, 'refs/tags/')) && startsWith(github.repository, 'RockChinQ/QChatGPT')
# run: export GITHUB_REF=${GITHUB_REF/refs\/tags\//}
- name: Check version
id: check_version
run: |
@@ -44,5 +38,5 @@ jobs:
run: docker login --username=${{ secrets.DOCKER_USERNAME }} --password ${{ secrets.DOCKER_PASSWORD }}
- name: Create Buildx
run: docker buildx create --name mybuilder --use
- name: Build # image name: rockchin/qchatgpt:<VERSION>
run: docker buildx build --platform linux/arm64,linux/amd64 -t rockchin/qchatgpt:${{ steps.check_version.outputs.version }} -t rockchin/qchatgpt:latest . --push
- name: Build # image name: rockchin/langbot:<VERSION>
run: docker buildx build --platform linux/arm64,linux/amd64 -t rockchin/langbot:${{ steps.check_version.outputs.version }} -t rockchin/langbot:latest . --push

View File

@@ -0,0 +1,52 @@
name: Build Release Artifacts
on:
workflow_dispatch:
## 发布release的时候会自动构建
release:
types: [published]
jobs:
build-artifacts:
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v2
- name: Check version
id: check_version
run: |
echo $GITHUB_REF
# 如果是tag则去掉refs/tags/前缀
if [[ $GITHUB_REF == refs/tags/* ]]; then
echo "It's a tag"
echo $GITHUB_REF
echo $GITHUB_REF | awk -F '/' '{print $3}'
echo ::set-output name=version::$(echo $GITHUB_REF | awk -F '/' '{print $3}')
else
echo "It's not a tag"
echo $GITHUB_REF
echo ::set-output name=version::${GITHUB_REF}
fi
- name: Make Temp Directory
run: |
mkdir -p /tmp/langbot_build_web
cp -r . /tmp/langbot_build_web
- name: Setup Node
uses: actions/setup-node@v2
with:
node-version: '22'
- name: Build Web
run: |
cd /tmp/langbot_build_web/web
npm install
npm run build
- name: Package Output
run: |
cp -r /tmp/langbot_build_web/web/dist ./web
- name: Upload Artifact
uses: actions/upload-artifact@v4
with:
name: langbot-${{ steps.check_version.outputs.version }}-all
path: .

View File

@@ -1,43 +0,0 @@
name: Update Wiki
on:
push:
branches:
- master
paths:
- 'res/wiki/**'
jobs:
update-wiki:
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v2
- name: Setup Git
run: |
git config --global user.name "GitHub Actions"
git config --global user.email "github-actions[bot]@users.noreply.github.com"
- name: Clone Wiki Repository
uses: actions/checkout@v2
with:
repository: RockChinQ/QChatGPT.wiki
path: wiki
- name: Delete old wiki content
run: |
rm -rf wiki/*
- name: Copy res/wiki content to wiki
run: |
cp -r res/wiki/* wiki/
- name: Check for changes
run: |
cd wiki
if git diff --quiet; then
echo "No changes to commit."
exit 0
fi
- name: Commit and Push Changes
run: |
cd wiki
git add .
git commit -m "Update wiki"
git push

8
.gitignore vendored
View File

@@ -3,9 +3,10 @@
__pycache__/
database.db
qchatgpt.log
langbot.log
/banlist.py
plugins/
!plugins/__init__.py
/plugins/
!/plugins/__init__.py
/revcfg.py
prompts/
logs/
@@ -34,4 +35,5 @@ bard.json
res/instance_id.json
.DS_Store
/data
botpy.log*
botpy.log*
/poc

View File

@@ -1,8 +1,19 @@
FROM node:22-alpine AS node
WORKDIR /app
COPY web ./web
RUN cd web && npm install && npm run build
FROM python:3.10.13-slim
WORKDIR /app
COPY . .
COPY --from=node /app/web/dist ./web/dist
RUN apt update \
&& apt install gcc -y \
&& python -m pip install -r requirements.txt \

View File

@@ -1,25 +1,25 @@
<p align="center">
<img src="https://qchatgpt.rockchin.top/logo.png" alt="QChatGPT" width="180" />
<img src="https://docs.langbot.app/chrome-512.png" alt="QChatGPT" width="180" />
</p>
<div align="center">
# QChatGPT
# LangBot
<a href="https://trendshift.io/repositories/6187" target="_blank"><img src="https://trendshift.io/api/badge/repositories/6187" alt="RockChinQ%2FQChatGPT | Trendshift" style="width: 250px; height: 55px;" width="250" height="55"/></a>
[![GitHub release (latest by date)](https://img.shields.io/github/v/release/RockChinQ/QChatGPT)](https://github.com/RockChinQ/QChatGPT/releases/latest)
<a href="https://hub.docker.com/repository/docker/rockchin/qchatgpt">
<img src="https://img.shields.io/docker/pulls/rockchin/qchatgpt?color=blue" alt="docker pull">
[![GitHub release (latest by date)](https://img.shields.io/github/v/release/RockChinQ/LangBot)](https://github.com/RockChinQ/LangBot/releases/latest)
<a href="https://hub.docker.com/repository/docker/rockchin/langbot">
<img src="https://img.shields.io/docker/pulls/rockchin/langbot?color=blue" alt="docker pull">
</a>
![Dynamic JSON Badge](https://img.shields.io/badge/dynamic/json?url=https%3A%2F%2Fapi.qchatgpt.rockchin.top%2Fapi%2Fv2%2Fview%2Frealtime%2Fcount_query%3Fminute%3D10080&query=%24.data.count&label=%E4%BD%BF%E7%94%A8%E9%87%8F%EF%BC%887%E6%97%A5%EF%BC%89)
![Dynamic JSON Badge](https://img.shields.io/badge/dynamic/json?url=https%3A%2F%2Fapi.docs.langbot.app%2Fapi%2Fv2%2Fview%2Frealtime%2Fcount_query%3Fminute%3D10080&query=%24.data.count&label=%E4%BD%BF%E7%94%A8%E9%87%8F%EF%BC%887%E6%97%A5%EF%BC%89)
![Wakapi Count](https://wakapi.rockchin.top/api/badge/RockChinQ/interval:any/project:QChatGPT)
<br/>
<img src="https://img.shields.io/badge/python-3.10 | 3.11 | 3.12-blue.svg" alt="python">
<a href="http://qm.qq.com/cgi-bin/qm/qr?_wv=1027&k=66-aWvn8cbP4c1ut_1YYkvvGVeEtyTH8&authKey=pTaKBK5C%2B8dFzQ4XlENf6MHTCLaHnlKcCRx7c14EeVVlpX2nRSaS8lJm8YeM4mCU&noverify=0&group_code=195992197">
<img alt="Static Badge" src="https://img.shields.io/badge/%E5%AE%98%E6%96%B9%E7%BE%A4-195992197-purple">
</a>
<a href="https://qm.qq.com/q/1yxEaIgXMA">
<a href="https://qm.qq.com/q/PClALFK242">
<img alt="Static Badge" src="https://img.shields.io/badge/%E7%A4%BE%E5%8C%BA%E7%BE%A4-619154800-purple">
</a>
<a href="https://codecov.io/gh/RockChinQ/QChatGPT" >
@@ -28,12 +28,12 @@
## 使用文档
<a href="https://qchatgpt.rockchin.top">项目主页</a>
<a href="https://qchatgpt.rockchin.top/posts/feature.html">功能介绍</a>
<a href="https://qchatgpt.rockchin.top/posts/deploy/">部署文档</a>
<a href="https://qchatgpt.rockchin.top/posts/error/">常见问题</a>
<a href="https://qchatgpt.rockchin.top/posts/plugin/intro.html">插件介绍</a>
<a href="https://github.com/RockChinQ/QChatGPT/issues/new?assignees=&labels=%E7%8B%AC%E7%AB%8B%E6%8F%92%E4%BB%B6&projects=&template=submit-plugin.yml&title=%5BPlugin%5D%3A+%E8%AF%B7%E6%B1%82%E7%99%BB%E8%AE%B0%E6%96%B0%E6%8F%92%E4%BB%B6">提交插件</a>
<a href="https://docs.langbot.app">项目主页</a>
<a href="https://docs.langbot.app/insight/intro.htmll">功能介绍</a>
<a href="https://docs.langbot.app/insight/guide.html">部署文档</a>
<a href="https://docs.langbot.app/usage/faq.html">常见问题</a>
<a href="https://docs.langbot.app/plugin/plugin-intro.html">插件介绍</a>
<a href="https://github.com/RockChinQ/LangBot/issues/new?assignees=&labels=%E7%8B%AC%E7%AB%8B%E6%8F%92%E4%BB%B6&projects=&template=submit-plugin.yml&title=%5BPlugin%5D%3A+%E8%AF%B7%E6%B1%82%E7%99%BB%E8%AE%B0%E6%96%B0%E6%8F%92%E4%BB%B6">提交插件</a>
## 相关链接
@@ -42,5 +42,5 @@
<a href="https://github.com/RockChinQ/qcg-center">遥测服务端源码</a>
<a href="https://github.com/the-lazy-me/QChatGPT-Wiki">官方文档储存库</a>
<img alt="回复效果(带有联网插件)" src="https://qchatgpt.rockchin.top/assets/image/QChatGPT-0516.png" width="500px"/>
<img alt="回复效果(带有联网插件)" src="https://docs.langbot.top/QChatGPT-0516.png" width="500px"/>
</div>

View File

@@ -1,10 +1,13 @@
version: "3"
services:
qchatgpt:
image: rockchin/qchatgpt:latest
langbot:
image: rockchin/langbot:latest
volumes:
- ./data:/app/data
- ./plugins:/app/plugins
restart: on-failure
# 根据具体环境配置网络
ports:
- 5300:5300 # 供 WebUI 使用
- 2280-2290:2280-2290 # 供消息平台适配器方向连接
# 根据具体环境配置网络

43
main.py
View File

@@ -1,19 +1,23 @@
# QChatGPT 终端启动入口
# LangBot 终端启动入口
# 在此层级解决依赖项检查。
# QChatGPT/main.py
# LangBot/main.py
asciiart = r"""
___ ___ _ _ ___ ___ _____
/ _ \ / __| |_ __ _| |_ / __| _ \_ _|
| (_) | (__| ' \/ _` | _| (_ | _/ | |
\__\_\\___|_||_\__,_|\__|\___|_| |_|
_ ___ _
| | __ _ _ _ __ _| _ ) ___| |_
| |__/ _` | ' \/ _` | _ \/ _ \ _|
|____\__,_|_||_\__, |___/\___/\__|
|___/
⭐️开源地址: https://github.com/RockChinQ/QChatGPT
📖文档地址: https://q.rkcn.top
⭐️开源地址: https://github.com/RockChinQ/LangBot
📖文档地址: https://docs.langbot.app
"""
async def main_entry():
import asyncio
async def main_entry(loop: asyncio.AbstractEventLoop):
print(asciiart)
import sys
@@ -46,13 +50,20 @@ async def main_entry():
sys.exit(0)
from pkg.core import boot
await boot.main()
await boot.main(loop)
if __name__ == '__main__':
import os
import sys
# 检查本目录是否有main.py且包含QChatGPT字符串
# 必须大于 3.10.1
if sys.version_info < (3, 10, 1):
print("需要 Python 3.10.1 及以上版本,当前 Python 版本为:", sys.version)
input("按任意键退出...")
exit(1)
# 检查本目录是否有main.py且包含LangBot字符串
invalid_pwd = False
if not os.path.exists('main.py'):
@@ -60,13 +71,13 @@ if __name__ == '__main__':
else:
with open('main.py', 'r', encoding='utf-8') as f:
content = f.read()
if "QChatGPT/main.py" not in content:
if "LangBot/main.py" not in content:
invalid_pwd = True
if invalid_pwd:
print("请在QChatGPT项目根目录下以命令形式运行此程序。")
print("请在 LangBot 项目根目录下以命令形式运行此程序。")
input("按任意键退出...")
exit(0)
exit(1)
import asyncio
loop = asyncio.new_event_loop()
asyncio.run(main_entry())
loop.run_until_complete(main_entry(loop))

View File

View File

@@ -0,0 +1,107 @@
from __future__ import annotations
import abc
import typing
import enum
import quart
from quart.typing import RouteCallable
from ....core import app
preregistered_groups: list[type[RouterGroup]] = []
"""RouterGroup 的预注册列表"""
def group_class(name: str, path: str) -> None:
"""注册一个 RouterGroup"""
def decorator(cls: typing.Type[RouterGroup]) -> typing.Type[RouterGroup]:
cls.name = name
cls.path = path
preregistered_groups.append(cls)
return cls
return decorator
class AuthType(enum.Enum):
"""认证类型"""
NONE = 'none'
USER_TOKEN = 'user-token'
class RouterGroup(abc.ABC):
name: str
path: str
ap: app.Application
quart_app: quart.Quart
def __init__(self, ap: app.Application, quart_app: quart.Quart) -> None:
self.ap = ap
self.quart_app = quart_app
@abc.abstractmethod
async def initialize(self) -> None:
pass
def route(self, rule: str, auth_type: AuthType = AuthType.USER_TOKEN, **options: typing.Any) -> typing.Callable[[RouteCallable], RouteCallable]: # decorator
"""注册一个路由"""
def decorator(f: RouteCallable) -> RouteCallable:
nonlocal rule
rule = self.path + rule
async def handler_error(*args, **kwargs):
if auth_type == AuthType.USER_TOKEN:
# 从Authorization头中获取token
token = quart.request.headers.get('Authorization', '').replace('Bearer ', '')
if not token:
return self.http_status(401, -1, '未提供有效的用户令牌')
try:
user_email = await self.ap.user_service.verify_jwt_token(token)
# 检查f是否接受user_email参数
if 'user_email' in f.__code__.co_varnames:
kwargs['user_email'] = user_email
except Exception as e:
return self.http_status(401, -1, str(e))
try:
return await f(*args, **kwargs)
except Exception as e: # 自动 500
return self.http_status(500, -2, str(e))
new_f = handler_error
new_f.__name__ = (self.name + rule).replace('/', '__')
new_f.__doc__ = f.__doc__
self.quart_app.route(rule, **options)(new_f)
return f
return decorator
def success(self, data: typing.Any = None) -> quart.Response:
"""返回一个 200 响应"""
return quart.jsonify({
'code': 0,
'msg': 'ok',
'data': data,
})
def fail(self, code: int, msg: str) -> quart.Response:
"""返回一个异常响应"""
return quart.jsonify({
'code': code,
'msg': msg,
})
def http_status(self, status: int, code: int, msg: str) -> quart.Response:
"""返回一个指定状态码的响应"""
return self.fail(code, msg), status

View File

@@ -0,0 +1,32 @@
from __future__ import annotations
import traceback
import quart
from .....core import app
from .. import group
@group.group_class('logs', '/api/v1/logs')
class LogsRouterGroup(group.RouterGroup):
async def initialize(self) -> None:
@self.route('', methods=['GET'])
async def _() -> str:
start_page_number = int(quart.request.args.get('start_page_number', 0))
start_offset = int(quart.request.args.get('start_offset', 0))
logs_str, end_page_number, end_offset = self.ap.log_cache.get_log_by_pointer(
start_page_number=start_page_number,
start_offset=start_offset
)
return self.success(
data={
"logs": logs_str,
"end_page_number": end_page_number,
"end_offset": end_offset
}
)

View File

@@ -0,0 +1,84 @@
from __future__ import annotations
import traceback
import quart
from .....core import app, taskmgr
from .. import group
@group.group_class('plugins', '/api/v1/plugins')
class PluginsRouterGroup(group.RouterGroup):
async def initialize(self) -> None:
@self.route('', methods=['GET'])
async def _() -> str:
plugins = self.ap.plugin_mgr.plugins()
plugins_data = [plugin.model_dump() for plugin in plugins]
return self.success(data={
'plugins': plugins_data
})
@self.route('/<author>/<plugin_name>/toggle', methods=['PUT'])
async def _(author: str, plugin_name: str) -> str:
data = await quart.request.json
target_enabled = data.get('target_enabled')
await self.ap.plugin_mgr.update_plugin_switch(plugin_name, target_enabled)
return self.success()
@self.route('/<author>/<plugin_name>/update', methods=['POST'])
async def _(author: str, plugin_name: str) -> str:
ctx = taskmgr.TaskContext.new()
wrapper = self.ap.task_mgr.create_user_task(
self.ap.plugin_mgr.update_plugin(plugin_name, task_context=ctx),
kind="plugin-operation",
name=f"plugin-update-{plugin_name}",
label=f"更新插件 {plugin_name}",
context=ctx
)
return self.success(data={
'task_id': wrapper.id
})
@self.route('/<author>/<plugin_name>', methods=['DELETE'])
async def _(author: str, plugin_name: str) -> str:
ctx = taskmgr.TaskContext.new()
wrapper = self.ap.task_mgr.create_user_task(
self.ap.plugin_mgr.uninstall_plugin(plugin_name, task_context=ctx),
kind="plugin-operation",
name=f'plugin-remove-{plugin_name}',
label=f'删除插件 {plugin_name}',
context=ctx
)
return self.success(data={
'task_id': wrapper.id
})
@self.route('/reorder', methods=['PUT'])
async def _() -> str:
data = await quart.request.json
await self.ap.plugin_mgr.reorder_plugins(data.get('plugins'))
return self.success()
@self.route('/install/github', methods=['POST'])
async def _() -> str:
data = await quart.request.json
ctx = taskmgr.TaskContext.new()
short_source_str = data['source'][-8:]
wrapper = self.ap.task_mgr.create_user_task(
self.ap.plugin_mgr.install_plugin(data['source'], task_context=ctx),
kind="plugin-operation",
name=f'plugin-install-github',
label=f'安装插件 ...{short_source_str}',
context=ctx
)
return self.success(data={
'task_id': wrapper.id
})

View File

@@ -0,0 +1,62 @@
import quart
from .....core import app
from .. import group
@group.group_class('settings', '/api/v1/settings')
class SettingsRouterGroup(group.RouterGroup):
async def initialize(self) -> None:
@self.route('', methods=['GET'])
async def _() -> str:
return self.success(
data={
"managers": [
{
"name": m.name,
"description": m.description,
}
for m in self.ap.settings_mgr.get_manager_list()
]
}
)
@self.route('/<manager_name>', methods=['GET'])
async def _(manager_name: str) -> str:
manager = self.ap.settings_mgr.get_manager(manager_name)
if manager is None:
return self.fail(1, '配置管理器不存在')
return self.success(
data={
"manager": {
"name": manager.name,
"description": manager.description,
"schema": manager.schema,
"file": manager.file.config_file_name,
"data": manager.data,
"doc_link": manager.doc_link
}
}
)
@self.route('/<manager_name>/data', methods=['PUT'])
async def _(manager_name: str) -> str:
data = await quart.request.json
manager = self.ap.settings_mgr.get_manager(manager_name)
if manager is None:
return self.fail(code=1, msg='配置管理器不存在')
# manager.data = data['data']
for k, v in data['data'].items():
manager.data[k] = v
await manager.dump_config()
return self.success(data={
"data": manager.data
})

View File

@@ -0,0 +1,23 @@
import quart
import asyncio
from .....core import app, taskmgr
from .. import group
@group.group_class('stats', '/api/v1/stats')
class StatsRouterGroup(group.RouterGroup):
async def initialize(self) -> None:
@self.route('/basic', methods=['GET'])
async def _() -> str:
conv_count = 0
for session in self.ap.sess_mgr.session_list:
conv_count += len(session.conversations if session.conversations is not None else [])
return self.success(data={
'active_session_count': len(self.ap.sess_mgr.session_list),
'conversation_count': conv_count,
'query_count': self.ap.query_pool.query_id_counter,
})

View File

@@ -0,0 +1,63 @@
import quart
import asyncio
from .....core import app, taskmgr
from .. import group
from .....utils import constants
@group.group_class('system', '/api/v1/system')
class SystemRouterGroup(group.RouterGroup):
async def initialize(self) -> None:
@self.route('/info', methods=['GET'], auth_type=group.AuthType.NONE)
async def _() -> str:
return self.success(
data={
"version": constants.semantic_version,
"debug": constants.debug_mode,
"enabled_platform_count": len(self.ap.platform_mgr.adapters)
}
)
@self.route('/tasks', methods=['GET'])
async def _() -> str:
task_type = quart.request.args.get("type")
if task_type == '':
task_type = None
return self.success(
data=self.ap.task_mgr.get_tasks_dict(task_type)
)
@self.route('/tasks/<task_id>', methods=['GET'])
async def _(task_id: str) -> str:
task = self.ap.task_mgr.get_task_by_id(int(task_id))
if task is None:
return self.http_status(404, 404, "Task not found")
return self.success(data=task.to_dict())
@self.route('/reload', methods=['POST'])
async def _() -> str:
json_data = await quart.request.json
scope = json_data.get("scope")
await self.ap.reload(
scope=scope
)
return self.success()
@self.route('/_debug/exec', methods=['POST'])
async def _() -> str:
if not constants.debug_mode:
return self.http_status(403, 403, "Forbidden")
py_code = await quart.request.data
ap = self.ap
return self.success(data=exec(py_code, {"ap": ap}))

View File

@@ -0,0 +1,43 @@
import quart
import sqlalchemy
from .. import group
from .....persistence.entities import user
@group.group_class('user', '/api/v1/user')
class UserRouterGroup(group.RouterGroup):
async def initialize(self) -> None:
@self.route('/init', methods=['GET', 'POST'], auth_type=group.AuthType.NONE)
async def _() -> str:
if quart.request.method == 'GET':
return self.success(data={
'initialized': await self.ap.user_service.is_initialized()
})
if await self.ap.user_service.is_initialized():
return self.fail(1, '系统已初始化')
json_data = await quart.request.json
user_email = json_data['user']
password = json_data['password']
await self.ap.user_service.create_user(user_email, password)
return self.success()
@self.route('/auth', methods=['POST'], auth_type=group.AuthType.NONE)
async def _() -> str:
json_data = await quart.request.json
token = await self.ap.user_service.authenticate(json_data['user'], json_data['password'])
return self.success(data={
'token': token
})
@self.route('/check-token', methods=['GET'])
async def _() -> str:
return self.success()

View File

@@ -0,0 +1,73 @@
from __future__ import annotations
import asyncio
import os
import quart
import quart_cors
from ....core import app, entities as core_entities
from .groups import logs, system, settings, plugins, stats, user
from . import group
class HTTPController:
ap: app.Application
quart_app: quart.Quart
def __init__(self, ap: app.Application) -> None:
self.ap = ap
self.quart_app = quart.Quart(__name__)
quart_cors.cors(self.quart_app, allow_origin="*")
async def initialize(self) -> None:
await self.register_routes()
async def run(self) -> None:
if self.ap.system_cfg.data["http-api"]["enable"]:
async def shutdown_trigger_placeholder():
while True:
await asyncio.sleep(1)
async def exception_handler(*args, **kwargs):
try:
await self.quart_app.run_task(
*args, **kwargs
)
except Exception as e:
self.ap.logger.error(f"启动 HTTP 服务失败: {e}")
self.ap.task_mgr.create_task(
exception_handler(
host=self.ap.system_cfg.data["http-api"]["host"],
port=self.ap.system_cfg.data["http-api"]["port"],
shutdown_trigger=shutdown_trigger_placeholder,
),
name="http-api-quart",
scopes=[core_entities.LifecycleControlScope.APPLICATION],
)
# await asyncio.sleep(5)
async def register_routes(self) -> None:
@self.quart_app.route("/healthz")
async def healthz():
return {"code": 0, "msg": "ok"}
for g in group.preregistered_groups:
ginst = g(self.ap, self.quart_app)
await ginst.initialize()
frontend_path = "web/dist"
@self.quart_app.route("/")
async def index():
return await quart.send_from_directory(frontend_path, "index.html")
@self.quart_app.route("/<path:path>")
async def static_file(path: str):
return await quart.send_from_directory(frontend_path, path)

View File

View File

@@ -0,0 +1,74 @@
from __future__ import annotations
import sqlalchemy
import argon2
import jwt
import datetime
from ....core import app
from ....persistence.entities import user
from ....utils import constants
class UserService:
ap: app.Application
def __init__(self, ap: app.Application) -> None:
self.ap = ap
async def is_initialized(self) -> bool:
result = await self.ap.persistence_mgr.execute_async(
sqlalchemy.select(user.User).limit(1)
)
result_list = result.all()
return result_list is not None and len(result_list) > 0
async def create_user(self, user_email: str, password: str) -> None:
ph = argon2.PasswordHasher()
hashed_password = ph.hash(password)
await self.ap.persistence_mgr.execute_async(
sqlalchemy.insert(user.User).values(
user=user_email,
password=hashed_password
)
)
async def authenticate(self, user_email: str, password: str) -> str | None:
result = await self.ap.persistence_mgr.execute_async(
sqlalchemy.select(user.User).where(user.User.user == user_email)
)
result_list = result.all()
if result_list is None or len(result_list) == 0:
raise ValueError('用户不存在')
user_obj = result_list[0]
ph = argon2.PasswordHasher()
if not ph.verify(user_obj.password, password):
raise ValueError('密码错误')
return await self.generate_jwt_token(user_email)
async def generate_jwt_token(self, user_email: str) -> str:
jwt_secret = self.ap.instance_secret_meta.data['jwt_secret']
jwt_expire = self.ap.system_cfg.data['http-api']['jwt-expire']
payload = {
'user': user_email,
'iss': 'LangBot-'+constants.edition,
'exp': datetime.datetime.now() + datetime.timedelta(seconds=jwt_expire)
}
return jwt.encode(payload, jwt_secret, algorithm='HS256')
async def verify_jwt_token(self, token: str) -> str:
jwt_secret = self.ap.instance_secret_meta.data['jwt_secret']
return jwt.decode(token, jwt_secret, algorithms=['HS256'])['user']

View File

@@ -9,11 +9,12 @@ import asyncio
import aiohttp
import requests
from ...core import app
from ...core import app, entities as core_entities
class APIGroup(metaclass=abc.ABCMeta):
"""API 组抽象类"""
_basic_info: dict = None
_runtime_info: dict = None
@@ -32,33 +33,28 @@ class APIGroup(metaclass=abc.ABCMeta):
data: dict = None,
params: dict = None,
headers: dict = {},
**kwargs
**kwargs,
):
"""
执行请求
"""
self._runtime_info['account_id'] = "-1"
self._runtime_info["account_id"] = "-1"
url = self.prefix + path
data = json.dumps(data)
headers['Content-Type'] = 'application/json'
headers["Content-Type"] = "application/json"
try:
async with aiohttp.ClientSession() as session:
async with session.request(
method,
url,
data=data,
params=params,
headers=headers,
**kwargs
method, url, data=data, params=params, headers=headers, **kwargs
) as resp:
self.ap.logger.debug("data: %s", data)
self.ap.logger.debug("ret: %s", await resp.text())
except Exception as e:
self.ap.logger.debug(f'上报失败: {e}')
self.ap.logger.debug(f"上报失败: {e}")
async def do(
self,
method: str,
@@ -66,27 +62,27 @@ class APIGroup(metaclass=abc.ABCMeta):
data: dict = None,
params: dict = None,
headers: dict = {},
**kwargs
**kwargs,
) -> asyncio.Task:
"""执行请求"""
asyncio.create_task(self._do(method, path, data, params, headers, **kwargs))
def gen_rid(
self
):
return self.ap.task_mgr.create_task(
self._do(method, path, data, params, headers, **kwargs),
kind="telemetry-operation",
name=f"{method} {path}",
scopes=[core_entities.LifecycleControlScope.APPLICATION],
).task
def gen_rid(self):
"""生成一个请求 ID"""
return str(uuid.uuid4())
def basic_info(
self
):
def basic_info(self):
"""获取基本信息"""
basic_info = APIGroup._basic_info.copy()
basic_info['rid'] = self.gen_rid()
basic_info["rid"] = self.gen_rid()
return basic_info
def runtime_info(
self
):
def runtime_info(self):
"""获取运行时信息"""
return APIGroup._runtime_info

View File

@@ -8,7 +8,7 @@ from . import entities, operator, errors
from ..config import manager as cfg_mgr
# 引入所有算子以便注册
from .operators import func, plugin, default, reset, list as list_cmd, last, next, delc, resend, prompt, cmd, help, version, update
from .operators import func, plugin, default, reset, list as list_cmd, last, next, delc, resend, prompt, cmd, help, version, update, ollama, model
class CommandManager:

View File

@@ -3,10 +3,10 @@ from __future__ import annotations
import typing
import pydantic
import mirai
from ..core import app, entities as core_entities
from . import errors, operator
from ..platform.types import message as platform_message
class CommandReturn(pydantic.BaseModel):
@@ -17,7 +17,7 @@ class CommandReturn(pydantic.BaseModel):
"""文本
"""
image: typing.Optional[mirai.Image] = None
image: typing.Optional[platform_message.Image] = None
"""弃用"""
image_url: typing.Optional[str] = None

View File

@@ -2,6 +2,7 @@ from __future__ import annotations
from typing import AsyncGenerator
from .. import operator, entities, cmdmgr
from ...plugin import context as plugin_context
@operator.operator_class(name="func", help="查看所有已注册的内容函数", usage='!func')
@@ -9,16 +10,18 @@ class FuncOperator(operator.CommandOperator):
async def execute(
self, context: entities.ExecuteContext
) -> AsyncGenerator[entities.CommandReturn, None]:
reply_str = "当前已加载的内容函数: \n\n"
reply_str = "当前已启用的内容函数: \n\n"
index = 1
all_functions = await self.ap.tool_mgr.get_all_functions()
all_functions = await self.ap.tool_mgr.get_all_functions(
plugin_enabled=True,
plugin_status=plugin_context.RuntimeContainerStatus.INITIALIZED,
)
for func in all_functions:
reply_str += "{}. {}{}:\n{}\n\n".format(
reply_str += "{}. {}:\n{}\n\n".format(
index,
("(已禁用) " if not func.enable else ""),
func.name,
func.description,
)

View File

@@ -0,0 +1,86 @@
from __future__ import annotations
import typing
from .. import operator, entities, cmdmgr, errors
@operator.operator_class(
name="model",
help='显示和切换模型列表',
usage='!model\n!model show <模型名>\n!model set <模型名>',
privilege=2
)
class ModelOperator(operator.CommandOperator):
"""Model命令"""
async def execute(self, context: entities.ExecuteContext) -> typing.AsyncGenerator[entities.CommandReturn, None]:
content = '模型列表:\n'
model_list = self.ap.model_mgr.model_list
for model in model_list:
content += f"\n名称: {model.name}\n"
content += f"请求器: {model.requester.name}\n"
content += f"\n当前对话使用模型: {context.query.use_model.name}\n"
content += f"新对话默认使用模型: {self.ap.provider_cfg.data.get('model')}\n"
yield entities.CommandReturn(text=content.strip())
@operator.operator_class(
name="show",
help='显示模型详情',
privilege=2,
parent_class=ModelOperator
)
class ModelShowOperator(operator.CommandOperator):
"""Model Show命令"""
async def execute(self, context: entities.ExecuteContext) -> typing.AsyncGenerator[entities.CommandReturn, None]:
model_name = context.crt_params[0]
model = None
for _model in self.ap.model_mgr.model_list:
if model_name == _model.name:
model = _model
break
if model is None:
yield entities.CommandReturn(error=errors.CommandError(f"未找到模型 {model_name}"))
else:
content = f"模型详情\n"
content += f"名称: {model.name}\n"
if model.model_name is not None:
content += f"请求模型名称: {model.model_name}\n"
content += f"请求器: {model.requester.name}\n"
content += f"密钥组: {model.token_mgr.provider}\n"
content += f"支持视觉: {model.vision_supported}\n"
content += f"支持工具: {model.tool_call_supported}\n"
yield entities.CommandReturn(text=content.strip())
@operator.operator_class(
name="set",
help='设置默认使用模型',
privilege=2,
parent_class=ModelOperator
)
class ModelSetOperator(operator.CommandOperator):
"""Model Set命令"""
async def execute(self, context: entities.ExecuteContext) -> typing.AsyncGenerator[entities.CommandReturn, None]:
model_name = context.crt_params[0]
model = None
for _model in self.ap.model_mgr.model_list:
if model_name == _model.name:
model = _model
break
if model is None:
yield entities.CommandReturn(error=errors.CommandError(f"未找到模型 {model_name}"))
else:
self.ap.provider_cfg.data['model'] = model_name
await self.ap.provider_cfg.dump_config()
yield entities.CommandReturn(text=f"已设置当前使用模型为 {model_name},重置会话以生效")

View File

@@ -0,0 +1,121 @@
from __future__ import annotations
import json
import typing
import traceback
import ollama
from .. import operator, entities, errors
@operator.operator_class(
name="ollama",
help="ollama平台操作",
usage="!ollama\n!ollama show <模型名>\n!ollama pull <模型名>\n!ollama del <模型名>"
)
class OllamaOperator(operator.CommandOperator):
async def execute(
self, context: entities.ExecuteContext
) -> typing.AsyncGenerator[entities.CommandReturn, None]:
try:
content: str = '模型列表:\n'
model_list: list = ollama.list().get('models', [])
for model in model_list:
content += f"名称: {model['name']}\n"
content += f"修改时间: {model['modified_at']}\n"
content += f"大小: {bytes_to_mb(model['size'])}MB\n\n"
yield entities.CommandReturn(text=f"{content.strip()}")
except ollama.ResponseError as e:
yield entities.CommandReturn(error=errors.CommandError(f"无法获取模型列表,请确认 Ollama 服务正常"))
def bytes_to_mb(num_bytes):
mb: float = num_bytes / 1024 / 1024
return format(mb, '.2f')
@operator.operator_class(
name="show",
help="ollama模型详情",
privilege=2,
parent_class=OllamaOperator
)
class OllamaShowOperator(operator.CommandOperator):
async def execute(
self, context: entities.ExecuteContext
) -> typing.AsyncGenerator[entities.CommandReturn, None]:
content: str = '模型详情:\n'
try:
show: dict = ollama.show(model=context.crt_params[0])
model_info: dict = show.get('model_info', {})
ignore_show: str = 'too long to show...'
for key in ['license', 'modelfile']:
show[key] = ignore_show
for key in ['tokenizer.chat_template.rag', 'tokenizer.chat_template.tool_use']:
model_info[key] = ignore_show
content += json.dumps(show, indent=4)
yield entities.CommandReturn(text=content.strip())
except ollama.ResponseError as e:
yield entities.CommandReturn(error=errors.CommandError(f"无法获取模型详情,请确认 Ollama 服务正常"))
@operator.operator_class(
name="pull",
help="ollama模型拉取",
privilege=2,
parent_class=OllamaOperator
)
class OllamaPullOperator(operator.CommandOperator):
async def execute(
self, context: entities.ExecuteContext
) -> typing.AsyncGenerator[entities.CommandReturn, None]:
try:
model_list: list = ollama.list().get('models', [])
if context.crt_params[0] in [model['name'] for model in model_list]:
yield entities.CommandReturn(text="模型已存在")
return
except ollama.ResponseError as e:
yield entities.CommandReturn(error=errors.CommandError(f"无法获取模型列表,请确认 Ollama 服务正常"))
return
on_progress: bool = False
progress_count: int = 0
try:
for resp in ollama.pull(model=context.crt_params[0], stream=True):
total: typing.Any = resp.get('total')
if not on_progress:
if total is not None:
on_progress = True
yield entities.CommandReturn(text=resp.get('status'))
else:
if total is None:
on_progress = False
completed: typing.Any = resp.get('completed')
if isinstance(completed, int) and isinstance(total, int):
percentage_completed = (completed / total) * 100
if percentage_completed > progress_count:
progress_count += 10
yield entities.CommandReturn(
text=f"下载进度: {completed}/{total} ({percentage_completed:.2f}%)")
except ollama.ResponseError as e:
yield entities.CommandReturn(text=f"拉取失败: {e.error}")
@operator.operator_class(
name="del",
help="ollama模型删除",
privilege=2,
parent_class=OllamaOperator
)
class OllamaDelOperator(operator.CommandOperator):
async def execute(
self, context: entities.ExecuteContext
) -> typing.AsyncGenerator[entities.CommandReturn, None]:
try:
ret: str = ollama.delete(model=context.crt_params[0])['status']
except ollama.ResponseError as e:
ret = f"{e.error}"
yield entities.CommandReturn(text=ret)

View File

@@ -18,7 +18,7 @@ class PluginOperator(operator.CommandOperator):
context: entities.ExecuteContext
) -> typing.AsyncGenerator[entities.CommandReturn, None]:
plugin_list = self.ap.plugin_mgr.plugins
plugin_list = self.ap.plugin_mgr.plugins()
reply_str = "所有插件({}):\n".format(len(plugin_list))
idx = 0
for plugin in plugin_list:
@@ -110,7 +110,7 @@ class PluginUpdateAllOperator(operator.CommandOperator):
try:
plugins = [
p.plugin_name
for p in self.ap.plugin_mgr.plugins
for p in self.ap.plugin_mgr.plugins()
]
if plugins:
@@ -163,24 +163,6 @@ class PluginDelOperator(operator.CommandOperator):
yield entities.CommandReturn(error=errors.CommandError("插件删除失败: "+str(e)))
async def update_plugin_status(plugin_name: str, new_status: bool, ap: app.Application):
if ap.plugin_mgr.get_plugin_by_name(plugin_name) is not None:
for plugin in ap.plugin_mgr.plugins:
if plugin.plugin_name == plugin_name:
plugin.enabled = new_status
for func in plugin.content_functions:
func.enable = new_status
await ap.plugin_mgr.setting.dump_container_setting(ap.plugin_mgr.plugins)
break
return True
else:
return False
@operator.operator_class(
name="on",
help="启用插件",
@@ -200,7 +182,7 @@ class PluginEnableOperator(operator.CommandOperator):
plugin_name = context.crt_params[0]
try:
if await update_plugin_status(plugin_name, True, self.ap):
if await self.ap.plugin_mgr.update_plugin_switch(plugin_name, True):
yield entities.CommandReturn(text="已启用插件: {}".format(plugin_name))
else:
yield entities.CommandReturn(error=errors.CommandError("插件状态修改失败: 未找到插件 {}".format(plugin_name)))
@@ -228,7 +210,7 @@ class PluginDisableOperator(operator.CommandOperator):
plugin_name = context.crt_params[0]
try:
if await update_plugin_status(plugin_name, False, self.ap):
if await self.ap.plugin_mgr.update_plugin_switch(plugin_name, False):
yield entities.CommandReturn(text="已禁用插件: {}".format(plugin_name))
else:
yield entities.CommandReturn(error=errors.CommandError("插件状态修改失败: 未找到插件 {}".format(plugin_name)))

View File

@@ -20,7 +20,7 @@ class VersionCommand(operator.CommandOperator):
try:
if await self.ap.ver_mgr.is_new_version_available():
reply_str += "\n\n有新版本可用, 使用 !update 更新"
reply_str += "\n\n有新版本可用"
except:
pass

View File

@@ -37,7 +37,10 @@ class JSONConfigFile(file_model.ConfigFile):
self.template_data = json.load(f)
with open(self.config_file_name, "r", encoding="utf-8") as f:
cfg = json.load(f)
try:
cfg = json.load(f)
except json.JSONDecodeError as e:
raise Exception(f"配置文件 {self.config_file_name} 语法错误: {e}")
if completion:

59
pkg/config/impls/yaml.py Normal file
View File

@@ -0,0 +1,59 @@
import os
import shutil
import yaml
from .. import model as file_model
class YAMLConfigFile(file_model.ConfigFile):
"""YAML配置文件"""
def __init__(
self, config_file_name: str, template_file_name: str = None, template_data: dict = None
) -> None:
self.config_file_name = config_file_name
self.template_file_name = template_file_name
self.template_data = template_data
def exists(self) -> bool:
return os.path.exists(self.config_file_name)
async def create(self):
if self.template_file_name is not None:
shutil.copyfile(self.template_file_name, self.config_file_name)
elif self.template_data is not None:
with open(self.config_file_name, "w", encoding="utf-8") as f:
yaml.dump(self.template_data, f, indent=4, allow_unicode=True)
else:
raise ValueError("template_file_name or template_data must be provided")
async def load(self, completion: bool=True) -> dict:
if not self.exists():
await self.create()
if self.template_file_name is not None:
with open(self.template_file_name, "r", encoding="utf-8") as f:
self.template_data = yaml.load(f, Loader=yaml.FullLoader)
with open(self.config_file_name, "r", encoding="utf-8") as f:
try:
cfg = yaml.load(f, Loader=yaml.FullLoader)
except yaml.YAMLError as e:
raise Exception(f"配置文件 {self.config_file_name} 语法错误: {e}")
if completion:
for key in self.template_data:
if key not in cfg:
cfg[key] = self.template_data[key]
return cfg
async def save(self, cfg: dict):
with open(self.config_file_name, "w", encoding="utf-8") as f:
yaml.dump(cfg, f, indent=4, allow_unicode=True)
def save_sync(self, cfg: dict):
with open(self.config_file_name, "w", encoding="utf-8") as f:
yaml.dump(cfg, f, indent=4, allow_unicode=True)

View File

@@ -1,14 +1,22 @@
from __future__ import annotations
from . import model as file_model
from .impls import pymodule, json as json_file
managers: ConfigManager = []
from .impls import pymodule, json as json_file, yaml as yaml_file
class ConfigManager:
"""配置文件管理器"""
name: str = None
"""配置管理器名"""
description: str = None
"""配置管理器描述"""
schema: dict = None
"""配置文件 schema
需要符合 JSON Schema Draft 7 规范
"""
file: file_model.ConfigFile = None
"""配置文件实例"""
@@ -16,6 +24,9 @@ class ConfigManager:
data: dict = None
"""配置数据"""
doc_link: str = None
"""配置文件文档链接"""
def __init__(self, cfg_file: file_model.ConfigFile) -> None:
self.file = cfg_file
self.data = {}
@@ -31,7 +42,16 @@ class ConfigManager:
async def load_python_module_config(config_name: str, template_name: str, completion: bool=True) -> ConfigManager:
"""加载Python模块配置文件"""
"""加载Python模块配置文件
Args:
config_name (str): 配置文件名
template_name (str): 模板文件名
completion (bool): 是否自动补全内存中的配置文件
Returns:
ConfigManager: 配置文件管理器
"""
cfg_inst = pymodule.PythonModuleConfigFile(
config_name,
template_name
@@ -44,7 +64,14 @@ async def load_python_module_config(config_name: str, template_name: str, comple
async def load_json_config(config_name: str, template_name: str=None, template_data: dict=None, completion: bool=True) -> ConfigManager:
"""加载JSON配置文件"""
"""加载JSON配置文件
Args:
config_name (str): 配置文件名
template_name (str): 模板文件名
template_data (dict): 模板数据
completion (bool): 是否自动补全内存中的配置文件
"""
cfg_inst = json_file.JSONConfigFile(
config_name,
template_name,
@@ -54,4 +81,28 @@ async def load_json_config(config_name: str, template_name: str=None, template_d
cfg_mgr = ConfigManager(cfg_inst)
await cfg_mgr.load_config(completion=completion)
return cfg_mgr
return cfg_mgr
async def load_yaml_config(config_name: str, template_name: str=None, template_data: dict=None, completion: bool=True) -> ConfigManager:
"""加载YAML配置文件
Args:
config_name (str): 配置文件名
template_name (str): 模板文件名
template_data (dict): 模板数据
completion (bool): 是否自动补全内存中的配置文件
Returns:
ConfigManager: 配置文件管理器
"""
cfg_inst = yaml_file.YAMLConfigFile(
config_name,
template_name,
template_data
)
cfg_mgr = ConfigManager(cfg_inst)
await cfg_mgr.load_config(completion=completion)
return cfg_mgr

75
pkg/config/settings.py Normal file
View File

@@ -0,0 +1,75 @@
from __future__ import annotations
from . import manager as config_manager
from ..core import app
class SettingsManager:
"""设置管理器
保存、管理多个配置文件管理器
"""
ap: app.Application
managers: list[config_manager.ConfigManager] = []
"""配置文件管理器列表"""
def __init__(self, ap: app.Application) -> None:
self.ap = ap
self.managers = []
async def initialize(self) -> None:
pass
def register_manager(
self,
name: str,
description: str,
manager: config_manager.ConfigManager,
schema: dict=None,
doc_link: str=None,
) -> None:
"""注册配置管理器
Args:
name (str): 配置管理器名
description (str): 配置管理器描述
manager (ConfigManager): 配置管理器
schema (dict): 配置文件 schema符合 JSON Schema Draft 7 规范
"""
for m in self.managers:
if m.name == name:
raise ValueError(f'配置管理器名 {name} 已存在')
manager.name = name
manager.description = description
manager.schema = schema
manager.doc_link = doc_link
self.managers.append(manager)
def get_manager(self, name: str) -> config_manager.ConfigManager | None:
"""获取配置管理器
Args:
name (str): 配置管理器名
Returns:
ConfigManager: 配置管理器
"""
for m in self.managers:
if m.name == name:
return m
return None
def get_manager_list(self) -> list[config_manager.ConfigManager]:
"""获取配置管理器列表
Returns:
list[ConfigManager]: 配置管理器列表
"""
return self.managers

View File

@@ -2,25 +2,42 @@ from __future__ import annotations
import logging
import asyncio
import threading
import traceback
import enum
import sys
import os
from ..platform import manager as im_mgr
from ..provider.session import sessionmgr as llm_session_mgr
from ..provider.modelmgr import modelmgr as llm_model_mgr
from ..provider.sysprompt import sysprompt as llm_prompt_mgr
from ..provider.tools import toolmgr as llm_tool_mgr
from ..provider import runnermgr
from ..config import manager as config_mgr
from ..config import settings as settings_mgr
from ..audit.center import v2 as center_mgr
from ..command import cmdmgr
from ..plugin import manager as plugin_mgr
from ..pipeline import pool
from ..pipeline import controller, stagemgr
from ..utils import version as version_mgr, proxy as proxy_mgr
from ..utils import version as version_mgr, proxy as proxy_mgr, announce as announce_mgr
from ..persistence import mgr as persistencemgr
from ..api.http.controller import main as http_controller
from ..api.http.service import user as user_service
from ..utils import logcache, ip
from . import taskmgr
from . import entities as core_entities
class Application:
"""运行时应用对象和上下文"""
event_loop: asyncio.AbstractEventLoop = None
# asyncio_tasks: list[asyncio.Task] = []
task_mgr: taskmgr.AsyncTaskManager = None
platform_mgr: im_mgr.PlatformManager = None
cmd_mgr: cmdmgr.CommandManager = None
@@ -33,6 +50,10 @@ class Application:
tool_mgr: llm_tool_mgr.ToolManager = None
runner_mgr: runnermgr.RunnerManager = None
settings_mgr: settings_mgr.SettingsManager = None
# ======= 配置管理器 =======
command_cfg: config_mgr.ConfigManager = None
@@ -55,6 +76,8 @@ class Application:
llm_models_meta: config_mgr.ConfigManager = None
instance_secret_meta: config_mgr.ConfigManager = None
# =========================
ctr_mgr: center_mgr.V2CenterAPI = None
@@ -69,10 +92,22 @@ class Application:
ver_mgr: version_mgr.VersionManager = None
ann_mgr: announce_mgr.AnnouncementManager = None
proxy_mgr: proxy_mgr.ProxyManager = None
logger: logging.Logger = None
persistence_mgr: persistencemgr.PersistenceManager = None
http_ctrl: http_controller.HTTPController = None
log_cache: logcache.LogCache = None
# ========= HTTP Services =========
user_service: user_service.UserService = None
def __init__(self):
pass
@@ -80,34 +115,89 @@ class Application:
pass
async def run(self):
await self.plugin_mgr.initialize_plugins()
tasks = []
try:
tasks = [
asyncio.create_task(self.platform_mgr.run()),
asyncio.create_task(self.ctrl.run())
]
await self.plugin_mgr.initialize_plugins()
# 后续可能会允许动态重启其他任务
# 故为了防止程序在非 Ctrl-C 情况下退出,这里创建一个不会结束的协程
async def never_ending():
while True:
await asyncio.sleep(1)
# 挂信号处理
import signal
def signal_handler(sig, frame):
for task in tasks:
task.cancel()
self.logger.info("程序退出.")
exit(0)
signal.signal(signal.SIGINT, signal_handler)
await asyncio.gather(*tasks, return_exceptions=True)
self.task_mgr.create_task(self.platform_mgr.run(), name="platform-manager", scopes=[core_entities.LifecycleControlScope.APPLICATION, core_entities.LifecycleControlScope.PLATFORM])
self.task_mgr.create_task(self.ctrl.run(), name="query-controller", scopes=[core_entities.LifecycleControlScope.APPLICATION])
self.task_mgr.create_task(self.http_ctrl.run(), name="http-api-controller", scopes=[core_entities.LifecycleControlScope.APPLICATION])
self.task_mgr.create_task(never_ending(), name="never-ending-task", scopes=[core_entities.LifecycleControlScope.APPLICATION])
await self.print_web_access_info()
await self.task_mgr.wait_all()
except asyncio.CancelledError:
pass
except Exception as e:
self.logger.error(f"应用运行致命异常: {e}")
self.logger.debug(f"Traceback: {traceback.format_exc()}")
async def print_web_access_info(self):
"""打印访问 webui 的提示"""
if not os.path.exists(os.path.join(".", "web/dist")):
self.logger.warning("WebUI 文件缺失请根据文档获取https://docs.langbot.app/webui/intro.html")
return
import socket
host_ip = socket.gethostbyname(socket.gethostname())
public_ip = await ip.get_myip()
port = self.system_cfg.data['http-api']['port']
tips = f"""
=======================================
✨ 您可通过以下方式访问管理面板
🏠 本地地址http://{host_ip}:{port}/
🌐 公网地址http://{public_ip}:{port}/
📌 如果您在容器中运行此程序,请确保容器的 {port} 端口已对外暴露
🔗 若要使用公网地址访问,请阅读以下须知
1. 公网地址仅供参考,请以您的主机公网 IP 为准;
2. 要使用公网地址访问,请确保您的主机具有公网 IP并且系统防火墙已放行 {port} 端口;
🤯 WebUI 仍处于 Beta 测试阶段,如有问题或建议请反馈到 https://github.com/RockChinQ/LangBot/issues
=======================================
""".strip()
for line in tips.split("\n"):
self.logger.info(line)
async def reload(
self,
scope: core_entities.LifecycleControlScope,
):
match scope:
case core_entities.LifecycleControlScope.PLATFORM.value:
self.logger.info("执行热重载 scope="+scope)
await self.platform_mgr.shutdown()
self.platform_mgr = im_mgr.PlatformManager(self)
await self.platform_mgr.initialize()
self.task_mgr.create_task(self.platform_mgr.run(), name="platform-manager", scopes=[core_entities.LifecycleControlScope.APPLICATION, core_entities.LifecycleControlScope.PLATFORM])
case core_entities.LifecycleControlScope.PLUGIN.value:
self.logger.info("执行热重载 scope="+scope)
await self.plugin_mgr.destroy_plugins()
# 删除 sys.module 中所有的 plugins/* 下的模块
for mod in list(sys.modules.keys()):
if mod.startswith("plugins."):
del sys.modules[mod]
self.plugin_mgr = plugin_mgr.PluginManager(self)
await self.plugin_mgr.initialize()
await self.plugin_mgr.initialize_plugins()
await self.plugin_mgr.load_plugins()
await self.plugin_mgr.initialize_plugins()
case _:
pass

View File

@@ -1,32 +1,45 @@
from __future__ import print_function
import traceback
import asyncio
import os
from . import app
from ..audit import identifier
from . import stage
from ..utils import constants
# 引入启动阶段实现以便注册
from .stages import load_config, setup_logger, build_app, migrate
from .stages import load_config, setup_logger, build_app, migrate, show_notes
stage_order = [
"LoadConfigStage",
"MigrationStage",
"SetupLoggerStage",
"BuildAppStage"
"BuildAppStage",
"ShowNotesStage"
]
async def make_app() -> app.Application:
async def make_app(loop: asyncio.AbstractEventLoop) -> app.Application:
# 生成标识符
identifier.init()
# 确定是否为调试模式
if "DEBUG" in os.environ and os.environ["DEBUG"] in ["true", "1"]:
constants.debug_mode = True
ap = app.Application()
ap.event_loop = loop
# 执行启动阶段
for stage_name in stage_order:
stage_cls = stage.preregistered_stages[stage_name]
stage_inst = stage_cls()
await stage_inst.run(ap)
await ap.initialize()
@@ -34,6 +47,23 @@ async def make_app() -> app.Application:
return ap
async def main():
app_inst = await make_app()
await app_inst.run()
async def main(loop: asyncio.AbstractEventLoop):
try:
# 挂系统信号处理
import signal
ap: app.Application
def signal_handler(sig, frame):
print("[Signal] 程序退出.")
# ap.shutdown()
os._exit(0)
signal.signal(signal.SIGINT, signal_handler)
app_inst = await make_app(loop)
ap = app_inst
await app_inst.run()
except Exception as e:
traceback.print_exc()

View File

@@ -8,16 +8,3 @@ from ...config.impls import pymodule
load_python_module_config = config_mgr.load_python_module_config
load_json_config = config_mgr.load_json_config
async def override_config_manager(cfg_mgr: config_mgr.ConfigManager) -> list[str]:
override_json = json.load(open("override.json", "r", encoding="utf-8"))
overrided = []
config = cfg_mgr.data
for key in override_json:
if key in config:
config[key] = override_json[key]
overrided.append(key)
return overrided

View File

@@ -5,9 +5,8 @@ required_deps = {
"openai": "openai",
"anthropic": "anthropic",
"colorlog": "colorlog",
"mirai": "yiri-mirai-rc",
"aiocqhttp": "aiocqhttp",
"botpy": "qq-botpy",
"botpy": "qq-botpy-rc",
"PIL": "pillow",
"nakuru": "nakuru-project-idk",
"tiktoken": "tiktoken",
@@ -15,6 +14,15 @@ required_deps = {
"aiohttp": "aiohttp",
"psutil": "psutil",
"async_lru": "async-lru",
"ollama": "ollama",
"quart": "quart",
"quart_cors": "quart-cors",
"sqlalchemy": "sqlalchemy[asyncio]",
"aiosqlite": "aiosqlite",
"aiofiles": "aiofiles",
"aioshutil": "aioshutil",
"argon2": "argon2-cffi",
"jwt": "pyjwt",
}

View File

@@ -5,6 +5,8 @@ import time
import colorlog
from ...utils import constants
log_colors_config = {
"DEBUG": "green", # cyan white
@@ -15,18 +17,18 @@ log_colors_config = {
}
async def init_logging() -> logging.Logger:
async def init_logging(extra_handlers: list[logging.Handler] = None) -> logging.Logger:
# 删除所有现有的logger
for handler in logging.root.handlers[:]:
logging.root.removeHandler(handler)
level = logging.INFO
if "DEBUG" in os.environ and os.environ["DEBUG"] in ["true", "1"]:
if constants.debug_mode:
level = logging.DEBUG
log_file_name = "data/logs/qcg-%s.log" % time.strftime(
"%Y-%m-%d-%H-%M-%S", time.localtime()
log_file_name = "data/logs/langbot-%s.log" % time.strftime(
"%Y-%m-%d", time.localtime()
)
qcg_logger = logging.getLogger("qcg")
@@ -34,14 +36,15 @@ async def init_logging() -> logging.Logger:
qcg_logger.setLevel(level)
color_formatter = colorlog.ColoredFormatter(
fmt="%(log_color)s[%(asctime)s.%(msecs)03d] %(pathname)s (%(lineno)d) - [%(levelname)s] :\n %(message)s",
datefmt="%Y-%m-%d %H:%M:%S",
fmt="%(log_color)s[%(asctime)s.%(msecs)03d] %(filename)s (%(lineno)d) - [%(levelname)s] : %(message)s",
datefmt="%m-%d %H:%M:%S",
log_colors=log_colors_config,
)
stream_handler = logging.StreamHandler(sys.stdout)
log_handlers: logging.Handler = [stream_handler, logging.FileHandler(log_file_name)]
log_handlers: list[logging.Handler] = [stream_handler, logging.FileHandler(log_file_name)]
log_handlers += extra_handlers if extra_handlers is not None else []
for handler in log_handlers:
handler.setLevel(level)

View File

@@ -6,13 +6,23 @@ import datetime
import asyncio
import pydantic
import mirai
from ..provider import entities as llm_entities
from ..provider.modelmgr import entities
from ..provider.sysprompt import entities as sysprompt_entities
from ..provider.tools import entities as tools_entities
from ..platform import adapter as msadapter
from ..platform.types import message as platform_message
from ..platform.types import events as platform_events
from ..platform.types import entities as platform_entities
class LifecycleControlScope(enum.Enum):
APPLICATION = "application"
PLATFORM = "platform"
PLUGIN = "plugin"
class LauncherTypes(enum.Enum):
@@ -40,10 +50,10 @@ class Query(pydantic.BaseModel):
sender_id: int
"""发送者IDplatform处理阶段设置"""
message_event: mirai.MessageEvent
message_event: platform_events.MessageEvent
"""事件platform收到的原始事件"""
message_chain: mirai.MessageChain
message_chain: platform_message.MessageChain
"""消息链platform收到的原始消息链"""
adapter: msadapter.MessageSourceAdapter
@@ -67,12 +77,15 @@ class Query(pydantic.BaseModel):
use_funcs: typing.Optional[list[tools_entities.LLMFunction]] = None
"""使用的函数,由前置处理器阶段设置"""
resp_messages: typing.Optional[list[llm_entities.Message]] | typing.Optional[list[mirai.MessageChain]] = []
resp_messages: typing.Optional[list[llm_entities.Message]] | typing.Optional[list[platform_message.MessageChain]] = []
"""由Process阶段生成的回复消息对象列表"""
resp_message_chain: typing.Optional[list[mirai.MessageChain]] = None
resp_message_chain: typing.Optional[list[platform_message.MessageChain]] = None
"""回复消息链从resp_messages包装而得"""
# ======= 内部保留 =======
current_stage: "pkg.pipeline.stagemgr.StageInstContainer" = None
class Config:
arbitrary_types_allowed = True
@@ -105,7 +118,7 @@ class Session(pydantic.BaseModel):
using_conversation: typing.Optional[Conversation] = None
conversations: typing.Optional[list[Conversation]] = []
conversations: typing.Optional[list[Conversation]] = pydantic.Field(default_factory=list)
create_time: typing.Optional[datetime.datetime] = pydantic.Field(default_factory=datetime.datetime.now)

View File

@@ -3,7 +3,7 @@ from __future__ import annotations
import abc
import typing
from ..core import app
from . import app
preregistered_migrations: list[typing.Type[Migration]] = []

View File

View File

@@ -0,0 +1,29 @@
from __future__ import annotations
from .. import migration
@migration.migration_class("ad-fixwin-cfg-migration", 8)
class AdFixwinConfigMigration(migration.Migration):
"""迁移"""
async def need_migrate(self) -> bool:
"""判断当前环境是否需要运行此迁移"""
return isinstance(
self.ap.pipeline_cfg.data["rate-limit"]["fixwin"]["default"],
int
)
async def run(self):
"""执行迁移"""
for session_name in self.ap.pipeline_cfg.data["rate-limit"]["fixwin"]:
temp_dict = {
"window-size": 60,
"limit": self.ap.pipeline_cfg.data["rate-limit"]["fixwin"][session_name]
}
self.ap.pipeline_cfg.data["rate-limit"]["fixwin"][session_name] = temp_dict
await self.ap.pipeline_cfg.dump_config()

View File

@@ -0,0 +1,24 @@
from __future__ import annotations
from .. import migration
@migration.migration_class("msg-truncator-cfg-migration", 9)
class MsgTruncatorConfigMigration(migration.Migration):
"""迁移"""
async def need_migrate(self) -> bool:
"""判断当前环境是否需要运行此迁移"""
return 'msg-truncate' not in self.ap.pipeline_cfg.data
async def run(self):
"""执行迁移"""
self.ap.pipeline_cfg.data['msg-truncate'] = {
'method': 'round',
'round': {
'max-round': 10
}
}
await self.ap.pipeline_cfg.dump_config()

View File

@@ -0,0 +1,23 @@
from __future__ import annotations
from .. import migration
@migration.migration_class("ollama-requester-config", 10)
class MsgTruncatorConfigMigration(migration.Migration):
"""迁移"""
async def need_migrate(self) -> bool:
"""判断当前环境是否需要运行此迁移"""
return 'ollama-chat' not in self.ap.provider_cfg.data['requester']
async def run(self):
"""执行迁移"""
self.ap.provider_cfg.data['requester']['ollama-chat'] = {
"base-url": "http://127.0.0.1:11434",
"args": {},
"timeout": 600
}
await self.ap.provider_cfg.dump_config()

View File

@@ -0,0 +1,21 @@
from __future__ import annotations
from .. import migration
@migration.migration_class("command-prefix-config", 11)
class CommandPrefixConfigMigration(migration.Migration):
"""迁移"""
async def need_migrate(self) -> bool:
"""判断当前环境是否需要运行此迁移"""
return 'command-prefix' not in self.ap.command_cfg.data
async def run(self):
"""执行迁移"""
self.ap.command_cfg.data['command-prefix'] = [
"!", ""
]
await self.ap.command_cfg.dump_config()

View File

@@ -0,0 +1,19 @@
from __future__ import annotations
from .. import migration
@migration.migration_class("runner-config", 12)
class RunnerConfigMigration(migration.Migration):
"""迁移"""
async def need_migrate(self) -> bool:
"""判断当前环境是否需要运行此迁移"""
return 'runner' not in self.ap.provider_cfg.data
async def run(self):
"""执行迁移"""
self.ap.provider_cfg.data['runner'] = 'local-agent'
await self.ap.provider_cfg.dump_config()

View File

@@ -0,0 +1,31 @@
from __future__ import annotations
from .. import migration
@migration.migration_class("http-api-config", 13)
class HttpApiConfigMigration(migration.Migration):
"""迁移"""
async def need_migrate(self) -> bool:
"""判断当前环境是否需要运行此迁移"""
return 'http-api' not in self.ap.system_cfg.data or "persistence" not in self.ap.system_cfg.data
async def run(self):
"""执行迁移"""
self.ap.system_cfg.data['http-api'] = {
"enable": True,
"host": "0.0.0.0",
"port": 5300,
"jwt-expire": 604800
}
self.ap.system_cfg.data['persistence'] = {
"sqlite": {
"path": "data/persistence.db"
},
"use": "sqlite"
}
await self.ap.system_cfg.dump_config()

View File

@@ -0,0 +1,22 @@
from __future__ import annotations
from .. import migration
@migration.migration_class("force-delay-config", 14)
class ForceDelayConfigMigration(migration.Migration):
"""迁移"""
async def need_migrate(self) -> bool:
"""判断当前环境是否需要运行此迁移"""
return type(self.ap.platform_cfg.data['force-delay']) == list
async def run(self):
"""执行迁移"""
self.ap.platform_cfg.data['force-delay'] = {
"min": self.ap.platform_cfg.data['force-delay'][0],
"max": self.ap.platform_cfg.data['force-delay'][1]
}
await self.ap.platform_cfg.dump_config()

44
pkg/core/note.py Normal file
View File

@@ -0,0 +1,44 @@
from __future__ import annotations
import abc
import typing
from . import app
preregistered_notes: list[typing.Type[LaunchNote]] = []
def note_class(name: str, number: int):
"""注册一个启动信息
"""
def decorator(cls: typing.Type[LaunchNote]) -> typing.Type[LaunchNote]:
cls.name = name
cls.number = number
preregistered_notes.append(cls)
return cls
return decorator
class LaunchNote(abc.ABC):
"""启动信息
"""
name: str
number: int
ap: app.Application
def __init__(self, ap: app.Application):
self.ap = ap
@abc.abstractmethod
async def need_show(self) -> bool:
"""判断当前环境是否需要显示此启动信息
"""
pass
@abc.abstractmethod
async def yield_note(self) -> typing.AsyncGenerator[typing.Tuple[str, int], None]:
"""生成启动信息
"""
pass

View File

View File

@@ -0,0 +1,20 @@
from __future__ import annotations
import typing
from .. import note, app
@note.note_class("ClassicNotes", 1)
class ClassicNotes(note.LaunchNote):
"""经典启动信息
"""
async def need_show(self) -> bool:
return True
async def yield_note(self) -> typing.AsyncGenerator[typing.Tuple[str, int], None]:
yield await self.ap.ann_mgr.show_announcements()
yield await self.ap.ver_mgr.show_version_update()

View File

@@ -0,0 +1,21 @@
from __future__ import annotations
import typing
import os
import sys
import logging
from .. import note, app
@note.note_class("SelectionModeOnWindows", 2)
class SelectionModeOnWindows(note.LaunchNote):
"""Windows 上的选择模式提示信息
"""
async def need_show(self) -> bool:
return os.name == 'nt'
async def yield_note(self) -> typing.AsyncGenerator[typing.Tuple[str, int], None]:
yield """您正在使用 Windows 系统,若窗口左上角显示处于”选择“模式,程序将被暂停运行,此时请右键窗口中空白区域退出选择模式。""", logging.INFO

View File

@@ -13,7 +13,14 @@ from ...provider.session import sessionmgr as llm_session_mgr
from ...provider.modelmgr import modelmgr as llm_model_mgr
from ...provider.sysprompt import sysprompt as llm_prompt_mgr
from ...provider.tools import toolmgr as llm_tool_mgr
from ...provider import runnermgr
from ...platform import manager as im_mgr
from ...persistence import mgr as persistencemgr
from ...api.http.controller import main as http_controller
from ...api.http.service import user as user_service
from ...utils import logcache
from .. import taskmgr
@stage.stage_class("BuildAppStage")
class BuildAppStage(stage.BootingStage):
@@ -23,6 +30,7 @@ class BuildAppStage(stage.BootingStage):
async def run(self, ap: app.Application):
"""构建app对象的各个组件对象并初始化
"""
ap.task_mgr = taskmgr.AsyncTaskManager(ap)
proxy_mgr = proxy.ProxyManager(ap)
await proxy_mgr.initialize()
@@ -53,11 +61,16 @@ class BuildAppStage(stage.BootingStage):
# 发送公告
ann_mgr = announce.AnnouncementManager(ap)
await ann_mgr.show_announcements()
ap.ann_mgr = ann_mgr
ap.query_pool = pool.QueryPool()
await ap.ver_mgr.show_version_update()
log_cache = logcache.LogCache()
ap.log_cache = log_cache
persistence_mgr_inst = persistencemgr.PersistenceManager(ap)
await persistence_mgr_inst.initialize()
ap.persistence_mgr = persistence_mgr_inst
plugin_mgr_inst = plugin_mgr.PluginManager(ap)
await plugin_mgr_inst.initialize()
@@ -83,6 +96,11 @@ class BuildAppStage(stage.BootingStage):
llm_tool_mgr_inst = llm_tool_mgr.ToolManager(ap)
await llm_tool_mgr_inst.initialize()
ap.tool_mgr = llm_tool_mgr_inst
runner_mgr_inst = runnermgr.RunnerManager(ap)
await runner_mgr_inst.initialize()
ap.runner_mgr = runner_mgr_inst
im_mgr_inst = im_mgr.PlatformManager(ap=ap)
await im_mgr_inst.initialize()
ap.platform_mgr = im_mgr_inst
@@ -91,6 +109,12 @@ class BuildAppStage(stage.BootingStage):
await stage_mgr.initialize()
ap.stage_mgr = stage_mgr
http_ctrl = http_controller.HTTPController(ap)
await http_ctrl.initialize()
ap.http_ctrl = http_ctrl
user_service_inst = user_service.UserService(ap)
ap.user_service = user_service_inst
ctrl = controller.Controller(ap)
ap.ctrl = ctrl

View File

@@ -1,7 +1,11 @@
from __future__ import annotations
import secrets
from .. import stage, app
from ..bootutils import config
from ...config import settings as settings_mgr
from ...utils import schema
@stage.stage_class("LoadConfigStage")
@@ -12,12 +16,56 @@ class LoadConfigStage(stage.BootingStage):
async def run(self, ap: app.Application):
"""启动
"""
ap.settings_mgr = settings_mgr.SettingsManager(ap)
await ap.settings_mgr.initialize()
ap.command_cfg = await config.load_json_config("data/config/command.json", "templates/command.json", completion=False)
ap.pipeline_cfg = await config.load_json_config("data/config/pipeline.json", "templates/pipeline.json", completion=False)
ap.platform_cfg = await config.load_json_config("data/config/platform.json", "templates/platform.json", completion=False)
ap.provider_cfg = await config.load_json_config("data/config/provider.json", "templates/provider.json", completion=False)
ap.system_cfg = await config.load_json_config("data/config/system.json", "templates/system.json", completion=False)
ap.settings_mgr.register_manager(
name="command.json",
description="命令配置",
manager=ap.command_cfg,
schema=schema.CONFIG_COMMAND_SCHEMA,
doc_link="https://docs.langbot.app/config/function/command.html"
)
ap.settings_mgr.register_manager(
name="pipeline.json",
description="消息处理流水线配置",
manager=ap.pipeline_cfg,
schema=schema.CONFIG_PIPELINE_SCHEMA,
doc_link="https://docs.langbot.app/config/function/pipeline.html"
)
ap.settings_mgr.register_manager(
name="platform.json",
description="消息平台配置",
manager=ap.platform_cfg,
schema=schema.CONFIG_PLATFORM_SCHEMA,
doc_link="https://docs.langbot.app/config/function/platform.html"
)
ap.settings_mgr.register_manager(
name="provider.json",
description="大模型能力配置",
manager=ap.provider_cfg,
schema=schema.CONFIG_PROVIDER_SCHEMA,
doc_link="https://docs.langbot.app/config/function/provider.html"
)
ap.settings_mgr.register_manager(
name="system.json",
description="系统配置",
manager=ap.system_cfg,
schema=schema.CONFIG_SYSTEM_SCHEMA,
doc_link="https://docs.langbot.app/config/function/system.html"
)
ap.plugin_setting_meta = await config.load_json_config("plugins/plugins.json", "templates/plugin-settings.json")
await ap.plugin_setting_meta.dump_config()
@@ -29,3 +77,8 @@ class LoadConfigStage(stage.BootingStage):
ap.llm_models_meta = await config.load_json_config("data/metadata/llm-models.json", "templates/metadata/llm-models.json")
await ap.llm_models_meta.dump_config()
ap.instance_secret_meta = await config.load_json_config("data/metadata/instance-secret.json", template_data={
'jwt_secret': secrets.token_hex(16)
})
await ap.instance_secret_meta.dump_config()

View File

@@ -3,9 +3,10 @@ from __future__ import annotations
import importlib
from .. import stage, app
from ...config import migration
from ...config.migrations import m001_sensitive_word_migration, m002_openai_config_migration, m003_anthropic_requester_cfg_completion, m004_moonshot_cfg_completion
from ...config.migrations import m005_deepseek_cfg_completion, m006_vision_config, m007_qcg_center_url
from .. import migration
from ..migrations import m001_sensitive_word_migration, m002_openai_config_migration, m003_anthropic_requester_cfg_completion, m004_moonshot_cfg_completion
from ..migrations import m005_deepseek_cfg_completion, m006_vision_config, m007_qcg_center_url, m008_ad_fixwin_config_migrate, m009_msg_truncator_cfg
from ..migrations import m010_ollama_requester_config, m011_command_prefix_config, m012_runner_config, m013_http_api_config, m014_force_delay_config
@stage.stage_class("MigrationStage")

View File

@@ -1,9 +1,38 @@
from __future__ import annotations
import logging
import asyncio
from datetime import datetime
from .. import stage, app
from ..bootutils import log
class PersistenceHandler(logging.Handler, object):
"""
保存日志到数据库
"""
ap: app.Application
def __init__(self, name, ap: app.Application):
logging.Handler.__init__(self)
self.ap = ap
def emit(self, record):
"""
emit函数为自定义handler类时必重写的函数这里可以根据需要对日志消息做一些处理比如发送日志到服务器
发出记录(Emit a record)
"""
try:
msg = self.format(record)
if self.ap.log_cache is not None:
self.ap.log_cache.add_log(msg)
except Exception:
self.handleError(record)
@stage.stage_class("SetupLoggerStage")
class SetupLoggerStage(stage.BootingStage):
"""设置日志器阶段
@@ -12,4 +41,9 @@ class SetupLoggerStage(stage.BootingStage):
async def run(self, ap: app.Application):
"""启动
"""
ap.logger = await log.init_logging()
persistence_handler = PersistenceHandler('LoggerHandler', ap)
extra_handlers = []
extra_handlers = [persistence_handler]
ap.logger = await log.init_logging(extra_handlers)

View File

@@ -0,0 +1,28 @@
from __future__ import annotations
from .. import stage, app, note
from ..notes import n001_classic_msgs, n002_selection_mode_on_windows
@stage.stage_class("ShowNotesStage")
class ShowNotesStage(stage.BootingStage):
"""显示启动信息阶段
"""
async def run(self, ap: app.Application):
# 排序
note.preregistered_notes.sort(key=lambda x: x.number)
for note_cls in note.preregistered_notes:
try:
note_inst = note_cls(ap)
if await note_inst.need_show():
async for ret in note_inst.yield_note():
if not ret:
continue
msg, level = ret
if msg:
ap.logger.log(level, msg)
except Exception as e:
continue

235
pkg/core/taskmgr.py Normal file
View File

@@ -0,0 +1,235 @@
from __future__ import annotations
import asyncio
import typing
import datetime
import traceback
from . import app
from . import entities as core_entities
class TaskContext:
"""任务跟踪上下文"""
current_action: str
"""当前正在执行的动作"""
log: str
"""记录日志"""
def __init__(self):
self.current_action = "default"
self.log = ""
def _log(self, msg: str):
self.log += msg + "\n"
def set_current_action(self, action: str):
self.current_action = action
def trace(
self,
msg: str,
action: str = None,
):
if action is not None:
self.set_current_action(action)
self._log(
f"{datetime.datetime.now().strftime('%Y-%m-%d %H:%M:%S')} | {self.current_action} | {msg}"
)
def to_dict(self) -> dict:
return {"current_action": self.current_action, "log": self.log}
@staticmethod
def new() -> TaskContext:
return TaskContext()
@staticmethod
def placeholder() -> TaskContext:
global placeholder_context
if placeholder_context is None:
placeholder_context = TaskContext()
return placeholder_context
placeholder_context: TaskContext | None = None
class TaskWrapper:
"""任务包装器"""
_id_index: int = 0
"""任务ID索引"""
id: int
"""任务ID"""
task_type: str = "system" # 任务类型: system 或 user
"""任务类型"""
kind: str = "system_task" # 由发起者确定任务种类,通常同质化的任务种类相同
"""任务种类"""
name: str = ""
"""任务唯一名称"""
label: str = ""
"""任务显示名称"""
task_context: TaskContext
"""任务上下文"""
task: asyncio.Task
"""任务"""
task_stack: list = None
"""任务堆栈"""
ap: app.Application
"""应用实例"""
scopes: list[core_entities.LifecycleControlScope]
"""任务所属生命周期控制范围"""
def __init__(
self,
ap: app.Application,
coro: typing.Coroutine,
task_type: str = "system",
kind: str = "system_task",
name: str = "",
label: str = "",
context: TaskContext = None,
scopes: list[core_entities.LifecycleControlScope] = [core_entities.LifecycleControlScope.APPLICATION],
):
self.id = TaskWrapper._id_index
TaskWrapper._id_index += 1
self.ap = ap
self.task_context = context or TaskContext()
self.task = self.ap.event_loop.create_task(coro)
self.task_type = task_type
self.kind = kind
self.name = name
self.label = label if label != "" else name
self.task.set_name(name)
self.scopes = scopes
def assume_exception(self):
try:
exception = self.task.exception()
if self.task_stack is None:
self.task_stack = self.task.get_stack()
return exception
except:
return None
def assume_result(self):
try:
return self.task.result()
except:
return None
def to_dict(self) -> dict:
exception_traceback = None
if self.assume_exception() is not None:
exception_traceback = 'Traceback (most recent call last):\n'
for frame in self.task_stack:
exception_traceback += f" File \"{frame.f_code.co_filename}\", line {frame.f_lineno}, in {frame.f_code.co_name}\n"
exception_traceback += f" {self.assume_exception().__str__()}\n"
return {
"id": self.id,
"task_type": self.task_type,
"kind": self.kind,
"name": self.name,
"label": self.label,
"scopes": [scope.value for scope in self.scopes],
"task_context": self.task_context.to_dict(),
"runtime": {
"done": self.task.done(),
"state": self.task._state,
"exception": self.assume_exception().__str__() if self.assume_exception() is not None else None,
"exception_traceback": exception_traceback,
"result": self.assume_result().__str__() if self.assume_result() is not None else None,
},
}
def cancel(self):
self.task.cancel()
class AsyncTaskManager:
"""保存app中的所有异步任务
包含系统级的和用户级(插件安装、更新等由用户直接发起的)的"""
ap: app.Application
tasks: list[TaskWrapper]
"""所有任务"""
def __init__(self, ap: app.Application):
self.ap = ap
self.tasks = []
def create_task(
self,
coro: typing.Coroutine,
task_type: str = "system",
kind: str = "system-task",
name: str = "",
label: str = "",
context: TaskContext = None,
scopes: list[core_entities.LifecycleControlScope] = [core_entities.LifecycleControlScope.APPLICATION],
) -> TaskWrapper:
wrapper = TaskWrapper(self.ap, coro, task_type, kind, name, label, context, scopes)
self.tasks.append(wrapper)
return wrapper
def create_user_task(
self,
coro: typing.Coroutine,
kind: str = "user-task",
name: str = "",
label: str = "",
context: TaskContext = None,
scopes: list[core_entities.LifecycleControlScope] = [core_entities.LifecycleControlScope.APPLICATION],
) -> TaskWrapper:
return self.create_task(coro, "user", kind, name, label, context, scopes)
async def wait_all(self):
await asyncio.gather(*[t.task for t in self.tasks], return_exceptions=True)
def get_all_tasks(self) -> list[TaskWrapper]:
return self.tasks
def get_tasks_dict(
self,
type: str = None,
) -> dict:
return {
"tasks": [
t.to_dict() for t in self.tasks if type is None or t.task_type == type
],
"id_index": TaskWrapper._id_index,
}
def get_task_by_id(self, id: int) -> TaskWrapper | None:
for t in self.tasks:
if t.id == id:
return t
return None
def cancel_by_scope(self, scope: core_entities.LifecycleControlScope):
for wrapper in self.tasks:
if not wrapper.task.done() and scope in wrapper.scopes:
wrapper.task.cancel()

View File

View File

@@ -0,0 +1,40 @@
from __future__ import annotations
import abc
import sqlalchemy.ext.asyncio as sqlalchemy_asyncio
from ..core import app
preregistered_managers: list[type[BaseDatabaseManager]] = []
def manager_class(name: str) -> None:
"""注册一个数据库管理类"""
def decorator(cls: type[BaseDatabaseManager]) -> type[BaseDatabaseManager]:
cls.name = name
preregistered_managers.append(cls)
return cls
return decorator
class BaseDatabaseManager(abc.ABC):
"""基础数据库管理类"""
name: str
ap: app.Application
engine: sqlalchemy_asyncio.AsyncEngine
def __init__(self, ap: app.Application) -> None:
self.ap = ap
@abc.abstractmethod
async def initialize(self) -> None:
pass
def get_engine(self) -> sqlalchemy_asyncio.AsyncEngine:
return self.engine

View File

View File

@@ -0,0 +1,13 @@
from __future__ import annotations
import sqlalchemy.ext.asyncio as sqlalchemy_asyncio
from .. import database
@database.manager_class("sqlite")
class SQLiteDatabaseManager(database.BaseDatabaseManager):
"""SQLite 数据库管理类"""
async def initialize(self) -> None:
self.engine = sqlalchemy_asyncio.create_async_engine(f"sqlite+aiosqlite:///{self.ap.system_cfg.data['persistence']['sqlite']['path']}")

View File

View File

@@ -0,0 +1,5 @@
import sqlalchemy.orm
class Base(sqlalchemy.orm.DeclarativeBase):
pass

View File

@@ -0,0 +1,11 @@
import sqlalchemy
from .base import Base
class User(Base):
__tablename__ = 'users'
id = sqlalchemy.Column(sqlalchemy.Integer, primary_key=True)
user = sqlalchemy.Column(sqlalchemy.String(255), nullable=False)
password = sqlalchemy.Column(sqlalchemy.String(255), nullable=False)

57
pkg/persistence/mgr.py Normal file
View File

@@ -0,0 +1,57 @@
from __future__ import annotations
import asyncio
import datetime
import sqlalchemy.ext.asyncio as sqlalchemy_asyncio
import sqlalchemy
from . import database
from .entities import user, base
from ..core import app
from .databases import sqlite
class PersistenceManager:
"""持久化模块管理器"""
ap: app.Application
db: database.BaseDatabaseManager
"""数据库管理器"""
meta: sqlalchemy.MetaData
def __init__(self, ap: app.Application):
self.ap = ap
self.meta = base.Base.metadata
async def initialize(self):
for manager in database.preregistered_managers:
self.db = manager(self.ap)
await self.db.initialize()
await self.create_tables()
async def create_tables(self):
# TODO: 对扩展友好
# 日志
async with self.get_db_engine().connect() as conn:
await conn.run_sync(self.meta.create_all)
await conn.commit()
async def execute_async(
self,
*args,
**kwargs
) -> sqlalchemy.engine.cursor.CursorResult:
async with self.get_db_engine().connect() as conn:
result = await conn.execute(*args, **kwargs)
await conn.commit()
return result
def get_db_engine(self) -> sqlalchemy_asyncio.AsyncEngine:
return self.db.get_engine()

View File

@@ -1,7 +1,5 @@
from __future__ import annotations
import mirai
from ...core import app
from .. import stage, entities, stagemgr
@@ -10,6 +8,9 @@ from ...config import manager as cfg_mgr
from . import filter as filter_model, entities as filter_entities
from .filters import cntignore, banwords, baiduexamine
from ...provider import entities as llm_entities
from ...platform.types import message as platform_message
from ...platform.types import events as platform_events
from ...platform.types import entities as platform_entities
@stage.stage_class('PostContentFilterStage')
@@ -63,6 +64,7 @@ class ContentFilterStage(stage.PipelineStage):
"""请求llm前处理消息
只要有一个不通过就不放行,只放行 PASS 的消息
"""
if not self.ap.pipeline_cfg.data['income-msg-check']:
return entities.StageProcessResult(
result_type=entities.ResultType.CONTINUE,
@@ -86,8 +88,8 @@ class ContentFilterStage(stage.PipelineStage):
elif result.level == filter_entities.ResultLevel.PASS: # 传到下一个
message = result.replacement
query.message_chain = mirai.MessageChain(
mirai.Plain(message)
query.message_chain = platform_message.MessageChain(
platform_message.Plain(message)
)
return entities.StageProcessResult(
@@ -145,11 +147,13 @@ class ContentFilterStage(stage.PipelineStage):
contain_non_text = False
text_components = [platform_message.Plain, platform_message.Source]
for me in query.message_chain:
if not isinstance(me, mirai.Plain):
if type(me) not in text_components:
contain_non_text = True
break
if contain_non_text:
self.ap.logger.debug(f"消息中包含非文本消息,跳过内容过滤器检查。")
return entities.StageProcessResult(

View File

@@ -7,6 +7,7 @@ import traceback
from ..core import app, entities
from . import entities as pipeline_entities
from ..plugin import events
from ..platform.types import message as platform_message
class Controller:
@@ -57,8 +58,13 @@ class Controller:
(await self.ap.sess_mgr.get_session(selected_query)).semaphore.release()
# 通知其他协程,有新的请求可以处理了
self.ap.query_pool.condition.notify_all()
asyncio.create_task(_process_query(selected_query))
self.ap.task_mgr.create_task(
_process_query(selected_query),
kind="query",
name=f"query-{selected_query.query_id}",
scopes=[entities.LifecycleControlScope.APPLICATION, entities.LifecycleControlScope.PLATFORM],
)
except Exception as e:
# traceback.print_exc()
self.ap.logger.error(f"控制器循环出错: {e}")
@@ -68,6 +74,17 @@ class Controller:
"""检查输出
"""
if result.user_notice:
# 处理str类型
if isinstance(result.user_notice, str):
result.user_notice = platform_message.MessageChain(
platform_message.Plain(result.user_notice)
)
elif isinstance(result.user_notice, list):
result.user_notice = platform_message.MessageChain(
*result.user_notice
)
await self.ap.platform_mgr.send(
query.message_event,
result.user_notice,
@@ -109,6 +126,8 @@ class Controller:
while i < len(self.ap.stage_mgr.stage_containers):
stage_container = self.ap.stage_mgr.stage_containers[i]
query.current_stage = stage_container # 标记到 Query 对象里
result = stage_container.inst.process(query, stage_container.inst_name)
@@ -144,14 +163,30 @@ class Controller:
async def process_query(self, query: entities.Query):
"""处理请求
"""
# ======== 触发 MessageReceived 事件 ========
event_type = events.PersonMessageReceived if query.launcher_type == entities.LauncherTypes.PERSON else events.GroupMessageReceived
event_ctx = await self.ap.plugin_mgr.emit_event(
event=event_type(
launcher_type=query.launcher_type.value,
launcher_id=query.launcher_id,
sender_id=query.sender_id,
message_chain=query.message_chain,
query=query
)
)
if event_ctx.is_prevented_default():
return
self.ap.logger.debug(f"Processing query {query}")
try:
await self._execute_from_stage(0, query)
except Exception as e:
self.ap.logger.error(f"处理请求时出错 query_id={query.query_id}: {e}")
self.ap.logger.error(f"处理请求时出错 query_id={query.query_id} stage={query.current_stage.inst_name} : {e}")
self.ap.logger.debug(f"Traceback: {traceback.format_exc()}")
# traceback.print_exc()
finally:
self.ap.logger.debug(f"Query {query} processed")

View File

@@ -4,8 +4,7 @@ import enum
import typing
import pydantic
import mirai
import mirai.models.message as mirai_message
from ..platform.types import message as platform_message
from ..core import entities
@@ -25,13 +24,9 @@ class StageProcessResult(pydantic.BaseModel):
new_query: entities.Query
user_notice: typing.Optional[typing.Union[str, list[mirai_message.MessageComponent], mirai.MessageChain, None]] = []
user_notice: typing.Optional[typing.Union[str, list[platform_message.MessageComponent], platform_message.MessageChain, None]] = []
"""只要设置了就会发送给用户"""
# TODO delete
# admin_notice: typing.Optional[typing.Union[str, list[mirai_message.MessageComponent], mirai.MessageChain, None]] = []
"""只要设置了就会发送给管理员"""
console_notice: typing.Optional[str] = ''
"""只要设置了就会输出到控制台"""

View File

@@ -3,7 +3,6 @@ import os
import traceback
from PIL import Image, ImageDraw, ImageFont
from mirai.models.message import MessageComponent, Plain, MessageChain
from ...core import app
from . import strategy
@@ -11,6 +10,7 @@ from .strategies import image, forward
from .. import stage, entities, stagemgr
from ...core import entities as core_entities
from ...config import manager as cfg_mgr
from ...platform.types import message as platform_message
@stage.stage_class("LongTextProcessStage")
@@ -63,14 +63,14 @@ class LongTextProcessStage(stage.PipelineStage):
contains_non_plain = False
for msg in query.resp_message_chain[-1]:
if not isinstance(msg, Plain):
if not isinstance(msg, platform_message.Plain):
contains_non_plain = True
break
if contains_non_plain:
self.ap.logger.debug("消息中包含非 Plain 组件,跳过长消息处理。")
elif len(str(query.resp_message_chain[-1])) > self.ap.platform_cfg.data['long-text-process']['threshold']:
query.resp_message_chain[-1] = MessageChain(await self.strategy_impl.process(str(query.resp_message_chain[-1]), query))
query.resp_message_chain[-1] = platform_message.MessageChain(await self.strategy_impl.process(str(query.resp_message_chain[-1]), query))
return entities.StageProcessResult(
result_type=entities.ResultType.CONTINUE,

View File

@@ -2,15 +2,14 @@
from __future__ import annotations
import typing
from mirai.models import MessageChain
from mirai.models.message import MessageComponent, ForwardMessageNode
from mirai.models.base import MiraiBaseModel
import pydantic
from .. import strategy as strategy_model
from ....core import entities as core_entities
from ....platform.types import message as platform_message
class ForwardMessageDiaplay(MiraiBaseModel):
class ForwardMessageDiaplay(pydantic.BaseModel):
title: str = "群聊的聊天记录"
brief: str = "[聊天记录]"
source: str = "聊天记录"
@@ -18,13 +17,13 @@ class ForwardMessageDiaplay(MiraiBaseModel):
summary: str = "查看x条转发消息"
class Forward(MessageComponent):
class Forward(platform_message.MessageComponent):
"""合并转发。"""
type: str = "Forward"
"""消息组件类型。"""
display: ForwardMessageDiaplay
"""显示信息"""
node_list: typing.List[ForwardMessageNode]
node_list: typing.List[platform_message.ForwardMessageNode]
"""转发消息节点列表。"""
def __init__(self, *args, **kwargs):
if len(args) == 1:
@@ -39,7 +38,7 @@ class Forward(MessageComponent):
@strategy_model.strategy_class("forward")
class ForwardComponentStrategy(strategy_model.LongTextStrategy):
async def process(self, message: str, query: core_entities.Query) -> list[MessageComponent]:
async def process(self, message: str, query: core_entities.Query) -> list[platform_message.MessageComponent]:
display = ForwardMessageDiaplay(
title="群聊的聊天记录",
brief="[聊天记录]",
@@ -49,10 +48,10 @@ class ForwardComponentStrategy(strategy_model.LongTextStrategy):
)
node_list = [
ForwardMessageNode(
platform_message.ForwardMessageNode(
sender_id=query.adapter.bot_account_id,
sender_name='QQ用户',
message_chain=MessageChain([message])
message_chain=platform_message.MessageChain([message])
)
]

View File

@@ -8,8 +8,7 @@ import re
from PIL import Image, ImageDraw, ImageFont
from mirai.models import MessageChain, Image as ImageComponent
from mirai.models.message import MessageComponent
from ....platform.types import message as platform_message
from .. import strategy as strategy_model
from ....core import entities as core_entities
@@ -23,7 +22,7 @@ class Text2ImageStrategy(strategy_model.LongTextStrategy):
async def initialize(self):
self.text_render_font = ImageFont.truetype(self.ap.platform_cfg.data['long-text-process']['font-path'], 32, encoding="utf-8")
async def process(self, message: str, query: core_entities.Query) -> list[MessageComponent]:
async def process(self, message: str, query: core_entities.Query) -> list[platform_message.MessageComponent]:
img_path = self.text_to_image(
text_str=message,
save_as='temp/{}.png'.format(int(time.time()))
@@ -46,7 +45,7 @@ class Text2ImageStrategy(strategy_model.LongTextStrategy):
os.remove(compressed_path)
return [
ImageComponent(
platform_message.Image(
base64=b64.decode('utf-8'),
)
]

View File

@@ -2,11 +2,10 @@ from __future__ import annotations
import abc
import typing
import mirai
from mirai.models.message import MessageComponent
from ...core import app
from ...core import entities as core_entities
from ...platform.types import message as platform_message
preregistered_strategies: list[typing.Type[LongTextStrategy]] = []
@@ -51,7 +50,7 @@ class LongTextStrategy(metaclass=abc.ABCMeta):
pass
@abc.abstractmethod
async def process(self, message: str, query: core_entities.Query) -> list[MessageComponent]:
async def process(self, message: str, query: core_entities.Query) -> list[platform_message.MessageComponent]:
"""处理长文本
在 platform.json 中配置 long-text-process 字段,只要 文本长度超过了 threshold 就会调用此方法
@@ -61,6 +60,6 @@ class LongTextStrategy(metaclass=abc.ABCMeta):
query (core_entities.Query): 此次请求的上下文对象
Returns:
list[mirai.models.messages.MessageComponent]: 转换后的 YiriMirai 消息组件列表
list[platform_message.MessageComponent]: 转换后的 平台 消息组件列表
"""
return []

View File

View File

@@ -0,0 +1,35 @@
from __future__ import annotations
from .. import stage, entities, stagemgr
from ...core import entities as core_entities
from . import truncator
from .truncators import round
@stage.stage_class("ConversationMessageTruncator")
class ConversationMessageTruncator(stage.PipelineStage):
"""会话消息截断器
用于截断会话消息链,以适应平台消息长度限制。
"""
trun: truncator.Truncator
async def initialize(self):
use_method = self.ap.pipeline_cfg.data['msg-truncate']['method']
for trun in truncator.preregistered_truncators:
if trun.name == use_method:
self.trun = trun(self.ap)
break
else:
raise ValueError(f"未知的截断器: {use_method}")
async def process(self, query: core_entities.Query, stage_inst_name: str) -> entities.StageProcessResult:
"""处理
"""
query = await self.trun.truncate(query)
return entities.StageProcessResult(
result_type=entities.ResultType.CONTINUE,
new_query=query
)

View File

@@ -0,0 +1,56 @@
from __future__ import annotations
import typing
import abc
from ...core import entities as core_entities, app
preregistered_truncators: list[typing.Type[Truncator]] = []
def truncator_class(
name: str
) -> typing.Callable[[typing.Type[Truncator]], typing.Type[Truncator]]:
"""截断器类装饰器
Args:
name (str): 截断器名称
Returns:
typing.Callable[[typing.Type[Truncator]], typing.Type[Truncator]]: 装饰器
"""
def decorator(cls: typing.Type[Truncator]) -> typing.Type[Truncator]:
assert issubclass(cls, Truncator)
cls.name = name
preregistered_truncators.append(cls)
return cls
return decorator
class Truncator(abc.ABC):
"""消息截断器基类
"""
name: str
ap: app.Application
def __init__(self, ap: app.Application):
self.ap = ap
async def initialize(self):
pass
@abc.abstractmethod
async def truncate(self, query: core_entities.Query) -> core_entities.Query:
"""截断
一般只需要操作query.messages也可以扩展操作query.prompt, query.user_message。
请勿操作其他字段。
"""
pass

View File

@@ -0,0 +1,32 @@
from __future__ import annotations
from .. import truncator
from ....core import entities as core_entities
@truncator.truncator_class("round")
class RoundTruncator(truncator.Truncator):
"""前文回合数阶段器
"""
async def truncate(self, query: core_entities.Query) -> core_entities.Query:
"""截断
"""
max_round = self.ap.pipeline_cfg.data['msg-truncate']['round']['max-round']
temp_messages = []
current_round = 0
# 从后往前遍历
for msg in query.messages[::-1]:
if current_round < max_round:
temp_messages.append(msg)
if msg.role == 'user':
current_round += 1
else:
break
query.messages = temp_messages[::-1]
return query

View File

@@ -2,10 +2,11 @@ from __future__ import annotations
import asyncio
import mirai
from ..core import entities
from ..platform import adapter as msadapter
from ..platform.types import message as platform_message
from ..platform.types import events as platform_events
class QueryPool:
@@ -30,8 +31,8 @@ class QueryPool:
launcher_type: entities.LauncherTypes,
launcher_id: int,
sender_id: int,
message_event: mirai.MessageEvent,
message_chain: mirai.MessageChain,
message_event: platform_events.MessageEvent,
message_chain: platform_message.MessageChain,
adapter: msadapter.MessageSourceAdapter
) -> entities.Query:
async with self.condition:

View File

@@ -1,11 +1,11 @@
from __future__ import annotations
import mirai
from .. import stage, entities, stagemgr
from ...core import entities as core_entities
from ...provider import entities as llm_entities
from ...plugin import events
from ...platform.types import message as platform_message
@stage.stage_class("PreProcessor")
@@ -55,11 +55,11 @@ class PreProcessor(stage.PipelineStage):
content_list = []
for me in query.message_chain:
if isinstance(me, mirai.Plain):
if isinstance(me, platform_message.Plain):
content_list.append(
llm_entities.ContentElement.from_text(me.text)
)
elif isinstance(me, mirai.Image):
elif isinstance(me, platform_message.Image):
if self.ap.provider_cfg.data['enable-vision'] and query.use_model.vision_supported:
if me.url is not None:
content_list.append(

View File

@@ -5,14 +5,15 @@ import time
import traceback
import json
import mirai
from .. import handler
from ... import entities
from ....core import entities as core_entities
from ....provider import entities as llm_entities
from ....provider import entities as llm_entities, runnermgr
from ....plugin import events
from ....platform.types import message as platform_message
class ChatMessageHandler(handler.MessageHandler):
@@ -40,7 +41,7 @@ class ChatMessageHandler(handler.MessageHandler):
if event_ctx.is_prevented_default():
if event_ctx.event.reply is not None:
mc = mirai.MessageChain(event_ctx.event.reply)
mc = platform_message.MessageChain(event_ctx.event.reply)
query.resp_messages.append(mc)
@@ -62,9 +63,8 @@ class ChatMessageHandler(handler.MessageHandler):
)
if event_ctx.event.alter is not None:
query.message_chain = mirai.MessageChain([
mirai.Plain(event_ctx.event.alter)
])
# if isinstance(event_ctx.event, str): # 现在暂时不考虑多模态alter
query.user_message.content = event_ctx.event.alter
text_length = 0
@@ -72,7 +72,9 @@ class ChatMessageHandler(handler.MessageHandler):
try:
async for result in self.runner(query):
runner = self.ap.runner_mgr.get_runner()
async for result in runner.run(query):
query.resp_messages.append(result)
self.ap.logger.info(f'对话({query.query_id})响应: {self.cut_str(result.readable_str())}')
@@ -109,64 +111,3 @@ class ChatMessageHandler(handler.MessageHandler):
response_seconds=int(time.time() - start_time),
retry_times=-1,
)
async def runner(
self,
query: core_entities.Query,
) -> typing.AsyncGenerator[llm_entities.Message, None]:
"""执行一个请求处理过程中的LLM接口请求、函数调用的循环
这是临时处理方案后续可能改为使用LangChain或者自研的工作流处理器
"""
await query.use_model.requester.preprocess(query)
pending_tool_calls = []
req_messages = query.prompt.messages.copy() + query.messages.copy() + [query.user_message]
# 首次请求
msg = await query.use_model.requester.call(query.use_model, req_messages, query.use_funcs)
yield msg
pending_tool_calls = msg.tool_calls
req_messages.append(msg)
# 持续请求,只要还有待处理的工具调用就继续处理调用
while pending_tool_calls:
for tool_call in pending_tool_calls:
try:
func = tool_call.function
parameters = json.loads(func.arguments)
func_ret = await self.ap.tool_mgr.execute_func_call(
query, func.name, parameters
)
msg = llm_entities.Message(
role="tool", content=json.dumps(func_ret, ensure_ascii=False), tool_call_id=tool_call.id
)
yield msg
req_messages.append(msg)
except Exception as e:
# 工具调用出错,添加一个报错信息到 req_messages
err_msg = llm_entities.Message(
role="tool", content=f"err: {e}", tool_call_id=tool_call.id
)
yield err_msg
req_messages.append(err_msg)
# 处理完所有调用,再次请求
msg = await query.use_model.requester.call(query.use_model, req_messages, query.use_funcs)
yield msg
pending_tool_calls = msg.tool_calls
req_messages.append(msg)

View File

@@ -1,13 +1,13 @@
from __future__ import annotations
import typing
import mirai
from .. import handler
from ... import entities
from ....core import entities as core_entities
from ....provider import entities as llm_entities
from ....plugin import events
from ....platform.types import message as platform_message
class CommandHandler(handler.MessageHandler):
@@ -46,7 +46,7 @@ class CommandHandler(handler.MessageHandler):
if event_ctx.is_prevented_default():
if event_ctx.event.reply is not None:
mc = mirai.MessageChain(event_ctx.event.reply)
mc = platform_message.MessageChain(event_ctx.event.reply)
query.resp_messages.append(mc)
@@ -63,8 +63,8 @@ class CommandHandler(handler.MessageHandler):
else:
if event_ctx.event.alter is not None:
query.message_chain = mirai.MessageChain([
mirai.Plain(event_ctx.event.alter)
query.message_chain = platform_message.MessageChain([
platform_message.Plain(event_ctx.event.alter)
])
session = await self.ap.sess_mgr.get_session(query)

View File

@@ -42,7 +42,9 @@ class Processor(stage.PipelineStage):
self.ap.logger.info(f"处理 {query.launcher_type.value}_{query.launcher_id} 的请求({query.query_id}): {message_text}")
async def generator():
if message_text.startswith('!') or message_text.startswith(''):
cmd_prefix = self.ap.command_cfg.data['command-prefix']
if any(message_text.startswith(prefix) for prefix in cmd_prefix):
async for result in self.cmd_handler.handle(query):
yield result
else:

View File

@@ -1,18 +1,15 @@
# 固定窗口算法
from __future__ import annotations
import asyncio
import time
from .. import algo
# 固定窗口算法
class SessionContainer:
wait_lock: asyncio.Lock
records: dict[int, int]
"""访问记录key为每分钟的起始时间戳value为访问次数"""
"""访问记录key为每窗口长度的起始时间戳value为访问次数"""
def __init__(self):
self.wait_lock = asyncio.Lock()
@@ -47,30 +44,34 @@ class FixedWindowAlgo(algo.ReteLimitAlgo):
# 等待锁
async with container.wait_lock:
# 获取窗口大小和限制
window_size = self.ap.pipeline_cfg.data['rate-limit']['fixwin']['default']['window-size']
limitation = self.ap.pipeline_cfg.data['rate-limit']['fixwin']['default']['limit']
if session_name in self.ap.pipeline_cfg.data['rate-limit']['fixwin']:
window_size = self.ap.pipeline_cfg.data['rate-limit']['fixwin'][session_name]['window-size']
limitation = self.ap.pipeline_cfg.data['rate-limit']['fixwin'][session_name]['limit']
# 获取当前时间戳
now = int(time.time())
# 获取当前分钟的起始时间戳
now = now - now % 60
# 获取当前窗口的起始时间戳
now = now - now % window_size
# 获取当前分钟的访问次数
# 获取当前窗口的访问次数
count = container.records.get(now, 0)
limitation = self.ap.pipeline_cfg.data['rate-limit']['fixwin']['default']
if session_name in self.ap.pipeline_cfg.data['rate-limit']['fixwin']:
limitation = self.ap.pipeline_cfg.data['rate-limit']['fixwin'][session_name]
# 如果访问次数超过了限制
if count >= limitation:
if self.ap.pipeline_cfg.data['rate-limit']['strategy'] == 'drop':
return False
elif self.ap.pipeline_cfg.data['rate-limit']['strategy'] == 'wait':
# 等待下一分钟
await asyncio.sleep(60 - time.time() % 60)
# 等待下一窗口
await asyncio.sleep(window_size - time.time() % window_size)
now = int(time.time())
now = now - now % 60
now = now - now % window_size
if now not in container.records:
container.records = {}

View File

@@ -3,7 +3,6 @@ from __future__ import annotations
import random
import asyncio
import mirai
from ...core import app
@@ -20,7 +19,10 @@ class SendResponseBackStage(stage.PipelineStage):
async def process(self, query: core_entities.Query, stage_inst_name: str) -> entities.StageProcessResult:
"""处理
"""
random_delay = random.uniform(*self.ap.platform_cfg.data['force-delay'])
random_range = (self.ap.platform_cfg.data['force-delay']['min'], self.ap.platform_cfg.data['force-delay']['max'])
random_delay = random.uniform(*random_range)
self.ap.logger.debug(
"根据规则强制延迟回复: %s s",

View File

@@ -1,9 +1,10 @@
import pydantic
import mirai
from ...platform.types import message as platform_message
class RuleJudgeResult(pydantic.BaseModel):
matching: bool = False
replacement: mirai.MessageChain = None
replacement: platform_message.MessageChain = None

View File

@@ -1,6 +1,5 @@
from __future__ import annotations
import mirai
from ...core import app
from . import entities as rule_entities, rule
@@ -44,8 +43,8 @@ class GroupRespondRuleCheckStage(stage.PipelineStage):
use_rule = rules['default']
if str(query.launcher_id) in use_rule:
use_rule = use_rule[str(query.launcher_id)]
if str(query.launcher_id) in rules:
use_rule = rules[str(query.launcher_id)]
for rule_matcher in self.rule_matchers: # 任意一个匹配就放行
res = await rule_matcher.match(str(query.message_chain), query.message_chain, use_rule, query)

View File

@@ -2,11 +2,11 @@ from __future__ import annotations
import abc
import typing
import mirai
from ...core import app, entities as core_entities
from . import entities
from ...platform.types import message as platform_message
preregisetered_rules: list[typing.Type[GroupRespondRule]] = []
@@ -35,7 +35,7 @@ class GroupRespondRule(metaclass=abc.ABCMeta):
async def match(
self,
message_text: str,
message_chain: mirai.MessageChain,
message_chain: platform_message.MessageChain,
rule_dict: dict,
query: core_entities.Query
) -> entities.RuleJudgeResult:

Some files were not shown because too many files have changed in this diff Show More