doc: 增加长消息处理功能

增加百度云内容审核

.gitignore

@@ -3,10 +3,10 @@ config.py
 __pycache__/
 database.db
 qchatgpt.log
-config.py
 /banlist.py
 plugins/
 !plugins/__init__.py
 /revcfg.py
 prompts/
 logs/
+sensitive.json

CONTRIBUTING.md

@@ -0,0 +1,19 @@
+## 参与项目
+
+欢迎为此项目贡献代码或其他支持，以使您的点子或众人期待的功能成为现实，助力社区成长。  
+
+### 贡献形式
+
+- 提交PR，解决issues中提到的bug或期待的功能
+- 提交PR，实现您设想的功能（请先提出issue与作者沟通）
+- 优化代码架构，使各个模块的组织更加整洁优雅
+- 在issues中提出发现的bug或者期待的功能
+- 为本项目在其他社交平台撰写文章、制作视频等
+- 为本项目的衍生项目作出贡献，或开发插件增加功能
+
+### 如何开始
+
+- 加入本项目交流群，一同探讨项目相关事务
+- 解决本项目或衍生项目的issues中亟待解决的问题
+- 阅读并完善本项目文档
+- 在各个社交媒体撰写本项目教程等

README.md

@@ -1,36 +1,48 @@
 # QChatGPT🤖
-
-### 🎉现已支持接入ChatGPT网页版，详情请完成部署并查看底部**插件**小节或[此仓库](https://github.com/RockChinQ/revLibs)
+> 2023/3/3  官方接口疑似被墙，可考虑使用网络代理 [#198](https://github.com/RockChinQ/QChatGPT/issues/198)  
+> 2023/3/3  现已在主线支持官方ChatGPT接口，使用方法查看[#195](https://github.com/RockChinQ/QChatGPT/issues/195)  
+> 2023/3/2  OpenAI已发布ChatGPT官方接口，我们正在全力接入，预计明日前完成，请查看[此PR](https://github.com/RockChinQ/QChatGPT/pull/194)  
+> 2023/2/16 现已支持接入ChatGPT网页版，详情请完成部署并查看底部**插件**小节或[此仓库](https://github.com/RockChinQ/revLibs)
 
 - 到[项目Wiki](https://github.com/RockChinQ/QChatGPT/wiki)可了解项目详细信息
 - 由bilibili TheLazy制作的[视频教程](https://www.bilibili.com/video/BV15v4y1X7aP)
-- 测试号: 2196084348（已加载逆向库插件、每分钟限速）、~~1480613886（已加载逆向库插件）~~（被封）
-- 交流、答疑群: ~~204785790~~（已满）、691226829 
-  - **进群提问前请您`确保`已经找遍文档和issue均无法解决**  
-  - **进群提问前请您`确保`已经找遍文档和issue均无法解决**  
+- 交流、答疑群: ~~204785790~~（已满）、691226829、656285629  
   - **进群提问前请您`确保`已经找遍文档和issue均无法解决**  
 - QQ频道机器人见[QQChannelChatGPT](https://github.com/Soulter/QQChannelChatGPT)
 
-通过调用OpenAI GPT-3模型提供的Completion API来实现一个更加智能的QQ机器人  
+通过调用OpenAI的ChatGPT等语言模型来实现一个更加智能的QQ机器人  
+
+## 🍺模型适配一览
+
+### 文字对话
+
+- OpenAI GPT-3.5模型(ChatGPT API), 本项目原生支持, 默认使用
+- OpenAI GPT-3模型, 本项目原生支持, 部署完成后前往config.py切换
+- ChatGPT网页版逆向API, 由[插件](https://github.com/RockChinQ/revLibs)接入
+
+### 故事续写
+
+- NovelAI API, 由[插件](https://github.com/dominoar/QCPNovelAi)接入
+
+### 图片绘制
+
+- OpenAI DALL·E模型, 本项目原生支持, 使用方法查看[Wiki功能使用页](https://github.com/RockChinQ/QChatGPT/wiki/%E5%8A%9F%E8%83%BD%E4%BD%BF%E7%94%A8#%E5%8A%9F%E8%83%BD%E7%82%B9%E5%88%97%E4%B8%BE)
+- NovelAI API, 由[插件](https://github.com/dominoar/QCPNovelAi)接入
+
+### 语音生成
+
+- TTS+VITS, 由[插件](https://github.com/dominoar/QChatPlugins)接入
 
 ## ✅功能
 
-<details>
-<summary>✅回复符合上下文</summary>
-
-  - 程序向模型发送近几次对话内容，模型根据上下文生成回复
-  - 您可在`config.py`中修改`prompt_submit_length`自定义联系上下文的范围
-
-</details>
-
 <details>
 <summary>✅支持敏感词过滤，避免账号风险</summary>
 
   - 难以监测机器人与用户对话时的内容，故引入此功能以减少机器人风险
+  - 加入了百度云内容审核，在`config.py`中修改`baidu_check`的值，并填写`baidu_api_key`和`baidu_secret_key`以开启此功能
   - 编辑`sensitive.json`，并在`config.py`中修改`sensitive_word_filter`的值以开启此功能
 </details>
 
-
 <details>
 <summary>✅群内多种响应规则，不必at</summary>
 
@@ -38,14 +50,6 @@
   - 详细见`config.py`中的`response_rules`字段
 </details>
 
-<details>
-<summary>✅使用官方api，不需要网络代理，稳定快捷</summary>
-
-  - 不使用ChatGPT逆向接口，而使用官方的Completion API，稳定性高
-  - 您可以在`config.py`中自定义`completion_api_params`字段，设置向官方API提交的参数以自定义机器人的风格
-
-</details>
-
 <details>
 <summary>✅完善的多api-key管理，超额自动切换</summary>
 
@@ -55,13 +59,6 @@
   - 运行期间向机器人说`!usage`以查看当前使用情况
 </details>
 
-<details>
-<summary>✅组件少，部署方便，提供一键安装器及Docker安装</summary>
-
-  - 手动部署步骤少
-  - 提供自动安装器及docker方式，详见以下安装步骤
-</details>
-
 <details>
 <summary>✅支持预设指令文字</summary>
 
@@ -70,13 +67,6 @@
   - 支持设置多个预设情景，并通过!reset、!default等指令控制，详细请查看[wiki指令](https://github.com/RockChinQ/QChatGPT/wiki/%E5%8A%9F%E8%83%BD%E4%BD%BF%E7%94%A8#%E6%9C%BA%E5%99%A8%E4%BA%BA%E6%8C%87%E4%BB%A4)
 </details>
 
-<details>
-<summary>✅完善的会话管理，重启不丢失</summary>
-
-  - 使用SQLite进行会话内容持久化
-  - 最后一次对话一定时间后自动保存，请到`config.py`中修改`session_expire_time`的值以自定义时间
-  - 运行期间可使用`!reset` `!list` `!last` `!next` `!prompt`等指令管理会话
-</details>
 <details>
 <summary>✅支持对话、绘图等模型，可玩性更高</summary>
 
@@ -102,6 +92,12 @@
   - 详见Wiki`加入黑名单`节
 </details>
 <details>
+<summary>✅长消息处理策略</summary>
+
+  - 支持将长消息转换成图片或消息记录组件，避免消息刷屏
+  - 请查看`config.py`中`blob_message_strategy`等字段
+</details>
+<details>
 <summary>✅回复速度限制</summary>
 
   - 支持限制单会话内每分钟可进行的对话次数
@@ -149,9 +145,6 @@
 <summary>手动部署适用于所有平台</summary>
 
 - 请使用Python 3.9.x以上版本   
-- 请注意OpenAI账号额度消耗  
-  - 每个账户仅有18美元免费额度，如未绑定银行卡，则会在超出时报错  
-  - OpenAI收费标准：默认使用的`text-davinci-003`模型 0.02美元/千字  
 
 #### 配置Mirai
 
@@ -194,7 +187,7 @@ python3 main.py
 
 **常见问题**
 
-- mirai登录提示`QQ版本过低`，见[此issue](https://github.com/RockChinQ/QChatGPT/issues/38)
+- mirai登录提示`QQ版本过低`，见[此issue](https://github.com/RockChinQ/QChatGPT/issues/137)
 - 如提示安装`uvicorn`或`hypercorn`请*不要*安装，这两个不是必需的，目前存在未知原因bug
 - 如报错`TypeError: As of 3.10, the *loop* parameter was removed from Lock() since it is no longer necessary`, 请参考 [此处](https://github.com/RockChinQ/QChatGPT/issues/5)
 
@@ -224,7 +217,7 @@ python3 main.py
 
 - [revLibs](https://github.com/RockChinQ/revLibs) - 将ChatGPT网页版接入此项目，关于[官方接口和网页版有什么区别](https://github.com/RockChinQ/QChatGPT/wiki/%E5%AE%98%E6%96%B9%E6%8E%A5%E5%8F%A3%E4%B8%8EChatGPT%E7%BD%91%E9%A1%B5%E7%89%88)
 - [hello_plugin](https://github.com/RockChinQ/hello_plugin) - `hello_plugin` 的储存库形式，插件开发模板
-- [dominoar/QchatPlugins](https://github.com/dominoar/QchatPlugins) - dominoar编写的诸多新功能插件（语言输出、Ranimg、屏蔽词规则等）
+- [dominoar/QChatPlugins](https://github.com/dominoar/QchatPlugins) - dominoar编写的诸多新功能插件（语言输出、Ranimg、屏蔽词规则等）
 - [dominoar/QCP-NovelAi](https://github.com/dominoar/QCP-NovelAi) - NovelAI 故事叙述与绘画
 
 ## 😘致谢
@@ -233,6 +226,7 @@ python3 main.py
 - [@mikumifa](https://github.com/mikumifa) 本项目Docker部署仓库开发者
 - [@dominoar](https://github.com/dominoar) 为本项目开发多种插件
 - [@hissincn](https://github.com/hissincn) 本项目贡献者
+- [@LINSTCL](https://github.com/LINSTCL)   GPT-3.5官方模型适配贡献者
 
 以及其他所有为本项目提供支持的朋友们。
 

config-template.py

@@ -20,6 +20,7 @@ mirai_http_api_config = {
 
 # [必需] OpenAI的配置
 # api_key: OpenAI的API Key
+# http_proxy: 请求OpenAI时使用的代理，None为不使用，https和socks5暂不能使用
 # 若只有一个api-key，请直接修改以下内容中的"openai_api_key"为你的api-key
 #
 # 如准备了多个api-key，可以以字典的形式填写，程序会自动选择可用的api-key
@@ -30,11 +31,13 @@ mirai_http_api_config = {
 #         "key1": "sk-xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx",
 #         "key2": "sk-xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx",
 #     },
+#     "http_proxy": "http://127.0.0.1:12345"
 # }
 openai_config = {
     "api_key": {
         "default": "openai_api_key"
     },
+    "http_proxy": None
 }
 
 # [必需] 管理员QQ号，用于接收报错等通知及执行管理员级别指令
@@ -99,10 +102,27 @@ ignore_rules = {
     "regexp": []
 }
 
+# 是否检查收到的消息中是否包含敏感词
+# 若收到的消息无法通过下方指定的敏感词检查策略，则发送提示信息
+income_msg_check = False
+
 # 敏感词过滤开关，以同样数量的*代替敏感词回复
 # 请在sensitive.json中添加敏感词
 sensitive_word_filter = True
 
+# 是否启用百度云内容安全审核
+# 注册方式查看 https://cloud.baidu.com/doc/ANTIPORN/s/Wkhu9d5iy
+baidu_check = False
+
+# 百度云API_KEY 24位英文数字字符串
+baidu_api_key = ""
+
+# 百度云SECRET_KEY 32位的英文数字字符串
+baidu_secret_key = ""
+
+# 不合规消息自定义返回
+inappropriate_message_tips = "[百度云]请珍惜机器人，当前返回内容不合规"
+
 # 启动时是否发送赞赏码
 # 仅当使用量已经超过2048字时发送
 encourage_sponsor_at_start = True
@@ -112,12 +132,25 @@ encourage_sponsor_at_start = True
 # 注意：较大的prompt_submit_length会导致OpenAI账户额度消耗更快
 prompt_submit_length = 1024
 
-# OpenAI的completion API的参数
+# OpenAI补全API的参数
+# 请在下方填写模型，程序自动选择接口
+# 现已支持的模型有：
+# 
+#    'gpt-3.5-turbo'
+#    'gpt-3.5-turbo-0301'
+#    'text-davinci-003'
+#    'text-davinci-002'
+#    'code-davinci-002'
+#    'code-cushman-001'
+#    'text-curie-001'
+#    'text-babbage-001'
+#    'text-ada-001'
+#
 # 具体请查看OpenAI的文档: https://beta.openai.com/docs/api-reference/completions/create
 completion_api_params = {
-    "model": "text-davinci-003",
+    "model": "gpt-3.5-turbo",
     "temperature": 0.9,  # 数值越低得到的回答越理性，取值范围[0, 1]
-    "max_tokens": 512,  # 每次获取OpenAI接口响应的文字量上限, 不高于4096
+    "max_tokens": 1024,  # 每次获取OpenAI接口响应的文字量上限, 不高于4096
     "top_p": 1,  # 生成的文本的文本与要求的符合度, 取值范围[0, 1]
     "frequency_penalty": 0.2,
     "presence_penalty": 1.0,
@@ -138,21 +171,18 @@ include_image_description = True
 # 消息处理的超时时间，单位为秒
 process_message_timeout = 30
 
-# 会话对象名称，此配置与会话对象管理相关，
-# 若不了解相关功能，无需修改此配置
-# 详细说明请查看：https://github.com/RockChinQ/QChatGPT/wiki/%E6%8A%80%E6%9C%AF%E4%BF%A1%E6%81%AF#%E4%BC%9A%E8%AF%9Dsession
-# user_name: 管理员(主人)的名字
-# bot_name: 机器人的名字
-user_name = 'You'
-bot_name = 'Bot'
-
-# [暂未实现] 群内会话是否启用多对象名称
-# 若不启用，群内会话的prompt只使用user_name和bot_name
-multi_subject = False
-
 # 回复消息时是否显示[GPT]前缀
 show_prefix = False
 
+# 应用长消息处理策略的阈值
+# 当回复消息长度超过此值时，将使用长消息处理策略
+blob_message_threshold = 256
+
+# 长消息处理策略
+# - "image": 将长消息转换为图片发送
+# - "forward": 将长消息转换为转发消息组件发送
+blob_message_strategy = "forward"
+
 # 消息处理超时重试次数
 retry_times = 3
 
@@ -186,6 +216,9 @@ rate_limit_strategy = "wait"
 # 若设置为空字符串，则不发送提示信息
 rate_limit_drop_tip = "本分钟对话次数超过限速次数，此对话被丢弃"
 
+# 是否在启动时进行依赖库更新
+upgrade_dependencies = True
+
 # 是否上报统计信息
 # 用于统计机器人的使用情况，不会收集任何用户信息
 # 仅上报时间、字数使用量、绘图使用量，其他信息不会上报

main.py

@@ -43,6 +43,11 @@ def init_db():
     database.initialize_database()
 
 
+def ensure_dependencies():
+    import pkg.utils.pkgmgr as pkgmgr
+    pkgmgr.run_pip(["install", "openai", "Pillow", "--upgrade"])
+
+
 known_exception_caught = False
 
 log_file_name = "qchatgpt.log"
@@ -309,6 +314,10 @@ if __name__ == '__main__':
     if not os.path.exists('banlist.py'):
         shutil.copy('banlist-template.py', 'banlist.py')
 
+    # 检查是否有sensitive.json,
+    if not os.path.exists("sensitive.json"):
+        shutil.copy("sensitive-template.json", "sensitive.json")
+
     if len(sys.argv) > 1 and sys.argv[1] == 'init_db':
         init_db()
         sys.exit(0)
@@ -333,4 +342,18 @@ if __name__ == '__main__':
     #
     # pkg.utils.configmgr.set_config_and_reload("quote_origin", False)
     requests.packages.urllib3.disable_warnings(InsecureRequestWarning)
+
+    import config
+    # 更新openai库到最新版本
+    if not hasattr(config, 'upgrade_dependencies') or config.upgrade_dependencies:
+        print("正在更新依赖库，请等待...")
+        if not hasattr(config, 'upgrade_dependencies'):
+            print("这个操作不是必须的,如果不想更新,请在config.py中添加upgrade_dependencies=False")
+        else:
+            print("这个操作不是必须的,如果不想更新,请在config.py中将upgrade_dependencies设置为False")
+        try:
+            ensure_dependencies()
+        except Exception as e:
+            print("更新openai库失败:{}, 请忽略或自行更新".format(e))
+
     main(True)

pkg/audit/gatherer.py

@@ -20,7 +20,7 @@ class DataGatherer:
         }
     }为值的字典"""
 
-    version_str = "0.1.0"
+    version_str = "undetermined"
 
     def __init__(self):
         self.load_from_db()

pkg/database/manager.py

@@ -206,7 +206,7 @@ class DatabaseManager:
         }
 
     # 列出与某个对象的所有对话session
-    def list_history(self, session_name: str, capacity: int, page: int, replace: str = ""):
+    def list_history(self, session_name: str, capacity: int, page: int):
         self.execute("""
         select `name`, `type`, `number`, `create_timestamp`, `last_interact_timestamp`, `prompt`, `status`
         from `sessions` where `name` = '{}' order by `last_interact_timestamp` desc limit {} offset {}
@@ -227,7 +227,7 @@ class DatabaseManager:
                 'subject_number': subject_number,
                 'create_timestamp': create_timestamp,
                 'last_interact_timestamp': last_interact_timestamp,
-                'prompt': prompt if replace == "" else prompt.replace(replace, "")
+                'prompt': prompt
             })
 
         return sessions

pkg/openai/manager.py

@@ -5,7 +5,7 @@ import openai
 import pkg.openai.keymgr
 import pkg.utils.context
 import pkg.audit.gatherer
-
+from pkg.openai.modelmgr import ModelRequest, create_openai_model_request
 
 # 为其他模块提供与OpenAI交互的接口
 class OpenAIInteract:
@@ -32,24 +32,31 @@ class OpenAIInteract:
         pkg.utils.context.set_openai_manager(self)
 
     # 请求OpenAI Completion
-    def request_completion(self, prompt, stop):
+    def request_completion(self, prompts):
         config = pkg.utils.context.get_config()
-        response = openai.Completion.create(
-            prompt=prompt,
-            stop=stop,
+
+        # 根据模型选择使用的接口
+        ai: ModelRequest = create_openai_model_request(
+            config.completion_api_params['model'], 
+            'user', 
+            config.openai_config["http_proxy"] if "http_proxy" in config.openai_config else None
+        )
+        ai.request(
+            prompts,
             **config.completion_api_params
         )
+        response = ai.get_response()
 
         logging.debug("OpenAI response: %s", response)
 
         if 'model' in config.completion_api_params:
             self.audit_mgr.report_text_model_usage(config.completion_api_params['model'],
-                                                   response['usage']['total_tokens'])
+                                                   ai.get_total_tokens())
         elif 'engine' in config.completion_api_params:
             self.audit_mgr.report_text_model_usage(config.completion_api_params['engine'],
                                                    response['usage']['total_tokens'])
 
-        return response
+        return ai.get_message()
 
     def request_image(self, prompt):
 

pkg/openai/modelmgr.py

@@ -1,7 +1,19 @@
 # 提供与模型交互的抽象接口
+import openai, logging, threading, asyncio
 
 COMPLETION_MODELS = {
-    'text-davinci-003'
+    'text-davinci-003',
+    'text-davinci-002',
+    'code-davinci-002',
+    'code-cushman-001',
+    'text-curie-001',
+    'text-babbage-001',
+    'text-ada-001',
+}
+
+CHAT_COMPLETION_MODELS = {
+    'gpt-3.5-turbo',
+    'gpt-3.5-turbo-0301',
 }
 
 EDIT_MODELS = {
@@ -13,22 +25,128 @@ IMAGE_MODELS = {
 }
 
 
-# ModelManager
-# 由session包含
-class ModelMgr(object):
+class ModelRequest():
+    """GPT父类"""
+    can_chat = False
+    runtime:threading.Thread = None
+    ret = ""
+    proxy:str = None
 
-    using_completion_model = ""
-    using_edit_model = ""
-    using_image_model = ""
+    def __init__(self, model_name, user_name, request_fun, http_proxy:str = None):
+        self.model_name = model_name
+        self.user_name = user_name
+        self.request_fun = request_fun
+        if http_proxy != None:
+            self.proxy = http_proxy
+            openai.proxy = self.proxy
 
-    def __init__(self):
-        pass
+    async def __a_request__(self, **kwargs):
+        self.ret = await self.request_fun(**kwargs)
 
-    def get_using_completion_model(self):
-        return self.using_completion_model
+    def request(self, **kwargs):
+        if self.proxy != None: #异步请求
+            loop = asyncio.new_event_loop()
+            self.runtime = threading.Thread(
+                target=loop.run_until_complete,
+                args=(self.__a_request__(**kwargs),)
+            )
+            self.runtime.start()
+        else: #同步请求
+            self.ret = self.request_fun(**kwargs)
 
-    def get_using_edit_model(self):
-        return self.using_edit_model
+    def __msg_handle__(self, msg):
+        """将prompt dict转换成接口需要的格式"""
+        return msg
     
-    def get_using_image_model(self):
-        return self.using_image_model
+    def ret_handle(self):
+        '''
+        API消息返回处理函数
+        若重写该方法，应检查异步线程状态，或在需要检查处super该方法
+        '''
+        if self.runtime != None and isinstance(self.runtime, threading.Thread):
+            self.runtime.join()
+        return
+
+    def get_total_tokens(self):
+        try:
+            return self.ret['usage']['total_tokens']
+        except Exception:
+            return 0
+
+    def get_message(self):
+        return self.message
+
+    def get_response(self):
+        return self.ret
+
+class ChatCompletionModel(ModelRequest):
+    """ChatCompletion类模型"""
+    Chat_role = ['system', 'user', 'assistant']
+    def __init__(self, model_name, user_name, http_proxy:str = None, **kwargs):
+        if http_proxy == None:
+            request_fun = openai.ChatCompletion.create
+        else:
+            request_fun = openai.ChatCompletion.acreate
+        self.can_chat = True
+        super().__init__(model_name, user_name, request_fun, http_proxy, **kwargs)
+
+    def request(self, prompts, **kwargs):
+        prompts = self.__msg_handle__(prompts)
+        kwargs['messages'] = prompts
+        super().request(**kwargs)
+        self.ret_handle()
+
+    def __msg_handle__(self, msgs):
+        temp_msgs = []
+        # 把msgs拷贝进temp_msgs
+        for msg in msgs:
+            temp_msgs.append(msg.copy())
+        return temp_msgs
+
+    def get_message(self):
+        return self.ret["choices"][0]["message"]['content'] #需要时直接加载加快请求速度，降低内存消耗
+
+
+class CompletionModel(ModelRequest):
+    """Completion类模型"""
+    def __init__(self, model_name, user_name, http_proxy:str = None, **kwargs):
+        if http_proxy == None:
+            request_fun = openai.Completion.create
+        else:
+            request_fun = openai.Completion.acreate
+        super().__init__(model_name, user_name, request_fun, http_proxy, **kwargs)
+
+    def request(self, prompts, **kwargs):
+        prompts = self.__msg_handle__(prompts)
+        kwargs['prompt'] = prompts
+        super().request(**kwargs)
+        self.ret_handle()
+
+    def __msg_handle__(self, msgs):
+        prompt = ''
+        for msg in msgs:
+            prompt = prompt + "{}: {}\n".format(msg['role'], msg['content'])
+        # for msg in msgs:
+        #     if msg['role'] == 'assistant':
+        #         prompt = prompt + "{}\n".format(msg['content'])
+        #     else:
+        #         prompt = prompt + "{}:{}\n".format(msg['role'] , msg['content'])
+        prompt = prompt + "assistant: "
+        return prompt
+
+    def get_message(self):
+        return self.ret["choices"][0]["text"]
+
+
+def create_openai_model_request(model_name: str, user_name: str = 'user', http_proxy:str = None) -> ModelRequest:
+    """使用给定的模型名称创建模型请求对象"""
+    if model_name in CHAT_COMPLETION_MODELS:
+        model = ChatCompletionModel(model_name, user_name, http_proxy)
+    elif model_name in COMPLETION_MODELS:
+        model = CompletionModel(model_name, user_name, http_proxy)
+    else :
+        log = "找不到模型[{}]，请检查配置文件".format(model_name)
+        logging.error(log)
+        raise IndexError(log)
+    logging.debug("使用接口[{}]创建模型请求[{}]".format(model.__class__.__name__, model_name))
+    return model

pkg/openai/session.py

@@ -1,8 +1,10 @@
 import logging
 import threading
 import time
+import json
 
 import pkg.openai.manager
+import pkg.openai.modelmgr
 import pkg.database.manager
 import pkg.utils.context
 
@@ -17,6 +19,32 @@ class SessionOfflineStatus:
     ON_GOING = 'on_going'
     EXPLICITLY_CLOSED = 'explicitly_closed'
 
+# 重置session.prompt
+def reset_session_prompt(session_name, prompt):
+    # 备份原始数据
+    bak_path = 'logs/{}-{}.bak'.format(
+        session_name,
+        time.strftime("%Y-%m-%d-%H-%M-%S", time.localtime())
+    )
+    f = open(bak_path, 'w+')
+    f.write(prompt)
+    f.close()
+    # 生成新数据
+    config = pkg.utils.context.get_config()
+    prompt = [
+        {
+            'role': 'system',
+            'content': config.default_prompt['default']
+        }
+    ]
+    # 警告
+    logging.warning(
+        """
+用户[{}]的数据已被重置，有可能是因为数据版本过旧或存储错误
+原始数据将备份在：
+{}""".format(session_name, bak_path)
+    )
+    return prompt
 
 # 从数据加载session
 def load_sessions():
@@ -33,7 +61,11 @@ def load_sessions():
         temp_session.name = session_name
         temp_session.create_timestamp = session_data[session_name]['create_timestamp']
         temp_session.last_interact_timestamp = session_data[session_name]['last_interact_timestamp']
-        temp_session.prompt = session_data[session_name]['prompt']
+        try:
+            temp_session.prompt = json.loads(session_data[session_name]['prompt'])
+        except Exception:
+            temp_session.prompt = reset_session_prompt(session_name, session_data[session_name]['prompt'])
+            temp_session.persistence()
 
         sessions[session_name] = temp_session
 
@@ -60,12 +92,7 @@ def dump_session(session_name: str):
 class Session:
     name = ''
 
-    prompt = ""
-
-    import config
-
-    user_name = config.user_name if hasattr(config, 'user_name') and config.user_name != '' else 'You'
-    bot_name = config.bot_name if hasattr(config, 'bot_name') and config.bot_name != '' else 'Bot'
+    prompt = []
 
     create_timestamp = 0
 
@@ -99,11 +126,15 @@ class Session:
         else:
             current_default_prompt = dprompt.get_prompt(use_default)
 
-        user_name = config.user_name if hasattr(config, 'user_name') and config.user_name != '' else 'You'
-        bot_name = config.bot_name if hasattr(config, 'bot_name') and config.bot_name != '' else 'Bot'
-
-        return (user_name + ":{}\n".format(current_default_prompt) + bot_name + ":好的\n") \
-            if current_default_prompt != '' else ''
+        return [
+            {
+                'role': 'user',
+                'content': current_default_prompt
+            },{
+                'role': 'assistant',
+                'content': 'ok'
+            }
+        ]
 
     def __init__(self, name: str):
         self.name = name
@@ -165,22 +196,16 @@ class Session:
             if event.is_prevented_default():
                 return None
 
-        # max_rounds = config.prompt_submit_round_amount if hasattr(config, 'prompt_submit_round_amount') else 7
         config = pkg.utils.context.get_config()
-        max_rounds = 1000  # 不再限制回合数
         max_length = config.prompt_submit_length if hasattr(config, "prompt_submit_length") else 1024
 
         # 向API请求补全
-        response = pkg.utils.context.get_openai_manager().request_completion(
-            self.cut_out(self.prompt + self.user_name + ':' +
-                         text + '\n' + self.bot_name + ':',
-                         max_rounds, max_length),
-            self.user_name + ':')
+        message = pkg.utils.context.get_openai_manager().request_completion(
+            self.cut_out(text, max_length),
+        )
 
-        self.prompt += self.user_name + ':' + text + '\n' + self.bot_name + ':'
-        # print(response)
-        # 处理回复
-        res_test = response["choices"][0]["text"]
+        # 成功获取，处理回复
+        res_test = message
         res_ans = res_test
 
         # 去除开头可能的提示
@@ -189,50 +214,55 @@ class Session:
             del (res_ans_spt[0])
             res_ans = '\n\n'.join(res_ans_spt)
 
-        self.prompt += "{}".format(res_ans) + '\n'
+        # 将此次对话的双方内容加入到prompt中
+        self.prompt.append({'role':'user', 'content':text})
+        self.prompt.append({'role':'assistant', 'content':res_ans})
 
         if self.just_switched_to_exist_session:
             self.just_switched_to_exist_session = False
             self.set_ongoing()
 
-        return res_ans
+        return res_ans if res_ans[0]!='\n' else res_ans[1:]
 
     # 删除上一回合并返回上一回合的问题
     def undo(self) -> str:
         self.last_interact_timestamp = int(time.time())
 
-        # 删除上一回合
-        to_delete = self.cut_out(self.prompt, 1, 1024)
+        # 删除最后两个消息
+        if len(self.prompt) < 2:
+            raise Exception('之前无对话，无法撤销')
     
-        self.prompt = self.prompt.replace(to_delete, '')
+        question = self.prompt[-2]['content']
+        self.prompt = self.prompt[:-2]
         
         # 返回上一回合的问题
-        return to_delete.split(self.bot_name + ':')[0].split(self.user_name + ':')[1].strip()
+        return question
 
-    # 从尾部截取prompt里不多于max_rounds个回合，长度不大于max_tokens的字符串
-    # 保证都是完整的对话
-    def cut_out(self, prompt: str, max_rounds: int, max_tokens: int) -> str:
-        # 分隔出每个回合
-        rounds_spt_by_user_name = prompt.split(self.user_name + ':')
+    # 构建对话体
+    def cut_out(self, msg: str, max_tokens: int) -> list:
+        """将现有prompt进行切割处理，使得新的prompt长度不超过max_tokens"""
+        # 如果用户消息长度超过max_tokens，直接返回
         
-        result = ''
+        temp_prompt = [
+                {
+                    'role': 'user',
+                    'content': msg
+                }
+            ]
 
-        checked_rounds = 0
-        # 从后往前遍历，加到result前面，检查result是否符合要求
-        for i in range(len(rounds_spt_by_user_name) - 1, 0, -1):
-            result_temp = self.user_name + ':' + rounds_spt_by_user_name[i] + result
-            checked_rounds += 1
-
-            if checked_rounds > max_rounds:
+        token_count = len(msg)
+        # 倒序遍历prompt
+        for i in range(len(self.prompt) - 1, -1, -1):
+            if token_count >= max_tokens:
                 break
 
-            if int((len(result_temp.encode('utf-8')) - len(result_temp)) / 2 + len(result_temp)) > max_tokens:
-                break
+            # 将prompt加到temp_prompt头部
+            temp_prompt.insert(0, self.prompt[i])
+            token_count += len(self.prompt[i]['content'])
 
-            result = result_temp
+        logging.debug('cut_out: {}'.format(str(temp_prompt)))
 
-        logging.debug('cut_out: {}'.format(result))
-        return result
+        return temp_prompt
 
     # 持久化session
     def persistence(self):
@@ -247,11 +277,11 @@ class Session:
         subject_number = int(name_spt[1])
 
         db_inst.persistence_session(subject_type, subject_number, self.create_timestamp, self.last_interact_timestamp,
-                                    self.prompt)
+                                    json.dumps(self.prompt))
 
     # 重置session
     def reset(self, explicit: bool = False, expired: bool = False, schedule_new: bool = True, use_prompt: str = None):
-        if not self.prompt.endswith(':好的\n'):
+        if self.prompt[-1]['role'] != "system":
             self.persistence()
             if explicit:
                 # 触发插件事件
@@ -291,7 +321,11 @@ class Session:
 
             self.create_timestamp = last_one['create_timestamp']
             self.last_interact_timestamp = last_one['last_interact_timestamp']
-            self.prompt = last_one['prompt']
+            try:
+                self.prompt = json.loads(last_one['prompt'])
+            except json.decoder.JSONDecodeError:
+                self.prompt = reset_session_prompt(self.name, last_one['prompt'])
+                self.persistence()
 
             self.just_switched_to_exist_session = True
             return self
@@ -306,14 +340,17 @@ class Session:
 
             self.create_timestamp = next_one['create_timestamp']
             self.last_interact_timestamp = next_one['last_interact_timestamp']
-            self.prompt = next_one['prompt']
+            try:
+                self.prompt = json.loads(next_one['prompt'])
+            except json.decoder.JSONDecodeError:
+                self.prompt = reset_session_prompt(self.name, next_one['prompt'])
+                self.persistence()
 
             self.just_switched_to_exist_session = True
             return self
 
     def list_history(self, capacity: int = 10, page: int = 0):
-        return pkg.utils.context.get_database_manager().list_history(self.name, capacity, page,
-                                                                     self.get_default_prompt())
+        return pkg.utils.context.get_database_manager().list_history(self.name, capacity, page)
 
     def draw_image(self, prompt: str):
         return pkg.utils.context.get_openai_manager().request_image(prompt)

pkg/qqbot/banlist.py

@@ -1,30 +1,34 @@
 import pkg.utils.context
 
 
-def is_banned(launcher_type: str, launcher_id: int) -> bool:
+def is_banned(launcher_type: str, launcher_id: int, sender_id: int) -> bool:
     if not pkg.utils.context.get_qqbot_manager().enable_banlist:
         return False
     
     result = False
     
     if launcher_type == 'group':
-        for group_rule in pkg.utils.context.get_qqbot_manager().ban_group:
-            if type(group_rule) == int:
-                if group_rule == launcher_id: # 此群群号被禁用
-                    result = True
-            elif type(group_rule) == str:
-                if group_rule.startswith('!'):
-                    # 截取!后面的字符串作为表达式，判断是否匹配
-                    reg_str = group_rule[1:]
-                    import re
-                    if re.match(reg_str, str(launcher_id)): # 被豁免，最高级别
-                        result = False
-                        break
-                else:
-                    # 判断是否匹配regexp
-                    import re
-                    if re.match(group_rule, str(launcher_id)): # 此群群号被禁用
+        # 检查是否显式声明发起人QQ要被person忽略
+        if sender_id in pkg.utils.context.get_qqbot_manager().ban_person:
+            result = True
+        else:
+            for group_rule in pkg.utils.context.get_qqbot_manager().ban_group:
+                if type(group_rule) == int:
+                    if group_rule == launcher_id:  # 此群群号被禁用
                         result = True
+                elif type(group_rule) == str:
+                    if group_rule.startswith('!'):
+                        # 截取!后面的字符串作为表达式，判断是否匹配
+                        reg_str = group_rule[1:]
+                        import re
+                        if re.match(reg_str, str(launcher_id)):  # 被豁免，最高级别
+                            result = False
+                            break
+                    else:
+                        # 判断是否匹配regexp
+                        import re
+                        if re.match(group_rule, str(launcher_id)):  # 此群群号被禁用
+                            result = True
                     
     else:
         # ban_person, 与群规则相同

pkg/qqbot/blob.py

@@ -0,0 +1,105 @@
+# 长消息处理相关
+import logging
+import os
+import time
+import base64
+
+import config
+from mirai.models.message import MessageComponent, MessageChain, Image
+from mirai.models.message import ForwardMessageNode
+from mirai.models.base import MiraiBaseModel
+from typing import List
+import pkg.utils.context as context
+import pkg.utils.text2img as text2img
+
+
+class ForwardMessageDiaplay(MiraiBaseModel):
+    title: str = "群聊的聊天记录"
+    brief: str = "[聊天记录]"
+    source: str = "聊天记录"
+    preview: List[str] = []
+    summary: str = "查看x条转发消息"
+
+
+class Forward(MessageComponent):
+    """合并转发。"""
+    type: str = "Forward"
+    """消息组件类型。"""
+    display: ForwardMessageDiaplay
+    """显示信息"""
+    node_list: List[ForwardMessageNode]
+    """转发消息节点列表。"""
+    def __init__(self, *args, **kwargs):
+        if len(args) == 1:
+            self.node_list = args[0]
+            super().__init__(**kwargs)
+        super().__init__(*args, **kwargs)
+
+    def __str__(self):
+        return '[聊天记录]'
+
+
+def text_to_image(text: str) -> MessageComponent:
+    """将文本转换成图片"""
+    # 检查temp文件夹是否存在
+    if not os.path.exists('temp'):
+        os.mkdir('temp')
+    img_path = text2img.text_to_image(text_str=text, save_as='temp/{}.png'.format(int(time.time())))
+    
+    compressed_path, size = text2img.compress_image(img_path, outfile="temp/{}_compressed.png".format(int(time.time())))
+    # 读取图片，转换成base64
+    with open(compressed_path, 'rb') as f:
+        img = f.read()
+
+    b64 = base64.b64encode(img)
+
+    # 删除图片
+    os.remove(img_path)
+
+    # 判断compressed_path是否存在
+    if os.path.exists(compressed_path):
+        os.remove(compressed_path)
+    # 返回图片
+    return Image(base64=b64.decode('utf-8'))
+
+
+def check_text(text: str) -> list:
+    """检查文本是否为长消息，并转换成该使用的消息链组件"""
+    if not hasattr(config, 'blob_message_threshold'):
+        return [text]
+    
+    if len(text) > config.blob_message_threshold:
+        if not hasattr(config, 'blob_message_strategy'):
+            raise AttributeError('未定义长消息处理策略')
+
+        # logging.info("长消息: {}".format(text))
+        if config.blob_message_strategy == 'image':
+            # 转换成图片
+            return [text_to_image(text)]
+        elif config.blob_message_strategy == 'forward':
+            # 敏感词屏蔽
+            text = context.get_qqbot_manager().reply_filter.process(text)
+
+            # 包装转发消息
+            display = ForwardMessageDiaplay(
+                title='群聊的聊天记录',
+                brief='[聊天记录]',
+                source='聊天记录',
+                preview=["bot: "+text],
+                summary="查看1条转发消息"
+            )
+
+            node = ForwardMessageNode(
+                sender_id=config.mirai_http_api_config['qq'],
+                sender_name='bot',
+                message_chain=MessageChain([text])
+            )
+
+            forward = Forward(
+                display=display,
+                node_list=[node]
+            )
+
+            return [forward]
+    else:
+        return [text]

pkg/qqbot/command.py

@@ -185,11 +185,7 @@ def process_command(session_name: str, text_message: str, mgr, config,
             else:
                 datetime_str = datetime.datetime.fromtimestamp(result.create_timestamp).strftime(
                     '%Y-%m-%d %H:%M:%S')
-                reply = ["[bot]已切换到前一次的对话：\n创建时间:{}\n".format(
-                    datetime_str) + result.prompt[
-                                    :min(100,
-                                         len(result.prompt))] + \
-                         ("..." if len(result.prompt) > 100 else "#END#")]
+                reply = ["[bot]已切换到前一次的对话：\n创建时间:{}\n".format(datetime_str)]
         elif cmd == 'next':
             result = pkg.openai.session.get_session(session_name).next_session()
             if result is None:
@@ -197,13 +193,18 @@ def process_command(session_name: str, text_message: str, mgr, config,
             else:
                 datetime_str = datetime.datetime.fromtimestamp(result.create_timestamp).strftime(
                     '%Y-%m-%d %H:%M:%S')
-                reply = ["[bot]已切换到后一次的对话：\n创建时间:{}\n".format(
-                    datetime_str) + result.prompt[
-                                    :min(100,
-                                         len(result.prompt))] + \
-                         ("..." if len(result.prompt) > 100 else "#END#")]
+                reply = ["[bot]已切换到后一次的对话：\n创建时间:{}\n".format(datetime_str)]
         elif cmd == 'prompt':
-            reply = ["[bot]当前对话所有内容：\n" + pkg.openai.session.get_session(session_name).prompt]
+            msgs = ""
+            session:list = pkg.openai.session.get_session(session_name).prompt
+            for msg in session:
+                if len(params) != 0 and params[0] in ['-all', '-a']:
+                    msgs = msgs + "{}: {}\n\n".format(msg['role'], msg['content'])
+                elif len(msg['content']) > 30:
+                    msgs = msgs + "[{}]: {}...\n\n".format(msg['role'], msg['content'][:30])
+                else:
+                    msgs = msgs + "[{}]: {}\n\n".format(msg['role'], msg['content'])
+            reply = ["[bot]当前对话所有内容：\n{}".format(msgs)]
         elif cmd == 'list':
             pkg.openai.session.get_session(session_name).persistence()
             page = 0
@@ -223,10 +224,21 @@ def process_command(session_name: str, text_message: str, mgr, config,
                 for i in range(len(results)):
                     # 时间(使用create_timestamp转换) 序号 部分内容
                     datetime_obj = datetime.datetime.fromtimestamp(results[i]['create_timestamp'])
-                    reply_str += "#{} 创建:{} {}\n".format(i + page * 10,
-                                                           datetime_obj.strftime("%Y-%m-%d %H:%M:%S"),
-                                                           results[i]['prompt'][
-                                                           :min(20, len(results[i]['prompt']))])
+                    msg = ""
+                    try:
+                        msg = json.loads(results[i]['prompt'])
+                    except json.decoder.JSONDecodeError:
+                        msg = pkg.openai.session.reset_session_prompt(session_name, results[i]['prompt'])
+                        # 持久化
+                        pkg.openai.session.get_session(session_name).persistence()
+                    if len(msg) >= 2:
+                        reply_str += "#{} 创建:{} {}\n".format(i + page * 10,
+                                                               datetime_obj.strftime("%Y-%m-%d %H:%M:%S"),
+                                                               msg[1]['content'])
+                    else:
+                        reply_str += "#{} 创建:{} {}\n".format(i + page * 10,
+                                                               datetime_obj.strftime("%Y-%m-%d %H:%M:%S"),
+                                                               "无内容")
                     if results[i]['create_timestamp'] == pkg.openai.session.get_session(
                             session_name).create_timestamp:
                         current = i + page * 10

pkg/qqbot/filter.py

@@ -1,19 +1,77 @@
 # 敏感词过滤模块
 import re
+import requests
+import json
+import logging
 
 
 class ReplyFilter:
-
     sensitive_words = []
 
+    # 默认值( 兼容性考虑 )
+    baidu_check = False
+    baidu_api_key = ""
+    baidu_secret_key = ""
+    inappropriate_message_tips = "[百度云]请珍惜机器人，当前返回内容不合规"
+
     def __init__(self, sensitive_words: list):
         self.sensitive_words = sensitive_words
+        import config
+        if hasattr(config, 'baidu_check') and hasattr(config, 'baidu_api_key') and hasattr(config, 'baidu_secret_key'):
+            self.baidu_check = config.baidu_check
+            self.baidu_api_key = config.baidu_api_key
+            self.baidu_secret_key = config.baidu_secret_key
+            self.inappropriate_message_tips = config.inappropriate_message_tips
+
+    def is_illegal(self, message: str) -> bool:
+        processed = self.process(message)
+        if processed != message:
+            return True
+        return False
 
     def process(self, message: str) -> str:
+
+        # 本地关键词屏蔽
         for word in self.sensitive_words:
             match = re.findall(word, message)
             if len(match) > 0:
                 for i in range(len(match)):
                     message = message.replace(match[i], "*" * len(match[i]))
 
+        # 百度云审核
+        if self.baidu_check:
+
+            # 百度云审核URL
+            baidu_url = "https://aip.baidubce.com/rest/2.0/solution/v1/text_censor/v2/user_defined?access_token=" + \
+                str(requests.post("https://aip.baidubce.com/oauth/2.0/token", 
+                      params={"grant_type": "client_credentials", 
+                              "client_id": self.baidu_api_key,
+                              "client_secret": self.baidu_secret_key}).json().get("access_token"))
+
+            # 百度云审核
+            payload = "text=" + message
+            logging.info("向百度云发送:" + payload)
+            headers = {'Content-Type': 'application/x-www-form-urlencoded', 'Accept': 'application/json'}
+
+            if isinstance(payload, str):
+                payload = payload.encode('utf-8')
+
+            response = requests.request("POST", baidu_url, headers=headers, data=payload)
+            response_dict = json.loads(response.text)
+
+            if "error_code" in response_dict:
+                error_msg = response_dict.get("error_msg")
+                logging.warning(f"百度云判定出错，错误信息：{error_msg}")
+                conclusion = f"百度云判定出错，错误信息：{error_msg}\n以下是原消息：{message}"
+            else:
+                conclusion = response_dict["conclusion"]
+                if conclusion in ("合规"):
+                    logging.info(f"百度云判定结果：{conclusion}")
+                    return message
+                else:
+                    logging.warning(f"百度云判定结果：{conclusion}")
+                    conclusion = self.inappropriate_message_tips
+            # 返回百度云审核结果
+            return conclusion
+        
         return message

pkg/qqbot/message.py

@@ -7,6 +7,7 @@ import pkg.openai.session
 
 import pkg.plugin.host as plugin_host
 import pkg.plugin.models as plugin_models
+import pkg.qqbot.blob as blob
 
 
 def handle_exception(notify_admin: str = "", set_reply: str = "") -> list:
@@ -63,7 +64,7 @@ def process_normal_message(text_message: str, mgr, config, launcher_type: str,
                 reply = event.get_return_value("reply")
 
             if not event.is_prevented_default():
-                reply = [prefix + text]
+                reply = blob.check_text(prefix + text)
         except openai.error.APIConnectionError as e:
             err_msg = str(e)
             if err_msg.__contains__('Error communicating with OpenAI'):

pkg/qqbot/process.py

@@ -49,7 +49,7 @@ def process_message(launcher_type: str, launcher_id: int, text_message: str, mes
     session_name = "{}_{}".format(launcher_type, launcher_id)
 
     # 检查发送方是否被禁用
-    if banlist.is_banned(launcher_type, launcher_id):
+    if banlist.is_banned(launcher_type, launcher_id, sender_id):
         logging.info("根据禁用列表忽略{}_{}的消息".format(launcher_type, launcher_id))
         return []
 
@@ -66,6 +66,11 @@ def process_message(launcher_type: str, launcher_id: int, text_message: str, mes
                                                                                 result.mute_time_remaining))
             return reply
 
+    import config
+    if hasattr(config, 'income_msg_check') and config.income_msg_check:
+        if mgr.reply_filter.is_illegal(text_message):
+            return MessageChain(Plain("[bot] 你的提问中有不合适的内容, 请更换措辞~"))
+
     pkg.openai.session.get_session(session_name).acquire_response_lock()
 
     text_message = text_message.strip()
@@ -153,7 +158,7 @@ def process_message(launcher_type: str, launcher_id: int, text_message: str, mes
                                                      "..." if len(reply[0]) > 100 else "")))
                 reply = [mgr.reply_filter.process(reply[0])]
             else:
-                logging.info("回复[{}]图片消息:{}".format(session_name, reply))
+                logging.info("回复[{}]消息".format(session_name))
 
         finally:
             processing.remove(session_name)

pkg/utils/pkgmgr.py

@@ -8,6 +8,11 @@ def install(package):
     main.reset_logging()
 
 
+def run_pip(params: list):
+    pipmain(params)
+    main.reset_logging()
+
+
 def install_requirements(file):
     pipmain(['install', '-r', file, "--upgrade"])
     main.reset_logging()

pkg/utils/text2img.py

@@ -0,0 +1,164 @@
+from PIL import Image, ImageDraw, ImageFont
+import re
+import os
+
+text_render_font = ImageFont.truetype("res/simhei.ttf", 32, encoding="utf-8")
+
+
+def indexNumber(path=''):
+    """
+    查找字符串中数字所在串中的位置
+    :param path:目标字符串
+    :return:<class 'list'>: <class 'list'>: [['1', 16], ['2', 35], ['1', 51]]
+    """
+    kv = []
+    nums = []
+    beforeDatas = re.findall('[\d]+', path)
+    for num in beforeDatas:
+        indexV = []
+        times = path.count(num)
+        if times > 1:
+            if num not in nums:
+                indexs = re.finditer(num, path)
+                for index in indexs:
+                    iV = []
+                    i = index.span()[0]
+                    iV.append(num)
+                    iV.append(i)
+                    kv.append(iV)
+            nums.append(num)
+        else:
+            index = path.find(num)
+            indexV.append(num)
+            indexV.append(index)
+            kv.append(indexV)
+    # 根据数字位置排序
+    indexSort = []
+    resultIndex = []
+    for vi in kv:
+        indexSort.append(vi[1])
+    indexSort.sort()
+    for i in indexSort:
+        for v in kv:
+            if i == v[1]:
+                resultIndex.append(v)
+    return resultIndex
+
+
+def get_size(file):
+    # 获取文件大小:KB
+    size = os.path.getsize(file)
+    return size / 1024
+
+
+def get_outfile(infile, outfile):
+    if outfile:
+        return outfile
+    dir, suffix = os.path.splitext(infile)
+    outfile = '{}-out{}'.format(dir, suffix)
+    return outfile
+
+
+def compress_image(infile, outfile='', kb=100, step=20, quality=90):
+    """不改变图片尺寸压缩到指定大小
+    :param infile: 压缩源文件
+    :param outfile: 压缩文件保存地址
+    :param mb: 压缩目标,KB
+    :param step: 每次调整的压缩比率
+    :param quality: 初始压缩比率
+    :return: 压缩文件地址，压缩文件大小
+    """
+    o_size = get_size(infile)
+    if o_size <= kb:
+        return infile, o_size
+    outfile = get_outfile(infile, outfile)
+    while o_size > kb:
+        im = Image.open(infile)
+        im.save(outfile, quality=quality)
+        if quality - step < 0:
+            break
+        quality -= step
+        o_size = get_size(outfile)
+    return outfile, get_size(outfile)
+
+
+def text_to_image(text_str: str, save_as="temp.png", width=800):
+    global text_render_font
+
+    text_str = text_str.replace("\t", "    ")
+    
+    # 分行
+    lines = text_str.split('\n')
+
+    # 计算并分割
+    final_lines = []
+
+    text_width = width-80
+    for line in lines:
+        # 如果长了就分割
+        line_width = text_render_font.getlength(line)
+        if line_width < text_width:
+            final_lines.append(line)
+            continue
+        else:
+            rest_text = line
+            while True:
+                # 分割最前面的一行
+                point = int(len(rest_text) * (text_width / line_width))
+
+                # 检查断点是否在数字中间
+                numbers = indexNumber(rest_text)
+
+                for number in numbers:
+                    if number[1] < point < number[1] + len(number[0]) and number[1] != 0:
+                        point = number[1]
+                        break
+
+                final_lines.append(rest_text[:point])
+                rest_text = rest_text[point:]
+                line_width = text_render_font.getlength(rest_text)
+                if line_width < text_width:
+                    final_lines.append(rest_text)
+                    break
+                else:
+                    continue
+    # 准备画布
+    img = Image.new('RGBA', (width, max(280, len(final_lines) * 35 + 45)), (255, 255, 255, 255))
+    draw = ImageDraw.Draw(img, mode='RGBA')
+
+    
+    # 绘制正文
+    line_number = 0
+    offset_x = 20
+    offset_y = 30
+    for final_line in final_lines:
+        draw.text((offset_x, offset_y + 35 * line_number), final_line, fill=(0, 0, 0), font=text_render_font)
+        # 遍历此行,检查是否有emoji
+        idx_in_line = 0
+        for ch in final_line:
+            # if self.is_emoji(ch):
+            #     emoji_img_valid = ensure_emoji(hex(ord(ch))[2:])
+            #     if emoji_img_valid:  # emoji图像可用,绘制到指定位置
+            #         emoji_image = Image.open("emojis/{}.png".format(hex(ord(ch))[2:]), mode='r').convert('RGBA')
+            #         emoji_image = emoji_image.resize((32, 32))
+
+            #         x, y = emoji_image.size
+
+            #         final_emoji_img = Image.new('RGBA', emoji_image.size, (255, 255, 255))
+            #         final_emoji_img.paste(emoji_image, (0, 0, x, y), emoji_image)
+
+            #         img.paste(final_emoji_img, box=(int(offset_x + idx_in_line * 32), offset_y + 35 * line_number))
+
+            # 检查字符占位宽
+            char_code = ord(ch)
+            if char_code >= 127:
+                idx_in_line += 1
+            else:
+                idx_in_line += 0.5
+
+        line_number += 1
+
+    
+    img.save(save_as)
+
+    return save_as

requirements.txt

@@ -1,5 +1,5 @@
 requests~=2.28.1
-openai~=0.26.5
+openai~=0.27.0
 pip~=22.3.1
 dulwich~=0.21.3
 colorlog~=6.6.0
@@ -7,3 +7,4 @@ yiri-mirai~=0.2.6.1
 websockets~=10.4
 urllib3~=1.26.10
 func_timeout~=4.3.5
+Pillow

sensitive-template.json

@@ -9,6 +9,7 @@
     "毛泽东",
     "邓小平",
     "周恩来",
+    "马克思",
     "社会主义",
     "共产党",
     "共产主义",
@@ -21,6 +22,8 @@
     "天安门",
     "六四",
     "政治局常委",
+    "两会",
+    "共青团",
     "学潮",
     "八九",
     "二十大",
@@ -48,6 +51,7 @@
     "作爱",
     "做爱",
     "性交",
+    "性爱",
     "自慰",
     "阴茎",
     "淫妇",

tests/compatibility_tests/models_and_interfaces.py

@@ -0,0 +1,46 @@
+import openai
+import time
+
+# 测试completion api
+models = [
+    'gpt-3.5-turbo',
+    'gpt-3.5-turbo-0301',
+    'text-davinci-003',
+    'text-davinci-002',
+    'code-davinci-002',
+    'code-cushman-001',
+    'text-curie-001',
+    'text-babbage-001',
+    'text-ada-001',
+]
+
+openai.api_key = "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"
+
+for model in models:
+    print('Testing model: ', model)
+
+    # completion api
+    try:
+        response = openai.Completion.create(
+                        model=model,
+                        prompt="Say this is a test",
+                        max_tokens=7,
+                        temperature=0
+                    )
+        print('    completion api: ', response['choices'][0]['text'].strip())
+    except Exception as e:
+        print('    completion api err: ', e)
+
+    # chat completion api
+    try:
+        completion = openai.ChatCompletion.create(
+                        model="gpt-3.5-turbo",
+                        messages=[
+                            {"role": "user", "content": "Hello!"}
+                        ]
+                    )
+        print("    chat api: ",completion.choices[0].message['content'].strip())
+    except Exception as e:
+        print('    chat api err: ', e)
+
+    time.sleep(60)