injectlib/MediaCrawler
1
0
Fork 0
mirror of https://github.com/NanmiCoder/MediaCrawler.git synced 2025-11-25 11:29:27 +08:00
Code Issues Packages Projects Releases Wiki Activity

feat: Add daily limit for video/post crawling in Bilibili and base config

Browse Source
This commit is contained in:
gaoxiaobei
2025-07-12 14:50:59 +08:00
parent ec0d29cf0f
commit cad9fc7af8
2 changed files with 8 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

3
config/base_config.py
View File

@@ -86,6 +86,9 @@ START_PAGE = 1
# 爬取视频/帖子的数量控制
CRAWLER_MAX_NOTES_COUNT = 200
# 每天爬取视频/帖子的数量控制
MAX_NOTES_PER_DAY = 20
# 并发爬虫数量控制
MAX_CONCURRENCY_NUM = 1