Commit Graph

Select branches

Hide Pull Requests

codex/fix-issue-with-mediacrawler-functionality

codex/replace-argparse-with-typer-for-cli

feature/config-refactor-20251018

main

#10

#112

#124

#136

#137

#142

#157

#177

#18

#182

#183

#184

#201

#202

#206

#209

#212

#213

#214

#216

#217

#223

#224

#226

#228

#228

#229

#234

#238

#24

#242

#243

#250

#251

#252

#253

#254

#255

#265

#273

#276

#278

#280

#281

#290

#292

#293

#294

#295

#298

#299

#300

#301

#303

#306

#307

#309

#314

#316

#318

#319

#32

#323

#325

#327

#328

#329

#330

#331

#332

#333

#336

#337

#343

#345

#346

#347

#348

#350

#353

#357

#360

#361

#370

#371

#374

#375

#376

#377

#378

#379

#380

#384

#385

#386

#387

#388

#389

#396

#399

#4

#40

#400

#401

#404

#405

#411

#413

#422

#425

#427

#433

#461

#462

#463

#477

#479

#506

#509

#513

#516

#519

#521

#528

#536

#538

#542

#547

#559

#572

#590

#592

#596

#599

#603

#607

#608

#616

#619

#621

#634

#635

#636

#637

#638

#639

#64

#641

#642

#649

#650

#652

#654

#655

#657

#658

#664

#674

#676

#680

#681

#684

#686

#687

#690

#699

#700

#704

#707

#716

#717

#718

#723

#726

#728

#729

#73

#738

#739

#74

#741

#743

#761

#764

#768

#772

#772

#773

#773

#82

#89

#94

#96

Mono Color

• e6db6be1ca Update __init__.py，添加了提取抖音笔记图片的逻辑 翟持江 2025-07-30 10:45:38 +08:00
• 72a4d194c0 Merge pull request #680 from korruz/main 程序员阿江-Relakkes 2025-07-29 14:21:48 +08:00
• 07a6e387ea refactor: move format_proxy_info to utils and update crawler classes to use it korruz 2025-07-29 14:16:24 +08:00
• 2935841872 Merge pull request #676 from andy9468/main 程序员阿江-Relakkes 2025-07-28 11:33:42 +08:00
• 3653fe2d37 add aiosqlite andy9468 2025-07-26 16:42:16 +08:00
• dbbc33a0df Merge pull request #674 from persist-1/chore 程序员阿江-Relakkes 2025-07-25 17:23:03 +08:00
• 19df1734f1 chore: 增加--help参数中文显示支持及douyin_aweme表music_download_url字段\n\n- 为命令行参数增加中文显示支持，提升用户体验\n- 在douyin_aweme表中新增music_download_url字段用于存储视频音乐下载链接\n- 更新相关数据库表结构文件(tables.sql, sqlite_tables.sql)\n- 实现音乐下载URL提取逻辑并集成到数据存储流程 persist-1 2025-07-24 22:39:53 +08:00
• fc06c783f5 fix: fixed xhs req headers 程序员阿江(Relakkes) 2025-07-23 13:28:58 +08:00
• b41896f4f3 docs: add a sponsor 程序员阿江(Relakkes) 2025-07-23 13:17:19 +08:00
• a4d9aaa34a refactor: xhs update 程序员阿江(Relakkes) 2025-07-21 21:26:16 +08:00
• 26a43358cb chore: update config 程序员阿江(Relakkes) 2025-07-20 14:34:56 +08:00
• 13b00f7a36 refactor: config update 程序员阿江(Relakkes) 2025-07-18 23:26:52 +08:00
• 122978b35c Merge pull request #652 from gaoxiaobei/dev 程序员阿江(Relakkes) 2025-07-18 21:41:20 +08:00
• 8105b053ed Merge remote-tracking branch 'origin/dev' into devdev gaoxiaobei 2025-07-18 17:37:29 +08:00
• 7176956e51 Merge branch 'NanmiCoder:main' into dev gaoxiaobei 2025-07-18 17:32:04 +08:00
• b913db64bb refactor(config): move platform-specific configs to separate files gaoxiaobei 2025-07-18 17:27:37 +08:00
• 2753e7631e Merge pull request #664 from cfl-chenfangliang/feat-dyCommentBug 程序员阿江(Relakkes) 2025-07-18 15:53:33 +08:00
• aa54dad9a5 feat: 修复抖音二级评论地理位置缺失问题 chenfangliang 2025-07-18 10:48:43 +08:00
• 5daae04c7d fix: conflict resolution errors gaoxiaobei 2025-07-17 16:54:17 +08:00
• 1dc8c1789f docs(config): update Bilibili search mode options gaoxiaobei 2025-07-17 07:51:27 +08:00
• 29b6cee408 Merge pull request #1 from gaoxiaobei/dev gaoxiaobei 2025-07-17 06:45:55 +08:00
• 6ced357096 Merge branch 'main' into dev gaoxiaobei 2025-07-17 06:45:30 +08:00
• 9fb396c7d1 fix(media_platform): handle edge cases and improve error handling for Bilibili client and crawler gaoxiaobei 2025-07-17 06:40:56 +08:00
• fb846e9060 Merge branch 'NanmiCoder:main' into main gaoxiaobei 2025-07-17 06:39:04 +08:00
• 08c28e6f7b Merge pull request #658 from persist-1/feature/sqlite-support 程序员阿江(Relakkes) 2025-07-16 20:35:58 +08:00
• 9457455c18 fix: 修复SQLite数据库初始化问题并重新生成数据库文件 买定不离手 2025-07-16 19:48:52 +08:00
• c5509ab91f fix: update ip proxy valid url for #662 程序员阿江(Relakkes) 2025-07-16 11:06:58 +08:00
• c795b1316a fix: import error for #663 程序员阿江(Relakkes) 2025-07-16 10:58:11 +08:00
• 3184e0a8d9 fix: sign data too long error #659 程序员阿江(Relakkes) 2025-07-14 18:43:17 +08:00
• 4d743f6c17 debug & resume default configuration gaoxiaobei 2025-07-14 08:00:48 +08:00
• 3365095c62 fix: 完善Bilibili和抖音平台SQLite SQL语句适配 买定不离手 2025-07-14 03:51:19 +08:00
• 1298022410 refactor: 更新各平台store模块初始化以支持SQLite 买定不离手 2025-07-14 03:51:08 +08:00
• 1673bd5c0c feat: 增强SQLite数据库配置和命令行参数支持 买定不离手 2025-07-14 03:50:54 +08:00
• 191dd5998d docs: 更新README文档以支持SQLite数据库存储 买定不离手 2025-07-14 03:50:32 +08:00
• c6b96b7e28 chore: 更新项目依赖锁定文件\n\n- 更新 uv.lock: 同步项目依赖版本 买定不离手 2025-07-14 03:37:10 +08:00
• 36d4a086dd docs: 在默认说明文档中加入SQLite支持说明\n\n- 更新 docs/index.md 文件，增加对SQLite数据库的支持说明\n- 添加SQLite使用示例和配置说明 买定不离手 2025-07-14 03:36:59 +08:00
• f5fbbb36ba fix: 修复SQLite数据库初始化和关闭逻辑\n\n- 更新 main.py: 修复数据库初始化条件，支持sqlite选项\n- 更新 db.py: 添加SQLite数据库初始化和关闭支持 买定不离手 2025-07-14 03:36:48 +08:00
• 6f274d476b feat: 添加各平台SQLite存储实现文件\n\n- 新增 store/bilibili/bilibili_store_impl.py: B站SQLite存储实现\n- 新增 store/douyin/douyin_store_impl.py: 抖音SQLite存储实现\n- 新增 store/kuaishou/kuaishou_store_impl.py: 快手SQLite存储实现\n- 新增 store/tieba/tieba_store_impl.py: 贴吧SQLite存储实现\n- 新增 store/weibo/weibo_store_impl.py: 微博SQLite存储实现\n- 新增 store/xhs/xhs_store_impl.py: 小红书SQLite存储实现\n- 新增 store/zhihu/zhihu_store_impl.py: 知乎SQLite存储实现 买定不离手 2025-07-14 03:36:36 +08:00
• fb938f38aa feat: 更新各平台store SQL文件以支持SQLite\n\n- 更新 store/kuaishou/kuaishou_store_sql.py: 快手平台SQLite适配\n- 更新 store/tieba/tieba_store_sql.py: 贴吧平台SQLite适配\n- 更新 store/weibo/weibo_store_sql.py: 微博平台SQLite适配\n- 更新 store/xhs/xhs_store_sql.py: 小红书平台SQLite适配\n- 更新 store/zhihu/zhihu_store_sql.py: 知乎平台SQLite适配 买定不离手 2025-07-14 03:36:20 +08:00
• 3a2959d86c feat: 添加SQLite数据库支持核心文件\n\n- 新增 async_sqlite_db.py: SQLite异步数据库操作封装\n- 新增 schema/sqlite_tables.sql: SQLite数据库表结构定义\n- 新增 schema/sqlite_tables.db: SQLite数据库文件 买定不离手 2025-07-14 03:36:06 +08:00
• e91ec750bb feat: Enhance Bilibili crawler with retry logic and robustness gaoxiaobei 2025-07-13 10:42:15 +08:00
• d0d7293926 feat(bilibili): Add flexible search modes and fix limit logic gaoxiaobei 2025-07-13 06:07:13 +08:00
• e103bfa1f3 Merge branch 'NanmiCoder:main' into main gaoxiaobei 2025-07-13 05:41:21 +08:00
• dd8a3f5db8 docs: add a Sponsor 程序员阿江(Relakkes) 2025-07-12 23:26:30 +08:00
• cad9fc7af8 feat: Add daily limit for video/post crawling in Bilibili and base config gaoxiaobei 2025-07-12 14:50:59 +08:00
• ec0d29cf0f Merge pull request #642 from cllei12/weibo-search-type 程序员阿江(Relakkes) 2025-07-07 15:05:29 +08:00
• 0d21a27b6e Merge pull request #641 from cllei12/main 程序员阿江(Relakkes) 2025-07-07 15:03:42 +08:00
• 355ed183dd 增加选择微博搜索类型的配置 Lei Cao 2025-07-05 22:14:31 +00:00
• eb03a4f68d Update playwright version to support Ubuntu 24.04 Lei Cao 2025-07-05 21:17:52 +00:00
• 3cb0e2f91f Merge pull request #637 from Mirza-Samad-Ahmed-Baig/fix/bilibili-creator-videos 程序员阿江(Relakkes) 2025-07-05 00:13:49 +08:00
• 7edf3bcc15 refactor(bilibili): process creator videos in batches mirza-samad-ahmed-baig 2025-07-04 21:04:10 +05:00
• 66a68fbb13 docs: update multi language badges size 程序员阿江(Relakkes) 2025-07-04 14:15:03 +08:00
• 8dcc540797 Merge pull request #635 from Root-FTW/main 程序员阿江(Relakkes) 2025-07-04 13:54:31 +08:00
• 4a110abebb feat: Add language navigation links to all README files Root-FTW 2025-07-03 17:14:41 -07:00
• 3b7726365c feat: Add localized README files in English and Spanish Root-FTW 2025-07-03 17:09:08 -07:00
• 848df2b491 feat: other platfrom support the cdp mode 程序员阿江(Relakkes) 2025-07-03 17:13:32 +08:00
• c892c3324c chore: update uv lock file use aliyun pypi 程序员阿江(Relakkes) 2025-07-03 16:07:07 +08:00
• 452aafeec8 chore: update uv depends source 程序员阿江(Relakkes) 2025-07-03 15:27:39 +08:00
• 89359aa259 docs: update README.md 程序员阿江(Relakkes) 2025-07-02 01:20:44 +08:00
• 0514758cff docs: 更新uv安装文档 程序员阿江(Relakkes) 2025-06-29 00:07:13 +08:00
• e83b2422d9 feat: 支持playwright通过cdp协议连接本地chrome浏览器 docs: 增加uv来管理python依赖的文档 程序员阿江(Relakkes) 2025-06-25 23:22:39 +08:00
• fbc9788d54 docs: update README.md 程序员阿江(Relakkes) 2025-06-24 17:44:43 +08:00
• 71210e3f38 chore: delete unnecessary file Relakkes 2025-06-20 16:26:02 +08:00
• fc45b22963 fix: vitepress path problem Relakkes 2025-06-20 16:10:44 +08:00
• 22ec34dca3 docs: add list of sponsors Relakkes 2025-06-20 16:04:04 +08:00
• fd33813f8f feat: add like_count field to bilibi for issue #623 Relakkes 2025-06-20 15:50:38 +08:00
• 31bcdb191f docs: update README.md Relakkes 2025-06-16 13:58:09 +08:00
• d55d8b1efa feat: Douyin supports obtaining video links and cover images. for issue #620 Relakkes 2025-06-14 23:59:08 +08:00
• ed1dc7916a docs: update README.md Relakkes 2025-06-08 15:56:02 +08:00
• 6323e2d45b Merge pull request #616 from chimeElm/main 程序员阿江(Relakkes) 2025-06-07 14:43:37 +08:00
• 26a845581e Update client.py chimeElm 2025-06-07 02:41:09 +08:00
• 23c8f8f87b docs: add english license Relakkes 2025-06-01 23:20:11 +08:00
• 1e7b950d3e Revert "chore: remove sponor" Relakkes 2025-05-26 22:35:18 +08:00
• 242c06c345 chore: remove sponor Relakkes 2025-05-25 11:54:38 +08:00
• ff41faeb00 Merge pull request #608 from Bowenwin/bili_expand 程序员阿江(Relakkes) 2025-05-22 23:14:58 +08:00
• 66843f216a finish_all_for_expand_bili Bowenwin 2025-05-22 22:26:30 +08:00
• 59619fff0a finish_all Bowenwin 2025-05-22 22:06:06 +08:00
• 44e3d370ff fix_words Bowenwin 2025-05-22 20:31:48 +08:00
• 7ed6621933 Merge pull request #603 from Bowenwin/fix_words 程序员阿江(Relakkes) 2025-05-19 23:16:12 +08:00
• 703a6e84cb fix_words Bowenwin 2025-05-19 20:07:20 +08:00
• 144b8bec6a fix_words Bowenwin 2025-05-19 20:04:00 +08:00
• a356358c21 get_fans_and_get_followings Bowenwin 2025-05-19 19:57:36 +08:00
• 654260cbce docs: update README.md Relakkes 2025-05-13 18:42:58 +08:00
• 79a9824f6a fix: modify dy schema Relakkes 2025-04-30 16:47:13 +08:00
• 67d31bf42a fix: dy update fp params Relakkes 2025-04-30 13:26:22 +08:00
• 2a41b684ad Merge pull request #590 from 2513502304/main 程序员阿江(Relakkes) 2025-04-20 14:14:55 +08:00
• af5a393a7a Update core.py，删除了其它代码贡献者所添加的try-catch语句，该段try-catch语句将会影响其代码的最终逻辑并令其失效，使其仅能爬取当天一天数据而无法跳转到下一天（原先的逻辑就是try-catch捕获异常从而进入下一天，不要再向该语句中添加捕获异常操作或者finally语句！） 翟持江 2025-04-19 04:34:24 +08:00
• b675547aab Update __init__.py，为bilibili的视频信息、up主信息、评论信息添加额外字段 翟持江 2025-04-19 02:29:22 +08:00
• ec97001451 Update tables.sql 翟持江 2025-04-19 02:22:22 +08:00
• 9935a07279 Add files via upload 翟持江 2025-04-19 02:18:52 +08:00
• cb2ae6cbab docs: add sponsor Relakkes 2025-04-16 16:49:49 +08:00
• 0d715a9f32 fix: bili qrcode login fix Relakkes 2025-04-08 21:11:40 +08:00
• 660fd18a95 fix: dy login fix Relakkes 2025-04-08 20:58:04 +08:00
• afbd4ec1bf Merge pull request #572 from crpa33/main 程序员阿江(Relakkes) 2025-04-02 13:34:41 +08:00
• 274d64aefc 处理xhs意外的评论信息为空的情况 crpa33 2025-04-02 11:59:27 +08:00
• a39b571d27 输出到日志-处理视频搜索页任务列表构造的错误 crpa33 2025-04-02 11:57:28 +08:00
• 413d91a520 输出到日志-author被封禁或存在错误 crpa33 2025-04-02 11:52:36 +08:00
• eaf14721f8 输出到日志-NoneType导致的推导式错误 crpa33 2025-04-02 11:48:36 +08:00
• 2c4af2337e douyin搜索页为空跳下一关键词 crpa33 2025-03-27 23:32:21 +08:00
• 3c72fc48b0 保护author为None但未被识别的情况 crpa33 2025-03-27 23:22:47 +08:00