injectlib/OpenList
1
0
Fork 0
mirror of https://github.com/OpenListTeam/OpenList.git synced 2025-11-25 03:15:19 +08:00
Code Issues Packages Projects Releases Wiki Activity
2,602 Commits 18 Branches 147 Tags
main
Commit Graph

120 Commits

Author SHA1 Message Date
varg1714
b22e211044 feat(fs): Add skipExisting option to move and copy, merge option to copy (#1556) 
* fix(fs): Add skipExisting option to move and copy.

* feat(fs): Add merge option to copy.

* feat(fs): Code smell.

* feat(fs): Code smell.
 2025-11-24 14:20:24 +08:00
ASLant
39dcf9bd19 feat(onedrive): support frontend direct upload (#1532) 
* OneDrive添加直连上传

* refactor

* fix: duplicate root path join

---------

Co-authored-by: KirCute <951206789@qq.com>
 2025-11-06 23:22:02 +08:00
ILoveScratch
febbcd6027 feat(cache): improve cache management (#1339) 
* feat(cache): improve cache management

* feat(disk-usage): add cache

* feat(disk-usage): add refresh

* fix(disk-usage): cache with ttl

* feat(cache): implement KeyedCache and TypedCache for improved caching mechanism

* fix(copy): update object retrieval to use Get instead of GetUnwrap

* refactor(cache): simplify DirectoryCache structure and improve object management

* fix(cache): correct cache entry initialization and key deletion logic in TypedCache

* refactor(driver): remove GetObjInfo interface and simplify Link function logic
https://github.com/OpenListTeam/OpenList/pull/888/files#r2430925783

* fix(link): optimize link retrieval and caching logic

* refactor(cache): consolidate cache management and improve directory cache handling

* fix(cache): add cache control based on storage configuration in List function

* .

* refactor: replace fmt.Sprintf with strconv for integer conversions

* refactor(cache): enhance cache entry management with Expirable interface

* fix(cache): improve link reference acquisition logic to handle expiration

* refactor: replace OnlyLinkMFile with NoLinkSF in driver configurations and logic

* refactor(link): enhance link caching logic with dynamic type keys based on IP and User-Agent

* feat(drivers): add LinkCacheType to driver configurations for enhanced caching

* refactor(cache): streamline directory object management in cache operations

* refactor(cache): remove unnecessary 'dirty' field from CacheEntry structure

* refactor(cache): replace 'dirty' field with bitwise flags

* refactor(io): 调高SyncClosers.AcquireReference的优先级

* refactor(link): 优化链接获取逻辑，增加重

* refactor(link): 添加RequireReference字段以增强链接管理

* refactor(link): 移除MFile字段，改用RangeReader

* refactor: 移除不必要的NoLinkSF字段

* refactor(cache): 修改目录缓存的脏标志定义和更新逻辑

* feat(cache): add expiration gc

---------

Co-authored-by: KirCute <951206789@qq.com>
Co-authored-by: KirCute <kircute@foxmail.com>
Co-authored-by: j2rong4cn <j2rong@qq.com>
 2025-10-18 21:47:18 +08:00
jenfonro
549e60136b fix(fs):fix retry task after restart (#1467) 
* fix retry task after restart

* fix: initialize SrcStorage and DstStorage in tasks to prevent nil pointer dereference

* feat: implement storage load signal mechanism for improved synchronization

* fix:  update StoragesLoaded logic

* refactor: reorganize storage loading logic and improve synchronization handling

---------

Co-authored-by: j2rong4cn <j2rong@qq.com>
 2025-10-18 20:52:02 +08:00
KirCute
a2fc38be8d perf(disk-usage): concurrently get details (#1326) 2025-09-29 23:26:56 +08:00
KirCute
cc16cb35bf feat(style): add driver icons and disk usage (#1274) 
* feat(style): add driver icons and disk usage

* feat(driver): add disk usage for 115_open, 123_open, aliyundrive_open and baidu_netdisk

* feat(driver): add disk usage for crypt, sftp and smb

* chore: clean unused variable

* feat(driver): add disk usage for cloudreve_v4

Signed-off-by: MadDogOwner <xiaoran@xrgzs.top>

* fix(local): disk label check when getting disk usage

* feat(style): return details when accessing the manage page

---------

Signed-off-by: MadDogOwner <xiaoran@xrgzs.top>
Co-authored-by: MadDogOwner <xiaoran@xrgzs.top>
 2025-09-19 11:59:11 +08:00
KirCute
e4c902dd93 feat(share): support more secure file sharing (#991) 
提供一种类似大多数网盘的文件分享操作，这种分享方式可以通过强制 Web 代理隐藏文件源路径，可以设置分享码、最大访问数和过期时间，并且不需要启用 guest 用户。

在全局设置中可以调整：
- 是否强制 Web 代理
- 是否允许预览
- 是否允许预览压缩文件
- 分享文件后，点击“复制链接”按钮复制的内容

前端部分：OpenListTeam/OpenList-Frontend#156
文档部分：OpenListTeam/OpenList-Docs#130

Close #183
Close #526
Close #860
Close #892
Close #1079


* feat(share): support more secure file sharing

* feat(share): add archive preview

* fix(share): fix some bugs

* feat(openlist_share): add openlist share driver

* fix(share): lack unwrap when get virtual path

* fix: use unwrapPath instead of path for virtual file name comparison

* fix(share): change request method of /api/share/list from GET to Any

* fix(share): path traversal vulnerability in sharing path check

* 修复分享alias驱动的文件 没开代理时无法获取URL

* fix(sharing): update error message for sharing root link extraction

---------

Co-authored-by: Suyunmeng <69945917+Suyunmeng@users.noreply.github.com>
Co-authored-by: j2rong4cn <j2rong@qq.com>
 2025-08-19 15:10:02 +08:00
j2rong4cn
57fceabcf4 perf(stream): improve file stream range reading and caching mechanism (#1001) 
* perf(stream): improve file stream range reading and caching mechanism

* 。

* add bytes_test.go

* fix(stream): handle EOF and buffer reading more gracefully

* 注释

* refactor: update CacheFullAndWriter to accept pointer for UpdateProgress

* update tests

* Update drivers/google_drive/util.go

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Signed-off-by: j2rong4cn <36783515+j2rong4cn@users.noreply.github.com>

* 更优雅的克隆Link

* 修复stream已缓存但无法重复读取

* 将Bytes类型重命名为Reader

* 修复栈溢出

* update tests

---------

Signed-off-by: j2rong4cn <36783515+j2rong4cn@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
 2025-08-11 23:41:22 +08:00
j2rong4cn
4e9c30f49d feat(fs): full support webdav cross-driver copy and move (#823) 
* fix(fs): restore webdav cross-driver copy and move

* fix bug

* webdav支持复制、移动 文件夹

* 优化

* 。
 2025-07-26 00:27:46 +08:00
Seven
e93ab76036 feat(task-group): introduce TaskGroupCoordinator for coordinated task execution (#721) 
* feat(task): add task hook,batch task
refactor(move): move use CopyTask

* Update internal/task/batch_task/refresh.go

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Signed-off-by: Seven <53081179+Seven66677731@users.noreply.github.com>

* fix: upload task allFinish judge

* Update internal/task/batch_task/refresh.go

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Signed-off-by: Seven <53081179+Seven66677731@users.noreply.github.com>

* feat: enhance concurrency safety

* 优化代码

* 解压缩

* 修复死锁

* refactor(move): move as task

* 重构，优化

* .

* 优化，修复bug

* .

* 修复bug

* feat: add task retry judge

* 代理Task.SetState函数来判断Task的生命周期

* chore: use OnSucceeded、OnFailed、OnBeforeRetry functions

* 优化

* 优化，去除重复代码

* .

* 优化

* .

* webdav

* Revert "fix(fs):After the file is copied or moved, flush the cache of the directory that was copied or moved to."

This reverts commit 5f03edd683.

---------

Signed-off-by: Seven <53081179+Seven66677731@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: j2rong4cn <j2rong@qq.com>
 2025-07-24 16:15:24 +08:00
Suyunjing
5f03edd683 fix(fs):After the file is copied or moved, flush the cache of the directory that was copied or moved to. (#592) 
* fix(fs):After the file is copied, the cache of the copied directory is refreshed

* fixed randomstring

* fixed EOL and Sync branch

chore(quark_uc): `webdav_policy` default to native_proxy

* fixed uuid and other bugs

* fixed comments

* fixed EOL

* add move refresh

* fixed builds

* fixed batch

* change betch to task.go

---------

Co-authored-by: Sumengjing <146963948+suyunjing-su@users.noreply.github.com>
 2025-07-24 10:24:12 +08:00
pnparadise
86324d2d6b fix(net): ensure accurate content-length in response (#749) 
* fix(fs): ensure accurate content-length in http2 requests

Chrome browsers were unable to preview thumbnails, reporting an
'ERR_HTTP_2_PROTOCOL_ERROR'. This was caused by an incorrect
content-length header in the server's response for thumbnail images.

This commit corrects the content-length calculation, allowing
Chrome and other compliant clients to render thumbnails correctly.

* fix(net): ensure accurate content-length in response

* 补缺

* .

---------

Co-authored-by: zhiqiang.huang <zhiqiang.tech@gmail.com>
Co-authored-by: j2rong4cn <j2rong@qq.com>
 2025-07-19 20:36:27 +08:00
j2rong4cn
0c461991f9 chore: standardize context keys with custom ContextKey type (#697) 
* chore: standardize context keys with custom ContextKey type

* fix bug

* 使用Request.Context
 2025-07-14 23:55:17 +08:00
j2rong4cn
cc01b410a4 perf(link): optimize concurrent response (#641) 
* fix(crypt): bug caused by link cache

* perf(crypt,mega,halalcloud,quark,uc): optimize concurrent response link

* chore: 删除无用代码

* ftp

* 修复bug；资源释放

* 添加SyncClosers

* local,sftp,smb

* 重构，优化，增强

* Update internal/stream/util.go

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Signed-off-by: j2rong4cn <36783515+j2rong4cn@users.noreply.github.com>

* chore

* chore

* 优化，修复bug

* .

---------

Signed-off-by: j2rong4cn <36783515+j2rong4cn@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
 2025-07-12 17:57:54 +08:00
Seven
3838ef0663 feat(traffic): update progress when caching file (#646) 
* feat(traffic): update progress when caching file

* 调整参数位置和命名

---------

Co-authored-by: j2rong4cn <j2rong@qq.com>
 2025-07-08 21:41:45 +08:00
j2rong4cn
e83f8e197a feat(offline-download): SimpleHttp: download stream direct upload (#523) 
* feat(offline-download): stream download to upload

* 重命名stream_put为upload_download_stream

* chore
 2025-07-02 18:47:16 +08:00
MadDogOwner
874dc292ae fix(gomod): go modules with tagged versions (#499) 
fix: go modules with tagged versions
 2025-07-01 09:54:50 +08:00
Suyunjing
52c93f2046 build:Update old dependencies to new hosting (#495) 
* build:Update old dependencies to new hosting
 2025-06-30 21:26:42 +08:00
j2rong4cn
103abc942e refactor: pass api_url through context (#457) 
* refactor: pass `api_url` through context

* 移除 LinkArgs.HttpReq

* pref(alias): 减少不必要下载代理

* 修复bug

* net: 支持1并发 分片下载
 2025-06-30 15:48:05 +08:00
Suyunjing
0fd602bc1b refactor(fs):Refactor the delete function and fix known issues (#353) 
* fix(move):Fix file move logic

* fix(move):fixed move logic

* fix(move):Fixed move logic

* Fixed errs

* fix(move):fixed movetask

* fix(move):fix movetask

* fixed

* fix(move):Refactoring the move structure

* fix(move):Fixed move system

* Fixed

* Fixed

* Fixed

* Fixed

* Fixed

* Rollback

* Fixed and Refactor

* fix(move):fixed

* fix(move):Solve related performance issues

* refacter(move):fixed build bugs
 2025-06-24 23:36:37 +08:00
Suyunjing
96cf2f7cf9 fix(fs):Repair file loss caused by special reasons when moving files (#321) 
* fix(move):Fix file move logic

* fix(move):fixed move logic
 2025-06-23 19:48:17 +08:00
ILoveScratch
0f5ed14fe2 feat(fs): add cross-storage move support (#211) 
* feat(fs): add cross-storage move support

* fix(fs): add check before moving files

* fix(fs): changed error detect method

---------

Co-authored-by: ShenLin <773933146@qq.com>
 2025-06-20 17:54:24 +08:00
Kuingsmile
fdcc2f136e chore: change module name to OpenListTeam/OpenList (#2) 
* Enable blank issue

* chore(README.md): update docs (temporally)

* Update FUNDING.yml

* chore: purge README.md

* chore: change module name to OpenListTeam/OpenList

* fix: fix link errors

* chore: remove v3 in module name

* fix: resolve some conficts

* fix: resolve conficts

* docs: update with latest file

---------

Co-authored-by: ShenLin <773933146@qq.com>
Co-authored-by: Hantong Chen <cxwdyx620@gmail.com>
Co-authored-by: joshua <i@joshua.su>
Co-authored-by: Hantong Chen <70561268+cxw620@users.noreply.github.com>
 2025-06-12 22:02:46 +08:00
j2rong4cn
ddffacf07b perf: optimize IO read/write usage (#8243) 
* perf: optimize IO read/write usage

* .

* Update drivers/139/driver.go

Co-authored-by: MadDogOwner <xiaoran@xrgzs.top>

---------

Co-authored-by: MadDogOwner <xiaoran@xrgzs.top>
 2025-04-12 16:55:31 +08:00
KirCute
1335f80362 feat(archive): support multipart archives (#8184 close #8015) 
* feat(archive): multipart support & sevenzip tool

* feat(archive): rardecode tool

* feat(archive): support decompress multi-selected

* fix(archive): decompress response filter internal

* feat(archive): support multipart zip

* fix: more applicable AcceptedMultipartExtensions interface
 2025-03-27 23:20:44 +08:00
KirCute
9a9aee9ac6 feat(alias): support writing to non-ambiguous paths (#8216) 
* feat(alias): support writing to non-ambiguous paths

* feat(alias): support extract concurrency

* fix(alias): extract url no pass query
 2025-03-27 23:17:45 +08:00
KirCute_ECT
d5ec998699 feat(task): allow retry canceled (#7852) 2025-01-27 20:18:10 +08:00
KirCute_ECT
11b6a6012f fix(copy): use Link and Put when the driver does not support copying (#7834) 2025-01-18 23:52:02 +08:00
KirCute_ECT
bb40e2e2cd feat(archive): archive manage (#7817) 
* feat(archive): archive management

* fix(ftp-server): remove duplicate ReadAtSeeker realization

* fix(archive): bad seeking of SeekableStream

* fix(archive): split internal and driver extraction api

* feat(archive): patch

* fix(shutdown): clear decompress upload tasks

* chore

* feat(archive): support .iso format

* chore
 2025-01-18 23:28:12 +08:00
KirCute_ECT
d7aa1608ac feat(task): add speed monitor (#7655) 2024-12-25 21:09:54 +08:00
KirCute_ECT
64ceb5afb6 feat: support general users view and cancel own tasks (#7416 close #7398) 
* feat: support general users view and cancel own tasks

Add a creator attribute to the upload, copy and offline download
tasks, so that a GENERAL task creator can view and cancel them.

BREAKING CHANGE:

1. A new internal package `task` including the struct `TaskWithCreator`
   which embeds `tache.Base` is created, and the past dependence on
   `tache.Task` will all be transferred to dependence on this package.
2. The API `/admin/task` can now also be accessed via `/task`, and the
   old endpoint is retained to ensure compatibility with legacy
   automation scripts.

Closes #7398

* fix(deps): update github.com/xhofe/tache to v0.1.3
 2024-11-01 23:32:26 +08:00
itsHenry
74f8295960 feat: persistant Task (#6925 close #5313) 2024-08-07 12:16:21 +08:00
j2rong4cn
29fe49fb87 fix(alias): Support forced refresh of file list (#6562) 2024-06-16 16:59:10 +08:00
Andy Hsu
026e944cbb feat: add task info to resp of add task api (close #5579) 2023-12-03 14:44:20 +08:00
Andy Hsu
7583c4d734 feat: customize workers and retry of task (close #5493 fix #5274) 2023-11-21 15:51:57 +08:00
Andy Hsu
11a30c5044 feat: refactor task module 2023-11-20 18:01:51 +08:00
Andy Hsu
352a6a741a feat(webdav): support copy directly without task (close #5206) 2023-09-13 15:45:57 +08:00
Sean
a3748af772 feat: misc improvements about upload/copy/hash (#5045) 
general: add createTime/updateTime support in webdav and some drivers
general: add hash support in some drivers
general: cross-storage rapid-upload support
general: enhance upload to avoid local temp file if possible
general: replace readseekcloser with File interface to speed upstream operations
feat(aliyun_open): same as above
feat(crypt): add hack for 139cloud

Close #4934 
Close #4819 

baidu_netdisk needs to improve the upload code to support rapid-upload
 2023-08-27 21:14:23 +08:00
Andy Hsu
5606c23768 perf(copy): use multi-thread downloader (close #5000) 2023-08-13 15:31:49 +08:00
Sean
7877184bee feat(baidu_netdisk): add retry to most operations (close #4863 in #4939) 2023-08-07 13:44:28 +08:00
Sean
3c21a9a520 feat: Crypt driver, improve http/webdav handling (#4884) 
this PR has several enhancements, fixes, and features:
- [x] Crypt: a transparent encryption driver. Anyone can easily, and safely store encrypted data on the remote storage provider.  Consider your data is safely stored in the safe, and the storage provider can only see the safe, but not your data.
  - [x] Optional: compatible with [Rclone Crypt](https://rclone.org/crypt/). More ways to manipulate the encrypted data.
  - [x] directory and filename encryption
  - [x] server-side encryption mode (server encrypts & decrypts all data, all data flows thru the server)
- [x] obfuscate sensitive information internally
- [x] introduced a server memory-cached multi-thread downloader.
  - [x] Driver: **Quark** enabled this feature, faster load in any single thread scenario. e.g. media player directly playing from the link, now it's faster.
- [x] general improvement on HTTP/WebDAV stream processing & header handling & response handling
  - [x] Driver: **Mega** driver support ranged http header
  - [x] Driver: **Quark** fix bug of not closing HTTP request to Quark server while user end has closed connection to alist

## Crypt, a transparent Encrypt/Decrypt Driver. (Rclone Crypt compatible)

e.g.  
Crypt mount path ->  /vault 
Crypt remote path -> /ali/encrypted
Aliyun mount paht -> /ali

when the user uploads a.jpg to /vault, the data will be encrypted and saved to /ali/encrypted/xxxxx. And when the user wants to access a.jpg,  it's automatically decrypted, and the user can do anything with it.
Since it's Rclone Crypt compatible, users can download /ali/encrypted/xxxxx  and decrypt it with rclone crypt tool. Or the user can mount this folder using rclone, then mount the decrypted folder in Linux...

NB.  Some breaking changes is made to make it follow global standard, e.g. processing the HTTP header properly.

close #4679 
close #4827 

Co-authored-by: Sean He <866155+seanhe26@users.noreply.github.com>
Co-authored-by: Andy Hsu <i@nn.ci>
 2023-08-02 14:40:36 +08:00
Andy Hsu
e700ce15e5 fix: missed progress in upload task 2023-07-29 17:09:26 +08:00
Andy Hsu
b2f5757f8d fix(copy): copy from driver that return writer (close #4291) 2023-05-26 21:57:43 +08:00
Noah Hsu
63de65be45 fix: increase timeout for http_client (close #4409) 2023-05-18 23:32:05 +08:00
XYUU
a3446720a2 fix: make TlsInsecureSkipVerify enable for all request (#4386) 2023-05-14 17:05:47 +08:00
Andy Hsu
7e3b13ea2d fix: fs/list interface conversion from copy alias (close #4279) 2023-05-01 15:45:45 +08:00
Andy Hsu
cc43238bd1 fix(alias): disable log completely (#4054) 2023-04-09 15:46:26 +08:00
Andy Hsu
c0a6beecea fix(alias): panic on nil pointer (close #4093) 2023-04-09 14:06:04 +08:00
Andy Hsu
0f8a84f67e perf(alias): disabled log on fs call (close #4054) 2023-04-07 00:02:07 +08:00
Andy Hsu
d9795ff22f feat(alias): support proxy and direct together 2023-03-14 13:46:27 +08:00