624 Commits

Author SHA1 Message Date
varg1714
b22e211044 feat(fs): Add skipExisting option to move and copy, merge option to copy (#1556)
* fix(fs): Add skipExisting option to move and copy.

* feat(fs): Add merge option to copy.

* feat(fs): Code smell.

* feat(fs): Code smell.
2025-11-24 14:20:24 +08:00
KirCute
72e2ae1f14 feat(fs): support manually trigger objs update hook (#1620)
* feat(fs): support manually trigger objs update hook

* fix: support driver internal copy & move case

* fix

* fix: apply suggestions of Copilot
2025-11-21 12:18:20 +08:00
Copilot
9de7561154 feat(upload): add optional system file filtering for uploads (#1634) 2025-11-14 14:45:39 +08:00
MadDogOwner
0866b9075f fix(link): correct link cache mode bitwise comparison (#1635)
* fix(link): correct link cache mode bitwise comparison

Signed-off-by: MadDogOwner <xiaoran@xrgzs.top>

* refactor(link): use explicit flag equality for link cache mode bitmask checks

Signed-off-by: MadDogOwner <xiaoran@xrgzs.top>

---------

Signed-off-by: MadDogOwner <xiaoran@xrgzs.top>
2025-11-13 13:52:33 +08:00
KirCute
055696f576 feat(s3): support frontend direct upload (#1631)
* feat(s3): support frontend direct upload

* feat(s3): support custom direct upload host

* fix: apply suggestions of Copilot
2025-11-13 13:22:17 +08:00
KirCute
ee2c77acd8 fix(archive/zip): user specific encoding for non-EFS zips (#1599)
* fix(archive/zip): user specific encoding for non-EFS zips

* fix(stream): simplify head cache initialization and improve reader retrieval logic

* fix: support multipart zips (.z01)

* chore(deps): update github.com/KirCute/zip to v1.0.1

---------

Co-authored-by: j2rong4cn <j2rong@qq.com>
Co-authored-by: Pikachu Ren <40362270+PIKACHUIM@users.noreply.github.com>
2025-11-10 19:08:50 +08:00
ASLant
39dcf9bd19 feat(onedrive): support frontend direct upload (#1532)
* OneDrive添加直连上传

* refactor

* fix: duplicate root path join

---------

Co-authored-by: KirCute <951206789@qq.com>
2025-11-06 23:22:02 +08:00
j2rong4cn
a1f1f98f94 refactor(stream): simplify code (#1590)
* refactor(stream): simplify Close method and update SeekableStream to use RangeReader interface

* refactor(stream):  improve RangeRead comments for clarity
2025-11-06 20:06:48 +08:00
j2rong4cn
174eae802a perf(stream): optimize CacheFullAndWriter for better memory management (#1584)
* perf(stream): optimize CacheFullAndWriter for better memory management

* fix(stream): ensure proper seek handling in CacheFullAndWriter for improved data integrity
2025-11-05 12:16:09 +08:00
KirCute
b9f058fcc9 fix(backup-restore): add shares (#1500) 2025-11-05 12:11:20 +08:00
j2rong4cn
6de15b6310 feat(stream): enhance GetRangeReaderFromLink rate limiting (#1528)
* feat(stream): enhance GetRangeReaderFromLink rate limiting

* refactor(stream): update GetRangeReaderFromMFile to return *model.FileRangeReader

* refactor(stream): simplify context error handling in RateLimitReader, RateLimitWriter, and RateLimitFile

* refactor(net): replace custom LimitedReadCloser with readers.NewLimitedReadCloser

* fix(model): update Link.ContentLength JSON tag for correct serialization

* docs(model): add clarification to FileRangeReader usage comment
2025-11-04 23:56:09 +08:00
jenfonro
d88f0e8f3c feat(net): support proxy configuration via config file (#1359)
* support proxy

* debug

* debug2

* del debug

* add proxy configuration with env var fallback

* comments to en

* refactor(env): fallback env

---------

Co-authored-by: jyxjjj <773933146@qq.com>
2025-11-04 09:01:35 +08:00
Seven
66d9809057 feat(strm): strm local file (#1127)
* feat(strm): strm local file

* feat: 代码优化

* feat: 访问被strm挂载路径时也更新

* fix: 路径最后带/判断缺失

* fix: 路径最后带/判断缺失

* refactor

* refactor

* fix: close seekable-stream in `generateStrm`

* refactor: lazy create local file

* 优化路径判断

---------

Co-authored-by: KirCute <kircute@foxmail.com>
2025-11-03 10:48:15 +08:00
ShenLin
525f26dc23 feat(command): add --config flag to set custom config path (#1479) 2025-10-22 19:35:34 +08:00
ILoveScratch
15f276537c fix(share): remove share when user delete (#1493) 2025-10-19 22:48:21 +08:00
MadDogOwner
623a12050e feat(openlist): add PassIPToUpsteam to driver (#1498) 2025-10-19 22:45:40 +08:00
ILoveScratch
febbcd6027 feat(cache): improve cache management (#1339)
* feat(cache): improve cache management

* feat(disk-usage): add cache

* feat(disk-usage): add refresh

* fix(disk-usage): cache with ttl

* feat(cache): implement KeyedCache and TypedCache for improved caching mechanism

* fix(copy): update object retrieval to use Get instead of GetUnwrap

* refactor(cache): simplify DirectoryCache structure and improve object management

* fix(cache): correct cache entry initialization and key deletion logic in TypedCache

* refactor(driver): remove GetObjInfo interface and simplify Link function logic
https://github.com/OpenListTeam/OpenList/pull/888/files#r2430925783

* fix(link): optimize link retrieval and caching logic

* refactor(cache): consolidate cache management and improve directory cache handling

* fix(cache): add cache control based on storage configuration in List function

* .

* refactor: replace fmt.Sprintf with strconv for integer conversions

* refactor(cache): enhance cache entry management with Expirable interface

* fix(cache): improve link reference acquisition logic to handle expiration

* refactor: replace OnlyLinkMFile with NoLinkSF in driver configurations and logic

* refactor(link): enhance link caching logic with dynamic type keys based on IP and User-Agent

* feat(drivers): add LinkCacheType to driver configurations for enhanced caching

* refactor(cache): streamline directory object management in cache operations

* refactor(cache): remove unnecessary 'dirty' field from CacheEntry structure

* refactor(cache): replace 'dirty' field with bitwise flags

* refactor(io): 调高SyncClosers.AcquireReference的优先级

* refactor(link): 优化链接获取逻辑,增加重

* refactor(link): 添加RequireReference字段以增强链接管理

* refactor(link): 移除MFile字段,改用RangeReader

* refactor: 移除不必要的NoLinkSF字段

* refactor(cache): 修改目录缓存的脏标志定义和更新逻辑

* feat(cache): add expiration gc

---------

Co-authored-by: KirCute <951206789@qq.com>
Co-authored-by: KirCute <kircute@foxmail.com>
Co-authored-by: j2rong4cn <j2rong@qq.com>
2025-10-18 21:47:18 +08:00
jenfonro
549e60136b fix(fs):fix retry task after restart (#1467)
* fix retry task after restart

* fix: initialize SrcStorage and DstStorage in tasks to prevent nil pointer dereference

* feat: implement storage load signal mechanism for improved synchronization

* fix:  update StoragesLoaded logic

* refactor: reorganize storage loading logic and improve synchronization handling

---------

Co-authored-by: j2rong4cn <j2rong@qq.com>
2025-10-18 20:52:02 +08:00
KirCute
670e0bdc45 chore(frontend): optimize user operations (#1449) 2025-10-14 16:02:50 +08:00
ShenLin
89235012af feat(http3|quic): add http3|quic support (#1466)
* feat(http3|quic): add http3|quic support

* revert(ai): fix ai error

* fix(shutdown): shutdown was using close

* feat(http3|quic): add config if needs h3

* feat(http3|quic): add Alt-Svc to expose h3
2025-10-14 15:57:16 +08:00
KirCute
2bfbad2874 feat(offline_download): add 123 open (#1427) 2025-10-07 01:13:25 +08:00
KirCute
4ba7696032 feat(pikpak): support disk usage (#1426)
* feat(pikpak): support disk usage

* fix(alias): cannot list with details

* refactor: rename `NewDiskUsageFromUsedAndTotal`

* fix(disk-usage): get details of storages that is not initialized
2025-10-06 16:37:00 +08:00
NewbieOrange
4153245f2c fix(drivers): free space underflow if used larger than total space (#1407) 2025-10-04 00:41:45 +08:00
j2rong4cn
2edc446ced feat(stream): support using temporary files as large buffer (#1399)
feat(stream): refactor StreamSectionReader to support using temporary files as large buffer
2025-10-01 18:43:20 +08:00
j2rong4cn
da0c734aa3 fix(net): unable to pass HttpStatusCode (#1397) 2025-10-01 00:16:36 +08:00
ILoveScratch
189cebe4c9 feat(drivers): add MediaFire driver support (#1322)
* feat(drivers): add MediaFire driver support (#9319)

- Implement complete MediaFire storage driver
- Add authentication via session_token and cookie
- Support all core operations: List, Get, Link, Put, Copy, Move, Remove, Rename, MakeDir
- Include thumbnail generation for media files
- Handle MediaFire's resumable upload API with multi-unit transfers
- Add proper error handling and progress reporting

Co-authored-by: Da3zKi7 <da3zki7@duck.com>

* fix(mediafire): fix code errors in mediafire

* fix(mediafire): fix code errors in mediafire

* fix(drivers): add session renewal cron for MediaFire driver (#9321)

- Implement automatic session token renewal every 6-9 minutes
- Add validation for required SessionToken and Cookie fields in Init
- Handle session expiration by calling renewToken on validation failure
- Prevent storage failures due to MediaFire session timeouts

Fixes session closure issues that occur after server restarts or extended periods.

Co-authored-by: Da3zKi7 <da3zki7@duck.com>

* docs: restore README changes

Signed-off-by: ILoveScratch <ilovescratch@foxmail.com>

* fix

* fix

* fix: add stream upload limit

* fix

* fix: clear action token on drop and refactor header setting

* feat(drivers/mediafire): optimize file caching - support direct stream processing

- Remove forced caching to *os.File type
- Support generic model.File interface for better flexibility
- Improve upload efficiency by avoiding unnecessary file conversions
- Fix return type to use model.Object instead of model.ObjThumb

* feat(drivers/mediafire): improve global rate limiting

- Ensure all API methods properly use context for rate limiting
- Fix context parameter usage in getDirectDownloadLink, getActionToken, getFileByHash
- Maintain consistent rate limiting across all MediaFire API calls

* feat(drivers/mediafire): unify return types - remove unnecessary ObjThumb

- Change MakeDir, Rename, Copy methods to return model.Object instead of model.ObjThumb
- Remove empty Thumbnail fields where not meaningful
- Keep ObjThumb only for fileToObj (List operations) which provides actual thumbnail URLs
- Improve code consistency and reduce unnecessary wrapper objects

* refactor(drivers/mediafire): extract common error handling logic

- Add checkAPIResult helper function to reduce code duplication
- Replace repetitive MediaFire API error checks with centralized function
- Maintain specific error messages for unique cases (token, upload, search)
- Improve code maintainability and consistency

* enhance(drivers/mediafire): improve quick upload implementation

- Add null check for existingFile to prevent potential issues
- Improve error handling in quick upload - continue normal upload if search fails
- Add detailed comments explaining quick upload logic
- Optimize getExistingFileInfo with clearer fallback strategy
- Ensure upload reliability even when file search encounters issues

* refactor(drivers/mediafire): optimize request method reusability

- Extract common HTTP request logic into apiRequest method
- Reduce code duplication between getForm and postForm methods
- Maintain backward compatibility with existing method signatures
- Centralize rate limiting and header management
- Support extensible HTTP method handling

* docs(drivers/mediafire): add comprehensive English comments

- Add function-level comments for all major driver methods
- Document Init, List, Link, MakeDir, Move, Rename, Copy, Remove, Put methods
- Add comments for key utility functions including session token management
- Improve code readability and maintainability for community collaboration
- Follow Go documentation conventions with clear, concise descriptions

* perf(mediafire): optimize memory allocation and type assertion performance

- Pre-allocate slice capacity in getFiles and bitmap conversion to reduce reallocations
- Cache file type check in uploadUnits to avoid repeated type assertions
- Add uploadSingleUnitOptimized for os.File to eliminate redundant type checks
- Optimize string to int conversion with proper error handling
- Improve memory efficiency in file upload operations

* fix(mediafire): upload without cache

* feat(mediafire): add rate limiting to all API methods

- Add WaitLimit(ctx) calls to all driver methods: List, Link, MakeDir, Move, Rename, Copy, Remove, Put
- Ensure consistent rate limiting across all MediaFire API interactions
- Follow project standard pattern used by other drivers

* feat(mediafire): improve error handling consistency

- Add context parameter to all HTTP API functions for proper context propagation
- Update getForm, postForm and apiRequest to accept context parameter
- Fix rate limiting to use caller context instead of background context
- Ensure consistent error handling patterns across all API calls
- Improve cancellation and timeout support

* feat(mediafire): refactor resumableUpload to use io.ReadSeeker and improve upload handling

* fix(mediafire): release section reader

* feat: add disk usage

* feat(drivers/mediafire): support concurrent upload (#1387)

* feat(drivers): add MediaFire driver with concurrent upload support

- Implement complete MediaFire storage driver with session token authentication
- Support all core operations: List, Get, Link, Put, Copy, Move, Remove, Rename, MakeDir
- Include thumbnail generation for media files
- Handle MediaFire's resumable upload with intelligent and multi-unit transfers
- Support concurrent chunk uploads using errgroup.NewOrderedGroupWithContext, using splitted file caching for large files
- Optimize memory usage with adaptive buffer sizing (10MB-100MB (default))
- Include rate limiting and retry logic for API requests
- Add proper error handling and progress reporting
- Handle MediaFire's bitmap-based resumable upload protocol

Closes PR #1322

* feat(stream): add DiscardSection method to StreamSectionReader for skipping data

* feat(mediafire): refactor resumableUpload logic for improved upload handling and error management

* fix(mediafire): stop cron job and clear action token in Drop method

* .

* fix(mediafire): optimize buffer sizing logic in uploadUnits method

* fix(docs): remove duplicate MediaFire

* fix(mediafire): revert 'optimization', large files should not be fully chached.

---------

Signed-off-by: j2rong4cn <36783515+j2rong4cn@users.noreply.github.com>
Co-authored-by: Da3zKi7 <da3zki7@duck.com>
Co-authored-by: D@' 3z K!7 <99719341+Da3zKi7@users.noreply.github.com>
Co-authored-by: j2rong4cn <j2rong@qq.com>
Co-authored-by: j2rong4cn <36783515+j2rong4cn@users.noreply.github.com>

* fix(mediafire): optimize hash calculation in uploadUnits function

* feat(drivers/mediafire): support concurrent upload  (#1366)

* feat(drivers): add MediaFire driver with concurrent upload support

- Implement complete MediaFire storage driver with session token authentication
- Support all core operations: List, Get, Link, Put, Copy, Move, Remove, Rename, MakeDir
- Include thumbnail generation for media files
- Handle MediaFire's resumable upload with intelligent and multi-unit transfers
- Support concurrent chunk uploads using errgroup.NewOrderedGroupWithContext, using splitted file caching for large files
- Optimize memory usage with adaptive buffer sizing (10MB-100MB (default))
- Include rate limiting and retry logic for API requests
- Add proper error handling and progress reporting
- Handle MediaFire's bitmap-based resumable upload protocol

Closes PR #1322

* feat(stream): add DiscardSection method to StreamSectionReader for skipping data

* feat(mediafire): refactor resumableUpload logic for improved upload handling and error management

* fix(mediafire): stop cron job and clear action token in Drop method

* .

* fix(mediafire): optimize buffer sizing logic in uploadUnits method

* fix(docs): remove duplicate MediaFire

* fix(mediafire): revert 'optimization', large files should not be fully chached.

---------

Signed-off-by: j2rong4cn <36783515+j2rong4cn@users.noreply.github.com>
Signed-off-by: D@' 3z K!7 <99719341+Da3zKi7@users.noreply.github.com>
Co-authored-by: j2rong4cn <j2rong@qq.com>
Co-authored-by: j2rong4cn <36783515+j2rong4cn@users.noreply.github.com>

---------

Signed-off-by: ILoveScratch <ilovescratch@foxmail.com>
Signed-off-by: j2rong4cn <36783515+j2rong4cn@users.noreply.github.com>
Signed-off-by: D@' 3z K!7 <99719341+Da3zKi7@users.noreply.github.com>
Co-authored-by: D@' 3z K!7 <99719341+Da3zKi7@users.noreply.github.com>
Co-authored-by: Da3zKi7 <da3zki7@duck.com>
Co-authored-by: KirCute <951206789@qq.com>
Co-authored-by: Suyunmeng <Susus0175@proton.me>
Co-authored-by: j2rong4cn <j2rong@qq.com>
Co-authored-by: j2rong4cn <36783515+j2rong4cn@users.noreply.github.com>
2025-09-30 21:55:41 +08:00
KirCute
a2fc38be8d perf(disk-usage): concurrently get details (#1326) 2025-09-29 23:26:56 +08:00
KirCute
e0414e7110 fix(offline_download/http): attach UA to the request (#1347)
fix(simple-http): attach UA to the request
2025-09-29 22:13:00 +08:00
KirCute
b486af0031 feat(sftp-server): support disable password login (#1357) 2025-09-29 21:46:55 +08:00
j2rong4cn
ea09ce4b8f fix(fs): improve error handling in op.Get (#1323) 2025-09-29 21:38:34 +08:00
ILoveScratch
3c07144211 chore(announcement): change default announcement text (#1319)
* chore(announcement): change default announcement text for better clarity  

- Improve site announcement style for a more polished look
- Adjust default value to avoid unclear meaning with only repo or single link
2025-09-20 23:06:40 +08:00
KirCute
3936e736e6 feat(drivers): add a driver that divides large files into multiple chunks (#1153) 2025-09-19 19:27:35 +08:00
KirCute
cc16cb35bf feat(style): add driver icons and disk usage (#1274)
* feat(style): add driver icons and disk usage

* feat(driver): add disk usage for 115_open, 123_open, aliyundrive_open and baidu_netdisk

* feat(driver): add disk usage for crypt, sftp and smb

* chore: clean unused variable

* feat(driver): add disk usage for cloudreve_v4

Signed-off-by: MadDogOwner <xiaoran@xrgzs.top>

* fix(local): disk label check when getting disk usage

* feat(style): return details when accessing the manage page

---------

Signed-off-by: MadDogOwner <xiaoran@xrgzs.top>
Co-authored-by: MadDogOwner <xiaoran@xrgzs.top>
2025-09-19 11:59:11 +08:00
TwoOnefour
cbbb5ad231 fix(stream): http chucked upload issue (#1152)
* fix(stream): http chucked upload issue

* fix(stream): use MmapThreshold

* fix(stream): improve caching mechanism and handle size=0 case

* fix bug

* fix(buffer): optimize ReadAt method for improved performance

* fix(upload): handle Content-Length and File-Size headers for better size management

* fix(189pc): 移除重复限速

* fix(upload): handle negative file size during streaming uploads

* fix(upload): update header key from File-Size to X-File-Size for size retrieval

---------

Co-authored-by: j2rong4cn <j2rong@qq.com>
2025-09-15 19:36:16 +08:00
hshpy
c1d03c5bcc fix(security): zip slip (#1228)
* fix(security): Zip Slip

* chore:remove repeat clean

* fix: archives,iso9660 and rardecode module

---------

Co-authored-by: ILoveScratch <ilovescratch@foxmail.com>
2025-09-15 13:25:21 +08:00
KirCute
bbb7c06504 feat(alias): support pass through provider (#1269) 2025-09-14 21:36:38 +08:00
walloo
89f35170b3 fix(fs): clear cache after directory rename to ensure consistency (#1193)
Clear cache after renaming the directory.
2025-09-01 18:47:54 +08:00
hshpy
04a5e58781 fix(server): can't edit .md source files (#1159)
* fix(server): can't edit .md source files

* chore

* add ignore direct link args
2025-08-28 16:19:57 +08:00
KirCute
e4c902dd93 feat(share): support more secure file sharing (#991)
提供一种类似大多数网盘的文件分享操作,这种分享方式可以通过强制 Web 代理隐藏文件源路径,可以设置分享码、最大访问数和过期时间,并且不需要启用 guest 用户。

在全局设置中可以调整:
- 是否强制 Web 代理
- 是否允许预览
- 是否允许预览压缩文件
- 分享文件后,点击“复制链接”按钮复制的内容

前端部分:OpenListTeam/OpenList-Frontend#156
文档部分:OpenListTeam/OpenList-Docs#130

Close #183
Close #526
Close #860
Close #892
Close #1079


* feat(share): support more secure file sharing

* feat(share): add archive preview

* fix(share): fix some bugs

* feat(openlist_share): add openlist share driver

* fix(share): lack unwrap when get virtual path

* fix: use unwrapPath instead of path for virtual file name comparison

* fix(share): change request method of /api/share/list from GET to Any

* fix(share): path traversal vulnerability in sharing path check

* 修复分享alias驱动的文件 没开代理时无法获取URL

* fix(sharing): update error message for sharing root link extraction

---------

Co-authored-by: Suyunmeng <69945917+Suyunmeng@users.noreply.github.com>
Co-authored-by: j2rong4cn <j2rong@qq.com>
2025-08-19 15:10:02 +08:00
j2rong4cn
016ed90efa feat(stream): fast buffer freeing for large cache (#1053)
Signed-off-by: j2rong4cn <36783515+j2rong4cn@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2025-08-16 17:19:52 +08:00
j2rong4cn
57fceabcf4 perf(stream): improve file stream range reading and caching mechanism (#1001)
* perf(stream): improve file stream range reading and caching mechanism

* 。

* add bytes_test.go

* fix(stream): handle EOF and buffer reading more gracefully

* 注释

* refactor: update CacheFullAndWriter to accept pointer for UpdateProgress

* update tests

* Update drivers/google_drive/util.go

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Signed-off-by: j2rong4cn <36783515+j2rong4cn@users.noreply.github.com>

* 更优雅的克隆Link

* 修复stream已缓存但无法重复读取

* 将Bytes类型重命名为Reader

* 修复栈溢出

* update tests

---------

Signed-off-by: j2rong4cn <36783515+j2rong4cn@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2025-08-11 23:41:22 +08:00
MadDogOwner
8c244a984d refactor(assets): migrate to resource domain (#975)
* refactor(assets): migrate to resource domain

* feat(bootstrap): add migration value for logo and favicon settings
2025-08-10 09:57:33 +08:00
j2rong4cn
9ac0484bc0 perf(ftp): improve concurrent Link response; fix alias/local driver issues (#974) 2025-08-06 13:32:37 +08:00
j2rong4cn
8cf15183a0 perf: optimize upload (#554)
* pref(115,123): optimize upload

* chore

* aliyun_open, google_drive

* fix bug

* chore

* cloudreve, cloudreve_v4, onedrive, onedrive_app

* chore(conf): add `max_buffer_limit` option

* 123pan multithread upload

* doubao

* google_drive

* chore

* chore

* chore: 计算分片数量的代码

* MaxBufferLimit自动挡

* MaxBufferLimit自动挡

* 189pc

* errorgroup添加Lifecycle

* 查缺补漏

* Conf.MaxBufferLimit单位为MB

* 。

---------

Co-authored-by: MadDogOwner <xiaoran@xrgzs.top>
2025-08-05 21:42:54 +08:00
j2rong4cn
1208bd0a83 fix(fs): nil interface not equal to nil (#971)
https://go.dev/doc/faq#nil_error
2025-08-03 23:51:11 +08:00
j2rong4cn
6b096bcad4 fix(fs): deadlock when get link error (#963) 2025-08-02 17:49:53 +08:00
Seven
58dbf088f9 fix(fs): forget cache when get link error (#956) 2025-08-02 11:03:34 +08:00
MadDogOwner
e458f2ab53 fix(bootstrap): add newline after initial admin password output (#943)
fix(bootstrap): add newline after initial admin  password output
2025-08-01 13:43:41 +08:00
hshpy
9469c95b14 fix(security): potential XSS vulnerabilities (#896) 2025-07-31 12:57:20 +08:00
MadDogOwner
cf912dcf7a fix(cmd): output to console (#920)
fix(cmd): output to terminal
2025-07-31 11:44:00 +08:00