Commit Graph

111 Commits

Author SHA1 Message Date
ILoveScratch
febbcd6027 feat(cache): improve cache management (#1339)
* feat(cache): improve cache management

* feat(disk-usage): add cache

* feat(disk-usage): add refresh

* fix(disk-usage): cache with ttl

* feat(cache): implement KeyedCache and TypedCache for improved caching mechanism

* fix(copy): update object retrieval to use Get instead of GetUnwrap

* refactor(cache): simplify DirectoryCache structure and improve object management

* fix(cache): correct cache entry initialization and key deletion logic in TypedCache

* refactor(driver): remove GetObjInfo interface and simplify Link function logic
https://github.com/OpenListTeam/OpenList/pull/888/files#r2430925783

* fix(link): optimize link retrieval and caching logic

* refactor(cache): consolidate cache management and improve directory cache handling

* fix(cache): add cache control based on storage configuration in List function

* .

* refactor: replace fmt.Sprintf with strconv for integer conversions

* refactor(cache): enhance cache entry management with Expirable interface

* fix(cache): improve link reference acquisition logic to handle expiration

* refactor: replace OnlyLinkMFile with NoLinkSF in driver configurations and logic

* refactor(link): enhance link caching logic with dynamic type keys based on IP and User-Agent

* feat(drivers): add LinkCacheType to driver configurations for enhanced caching

* refactor(cache): streamline directory object management in cache operations

* refactor(cache): remove unnecessary 'dirty' field from CacheEntry structure

* refactor(cache): replace 'dirty' field with bitwise flags

* refactor(io): 调高SyncClosers.AcquireReference的优先级

* refactor(link): 优化链接获取逻辑,增加重

* refactor(link): 添加RequireReference字段以增强链接管理

* refactor(link): 移除MFile字段,改用RangeReader

* refactor: 移除不必要的NoLinkSF字段

* refactor(cache): 修改目录缓存的脏标志定义和更新逻辑

* feat(cache): add expiration gc

---------

Co-authored-by: KirCute <951206789@qq.com>
Co-authored-by: KirCute <kircute@foxmail.com>
Co-authored-by: j2rong4cn <j2rong@qq.com>
2025-10-18 21:47:18 +08:00
MadDogOwner
b04677b806 feat(server): add error page and status code (#1099) 2025-08-19 15:18:12 +08:00
SenkjM
5a0d8ee1b8 feat(proxy): add disable proxy sign (#764)
* feat(proxy): add disable proxy sign

* Update driver.go

* GenerateDownProxyUrl

* .

* Update internal/op/driver.go

Signed-off-by: j2rong4cn <36783515+j2rong4cn@users.noreply.github.com>

* .

---------

Signed-off-by: j2rong4cn <36783515+j2rong4cn@users.noreply.github.com>
Co-authored-by: j2rong4cn <j2rong@qq.com>
Co-authored-by: j2rong4cn <36783515+j2rong4cn@users.noreply.github.com>
2025-07-21 17:03:08 +08:00
pnparadise
86324d2d6b fix(net): ensure accurate content-length in response (#749)
* fix(fs): ensure accurate content-length in http2 requests

Chrome browsers were unable to preview thumbnails, reporting an
'ERR_HTTP_2_PROTOCOL_ERROR'. This was caused by an incorrect
content-length header in the server's response for thumbnail images.

This commit corrects the content-length calculation, allowing
Chrome and other compliant clients to render thumbnails correctly.

* fix(net): ensure accurate content-length in response

* 补缺

* .

---------

Co-authored-by: zhiqiang.huang <zhiqiang.tech@gmail.com>
Co-authored-by: j2rong4cn <j2rong@qq.com>
2025-07-19 20:36:27 +08:00
j2rong4cn
0c461991f9 chore: standardize context keys with custom ContextKey type (#697)
* chore: standardize context keys with custom ContextKey type

* fix bug

* 使用Request.Context
2025-07-14 23:55:17 +08:00
j2rong4cn
2a4c546a8b feat: default settings api (#716)
* feat: default settings api

* fix logic bug

* chore
2025-07-14 23:41:34 +08:00
j2rong4cn
cc01b410a4 perf(link): optimize concurrent response (#641)
* fix(crypt): bug caused by link cache

* perf(crypt,mega,halalcloud,quark,uc): optimize concurrent response link

* chore: 删除无用代码

* ftp

* 修复bug;资源释放

* 添加SyncClosers

* local,sftp,smb

* 重构,优化,增强

* Update internal/stream/util.go

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Signed-off-by: j2rong4cn <36783515+j2rong4cn@users.noreply.github.com>

* chore

* chore

* 优化,修复bug

* .

---------

Signed-off-by: j2rong4cn <36783515+j2rong4cn@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2025-07-12 17:57:54 +08:00
j2rong4cn
ffb6c2a180 refactor: optimize stream, link, and resource management (#486)
* refactor: optimize stream, link, and resource management

* Link.MFile改为io.ReadSeeker类型

* fix (crypt): read on closed response body

* chore

* chore

* chore
2025-07-03 10:39:34 +08:00
MadDogOwner
874dc292ae fix(gomod): go modules with tagged versions (#499)
fix: go modules with tagged versions
2025-07-01 09:54:50 +08:00
Suyunjing
52c93f2046 build:Update old dependencies to new hosting (#495)
* build:Update old dependencies to new hosting
2025-06-30 21:26:42 +08:00
j2rong4cn
103abc942e refactor: pass api_url through context (#457)
* refactor: pass `api_url` through context

* 移除 LinkArgs.HttpReq

* pref(alias): 减少不必要下载代理

* 修复bug

* net: 支持1并发 分片下载
2025-06-30 15:48:05 +08:00
XZB-1248
02031bd835 feat(s3): add Content-Disposition header (#365)
* add(s3): add Content-Disposition header

* Update driver.go

Signed-off-by: XZB-1248 <28593573+XZB-1248@users.noreply.github.com>

* Update driver.go

Signed-off-by: XZB-1248 <28593573+XZB-1248@users.noreply.github.com>

---------

Signed-off-by: XZB-1248 <28593573+XZB-1248@users.noreply.github.com>
Co-authored-by: XZB-1248 <i@1248.ink>
Co-authored-by: Suyunjing <69945917+Suyunmeng@users.noreply.github.com>
2025-06-27 15:29:08 +08:00
Kuingsmile
fdcc2f136e chore: change module name to OpenListTeam/OpenList (#2)
* Enable blank issue

* chore(README.md): update docs (temporally)

* Update FUNDING.yml

* chore: purge README.md

* chore: change module name to OpenListTeam/OpenList

* fix: fix link errors

* chore: remove v3 in module name

* fix: resolve some conficts

* fix: resolve conficts

* docs: update with latest file

---------

Co-authored-by: ShenLin <773933146@qq.com>
Co-authored-by: Hantong Chen <cxwdyx620@gmail.com>
Co-authored-by: joshua <i@joshua.su>
Co-authored-by: Hantong Chen <70561268+cxw620@users.noreply.github.com>
2025-06-12 22:02:46 +08:00
j2rong4cn
a2f266277c fix(net): unexpected write (#8291 close #8281) 2025-04-12 17:01:52 +08:00
j2rong4cn
a6304285b6 fix: revert "refactor(net): pass request header" (#8269)
5be50e77d9
2025-04-03 20:35:52 +08:00
j2rong4cn
b4e6ab12d9 refactor: FilterReadMeScripts (#8154 close #8150)
* refactor: FilterReadMeScripts

* .
2025-03-18 22:02:33 +08:00
hshpy
d20f41d687 fix: missing handling of RangeReadCloser (#8146) 2025-03-16 22:14:44 +08:00
hshpy
c82e632ee1 fix: potential XSS vulnerabilities (#7923)
* fix: potential XSS vulnerabilities

* feat: support filter and render for readme.md

* chore: set ReadMeAutoRender to true

* fix attachFileName undefined

---------

Co-authored-by: Andy Hsu <i@nn.ci>
2025-03-15 23:28:40 +08:00
j2rong4cn
0126af4de0 fix(crypt): premature close of MFile (#8132 close #8119)
* fix(crypt): premature close of MFile

* refactor
2025-03-15 00:13:30 +08:00
j2rong4cn
4145734c18 refactor(net): pass request header (#8031 close #8008)
* refactor(net): pass request header

* feat(proxy): add `Etag` to response header

* refactor
2025-03-01 18:35:34 +08:00
KirCute_ECT
3b71500f23 feat(traffic): support limit task worker count & file stream rate (#7948)
* feat: set task workers num & client stream rate limit

* feat: server stream rate limit

* upgrade xhofe/tache

* .
2025-02-16 12:22:11 +08:00
Jealous
b9ad18bd0a feat(recursive-move): Advanced conflict policy for preventing unintentional overwriting (#7906) 2025-02-09 18:32:57 +08:00
j2rong4cn
2be0c3d1a0 feat(alias): add DownloadConcurrency and DownloadPartSize option (#7829)
* fix(net): goroutine logic bug (AlistGo/alist#7215)

* Fix goroutine logic bug

* Fix bug

---------

Co-authored-by: hpy hs <hshpy.pengyu@gmail.com>

* perf(net): sequential and dynamic concurrency

* fix(net): incorrect error return

* feat(alias):  add `DownloadConcurrency` and `DownloadPartSize` option

* feat(net): add `ConcurrencyLimit`

* pref(net): create `chunk` on demand

* refactor

* refactor

* fix(net): `r.Closers.Add` has no effect

* refactor

---------

Co-authored-by: hpy hs <hshpy.pengyu@gmail.com>
2025-01-27 20:08:39 +08:00
j2rong4cn
db5c601cfe fix(crypt): add sign to thumbnail (#6611) 2024-12-25 21:13:54 +08:00
j2rong4cn
db99224126 perf: Speed ​​of database initialization (#7694)
* perf: 优化非sqlite3数据库时初始化慢的问题

* refactor
2024-12-25 21:08:22 +08:00
itsHenry
81258d3e8a feat: invalidate token on logout (#6923 close #6792) 2024-08-04 12:32:39 +08:00
j2rong4cn
5f60b51cf8 feat: add proxy_range option for 139Yun Alias AList V3 (#6496) 2024-05-22 23:31:42 +08:00
Andy Hsu
57bac9e0d2 fix: some missing regexp lib modified 2024-01-01 18:44:59 +08:00
Andy Hsu
d26887d211 fix: content-type conflicts with #5420 2023-11-24 19:22:19 +08:00
MuGu
867accafd1 fix(local): video file thumbnails not displaying on iOS Safari (#5420)
* perf(webdav): support for cookies on webdav drive

* fix(local): video file thumbnails not displaying on iOS Safari
2023-11-18 22:36:41 +08:00
Andy Hsu
3d51845f57 feat: invalidate old token after changing the password (close #5515) 2023-11-13 15:22:42 +08:00
Andy Hsu
4355dae491 fix: incorrect content-type of apk files (close #5385) 2023-11-06 18:20:25 +08:00
Andy Hsu
34fadecc2c fix(ftp): dead lock on Read (close #5128) 2023-08-31 15:10:47 +08:00
Sean
a3748af772 feat: misc improvements about upload/copy/hash (#5045)
general: add createTime/updateTime support in webdav and some drivers
general: add hash support in some drivers
general: cross-storage rapid-upload support
general: enhance upload to avoid local temp file if possible
general: replace readseekcloser with File interface to speed upstream operations
feat(aliyun_open): same as above
feat(crypt): add hack for 139cloud

Close #4934 
Close #4819 

baidu_netdisk needs to improve the upload code to support rapid-upload
2023-08-27 21:14:23 +08:00
Andy Hsu
5606c23768 perf(copy): use multi-thread downloader (close #5000) 2023-08-13 15:31:49 +08:00
Sean
15b7169df4 perf: multi-thread downloader, Content-Disposition (#4921)
general: enhance multi-thread downloader with cancelable context, immediately stop all stream processes when canceled;
feat(crypt): improve stream closing;
general: fix the bug of downloading files becomes previewing stream on modern browsers;

Co-authored-by: Sean He <866155+seanhe26@users.noreply.github.com>
Co-authored-by: Andy Hsu <i@nn.ci>
2023-08-04 15:29:54 +08:00
Sean
3c21a9a520 feat: Crypt driver, improve http/webdav handling (#4884)
this PR has several enhancements, fixes, and features:
- [x] Crypt: a transparent encryption driver. Anyone can easily, and safely store encrypted data on the remote storage provider.  Consider your data is safely stored in the safe, and the storage provider can only see the safe, but not your data.
  - [x] Optional: compatible with [Rclone Crypt](https://rclone.org/crypt/). More ways to manipulate the encrypted data.
  - [x] directory and filename encryption
  - [x] server-side encryption mode (server encrypts & decrypts all data, all data flows thru the server)
- [x] obfuscate sensitive information internally
- [x] introduced a server memory-cached multi-thread downloader.
  - [x] Driver: **Quark** enabled this feature, faster load in any single thread scenario. e.g. media player directly playing from the link, now it's faster.
- [x] general improvement on HTTP/WebDAV stream processing & header handling & response handling
  - [x] Driver: **Mega** driver support ranged http header
  - [x] Driver: **Quark** fix bug of not closing HTTP request to Quark server while user end has closed connection to alist

## Crypt, a transparent Encrypt/Decrypt Driver. (Rclone Crypt compatible)

e.g.  
Crypt mount path ->  /vault 
Crypt remote path -> /ali/encrypted
Aliyun mount paht -> /ali

when the user uploads a.jpg to /vault, the data will be encrypted and saved to /ali/encrypted/xxxxx. And when the user wants to access a.jpg,  it's automatically decrypted, and the user can do anything with it.
Since it's Rclone Crypt compatible, users can download /ali/encrypted/xxxxx  and decrypt it with rclone crypt tool. Or the user can mount this folder using rclone, then mount the decrypted folder in Linux...

NB.  Some breaking changes is made to make it follow global standard, e.g. processing the HTTP header properly.

close #4679 
close #4827 

Co-authored-by: Sean He <866155+seanhe26@users.noreply.github.com>
Co-authored-by: Andy Hsu <i@nn.ci>
2023-08-02 14:40:36 +08:00
Andy Hsu
3f8b3da52b feat(server): add HEAD method support (close #4740) 2023-07-11 13:47:49 +08:00
Andy Hsu
b2f5757f8d fix(copy): copy from driver that return writer (close #4291) 2023-05-26 21:57:43 +08:00
XYUU
a3446720a2 fix: make TlsInsecureSkipVerify enable for all request (#4386) 2023-05-14 17:05:47 +08:00
Andy Hsu
8711f2a1c5 feat(quark): shard request file (close #4175) 2023-04-17 15:33:38 +08:00
Andy Hsu
7f35aab071 revert(quark): remove preset range header 2023-04-17 14:39:21 +08:00
Andy Hsu
ecd167d2f9 feat(quark): add preset range header (close #4166) 2023-04-16 19:26:03 +08:00
Andy Hsu
3b07c72f88 fix(proxy): ignore Referer if got redirect (close #3996) 2023-03-31 20:29:55 +08:00
BaiYi
112363031a feat: add fine-grained control for link signing (#3924)
* Determine whether the URL requires Sign

* Add File and Mem based KV

NOT TESTED: TokenKV Function

* Change Token KV func to common func.

Add File based KV func

* Remove KV, Remove Token

I found that the original Sign function is enough to complete the link signature, and only need to add simple configuration items to meet the requirements.

* Add IsStorageSigned func to judge if Signing is enabled in the storage settings.

It should be working now.

* Add a SIGN button to the management panel.

* Add enable_sign to the basic storage struct.

Can enable sign for every driver now.

Bug: When sign enabled, in download page, Copy link doesn't contain a sign.

(Not done yet)

* Fix a bug from commit 8f6c25f.

Response of fsread function does not contain sign.

* Optimize code and follow advices.

- Add back public/dist/README.md

- Enable sign when DownProxyUrl is enabled

- Merge needSign() to isEncrypt() in fsread.go

* simplify code

---------

Co-authored-by: Andy Hsu <i@nn.ci>
2023-03-24 22:44:33 +08:00
XZB-1248
0eab31bdf5 fix(local): filename with whitespace issue (#3928)
* fix(local): filename whitespace problem

* fix(deps): remove deprecated package io/ioutil

---------

Co-authored-by: XZB <i@1248.ink>
2023-03-23 15:18:37 +08:00
Andy Hsu
d9795ff22f feat(alias): support proxy and direct together 2023-03-14 13:46:27 +08:00
XZB-1248
c4108007cd fix: spaces in filename will be replaced with plus sign (#3841)
Co-authored-by: XZB <i@1248.ink>
2023-03-14 12:27:42 +08:00
Andy Hsu
43de823058 fix: path IsApply check (close #3784) 2023-03-09 21:03:56 +08:00
Andy Hsu
12a095a1d6 fix: slice bounds out of range on CanAccess check 2023-03-05 15:29:53 +08:00