- Introduced `sanitizeName` function to remove four-byte characters (e.g., emojis) from names before upload or creation.
- Added `StripEmoji` option in driver configurations for cloud189 and cloud189pc.
- Updated file and folder operations (upload, rename, and creation) to use sanitized names.
- Ensured compatibility with both cloud189 and cloud189pc implementations.
* feat(driver): Added support for Gitee driver
- Implemented core driver functions including initialization, file listing, and file linking
- Added Gitee-specific API interaction and object mapping
- Registered Gitee driver in the driver registry
* feat(driver): Added cookie-based authentication support for Gitee driver
- Extended request handling to include `Cookie` header if provided
- Updated metadata to include `cookie` field with appropriate documentation
- Adjusted file link generation to propagate `Cookie` headers in requests
Introduce a `DeviceFingerprint` field to the request metadata.
This field is used to conditionally set the `X-Device-Fingerprint`
HTTP header in outgoing requests if its value is not empty.
* feat(bitqiu): Add Bitqiu cloud drive support
- Implement the new Bitqiu cloud drive.
- Add core driver logic, metadata handling, and utility functions.
- Register the Bitqiu driver for use.
* feat(driver): Implement GetLink, CreateDir, and Move operations
- Implement `GetLink` method to retrieve download links for files.
- Implement `CreateDir` method to create new directories.
- Implement `Move` method to relocate files and directories.
- Add new API endpoints and data structures for download and directory creation responses.
- Integrate retry logic with re-authentication for API calls in implemented methods.
- Update HTTP request headers to include `x-requested-with`.
* feat(bitqiu): Add rename, copy, and delete operations
- Implement `Rename` operation with retry logic and API calls.
- Implement `Copy` operation, including asynchronous handling, polling for completion, and status checks.
- Implement `Remove` operation with retry logic and API calls.
- Add new API endpoint URLs for rename, copy, and delete, and a new copy success code.
- Introduce `AsyncManagerData`, `AsyncTask`, and `AsyncTaskInfo` types to support async copy status monitoring.
- Add utility functions `updateObjectName` and `parentPathOf` for object manipulation.
- Integrate login retry mechanism for all file operations.
* feat(bitqiu-upload): Implement chunked file upload support
- Implement multi-part chunked upload logic for the BitQiu service.
- Introduce `UploadInitData` and `ChunkUploadResponse` structs for structured API communication.
- Refactor the `Save` method to orchestrate initial upload, chunked data transfer, and finalization.
- Add `uploadFileInChunks` function to handle sequential uploading of file parts.
- Add `completeChunkUpload` function to finalize the chunked upload process on the server.
- Ensure proper temporary file cleanup using `defer tmpFile.Close()`.
* feat(driver): Implement automatic root folder ID retrieval
- Add `userInfoURL` constant for fetching user information.
- Implement `ensureRootFolderID` function to retrieve and set the driver's root folder ID if not already present.
- Integrate `ensureRootFolderID` into the driver's `Init` process.
- Define `UserInfoData` struct to parse the `rootDirId` from user information responses.
* feat(client): Implement configurable user agent
* Introduce a configurable `UserAgent` field in the client's settings.
* Add a `userAgent()` method to retrieve the user agent, prioritizing the custom setting or using a predefined default.
* Apply the determined user agent to all outbound HTTP requests made by the `BitQiu` client.
* feat(s3): Add support for S3 object storage classes
Introduces a new 'storage_class' configuration option for S3 providers. Users can now specify the desired storage class (e.g., Standard, GLACIER, DEEP_ARCHIVE) for objects uploaded to S3-compatible services like AWS S3 and Tencent COS.
The input storage class string is normalized to match AWS SDK constants, supporting various common aliases. If an unknown storage class is provided, it will be used as a raw value with a warning. This enhancement provides greater control over storage costs and data access patterns.
* feat(storage): Support for displaying file storage classes
Adds storage class information to file metadata and API responses.
This change introduces the ability to store file storage classes in file metadata and display them in API responses. This allows users to view a file's storage tier (e.g., S3 Standard, Glacier), enhancing data management capabilities.
Implementation details include:
- Introducing the StorageClassProvider interface and the ObjWrapStorageClass structure to uniformly handle and communicate object storage class information.
- Updated file metadata structures (e.g., ArchiveObj, FileInfo, RespFile) to include a StorageClass field.
- Modified relevant API response functions (e.g., GetFileInfo, GetFileList) to populate and return storage classes.
- Integrated functionality for retrieving object storage classes from underlying storage systems (e.g., S3) and wrapping them in lists.
* feat(driver/s3): Added the "Other" interface and implemented it by the S3 driver.
A new `driver.Other` interface has been added and defined in the `other.go` file.
The S3 driver has been updated to implement this new interface, extending its functionality.
* feat(s3): Add S3 object archive and thaw task management
This commit introduces comprehensive support for S3 object archive and thaw operations, managed asynchronously through a new task system.
- **S3 Transition Task System**:
- Adds a new `S3Transition` task configuration, including workers, max retries, and persistence options.
- Initializes `S3TransitionTaskManager` to handle asynchronous S3 archive/thaw requests.
- Registers dedicated API routes for monitoring S3 transition tasks.
- **Integrate S3 Archive/Thaw with Other API**:
- Modifies the `Other` API handler to intercept `archive` and `thaw` methods for S3 storage drivers.
- Dispatches these operations as `S3TransitionTask` instances to the task manager for background processing.
- Returns a task ID to the client for tracking the status of the dispatched operation.
- **Refactor `other` package for improved API consistency**:
- Exports previously internal structs such as `archiveRequest`, `thawRequest`, `objectDescriptor`, `archiveResponse`, `thawResponse`, and `restoreStatus` by making their names public.
- Makes helper functions like `decodeOtherArgs`, `normalizeStorageClass`, and `normalizeRestoreTier` public.
- Introduces new constants for various S3 `Other` API methods.
Introduces the ability to sign generated URLs for enhanced security and access control.
This feature is activated by configuring a `PrivateKey`, `UID`, and `ValidDuration` in the driver settings. If a private key is provided, the driver will sign the output URLs, making them time-limited based on the `ValidDuration`. The `ValidDuration` defaults to 30 minutes if not specified.
The core signing logic is encapsulated in the new `sign.go` file. The `driver.go` file integrates this signing process before returning the final URL.
- Implement complete ProtonDrive storage driver with end-to-end encryption support
- Add authentication via username/password with credential caching and reusable login
- Support all core operations: List, Link, Put, Copy, Move, Remove, Rename, MakeDir
- Include encrypted file operations with PGP key management and node passphrase handling
- Add temporary HTTP server for secure file downloads with range request support
- Support media streaming using temp server range requests
- Implement progress tracking for uploads and downloads
- Support directory operations with circular move detection
- Add proper error handling and panic recovery for external library integration
Closes#9312
- Implement OAuth2 authentication with US/EU region support
- Add file operations (list, upload, download, delete, rename, move, copy)
- Add folder operations (create, rename, move, delete)
- Enhance error handling with pCloud-specific retry logic
- Use correct API methods: GET for reads, POST for writes
- Implement direct upload approach for better performance
- Add exponential backoff for failed requests with 4xxx/5xxx classification
* feat(driver): add Gofile storage driver
Add support for Gofile.io cloud storage service with full CRUD operations.
Features:
- File and folder listing
- Upload and download functionality
- Create, move, rename, copy, and delete operations
- Direct link generation for file access
- API token authentication
The driver implements all required driver interfaces and follows
the existing driver patterns in the codebase.
* feat(gofile): add configurable link expiration handling
- Adjusts driver addition metadata to accept LinkExpiry and DirectLinkExpiry options for caching and API expiry control (drivers/gofile/meta.go:10).
- Applies the new options when building file links, setting optional local cache expiration (drivers/gofile/driver.go:101) and sending an expireTime to the direct-link API (drivers/gofile/util.go:202).
- Logs Gofile API error payloads and validates the structured error response before returning it (drivers/gofile/util.go:141).
- Adds the required imports and returns the configured model.Link instance (drivers/gofile/driver.go:6).
- Use `filePath` variable to simplify path handling
- Replace `isSymlinkDir` with `isLinkedDir` in `isFolder` check
- Use simplified path variables in `times.Stat` function calls
refactor(util): Optimized directory link check functions
- Renamed `isSymlinkDir` to `isLinkedDir` to expand Windows platform support
- Corrected path resolution logic to ensure link paths are absolute
- Added error handling to prevent path resolution failures
- Implement automatic session token renewal every 6-9 minutes
- Add validation for required SessionToken and Cookie fields in Init
- Handle session expiration by calling renewToken on validation failure
- Prevent storage failures due to MediaFire session timeouts
Fixes session closure issues that occur after server restarts or extended periods.
Co-authored-by: Da3zKi7 <da3zki7@duck.com>
Add support for Gofile.io cloud storage service with full CRUD operations.
Features:
- File and folder listing
- Upload and download functionality
- Create, move, rename, copy, and delete operations
- Direct link generation for file access
- API token authentication
The driver implements all required driver interfaces and follows
the existing driver patterns in the codebase.
* feat(meta): Added a SafePassword field
- Added the SafePassword field to meta.go
- Revised the field format to align with the code style
- The SafePassword field is used to supplement the extended functionality
* feat(driver): Added support for safe unlocking logic
- Added safe file unlocking logic in `driver.go`, returning an error if unlocking fails.
- Introduced the `safeBoxUnlocked` variable of type `sync.Map` to record the IDs of unlocked files.
- Enhanced error handling logic to automatically attempt to unlock safe files and re-retrieve the file list.
- Added the `IsLock` field to file types in `types.go` to identify whether they are safe files.
- Added a constant definition for the `SafeBoxUnlock` interface address in `util.go`.
- Added the `unlockSafeBox` method to unlock a safe with a specified file ID via the API.
- Optimized the file retrieval logic to automatically call the unlock method when the safe is locked.
* Refactor (driver): Optimize lock field type
- Changed the `IsLock` field type from `int` to `bool` for better semantics.
- Updated the check logic to use direct Boolean comparisons to improve code readability and accuracy.
- Adjusted logic to skip incrementing index when exiting comments.
- Added checks to continue loop if inside a single-line or block comment.
- Prevents erroneous parsing and retains intended comment exclusion.
- Add `IntSlice` type to handle both single int and array in JSON.
- Modify `MeResp` struct to use `IntSlice` for `Role` field.
- Import `encoding/json` for JSON operations.
- Implement new driver for 123 Open service, enabling file operations
such as listing, uploading, moving, and removing files.
- Introduce token management for authentication and authorization.
- Add API integration for various file operations and actions.
- Include utility functions for handling API requests and responses.
- Register the new driver in the existing drivers' list.
* 标签管理
* pr检查优化
* feat(role): Implement role management functionality
- Add role management routes in `server/router.go` for listing, getting, creating, updating, and deleting roles
- Introduce `initRoles()` in `internal/bootstrap/data/data.go` for initializing roles during bootstrap
- Create `internal/op/role.go` to handle role operations including caching and singleflight
- Implement role handler functions in `server/handles/role.go` for API responses
- Define database operations for roles in `internal/db/role.go`
- Extend `internal/db/db.go` for role model auto-migration
- Design `internal/model/role.go` to represent role structure with ID, name, description, base path, and permissions
- Initialize default roles (`admin` and `guest`) in `internal/bootstrap/data/role.go` during startup
* refactor(user roles): Support multiple roles for users
- Change the `Role` field type from `int` to `[]int` in `drivers/alist_v3/types.go` and `drivers/quqi/types.go`.
- Update the `Role` field in `internal/model/user.go` to use a new `Roles` type with JSON and database support.
- Modify `IsGuest` and `IsAdmin` methods to check for roles using `Contains` method.
- Update `GetUserByRole` method in `internal/db/user.go` to handle multiple roles.
- Add `roles.go` to define a new `Roles` type with JSON marshalling and scanning capabilities.
- Adjust code in `server/handles/user.go` to compare roles with `utils.SliceEqual`.
- Change role initialization for users in `internal/bootstrap/data/dev.go` and `internal/bootstrap/data/user.go`.
- Update `Role` handling in `server/handles/task.go`, `server/handles/ssologin.go`, and `server/handles/ldap_login.go`.
* feat(user/role): Add path limit check for user and role permissions
- Add new permission bit for checking path limits in `user.go`
- Implement `CheckPathLimit` method in `User` struct to validate path access
- Modify `JoinPath` method in `User` to enforce path limit checks
- Update `role.go` to include path limit logic in `Role` struct
- Document new permission bit in `Role` and `User` comments for clarity
* feat(permission): Add role-based permission handling
- Introduce `role_perm.go` for managing user permissions based on roles.
- Implement `HasPermission` and `MergeRolePermissions` functions.
- Update `webdav.go` to utilize role-based permissions instead of direct user checks.
- Modify `fsup.go` to integrate `CanAccessWithRoles` function.
- Refactor `fsread.go` to use `common.HasPermission` for permission validation.
- Adjust `fsmanage.go` for role-based access control checks.
- Enhance `ftp.go` and `sftp.go` to manage FTP access via roles.
- Update `fsbatch.go` to employ `MergeRolePermissions` for batch operations.
- Replace direct user permission checks with role-based permission handling across various modules.
* refactor(user): Replace integer role values with role IDs
- Change `GetAdmin()` and `GetGuest()` functions to retrieve role by name and use role ID.
- Add patch for version `v3.45.2` to convert legacy integer roles to role IDs.
- Update `dev.go` and `user.go` to use role IDs instead of integer values for roles.
- Remove redundant code in `role.go` related to guest role creation.
- Modify `ssologin.go` and `ldap_login.go` to set user roles to nil instead of using integer roles.
- Introduce `convert_roles.go` to handle conversion of legacy roles and ensure role existence in the database.
* feat(role_perm): implement support for multiple base paths for roles
- Modify role permission checks to support multiple base paths
- Update role creation and update functions to handle multiple base paths
- Add migration script to convert old base_path to base_paths
- Define new Paths type for handling multiple paths in the model
- Adjust role model to replace BasePath with BasePaths
- Update existing patches to handle roles with multiple base paths
- Update bootstrap data to reflect the new base_paths field
* feat(role): Restrict modifications to default roles (admin and guest)
- Add validation to prevent changes to "admin" and "guest" roles in `UpdateRole` and `DeleteRole` functions.
- Introduce `ErrChangeDefaultRole` error in `internal/errs/role.go` to standardize error messaging.
- Update role-related API handlers in `server/handles/role.go` to enforce the new restriction.
- Enhance comments in `internal/bootstrap/data/role.go` to clarify the significance of default roles.
- Ensure consistent error responses for unauthorized role modifications across the application.
* 🔄 **refactor(role): Enhance role permission handling**
- Replaced `BasePaths` with `PermissionPaths` in `Role` struct for better permission granularity.
- Introduced JSON serialization for `PermissionPaths` using `RawPermission` field in `Role` struct.
- Implemented `BeforeSave` and `AfterFind` GORM hooks for handling `PermissionPaths` serialization.
- Refactored permission calculation logic in `role_perm.go` to work with `PermissionPaths`.
- Updated role creation logic to initialize `PermissionPaths` for `admin` and `guest` roles.
- Removed deprecated `CheckPathLimit` method from `Role` struct.
* fix(model/user/role): update permission settings for admin and role
- Change `RawPermission` field in `role.go` to hide JSON representation
- Update `Permission` field in `user.go` to `0xFFFF` for full access
- Modify `PermissionScopes` in `role.go` to `0xFFFF` for enhanced permissions
* 🔒 feat(role-permissions): Enhance role-based access control
- Introduce `canReadPathByRole` function in `role_perm.go` to verify path access based on user roles
- Modify `CanAccessWithRoles` to include role-based path read check
- Add `RoleNames` and `Permissions` to `UserResp` struct in `auth.go` for enhanced user role and permission details
- Implement role details aggregation in `auth.go` to populate `RoleNames` and `Permissions`
- Update `User` struct in `user.go` to include `RolesDetail` for more detailed role information
- Enhance middleware in `auth.go` to load and verify detailed role information for users
- Move `guest` user initialization logic in `user.go` to improve code organization and avoid repetition
* 🔒 fix(permissions): Add permission checks for archive operations
- Add `MergeRolePermissions` and `HasPermission` checks to validate user access for reading archives
- Ensure users have `PermReadArchives` before proceeding with `GetNearestMeta` in specific archive paths
- Implement permission checks for decompress operations, requiring `PermDecompress` for source paths
- Return `PermissionDenied` errors with 403 status if user lacks necessary permissions
* 🔒 fix(server): Add permission check for offline download
- Add permission merging logic for user roles
- Check user has permission for offline download addition
- Return error response with "permission denied" if check fails
* ✨ feat(role-permission): Implement path-based role permission checks
- Add `CheckPathLimitWithRoles` function to validate access based on `PermPathLimit` permission.
- Integrate `CheckPathLimitWithRoles` in `offline_download` to enforce path-based access control.
- Apply `CheckPathLimitWithRoles` across file system management operations (e.g., creation, movement, deletion).
- Ensure `CheckPathLimitWithRoles` is invoked for batch operations and archive-related actions.
- Update error handling to return `PermissionDenied` if the path validation fails.
- Import `errs` package in `offline_download` for consistent error responses.
* ✨ feat(role-permission): Implement path-based role permission checks
- Add `CheckPathLimitWithRoles` function to validate access based on `PermPathLimit` permission.
- Integrate `CheckPathLimitWithRoles` in `offline_download` to enforce path-based access control.
- Apply `CheckPathLimitWithRoles` across file system management operations (e.g., creation, movement, deletion).
- Ensure `CheckPathLimitWithRoles` is invoked for batch operations and archive-related actions.
- Update error handling to return `PermissionDenied` if the path validation fails.
- Import `errs` package in `offline_download` for consistent error responses.
* ♻️ refactor(access-control): Update access control logic to use role-based checks
- Remove deprecated logic from `CanAccess` function in `check.go`, replacing it with `CanAccessWithRoles` for improved role-based access control.
- Modify calls in `search.go` to use `CanAccessWithRoles` for more precise handling of permissions.
- Update `fsread.go` to utilize `CanAccessWithRoles`, ensuring accurate access validation based on user roles.
- Simplify import statements in `check.go` by removing unused packages to clean up the codebase.
* ✨ feat(fs): Improve visibility logic for hidden files
- Import `server/common` package to handle permissions more robustly
- Update `whetherHide` function to use `MergeRolePermissions` for user-specific path permissions
- Replace direct user checks with `HasPermission` for `PermSeeHides`
- Enhance logic to ensure `nil` user cases are handled explicitly
* 标签管理
* feat(db/auth/user): Enhance role handling and clean permission paths
- Comment out role modification checks in `server/handles/user.go` to allow flexible role changes.
- Improve permission path handling in `server/handles/auth.go` by normalizing and deduplicating paths.
- Introduce `addedPaths` map in `CurrentUser` to prevent duplicate permissions.
* feat(storage/db): Implement role permissions path prefix update
- Add `UpdateRolePermissionsPathPrefix` function in `role.go` to update role permissions paths.
- Modify `storage.go` to call the new function when the mount path is renamed.
- Introduce path cleaning and prefix matching logic for accurate path updates.
- Ensure roles are updated only if their permission scopes are modified.
- Handle potential errors with informative messages during database operations.
* feat(role-migration): Implement role conversion and introduce NEWGENERAL role
- Add `NEWGENERAL` to the roles enumeration in `user.go`
- Create new file `convert_role.go` for migrating legacy roles to new model
- Implement `ConvertLegacyRoles` function to handle role conversion with permission scopes
- Add `convert_role.go` patch to `all.go` under version `v3.46.0`
* feat(role/auth): Add role retrieval by user ID and update path prefixes
- Add `GetRolesByUserID` function for efficient role retrieval by user ID
- Implement `UpdateUserBasePathPrefix` to update user base paths
- Modify `UpdateRolePermissionsPathPrefix` to return modified role IDs
- Update `auth.go` middleware to use the new role retrieval function
- Refresh role and user caches upon path prefix updates to maintain consistency
---------
Co-authored-by: Leslie-Xy <540049476@qq.com>
Mega supports duplicate names but alist does not support.
In `List()` method, driver will return multiple files with same name.
That makes alist to use oldest version file for listing/downloading.
So it is necessary to filter old same name files in a folder.
After fixes, all CRUD work normally.
Refs #8344
* feat(local): support percent for video thumbnail
The percentage determines the point in the video (as a percentage of the total duration) at which the thumbnail will be generated.
* feat(local): support both time and percent for video thumbnail
* refactor(local): avoid duplicate parsing of VideoThumbPos
* feat(lanzou): add RemoveJSComment function to clean JavaScript comments from HTML
* feat(lanzou): remove comments from share page data in getFilesByShareUrl function
* fix(lanzou): optimize RemoveJSComment function to improve comment removal logic
* fix(aliyundrive_open): resolve file duplication issues and improve path handling
1. Fix file duplication by implementing a new removeDuplicateFiles method that cleans up duplicate files after operations
2. Change Move operation to use "ignore" for check_name_mode instead of "refuse" to allow moves when destination has same filename
3. Set Copy operation to handle duplicates by removing them after successful copy
4. Improve path handling for all file operations (Move, Rename, Put, MakeDir) by properly maintaining the full path of objects
5. Implement GetRoot interface for proper root object initialization with correct path
6. Add proper path management in List operation to ensure objects have correct paths
7. Fix path handling in error cases and improve logging of failures
* refactor(aliyundrive_open): change error logging to warnings for duplicate file removal
Updated the Move, Rename, and Copy methods to log warnings instead of errors when duplicate file removal fails, as the primary operations have already completed successfully. This improves the clarity of logs without affecting the functionality.
* Update drivers/aliyundrive_open/util.go
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
---------
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
