MinIO

Commit Graph

Author	SHA1	Message	Date
Harshavardhana	8c4561b8da	add all missing go.mod for debugging tools (#18049 )	2 years ago
Harshavardhana	fd421ddd6f	skip tiered objects to GLACIER in batch replication (#18044 ) tiered objects to GLACIER are not readable until they are restored, we skip these as unreadable	2 years ago
jiuker	9947c01c8e	feat: SSE-KMS use uuid instead of read all data to md5. (#17958 )	2 years ago
Eng Zer Jun	a00db4267c	data-usage-cache: remove redundant nil check (#17970 ) From the Go specification: "3. If the map is nil, the number of iterations is 0." [1] Therefore, an additional nil check for before the loop is unnecessary. [1]: https://go.dev/ref/spec#For_range Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>	2 years ago
Harshavardhana	36385010f5	use optimized pathJoin instead of path.Join (#18042 ) this avoids allocations in scanner routine, they are tiny but they allocate a lot over many cycles of the scanner.	2 years ago
Harshavardhana	fa6d082bfd	reduce all major allocations in replication path (#18032 ) - remove targetClient for passing around via replicationObjectInfo{} - remove cloing to object info unnecessarily - remove objectInfo from replicationObjectInfo{} (only require necessary fields)	2 years ago
Minio Trusted	9fab91852a	Update yaml files to latest version RELEASE.2023-09-16T01-01-47Z	2 years ago
Poorna	b733e6e83c	site replication turn off retry login for admin API calls (#18039 ) additionally also mark site offline if n/w is down	2 years ago
Harshavardhana	ce05bb69dc	update console v0.39.0 (#18038 ) Signed-off-by: Harshavardhana <harsha@minio.io>	2 years ago
Anis Eleuch	37aa5934a1	scanner: Fix loading data usage cache structure (#18037 ) Return an empty data usage cache structure when the data usage cache file does not exist, otherwise, the scanner won't work.	2 years ago
Harshavardhana	1647fc7edc	fix: optimize listMultipartUploads to serve via local disks (#18034 ) and remove unused getLoadBalancedDisks()	2 years ago
Harshavardhana	7b92687397	remove generating presignedURLs with range header for lambda (#18033 )	2 years ago
Anis Eleuch	419e5baf16	fix: webhook notify endpoint with standard ports (#18016 )	2 years ago
Alex	dc48cd841a	Added MINIO_PROMETHEUS_AUTH_TOKEN env support (#18028 ) Signed-off-by: Benjamin Perez <benjamin@bexsoft.net>	2 years ago
Anis Eleuch	b0e1776d6d	Do not use a chain for S3 tiering to return better error messages (#18030 ) When using a chain provider all providers do not return a valid access and secret key, an anonymous request is sent, which makes it hard for users to figure out what is going on In the case of S3 tiering, when AWS IAM temporary account generation returns an error, an anonymous login will be used because of the chain provider. Avoid this and use the AWS IAM provider directly to get a good error message.	2 years ago
Aditya Manthramurthy	7a7068ee47	Move IAM periodic ops to a single go routine (#18026 ) This helps reduce disk operations as these periodic routines would not run concurrently any more. Also add expired STS purging periodic operation: Since we do not scan the on-disk STS credentials (and instead only load them on-demand) a separate routine is needed to purge expired credentials from storage. Currently this runs about a quarter as often as IAM refresh. Also fix a bug where with etcd, STS accounts could get loaded into the iamUsersMap instead of the iamSTSAccountsMap.	2 years ago
Aditya Manthramurthy	cbc0ef459b	Fix policy package import name (#18031 ) We do not need to rename the import of minio/pkg/v2/policy as iampolicy any more.	2 years ago
Harshavardhana	a2aabfabd9	add backups for usage-caches to rely on upon error (#18029 ) This allows scanner to avoid lengthy scans, skip things appropriately and also not lose metrics in any manner. reduce longer deadlines for usage-cache loads/saves to match the disk timeout which is 2minutes now per IOP.	2 years ago
Harshavardhana	822cbd4b43	add couple of missing things from #18027	2 years ago
Ravind Kumar	3c19a9308d	DOCS-987: Reorganizing list.md for better RST compatibility (#18027 )	2 years ago
Harshavardhana	32890342ce	introduce MINIO_BROWSER_REDIRECT env to enable/disable auto-redirect (#18025 )	2 years ago
Aditya Manthramurthy	ed2c2a285f	Load STS accounts into IAM cache lazily (#17994 ) In situations with large number of STS credentials on disk, IAM load time is high. To mitigate this, STS accounts will now be loaded into memory only on demand - i.e. when the credential is used. In each IAM cache (re)load we skip loading STS credentials and STS policy mappings into memory. Since STS accounts only expire and cannot be deleted, there is no risk of invalid credentials being reused, because credential validity is checked when it is used.	2 years ago
Poorna	18e23bafd9	replication resync: report only the on-disk status (#18017 ) Avoid reporting in-memory status since results can vary if different nodes are queried, resync always runs at a single node.	2 years ago
Harshavardhana	8b8be2695f	optimize mkdir calls to avoid base-dir `Mkdir` attempts (#18021 ) Currently we have IOPs of these patterns ``` [OS] os.Mkdir play.min.io:9000 /disk1 2.718µs [OS] os.Mkdir play.min.io:9000 /disk1/data 2.406µs [OS] os.Mkdir play.min.io:9000 /disk1/data/.minio.sys 4.068µs [OS] os.Mkdir play.min.io:9000 /disk1/data/.minio.sys/tmp 2.843µs [OS] os.Mkdir play.min.io:9000 /disk1/data/.minio.sys/tmp/d89c8ceb-f8d1-4cc6-b483-280f87c4719f 20.152µs ``` It can be seen that we can save quite Nx levels such as if your drive is mounted at `/disk1/minio` you can simply skip sending an `Mkdir /disk1/` and `Mkdir /disk1/minio`. Since they are expected to exist already, this PR adds a way for us to ignore all paths upto the mount or a directory which ever has been provided to MinIO setup.	2 years ago
Poorna	96fbf18201	replication: queue existing objects to same workers as incoming (#18020 ) Previously existing objects were queued to single worker and MRF re-queues are also handled by same worker - this does not fully use the available bandwidth in case there is no incoming workload.	2 years ago
Harshavardhana	c8a57a8fa2	fix: send content-md5 for AWS S3 proactively (#18018 ) fixes #17977	2 years ago
Harshavardhana	b1c2dacab3	fix: allow dynamic ports for API only in non-distributed setups (#18019 ) fixes #17998	2 years ago
Harshavardhana	65939913b4	update all dependencies (#18012 )	2 years ago
Harshavardhana	08b3a466e8	fix: allow concurrent SFTP connections (#18013 ) current implementation did not fully implement the concurrent SFTP connection implementation, this PR properly handles this. fixes #17914	2 years ago
Harshavardhana	5aa7c38035	update pkg to v2.0.1 to extend admin actions (#18008 )	2 years ago
Harshavardhana	1df5e31706	optimize MRF replication queue to avoid memory leaks (#18007 )	2 years ago
Harshavardhana	9f7044aed0	fix: ignore transient errors in read path (#18006 ) Errors such as ``` returned an error (context deadline exceeded) (fmt.wrapError) ``` ``` (msgp: too few bytes left to read object) (fmt.wrapError) ```	2 years ago
Anis Eleuch	41de53996b	heal: calculate the number of workers based on NRRequests (#17945 )	2 years ago
Harshavardhana	9878031cfd	fix: change DISK_ to DRIVE_ for some drive related envs (#18005 )	2 years ago
Harshavardhana	e3fbcaeb72	allow scanner key cycle to be empty (#18001 ) configs from 2020 server throws an error due to deprecation of the keys however an attempt is made to parse them, we should have chosen existing defaults - this PR fixes that.	2 years ago
Harshavardhana	ca6dd8be5e	use go1.21.1 for vulncheck	2 years ago
Minio Trusted	fba0924b1d	Update yaml files to latest version RELEASE.2023-09-07T02-05-02Z	2 years ago
Poorna	703ed46d79	fix: replication of tags while removing (#17989 ) A tag removal was not being replicated prior to this change	2 years ago
Harshavardhana	f7ca6c63c2	fix: bucket quota clear and honor existing quota config (#17988 )	2 years ago
Harshavardhana	ad69b9907f	fix: report bucket metrics for only existing buckets (#17987 )	2 years ago
Anis Eleuch	b9269151a4	fix: drive rotational calculation status for partitions (#17986 ) Fix drive rotational calculation status If a MinIO drive path is mounted to a partition and not a real disk, getting the rotational status would fail because Linux does not expose that status to partition; In other words, /sys/block/drive-partition-name/queue/rotational does not exist; To fix the issue, the code will search for the rotational status of the disk that hosts the partition, and this can be calculated from the real path of /sys/class/block/<drive-partition-name>	2 years ago
Shubhendu	bfddbb8b40	Embed file in ZIP with custom permissions (#17954 ) This change enables embedding files in ZIP with custom permissions. Also uses default creds for starting MinIO based on inspect data. Signed-off-by: Shubhendu Ram Tripathi <shubhendu@minio.io>	2 years ago
Poorna	13a2dc8485	replication resync: avoid blocking on results channel. (#17981 ) continues fix in #17775	2 years ago
Harshavardhana	1e51424e8a	use syscall.Rename() directly instead of os.Rename() (#17982 )	2 years ago
Harshavardhana	5b114b43f7	refactor bandwidth throttling for replication target (#17980 ) This refactor is to allow using the bandwidth throttling for other purposes.	2 years ago
Poorna	812f5a02d7	metrics: fix panic in replication stats reporting (#17979 )	2 years ago
Minio Trusted	19f70dbfbf	Update yaml files to latest version RELEASE.2023-09-04T19-57-37Z	2 years ago
Aditya Manthramurthy	1c99fb106c	Update to minio/pkg/v2 (#17967 )	2 years ago
Krishnan Parthasarathi	71c32e9b48	Return successorModTime in quorum when available (#17925 )	2 years ago
Harshavardhana	380a59520b	add missing testdata for benchmarking	2 years ago

1 2 3 4 5 ...

11072 Commits (8c4561b8da0b0cd93629b23c92afe8f67ef031c9) All Branches Search

11072 Commits (8c4561b8da0b0cd93629b23c92afe8f67ef031c9)

All Branches