 perf: websocket grid connectivity for all internode communication (#18461)
This PR adds a WebSocket grid feature that allows servers to communicate via
a single two-way connection.
There are two request types:
* Single requests, which are `[]byte => ([]byte, error)`. This is for efficient small
roundtrips with small payloads.
* Streaming requests which are `[]byte, chan []byte => chan []byte (and error)`,
which allows for different combinations of full two-way streams with an initial payload.
Only a single stream is created between two machines - and there is, as such, no
server/client relation since both sides can initiate and handle requests. Which server
initiates the request is decided deterministically on the server names.
Requests are made through a mux client and server, which handles message
passing, congestion, cancelation, timeouts, etc.
If a connection is lost, all requests are canceled, and the calling server will try
to reconnect. Registered handlers can operate directly on byte
slices or use a higher-level generics abstraction.
There is no versioning of handlers/clients, and incompatible changes should
be handled by adding new handlers.
The request path can be changed to a new one for any protocol changes.
First, all servers create a "Manager." The manager must know its address
as well as all remote addresses. This will manage all connections.
To get a connection to any remote, ask the manager to provide it given
the remote address using.
```
func (m *Manager) Connection(host string) *Connection
```
All serverside handlers must also be registered on the manager. This will
make sure that all incoming requests are served. The number of in-flight
requests and responses must also be given for streaming requests.
The "Connection" returned manages the mux-clients. Requests issued
to the connection will be sent to the remote.
* `func (c *Connection) Request(ctx context.Context, h HandlerID, req []byte) ([]byte, error)`
performs a single request and returns the result. Any deadline provided on the request is
forwarded to the server, and canceling the context will make the function return at once.
* `func (c *Connection) NewStream(ctx context.Context, h HandlerID, payload []byte) (st *Stream, err error)`
will initiate a remote call and send the initial payload.
```Go
// A Stream is a two-way stream.
// All responses *must* be read by the caller.
// If the call is canceled through the context,
//The appropriate error will be returned.
type Stream struct {
// Responses from the remote server.
// Channel will be closed after an error or when the remote closes.
// All responses *must* be read by the caller until either an error is returned or the channel is closed.
// Canceling the context will cause the context cancellation error to be returned.
Responses <-chan Response
// Requests sent to the server.
// If the handler is defined with 0 incoming capacity this will be nil.
// Channel *must* be closed to signal the end of the stream.
// If the request context is canceled, the stream will no longer process requests.
Requests chan<- []byte
}
type Response struct {
Msg []byte
Err error
}
```
There are generic versions of the server/client handlers that allow the use of type
safe implementations for data types that support msgpack marshal/unmarshal.
2 years ago |
|
module github.com/minio/minio
go 1.23.0
toolchain go1.23.6
require ( cloud.google.com/go/storage v1.46.0 github.com/Azure/azure-sdk-for-go/sdk/azcore v1.16.0 github.com/Azure/azure-sdk-for-go/sdk/azidentity v1.8.0 github.com/Azure/azure-sdk-for-go/sdk/storage/azblob v1.5.0 github.com/IBM/sarama v1.43.3 github.com/alecthomas/participle v0.7.1 github.com/beevik/ntp v1.4.3 github.com/buger/jsonparser v1.1.1 github.com/cespare/xxhash/v2 v2.3.0 github.com/cheggaaa/pb v1.0.29 github.com/coreos/go-oidc/v3 v3.11.0 github.com/coreos/go-systemd/v22 v22.5.0 github.com/cosnicolaou/pbzip2 v1.0.5 github.com/dchest/siphash v1.2.3 github.com/dustin/go-humanize v1.0.1 github.com/eclipse/paho.mqtt.golang v1.5.0 github.com/elastic/go-elasticsearch/v7 v7.17.10 github.com/fatih/color v1.18.0 github.com/felixge/fgprof v0.9.5 github.com/fraugster/parquet-go v0.12.0 github.com/go-ldap/ldap/v3 v3.4.8 github.com/go-openapi/loads v0.22.0 github.com/go-sql-driver/mysql v1.8.1 github.com/gobwas/ws v1.4.0 github.com/golang-jwt/jwt/v4 v4.5.1 github.com/gomodule/redigo v1.9.2 github.com/google/uuid v1.6.0 github.com/inconshreveable/mousetrap v1.1.0 github.com/json-iterator/go v1.1.12 github.com/klauspost/compress v1.17.11 github.com/klauspost/cpuid/v2 v2.2.9 github.com/klauspost/filepathx v1.1.1 github.com/klauspost/pgzip v1.2.6 github.com/klauspost/readahead v1.4.0 github.com/klauspost/reedsolomon v1.12.4 github.com/lib/pq v1.10.9 github.com/lithammer/shortuuid/v4 v4.0.0 github.com/miekg/dns v1.1.62 github.com/minio/cli v1.24.2 github.com/minio/console v1.7.6 github.com/minio/csvparser v1.0.0 github.com/minio/dnscache v0.1.1 github.com/minio/dperf v0.6.3 github.com/minio/highwayhash v1.0.3 github.com/minio/kms-go/kes v0.3.1 github.com/minio/kms-go/kms v0.4.0 github.com/minio/madmin-go/v3 v3.0.94 github.com/minio/minio-go/v7 v7.0.85 github.com/minio/mux v1.9.0 github.com/minio/pkg/v3 v3.0.29 github.com/minio/selfupdate v0.6.0 github.com/minio/simdjson-go v0.4.5 github.com/minio/sio v0.4.1 github.com/minio/xxml v0.0.3 github.com/minio/zipindex v0.4.0 github.com/mitchellh/go-homedir v1.1.0 github.com/nats-io/nats-server/v2 v2.9.23 github.com/nats-io/nats.go v1.37.0 github.com/nats-io/stan.go v0.10.4 github.com/ncw/directio v1.0.5 github.com/nsqio/go-nsq v1.1.0 github.com/philhofer/fwd v1.1.3-0.20240916144458-20a13a1f6b7c github.com/pierrec/lz4/v4 v4.1.21 github.com/pkg/errors v0.9.1 github.com/pkg/sftp v1.13.7 github.com/pkg/xattr v0.4.10 github.com/prometheus/client_golang v1.20.5 github.com/prometheus/client_model v0.6.1 github.com/prometheus/common v0.62.0 github.com/prometheus/procfs v0.15.1 github.com/puzpuzpuz/xsync/v3 v3.4.0 github.com/rabbitmq/amqp091-go v1.10.0 github.com/rcrowley/go-metrics v0.0.0-20201227073835-cf1acfcdf475 github.com/rs/cors v1.11.1 github.com/secure-io/sio-go v0.3.1 github.com/shirou/gopsutil/v3 v3.24.5 github.com/tinylib/msgp v1.2.5 github.com/valyala/bytebufferpool v1.0.0 github.com/xdg/scram v1.0.5 github.com/zeebo/xxh3 v1.0.2 go.etcd.io/etcd/api/v3 v3.5.18 go.etcd.io/etcd/client/v3 v3.5.18 go.uber.org/atomic v1.11.0 go.uber.org/zap v1.27.0 goftp.io/server/v2 v2.0.1 golang.org/x/crypto v0.35.0 golang.org/x/oauth2 v0.26.0 golang.org/x/sync v0.11.0 golang.org/x/sys v0.30.0 golang.org/x/term v0.29.0 golang.org/x/time v0.8.0 google.golang.org/api v0.213.0 gopkg.in/yaml.v2 v2.4.0 gopkg.in/yaml.v3 v3.0.1)
require ( aead.dev/mem v0.2.0 // indirect aead.dev/minisign v0.3.0 // indirect cel.dev/expr v0.19.0 // indirect cloud.google.com/go v0.116.0 // indirect cloud.google.com/go/auth v0.13.0 // indirect cloud.google.com/go/auth/oauth2adapt v0.2.6 // indirect cloud.google.com/go/compute/metadata v0.6.0 // indirect cloud.google.com/go/iam v1.2.2 // indirect cloud.google.com/go/monitoring v1.21.2 // indirect filippo.io/edwards25519 v1.1.0 // indirect github.com/Azure/azure-sdk-for-go/sdk/internal v1.10.0 // indirect github.com/Azure/go-ntlmssp v0.0.0-20221128193559-754e69321358 // indirect github.com/AzureAD/microsoft-authentication-library-for-go v1.3.1 // indirect github.com/GoogleCloudPlatform/opentelemetry-operations-go/detectors/gcp v1.25.0 // indirect github.com/GoogleCloudPlatform/opentelemetry-operations-go/exporter/metric v0.49.0 // indirect github.com/GoogleCloudPlatform/opentelemetry-operations-go/internal/resourcemapping v0.49.0 // indirect github.com/VividCortex/ewma v1.2.0 // indirect github.com/acarl005/stripansi v0.0.0-20180116102854-5a71ef0e047d // indirect github.com/apache/thrift v0.21.0 // indirect github.com/asaskevich/govalidator v0.0.0-20230301143203-a9d515a09cc2 // indirect github.com/aymanbagabas/go-osc52/v2 v2.0.1 // indirect github.com/beorn7/perks v1.0.1 // indirect github.com/census-instrumentation/opencensus-proto v0.4.1 // indirect github.com/charmbracelet/bubbles v0.20.0 // indirect github.com/charmbracelet/bubbletea v1.3.0 // indirect github.com/charmbracelet/lipgloss v1.0.0 // indirect github.com/charmbracelet/x/ansi v0.8.0 // indirect github.com/charmbracelet/x/term v0.2.1 // indirect github.com/cncf/xds/go v0.0.0-20240905190251-b4127c9b8d78 // indirect github.com/coreos/go-semver v0.3.1 // indirect github.com/davecgh/go-spew v1.1.2-0.20180830191138-d8f796af33cc // indirect github.com/decred/dcrd/dcrec/secp256k1/v4 v4.3.0 // indirect github.com/docker/go-units v0.5.0 // indirect github.com/eapache/go-resiliency v1.7.0 // indirect github.com/eapache/go-xerial-snappy v0.0.0-20230731223053-c322873962e3 // indirect github.com/eapache/queue v1.1.0 // indirect github.com/envoyproxy/go-control-plane v0.13.1 // indirect github.com/envoyproxy/protoc-gen-validate v1.1.0 // indirect github.com/erikgeiser/coninput v0.0.0-20211004153227-1c3628e74d0f // indirect github.com/fatih/structs v1.1.0 // indirect github.com/felixge/httpsnoop v1.0.4 // indirect github.com/go-asn1-ber/asn1-ber v1.5.7 // indirect github.com/go-ini/ini v1.67.0 // indirect github.com/go-jose/go-jose/v4 v4.0.5 // indirect github.com/go-logr/logr v1.4.2 // indirect github.com/go-logr/stdr v1.2.2 // indirect github.com/go-ole/go-ole v1.3.0 // indirect github.com/go-openapi/analysis v0.23.0 // indirect github.com/go-openapi/errors v0.22.0 // indirect github.com/go-openapi/jsonpointer v0.21.0 // indirect github.com/go-openapi/jsonreference v0.21.0 // indirect github.com/go-openapi/runtime v0.28.0 // indirect github.com/go-openapi/spec v0.21.0 // indirect github.com/go-openapi/strfmt v0.23.0 // indirect github.com/go-openapi/swag v0.23.0 // indirect github.com/go-openapi/validate v0.24.0 // indirect github.com/gobwas/httphead v0.1.0 // indirect github.com/gobwas/pool v0.2.1 // indirect github.com/goccy/go-json v0.10.5 // indirect github.com/gogo/protobuf v1.3.2 // indirect github.com/golang-jwt/jwt/v5 v5.2.1 // indirect github.com/golang/groupcache v0.0.0-20210331224755-41bb18bfe9da // indirect github.com/golang/protobuf v1.5.4 // indirect github.com/golang/snappy v0.0.4 // indirect github.com/google/pprof v0.0.0-20241210010833-40e02aabc2ad // indirect github.com/google/s2a-go v0.1.8 // indirect github.com/google/shlex v0.0.0-20191202100458-e7afc7fbc510 // indirect github.com/googleapis/enterprise-certificate-proxy v0.3.4 // indirect github.com/googleapis/gax-go/v2 v2.14.0 // indirect github.com/gorilla/websocket v1.5.3 // indirect github.com/hashicorp/errwrap v1.1.0 // indirect github.com/hashicorp/go-multierror v1.1.1 // indirect github.com/hashicorp/go-uuid v1.0.3 // indirect github.com/hashicorp/golang-lru v1.0.2 // indirect github.com/jcmturner/aescts/v2 v2.0.0 // indirect github.com/jcmturner/dnsutils/v2 v2.0.0 // indirect github.com/jcmturner/gofork v1.7.6 // indirect github.com/jcmturner/gokrb5/v8 v8.4.4 // indirect github.com/jcmturner/rpc/v2 v2.0.3 // indirect github.com/jedib0t/go-pretty/v6 v6.6.5 // indirect github.com/jessevdk/go-flags v1.6.1 // indirect github.com/josharian/intern v1.0.0 // indirect github.com/juju/ratelimit v1.0.2 // indirect github.com/kr/fs v0.1.0 // indirect github.com/kylelemons/godebug v1.1.0 // indirect github.com/lestrrat-go/blackmagic v1.0.2 // indirect github.com/lestrrat-go/httpcc v1.0.1 // indirect github.com/lestrrat-go/httprc v1.0.6 // indirect github.com/lestrrat-go/iter v1.0.2 // indirect github.com/lestrrat-go/jwx/v2 v2.1.3 // indirect github.com/lestrrat-go/option v1.0.1 // indirect github.com/lucasb-eyer/go-colorful v1.2.0 // indirect github.com/lufia/plan9stats v0.0.0-20240909124753-873cd0166683 // indirect github.com/mailru/easyjson v0.9.0 // indirect github.com/mattn/go-colorable v0.1.14 // indirect github.com/mattn/go-ieproxy v0.0.12 // indirect github.com/mattn/go-isatty v0.0.20 // indirect github.com/mattn/go-localereader v0.0.1 // indirect github.com/mattn/go-runewidth v0.0.16 // indirect github.com/matttproud/golang_protobuf_extensions v1.0.4 // indirect github.com/minio/colorjson v1.0.8 // indirect github.com/minio/filepath v1.0.0 // indirect github.com/minio/mc v0.0.0-20250208210632-10c50368c526 // indirect github.com/minio/md5-simd v1.1.2 // indirect github.com/minio/websocket v1.6.0 // indirect github.com/mitchellh/mapstructure v1.5.0 // indirect github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd // indirect github.com/modern-go/reflect2 v1.0.2 // indirect github.com/montanaflynn/stats v0.7.1 // indirect github.com/muesli/ansi v0.0.0-20230316100256-276c6243b2f6 // indirect github.com/muesli/cancelreader v0.2.2 // indirect github.com/muesli/reflow v0.3.0 // indirect github.com/muesli/termenv v0.15.2 // indirect github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822 // indirect github.com/nats-io/jwt/v2 v2.5.0 // indirect github.com/nats-io/nats-streaming-server v0.24.6 // indirect github.com/nats-io/nkeys v0.4.7 // indirect github.com/nats-io/nuid v1.0.1 // indirect github.com/oklog/ulid v1.3.1 // indirect github.com/olekukonko/tablewriter v0.0.5 // indirect github.com/pkg/browser v0.0.0-20240102092130-5ac0b6a4141c // indirect github.com/planetscale/vtprotobuf v0.6.1-0.20240319094008-0393e58bdf10 // indirect github.com/posener/complete v1.2.3 // indirect github.com/power-devops/perfstat v0.0.0-20240221224432-82ca36839d55 // indirect github.com/prometheus/prom2json v1.4.1 // indirect github.com/prometheus/prometheus v0.301.0 // indirect github.com/rivo/uniseg v0.4.7 // indirect github.com/rjeczalik/notify v0.9.3 // indirect github.com/rs/xid v1.6.0 // indirect github.com/safchain/ethtool v0.5.10 // indirect github.com/segmentio/asm v1.2.0 // indirect github.com/shoenig/go-m1cpu v0.1.6 // indirect github.com/tidwall/gjson v1.18.0 // indirect github.com/tidwall/match v1.1.1 // indirect github.com/tidwall/pretty v1.2.1 // indirect github.com/tklauser/go-sysconf v0.3.14 // indirect github.com/tklauser/numcpus v0.9.0 // indirect github.com/unrolled/secure v1.17.0 // indirect github.com/vbauerster/mpb/v8 v8.9.2 // indirect github.com/xdg/stringprep v1.0.3 // indirect github.com/yusufpapurcu/wmi v1.2.4 // indirect go.etcd.io/etcd/client/pkg/v3 v3.5.18 // indirect go.mongodb.org/mongo-driver v1.17.2 // indirect go.opencensus.io v0.24.0 // indirect go.opentelemetry.io/auto/sdk v1.1.0 // indirect go.opentelemetry.io/contrib/detectors/gcp v1.32.0 // indirect go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc v0.57.0 // indirect go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.58.0 // indirect go.opentelemetry.io/otel v1.33.0 // indirect go.opentelemetry.io/otel/metric v1.33.0 // indirect go.opentelemetry.io/otel/sdk v1.33.0 // indirect go.opentelemetry.io/otel/sdk/metric v1.32.0 // indirect go.opentelemetry.io/otel/trace v1.33.0 // indirect go.uber.org/multierr v1.11.0 // indirect golang.org/x/mod v0.22.0 // indirect golang.org/x/net v0.35.0 // indirect golang.org/x/text v0.22.0 // indirect golang.org/x/tools v0.28.0 // indirect google.golang.org/genproto v0.0.0-20241113202542-65e8d215514f // indirect google.golang.org/genproto/googleapis/api v0.0.0-20250207221924-e9438ea467c6 // indirect google.golang.org/genproto/googleapis/rpc v0.0.0-20250207221924-e9438ea467c6 // indirect google.golang.org/grpc v1.70.0 // indirect google.golang.org/protobuf v1.36.5 // indirect)
|
