 perf: websocket grid connectivity for all internode communication (#18461)
This PR adds a WebSocket grid feature that allows servers to communicate via
a single two-way connection.
There are two request types:
* Single requests, which are `[]byte => ([]byte, error)`. This is for efficient small
roundtrips with small payloads.
* Streaming requests which are `[]byte, chan []byte => chan []byte (and error)`,
which allows for different combinations of full two-way streams with an initial payload.
Only a single stream is created between two machines - and there is, as such, no
server/client relation since both sides can initiate and handle requests. Which server
initiates the request is decided deterministically on the server names.
Requests are made through a mux client and server, which handles message
passing, congestion, cancelation, timeouts, etc.
If a connection is lost, all requests are canceled, and the calling server will try
to reconnect. Registered handlers can operate directly on byte
slices or use a higher-level generics abstraction.
There is no versioning of handlers/clients, and incompatible changes should
be handled by adding new handlers.
The request path can be changed to a new one for any protocol changes.
First, all servers create a "Manager." The manager must know its address
as well as all remote addresses. This will manage all connections.
To get a connection to any remote, ask the manager to provide it given
the remote address using.
```
func (m *Manager) Connection(host string) *Connection
```
All serverside handlers must also be registered on the manager. This will
make sure that all incoming requests are served. The number of in-flight
requests and responses must also be given for streaming requests.
The "Connection" returned manages the mux-clients. Requests issued
to the connection will be sent to the remote.
* `func (c *Connection) Request(ctx context.Context, h HandlerID, req []byte) ([]byte, error)`
performs a single request and returns the result. Any deadline provided on the request is
forwarded to the server, and canceling the context will make the function return at once.
* `func (c *Connection) NewStream(ctx context.Context, h HandlerID, payload []byte) (st *Stream, err error)`
will initiate a remote call and send the initial payload.
```Go
// A Stream is a two-way stream.
// All responses *must* be read by the caller.
// If the call is canceled through the context,
//The appropriate error will be returned.
type Stream struct {
// Responses from the remote server.
// Channel will be closed after an error or when the remote closes.
// All responses *must* be read by the caller until either an error is returned or the channel is closed.
// Canceling the context will cause the context cancellation error to be returned.
Responses <-chan Response
// Requests sent to the server.
// If the handler is defined with 0 incoming capacity this will be nil.
// Channel *must* be closed to signal the end of the stream.
// If the request context is canceled, the stream will no longer process requests.
Requests chan<- []byte
}
type Response struct {
Msg []byte
Err error
}
```
There are generic versions of the server/client handlers that allow the use of type
safe implementations for data types that support msgpack marshal/unmarshal.
2 years ago |
|
module github.com/minio/minio
go 1.21
require ( cloud.google.com/go/storage v1.41.0 github.com/Azure/azure-storage-blob-go v0.15.0 github.com/Azure/go-autorest/autorest v0.11.29 github.com/Azure/go-autorest/autorest/adal v0.9.24 github.com/IBM/sarama v1.43.2 github.com/alecthomas/participle v0.7.1 github.com/bcicen/jstream v1.0.1 github.com/beevik/ntp v1.4.3 github.com/buger/jsonparser v1.1.1 github.com/cespare/xxhash/v2 v2.3.0 github.com/cheggaaa/pb v1.0.29 github.com/coredns/coredns v1.11.3 github.com/coreos/go-oidc/v3 v3.10.0 github.com/coreos/go-systemd/v22 v22.5.0 github.com/cosnicolaou/pbzip2 v1.0.3 github.com/dchest/siphash v1.2.3 github.com/dustin/go-humanize v1.0.1 github.com/eclipse/paho.mqtt.golang v1.4.3 github.com/elastic/go-elasticsearch/v7 v7.17.10 github.com/fatih/color v1.17.0 github.com/felixge/fgprof v0.9.4 github.com/fraugster/parquet-go v0.12.0 github.com/go-ldap/ldap/v3 v3.4.8 github.com/go-openapi/loads v0.22.0 github.com/go-sql-driver/mysql v1.8.1 github.com/gobwas/ws v1.4.0 github.com/golang-jwt/jwt/v4 v4.5.0 github.com/gomodule/redigo v1.9.2 github.com/google/uuid v1.6.0 github.com/hashicorp/golang-lru/v2 v2.0.7 github.com/inconshreveable/mousetrap v1.1.0 github.com/json-iterator/go v1.1.12 github.com/klauspost/compress v1.17.8 github.com/klauspost/cpuid/v2 v2.2.7 github.com/klauspost/filepathx v1.1.1 github.com/klauspost/pgzip v1.2.6 github.com/klauspost/readahead v1.4.0 github.com/klauspost/reedsolomon v1.12.1 github.com/lib/pq v1.10.9 github.com/lithammer/shortuuid/v4 v4.0.0 github.com/miekg/dns v1.1.59 github.com/minio/cli v1.24.2 github.com/minio/console v1.5.0 github.com/minio/csvparser v1.0.0 github.com/minio/dnscache v0.1.1 github.com/minio/dperf v0.5.3 github.com/minio/highwayhash v1.0.2 github.com/minio/kms-go/kes v0.3.0 github.com/minio/kms-go/kms v0.4.0 github.com/minio/madmin-go/v3 v3.0.55-0.20240603092915-420a67132c32 github.com/minio/minio-go/v7 v7.0.72-0.20240610154810-fa174cbf14b0 github.com/minio/mux v1.9.0 github.com/minio/pkg/v3 v3.0.1 github.com/minio/selfupdate v0.6.0 github.com/minio/simdjson-go v0.4.5 github.com/minio/sio v0.4.0 github.com/minio/xxml v0.0.3 github.com/minio/zipindex v0.3.0 github.com/mitchellh/go-homedir v1.1.0 github.com/nats-io/nats-server/v2 v2.9.23 github.com/nats-io/nats.go v1.35.0 github.com/nats-io/stan.go v0.10.4 github.com/ncw/directio v1.0.5 github.com/nsqio/go-nsq v1.1.0 github.com/philhofer/fwd v1.1.2 github.com/pierrec/lz4 v2.6.1+incompatible github.com/pkg/errors v0.9.1 github.com/pkg/sftp v1.13.6 github.com/pkg/xattr v0.4.9 github.com/prometheus/client_golang v1.19.1 github.com/prometheus/client_model v0.6.1 github.com/prometheus/common v0.54.0 github.com/prometheus/procfs v0.15.1 github.com/puzpuzpuz/xsync/v3 v3.1.0 github.com/rabbitmq/amqp091-go v1.10.0 github.com/rcrowley/go-metrics v0.0.0-20201227073835-cf1acfcdf475 github.com/rs/cors v1.11.0 github.com/secure-io/sio-go v0.3.1 github.com/shirou/gopsutil/v3 v3.24.5 github.com/tidwall/gjson v1.17.1 github.com/tinylib/msgp v1.1.9 github.com/valyala/bytebufferpool v1.0.0 github.com/xdg/scram v1.0.5 github.com/zeebo/xxh3 v1.0.2 go.etcd.io/etcd/api/v3 v3.5.14 go.etcd.io/etcd/client/v3 v3.5.14 go.uber.org/atomic v1.11.0 go.uber.org/zap v1.27.0 goftp.io/server/v2 v2.0.1 golang.org/x/crypto v0.24.0 golang.org/x/exp v0.0.0-20240531132922-fd00a4e0eefc golang.org/x/oauth2 v0.21.0 golang.org/x/sys v0.21.0 golang.org/x/term v0.21.0 golang.org/x/time v0.5.0 google.golang.org/api v0.182.0 gopkg.in/yaml.v2 v2.4.0 gopkg.in/yaml.v3 v3.0.1)
require ( aead.dev/mem v0.2.0 // indirect aead.dev/minisign v0.3.0 // indirect cloud.google.com/go v0.114.0 // indirect cloud.google.com/go/auth v0.5.1 // indirect cloud.google.com/go/auth/oauth2adapt v0.2.2 // indirect cloud.google.com/go/compute/metadata v0.3.0 // indirect cloud.google.com/go/iam v1.1.8 // indirect filippo.io/edwards25519 v1.1.0 // indirect github.com/Azure/azure-pipeline-go v0.2.3 // indirect github.com/Azure/go-autorest v14.2.0+incompatible // indirect github.com/Azure/go-autorest/autorest/date v0.3.0 // indirect github.com/Azure/go-autorest/logger v0.2.1 // indirect github.com/Azure/go-autorest/tracing v0.6.0 // indirect github.com/Azure/go-ntlmssp v0.0.0-20221128193559-754e69321358 // indirect github.com/VividCortex/ewma v1.2.0 // indirect github.com/acarl005/stripansi v0.0.0-20180116102854-5a71ef0e047d // indirect github.com/apache/thrift v0.20.0 // indirect github.com/armon/go-metrics v0.4.0 // indirect github.com/asaskevich/govalidator v0.0.0-20230301143203-a9d515a09cc2 // indirect github.com/aymanbagabas/go-osc52/v2 v2.0.1 // indirect github.com/beorn7/perks v1.0.1 // indirect github.com/charmbracelet/bubbles v0.18.0 // indirect github.com/charmbracelet/bubbletea v0.26.4 // indirect github.com/charmbracelet/lipgloss v0.11.0 // indirect github.com/charmbracelet/x/ansi v0.1.2 // indirect github.com/charmbracelet/x/input v0.1.2 // indirect github.com/charmbracelet/x/term v0.1.1 // indirect github.com/charmbracelet/x/windows v0.1.2 // indirect github.com/coreos/go-semver v0.3.1 // indirect github.com/davecgh/go-spew v1.1.2-0.20180830191138-d8f796af33cc // indirect github.com/decred/dcrd/dcrec/secp256k1/v4 v4.3.0 // indirect github.com/docker/go-units v0.5.0 // indirect github.com/eapache/go-resiliency v1.6.0 // indirect github.com/eapache/go-xerial-snappy v0.0.0-20230731223053-c322873962e3 // indirect github.com/eapache/queue v1.1.0 // indirect github.com/erikgeiser/coninput v0.0.0-20211004153227-1c3628e74d0f // indirect github.com/fatih/structs v1.1.0 // indirect github.com/felixge/httpsnoop v1.0.4 // indirect github.com/frankban/quicktest v1.14.4 // indirect github.com/go-asn1-ber/asn1-ber v1.5.7 // indirect github.com/go-jose/go-jose/v4 v4.0.1 // indirect github.com/go-logr/logr v1.4.2 // indirect github.com/go-logr/stdr v1.2.2 // indirect github.com/go-ole/go-ole v1.3.0 // indirect github.com/go-openapi/analysis v0.23.0 // indirect github.com/go-openapi/errors v0.22.0 // indirect github.com/go-openapi/jsonpointer v0.21.0 // indirect github.com/go-openapi/jsonreference v0.21.0 // indirect github.com/go-openapi/runtime v0.28.0 // indirect github.com/go-openapi/spec v0.21.0 // indirect github.com/go-openapi/strfmt v0.23.0 // indirect github.com/go-openapi/swag v0.23.0 // indirect github.com/go-openapi/validate v0.24.0 // indirect github.com/gobwas/httphead v0.1.0 // indirect github.com/gobwas/pool v0.2.1 // indirect github.com/goccy/go-json v0.10.3 // indirect github.com/gogo/protobuf v1.3.2 // indirect github.com/golang/groupcache v0.0.0-20210331224755-41bb18bfe9da // indirect github.com/golang/protobuf v1.5.4 // indirect github.com/golang/snappy v0.0.4 // indirect github.com/google/pprof v0.0.0-20240528025155-186aa0362fba // indirect github.com/google/s2a-go v0.1.7 // indirect github.com/google/shlex v0.0.0-20191202100458-e7afc7fbc510 // indirect github.com/googleapis/enterprise-certificate-proxy v0.3.2 // indirect github.com/googleapis/gax-go/v2 v2.12.4 // indirect github.com/gorilla/websocket v1.5.1 // indirect github.com/hashicorp/errwrap v1.1.0 // indirect github.com/hashicorp/go-hclog v1.2.0 // indirect github.com/hashicorp/go-immutable-radix v1.3.1 // indirect github.com/hashicorp/go-multierror v1.1.1 // indirect github.com/hashicorp/go-uuid v1.0.3 // indirect github.com/hashicorp/golang-lru v1.0.2 // indirect github.com/jcmturner/aescts/v2 v2.0.0 // indirect github.com/jcmturner/dnsutils/v2 v2.0.0 // indirect github.com/jcmturner/gofork v1.7.6 // indirect github.com/jcmturner/gokrb5/v8 v8.4.4 // indirect github.com/jcmturner/rpc/v2 v2.0.3 // indirect github.com/jedib0t/go-pretty/v6 v6.5.9 // indirect github.com/jessevdk/go-flags v1.5.0 // indirect github.com/josharian/intern v1.0.0 // indirect github.com/juju/ratelimit v1.0.2 // indirect github.com/kr/fs v0.1.0 // indirect github.com/lestrrat-go/backoff/v2 v2.0.8 // indirect github.com/lestrrat-go/blackmagic v1.0.2 // indirect github.com/lestrrat-go/httpcc v1.0.1 // indirect github.com/lestrrat-go/iter v1.0.2 // indirect github.com/lestrrat-go/jwx v1.2.29 // indirect github.com/lestrrat-go/option v1.0.1 // indirect github.com/lucasb-eyer/go-colorful v1.2.0 // indirect github.com/lufia/plan9stats v0.0.0-20240513124658-fba389f38bae // indirect github.com/mailru/easyjson v0.7.7 // indirect github.com/mattn/go-colorable v0.1.13 // indirect github.com/mattn/go-ieproxy v0.0.12 // indirect github.com/mattn/go-isatty v0.0.20 // indirect github.com/mattn/go-localereader v0.0.1 // indirect github.com/mattn/go-runewidth v0.0.15 // indirect github.com/matttproud/golang_protobuf_extensions v1.0.4 // indirect github.com/minio/colorjson v1.0.8 // indirect github.com/minio/filepath v1.0.0 // indirect github.com/minio/mc v0.0.0-20240605181330-17adf0abbbca // indirect github.com/minio/md5-simd v1.1.2 // indirect github.com/minio/pkg/v2 v2.0.19 // indirect github.com/minio/websocket v1.6.0 // indirect github.com/mitchellh/mapstructure v1.5.0 // indirect github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd // indirect github.com/modern-go/reflect2 v1.0.2 // indirect github.com/montanaflynn/stats v0.7.1 // indirect github.com/muesli/ansi v0.0.0-20230316100256-276c6243b2f6 // indirect github.com/muesli/cancelreader v0.2.2 // indirect github.com/muesli/reflow v0.3.0 // indirect github.com/muesli/termenv v0.15.2 // indirect github.com/nats-io/jwt/v2 v2.5.0 // indirect github.com/nats-io/nats-streaming-server v0.24.3 // indirect github.com/nats-io/nkeys v0.4.7 // indirect github.com/nats-io/nuid v1.0.1 // indirect github.com/oklog/ulid v1.3.1 // indirect github.com/olekukonko/tablewriter v0.0.5 // indirect github.com/pierrec/lz4/v4 v4.1.21 // indirect github.com/posener/complete v1.2.3 // indirect github.com/power-devops/perfstat v0.0.0-20240221224432-82ca36839d55 // indirect github.com/prometheus/prom2json v1.3.3 // indirect github.com/rivo/uniseg v0.4.7 // indirect github.com/rjeczalik/notify v0.9.3 // indirect github.com/rs/xid v1.5.0 // indirect github.com/safchain/ethtool v0.3.0 // indirect github.com/shoenig/go-m1cpu v0.1.6 // indirect github.com/tidwall/match v1.1.1 // indirect github.com/tidwall/pretty v1.2.1 // indirect github.com/tklauser/go-sysconf v0.3.14 // indirect github.com/tklauser/numcpus v0.8.0 // indirect github.com/unrolled/secure v1.14.0 // indirect github.com/vbauerster/mpb/v8 v8.7.3 // indirect github.com/xdg/stringprep v1.0.3 // indirect github.com/xo/terminfo v0.0.0-20220910002029-abceb7e1c41e // indirect github.com/yusufpapurcu/wmi v1.2.4 // indirect go.etcd.io/etcd/client/pkg/v3 v3.5.14 // indirect go.mongodb.org/mongo-driver v1.15.0 // indirect go.opencensus.io v0.24.0 // indirect go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc v0.52.0 // indirect go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.52.0 // indirect go.opentelemetry.io/otel v1.27.0 // indirect go.opentelemetry.io/otel/metric v1.27.0 // indirect go.opentelemetry.io/otel/trace v1.27.0 // indirect go.uber.org/multierr v1.11.0 // indirect golang.org/x/mod v0.18.0 // indirect golang.org/x/net v0.26.0 // indirect golang.org/x/sync v0.7.0 // indirect golang.org/x/text v0.16.0 // indirect golang.org/x/tools v0.21.1-0.20240508182429-e35e4ccd0d2d // indirect google.golang.org/genproto v0.0.0-20240528184218-531527333157 // indirect google.golang.org/genproto/googleapis/api v0.0.0-20240604185151-ef581f913117 // indirect google.golang.org/genproto/googleapis/rpc v0.0.0-20240604185151-ef581f913117 // indirect google.golang.org/grpc v1.64.0 // indirect google.golang.org/protobuf v1.34.1 // indirect gopkg.in/ini.v1 v1.67.0 // indirect)
|
