Elastic containers¶

Elastic containers are containers that can be adapted to changing node clusters. To create an elastic container, you can use a policy that includes the REP * and SELECT * statements, or a combination of both.

Before we explain these statements, let's consider the following netmap:

Node 1: 022bb4041c50d607ff871dec7e4cd7778388e0ea6849d84ccbd9aa8f32e16a8131 ONLINE /dns4/s01.frostfs.devenv/tcp/8080
        Continent: Asia
        Country: Russia
        CountryCode: RU
        Location: Mishkino
        Price: 22
        UN-LOCODE: RU MSK
        User-Agent: FrostFS/0.34
Node 2: 02ac920cd7df0b61b289072e6b946e2da4e1a31b9ab1c621bb475e30fa4ab102c3 ONLINE /dns4/s03.frostfs.devenv/tcp/8080
        Continent: Europe
        Country: Sweden
        CountryCode: SE
        Location: Stockholm
        Price: 11
        SubDiv: Stockholms l�n
        SubDivCode: AB
        UN-LOCODE: SE STO
        User-Agent: FrostFS/0.34
Node 3: 038c862959e56b43e20f79187c4fe9e0bc7c8c66c1603e6cf0ec7f87ab6b08dc35 ONLINE /dns4/s04.frostfs.devenv/tcp/8082/tls /dns4/s04.frostfs.devenv/tcp/8080
        Continent: Europe
        Country: Finland
        CountryCode: FI
        Location: Helsinki (Helsingfors)
        Price: 44
        SubDiv: Uusimaa
        SubDivCode: 18
        UN-LOCODE: FI HEL
        User-Agent: FrostFS/0.34
Node 4: 03ff65b6ae79134a4dce9d0d39d3851e9bab4ee97abf86e81e1c5bbc50cd2826ae ONLINE /dns4/s02.frostfs.devenv/tcp/8080
        Continent: Europe
        Country: Russia
        CountryCode: RU
        Location: Saint Petersburg (ex Leningrad)
        Price: 33
        SubDiv: Sankt-Peterburg
        SubDivCode: SPE
        UN-LOCODE: RU LED
        User-Agent: FrostFS/0.34

SELECT *¶

The statement means that a placement vector is created from all the nodes in the selection.

Policy	Placement vector	Explanation
`REP 4 CBF 1 SELECT * FROM *`	s01, s02, s03, s04	OK. All available nodes are selected
`REP 2 CBF 1 SELECT * FROM nonRU FILTER Country NE Russia AS nonRU`	s03, s04	OK. All nodes with non-'Russia' `Country` attribute value are selected
`REP 4 CBF 1 SELECT * FROM nonRU FILTER Country NE Russia AS nonRU`	s03, s04	There are not enough nodes to replicate the object (4 > 2). If a container is forcefully created with this placement vector, then any PUT operation will result in the `INCOMPLETE PUT` error.
`REP 1 CBF 1 SELECT * IN SAME Country FROM *`	s04	OK. The clause `SAME` splits the original node set to 3 subsets (a.k.a buckets) by `Country` attribute. As `` wildcard is used, the number of nodes in each bucket is not restricted. So, these buckets are considered: `[s04], [s01, s02], [s03]`. These 3 buckets are sorted by the attribute 'Country': `'Finland', ['Russia', 'Russia'], 'Sweden'` - this behaviour is specified within `frostfs-sdk-go`. `SAME` trims the resulting vector to a size of 1: `[s04], [s01, s02], [s03]` -> `[s04]`, because the current placement vector must contain nodes with same* attribute value. If the node `s04` goes off the netmap, then the placement vector will be changed. `[s04], [s01, s02], [s03]` -> `[X], [s01, s02], [s03]` -> `[s01, s02], [s03]` -> `[s01]`. This means object created on `s04` will not be replicated to `s01` and vice versa.
`REP 2 CBF 1 SELECT * IN SAME Country FROM *`	s04	There are not enough nodes to replicate the object (2 > 1). If a container is forcefully created with this placement vector, then any PUT operation will result in the `INCOMPLETE PUT` error.
`REP 2 CBF 1 SELECT * IN SAME Country FROM russian FILTER Country EQ Russia AS russian`	s01, s02	OK. Non-'Russia' nodes are filtered out. Therefore, it gets a placement vector with two nodes: `[s01, s02]`
`REP 1 CBF 1 SELECT * IN DISTINCT Country FROM *`	s01, s03, s04	OK. The clause `DISTINCT` splits the original node set to 3 buckets by `Country` attribute. Each bucket's size is restricted by `1 * CBF`, i.e. `1`. Therefore, `[s04], [s01, s02], [s03]` -> `[s04, s01, s03]`. If the node `s01` goes off the netmap, then the resulting placement vector will be `[s04], [X, s02], [s03]` -> `[s04, s02, s03]` (s01 -> s02). If both `s01` and `s02` are not in the netmap, then the placement vector will contain nodes `s03` and `s04`. If all nodes are not in the netmap, which means the placement vector is empty, then any PUT operation will result in the `INCOMPLETE PUT` error.
`REP 1 CBF 2 SELECT * IN DISTINCT Country FROM *`	s01, s02, s03, s04	OK. Each bucket's size is restricted by `2` as `CBF = 2`.
`REP 1 CBF 2 SELECT * IN DISTINCT Country FROM russian FILTER Country EQ Russia AS russian`	s01, s02	OK. Only one bucket with `Country=Russia`

REP *¶

The statement means that the object is replicated to all nodes from the placement vector

Policy	Placement vector	Explanation
`REP * CBF 1 SELECT 4 FROM *`	s01, s02, s03, s04	OK. The object is replicated to all 4 nodes
`REP * CBF 1 SELECT * FROM noRu FILTER Country NE Russia AS noRu`	s03, s04	OK. The object is replicated to 2 non-'Russia' nodes
`REP * CBF 1 SELECT * IN SAME Country FROM *`	s04	OK. See the table above for what happens if `s04` goes off the netmap
`REP * CBF 1 SELECT * IN SAME Country FROM russian FILTER Country EQ Russia AS russian`	s01, s02	OK. If the node `s02` goes off the netmap, which means the placement vector has changed, then PUT operation won't result in the `INCOMPLETE PUT` error. This is because `REP *` adapts to the current placement vector. If the node `s02` comes back, all objects were created will be replicated from `s01` to `s02`. If both `s01` and `s02` are not in the netmap, which means the placement vector is empty, then PUT operation result in the `INCOMPLETE PUT` error.
`REP * CBF 1 SELECT * IN DISTINCT Country FROM *`	s01, s03, s04	OK. The object is replicated to all 3 nodes. As it mentioned earlier, `DISTINCT` clause splits the original node set to 3 buckets: `[s04], [s01, s02], [s03]`. Then, it trims each bucket to `1 * CBF`, i.e. 1. Considering this fact, if the node `s01` goes off the netmap, then the resulting placement vector will be `[s04, s02, s03]` (s01 -> s02). If both `s01` and `s02` are not in the netmap, then the object will be replicated only to `s03` and `s04` nodes - `REP *` adapts to the current placement vector.
`REP * CBF 2 SELECT * IN DISTINCT Country FROM *`	s01, s02, s03, s04	OK. The object is replicated to 4 nodes. The object can be created and replicated as long as the placement vector isn't empty.
`REP * CBF 2 SELECT * IN DISTINCT Country FROM russian FILTER Country EQ Russia AS russian`	s01, s02	OK. All nodes with Country='Russia' contain replica. The object can be created and replicated as long as in the placement vector isn't empty.
`REP * REP * IN selection1 CBF 1 SELECT 1 IN DISTINCT Country FROM * as selection1`	`[[s01, s02, s03, s04], [s01, s03, s04]]`	OK. First `REP ` goes without explicit* SELECT statement -> this is interpreted as `REP * SELECT * FROM *` -> all nodes are selected. The second placement vector is created from explicit SELECT with DISTINCT clause.
`REP * SELECT * FROM *`	s01, s02, s03, s04	OK. Basically, this policy means "replicate the object to the maximum number of nodes (from the netmap) possible". The PUT operation possible if at least one node is present in the placement vector. If new nodes appear in the netmap, then the placement vector grows and, therefore, objects will be replicated to these nodes. If some nodes go off the netmap, then, consequently, the placement vector is shrunk. If the placement vector is empty (happens when nodes are not it the netmap yet), then PUT operation returns the `INCOMPLETE PUT` error