membership

Contents

• Functions
• Tables
• Encryption Functions
• Subscription Functions
• membership.options Functions

Modules

• membership

Topics

• README
• CHANGELOG

Module membership

Membership library for Tarantool based on a gossip protocol.

This library builds a mesh from multiple tarantool instances. The mesh monitors itself, helps members discover everyone else and get notified about their status changes with low latency.

It is built upon the ideas from consul, or, more precisely, the SWIM algorithm.

Membership module works over UDP protocol and can operate even before tarantool box.cfg was initialized.

Functions

init (advertise_host, port)	Initialize the membership module.
broadcast ()	Discover members in local network.
leave ()	Gracefully leave the membership group.
mark_left ()	Forcefully send leave message about an instance.
members ()	Obtain all members known to the current instance.
pairs ()	Iterate over members.
get_member (uri)	Get info about member with the given URI.
myself ()	Get info about the current instance.
add_member (uri)	Add a member to the group.
probe_uri (uri)	Send a ping to a member.
set_payload (key, value)	Update payload and disseminate it along with the member status.
remove_member (string)	Remove a member.
set_allowed_members (table)	Filter out members from the list.

Tables

MemberInfo

Member data structure.

Encryption Functions

get_encryption_key ()	Retrieve the encryption key that is currently in use.
set_encryption_key (key)	Set the key used for low-level message encryption.

Subscription Functions

subscribe ()	Subscribe for updates in the members table.
unsubscribe (cond)	Unsubscribe from membership updates.

membership.options Functions

options.PROTOCOL_PERIOD_SECONDS	Period of sending direct PINGs.
options.ACK_TIMEOUT_SECONDS	Time to wait for ACK message after PING.
options.ANTI_ENTROPY_PERIOD_SECONDS	Period to perform anti-entropy sync.
options.SUSPICIOUSNESS	Toggle producing suspect rumors when ping fails.
options.SUSPECT_TIMEOUT_SECONDS	Timeout to mark suspect members as dead.
options.NUM_FAILURE_DETECTION_SUBGROUPS	Number of members to try indirectly pinging a suspect.
options.MAX_PACKET_SIZE	Maximum size of UPD packets to send.
options.ENCRYPTION_INIT	Initialization vector for aes256 CBC encryption.

Functions

init (advertise_host, port)

Initialize the membership module. Bind a UDP socket to 0.0.0.0:<port>, set the advertise_uri parameter to <advertise_host>:<port>, and incarnation to 1.

The init() function can be called several times, the old socket will be closed and a new one opened.

If the advertise_uri changes during the next init(), the old URI is considered DEAD. In order to leave the group gracefully use the leave function.

Parameters:

• advertise_host string either hostname or IP address being advertised to other members
• port number UDP port to bind and advertise

Returns:

boolean true

Raises:

Socket bind error

broadcast ()

Discover members in local network. Send UDP broadcast to all networks discovered by getifaddrs() C call

Returns:

true if broadcast was sent

Or

false if getifaddrs() fails.

leave ()

Gracefully leave the membership group. The node will be marked with the status left and no other members will ever try to reconnect it.

Returns:

boolean true if call succeeds, false if member has already left.

mark_left ()

Forcefully send leave message about an instance.

Returns:

boolean true if call succeeds, false if member has already left.

members ()

Obtain all members known to the current instance.

Editing this table has no effect.

Returns:

table a table with URIs as keys and corresponding MemberInfo as values.

pairs ()

Iterate over members. A shorthand for pairs(membership.members()).

Returns:

Lua iterator

Usage:

for uri, member in membership.pairs() do end

get_member (uri)

Get info about member with the given URI.

Parameters:

• uri string <advertise_uri> of member of interest

Returns:

MemberInfo the member data structure of the instance with the given URI.

myself ()

Get info about the current instance.

Returns:

MemberInfo the member data structure of the current instance.

add_member (uri)

Add a member to the group. Also propagate this event to other members. Adding a member to a single instance is enough as everybody else in the group will receive the update with time. It does not matter who adds whom.

Warning: The gossip protocol guarantees that every member in the group becomes aware of any status change in two communication cycles.

Parameters:

• uri string <advertise_uri> of member to add

Returns:

1. true or nil
2. optional string

   Possible errors:

   • "parse error" - if the URI can not be parsed

probe_uri (uri)

Send a ping to a member. Send a ping-message to a member to make sure it is in the group.

If the member responds but not in the group, it is added.

If it already is in the group, nothing happens.

Warning: When destination IP can be resolved in several diffent ways (by different hostnames) it is possible that probe_uri() function returns "no response" error, but the member is added to the group with another URI, corresponding to its <advertise_uri>.

Parameters:

• uri string <advertise_uri> of member to ping

Returns:

1. true or nil
2. optional string

   Possible errors:

   • "parse error" - if the URI can not be parsed
   • "ping was not sent" - if hostname could not be reloved
   • "no reponce" - if member does not responf within 0.2 seconds

set_payload (key, value)

Update payload and disseminate it along with the member status. Also increments incarnation.

Parameters:

• key string a key to set in payload table
• value auxiliary data

remove_member (string)

Remove a member. Don't use it unless you having a trouble with stale members.

Parameters:

• string uri

set_allowed_members (table)

Filter out members from the list. If the function wasn't called or allowed uri list if empty, all members are allowed.

Parameters:

• table uris URIs to leave in the list

Tables

MemberInfo

Member data structure. A member is represented by the table with the following fields:

Fields:

• uri string <advertise_uri> of a member
• status string

  a string that takes one of the values below

  • alive: a member that replies to ping-messages is alive and well.
  • suspect: if any member in the group cannot get a reply from any other member, the first member asks three other alive members to send a ping-message to the member in question. If there is no response, the latter becomes a suspect.
  • dead: a suspect becomes dead after a timeout.
  • left: a member gets the left status after executing the leave function.
• incarnation number a value incremented every time the instance status changes, or its payload is updated
• payload table an auxiliary data that can be used by various modules
• timestamp number a value of fiber.time64() which corresponds to the last update of status or incarnation; it is always local and does not depend on other members’ clock setting.
• clock_delta number difference of clocks (fiber.time64) between self and peer calculated during ping/ack protocol step or while probe_uri call

Usage:

tarantool> membership.myself()
---
uri: "localhost:33001"
status: "alive"
incarnation: 1
payload:
    uuid: "2d00c500-2570-4019-bfcc-ab25e5096b73"
timestamp: 1522427330993752
clock_delta: 700
...

Encryption Functions

get_encryption_key ()

Retrieve the encryption key that is currently in use.

Returns:

string encryption key

set_encryption_key (key)

Set the key used for low-level message encryption. The key is either trimmed or padded automatically to be exactly 32 bytes. If the key value is nil, the encryption is disabled.

Parameters:

• key string encryption key

Returns:

nil

Subscription Functions

subscribe ()

Subscribe for updates in the members table.

Returns:

fiber.cond object which is broadcasted whenever the members table changes

unsubscribe (cond)

Unsubscribe from membership updates. Remove subscription on cond object.

If parameter passed is already unsubscribed o invaled nothing happens.

Parameters:

• cond fiber.cond object obtained from subscribe function

Returns:

nil

membership.options Functions

options.PROTOCOL_PERIOD_SECONDS

Period of sending direct PINGs. Denoted as T' in SWIM paper.

Default is 1

options.ACK_TIMEOUT_SECONDS

Time to wait for ACK message after PING. If a member does not reply within this time, the indirect ping algorithm is invoked.

Default is 0.2

options.ANTI_ENTROPY_PERIOD_SECONDS

Period to perform anti-entropy sync. Algorithm is described in SWIM paper.

Default is 10

options.SUSPICIOUSNESS

Toggle producing suspect rumors when ping fails. Even if disabled, it doesn't affect neither gossip dissemination nor other statuses generation (e.g. dead and non-decryptable).

Default is true

options.SUSPECT_TIMEOUT_SECONDS

Timeout to mark suspect members as dead.

Default is 3

options.NUM_FAILURE_DETECTION_SUBGROUPS

Number of members to try indirectly pinging a suspect. Denoted as k in SWIM paper.

Default is 3

options.MAX_PACKET_SIZE

Maximum size of UPD packets to send.

Default is 1472 (Default-MTU (1500) - IP-Header (20) - UDP-Header (8))

options.ENCRYPTION_INIT

Initialization vector for aes256 CBC encryption.