Cellframe Wiki

Search

SearchSearch

Create your own Cluster with Python

Apr 22, 20253 min read

To create your own cluster in the Cellframe network, you should firstly know more about cluster architecture in our blockchain. If you already know what cluster are, click on the link python example.

Network clusterization

Clusterization is the next stage in the evolution of our blockchain, and also preparation for 2-level sharding (between database and chains).

Definition

A cluster is a set of nodes, connected with each other based on their similar properties, and work together as one.

Cluster architecture is a concept which allows us to represent a network as an open cluster. Such a cluster is able to accept new members by request from nodes. This concept allows for the use of a common synchronization algorithm for both opened and closed clusters using the standard network protocol (TCP).

This clustered architecture lays the foundation for the future implementation of sharding, a technology that will significantly enhance the network’s scalability and performance. Sharding is a crucial mechanism that divides the network into segments or shards.

Sharding helps distribute network load more efficiently and increases its throughput. In our case, sharding is particularly important because we utilize “heavy” post-quantum algorithms, which increase transaction weight and impact network speed.

Cellframe clusters

Clusters can have different types; depending on the cluster’s type, both its link management and its ability to automatically add members to the cluster will vary.

In the Cellframe network we have three types of clusters:

  • GDB Group Clusters are collections of nodes responsible for data storage and node list management. For instance, the KelVPN node cluster includes groups like KelVPN.nodes and KelVPN.nodes.aliases.
  • Role Clusters are groups of nodes with specific privileges, such as mainchain network validator, root, etc.
  • Link Clusters are nodes grouped based on link management.

Currently, we use the SYSTEM, AUTONOMIC, and EMBEDDED types of Link Cluster. The ISOLATED type is also defined, but full support for this type has not been added yet.

  • SYSTEM clusters are basic clusters with “local.” and “global.” groups, which either contain a single local node or all authorized nodes that have connected to the current one. Node authorization is based on its address. Unauthorized nodes cannot participate in clusters.

  • EMBEDDED clusters are network clusters. Links in these clusters are created through a load balancer, which is called back by the link manager. Anyone can join a network by announcing themselves using a special protocol. They will simultaneously be added to the cluster of that network. There is also the possibility of denunciation, which allows sharing a single link among multiple networks, as well as with static clusters represented by the type.

  • AUTONOMIC clusters are specifically designed for emission center tasks and are also known as static clusters. This cluster type establishes links to all members assigned roles within the cluster. No other members have access to the cluster.

Role Cluster types are presented in the next enums:

  • ROLE_NOBODY - nobody, member with this node address can’t have access to cluster in any way
  • ROLE_GUEST-  guest, member with this node address has access to cluster only in read mode
  • ROLE_USER - user, can write to GDB groups of cluster and read from them
  • ROLE_ROOT - super user, has additional right to rewrite/delete records in GDB groups of clusters.

The role is written by default for clusters with EMBEDDED type upon auto adding new members. All new members automatically get exactly this role. For static clusters (AUTONOMIC type) this parameter is not necessary, but you can set ROLE_NOBODY which will be correct.

The owner_root_access parameter allows the user, who created the record in GDB, to independently change/remove it. Other users, except super users, can’t do that.

There is also a parameter called TTL (time to live) - the lifetime of records in cluster groups. This parameter is set in hours for convenience, but the internal value is calculated in seconds.

Link to original

Python example

Here is the example of how to create your own AUTONOMIC cluster in the Backbone network using python script for 3 nodes with ROOT role.

from pycfhelpers.node.net import CFNodeAddress
from pycfhelpers.node import CFNet
from pycfhelpers.node.crypto import CFGUUID
 
# imported necessary python libraries
 
BACKBONE_NET = CFNet("Backbone")
MY_CLUSTER_ID = 0xAA
CLUSTER_NODES = [CFNodeAddress("AAAA::AAAA::AAAA::AAAA"),
                CFNodeAddress("BBBB::BBBB::BBBB::BBBB"), 
                CFNodeAddress("CCCC::CCCC::CCCC::CCCC")]
                
 
# created three nodes and specified their addresses
 
def setup_cluster():
    net = BACKBONE_NET
    cluster = CFGDBCluster("myclustername", #mnemomic
                             CFGUUID.compose(net.id.long, MY_CLUSTER_ID), #cluster id
                             "mycluster.gdb.group.mask.*", #gdb groups mask
                              3, #TTL of objects in cluster DB (hours)
                              True, # Root access for cluster owner
                              CFGDBCluster.MemberRole.NOBODY,  # ignores for AUTONOMIC
                              CFGDBCluster.ClusterRole.AUTONOMIC) #Autonomic cluster
 
# created constructor for new cluster with specified parameters (full description in more detailed chapter)
 
    cluster.add_net_associate(net) #important, cluster will go online if net does
 
# associated cluster with the network
 
   #add members
    for member in CLUSTER_NODES:
        cluster.member_add(member, CFGDBCluster.MemberRole.ROOT)
 
 # added members and gave them root role

More detailed description of the snippet

Imports:

from pycfhelpers.node.net import CFNodeAddress
from pycfhelpers.node import CFNet
from pycfhelpers.node.crypto import CFGUUID

These lines import necessary modules from the pycfhelpers library:

Important

For more information about pycfhelpers library, see article Sphinx AutoDocumentation

  • CFNodeAddress: This class represents an address for a node in the network.
  • CFNet: This class represents a specific network within the system.
  • CFGUUID: This class deals with generating unique identifiers.

Constants:

BACKBONE_NET = CFNet("Backbone")
MY_CLUSTER_ID = 0xAA
CLUSTER_NODES = [CFNodeAddress("AAAA::AAAA::AAAA::AAAA"),
                 CFNodeAddress("BBBB::BBBB::BBBB::BBBB"),
                 CFNodeAddress("CCCC::CCCC::CCCC::CCCC")]
  • BACKBONE_NET: Defines a network named “Backbone” using the CFNet class.
  • MY_CLUSTER_ID: Sets a constant cluster ID with the value 0xAA.
  • CLUSTER_NODES: Defines a list containing three CFNodeAddress objects with addresses specified.

Function Definition:

def setup_cluster():

This line defines a function named setup_cluster for setting up a cluster.

Function Body:

  net = BACKBONE_NET

This line assigns the previously defined BACKBONE_NET constant to a local variable net.

Cluster Creation:

  cluster = CFGDBCluster("myclustername",  # mnemomic 
                         CFGUUID.compose(net.id.long, MY_CLUSTER_ID),  # cluster ID
                         "mycluster.gdb.group.mask.*",  # gdb groups mask
                         3,  # TTL of objects in cluster DB (in hours)
                         True, # Root access for owner
                         CFGDBCluster.MemberRole.NOBODY,  # ignores for AUTONOMIC (role of the used node)
                         CFGDBCluster.ClusterRole.AUTONOMIC)  # Autonomic cluster (must be common)

This line creates a cluster object using a class named CFGDBCluster. The arguments passed to the constructor are:

  • "myclustername": (string) A name for the cluster
  • CFGUUID.compose(net.id.long, MY_CLUSTER_ID): (method which returns CFGUUID) Creates a unique cluster ID using the CFGUUID class. It combines the network ID with the MY_CLUSTER_ID constant.
  • "mycluster.gdb.group.mask.*": (string) Defines a mask for GDB groups, which is used in internal communication between nodes.
  • 3: (integer) Sets the Time-To-Live (TTL) for objects in the cluster database to 3 (hours).
  • True: (boolean) Defines whether cluster owner get ROOT access or not (yes if true).
  • CFGDBCluster.MemberRole.NOBODY: (constant) Sets NOBODY as member role.
  • CFGDBCluster.ClusterRole.AUTONOMIC: (constant) Sets AUTONOMIC as cluster role.

Associating Network:

  cluster.add_net_associate(net)  # important, cluster will go online if net does

This line associates the previously defined net (Backbone network) with the cluster. It is an important step for the cluster to become operational.

Adding Members:

  # add members
  for member in CLUSTER_NODES:
    cluster.member_add(member, CFGDBCluster.MemberRole.ROOT)

This loop iterates through the CLUSTER_NODES list (containing node addresses) and adds each node as a member to the cluster. The member_add function takes two arguments:

  • member: The address of the node to be added (from CLUSTER_NODES).
  • CFGDBCluster.MemberRole.ROOT: This sets the role of the added member to ROOT.

Graph View

Backlinks