VIFIB DESCENTRALIZED CLOUD COMPUTING

SlapOS is a decentralized Cloud Computing technology that can automate the deployment and configuration of applications in a heterogeneous environment.

Format SlapOS Node

This document will cover formatting a SlapOS node. Formatting may be necessary on a newly installed SlapOS node in case the recommended single line installer (which includes initial formatting) was not used.

Formatting may also be necessary when a node needs to be repurposed for example to increase the number of computer partitions or adding more storage capacity.

For this how-to command line access to the node is required. Please refer to the SlapOS core documentation for more information on the command line commmands available.

Table of Content

  • Initial Formatting
  • Repurpose Existing Node
  • Reconfiguring Node for Virtual Machines
  • Reformat after adding SSD Storage

Intial Formatting

SlapOS Format - Initial formatting of a node created from packages or source

Initial formatting of a node is only required when installing a node from source or packages. The recommended single line installer already handles default formatting during the installation process.

Formatting always includes registering a node with a master. This is not mandatory but recommend as a node is supposed to be run from a master. In order to register, a token (key and X509 certificate) is required. Follow the steps outlined in installing a Slave node to request a token before formatting a node.

Register SlapOS node

SlapOS Format - Register node

To create the configuration files necessary to format a SlapOS node, choose a recognizeable computer name to identify the node in a network and run:

sudo su
# slapos node register --interface-name lo --partition-number 20 [COMPUTER_NAME]

This will generate several files:

  • /etc/opt/slapos/slapos.cfg: The configuration of your SlapOS Node
  • /etc/opt/slapos/ssl/certificate: Your server SSL Cetificate
  • /etc/opt/slapos/ssl/key: Your server SSL Private Key

Next the default slapos.cfg has to be configured for IPv6.

IPv6 Support

SlapOS Format - IPv6 support

A node must have IPv6 installed. As described in the SlapOS architecture IPv4 and VPN work in theory - provided availability of a large enough address space to add 100+ IPv4 adresses to every node.

When using the single line installer, adding IPv6 is outlined in how to add IPv6 to a SlapOS node. The process for an unconfigured node is the same but some presets have to be made before adding IPv6. In the configuration file /etc/opt/slapos/slapos.cfg (consider eth0 to be the internet interface), XXX does this parameter still exist?

# if using re6st add:
interface_name = eth0
ipv6_interface = lo

# if eth0 already has IPv6, only add:
interface_name = eth0

# for tapVPN add:
ipv6_interface = tapVPN
interface_name = eth0

Slapformat

SlapOS Format - Calling slapformat

To format or reformat a SlapOS node the slapformat command is used (see all command line options). This will configure a node based on the configuration in /etc/opt/slapos/slapos.cfg. Running slapformat without creating or changing the configuration will only "ping" the slapos master

sudo su
# slapos node format --now
2018-03-28 15:32:58 slapos[18922] INFO Updating computer
2018-03-28 15:33:30 slapos[18922] INFO Posting information to 'https://slap.vifib.com/'
2018-03-28 15:33:31 slapos[18922] INFO slapos successfully prepared the computer.

If a node has not been registered or formatted before, slapformat will create 20 computer partitions along with users, their directories, tap interfaces and IPv6 addresses for every service of each computer partition (see SlapOS architecture for more info on what a node contains).

Note, for initial formatting after registering a node, the alter_user option has to be set:

sudo su
# slapos node format --alter_user=True --now

Repurpose Existing Node

SlapOS Format - Repurposing an existing node

slapformat will look for the slapos.cfg file and update a node according to the configuration file. In case a node needs to be repurposed, the slapos.cfg has to be updated before slapformat is called. XXX ? what else, change master, delete ?

Reconfiguring Node for Virtual Machines

SlapOS Format - Reconfiguring a node to support virtual machines

Hosting virtual machines and serving VM instances requires modifying the configuration of a SlapOS node - unless the node was installed from a dedicated disk image like used by Vifib master.

Network Configuration

SlapOS Format - Network Configuration

To host virtual machines, a bridge needs to be configured to support multiple tap interfaces used in virtual machines like a KVM before calling slapformat to acknowledge and modify the configuration. As using bridges on a computer/server can be tricky they are not enabled by default on SlapOS (unless using the Vifib master disk image).

Note: KVM instances will have no internet access without a bridge being configured (no problem if only used for planning/testing). And note, that setting network configuration will not work over wifi.

Debian Bridge

SlapOS Format - Debian Bridge for hosting KVM virtual machines

On a Debian distribution (Debian, Ubuntu, Mint, et al.), setting up a bridge is done by changing the file /etc/network/interfaces and restarting the network. Make sure you add relevant physical interfaces to the bridge. Make also sure that if you add a bridge to a remote server, you still have a way to contact the server in case of failure of the configuration (this happens frequently).

# /etc/network/interfaces
iface eth0 inet manual
auto br0
iface br0 inet dhcp
bridge_ports eth0

# /etc/rc.local
/opt/slapos/bin/slapformat -c /etc/opt/slapos/slapos.cfg
/opt/slapos/bin/bang /etc/opt/slapos/slapos.cfg

Generic Bridge

SlapOS Fromat - Generic Bridge for hosting KVM virtual machines

On a generic GNU/Linux distribution, a bridge setup can be setup entirely from /etc/rc.local, together with slapformat. This is not the most beautiful way to achieve the expected result but it works. Note, that any interface added to a bridge should no longer be configured by the system, meaning the configuration file for that interface should be removed.

# /etc/rc.local
brctl addbr br0
brctl addif br0 eth0
ip l s dev br0 up

Update SlapOS Configuration

SlapOS Format - Updating SlapOS Configuration

Start by updating slapos.cfg to use the newly created br0 bridge as interface:

nano /etc/opt/slapos/slapos.cfg

# under [slapformat] change parameters to:
interface_name = br0
create_tap = true

Then call slapformat (which also creates partitions and allocates IPs) to also create the "TAP" virtual interfaces for each partition and connect them to the bridge. Note to make sure and use the lastest SlapOS version as the create_tap options was not available on older releases.

slapos node format --now

KVM Configuration

SlapOS Format - Configuring KVM Settings

Load the required kernel modules and change the rights of 666 or 777/dev/kvm to 666.

# modprobe kvm
# modprobe kvm_intel
# modprobe kvm_amd
# chmod 777 /dev/kvm

and under [slapformat] change the parameters to:

interface_name = br0
use_tap = true

Note, for this to work, virtualization must be activated in the BIOS of your computer.

Reformat after adding SSD Storage

SlapOS Format - Reformatting after Hardware Modification

This section will explain how to directly use newly mounted SSD storage disks from running services in SlapOS computer partitions.

Additinal SSD may be required because of growing backup/log files or to attach a multi-storage, for example for reducing the I/O load on Cloudera data nodes with multiple virtual disks (one system disk, many data disks) by adding storage to eventually have a phyiscal disk per virtual disk.

Mount and Configure Disks

SlapOS Format - Mount and Configure additional SSD disks

After plugging in the new disk(s) they need to be mounted. Assume you have two disks, /dev/sdb1 and /dev/sdc1 which should be used to extend partitions. The disk will be formatted like a normal SlapOS partition to allow deployed instances to access the disks directly from their partition folder. Start by creating a base storage folder /data where disks will be mounted:

mkdir /data

Create a data[x] folder for each storage to mount into our storage.

mkdir /data/data1 /data/data2

Then mount the disks

mount -t ext2 /dev/sdb1 /data/data1
mount -t ext2 /dev/sbc1 /data/data2

In case of server reboot, we want to auto remount storage.

# vim /etc/fstab
UUID=STORAGE_SDB1_UUID              /data/data1                ext2    default    0 2
UUID=STORAGE_SDC1_UUID              /data/data2                ext2    default    0 2

where STORAGE_SDB1_UUID and STORAGE_SDC1_UUID can be retrieved using the command blkid.

Enable Support for Attached Storage

SlapOS Format - Enabling Support for newly attached Storages

After mounting, SlapOS needs to be informed where to find the attached storage disk. This is done by editing the configuration in the SlapOS config file at /etc/opt/slapos/slapos.cfg and then setting the instance_storage_home parameter to the base storage folder created earlier.

# vim /etc/opt/slapos/slapos.cfg
[slapos]
...
instance_storage_home = /data

Finish by running slapos format to reconfigre the attached storage.

slapos node format --now -v

When formatting finishes, the storage /data/data[x] will contain new folders created and managed by SlapOS to be used by slap partitions when new instances will be deployed.

# ls /data/data2/
total 120
drwxr-x--- 2 slapuser0  slapuser0  4096 Mar 23  2015 slappart0
drwxr-x--- 2 slapuser1  slapuser1  4096 Mar 23  2015 slappart1
drwxr-x--- 2 slapuser10 slapuser10 4096 Nov  7 02:29 slappart10
drwxr-x--- 2 slapuser11 slapuser11 4096 Nov  7 02:29 slappart11
...

Each partition on the other hand will have one additional new folder called DATA which contains all mounted disks (data1 and data2). Instance can use them as part of their partition file system.

# ls -l /srv/slapgrid/slappart19/
total 96
drwxr-x--- 2 slapuser19 slapuser19  4096 Jun 15 14:10 bin
-rw-r----- 1 root       slapuser19  9376 Nov  8 12:24 buildout.cfg
-rw-r----- 1 slapuser19 slapuser19  8282 Nov  8 12:24 buildout-switch-softwaretype.cfg
drwx------ 2 slapuser19 slapuser19  4096 Jun 15 14:10 DATA

The folder DATA also contains all mounted data folders:

# ls -l /srv/slapgrid/slappart19/DATA/*/
/srv/slapgrid/slappart19/DATA/data1/:
total 0
/srv/slapgrid/slappart19/DATA/data2/:
total 0

Note, that if the instance is destroyed, all attached storage content will be removed when SlapOS will clean up the partition.

Thank You

Image Nexedi Office
  • Nexedi GmbH
  • 147 Rue du Ballon
  • 59110 La Madeleine
  • France