VIFIB DESCENTRALIZED CLOUD COMPUTING

SlapOS is a decentralized Cloud Computing technology that can automate the deployment and configuration of applications in a heterogeneous environment.

SlapOS Economic Rationale

The goal of this document is to introduce the economic rationale which drives SlapOS research and development.

Since 2007, many companies which business is to sell open source software integration services have migrated their groupware from an open source platform to a proprietary cloud platform. Industrial companies that had been using proprietary groupware happily for more than 10 years have also opted for similar strategy. Costs seem to be the driving motivation for change.

This presentation will first analyze the costs of different approaches (traditional vs. Cloud) and later introduce a list of economic requirements which an open source / Free Cloud solution should meet in order to provide a competitive advantage over proprietary Cloud.

Agenda

  • Traditional Groupware Costs
  • Cloud Groupware Costs
  • Cost Comparison
  • Job Impact
  • Data Loss Risk
  • SlapOS Goals

We will first define a introduce scenario of groupware implementation in a company based on open source or proprietary software. Then we shall review the different commercial offers which are proposed to companies to implement similar solutions through the Cloud. Some commercial offers are free (gratis) while others require to pay (paid). We will then depict the impact of Cloud for long term IT jobs which are implied by the migration of many companies from traditional IT to Cloud based IT.

We will then focus on one hidden cost of both Cloud and traditional IT which is often absent of studies: the risk of data loss due to incorrect backup procedures or force majeure. SlapOS requirements will then be defined by combining an architecture which reduces the risk of data loss (or leaks) and at the same time matches or surpasses the best Cloud offers in terms of costs.

Traditional Groupware (Open Source)

  • Open Source Software: 0€
  • Server: 2 * 1000 / 100 / 24 = 0.8 €
  • System Administrator: 800 / 100 = 8 €
Total: 8.8 € / user / month

Until recently, one of the most cost efficient way to implement groupware in a company was to select an open source groupware suite such as Horde, OBM, Group Office, Feng Office, etc. purchase two servers (so that one server can replace the other one in case of hardware failure) and ask a system administrator to come one day a month to make sure everything is going well on both servers. This approach is quite typical of small companies.

For 100 users, a typical server costs 1000€ and is used in production for 2 years or 24 months. This represents a cost of 0.8€ per user and per month. A system adminstrator costs 800 € per day and operates on the server once a month in average to create accounts, make sure there is enough disk space, install system security upgrades, etc. System administration represents a cost of 8 € per month and per user. Overall, the total cost is 8.8 € per user and per month

By hosting more users on the server (ex. 1000) and using lower cost labour (ex. 200 € per day), the total cost per user can be reduced to less than 0.2 € per month. Therefore, what would cost 8€ in a small company in a country with high labour costs, could cost 0.2 € in a country with lower labour costs and in medium sized businesses.

Traditional Groupware (Proprietary)

  • Proprietary Software: 20€
  • Server: 2 * 1000 / 100.0 / 24 = 0.8 €
  • System Administration: 800 / 100 = 8 €
Total: 28.8 € / user / month

Large organizations used to rely on proprietary groupware such as Lotus Notes. Some of them such as Valeo later migrated to Cloud. The cost of traditional groupware implementation based on proprietary software is similar to the cost of open source with extra costs for software licenses which include not only the groupware application as such but also a fairly complex system environment made of LDAP directories, proprietary operating system, proprietary monitoring tools, proprietary office suite and proprietary asset management. Typical yearly cost for the groupware environment licenses could be at least 240 € per user, resulting in 20 € monthly cost.

The total cost per user per month of the groupware environment is thus close to30 €. Increasing the number of users or reducing labour cost has little impact on the total cost since most costs are licenses.

XMail Cloud (gratis)

  • Advertising: 0.5 €
  • Hard Disk: 100 * 7 / 1000.0 / 24 = 0.03 €
  • System Admin.: 100 * 10000 / 10000000 = 0.1 €
  • R&D: 20 * 10000 / 10000000 = 0.02 €
Total: 0.15 € / user / month

Many entrepreuneurs have opted for free (gratis) groupware since 2007. A well known French open source system integrator, which provides to its customers open source software integration including open source groupware integration, uses for its own staff a proprietary Cloud groupware suite, mainly because it was free (gratis) when the company was created. The company has now hundred employee and still uses proprietary Cloud groupware, without paying any monthly fee or system administration cost.

It is quite difficult to estimate a cost for such a solution since the solution is gratis. However, since the solution is sponsored by commercial links, a measure of the cost can be estimated through the number of clicks made by each user every month. Another measure of the cost can be estimated from the use of resources required to provide the Cloud service.

We found that typical users tends to click once a month and that commercial links are priced between 0.1€ and 1€, with a typical price of 0.5€. This gives a low estimate of the "price" of gratis Cloud groupware. If we included the time spent by employee to watch commercial links, this cost would probably be higher.

Another estimation is based on the usage of physical resources and human sources required to produce the service.

The hardware cost of the service depends mostly on the cost of storage rather than on the cost of CPU. With a typical cost of 100 € for 1 terabyte of storage on a hard disk and an average of 7 GB allocated per user, we reach a hardware cost of 0.03€ per month. We took into account in this calculation the fact that storage is duplicated (ie. 14 GB are needed to provided 7 GB of redundant storage). We also took into account the fact that some users use all the 7GB storage while others mostly nothing, with an average usage of 3.5 GB. We consider the cost of CPU or RAM to be close to zero since most users are only connected a small part of the day. Memory is then shared very efficiently by using a kind of multi-tenant architecture.

Intangible costs per user depend on the number of real users, since only those users are likely to provide turnover through commercial links and can be considered as equivalent to users of a traditional approach.

Gratis cloud services usually have many subscribers and much less real users, especially simultaneous users. The number of real users is a figure which is difficult de find and is suprisingly lower than the figures published by marketing departments. For example, one of the largest dating services in the world reaches daily a peak of 30,000 simultaneous connections. The number of "real" users can thus be estimated to be 3 to 10 times this value if we take into account time differences, which gives a figure 100,000 real users. We could imagine that 1% proprierty cloud groupware users also use dating services, which gives a value of 10,000,000 real and active users for cloud groupware (the 1% ratio is a typical low ratio found for all love related commercial services). Another estimate could be based on the number of registered users in facebook, less than one billion, and a 1% ratio to convert registered users into real users. We also reach by this reasoning the same figure of 10,000,000 real users. The 1% ratio is quite typical of Web 2.0 services: people subscribe to them much more than they actually use them. The 1% ratio is probably overestimated since many gratis services rather show a 1 to 1000 rate between real users and subscribed users.

If system administration is handled by a team of 100 engineers, the cost of system administration for 10,000,000 users is equal to 0.1 € per user and per month. If the R&D staff consists of a small team of typically 20 people, R&D costs are equal to 0.02 € per user and per month. The cost of gratis cloud groupware is thus equal to about 0.15 € per user and per month.

We have found through different approaches a montly cost per user of  0.1€ to 0.5€. This cost does not directly translate into a price, since gratis cloud is financed by advertising. However, it provides a target value which any open source cloud solution should match to be competitive in the long term, or which any open source cloud solution should match to be compatible with a business model based on advertising.

XMail Cloud (paid)

  • Google Apps Premier: 5 €
  • OVH MS Exchange: 3.3 €
  • Group Office: 3 €
Total: 3 - 5 € / user / month

Commercial cloud groupware offers also exist and are much easier to analyze since their price are public for most of them.

Google provides a "Premier" version of Google Apps at a cost of 5€ per user and per month. OVH provides hosted Microsoft Exchange at a cost of 3.3€ per user and per month. Group Office is a hosted version of the open source Group Office open source groupware. It is interesting to notice that the price levels of commercial cloud offers are 30 times higher that their costs as we calculated them previously. There is thus still a lot of room for competition.

Price Comparison

  • Traditional Open Source: 0.2-8.8 €
  • Traditional Proprietary: 28.8 €
  • Google Apps Gratis: 0.1-0.5 €
  • Google Apps Premier: 5 €
  • OVH Hosted MS Exchange: 3.3 €
  • Group Office: 3 €

By comparing different approaches to implement groupware in a 100 people company in a developped country, we found that cloud based solutions offer price levels which are 60 times lower than traditional open source solutions (0.1€ to 0.5€ vs. 8.8€). Paid cloud solutions lower prices by a magnitude of 10 compared to traditional proprietary solutions. This explains why many companies (Jaguar, Rover, BBVA) abandon traditional implementation of groupware and migrate to cloud, especially in countries with high labour costs and unflexible labour market.

However, the level of profits generated by current  paid Cloud offers is extremely high. Lower cost Cloud offers are likely to reach the market very soon.

Traditional open source approaches may also remain a competitive option in countries with low labour costs and in companies with more than 1000 users. Economies of scale and mature open source software packages can reduce the cost of open source groupware to 0.2€ per month and per user in some cases.

Job Impact

  • Google: 10000000 / 100 = 100000
  • Traditional: 20 * 100 = 2000
Productivity increase: 50 x

Cloud will have a major impact on the IT job market by reducing the number of open positions for system administrators and consultants.

A team of 100 system administrators can manage 10 million groupware accounts, whereas in a traditional approach, a single consultant can serve 100 users by spending one day per month, which means for a month of 20 working days, 2000 users. This figure of 2000 users compares to 100,000 users per system administrator for cloud based approaches.

Cloud can improve IT productivity by  magnitude of 50. In other terms, what used to require 100 system administrators in traditional IT, only requires 2 with cloud, leaving 98 system adminstrators to an unpredicatble destiny.

Cloud plays here the same role as ERP for accounting, robots for industry or mechanization for agriculture: it automates the work of certains categories of human beings, who then become less competitive than machines. This economic process has been depicted in the essay  "The End of Work" by Jeremy Rifkin. With Cloud, it is probably the first time that a category of workers (IT engineers) produces tools which will create massive unemployment for the same category of workers (IT engineers). 

Understanding the impact of Cloud on the IT job market is also a better way to define what "Cloud Computing" is: the automation through software of IT processes which used to be managed by human beings. Currently, Cloud computing focuses mostly on the automation of system administration. However, latest researches in Cloud Computing show that the process of tailoring ERPs for small companies can be automated and that the configuration of applications could be done by software rather than by consultants.

Data Loss Risk

An interesting aspect of cost comparison which is seldom taken into account is the risk of data loss. Corporate data can be very valuable. Losing accounting data or customer data can lead a stable and profitable company to bankrupcy in a matter of months.

There are already a few examples of data loss on the Cloud, either for technical reasons (ex. "Sidekick story") or for social reasons (see the "story of erased google account").  We also all know many stories of companies that use Storage Area Network (SAN) device and backup solutions from reputable makers but that lost their data because they did not monitor well enough the alarms of either system. The SAN filesystems got currupted and the backup solution had no longer been running for 6 months.

Is cloud safer or not than traditional IT from this point of view?

It is hard to tell currently without figures. Some companies with perfect backup plan could bear less risk than Cloud. Many companies implement a poor backup plan and would reduce their risk by migrating to Cloud.

One thing is certain however: current Cloud centralized architectures are based on large data centers and are operated by one same company all over the world. Centralized architectures are quite weak in case of strike, of bug, of virus or terrorist attack. It is not only a matter of lack of technical redundancy but also of lack of management redundancy.

Also, the perceived risk of data loss in Cloud is becoming higher and higher. Just like with air transportation, which is the safest transportation system, fears of accident can lead some users to reject it. A Cloud disaster, which is already predicted to happen in 2012, could generate a lot of distrust for Cloud.

SlapOS Goals

  • Share application R&D
  • Share automation software
  • Share hardware capacity
  • Share data loss risk
  • Share management
Goal: <= 0.15 € / user / month incl. data loss risk

The goal of SlapOS is to provide a competitive Cloud solution which does better to reduce risks of data loss than any other solution.

A single word summarizes the requirements to build such a solution: share

Application R&D is shared by using open source applications such as Horde or Wordpress which are widely spread and maintained by community. The cost of R&D is thus close to zero for end users and competes with the costs of R&D of companies with million users.

Automation software, the core of SlapOS, is based on existing open source tools maintained by community such as buildout or supervisord. This reduces the cost of development of automation tools by a magnitude of 10 or more. Other automation components are shared among all applications, through a concept of stack which is itself open source. All LAMP (Linux, Apache, MariaDB, PHP) could for example share the same stack and benefit from improvements made to the stack, which providing enough freedom for each application to implement fine tuning. There are already more than 100 LAMP applications in SlapOS. Adding one more application with complete automation of deployment, backup, resilience and front-end acceleration taks less than four hours.

Hardware capacity should be shared (and waste eliminated) by efficient system design and capacity sharing among providers.

A single 1000 € server should be able to host at least 1000 to 10,000. A form a multi-tenancy must thus be provided by default by SlapOS, since most open source applications are not multi-tenant. This form of multi-tenancy is called in SlapOS: POSIX, an operating system which allows multiple processes with different users to share the system RAM and CPU, and share the same application code through shared libraries. By using POSIX processes (rather than virtual machines), it is possible to implement dynamic allocation policies which can split a single server into 100 containers, each of which with a groupware instance of Horde and 10 users. By starting and stopping automatically POSIX processes, the number of containers can be increased to 200 or more. The cost of a complete groupware suite for a single company can be as low as 1€ / month. This figure should be compared to approaches based on Amazon EC2  which price is 10 times higher if not 100 times. And since a kvm virtual machine can be a POSIX process, using POSIX processes as the common model for Cloud covers more possibilities than Cloud models based on virtual machines.

Another form of hardware capacity sharing consists for one Cloud provider to supply any extra unused capacity to another Cloud provider. This approach reduces the cost of hardware resources for every Cloud provider since it optimizes their usage.

Risk of data loss should be reduced by sharing risk accross continents. Every SlapOS hosted application should be replicated in 2 or 3 regions in the world. In case of failure of SlapOS Cloud resources in one region (electricity, connectivity, natural disaster, etc.), another replica of the application can provide continued service.

SlapOS should be designed to resist management mistakes. Taking a wrong management decision (mistake, insanity, etc.) on a single SlapOS node should not destroy the entire system. Management should thus be distributed and shared accross multiple organizations in the world, each running its own SlapOS cloud.

By meeting all requirements, SlapOS can compete in price with the lowest cost proprietary Cloud yet provide higher resilience and data protection than any other solution.