Manageacloud

Web Based Terminal

We have upgraded the web-based terminal. It is now a filly compatible terminal emulator. Until now, we have been recommending the usage of you favourite terminal if you were performing longer sessions or if you need to run programs that are graphically complex.

With the new implementation of the web-based terminal, you can run long sessions and complex visual programs with ease. You are still able to use your favourite terminal emulator if you prefer, but it is not longer mandatory.

This upgraded removes the need for any external program when you are creating and maintaining the server configurations with Mananageacloud. You web browser is everything you need.

Web based terminal


Ubuntu Utopic Unicorn 14.10

Manageacloud is now compatible with Ubuntu Utopic Unicorn 14.10 released on 23th of October. Ubuntu Utopic Unicorn 14.10 is currently compatible with Rackspace and DigitalOcean. We are looking toward compatibility with Amazon Web Service too.

If you want to automate the deployment of an application in Ubuntu Utopic Unicorn, try manageacloud now!


The Sysadmin IDE

A few weeks ago, I was working with a well known programming Integrated Development Environment (IDE) for Java. I was analysing a plain SQL when I decided to press CTRL over the table name and click on it. I'd never tried this before. Suddenly the right side of the IDE opened up, connected to the database and fetched the table schema. What a wonderful feature!.

And the truth is that the IDE for programming is so useful that programmers don't work without it. It is so much quicker and easier to use.

When you automate systems, you will iterate over three basic steps:

  1. Create new functionality (or fix an existing one)
  2. Test the code
  3. Analyse new problems

The Sysadmin IDE is designed to assist you in those iterations.

 

Create new functionality (or fix an existing one)

The new functionality is created through the Sysadmin IDE. If you know your Linux distribution well enough, you won't need much training to become proficient in this technology.

This IDE is based in the Package Centric Design Pattern.

The package is the pointer to where the configuration resides. You can customise its files:

  1. Add, modify or delete files and folders
  2. Modify file and folder permissions

If you need to create a very specific behaviour, (like cloning from a it repository or adding a user) you do so through hook scripts, than can be written in any language. These hook scripts can be pre-hooked (executed before the package is installed) or post-hooked, (executed after the package is installed)

Along with these configuration capabilities, you also have access to all the documentation related to the package you are customising.

Test the configuration

You can test the configuration by creating temporary virtual servers with the base operating system in the cloud.

Once the virtual server is created, you can run the configuration.

When you install several packages at once, you can control the order by the dependencies in the hooks. If there are no hooks with dependencies, we assume that the order of installation is unimportant. The different configuration items are installed in this order:

  1. Repository configuration
  2. Pre hooks
  3. Package
  4. Post hooks

Repository configuration, package and hooks will be performed when the conditions are met. For example, if you mark apache2 as 'installed' and it is not in the system, it will be installed automatically.

The hooks runs successfully once per session.  If they fail, they will be executed again the next time you decide to run the configuration.

If the virtual server becomes unusable, or if you want to test the configuration from a new box, you can delete and recreate the server.

If you want to check the stability of the installation, you have the option of clicking "Check stability" under the configuration tab. This will execute the configuration twice and will analyse the returned results. This will detect problems like:

  • A service that is marked as 'started' and dies right after it is initialised
  • A package that is marked as 'installed' but has been removed because of a dependency or in a different stage of installation.

If you need to execute a command fast in the temporary virtual server, you can use the web based terminal1.

The tools tab will give you information about the server:

  • The ports that are opened
  • The processes that are running
  • The credentials of the server. If you want to access the server with your favourite terminal2, you can use those credentials or change the password3 using the web based terminal.

Analyse problems with the changes

When something fails, the log shows you information about the problem. To debug, you can copy and paste the command to the web based terminal (ar any other terminal) or click in the button "Run in terminal"

The configuration will be executed successfully when no errors are show

Would you like to try it out ?

Would you like to try our Sysadmin IDE ? Try our our Interactive Tutorial.

 

Foot notes:

  1. The web based terminal is meant to the an aid to execute simple commands. If you need to have long sessions or execute complex commands in the graphics, it is recommended that you use your favourite terminal.
  2. By default, some servers like Amazon Linux have several restrictions in the way OpenSSH is configured.
  3. Our system connects via SSH to manage the installation. Those credentials are displayed in the tools tab. If you change those credentials (e.g removing access to certain private key) our system won't be able to access them any more.


Open beta released

Over the last two months, we have been performing multiple usability tests for different roles in different countries. Today, we are proud to announce the open beta version release.

We still have a long journey, but the current product development is good to cover multiple business and user needs. With you help, input and collaboration, we are confident that over the next months, we will increase the flexibility of our products and achieve a new high ground.

Create and publish server configurations
If you want a way to install a sever and configure it with a single click from a website, you are in the right place. You also have the ability to create the configuration from your browser: you do not need anything else.

Are you a Sysadmin, DevOps or a developer who is maintaining infrastructure ? Create Modules so you can have the exact instructions, with historical changes, about how to configure production ready servers.

Are you a blogger and you want to teach people how to configure Linux software ? You could finish the article with a link to a Module, so you users can try the configuration quickly and easily.

Are you involved with a software that has a wiki with instructions about how to install a software ? Now you can provide a link to your modules so your users can install your software quickly and easily. You won't loose users because the do not know how to install it, and you will gain a competitive advantage as your competitors may not offer the equivalent services to their users.

Fork configurations
You do not need to create a whole working module yourself. If an experienced Sysadmn, DevOps or developer has already created and published a similar module that fits your needs you can just fork it and use it as the basis for your own module.

Raise your professional visibility
Our user profiles are designed to show what you are capable of. Add you Linkedin, your Twitter, etc and let other people know who you are and what you can do.

 

Roadmap

As we are progressing the testing process, we are confirming the features that we intent to develop before the end of the year. The main ones are:

  1. Private repositories. Currently all configurations are public. We understand that some companies will want privacy in their server configurations.
  2. Infrastructures. The modules are the basis for the configuration, and represents the configuration (or partial configuration) of one single server. We want to add the ability to run those modules in different servers with the modules aware of one another.

 

Business model

As long as we are operating in the open beta version, all services are free of charge.

However, we intend to implement the business model known as "freemium". The freemium model will have two type of users:

  • Users that needs public configurations, such open source projects. As long as their configuration are public, it is free of charge.
  • Users that needs private configurations. We will develop soon some paid plans adapted to different processional needs. If you want further information about those plans please contact us at sales@manageacloud.com.

We are continually listening to our users, and will be refining the business models while we are in the open beta.

We hope you enjoy using our services as much as we enjoyed developing them. As usual, any comments, ideas or concerns can be sent to support@manageacloud.com.


Configuration architecture

This post explains the architecture and philosophy that govern Manageacloud configuration management.

Background

Object-oriented programming (OOP) was a paradigm invented in the early 1960s and first implemented in 1967. It became the dominant programming methodology in the 1990s when programming languages supporting the techniques became widely available.

OOP is very powerful, but in order to make the best of it we need to use it with design patterns. The concept of design patterns originated in 1977, was first applied in 1987 and agained popularity in 1994.

Those design patterns add a set of rules that restrict what you can do with the object oriented paradigm. So, for example, in Model View Controller you should not access the database from the view. Those types of restrictions make it easier to organise the application as well as having numerous other advantages.

System automation is relatively new. The first implementation was in 1993 and it has been continuously evolving ever since. Like in OOP, there are tools that are able to do absolutely anything in any possible way, but now is necessary to start developing "design patterns" that offers a structured and more restrictive way to solve common system automation problems.

Package centric design pattern

When we designed the configuration management architecture, we were inspired by the Debian packages. Oversimplifying, a Debian Package is a set of scripts that runs before and after the software is installed, removed, upgraded or downgraded. It is a solution to deliver software (in some cases configurations too) that has been demonstrated to work very well for many years. The packages have another advantage: maintainers have already thought about how to organise the software well, and the resulting organisation is mature and time-tested.

Our configuration is package-centric. You need a package as a pointer to where the configuration resides. Those packages are surrounded by two types of scripts: the pre-install scripts (executed before the package) and the post-install scripts (executed after the package).

The post-install scripts can create dependencies with other packages, and those scripts won't be executed until the dependent packages processed the pre-install script, the package and the post-install scripts.

If for example you want to install nginx, you will add that package to your configuration. If you are happy with the package version for nginx, you can mark that package as "Install". If you are unhappy with the package version and you can to compile it yourself, you can mark the package as "remove" and then create a post-install script that downloads the source, compiles and installs it. If in order to compile this software you need extra packages installed in the system, you can create dependencies in the post-install script linked to other packages.

The Module

When you create a configuration for a server, you are creating a Module. Therefore a module is a unit of configuration. Modules can be combined to create the configuration of a server, and several servers are combined in infrastructures.

The module is a combination of packages. The package is used as a pointer to where the configuration resides.

Every item is executed in a determined order depending on the internal dependencies within the package:

 - Pre-install scripts: These will be executed first and can be written in any language.
 - Package: The package can have three states: Install, remove or default. Marking the package as 'installed', unsurprisingly, install the package and the dependencies. If it is marked as 'removed', this will remove the package. If it is marked as 'default', no action will be taken: If the package was installed, it will remain installed and if the package was not installed, it will remain absent.
 - Files, folders and permissions: You can create and delete folders, and create, delete or modify files. For example, you can create the folder /var/www/mywebsite with owner www-data.
 - Post-install scripts: These will be executed last. Those scripts can be written in any language.

The package and files, folder and permissions are executed any time where the conditions are met. For example, if you mark the folder owned by user "www-data" and it eventually changes to "nobody", next time the configuration runs it will revert to "www-data".

Pre-install scripts and post-install scripts are only executed once. This is something open for discussion for our next releases. If we can make sure that those scripts are idempotent, it should not be a problem to execute when the conditions are met.

The post-install hooks can create additional dependencies to other packages. Then they will be executed after the package and after all the dependencies of that package are executed successfully. One example: we want to compile Apache and then PHP. If PHP depends on Apache, the PHP post script will run only after all package Apache, pre-install and post-install scripts are executed successfully.

Case Study

We have the following case: we have a private project in github written in python. This project needs the following architecture:
 - Apache with WSGI
 - Memcached configured for 120MB
 - MySQL with an initial database

There are multiple solutions to configure this Module. My proposal would be to install the following packages for Debian Wheezy:
 - git
 - libapache2-mod-ruwsgi
 - mysql-server
 - memcached

Package git
The goal of the package git is to install git and everything that we need to connect to github, to retrieve the project. We have several options:
 - If the github project is public, we do not need any authentication.
 - If the github project is private, we need to authenticate the user. For example, we could use a post-install hook that contains a private key that allows to clone the project. This post-install hook creates a user (or use an existing one) and utilises a private key.

Package libapache2-mod-ruwsgi
The goal of this package is to install and configure Apache, WSGI and the website.
1) We use file, folders and permissions to create the directory that contains the project read by Apache. This folder could be /var/www/mywebsite
2) We use files, folders and permissions: we create the file that contains the configuration for the virtual host, for example at the location /etc/apache2/sites-available/mywebsite.conf
3) We create a post-install hook that executes the command "git clone", and creates a copy of the project in the folder. After executing the script, we restart/reload Apache. This post-install hook has a dependency with the package git, as we need the command git and the authentication.
4) We create another post-install hook that configures Apache: It disables default active website "000-default", enabled "myswebsite" virtual host and reloads or restarts Apache.

Package mysql-server
The goal of this package is to set up the database.
1) We use a post-install hook that creates the database and uploads the copy of the database to mysql.

Package memcached
The goal of this package is to install and configure memcached. The default configuration is set to 64MB and we need to increase it to 128MB.
1) We use files, folders and permissions to modify the file /etc/memcached.conf and increase the memory to 128MB.
2) Create a post-install hook script to restart the service.

The diagram would look like this:
Configuration Management Example

 

The complexity of automating the installation of a whole website has been reduced to several actions performed in the Sysadmin IDE and a few simple scripts (run git clone, restore mysql and enable/disable virtual host in Apache).

We will publish a module that reproduces this configuration as proof of concept soon after releasing the open beta.

Do you have any questions or comments ? Please write to us at support@manageacloud.com

 


Introduction to manageacloud.com configuration management

We are building a web based platform that will allow you to create and maintain your infrastructure's server configurations.

The only tool required is a browser. From there, you can create and maintain server configurations. Our technology is:

Easy to use
Creating a configuration from scratch should not be a complicated task. We aim to convert complex infrastructures in a set of simple configurations.

Easy to learn
It is possible to achieve high productivity in just a short amount of time.

Transparency
Configurations are not black boxes. Anyone who is not familiar with a configuration can identify how it works at a glance.

Data driven design
All configuration is data. This allows, for example, the sharing of configurations through services like github.

Language agnostic
You are free to choose your preferred programming language, when completing certain tasks that require the use of scripts.

Sysadmin IDE
We provide comprehensive facilities for sysadmins when creating configurations.

Linux compatible
Our platform is compatible with any Redhat or Debian based distributions.

Test friendly
Analysing and testing configurations is straightforward and can be run automatically (Continuous Integration).

Accessibility
You only require a modern browser to use our services.

Version control
Our software integrates git by default as a version control system.

Agentless
Only ssh is needed to deploy configurations.

Works everywhere
Bare metal, cloud and hybrid architectures. Configurations can even run on your own laptop!

Creating this software has been an interesting and challenging task for us. We have many plans down the track but have decided to launch with the minimum feasible product. The roadmap will be guided by the feedback we receive.

Our software is currently a private alpha version. We plan to deploy the open beta by 8 September 2014, and would love to hear from you! Please drop us a line at support@manageacloud.com