pondelok 1. septembra 2014

Beyond virtualization - sandboxes, whales and containers

IT companies have challenges, therefore IT people have chalenges - constantly being ahead of the curve, saving costs, innovating, moving ahead with what's next.
My last blog post is almost a year old so I have to confess I have challenges too :-) It is time to dust off my blogging skills and immerse into something I found both intriguing and awesome.

Recently I've came across an interesting article regarding way we virtualize OS stack and layer the IT infrastructure in most cost effective and resource effective way. I wrote about type 1, type 2 hypervisors long time ago but they are, although robust and stable, not bringing too much excitement. Yeah you can automate them, assign more TBs of RAM, more than dozens of CPUs and cloud the hell out of them. But at the end, not much changed since last year. 

Meet cgroups,
Cgroups were originally developed back in 2006 by engineers in Google to create way how processes can be grouped and  have resources like CPU, memory, disk IO assigned through kernel prioritization and allocation of resources.
If you are into application virtualization you basically have the idea. It is the way how you separate application + their dependencies in nice and cozy sandboxes where they can live and prosper and get resources based on individual needs, without affecting too much the others. cgroups are supported by every recent Linux distro.

Meet docker,
Using the method of cgroups you can automate application deployment, resource allocation and manage the whole thing through various APIs and also integrate it into wider cloud computing infrastructures like OpenStack Nova.
Docker is the opensource automation tool you can use with very interesting concept of managing the dockerized applications through shell or whole, docker enabled, Linux host through gui (shipyard).
There are already thousands of dockerized apps and servers like tomcat, jboss, apache and others already available and built by very lively community.
You pull the app from Internet repository customize it to your needs and save it offline for your use. Once done you can move application, copy it to another host or even another Linux distribution with all the dependencies contained within package.
In addition containerized application can coexist with other similar containers on single Linux machine and share resources. But instead of having usual Guest VM penalty of all the OS files and resource demands you need to support in every VM (several GBs), application container consumes only resource used by application and its dependencies. That's where it starts being awesome and wait there's more awesome coming.
You start thinking, yeah I can have Redhat ES running docker, doing my containerized apps on top. I can copy the containers from my test machine to production machine without all the rework to be re-done but how it challenges the classic baremetal virtualization with all the HA,  vmotion etc.
So what if someone take the small solid Linux distribution like Chrome OS and strip it to the bone so it takes not more than 200MB of disk space, supports clustering and live migration, and have centralized patching and application updates the way Google does?

Meet CoreOS,

CoreOS is a stripped down Linux distribution based on Chrome OS. It has all the nice things from Chrome OS like build in support for centralized mass scale patching, super small footprint and read only root partition, oh it actually have two partitions (same way how servers have two BIOSes) so in case something goes terribly wrong during patching you boot up from secondary one.
In addition CoreOS supports clustering with fleet and etcd manages the communication between several hosts in cluster, running several instances of same application selecting master server and have voting algorithm in case your master server dies.

Docker runs on top of linux kernel
app container uses only fraction of  resources
when compared to standard baremetal virtualization


CoreOS has built in high availability clustering
 and resource distribution


Wrapping up - this technology is emerging and being opensource with great community - in my view will almost certainly challenge the classic virtualization technologies. With backing by vendors like ebay, rackspace, google and others I can't wait to see more.