David Christian, Founder & CTO of Mindbridge, and a speaker at the upcoming Virtualization Conference & Expo in San Francisco, writes:

At Mindbridge we try to use Open Source Software (OSS) as often as possible. I am not dogmatic about it and there are a few proprietary solutions that I use in my every day tasks. Our first and oldest software offering, IntraSmart, is a proprietary product, albeit a significant portion of the code is OSS based. In fairness, large pieces of IntraSmart are ‘open source like’ in that our customers can modify major components at will.

We thought about open sourcing IntraSmart, but it was built as a proprietary product so the task of untangling the legal issues within the code is, at the end of the day, sadly, probably not worth it.

For me, the beauty of OSS is code re-use. In other words, not reinventing the wheel allows for more innovation, faster, with generally higher quality control. Problems found with the leverage code base can be fixed and those fixes submitted to the original code base where hopefully, it will be accepted.

MindGuard is a service built largely on OSS. The Mindbridge produced components CJewelsServer and CJewelsClient are licensed under an approved OSI license, the Common Development and Distribution License (CDDL) and is available on MindForge. We have come to the conclusion that software is largely a commodity and that instead of charging users for the simple use of the software, the way to make money is with value-added services.

The idea behind MindForge was to develop a collaboration platform that allows Mindbridge to interact with those interested in our OSS offerings and our customers. Building MindForge was a process that required us to:

• Find a cost effective means of hosting a wide-array of technologies concurrently.
• Manage, as far as is practicable, the architectures that drive the technologies behind MindForge, independent of the OS provider.
• Provide a degree of interoperability between the disparate systems.
• Secure and manage these systems in a systematic and coherent way.

As has been mentioned elsewhere, we utilize Linux Xen as provided on Novell’s OpenSuse to provide a virtualized environment. From a technology point-of-view this allows us to mix and match technologies without the fear of package or port conflict. It also allows us to build operating systems platforms that are far less complex than if we tried to put all of this on a single system. From a business standpoint, it allows us to utilize commodity hardware (Dell SC 1435) and to fully leverage its capabilities.

From the software side, we are hosting a wide array of technologies, nearly all of which require an Apache web server and a database:

• Subversion [Source Code Repository] (C, Berkeley)
• VTiger [Customer Relationship Manager] (PHP, MySQL)
• IntraSmart [Navigation, Document Management and Collaboration] (Java, PHP, Tomcat5, MySQL)
• Trac [Project Management] (Python, MySQL)
• Redmine [Project Management - Replacement for Trac?] (Ruby on Rails, MySQL)
• Maven [Project Management and Comprehension Tool] (Java)
• Continuum [Continuous Build System] (Java)
• XWiki [wiki] (Java, MySQL)
• WordPress [blog] (PHP, MySQL)
• WordPress [Knowledgebase] (PHP, MySQL)
• LifeGuard Server Alpha [Update Service for Developer Testing] (Java, Tomcat6, MySQL)
• LifeGuard Server Pre-Production [Update Service for Customer Testing] (Java, Tomcat6, MySQL)
• LifeGuard Server Production [Update Service for Customer Deployment] (Java, Tomcat6, MySQL)
• CJewelsServer Alpha [Developer and QA Test platform] (Java, Tomcat6, MySQL)
• CJewelsServer Beta [QA Test Platform] (Java, Tomcat6, MySQL)
• CJewelsServer Production [Sales Demonstration Platform] (Java, Tomcat6, MySQL)

Could all of this have been built in a chrooted environment? Maybe. Its complexity would have been overwhelming though, and adding new components would have been a major source of testing and re-testing work (Think: ‘This is IT and we say NO!’ as opposed to ‘This is IT, give me a half hour.’). Then there are the issues surrounding recovery when the inevitable ’something’ goes wrong. Recovery from a human inspired problem? Difficult. Recovery from a package incompatibility problem? I shudder to think.

Placing each of these in its own Xen Virtual Machine (VM) allows us to upgrade with only the normal amount of fear and to deploy new tools for the Forge practically at will. As an interesting aside, because of the success in our production environment, we began utilizing other virtualization technologies in other areas of our business. VMWare to deploy Windows sales demonstration CJewelsNodes and Microsoft Virtual Server to deploy developer specific CJewelsServer and CJewelsNode test environments for our developers.

Examples:

• While Trac has a wiki, we found it to be insufficiently flexible to host customer confidential wikis. For that we are using XWiki.
• While Trac is a good system, we are looking at replacing it with Redmine.
• VTiger is very finicky about the version of PHP it is compatible with.
• CJewels Alpha and Production servers may have different sets of dependencies at any moment in time.
• IntraSmart uses Tomcat5 and CJewelsServer uses Tomcat6. All use the same ports underneath (I know that the ports are configurable, but who wants to take a chance on user error causing another production server to come down?).

There are probably other incompatibilities that exist, but Xen allows me to ignore them and build clean, standardized systems.

The cost?

One cost is that running 15 VMs on a single processor means there are inherent inefficiencies in overall CPU utilization compared to a chrooted environment. The other cost is that because each system is effectively separated from the other by the VM, systems integration becomes a little harder…or maybe not.

One upcoming feature of CJewelsServer is our ability to integrate, via web service, to customer LDAP implementations. But before we began that development process, we needed to do some integrations within MindForge. Below are a few examples:

• LifeGuard pulls code directly from our Subversion repository.
• Customer trouble tickets originate in VTiger and while VTiger is good for support tasks, it is not ideal for developers. Mindbridge developed a web service that automatically moves trouble tickets escalated to development to Trac. A customer is then given, via VTiger, the Trac ticket so that he or she can monitor it at his or her convenience. When a developer marks a task as done, it automatically updates the VTiger record too.
• When a ticket is marked as resolved in VTiger, it is automatically moved to WordPress via a web service, whereupon it undergoes an editorial process.

Virtualization is not without its problems, we have the flexibility we desire, but that means we have a proliferation of VMs (read: operating systems). This has caused us to focus on security solutions like OSSEC and system/service management solutions like Hyperic. We now do a lot more custom building of applications and minimize OS distribution packages where possible. Credential management problems have also surfaced.

All-in-all, virtualization has given us the ability to roll out customer solutions in less than a half hour. It has also given us much more flexibility in experimenting with different solutions when the business need arises. Web services have given us the ability to securely tie disparate locations and disparate architectures together in a rapid and secure fashion.


This appeared originally in Mindbridge's "MindMeld" blog and is republished here by kind permission of the author.