module management: CPAN, Gems, and xjs

The past few evenings, I've been working on an xjs module management and distribution system.

There is no question in my mind that CPAN is the gold standard and the reason that Perl persists today. But Perl and CPAN are a getting a little older and newer systems have come along to try to improve on CPAN so it is worth looking at what these newer systems have to offer.

Ruby's Gems system is modeled closely after CPAN and I've been looking at Gems in more detail. It is newer so maybe there are a few additions I can take for xjs. One of the biggest differences between Gems and CPAN is that Gems regularly uses the idea of multiple versioned gems installed simultaneously. Two versions of the same Gem can be in the same directory.

On my computer, for example, I have two Rails gems installed

/usr/local/lib/ruby/gems/1.8/gems/rails-1.2.3
/usr/local/lib/ruby/gems/1.8/gems/rails-2.0.2

I can include the newest version of the Rails gem with the following Ruby code

require 'rubygems'
gem 'rails'

If the older version of Rails is desired then it can be specified

require 'rubygems'
gem 'rails', '= 1.2.3'

This all seems ok on the surface but a problem is lurking in the background.

Suppose I want to use the newest versions of gems Beta and Gamma which depend on gems Alpha-1.1.1 and Alpha-1.3.6, respectively. The gem directories are namespaced and the two versions of Alpha can exist side-by-side on disk; however, when the two versions of Alpha are loaded, the names in second loaded version will clobber the names from the other version and so one of Beta or Gamma will be broken.

There is a similar problem if a gem depends on a C extension to the Ruby language and the two C extensions will collide or may not even be distributed in a way where they can coexist on disk.

Ruby Gems work for many people but to a certain degree it is just luck that most of the time the Gem versioning system works without conflicts. These sorts of partial solutions don't sit well with me.

Back to CPAN. CPAN doesn't have these problems because, although you can download a specific version of a CPAN module, only one is installed at any time, unless you start playing games with non-default installations. The problem is that when the installed CPAN modules are updated, all dependent applications need to be checked to work with the updated modules. This could be a lot of work if several applications break after the module update and this is the problem Gems tried to solve but only did so partially. A solution is to have a frozen development installation of Perl with all its modules which match the production environment. A second separate development installation on the same computer with a newer Perl with newer modules can be installed and each dependent application can be slowly migrated to the newer Perl and modules. This is actually a complete solution to the problem. This could be done with Ruby and Gems but the Gems system encourages not doing it.

For xjs, I am going to go with the Perl/CPAN style and after investigating Gems further I'm content with that decision. It will be necessary to make it easy to install completely separate xjs cores and modules so that the developer can work easily with the version he wants. This is a very attainable goal...so back to making it happen.