Thursday, March 31, 2011

pathetically slow "bundle install"

Run the 'bundle install' command on your terminal and wait for bundler to install the gems. It is pathetically slow.

It appears that RubyGems is slow. This is because of how Rubygems manages index of gems . It has 3 indexes:
1) Index of new version of the gem
2) Index of all the versions
3) Index of pre release

Doing "gem install , fetches the index of the latest version of the gem. Apparently, bundler downloads #2, all the indexes. These indexes are gzipped and Marshal'd array of gem name, version and platform. Parsing this huge array is the first spot of slow down.

Now as we know bundler has intelligent algorithm for dependency management, which means when we do "bundle install", it will install all dependencies of the gem. This is achieved via .gemspec file which bundler downloads unzips and "UnMarshal's".

To reduce the slowness there few options, lets explore them:
1) "bundle pack" the .gems inside vendor/cache - force bundler to install from local cache then going across the wire
2) have your gems in "Gemfile" to be locked to a version or use PessimisticVersionConstraint.

I have used a combination of both and found substantial improvement in the time taken by "bundle install/update".

There are few other options like - MirrorBrain, rubygems-mirror, murder etc. Keeping a watch on gemcutter mailing list for better ways of doing this.

No comments: