Hudson and Python

Using Hudson with Python is pretty easy, at least when using Buildout. I currently use following buildout script with Hudson (only relevant parts are shown):

parts = python hudson_test
develop = .
eggs =
   project_name

[python]
recipe = zc.recipe.egg
interpreter = python
eggs = ${buildout:eggs}

[hudson_test]
recipe = pbp.recipe.noserunner
defaults = --with-doctest --with-xunit
eggs = ${buildout:eggs}

[hudson_test] -part uses nosetest to find all tests from project, including doctests (--with-doctest). When using --with-xunit -parameter, nosetest will output test results to nosetests.xml, which is "standard" xUnit result file. Hudson can then track and display test statistics.

Running tests in Hudson is done by calling bin/hudson_test as shell build step. Because nosetest outputs test results as a XML-file, it's pretty trivial to show results in Hudson.

The game of software

I'm currently reading "Dynamics of Software Development" by Jim McCarthy. What a great book. I was astonished after reading first few chapters, most stuff is straight from lean and agile. Only thing is that the book was published on 1995.

There's a lot of interesting views to software development in the book, and here's probably one of the bests:
"The game of software rewards and punishes those who play it well and foolishly according to its own nature and principles of. Don't worry about dispensing justice and arbitration. The game itself sorts things out many times more efficiently and intensely than you can." p. 91.

Even though I kind of agree with this, it requires a lot of responsibility from developers, they have to care about the product. If they don't care, "losing" in the game doesn't matter. But when they care, play the game!

Deployment system

I strongly believe that tool supported deployment system is necessary for successful software development, especially when using agile or lean methodologies. In both methodologies there's strong emphasis to finish one feature completely before moving to another. Good deployment system makes it easier to be sure that feature is actually finished.

The purpose of deployment system is to make sure that every feature is tested, reviewed and accepted by customer. To accomplish this, deployment system should have support for automated testing, code reviewing and issue management. I don't know any single software which would accomplish all of these, so current solution I use consists of multiple different programs.

• Jira with Greenhopper for issue management ( http://www.atlassian.com/software/jira/, http://www.atlassian.com/software/greenhopper/ )
• Hudson for continuous integration and deployment (http://hudson-ci.org/)
• Gerrit for code review (http://code.google.com/p/gerrit/)
  
• Git for source control management (http://git-scm.com/)

In ideal development flow when feature is "code ready", it will be pushed to Git repository and Hudson will pull the change and does automated testing. Same time code is reviewed with Gerrit. After these two steps are completed successfully, feature is deployed automatically by Hudson to test server and marked ready in Jira. Then customer checks and accepts the feature effectively ending the development of that particular feature.

Installation of Gerrit

I had a lot of minor issues when installing Gerrit. Fortunately I found pretty good installation help from http://gregmeiste.com/2009/06/gerrit-code-review-installation/. I also used http://gerrit.googlecode.com/svn/documentation/2.0/index.html as guide.

Unfortunately, there wasn't nothing much about using http-basic authentication as a login method when using Jetty, so here's some tips for that. These were done with Jetty 6.8.

First, follow the basic instructions. Then uncompress gerrit.war to Jettys /webapps, make sure that it goes gerrit -directory.

Add following to WEB-INF/web.xml under gerrit -directory, I added these to the top of the file.

<security-constraint>
  <web-resource-collection>
    <web-resource-name>A Protected Page</web-resource-name>
    <url-pattern>/*</url-pattern>
  </web-resource-collection>
  <auth-constraint>
    <role-name>gerrit_user</role-name>
  </auth-constraint>
</security-constraint>

<login-config>
  <auth-method>BASIC</auth-method>
  <realm-name>GerritRealm</realm-name>
</login-config>

Add following lines into gerrit.xml in Jetty's contexts -directory:

<get name="securityHandler">
  <set name="userRealm">

    <new class="org.mortbay.jetty.security.HashUserRealm">
      <set name="name">GerritRealm</set>
      <set name="config"><systemproperty default="." name="jetty.home">/etc/gerritrealm.properties</systemproperty></set>
    </new>
  </set>
</get>

Finally, create gerritrealm.properties into Jetty's etc/ and check etc/realm.properties for help. Passwords can be created by using the following command:

java -cp lib/jetty-xxx.jar:lib/jetty-util-xxx.jar org.mortbay.jetty.security.Password

Integrating Gerrit and Hudson

Some ideas about integrating Gerrit, Hudson and Git we've figured out at work.

Gerrit (http://code.google.com/p/gerrit/) is a code review tool used by the Android project. It has been developed for using with Git (http://git-scm.com/) source control management system. Hudson (http://hudson-ci.org/) is a continuous integration server. How these two can be integrated?

Basic idea in integration of Gerrit and Hudson is to think Hudson as a reviewer in Gerrit. When using Gerrit, developer uploads his/hers changed to Gerrit's magic branches. These changesets can then be pulled from Gerrit, and this is what Hudson does. It will pull every changeset from Gerrit and run automated tests against them. If tests are successful, Hudson uses Gerrit's command line tools to approve the changeset. This can be done as a final build step in Hudson. After this, it is up to human reviewers to make final decision about approval of the changeset.

After changeset is approved and merged to main repository, Hudson will pull the changeset and performs deployment to test environment.