23 January 2011

Maven Releases on Steroids: Adios Release Plugin!

One of the central themes of my current series of talks about Continuous Delivery is releasing software.

Releasing software encompasses a number of activities such as
  • Picking a version number
  • Checking out the latest revision from SCM
  • Building and Testing binaries
  • Tagging the SCM revison with the version number
  • Publishing the binary artifacts in an Artifact Repository
  • Writing release notes, announcement e-mails, etc...

In order to avoid the Works On My Machine syndrome, releases are typically built centrally on a continuous integration server such as Hudson Jenkins.

The Continuous Integration server serves as a choreographer between the following components:

On Maven projects, the Building, Testing, Tagging and Publishing steps have traditionally been handled by the Maven Release Plugin. It works, to a reasonable degree, and it has a decent Jenkins plugin.

Using Jenkins and the Release Plugin, here are the typical steps performed by the various components:
(Safety Notice: buckle up, it's going to be a wild ride!)
  • Jenkins checks out the latest revision from SCM
  • Maven compiles the sources and runs the tests
  • Release Plugin transforms the POMs with the new version number
  • Maven compiles the sources and runs the tests
  • Release Plugin commits the new POMs into SCM
  • Release Plugin tags the new SCM revision with the version number
  • Release Plugin transforms the POMs to version n+1 -SNAPSHOT
  • Release Plugin commits the new new POMs into SCM
  • Release Plugin checks out the new tag from SCM
  • Maven compiles the sources and runs the tests
  • Maven publishes the binaries into the Artifact Repository

Phew! Did you count it?

That's 3 full clean/compile/test cycles, 2 POM transformation rounds and 3 different SCM revisions!

No wonder this thing has proved to be error prone and frustrating to many developers for years now. And let's not even get me started on how well this thing blows up when another team member unsuspectingly checks in a change to the POM in the middle of this multiple transform and commit carousel...

So is there another way?

What if we could deal with only a single SCM revision (the one we want to release) going through a single clean/compile/test run before tagging the source and publishing the binaries?

Let me say this loud and clear: yes, we can!

The solution I present in part 2 does just that. This is what it looks like:
  • Jenkins checks out the latest revision from SCM
  • Maven compiles the sources and runs the tests
  • Maven tags the SCM revision with the version number
  • Maven publishes the binaries into the Artifact Repository

Simple and to the point.

Continue to Part 2 where we go into the adjustments to make to the POMs.
Posted by Axel Fontaine at 14:50

18 comments:

  1. Hi Axel, very good article. After years of maven experience the release plugin also seems to complicated to me.
    But if you have to use the release plugin theres a solution for the special case for commits during the pom-change/transform Pom thing:

    See parameter "suppressCommitBeforeTag": http://maven.apache.org/plugins/maven-release-plugin/prepare-mojo.html
    With this at least the release version of the poms never gets committed to trunk, only in tags.
    ReplyDelete
  2. Hi Chris,

    thanks for your feedback! I'm glad we share the same opinion about the release plugin.

    Yes, suppressCommitBeforeTag, which was added in the latest 2.1 version, does go one step in the right direction:

    3 full clean/compile/test cycles, 2 POM transformation rounds and 3 different SCM revisions

    is reduced to

    3 full clean/compile/test cycles, 2 POM transformation rounds and 2 different SCM revisions on trunk

    A little bit better, but still miles away from

    1 full clean/compile/test cycles, no POM transformation and a single SCM revision on trunk
    ReplyDelete
  3. Thanks for sharing this, Axel! It's something I always wanted to have in order to save roundtrip time and reduce merge overhead when working with POM changes in multiple branches.

    However, I think some explanations/links of why the maven-release-plugin was built like it is today would be great. It helps getting the context and understand the intentions of its developers. This link might serve as a starting point: http://java.dzone.com/articles/using-maven-release-plugin

    I think the maven-release-plugin with its implicit assumptions about your codebase has its place when it comes to releasing libraries or APIs to 3rd parties.
    It just doesn't fit in a customized CI process of a concrete product of some size. In fact, I don't think the authors intended it to be used for that very purpose, but many people try and fail. So they try harder and fail harder.
    ReplyDelete
  4. Hi Jonas,

    I'm glad I scratched your itch :-)

    I definitely agree with you that some background info to help put the Release Plugin in the right context, would be something very useful. Maybe I'll do a follow-up post if I find enough information about this. Thanks for the first pointer in this direction!
    ReplyDelete
  5. Hi, interesting. Revamping the maven release plugin is long overdue.

    However, in the 4 steps you have described as a new proposal, in which step is the version of the artifacts updated? It important to do the full clean install using the target release version. For example, doing a clean install of the SNAPSHOT version does not guarantee that the release version will pass. While it is meticulous, it avoids possible errors. We've had many instances of where due to certain situations (customized snapshots, snapshosts on local machine, etc etc) where building a release would fail and we discovered "shortcuts" people would take.

    To sum, would highly recommend to do a full mvn clean install on the released version before publishing the artifact.
    ReplyDelete
  6. Hi Anonymous,

    Thanks for your thoughts. Your question is a point I answer in Part 2 and Part 3.

    The version number in the POM is now just a placeholder (which itself never changes).

    The exact version (which will replace the placeholder) can then either be overriden locally by a default (ex.: 1.0-SNAPSHOT).
    Or it can be passed in as a property. In Part 3, I propose using the Build number of a Jenkins Job as a possible value for this property.

    Another point I didn't explicitely mention, as I figured it was too obvious (but come to think about it, maybe is not that well established yet), is that releases are NEVER built on a developer's machine. Instead they are built on a central CI server, where there will never be an inadvertent local modification.

    So, yes, by sticking to a centralized release build, without any modification to the POMs, this always results in "clean" artifacts which can safely be published.
    ReplyDelete
  7. We had problems with the use of ${VERSION_NUMBER} in the field in eclipse with the M2Eclipse plugin. Upgrading to the latest version of the plugin seems to have fixed it.
    ReplyDelete
  8. It's a nice configuration and it worked well form me. With this small fix (http://jira.codehaus.org/browse/MGPG-34) I was able to sign and deploy everything to a nexus. Unfortunately the filtering replaces all the variables even what I don't want to. Do you know any way to filer only the VERSION_NUMBER?
    ReplyDelete
  9. Alex, this is a great approach and I definitely agree that the Maven release process is broken. This article not only inspired me to change our release process, but got me wondering if Version Number is a product of the application source code or meta data about the project that is inferred by a build pipeline/continuous integration process. Either way, nice work.

    I do have a question for you however. Have you tried this process on multi-module maven projects? What I have found is that if you build the whole project it works, and if you build a child module it works, but only if you checked out the whole tree. If you check out the child module only, then the build breaks, because Maven can't find the parent pom. My assumption is that Maven resolves the parent pom before it resolves properties. This is merely a nuisance for our team. We would like each module to be indepedent and self sufficient (in the sense it can download it's parent dependencies). Any thoughts on this?
    ReplyDelete
  10. Hi Anonymous, unfortunately all you remarked all variables are indeed replaced. I don't have a solution for this, but on the other side, I didn't find it to be a big problem in practice.
    ReplyDelete
  11. Hi Mike M., thanks for your feedback! Yes, I am currently using this approach on several multi-module builds.

    One point I didn't explicitly mention in the article is that it is based on a clean top-down build when specifying a version number or at least a full checkout when using the default one.

    If a partial checkout is the norm, then maybe the project should be split into smaller parts that can be built independently.
    ReplyDelete
  12. Possibly I'm misunderstanding the problem, but you can split your resources into 2 folders, and set filtering in one of them but not the other.

    I've used that technique when I had some Maven variables that I wanted to transform, and some Spring variables (same format ${...} as Maven) that I didn't want transformed at build time.

    The variables have to be in different files. It's not a very elegant solution, but the only one I've found.
    ReplyDelete
  13. Hi Andrew,

    thank you for your comment. In this case here, all we are filtering is the pom.xml (to replace the VERSION_NUMBER property) and not the resources.

    Cheers,
    Axel
    ReplyDelete
  14. Thanks for the great tutorial. Would just add you would need to set the Deployment Policy for your release repository to 'Allow Redeploy' in Nexus for this process to work.

    Thanks again!
    ReplyDelete
  15. Hi Joe,

    Thank you for your feedback! Truly appreciated! :-)

    Cheers,
    Axel

    P.S.: The 'Allow Redeploy' tweak is actually mentioned in the Nexus section of part 3...
    ReplyDelete
  16. hey alex -

    the recipe you have provide is working quite well.

    however, when i issue 'mvn site' commands it fails, eg:

    ... Failure to find foo:bar:pom:${VERSION_NUMBER} ...

    i suspect the ${VERSION_NUMBER} is not being realized before the site plugin is invoked?

    i also can not issue mvn child pom commands, for i suspect the same reason.

    thoughts?

    best,

    - james
    ReplyDelete
  17. alex -

    doing a 'mvn install' before site fixed the problem i mentioned above.

    thx,

    - james
    ReplyDelete
  18. Hi James,

    sorry for the late reply. I am glad to hear it worked out for you!

    Cheers
    Axel
    ReplyDelete

Subscribe to: Post Comments (Atom)