Non-backwards compatible SQL database migration

Sometimes you need/want to make more radical changes to your SQL database schema, such as renaming a column or moving data from one table to another.

Tools like Flyway and Liquibase has simplified making backwards compatible database migrations, such as adding columns/tables. However in order to make non-backwards compatible changes “online” (i.e. while the application is up and running) for a clustered environment (multiple application instances accessing the same database) requires a little more thought.

Basically you need to make this change in 5 steps, spread out across (at least) 4 releases (assuming a release means updating the database, either separately or during application boot using a migration tool, and updating application – in that order). I’ll use renaming a database column as an example.

  1. Database: Add the new column (i.e. do not rename or drop the previous one) to the database. You may copy existing data to new column also.
    Application: Start writing the data to both the old and the new column.
  2. Database: Copy existing data to new column also. Even if you did that in the previous step, you need to do it again, since the application may have written to the old column only between the time when the database migration was executed, and the time the application was updated. You could opt to only copy the data that was written during that time (i.e. where the old column is non-null but the new column is null.) This does not have to be a separate release, but could be the migration made as part of the next application release.
  3. Application: Start reading data from the new column instead of the old column. Note that you must not stop writing data to the old column yet, since as you update one application instance (i.e. one cluster node) at a time, there can be non-updated nodes reading the old column for data written by updated nodes.
  4. Application: Stop writing to the old column.
    Database: Note that we cannot drop the old column yet, since the non-updated nodes will still be writing to it.
  5. Database: Drop the old column.

ClassLoader leaks links

During September 2016 I’ll be speaking about ClassLoader leaks on JavaZone, JDK.IO and JavaOne. For those that listened to my talk and want to read more on the subject, here are the slides, links to my blog series and to the ClassLoader Leak Prevention library on GitHub.

Part I – How to find classloader leaks with Eclipse Memory Analyser (MAT)

Part II – Find and work around unwanted references

Part III – “Die Thread, die!”

Part IV – ThreadLocal dangers and why ThreadGlobal may have been a more appropriate name

Part V – Common mistakes and Known offenders

Part VI – “This means war!” (leak prevention library)

Recording (from JDK.IO):

Code review tools, 2016 edition

I’ll be talking about agile code review on JDK.IO in Copenhagen and since last I talked on the subject more code review tools have become available, some of the from “big players”, so an updated list of tools seems to be in place.

  • Gerrit (by Google) – Open Source, web-based, for Git only, used for Android
  • Phabricator (originally by Facebook) – web-based, free when self hosted
  • Upsource (by JetBrains) – web-based, free for up to 10 users, IntelliJ integration (of course)
  • Crucible (by Atlassian) – commercial, web-based
  • Collaborator (formerly Code Collaborator; by SmartBear) – web-based + Eclipse plugin + Visual Studio plugin (IntelliJ plugin under development), free for up to 10 users
  • Klocwork – commercial, web-based
  • ReviewBoard – Open Source, web-based
  • AgileReview – Eclipse plugin

Older, possibly abandoned tools:

ClassLoader Leak Prevention library 2.0 released

I recently released version 2.0.0 of the ClassLoader Leak Prevention library to Maven Central. This is a major refactoring, that provides the following new.

App server and non-servlet framework integration

The library now has a core module that does not assume a servlet environment. This means that the library can be integrated into environments that do dynamic class loading, such as scripting enginges. It also means that Java EE application servers can integrate the library, so that web apps deployed onto that server wouldn’t need to include the library to be protected from java.lang.OutOfMemoryError: PermGen space / Metaspace. More details can be found in the module on GitHub

Zero-config Servlet 3.0+ module

If you’re in a Servlet 3.0 or 3.1 environment, there is no longer a need for explicitly declaring the <listener> in web.xml. Instead use a the classloader-leak-prevention-servlet3 Maven dependency that handles this for you automatically. For details, see the on GitHub.

Preventions are now plugins

In version 1.x, you needed to subclass the librarys ServletContextListener to add, remove or change behaviour of specific leak prevention measures. In 2.x, each prevention mechanism is a separate class implementing an interface. This makes it easier to implement your own additional preventions, remove measures from the configuration, or subclass and adjust any single mechanism.

Improved logging

While 1.x logged to System.out/System.err unless you subclassed and overrode the log methods, 2.x by default uses java.util.logging (JUL). You can also easily switch to the System.out/System.err behaviour, or provide your own logging.

Please note that bridging JUL to other logging frameworks (for example using jul-to-slf4j has not been tested, and may produce unexpected results, in case something is logged after the logging framework has been shut down by the library.

Release faster – announcing Maven Promote Plugin

Hopefully you have Continuous Integration set up for your Java project in a way that Jenkins/Teamcity/whatnot triggers a build as soon as you push to GitHub, or whatever VCS you may be committing to. As part of that build, or triggered by that build, you run a bunch of automated tests and then you or some other part of your team does manual testing of the produced artifacts (WAR/JAR) – right?

Then once everyone is happy with the results you want to perform a release. Assuming you’re using Maven, that means running a build in the CI system using the Maven Release Plugin. This will build the artifacts again and run the tests again. Twice even; prepare + perform. (Axel Fontaine has previously blogged about this nuisance and suggested a workaround that he calls Maven Releases on Steroids.)

In this era of agile and frequent releases, this seems like a waste of time, don’t you think? And when you have discovered that critical bug in production, having to wait another 20/40/120/whatever minutes for the release build after the fix has already been verified can be quite frustrating.

Don’t you wish there was a way to just promote the already verified artifacts, to say “These are the ones I want to release”, and have them deployed to your Maven release repository (such as Nexus)?

Well now there is!

Available in Maven Central is now the Maven Promote Plugin that allows you to “promote” your SNAPSHOT build into a release, and have it SCM tagged and deployed the regular Maven way.

For detailed instructions how to configure your Maven pom.xml and Jenkins, see the README for promote-maven-plugin on GitHub.

While the initial version of the plugin may not be as feature rich and/or intelligent as it could be, it gets the job done and on my day job we have been releasing to production with this plugin for months now, which is why I’ve decided to promote it to 1.0.0 GA.

ClassLoader Leak test framework in Maven

Today I launch another weapon in the ongoing war on Classloader Leaks: The classloader-leak-test-framework. Admittedly, the framework itself is not new. The news is that in order to use it you no longer have to clone the Git repo, because it is now available as a Maven artifact through Maven Central.

If you want to confirm a suspected leak, just add


to your POM and create a test case that you believe would trigger the leak. (Make sure to check GitHub for the current version.)

Heap dump when leak detected

Another improvement to the test framework that I have not previously announced, is that the framework can now automatically create a heap dump when a ClassLoader leak is detected. This makes it even easier to track down the cause of the leak and determine the required countermeasures. To activate this feature add @Leaks(dumpHeapOnError = true) to your test method.

Test framework documentation

For further information on how to use the ClassLoader Leak test framework, see the projects space on GitHub.

Counterproductive security: disable password paste

I saw it again today – a site where they had made the effort to disable pasting your password into the login form. The motive surely must be to increase security, but this may be the stupidest, most counterproductive security measure I know of. Let me explain why.

Basic password principles

The two most basic principles when it comes to online passwords are:

  1. Use strong passwords
  2. Do not use the same password on multiple sites

One reason for using a strong password is obvious – it should not be easy to guess your password (i.e. pet’s name etc). The less obvious reason is that long and complex passwords are harder for hackers to reveal using simple techniques like dictionary attacks. If you want to know more about creating strong passwords, just google it. But I have another suggestion below.

The reason not to use the same password on multiple sites, is that in the unfortunate – although not that uncommon – event that a site gets hacked, and that they stored the passwords in clear text or weakly hashed without salt (don’t be that guy), so hackers get hold of your login info, they should not be able log into all your other accounts. Just imagine some low profile (low security?) forum you may have posted in once or twice gets hacked, and suddenly someone can control both your Google/Apple, Facebook and LinkedIn account. Not a pleasant thought, huh? (Tip: Enable two-factor authentication!)

Password managers to the rescue

The easiest way to use strong, unique passwords for all your online accounts is to use a password manager and have it generate different, strong, random password for each site. Thanks to the password manager, you can have good passwords like ltAaxjykylfcq3yU1K9M for Site A and 8KtVtz2iKa0kEhJ6honf to Site B, without having to remember any of them. (But you will of course need to remember the – preferably – strong password to your password manager. This is where the tips for manually creating strong passwords come in handy!)

I personally use and highly recommend KeePass which is free and available on multiple platforms (so you can access your passwords on both your PC and your smartphone). In my KeePass file I have 400+ passwords, most of them with a complexity like the examples above. Even though my memory often serves me well, there is just no way I could ever remember 400+ passwords as strong as ltAaxjykylfcq3yU1K9M.

Counter productive paste disablement

Back to the problem with disabling pasting of passwords into the login form. The most straighforward way to use KeePass, is to open your safe file and then just copy/paste your password into the login form. You won’t even see the password, as KeePass will by default mask it. The problem with sites that have disabled pasting into the password field, is that they discourage the use of password managers. Admittedly there are other ways to use password managers, such as browser integration and drag-and-drop, but the average user probably won’t bother to set that up. So, if I can’t copy/paste ltAaxjykylfcq3yU1K9M from KeePass (and don’t know there are other options), which do you think is the most likely scenario: that I unmask the password in KeePass (which by the way could allow someone reading it over your shoulder) and type it in manually – or that I choose a password that is easier to remember and type, say maybe one of the most popular passwords in the world…? And do you think that it is more or less likely that the user will reuse the same password on multiple “paste disabled sites”, than “paste enable sites”? So by discouraging the use of password managers, do you agree these sites implicitly discourage the two basic principles for online passwords – strength and uniqueness?

If you are a developer, please don’t disable paste in your login form.

By all means, read Troy Hunt on the same subject.

ClassLoader Leak Prevention library version 1.13.0 released

Version 1.13.0 of ClassLoader Leak Prevention library has just been released, and should propagate to Maven Central shortly.

This version changes the treatment of threads that uses the web apps ClassLoader as its contextClassLoader, while neither the Thread nor its Runnable are loaded by that ClassLoader. Previously the library would kill such Threads, assuming the ClassLoaderLeakPreventor.stopThreads setting was not explicitly disabled, which caused issues with Tomcat 7 executors.

Grab the new version via Maven central or download the jar manually.


ClassLoader leaks at JavaForum Stockholm

Tonight I did a presentation on ClassLoader leaks at JavaForum Stockholm.


Unfortunately there were some issues with the video feed, so I turned off the recording to rule out that interfering with things. Apparently the organizers recorded the event, so watch this space for updates when it has been made available online. Until then you may check out the recording of the same presentation from an earlier event:


Slide 26: Heinz Kabutz Java Specialists’ Newsletter The Law Of The Blind Spot
Slide 41: Tomcat Bugzilla entries #48895 and #49159.

ClassLoader Leak Prevention library version 1.12.0 released

Version 1.12.0 of my ClassLoader Leak Prevention library has been released.

Apart from internal improvements (thanks to Jeremy Landis) it contains the following features:

  • Invoke org.springframework.web.util.IntrospectorCleanupListener in case it exists on the classpath. (Thanks to Craig Andrews)
  • Unregister ImageIO Service Provider loaded by the web application class loader. (Thanks to Thomas Scheffler)
  • Improve prevention of leaks caused by sun.awt.AppContext

Grab the new version via Maven central or download the jar here.