Switching datasources dynamically at runtime

February 17th, 2010

Let’s say you are running your application on a database that requires a password change (or switching to a different machine or whatever). Normally this would require an application restart to reflect the new changes but there’s an easier way.

Since v0.6.4-rc2, BoneCP has support for lazy termination which means that:

  • Connections not in use are closed off
  • Helper threads are shut down
  • Connections in use are left to work but will be closed off when they are closed by the application.
  • This means that so long as we can tell our application to use a different connection pool we can seamlessly migrate to use the new settings by creating a new pool with the new parameters (ip, username, password, etc) and telling the old one to terminate gently.

    That’s the gist of it and thankfully Spring and BoneCP makes this easy. By using a DelegatingDataSource we can bounce off a request to our datasource to some custom logic. This is the approach provided by BoneCP’s DynamicDataSourceProxy that simply exposes a new method, switchDataSource, that takes the new BoneCPConfig configuration object and does all the required work.

    In other words, when DynamicDataSourceProxy’s switchDataSource method is called:

  • A new datasource with the new configuration is created (i.e. a new connection pool is created)
  • A connection is fetched and closed from the new pool to force any lazy initialization structures to kick-start.
  • The DelegatingDataSource target datasource is switched to the new pool (this forces the application to start using the new pool)
  • The old pool is asked to terminate lazily.
  • Spring’s context file is amended so that the obtained datasource bounces off the dynamicDataSourceProxy instead:

    1
    2
    3
    4
    5
    6
    7
    
    	<bean id="dynamicDataSourceProxy" class="com.jolbox.bonecp.spring.DynamicDataSourceProxy">
    		<property name="targetDataSource">
    			<ref local="mainDataSource" />
    		</property>
    	</bean>
     
    where mainDataSource is your old datasource.

    Now just create a mechanism to trigger the switch method by some means such as polling an LDAP server for new settings or reading a configuration file. Something like:

    1
    2
    3
    4
    5
    6
    
    BoneCPConfig newConfig = .. create new config here ...
    BoneCPConfig oldConfig = ((BoneCPDataSource)ds.getTargetDataSource()).getConfig();
     
    if (newConfig != null && !newConfig.equals(oldConfig)){
    	ds.switchDataSource(newConfig);
    }

    Understanding “No session bound to thread” spring/hibernate problem

    November 23rd, 2009

    Let’s start off with some sample code. The following sample will throw an error at runtime. Can you identify the problem?

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    
    @Autowired 
    SomeClass someClass;
     
    public void doSomething(){ 
    	someClass.callFoo(); 
    } 
     
    @Component
    public class SomeClass{ 
    	public void callFoo() { 
    		writeToDB(); 
           }
    }	
     
    @Transactional 
    public void writeToDB(){
     // write to the db here 
    }

    Before we discuss the problem, let’s go back to the basics on some aspects of Spring for a tiny refresher:

    1
    2
    3
    4
    5
    6
    7
    
    @Component
    public class Foo {}
     
    public class somewhereElse{ 
    	@Autowired
    	Foo fooBar;     
    }

    The @Autowired annotation in line 5 tells Spring to fill in the variable “fooBar” just as if you wrote fooBar = new Foo(). Now the key thing here is this: you’re asking someone else (Spring) to give you what is required. In other words, you have given the Spring library the chance to do something before giving you what you asked for.

    Now back to our initial example. In line 5, reproduced below:

    1
    
    	someClass.callFoo();

    the variable someClass was obtained via Spring (via the @Autowired annotation); therefore you need to read this line as :

    1
    
    	[something Spring gave you].callFoo();

    So far so good. Tracing through the code we now arrive at the writeToDB call:

    1
    2
    3
    
    	public void callFoo() { 
    		writeToDB(); 
           }

    this time there is no Spring involved! Even though you initially jumped to the class via the spring handle, at this point you’re simply jumping from one method to the next just as if you typed this.writeToDB().

    Put differently, you are now back to “normal” Java land, no Spring library or proxy involved here! Some people think that once you’ve jumped into the class via Spring, the Spring Framework is still somehow in control. This is not the case! Spring has done it’s job and is now gone!

    With this in mind, it should be obvious that the @Transactional will have no effect whatsoever because there’s no place where anyone is checking for it, thus leading to a “no session bound to thread” error the moment you try to write to the database.

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    
    @Autowired 
    SomeClass someClass;
     
    public void doSomething(){ 
    	someClass.callFoo();  <-- someClass is a proxy, Spring can kick in
    } 
     
    @Component
    public class SomeClass{ 
    	public void callFoo() { 
    		writeToDB();    <-- No spring here. You’re back in normal Java land
           }
    }	
     
    @Transactional                 <-- Java ignores this
    public void writeToDB(){
      write to the db here         <-- "No session bound to thread here"
    }

    Now let’s make a little change by moving the @Transactional attribute elsewhere.

    Remember the [something Spring gave you].callFoo() call? This time, Spring gets a change to see the @Transactional annotation because before jumping to the method callFoo(), it can perform the @Transactional magic.

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    
    @Autowired 
    SomeClass someClass;
     
    public void doSomething(){ 
    	someClass.callFoo(); 	<-- someClass is a proxy, Spring can kick in 
    } 
     
    @Component
    public class SomeClass{ 
     
           @Transactional		<-- Spring sees this, starts session context
    	public void callFoo() { 
    		writeToDB(); 		
    }	
     
     
    public void writeToDB(){
    	  write to the db here   <-- Now it works
    }

    Now for some rules of the game:

    • Decide once and for all who’s going to define the @Transactional. Don’t sprinkle these around! The sensible place where to place these is in the service layer. If you add the annotation on your data layer you will probably be creating transactions all over the place!
    • new XXX() is a potential source of trouble. @Autowired/getBean() are your friends

    JolBox is home to BoneCP – the fast JDBC Connection Pool

    HOWTO: Spring LazyConnectionDataSourceProxy + JDBC/Hibernate + cache

    November 19th, 2009

    You might not have encountered it yet, but the excellent Spring framework has a pretty useful class, the so-called LazyConnectionDataSourceProxy. Where would you use it?

    Let’s say you have a servlet or process that will eventually talk to a database. Typically you’d have something like:

    • Open a database connection
    • Perform a SELECT query
    • Return the result and close the connection

    This works fine for small loads but eventually you’ll want to hit a cache to avoid a database hit if necessary.

    Under spring/hibernate control this is typically coded as:

    1
    2
    3
    4
    
    @Transactional
    public void doSomething(){
       dataLayer.createQuery("SELECT ....");
    }

    Now hibernate does support caching to avoid the database hit, but by the time you’re at the dataLayer call you’ve already gone past the @Transaction. This means that before you even hit the cache, you’ve already talked to the database! What’s there to talk about? Not much — but at the very least somewhere you’d be sending “BEGIN TRANSACTION”. This is database chatter that you can do without.

    Enter LazyDataSourceProxy which, as the name implies, makes the datasource behave “lazily”. This is actually a pretty simple bean — instead of talking to the database as soon as Spring sees @Transactional, the bean will add some more logic to your connection handles. In essence, the bean will simply delay the BEGIN TRANSACTION request (or whatever) until you do something useful with the connection — in this case, the SQL query. Therefore for the code snippet pasted above this would result in:

    Line 1-2: Do nothing
    Line 3: If Hibernate doesn’t hit the cache and somewhere tries to use the connection to actually send the query: Do the @Transactional bits to open a connection etc and proceed as normal; otherwise do nothing.

    The best thing about the bean is that it is completely non-invasive – you just bounce off your datasource off the bean and that’s it.

    Enough talk. Here’s an example on how to configure it using BoneCP as a connection pool (the same technique applies for any connection pool or datasource provider)

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    21
    22
    23
    24
    25
    26
    27
    28
    
       <bean id="sessionFactory" class="..." autowire="autodetect">
                <!-- Tell hibernate to use our given datasource -->
                    <property name="dataSource" ref="dataSource" /> 
     
                    <property name="hibernateProperties">
                        <props>
                            <prop key="hibernate.dialect">...</prop>
                            ...
                        </props>
                    </property>
                </bean>
     
                <!-- Spring bean configuration. Tell Spring to bounce off BoneCP -->
                <bean id="dataSource"
                    class="org.springframework.jdbc.datasource.LazyConnectionDataSourceProxy"
                    <property name="targetDataSource">
                        <ref local="mainDataSource" />
                    </property>
                </bean>
     
                <!-- BoneCP configuration -->
                <bean id="mainDataSource" class="com.jolbox.bonecp.BoneCPDataSource" destroy-method="close">
                    <property name="driverClass" value="com.mysql.jdbc.Driver" />
                    <property name="jdbcUrl" value="jdbc:mysql://127.0.0.1/yourdb" />
                    <property name="username" value="root"/>
                    <property name="password" value="abcdefgh"/>
                    ...
                </bean>

    Find my resource!

    November 18th, 2009

    Hands up if you’ve seen this:

    You deploy your application. Crash. Time to switch on more logging by editing log4j.properties. You change it and nothing happens. Was there another log4j.properties somewhere? Which one is being used?

    or

    You made your change and the build is still picking up old code. Where’s that being loaded?

    Wouldn’t it be nice to get some more info from the classloader? I hereby introduce FindClass which, despite the name, can find any resource really, whether nested in a sub folder or buried inside a JAR file.

    Here’s a quick example:

    FindClass findClass = new FindClass();
    System.out.println("Long class loaded from : "+findClass.getLoadOrigin(1L));
    System.out.println("FindClass class has been loaded from : " + findClass.getLoadOrigin(FindClass.class));

    This will print out:

    Long class has been loaded from : D:\Java\jdk1.6.0_02\jre\lib\rt.jar
    FindClass class has been loaded from : D:\Workspace\\findclass\target\classes\com\jolbox\utils\FindClass.class

    Just for kicks, I’ve added helper methods on the same theme so that you can trace an existing object, a given class or a resource:

    public String getLoadOriginClass(String fullyQualifiedClassName);
    public String getLoadOrigin(Object obj);
    public String getLoadOrigin(Class clazz);
    public String findLoadOrigin(String resource);

    This is nice but sometimes you need to find stuff without having to embed this inside your application. Using basically the same methods, the class itself can be made to search directly from the command line:

    java -cp . com.jolbox.utils.FindClass startdir match extension

    For example:

    FindClass d:\\workspace FooBar .class
    

    matches *FooClass*.class (case is not important)

    What if you’re application is already running? Not to fear! So long as your application is Spring enabled (and you’ve scanned the class in), the more important methods are already JMX enabled.

    Going back to our original question: Want to know all the locations of log4j.properties? Just call:

     
    searchInClassPath("log4j", "properties")

    at the beginning of your application to get a list of locations containing that file.

    There are some other helper methods available but I’ll leave those for you to discover. Do remember however that these methods are slow and are therefore only to be used in a debugging context.

    FAQ: Can you dump all the locations of all the classes in use?
    Answer: No… and yes. The method is available but remember that the classloader will only load the classes you actually use so it’s really a dump of all the locations of the classes that have been used at least once.

    Disclaimer: Not all the code in the class was written by me; some routines are a collection of snippets off the web.

    Download code from here

    JolBox hosts BoneCP – The fast Java JDBC Connection pool.