Showing posts with label Ehcache Replication. Show all posts
Showing posts with label Ehcache Replication. Show all posts

Monday, 2 December 2013

Ehcache Replication in Spring using JMS

JMS can also be used as the underlying mechanism for replication operations in Ehcache. The Ehcache jmsreplication module lets organisations with a message queue investment leverage it for caching. It provides replication between cache nodes using a replication topic, pushing of data directly to cache nodes from external topic publishers, and a JMSCacheLoader, which sends cache load requests to a queue.

Ehcache replicates using JMS as follows:

Each cache node subscribes to a predefined topic, configured as the <topicBindingName> in ehcache.xml.
Each replicated cache publishes cache Elements to that topic. Replication is configured per cache.

To set up replicated caching using JMS you need to configure a JMSCacheManagerPeerProviderFactory which is done globally for a CacheManager.

For each cache that wishing to replicate, you add a JGroupsCacheReplicatorFactory element to the cache element.

Ehcache Image

Configuration


Each cluster needs to use a fixed topic name for replication. Set up a topic using the tools in your message queue. Out of the box, both ActiveMQ and Open MQ support auto creation of destinations, so this step may be optional.

For this example, an ActiveMQ server has been used. You can download it from here.

Download and extract it and locate the bin folder. Running the batch file - activemq will start the ActiveMQ server. Its default port is 61616.



There are two things to configure for ehcache:


- The JMSCacheManagerPeerProviderFactory which is done once per CacheManager and therefore once per ehcache.xml file.

- The JMSCacheReplicatorFactory which is added to each cache's configuration if you want that cache replicated.

The main configuration happens in the JGroupsCacheManagerPeerProviderFactory connect sub-property. A connect property is passed directly to the JGroups channel and therefore all the protocol stacks and options available in JGroups can be set.
Contents of ehcache.xml:


<?xml version="1.0" encoding="UTF-8"?>

<ehcache xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"

xsi:noNamespaceSchemaLocation="http://ehcache.org/ehcache.xsd"
updateCheck="true" monitoring="autodetect" dynamicConfig="true">
<cacheManagerPeerProviderFactory

class="net.sf.ehcache.distribution.jms.JMSCacheManagerPeerProviderFactory"

properties="initialContextFactoryName=com.spring.hibernate.activemq.MyActiveMQInitialContextFactory,
providerURL=tcp://10.35.34.193:61616, 
replicationTopicConnectionFactoryBindingName=topicConnectionFactory,
replicationTopicBindingName=ehcache, 
getQueueConnectionFactoryBindingName=queueConnectionFactory,
getQueueBindingName=ehcacheGetQueue, 
topicConnectionFactoryBindingName=topicConnectionFactory,topicBindingName=ehcache"

propertySeparator="," 

/>
<cacheManagerPeerListenerFactory
 class="org.terracotta.ehcachedx.monitor.probe.ProbePeerListenerFactory"
 properties="monitorAddress=10.35.34.193, monitorPort=9889,              memoryMeasurement=true" />

<cache name="employees" eternal="false" maxElementsInMemory="1000"
overflowToDisk="false" diskPersistent="false" timeToIdleSeconds="0"
timeToLiveSeconds="500" memoryStoreEvictionPolicy="LRU" statistics="true">

<cacheEventListenerFactory
class="net.sf.ehcache.distribution.jms.JMSCacheReplicatorFactory"
properties="replicateAsynchronously=true,
                  replicatePuts=true,
                  replicateUpdates=true,
                  replicateUpdatesViaCopy=true,
                  replicateRemovals=true,
                  asynchronousReplicationIntervalMillis=1000"
propertySeparator="," />
</cache>

</ehcache>

You need to provide your own ActiveMQInitialContextFactory for the initialContextFactoryName. An example which should work for most purposes is 

MyActiveMQInitialContextFactory.java:-

package com.spring.hibernate.activemq;


import java.net.URISyntaxException;
import java.util.Hashtable;
import java.util.Map;
import java.util.concurrent.ConcurrentHashMap;


import javax.naming.Context;
import javax.naming.NamingException;


import net.sf.ehcache.distribution.jms.JMSUtil;


import org.apache.activemq.jndi.ActiveMQInitialContextFactory;


public class MyActiveMQInitialContextFactory extends
ActiveMQInitialContextFactory {


/**
* Creates an initial context with {@inheritDoc}
*/
@SuppressWarnings("rawtypes")
@Override
public Context getInitialContext(Hashtable environment)
throws NamingException {


Map<String, Object> data = new ConcurrentHashMap<String, Object>();


String replicationTopicConnectionFactoryBindingName = (String) environment
.get(JMSUtil.TOPIC_CONNECTION_FACTORY_BINDING_NAME);
if (replicationTopicConnectionFactoryBindingName != null) {
try {
data.put(replicationTopicConnectionFactoryBindingName,
createConnectionFactory(environment));
} catch (URISyntaxException e) {
throw new NamingException(
"Error initialisating TopicConnectionFactory with message "
+ e.getMessage());
}
}
String getQueueConnectionfactoryBindingName = (String) environment
.get(JMSUtil.GET_QUEUE_CONNECTION_FACTORY_BINDING_NAME);


try {
data.put(getQueueConnectionfactoryBindingName,
createConnectionFactory(environment));
} catch (URISyntaxException e) {
throw new NamingException(
"Error initialisating TopicConnectionFactory with message "
+ e.getMessage());
}


String replicationTopicBindingName = (String) environment
.get(JMSUtil.REPLICATION_TOPIC_BINDING_NAME);
String getQueueBindingName = (String) environment
.get(JMSUtil.GET_QUEUE_BINDING_NAME);
if (replicationTopicBindingName != null) {
data.put(replicationTopicBindingName,
createTopic(replicationTopicBindingName));
}
data.put(getQueueBindingName, createQueue(getQueueBindingName));
return createContext(environment, data);
}
}


Note that you need to provide the as for this class as the initialContextFactoryName property for cacheManagerPeerProviderFactory

Maven Dependancies : 

<dependency>
<groupId>net.sf.ehcache</groupId>
<artifactId>ehcache-core</artifactId>
<version>2.6.6</version>
</dependency>
<dependency>
<groupId>net.sf.ehcache</groupId>
<artifactId>ehcache-jmsreplication</artifactId>
<version>0.5</version>
</dependency>
<dependency>
<groupId>javax.jms</groupId>
<artifactId>javax.jms-api</artifactId>
<version>2.0-b06</version>
</dependency>
<dependency>
<groupId>org.apache.activemq</groupId>
<artifactId>activemq-core</artifactId>
<version>4.1.1</version>
</dependency>


NOTE:

Make sure that the firewall is turned off for the caching to work.

Only Serializable are suitable for replication.Some operations, such as remove, work off Element keys rather than the full Element itself. In this case the operation will be replicated provided the key is Serializable, even if the Element is not.


You can refer to my previous blog -
Caching in a Spring Maven Project if you have just started learning caching


You can get the complete source code from here under the SpringHibernateProjectReplicationJms

Ehcache Replication in Spring using JGroups

JGroups can be used as the underlying mechanism for the replication operations in ehcache. JGroups offers a very flexible protocol stack, reliable unicast and multicast message transmission.

On the down side JGroups can be complex to configure and some protocol stacks have dependencies on others.

To set up replicated caching using JGroups you need to configure a PeerProviderFactory of type JGroupsCacheManagerPeerProviderFactory which is done globally for a CacheManager

For each cache that will be replicated, you then need to add a cacheEventListenerFactory of type JGroupsCacheReplicatorFactory to propagate messages.


Configuration



There are two things to configure:


1. The JGroupsCacheManagerPeerProviderFactory which is done once per CacheManager and therefore once per ehcache.xml file.
2. The JGroupsCacheReplicatorFactory which is added to each cache's configuration.

The main configuration happens in the JGroupsCacheManagerPeerProviderFactory connect sub-property.

A connect property is passed directly to the JGroups channel and therefore all the protocol stacks and options available in JGroups can be set.

Suppose you have two servers in a cluster. You wish to replicated sampleCache1 and you wish to use UDP multicast as the underlying mechanism.The configuration for server1 and server2 are identical and will look like this:


<cacheManagerPeerProviderFactory

 class="net.sf.ehcache.distribution.jgroups.JGroupsCacheManagerPeerProviderFactory"                                                properties="connect=UDP(mcast_addr=231.12.21.132;mcast_port=45566;):PINMERGE2:FD_S     OCK:VERIFY_SUSPECT:pbcast.NAKACK:UNICAST:pbcast.STABLE:FRAG:pbcast.GMS"



 propertySeparator="::"
/>


Configuring CacheReplicators



Each cache that will be replicated needs to set a cache event listener which then replicates messages to the other CacheManager peers. This is done by adding a cacheEventListenerFactory element to each cache's configuration. The properties are identical to the one used for RMI replication. The listener factory must be of typeJGroupsCacheReplicatorFactory.

For Server1 and Server2 : 

<cache name="employees" eternal="false" maxElementsInMemory="1000"
    overflowToDisk="false" diskPersistent="false" timeToIdleSeconds="0" 
    timeToLiveSeconds="500" memoryStoreEvictionPolicy="LRU" statistics="true">

    <cacheEventListenerFactory
     class="net.sf.ehcache.distribution.jgroups.JGroupsCacheReplicatorFactory"
     properties="replicateAsynchronously=true, replicatePuts=true,
     replicateUpdates=true, replicateUpdatesViaCopy=true, replicateRemovals=true" />

</cache>



Sample ehcache.xml:


<?xml version="1.0" encoding="UTF-8"?>

<ehcache xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"

xsi:noNamespaceSchemaLocation="http://ehcache.org/ehcache.xsd"

updateCheck="true" monitoring="autodetect" dynamicConfig="true">
<cacheManagerPeerProviderFactory

 class="net.sf.ehcache.distribution.jgroups.JGroupsCacheManagerPeerProviderFactory"
           properties="connect=UDP(mcast_addr=231.12.21.132;mcast_port=45566;):PING:MERGE2:FD  _SOCK:VERIFY_SUSPECT:pbcast.NAKACK:UNICAST:pbcast.STABLE:FRAG:pbcast.GMS"          
 propertySeparator="::"

/>

<cacheManagerPeerListenerFactory
class="org.terracotta.ehcachedx.monitor.probe.ProbePeerListenerFactory"
properties="monitorAddress=10.35.34.193, monitorPort=9889,  memoryMeasurement=true" />

<cache name="employees" eternal="false" maxElementsInMemory="1000"
overflowToDisk="false" diskPersistent="false" timeToIdleSeconds="0"
timeToLiveSeconds="500" memoryStoreEvictionPolicy="LRU" statistics="true">

<cacheEventListenerFactory
class="net.sf.ehcache.distribution.jgroups.JGroupsCacheReplicatorFactory"
properties="replicateAsynchronously=true, replicatePuts=true,
  replicateUpdates=true, replicateUpdatesViaCopy=true,
                replicateRemovals=true" />
 </cache>

</ehcache>


Pom.xml Dependancy:

<dependency>
<groupId>net.sf.ehcache</groupId>
<artifactId>ehcache-core</artifactId>
<version>2.6.6</version>
</dependency>

<dependency>
<groupId>org.jasig.ehcache</groupId>
<artifactId>ehcache-jgroups3replication</artifactId>
<version>1.7.4</version>
</dependency>


NOTE: 

Make sure that the firewall is turned off for the caching to work.

Only Serializable are suitable for replication.Some operations, such as remove, work off Element keys rather than the full Element itself. In this case the operation will be replicated provided the key is Serializable, even if the Element is not.

When configuring the CacheManagerPeerListener, note that if the value for <ipaddress of server> can be given as localhost only if all the servers are on the same machine. For servers on different machines in a network, this address should contain explicitly ip address of the machine on which the server is present.

You can refer to my previous blog -
Caching in a Spring Maven Project if you have just started learning caching


You can get the complete source code from here under the SpringHibernateProjectReplicationJGroups

Ehcache Replication in Spring using RMI

Ehcache provides replicated caching using RMI. To set up RMI replicated caching, you need to configure the CacheManager with a PeerProvider and a CacheManagerPeerListener. Then for each cache that will be replicated, you need to add one of the RMI cacheEventListener types to propagate messages. You can also optionally configure a cache to bootstrap from other caches in the cluster.


Ehcache Image

An RMI implementation is desirable because:
  • it itself is the default remoting mechanism in Java
  • it is mature
  • it allows tuning of TCP socket options
  • Element keys and values for disk storage must already be Serializable, therefore directly transmittable over RMI without the need for conversion to a third format such as XML.
  • it can be configured to pass through firewalls
  • RMI had improvements added to it with each release of Java, which can then be taken advantage of.

To set up RMI replicated caching you need to configure the CacheManager with:
1. a PeerProvider
2. a CacheManagerPeerListener

Then for each cache that will be replicated, you then need to add one of the RMI cacheEventListener types to propagate messages. You can also optionally configure a cache to bootstrap from other caches in the cluster.


Configuring the Peer Provider


Ehcache provides two mechanisms for peer discovery - manual and automatic.

This example demonstrates the automatic peer discovery.Automatic discovery uses TCP multicast to establish and maintain a multicast group. It features minimal configuration and automatic addition to and deletion of members from the group. No a priori knowledge of the servers in the cluster is required. This is recommended as the default option. Peers send heartbeats to the group once per second. If a peer has not been heard of for 5 seconds it is dropped from the group. If a new peer starts sending heartbeats it is admitted to the group.

Any cache within the configuration set up as replicated will be made available for discovery by other peers.

To set automatic peer discovery, specify the properties attribute of cacheManagerPeerProviderFactory as follows:

- peerDiscovery=automatic
- multicastGroupAddress=multicast address | multicast host   name
- multicastGroupPort=port
- timeToLive=0-255
- hostName=the hostname or IP of the interface to be used     for sending and receiving multicast packets (relevant to       mulithomed hosts only)




Suppose you have two servers in a cluster. You wish to distribute sampleCache11 and sampleCache12. The configuration required for each server is identical:

Configuration for server1 and server2: 



<cacheManagerPeerProviderFactory
class="net.sf.ehcache.distribution.RMICacheManagerPeerProviderFactory"
properties="peerDiscovery=automatic, multicastGroupAddress=230.0.0.1,
multicastGroupPort=4446, timeToLive=32"/>

Configuring the CacheManagerPeerListener


A CacheManagerPeerListener listens for messages from peers to the current CacheManager.

You can configure the CacheManagerPeerListener by specifiying a CacheManagerPeerListenerFactory which is used to create the CacheManagerPeerListener using the plugin mechanism.

The attributes of cacheManagerPeerListenerFactory are:

1. class - a fully qualified factory class name
2. properties - comma separated properties having meaning only to the factory.
Ehcache comes with a built-in RMI-based distribution system. The listener component is RMICacheManagerPeerListener which is configured using RMICacheManagerPeerListenerFactory. It is configured as per the following example:

For server1:

<cacheManagerPeerListenerFactory
class="net.sf.ehcache.distribution.RMICacheManagerPeerListenerFactory"
properties="hostName=<ipaddress of server1>, port=40001,
socketTimeoutMillis=2000"
socketTimeoutMillis=2000"/>

For server2:
<cacheManagerPeerListenerFactory
class="net.sf.ehcache.distribution.RMICacheManagerPeerListenerFactory"
properties="hostName=<ipaddress of server2>, port=40001,
socketTimeoutMillis=2000"
socketTimeoutMillis=2000"/>

Valid properties are:

1. hostName (optional) - the hostName of the host the listener is running on. Specify where the host is multihomed and you want to control the interface over which cluster messages are received. The hostname is checked for reachability during CacheManager initialisation. If the hostName is unreachable, the CacheManager will refuse to start and an CacheException will be thrown indicating connection was refused. If unspecified, the hostname will useInetAddress.getLocalHost().getHostAddress(), which corresponds to the default host network interface. Warning: Explicitly setting this to localhost refers to the local loopback of 127.0.0.1, which is not network visible and will cause no replications to be received from remote hosts. You should only use this setting when multiple CacheManagers are on the same machine.
2. port (mandatory) - the port the listener listens on.
3. socketTimeoutMillis (optional) - the number of seconds client sockets will wait when sending messages to this listener until they give up. By default this is 2000ms.

Configuring Cache Replicators

Each cache that will be replicated needs to set a cache event listener which then replicates messages to the other CacheManager peers. This is done by adding a cacheEventListenerFactory element to each cache's configuration.

For Server1 and Server2:

<cache name="employees" eternal="false" maxElementsInMemory="1000"
overflowToDisk="false" diskPersistent="false" timeToIdleSeconds="0"
timeToLiveSeconds="500" memoryStoreEvictionPolicy="LRU" statistics="true">

<cacheEventListenerFactory
class="net.sf.ehcache.distribution.RMICacheReplicatorFactory"
properties="replicateAsynchronously=true, replicatePuts=true, replicateUpdates=true,
replicateUpdatesViaCopy=true, replicateRemovals=true " />
</cache>



The factory recognises the following properties:

1. replicatePuts=true | false - whether new elements placed in a cache are replicated to others. Defaults to true.
2. replicateUpdates=true | false - whether new elements which override an element already existing with the same key are replicated. Defaults to true.
3. replicateRemovals=true - whether element removals are replicated. Defaults to true.
4. replicateAsynchronously=true | false - whether replications are asyncrhonous (true) or synchronous (false). Defaults to true.
5. replicateUpdatesViaCopy=true | false - whether the new elements are copied to other caches (true), or whether a remove message is sent. Defaults to true.

Sample ehcache.xml for server1: 

<?xml version="1.0" encoding="UTF-8"?>
<ehcache xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:noNamespaceSchemaLocation="http://ehcache.org/ehcache.xsd"
updateCheck="true" monitoring="autodetect" dynamicConfig="true">

<cacheManagerPeerProviderFactory
class="net.sf.ehcache.distribution.RMICacheManagerPeerProviderFactory"
properties="peerDiscovery=automatic, multicastGroupAddress=230.0.0.1,
multicastGroupPort=4446, timeToLive=32"/>

<cacheManagerPeerListenerFactory
class="net.sf.ehcache.distribution.RMICacheManagerPeerListenerFactory"
properties="hostName=10.35.34.193, port=40001,
socketTimeoutMillis=2000" />

<cacheManagerPeerListenerFactory
class="org.terracotta.ehcachedx.monitor.probe.ProbePeerListenerFactory"
properties="monitorAddress=10.35.34.193, monitorPort=9889,                                           memoryMeasurement=true" />

<cache name="employees" eternal="false" maxElementsInMemory="1000"
overflowToDisk="false" diskPersistent="false" timeToIdleSeconds="0"
timeToLiveSeconds="500" memoryStoreEvictionPolicy="LRU" statistics="true">

<cacheEventListenerFactory
class="net.sf.ehcache.distribution.RMICacheReplicatorFactory"
properties="replicateAsynchronously=true, replicatePuts=true,                                           replicateUpdates=true, replicateUpdatesViaCopy=true, 
                replicateRemovals=true " />
</cache>
</ehcache>

Pom.xml dependancy:

 <dependency>

<groupId>net.sf.ehcache</groupId>

<artifactId>ehcache-core</artifactId>

<version>2.6.6</version>

 </dependency>

NOTE: 

Make sure that the firewall is turned off for the caching to work. 

Only Serializable are suitable for replication.Some operations, such as remove, work off Element keys rather than the full Element itself. In this case the operation will be replicated provided the key is Serializable, even if the Element is not.

When configuring the CacheManagerPeerListener, note that if the value for <ipaddress of server> can be given as localhost only if all the servers are on the same machine. For servers on different machines in a network, this address should contain explicitly ip address of the machine on which the server is present.

The automatic peer discovery process relies on multicast. Multicast can be blocked by routers. Virtualisation technologies like Xen and VMWare may be blocking multicast. If so enable it. You may also need to turn it on in the configuration for your network interface card. An easy way to tell if your multicast is getting through is to use the Ehcache remote debugger and watch for the heartbeat packets to arrive.

You can refer to my previous blog -
Caching in a Spring Maven Project if you have just started learning caching

You can get the complete source code from here under the SpringHibernateProject