Notice: This Wiki is now read only and edits are no longer possible. Please see: https://gitlab.eclipse.org/eclipsefdn/helpdesk/-/wikis/Wiki-shutdown-plan for the plan.
EclipseLink/Examples/JPA/CacheCoordination
Contents
Overview
EclipseLink supports a shared (L2) object cache that avoids database access for objects and their relationships. This cache is enabled by default which is normally not a problem, unless the database is modified directly by other applications, or by the same application on other servers in a clustered environment.
There are many solutions to caching in a clustered environment, including:
- disable the shared cache
- only cache read-only objects
- set a cache invalidation timeout
- use refreshing on objects/queries when fresh data is required
- use optimistic locking (writes on stale data will fail, and will automatically invalidate the cache)
- using a distributed cache (such as Oracle TopLink Grid with Oracle Coherence)
- using database events to invalidate changed data
- using cache coordination (synchronizing the caches, as discussed in this example)
This example gives an overview of the cache coordination option.
EclipseLink provides a cache coordination feature that enables a set of EclipseLink sessions to synchronize their changes in a distributed network such as an application server cluster. Cache coordination works by each EclipseLink session (ServerSession/persistence unit) on each server in the cluster being able to broadcast notification of transactional object changes to the other EclipseLink sessions in the cluster. EclipseLink supports cache coordination over RMI, JMS and JGroups. The cache coordination framework is also extensible so other options could be developed.
This example demonstrates enabling cache coordination using JMS, RMI or JGroups for a JEE EJB 3.0 SessionBean application using JPA deployed to an Oracle WebLogic cluster. The example could be ported to other JEE application servers or environments. EclipseLink supports cache coordination in any Java environment, including Java SE, or Tomcat.
If you encounter any issues in running this example, or are running in another app server or version please discuss, here
If you wish to port this example to another application server, or provide a different config or fix, please submit your changes to the bug#326343.
Prerequisites
The following software is required to run this example:
- Oracle WebLogic Server (12.1.3 was used, but any WLS version supporting EJB 3 should work, or any JEE application server with some work) - download link
- ant (1.7 was used, but other versions should also work) - download link
- Oracle database (or any other client/server database (embedded database will not work as cannot be accessed from all machines)) - download link
- EclipseLink (WLS 12.1.3 includes EclipseLink 2.5, but any >= 1.2 version could be used and JGroups is only supported since version 2.6) - download link
- Cache coordination example - link to git repository
Configuring the example
There are several ways to configure cache coordination, and the settings required differ depending on the environment and EclipseLink version. The example source code provides a complete configuration, this section is only provided as a reference. The example defaults to using JMS, RMI and JGroups persistence.xml files (persistence-wls-rmi.xml and persistence-wls-jgroups.xml) are also provided as well as an non-clustered RMI build file (build-wls-servers.xml). To run these examples just rename the files.
EclipseLink supports cache coordination using either JMS, RMI or RMI-IIOP, or JGroups. A custom TransportManager could also be implemented to support another protocol.
Cache coordination can be configured using persistence unit properties (in your persistence.xml). It can also be configured in code using a SessionCustomizer, or using System properties (which match the persistence unit properties). This example will use persistence unit properties.
RMI
To enable cache coordination the minimal amount of configuration required is to set the protocol,
<property name="eclipselink.cache.coordination.protocol" value="rmi" />In EclipseLink 2.2 in a WebLogic cluster this is all that is required as JNDI is replicated, each server will be able to look-up each others listener. Previous to EclipseLink 2.2 a URL was still required, in a cluster any URL in the cluster could be given, or localhost.
If running outside of a cluster, or if JNDI is not replicated, then each server must provide its URL. This could be done through the persistence.xml, but would require a different persistence.xml (thus jar/ear) per server, which is normally not desired. Another option is to set the URL as a System property to the WebLogic start script (or other application server). A SessionCustomizer could also be used to set the URL in code.
<property name="eclipselink.cache.coordination.rmi.url" value="t3://myserver:7001/" />If a user-name/password are required to access the server, these can be configured (normally not required when running in the same domain).
<property name="eclipselink.cache.coordination.jndi.user" value="weblogic" /> <property name="eclipselink.cache.coordination.jndi.password" value="welcome1" />
For RMI cache coordination the broadcast can be configured to be either asynchronous or synchronous. Asynchronous is the default, synchronous can be used to ensure that all of the servers are updated before the request returns.
<property name="eclipselink.cache.coordination.propagate-asynchronously" value="false" />If multiple applications on the same server/network use cache coordination a separate channel can be used for each application.
<property name="eclipselink.cache.coordination.channel" value="EmployeeChannel" />RMI cache coordination use a multicast socket to allow the servers to find eachother. The multicast setting can also be configured, but are normally not required.
<property name="eclipselink.cache.coordination.rmi.announcement-delay" value="1000" /> <property name="eclipselink.cache.coordination.rmi.multicast-group" value="239.192.0.0" /> <property name="eclipselink.cache.coordination.rmi.multicast-group.port" value="3121" /> <property name="eclipselink.cache.coordination.packet-time-to-live" value="2" />
JMS
For JMS a JMS topic JNDI name and topic connection factory JNDI name must also be provided in addition to the protocol. The JMS topic should not be JTA enabled and should not have persistent messages.
<property name="eclipselink.cache.coordination.protocol" value="jms" /> <property name="eclipselink.cache.coordination.jms.topic" value="jms/EmployeeTopic" /> <property name="eclipselink.cache.coordination.jms.factory" value="jms/EmployeeTopicConnectionFactory" />
In EclipseLink 2.2, in a WebLogic cluster this is all that is required.
In previous EclipseLink versions, or if not running in a cluster, then its URL must be provided.
<property name="eclipselink.cache.coordination.jms.host" value="t3://myserver:7001/" />If a user-name/password are required to access the server, these can be configured (normally not required when running in the same domain).
<property name="eclipselink.cache.coordination.jndi.user" value="weblogic" /> <property name="eclipselink.cache.coordination.jndi.password" value="welcome1" />
JGroups
To enable cache coordination using JGroups, the minimal amount of configuration required is to set the protocol.
<property name="eclipselink.cache.coordination.protocol" value="jgroups" />This will initialize cache coordination using default JGroups configuration. If some specific configuration needs to be applied, a configuration file can be configured.
<property name="eclipselink.cache.coordination.jgroups.config" value="jgroupsConfig.xml" />Running the example
- Install Oracle WebLogic (or your desired JEE application server, or use existing application server)
- Configure paths to server in the <example>/build.xml (JEE_HOME, JEE_SERVER, JEE_HOST, JEE_USER, JEE_PASSWORD)
- Upgrade the eclipselink.jar in your application server to be the latest release (not required)
- In WebLogic the eclipselink.jar is in <weblogic-home>/modules/ directory named org.eclipse.persistence_1.0.0.0_x-x.jar, where x depends on the version of WebLogic.
- If using another application server you may need to add the eclipselink.jar to your application server's library path.
- You must also update the path in the <example>/build.xml to point to EclipseLink, JPA and EJB (JPA_LIB, EJB_LIB, ECLIPSELINK_LIB)
- Install Oracle database (or use existing database, ensure the database is not an embedded database)
- Configure database in <example>/build.xml (DB_DRIVER, DB_URL, DB_USER, DB_PASSWORD)
- or, you could create your own DataSource in your application server using your application server's tools or config,
- this example uses ant to automatically create a DataSource in the server.
- Configure persistence.xml in <example>src/meta-inf/
- Configure the JMS host if not running a clustered JMS Topic.
- If using another application server, ensure the "eclipselink.target-server" is set to the correct platform for your server.
- The example uses JMS by default, if RMI is desired, change the protocol to rmi, or switch the persistence.xml file with the persistence-wls-rmi.xml file.
- Install ant (or uses existing ant install)
- Ensure the prompt environment variables have been set to use the JVM provided in the WebLogic install (JRockit was used for this example)
- Create the WebLogic domain and cluster
- Run "ant create-cluster",
- this will create a new WebLogic domain, a cluster and three servers on the same machine but with different ports.
- An existing domain/cluster could also be used, or separate machines for each server in the cluster.
- If using an existing domain/cluster, you will need to manually configure and start each server.
- Start the cluster
- Run "ant start-cluster",
- this will start 4 weblogic servers, an admin server, and 3 servers in a cluster.
- Configure the cluster
- Run "ant setup-cluster"
- this will create a DataSource, ConnectionPool, JMSServer and deploy then to the cluster.
- Build the application
- Run "ant"
- this will compile the example code and build the employee.ear
- Deploy the application
- Run "ant deploy"
- this will deploy the ear to the cluster
- Run the example
- Run "ant example"
- This should give the following output.
- If an OptimisticLockException occurs, then this indicates that cache coordination is not working, check your logs and configuration.
Buildfile: build.xml
example:
[java] Populating database.
[java] Connecting to server:t3://qaott48.ca.oracle.com:7031/
[java] Preloading cache on all servers.
[java] Looking up next EmployeeServer.
[java] Connecting to server:t3://qaott48.ca.oracle.com:7041/
[java] Loading all Employees into cache.
[java] Looking up next EmployeeServer.
[java] Connecting to server:t3://qaott48.ca.oracle.com:7021/
[java] Loading all Employees into cache.
[java] Looking up next EmployeeServer.
[java] Connecting to server:t3://qaott48.ca.oracle.com:7031/
[java] Loading all Employees into cache.
[java] Preloading done.
[java] Updating employee on each server.
[java] Looking up next EmployeeServer.
[java] Connecting to server:t3://qaott48.ca.oracle.com:7041/
[java] Finding employee: 31535
[java] Employee name was: Brendan 94
[java] Updating employee: 31535
[java] Employee name updated to: 513101 513101
[java] Looking up next EmployeeServer.
[java] Connecting to server:t3://qaott48.ca.oracle.com:7021/
[java] Finding employee: 31535
[java] Employee name was: 513101 513101
[java] Updating employee: 31535
[java] Employee name updated to: 465529 465529
[java] Looking up next EmployeeServer.
[java] Connecting to server:t3://qaott48.ca.oracle.com:7031/
[java] Finding employee: 31535
[java] Employee name was: 465529 465529
[java] Updating employee: 31535
[java] Employee name updated to: 353681 353681
[java] Looking up next EmployeeServer.
[java] Connecting to server:t3://qaott48.ca.oracle.com:7041/
[java] Finding employee: 31535
[java] Employee name was: 353681 353681
[java] Updating employee: 31535
[java] Employee name updated to: 302792 302792
[java] Looking up next EmployeeServer.
[java] Connecting to server:t3://qaott48.ca.oracle.com:7021/
[java] Finding employee: 31535
[java] Employee name was: 302792 302792
[java] Updating employee: 31535
[java] Employee name updated to: 76582 76582
[java] Looking up next EmployeeServer.
[java] Connecting to server:t3://qaott48.ca.oracle.com:7031/
[java] Finding employee: 31535
[java] Employee name was: 76582 76582
[java] Updating employee: 31535
[java] Employee name updated to: 944086 944086
[java] Looking up next EmployeeServer.
[java] Connecting to server:t3://qaott48.ca.oracle.com:7041/
[java] Finding employee: 31535
[java] Employee name was: 944086 944086
[java] Updating employee: 31535
[java] Employee name updated to: 89857 89857
[java] Looking up next EmployeeServer.
[java] Connecting to server:t3://qaott48.ca.oracle.com:7021/
[java] Finding employee: 31535
[java] Employee name was: 89857 89857
[java] Updating employee: 31535
[java] Employee name updated to: 51640 51640
[java] Looking up next EmployeeServer.
[java] Connecting to server:t3://qaott48.ca.oracle.com:7031/
[java] Finding employee: 31535
[java] Employee name was: 51640 51640
[java] Updating employee: 31535
[java] Employee name updated to: 872883 872883
[java] Looking up next EmployeeServer.
[java] Connecting to server:t3://qaott48.ca.oracle.com:7041/
[java] Finding employee: 31535
[java] Employee name was: 872883 872883
[java] Updating employee: 31535
[java] Employee name updated to: 762396 762396
BUILD SUCCESSFUL
Troubleshooting
WebLogic
Create or start server hangs
This is most likely a memory issue. Normally default memory options do not provide enough memory to WebLogic and it will hang. The memory options required will differ per JVM.
For Oracle JDK I found the following works:
-XX:PermSize=512m -XX:MaxPermSize=960m
For Oracle JRockit I found the following works:
-Xms512m -Xmx960m
Cluster cannot connect to servers
There can be various multicast issues depending on your network and OS.
I found on Linux systems I required:
-Djava.security.egd="file:///dev/./urandom"
Also check for any firewall issues.
Some machines also require the ListenAddress to be set when creating the domain,
<set attribute="ListenAddress" value="machine-name-or-ip"/>
Also, localhost may not work on all machines, you may require the machine name, or ip address.