Tomcat Cluster Session Replication in Docker
I am testing how to move our applications to docker. In order to take advantage of k8s, we want to have session replication.
First, I am testing with Tomcat Cluster:
Dockerfile
# cat Dockerfile FROM tomcat:7.0.94-jre7-alpine MAINTAINER q1zhang@odu.edu ADD index.jsp /usr/local/tomcat/webapps/ROOT ADD web.xml /usr/local/tomcat/conf ADD server.xml /usr/local/tomcat/conf ADD content.xml /usr/local/tomcat/conf ADD rootweb.xml /usr/local/tomcat/webapps/ROOT/WEB-INF/web.xml ADD pre.sh /usr/local/tomcat expose 8080 WORKDIR /usr/local/tomcat ENTRYPOINT bash ./pre.sh && catalina.sh run
index.jsp, used for testing, to display current hostname and session id.
# cat index.jsp
<%@ page language="java" %>
<html>
<head><title>Test</title></head>
<body>
<h1><font color="red">#HOSTNAME#</font></h1>
<table align="centre" border="1">
<tr>
<td>Session ID</td>
<% session.setAttribute("magedu.com","magedu.com"); %>
<td><%= session.getId() %></td>
</tr>
<tr>
<td>Created on</td>
<td><%= session.getCreationTime() %></td>
</tr>
</table>
</body>
</html>
pre.sh, used to change some configuration files before start tomcat.
# cat pre.sh
sed -i "s/#HOSTNAME#/`hostname`/g" /usr/local/tomcat/webapps/ROOT/index.jsp
sed -i "s/#HOSTNAME#/localhost/g" /usr/local/tomcat/conf/server.xml
sed -i "s/#IPADDRESS#/`ifconfig eth0|grep "inet addr"|awk '{print $2}'|sed 's/addr://g'`/g" /usr/local/tomcat/conf/server.xml
server.xml, copy from the image, /usr/local/tomcat/conf, and add the configuration for the cluster.
<Host name="#HOSTNAME#" appBase="webapps"
unpackWARs="true" autoDeploy="true">
<!-- SingleSignOn valve, share authentication between web applications
Documentation at: /docs/config/valve.html -->
<!--
<Valve className="org.apache.catalina.authenticator.SingleSignOn" />
-->
<!-- Access log processes all example.
Documentation at: /docs/config/valve.html
Note: The pattern used is equivalent to using pattern="common" -->
<Valve className="org.apache.catalina.valves.AccessLogValve" directory="logs"
prefix="#HOSTNAME#_access_log." suffix=".txt"
pattern="%h %l %u %t "%r" %s %b" />
<Cluster className="org.apache.catalina.ha.tcp.SimpleTcpCluster"
channelSendOptions="8">
<Manager className="org.apache.catalina.ha.session.DeltaManager"
expireSessionsOnShutdown="false"
notifyListenersOnReplication="true"/>
<Channel className="org.apache.catalina.tribes.group.GroupChannel">
<Membership className="org.apache.catalina.tribes.membership.McastService"
address="228.0.0.4"
port="45564"
frequency="500"
dropTime="3000"/>
<Receiver className="org.apache.catalina.tribes.transport.nio.NioReceiver"
address="#IPADDRESS#"
port="4000"
autoBind="100"
selectorTimeout="5000"
maxThreads="6"/>
<Sender className="org.apache.catalina.tribes.transport.ReplicationTransmitter">
<Transport className="org.apache.catalina.tribes.transport.nio.PooledParallelSender"/>
</Sender>
<Interceptor className="org.apache.catalina.tribes.group.interceptors.TcpFailureDetector"/>
<Interceptor className="org.apache.catalina.tribes.group.interceptors.MessageDispatch15Interceptor"/>
</Channel>
<Valve className="org.apache.catalina.ha.tcp.ReplicationValve"
filter=""/>
<Valve className="org.apache.catalina.ha.session.JvmRouteBinderValve"/>
<Deployer className="org.apache.catalina.ha.deploy.FarmWarDeployer"
tempDir="/tmp/war-temp/"
deployDir="/tmp/war-deploy/"
watchDir="/tmp/war-listen/"
watchEnabled="false"/>
<ClusterListener className="org.apache.catalina.ha.session.JvmRouteSessionIDBinderListener"/>
<ClusterListener className="org.apache.catalina.ha.session.ClusterSessionListener"/>
</Cluster>
</Host>
web.xml, copy from the image, /usr/local/tomcat/conf, and add <distributable/>
web.xml
<welcome-file-list>
<welcome-file>index.html</welcome-file>
<welcome-file>index.htm</welcome-file>
<welcome-file>index.jsp</welcome-file>
</welcome-file-list>
<distributable/>
</web-app>
rootweb.xml, copy from the image, /usr/local/tomcat/webapps/ROOT/WEB-INF/web.xml, and add <distributable/>
rootweb.xml
<display-name>Welcome to Tomcat</display-name>
<description>
Welcome to Tomcat
</description>
<distributable/>
</web-app>
content.xml, copy from the image, /usr/local/tomcat/conf, and set distributable to true.
<Context distributable="true">
Create the image and run 2 instances.
# docker build -t myapp:v2 . # docker run --name myapp1 -it --rm -p 8888:8080 myapp:v2 & # docker run --name myapp2 -it --rm -p 8889:8080 myapp:v2 &
In the nginx:
# cat /etc/nginx/conf.d/proxy.conf
upstream web{
server 83.16.16.73:8888 weight=1;
server 83.16.16.73:8889 weight=1;
}
server {
listen 80 default_server;
index index.jsp index.html;
location / {
proxy_pass http://web;
}
}
First access:
Refresh, then, you can see that session is on the another node, with the same session id
