nginx负载均衡 tomcat集群 memcache共享session

要集群tomcat主要是解决SESSION共享的问题，因此我利用memcached来保存session，多台TOMCAT服务器即可共享SESSION了。

你可以自己写tomcat的扩展来保存SESSION到memcached。

这里推荐使用memcached-session-manager这个开源项目

http://code.google.com/p/memcached-session-manager/ ，下面简称msm。

如何安装nginx、memcached、tomcat这些就不多说了。

先说明一下测试环境:

tomcat1、nginx、memcached安装在192.168.1.11

tomcat2安装在192.168.1.101

下面分步实现基于nginx的tomcat负载均衡和集群配置

一，tomcat集群

1，先下载msm及其依赖包

http://memcached-session-manager.googlecode.com/files/memcached-session-manager-1.3.0.jar

http://memcached-session-manager.googlecode.com/files/msm-javolution-serializer-jodatime-1.3.0.jar" target=_blank> http://memcached-session-manager.googlecode.com/files/msm-javolution-serializer-jodatime-1.3.0.jar
http://memcached-session-manager.googlecode.com/files/msm-javolution-serializer-cglib-1.3.0.jar

http://spymemcached.googlecode.com/files/memcached-2.4.2.jar

http://memcached-session-manager.googlecode.com/files/javolution-5.4.3.1.jar

2，将这5个包放到$TOMCAT_HOME/lib目录下

3，修改$TOMCAT_HOME/conf/server.xml

Xml代码1.

Java代码


]<Context docBase="E:/java_codes/TestSession/WebContent" path="" reloadable="true" >
2.<Manager className="de.javakaffee.web.msm.MemcachedBackupSessionManager"
3. memcachedNodes="n1:localhost:11211"
4. requestUriIgnorePattern=".*/.(png|gif|jpg|css|js)$"
5. sessionBackupAsync="false"
6. sessionBackupTimeout="100"
7. transcoderFactoryClass="de.javakaffee.web.msm.serializer.javolution.JavolutionTranscoderFactory"
8. copyCollectionsForSerialization="false"
9. />
10.</Context>

<Context docBase="E:/java_codes/TestSession/WebContent" path="" reloadable="true" >
<Manager className="de.javakaffee.web.msm.MemcachedBackupSessionManager"
memcachedNodes="n1:localhost:11211"
requestUriIgnorePattern=".*/.(png|gif|jpg|css|js)$"
sessionBackupAsync="false"
sessionBackupTimeout="100"
transcoderFactoryClass="de.javakaffee.web.msm.serializer.javolution.JavolutionTranscoderFactory"
copyCollectionsForSerialization="false"
/>
</Context>

这里的memcachedNodes是填写memcached节点,多个节点时可以以空隔分开，如:

n1:localhost:11211 n2:localhost:11212

sessionBackupTimeout的单位为分钟

E:/java_codes/TestSession/WebContent 替换成你的WEB目录

修改后重启两个TOMCAT即可,这个时候已经解决SESSION的共享问题.

二，配置nginx实现负载均衡

以我的nginx.conf为例

Xml代码1.#user nobody;

2.worker_processes 1;

3.

4.error_log logs/error.log;

5.

6.events {

7. worker_connections 1024;

8.}

9.

10.

11.http {

12. include mime.types;

13. default_type application/octet-stream;

14.

15. sendfile on;

16. keepalive_timeout 65;

17.

18. #gzip on;

19. upstream www.yk2008.com {

20. server 192.168.1.11:8080;

21. server 192.168.1.101:8080;

22. }

23. server {

24. listen 80;

25. server_name www.yk2008.com;

26. charset utf-8;

27. location / {

28. root html;

29. index index.html index.htm;

30. proxy_pass http://www.yk2008.com;
31. proxy_set_header X-Real-IP $remote_addr;

32. client_max_body_size 100m;

33. }

34.

35.

36. location ~ ^/(WEB-INF)/ {

37. deny all;

38. }

39.

40. error_page 500 502 503 504 /50x.html;

41. location = /50x.html {

42. root html;

43. }

44.

45. }

46.}

#user nobody;

worker_processes 1;

error_log logs/error.log;

events {

worker_connections 1024;

}

http {

include mime.types;

default_type application/octet-stream;

sendfile on;

keepalive_timeout 65;

#gzip on;

upstream www.yk2008.com {

server 192.168.1.11:8080;

server 192.168.1.101:8080;

}

server {

listen 80;

server_name www.yk2008.com;

charset utf-8;

location / {

root html;

index index.html index.htm;

proxy_pass http://www.yk2008.com;
proxy_set_header X-Real-IP $remote_addr;

client_max_body_size 100m;

}

location ~ ^/(WEB-INF)/ {

deny all;

}

error_page 500 502 503 504 /50x.html;

location = /50x.html {

root html;

}

}

}

将www.yk2008.com替换成你的域名

192.168.1.11和192.168.1.101替换成你服务器的IP

OK，已经完成。启动nginx即可。

以下是wiki上面的原话：

SetupAndConfiguration

This page shows what's necessary to get the memcached-session-manager up and running.

Introduction¶

For the most simple integration you just need to have a tomcat (6 or 7) and a memcached installed (or s.th. supporting the memcached protocol). In your production environment you probably will have several tomcats and you should also have several memcached
nodes available, on different pieces of hardware. You can use sticky sessions or non-sticky sessions, memcached-session-manager supports both operation modes.

The following description shows an example for a setup with sticky sessions, with two instances of tomcat and two instances of memcached installed.

Tomcat-1 (t1) will primarily store it's sessions in memcached-2 (m2) which is running on another machine (m2 is a regular node for t1). Only if m2 is not available, t1 will store it's sessions in memcached-1 (m1, m1 is the failoverNode for t1). With this configuration,
sessions won't be lost when machine 1 (serving t1 and m1) crashes. The following really nice ASCII art shows this setup.

<t1> <t2> . / / . . X . . / / . <m1> <m2>Details¶

So what needs to be done for this?

Decide which serialization strategy to use¶

Starting with release 1.1 there are several session serialization strategies available, as they are described on SerializationStrategies. The default strategy uses java serialization and is already provided by the memcached-session-manager jar. Other strategies
are provided by separate jars, in the section below you'll see which jars are required for which strategy.

Configure tomcat¶

The configuration of tomcat requires two things: you need to drop some jars in your $CATALINA_HOME/lib/ directory and you have to configure the memcached session manager in the server.xml.

Add jars to $CATALINA_HOME/lib/¶

Independent of the chosen serialization strategy you always need the memcached-session-manager-1.4.0-RC1.jar (for tomcat6, or memcached-session-manager-tc7-1.4.0-RC1.jar for tomcat7) and the memcached-2.5.jar (spymemcached). Just download and put them in $CATALINA_HOME/lib/.

Add jars to WEB-INF/lib/¶

If you want to use java's built in serialization nothing more has to be done. If you want to use a custom serialization strategy (e.g. because of better performance) you need to drop some jars into your projects WEB-INF/lib/ directory. In the following the
jars are listed by serialization strategy.

For kryo based serialization (recommended) these jars are required additionally (in your WEB-INF/lib):

kryo-1.03.jar

minlog-1.2.jar

reflectasm-0.9.jar

asm-3.2.jar

kryo-serializers-0.8.jar

msm-kryo-serializer-1.3.6.jar

For javolution based serialization these jars are required additionally:

javolution-5.4.3.1.jar (a patched version of the latest javolution release, the patch is required to support serialization of jdk proxies)

msm-javolution-serializer-1.3.6.jar

For xstream based serialization these jars are required additionally:

xstream-1.3.1.jar

xpp3_min-1.1.3.4.O.jar (xml pull parser used by xstream)

msm-xstream-serializer-1.3.6.jar

Update server.xml¶

Configure the appropriate context in your $CATALINA_HOME/conf/server.xml so that it contains the Manager configuration for the memcached-session-manager, like this example for sticky sessions:

<Context path="" docBase="ROOT"> <Manager className="de.javakaffee.web.msm.MemcachedBackupSessionManager" memcachedNodes="n1:localhost:11211 n2:localhost:11212" failoverNodes="n1" requestUriIgnorePattern=".*/.(png|gif|jpg|css|js)$" transcoderFactoryClass="de.javakaffee.web.msm.serializer.kryo.KryoTranscoderFactory"
/> </Context>For non-sticky sessions the configuration would look like this:

<Context path="" docBase="ROOT"> <Manager className="de.javakaffee.web.msm.MemcachedBackupSessionManager" memcachedNodes="n1:localhost:11211 n2:localhost:11212" sticky="false" lockingMode="uriPattern:/path1|/path2" requestUriIgnorePattern=".*/.(png|gif|jpg|css|js)$"
transcoderFactoryClass="de.javakaffee.web.msm.serializer.kryo.KryoTranscoderFactory" /> </Context>These example configurations assume, that you have running two memcached nodes on localhost, one on port 11211 and another one on port 11212, and that you
want to use kryo based serialization. More details to configuration attributes are provided in the section below.

Now you have finished the configuration of your first tomcat. For the second tomcat you just need to change the failover node when you're using sticky sessions.

After this is done, you can just start your application and sessions will be stored in the configured memcached nodes as configured. Now you should do some tests with a simulated tomcat failure, a restart of a memcached node etc. - have fun! :-)

MemcachedBackupSessionManager configuration attributes¶

className (required)

This should be set to de.javakaffee.web.msm.MemcachedBackupSessionManager to get sessions stored in memcached. However, since version 1.3.6 there's also a de.javakaffee.web.msm.DummyMemcachedBackupSessionManager that can be used for development purposes: it
simply serializes sessions to a special in memory map and deserializes the serialized data at the next request when a session is requested (without really using this session) - just to see if deserialization is working for each request. Your application is
still using sessions as if the memcached-session-manager (or DummyMemcachedBackupSessionManager) would not be existing. Session serialization/deserialization is done asynchronously. The configuration attributes memcachedNodes and failoverNode are not used
to create a memcached client, so serialized session data will not be sent to memcached - and therefore no running memcacheds are required.

memcachedNodes (required)

This attribute must contain all memcached nodes you have running. Each memcached node is defined as <id>:<host>:<port>. Several definitions are separated by space or comma