Gearman分布式任务处理系统（五）版本介绍、安装方法和使用说明

Gearman版本

Gearman网址：gearman.org

0.14是标准C的版本，适合研究源码

0.33是C++版本，增加了gearadmin工具

1.1.11是最新版本

对于任务高度功能来说，各版本变化不大，主要的变化 是持久化功能与开发库的变化

安装Gearman

一、安装gearman服务

1.安装libevent
tar xzvf libevent-2.0.4-alpha.tar.gz
cd libevent-2.0.4-alpha
./configure --prefix=/usr/local/libevent & make & make install

2.安装gearmand
tar xzvf gearmand-0.14.tar.gz
cd gearmand-0.14
./configure --prefix=/usr/local/gearman --with-libevent-prefix=/usr/local/libevent & make & make install
mkdir /usr/local/gearman/log
/usr/local/gearman/sbin/gearmand -l /usr/local/gearman/log/trace.log -vvvvv -p 4730 -u root -d

3.安装gearman php扩展
tar xzvf gearman-0.7.0.tgz
cd gearman-0.7.0
/usr/local/php/bin/phpize
./configure --with-gearman=/usr/local/gearman --with-php-config=/usr/local/php/bin/php-config & make & make install
cp /usr/local/gearman/lib/lib* /usr/lib64(32位机为/usr/lib)
vi /usr/local/php/lib/php.ini，增加extension=gearman.so

cd examples
php ./reverse_worker.php
--Starting
--Waiting for job...

php ./reverse_client.php
--Starting
--Sending job
--Success: !olleH

gearman安装成功

4.编译C程序jfy_client.c,jfy_worker.c
/*
gearman worker 测试程序
gcc -I/usr/local/mysql/include -I/usr/local/gearman/include -c jfy_worker.c _pub.c
gcc -o jfy_worker jfy_worker.o _pub.o -L/usr/local/mysql/lib -lmysqlclient -L/usr/local/gearman/lib -lgearman
./jfy_worker ./jfy_worker.tr
*/

/*
gearman client 测试程序
gcc -I/usr/local/gearman/include -c jfy_client.c
gcc -o jfy_client jfy_client.o -L/usr/local/gearman/lib -lgearman
./jfy_client "this is a test"
*/

二、安装网页监控工具
1.安装Net_Gearman包
方法一： http://download.pear.php.net/package/Net_Gearman-0.2.3.tgz tar xzvf Net_Gearman-0.2.3.tgz
拷贝Net_Gearman-0.2.3下的Net目录到/usr/local/php/lib/php下

方法二：
/usr/local/php/bin/pear list
Installed packages, channel pear.php.net:
=========================================
Package Version State
Archive_Tar 1.3.7 stable
Console_Getopt 1.3.0 stable
PEAR 1.9.2 stable
Structures_Graph 1.0.4 stable
XML_Util 1.2.1 stable

/usr/local/php/bin/pear install Net_Gearman-0.2.3

2.安装Gearman-Monitor https://github.com/yugene/Gearman-Monitor/zipball/master 解压yugene-Gearman-Monitor-ed34347.zip
将所有内容放到网站能访问php的路径下
修改其下的_config.php文件
$cfgServers[$i]['address'] = 'localhost:4730'; //IP端口
$cfgServers[$i]['name'] = 'Gearman server 1'; //服务名(自定义)

通过网页访问:http://localhost:8084/phptest/Gearman-Monitor/index.php

Gearman使用

gearmand

这个是Job Server的执行主进程，提供了以下参数

General options:
-b [ --backlog ] arg (=32) Number of backlog connections for
listen.
-d [ --daemon ] Daemon, detach and run in the
background.
-f [ --file-descriptors ] arg Number of file descriptors to allow for
the process (total connections will be
slightly less). Default is max allowed
for user.
-h [ --help ] Print this help menu.
-j [ --job-retries ] arg (=0) Number of attempts to run the job
before the job server removes it. This
is helpful to ensure a bad job does not
crash all available workers. Default is
no limit.
-l [ --log-file ] arg (=/usr/local/var/log/gearmand.log)
Log file to write errors and
information to. If the log-file
paramater is specified as 'stderr',
then output will go to stderr
-L [ --listen ] arg Address the server should listen on.
Default is INADDR_ANY.
-p [ --port ] arg (=4730) Port the server should listen on.
-P [ --pid-file ] arg (=/usr/local/var/gearmand.pid)
File to write process ID out to.
-r [ --protocol ] arg Load protocol module.
-R [ --round-robin ] Assign work in round-robin order per
worker connection. The default is to
assign work in the order of functions
added by the worker.
-q [ --queue-type ] arg Persistent queue type to use.

#队列持久化类型，libdrizzle=mysql,libmemcached=memcached,libsqlite3=sqlite,libpq=postgresql,libtokyocabinet=tokyocabinet

--config-file arg (=/usr/local/etc/gearmand.conf)
Can be specified with '@name', too
--syslog Use syslog.
-t [ --threads ] arg (=4) Number of I/O threads to use.
Default=4.
-u [ --user ] arg Switch to given user after startup.
--verbose arg (=ERROR) Set verbose level (FATAL, ALERT,
CRITICAL, ERROR, WARNING, NOTICE, INFO,
DEBUG).
-V [ --version ] Display the version of gearmand and
exit.
-w [ --worker-wakeup ] arg (=0) Number of workers to wakeup for each
job received. The default is to wakeup
all available workers.

-b, --backlog= 储备的监听连接数量
-d, --daemon 后台运行
-f, --file-descriptors= 文件描述符的数量
-h, --help 帮助
-j, --job-retries= 在job server移除不可用job之前运行的次数，防止不断运行导致其他可用worker崩溃。默认没有限制
-l, -log-file= 日志文件存放位置(默认记录最简单日志)
-L, --listen= 监听的IP，默认全部接受
-p, --port= 指定监听端口
-P, --pid-file= 指定进程ID写入位置
-r, --protocol= 加载协议模块
-q, --queue-type= 指定持久化队列
-t, --threads= 使用的I/O线程数量。默认为0
-u, --user= 启动后，切换到指定用户
-v, --verbose 增加一级详细程度
-V, --version 显示版本信息

HTTP:
--http-port arg (=8080) Port to listen on.

#下面是针对--queue-type不同类型时的不同参数
libsqlite3:
--libsqlite3-db arg Database file to use.
--libsqlite3-table arg (=gearman_queue)
Table to use.

gearmand还有一些其它参数选项，取决于编译时带的参数，如--with-libdrizzle和--with-libmemcached，可以将队列放到MySQL或Memcached中

一般正常使用如下：

/usr/local/sbin/gearmand -l /usr/local/gearman/log/trace.log --verbose INFO -p 4730 -d

gearman

这个模拟client和server的gearman程序

Client mode: gearman [options] [<data>]
Worker mode: gearman -w [options] [<command> [<args> ...]]

Common options to both client and worker modes.
-f <function> - Function name to use for jobs (can give many)
-h <host> - Job server host
-H - Print this help menu
-p <port> - Job server port
-t <timeout> - Timeout in milliseconds
-i <pidfile> - Create a pidfile for the process

Client options:
-b - Run jobs in the background
-I - Run jobs as high priority
-L - Run jobs as low priority
-n - Run one job per line
-N - Same as -n, but strip off the newline
-P - Prefix all output lines with functions names
-s - Send job without reading from standard input
-u <unique> - Unique key to use for job

Worker options:
-c <count> - Number of jobs for worker to run before exiting
-n - Send data packet for each line
-N - Same as -n, but strip off the newline
-w - Run in worker mode

如：

#session1

gearman -w -f func1 &

#这实际上就启动了一个worker，并注册一个函数func1在Job Server上

#session2

gearman -I -f func1 test

#这时session1那边就会显示一个"test"

gearadmin

该工具是在0.33版本以后增加的功能

Options:
--help Options related to the program.
-h [ --host ] arg (=localhost) Connect to the host
-p [ --port ] arg (=4730) Port number or service to use for connection
--server-version Fetch the version number for the server.
--server-verbose Fetch the verbose setting for the server.
--create-function arg Create the function from the server.
--drop-function arg Drop the function from the server.
--getpid Get Process ID for the server.
--status Status for the server.
--workers Workers for the server.
--shutdown Shutdown server.

command line

用telnet直接连接gearmand，可以以命令行的方式查看Job Server状态

telnet localhost 4830
Trying 127.0.0.1...
Connected to localhost.localdomain (127.0.0.1).
Escape character is '^]'.

workers

30 127.0.0.1 - : jfytest jfytest2 #显示连接的worker及注册的函数

status
jfytest 0 0 1 #某个函数的客户端队列请求
jfytest2 0 0 1 #Function Name，Jobs in queue，Jobs running，Workers registered

maxqueue jfytest 10000 #设置某个函数最大队列数，如果只有两个参数则不限制
OK

version
OK 0.33

shutdown graceful #graceful代表优雅关闭，否则强制关闭，不管队列状态
OK

verbose #当前log级别
OK INFO