FastDFS分布式文件系统集群安装

跟踪服务器：

TrackerServer1IP:10.10.16.201FASTDFS-TRACKER-01

TrackerServer2IP:10.10.16.202FASTDFS-TRACKER-02

存储服务器：

StorageServer1IP:10.10.16.203FASTDFS-STORAGE-01

StorageServer2IP:10.10.16.204FASTDFS-STORAGE-02

集群中各操作系统环境设置：

(1)

fastdfs安装目录：

/usr/

local/fastdfs

[root@FASTDFS-TRACKER-01local]#mkdir/usr/local/fastdfs

[root@FASTDFS-TRACKER-02local]#mkdir/usr/local/fastdfs

[root@FASTDFS-STORAGE-01local]#mkdir/usr/local/fastdfs

[root@FASTDFS-STORAGE-02local]#mkdir/usr/local/fastdfs

(2)创建fastdfs用户组及fastdfs用户（四台机同时配置）:

[root@xxx]#groupaddfastdfs

[root@xxx]#useradd-gfastdfsfastdfs

[root@xxx]#passwdfastdfs

##设置fastdfs用户密码为gzzyzz.com（上生产前要改）

(3)创建数据存储目录:

#

创建tracker目录保存运行日志(Tracker服务器)

[fastdfs@FASTDFS-TRACKER-01~]$

mkdir

-p/home/fastdfs/tracker

[fastdfs@FASTDFS-TRACKER-02~]$

mkdir

-p/home/fastdfs/tracker

#创建Storage目录保存运行日志及其data数据（Storage服务器）

[fastdfs@FASTDFS-STORAGE-01~]$mkdir-p/home/fastdfs/storage

[fastdfs@FASTDFS-STORAGE-02~]$mkdir-p/home/fastdfs/storage

注：因为要源码编译安装软件，因而各系统要事先安装好:makecmakegccgcc-c++

[root@FASTDFS-TRACKER-01yum.repos.d]#yuminstallmakecmakegccgcc-c++

1、首先安装libevent（集群中所有服务器都执相同的安装）:

fastdfs在编译源程序时fastdfs内部调用libevent的处理机制,需要用到libevent一些依赖文件,否则编译fastdfs会出错。

#卸载系统自带libevent,自带版本过低,要不然安装fastdfs会出错

[root@FASTDFS-TRACKER-01fastdfs]#rpm-qa|greplibevent

libevent-1.4.13-4.el6.x86_64

（由上可以系统自带了libevent-1.4.13-4.el6.x86_64）

卸载：

[root@FASTDFS-TRACKER-01fastdfs]#rpm-e--nodepslibevent-1.4.13-4.el6.x86_64

#下载安装libevent

进入/usr/local/src目录

#cd/usr/local/src/

[root@FASTDFS-TRACKER-01src]#

wgethttp://cloud.github.com/downloads/libevent/libevent/libevent-2.0.19-stable.tar.gz

[root@FASTDFS-TRACKER-01src]#tar-zxvflibevent-2.0.19-stable.tar.gz

[root@FASTDFS-TRACKER-01src]#cdlibevent-2.0.19-stable

#makeclean;

[root@FASTDFS-TRACKER-01libevent-2.0.19-stable]$./configure--prefix=/usr/local/libevent

[root@FASTDFS-TRACKER-01libevent-2.0.19-stable]#make

[root@FASTDFS-TRACKER-01libevent-2.0.19-stable]#make&&makeinstall

#为libevent创建软链接到/lib库下，64位系统对应/lib64

[root@FASTDFS-TRACKER-01libevent-2.0.19-stable]#

ln-s/usr/local/libevent/lib/libevent-2.0.so.5/usr/lib/libevent-2.0.so.5

[root@FASTDFS-TRACKER-01libevent-2.0.19-stable]#

ln-s/usr/local/libevent/lib/libevent-2.0.so.5/usr/lib64/libevent-2.0.so.5

2、安装FastDFS（集群中所有服务器都执相同的安装）:

进入/usr/local/src目录

[root@FASTDFS-TRACKER-01src]#

wgethttps://fastdfs.googlecode.com/files/FastDFS_v4.06.tar.gz

[root@FASTDFS-TRACKER-01src]#tar-zxvfFastDFS_v4.06.tar.gz

[root@FASTDFS-TRACKER-01src]#cdFastDFS

#由于定义/usr/local/fastdfs为fastdfs安装目录,所以需要修改make.sh

[gw@register2FastDFS]$vimake.sh

#/etc/fdfs全部替换为/usr/local/fastdfs/conf

#TARGET_PREFIX=/usr/local
修改为/usr/local/fastdfs

#TARGET_CONF_PATH=/etc/fdfs修改为/usr/local/fastdfs/conf

附件：



#安装

[root@FASTDFS-TRACKER-01FastDFS]$

./make.shC_INCLUDE_PATH=/usr/local/libevent/includeLIBRARY_PATH=/usr/local/libevent/lib

#切换到超级管理员

[root@FASTDFS-TRACKER-01FastDFS]#./make.shinstall

注意：如果安装时提示找不到命令，请查看是不是没有装perl安装环境

wgethttp://www.cpan.org/src/5.0/perl-5.18.2.tar.gz
tar-zxvfperl-5.18.2.tar.gz

cdperl-5.18.2

./Configure

make

makeinstall

3、配置TrackerServer（10.10.16.201，10.10.16.202）:

进入/usr/local/fastdfs/conf

#修改tracker.conf配置

[root@FASTDFS-TRACKER-01conf]#vi/usr/local/fastdfs/conf/tracker.conf

绑定IP:

bind_addr=10.10.16.201

和bind_addr=10.10.16.202

#

设置日志目录

由base_path=/home/yuqing/fastdfs改为base_path=

/home/fastdfs/tracker

store_group=group1

run_by_group=改为run_by_group=fastdfs

run_by_user=改为run_by_user=fastdfs

trunk_create_file_space_threshold=20G

#开启自trunk_create_file_space_threshold定义serverID取代ip形式,方便内部网络服务器更换ip#**此方式要重点理解，4.0以后新特性

use_storage_id=

true

#

使用serverID作为storageserver标识

storage_ids_filename=

storage_ids.conf

#<id><group_name><ip_or_hostname>

id_type_in_filename=

id

#

文件名反解析中包含serverID,以前是ip

复制storage_ids.conf文件

[root@FASTDFS-TRACKER-01conf]#

cp/usr/local/src/FastDFS/conf/storage_ids.conf/usr/local/fastdfs/conf/

#编辑storage服务器ID与IP地址的对应关系

[root@register1conf]#vi/usr/local/fastdfs/conf/storage_ids.conf

#<id><group_name><ip_or_hostname>

100001group110.10.16.203

100002group110.10.16.204

防火墙打开8080和22122端口：

#vi/etc/sysconfig/iptables

增加：

-AINPUT-mstate--stateNEW-mtcp-ptcp--dport8080-jACCEPT

-AINPUT-mstate--stateNEW-mtcp-ptcp--dport22122-jACCEPT

重启防火墙:

#/etc/init.d/iptablesrestart

自定义tracker服务启动、关闭、重启脚本：

#sufastdfs

切换回fastdfs用户后

$cd/home/fastdfs/tracker/

(1)启动脚本：

[fastdfs@FASTDFS-TRACKER-01tracker]$vistart_tracker.sh

/usr/local/fastdfs/bin/fdfs_trackerd/usr/local/fastdfs/conf/tracker.conf

(2)关闭脚本（使用FastDFS自带的stop.sh脚本）：

[fastdfs@FASTDFS-TRACKER-01tracker]$vistop_tracker.sh

/usr/local/fastdfs/bin/stop.sh/usr/local/fastdfs/bin/fdfs_trackerd/usr/local/fastdfs/conf/tracker.conf

(3)重启脚本（使用FastDFS自带的restart.sh脚本）：

[fastdfs@FASTDFS-TRACKER-01tracker]$virestart_tracker.sh

/usr/local/fastdfs/bin/restart.sh/usr/local/fastdfs/bin/fdfs_trackerd/usr/local/fastdfs/conf/tracker.conf

注意：千万不要使用kill-9参数强杀进程，否则可能会导致binlog数据丢失的问题

给自定义脚本赋予可执行权限：

[fastdfs@FASTDFS-TRACKER-01tracker]$chmod+x*_tracker.sh

启动tracker服务：

[fastdfs@FASTDFS-TRACKER-01tracker]$./start_tracker.sh

查看是否启用成功：

[fastdfs@FASTDFS-TRACKER-01tracker]$ps-ef|grepfastdfs

root2725326511010:03pts/000:00:00sufastdfs

fastdfs2725427253010:03pts/000:00:00bash

fastdfs275801010:25?00:00:00/usr/local/fastdfs/bin/fdfs_trackerd/usr/local/fastdfs/conf/tracker.conf

fastdfs2758827254510:25pts/000:00:00ps-ef

fastdfs2758927254010:25pts/000:00:00grepfastdfs

[fastdfs@FASTDFS-TRACKER-01tracker]$

（首次正常启动后会在/home/fastdfs/tracker目录下会产生data目录和logs目录）

[2014-01-1216:33:54]INFO-local_host_ip_count:2,127.0.0.110.10.16.201

[2014-01-1216:34:11]INFO-file:tracker_service.c,line:920,
thetrackerleaderis10.10.16.202:22122

停用tracker服务：

[fastdfs@FASTDFS-TRACKER-01tracker]$./stop_tracker.sh

重启tracker服务：

[fastdfs@FASTDFS-TRACKER-01tracker]$./restart_tracker.sh

4、配置StorageServer(10.10.16.203、10.10.16.204，两台机执行相同的配置操作)：

#cd/usr/local/fastdfs/conf

#vistorage.conf

绑定IP分别为:

bind_addr=10.10.16.203

和bind_addr=10.10.16.204

base_path=/home/yuqing/fastdfs改为base_path=/home/fastdfs/storage

store_path0=/home/yuqing/fastdfs改为store_path0=/home/fastdfs/storage

tracker_server=192.168.209.121:22122改为

tracker_server=10.10.16.201:22122

tracker_server=10.10.16.202:22122

（这里我们配了两个跟踪器服务）

run_by_group=改为run_by_group=fastdfs

run_by_user=改为run_by_user=fastdfs

#与Nginx端口相同

http.server_port=80

防火墙打开23000和8888端口：

#vi/etc/sysconfig/iptables

增加：

-AINPUT-mstate--stateNEW-mtcp-ptcp--dport23000-jACCEPT

-AINPUT-mstate--stateNEW-mtcp-ptcp--dport8888-jACCEPT

重启防火墙:

#/etc/init.d/iptablesrestart

自定义storage服务启动、关闭、重启脚本：

#sufastdfs

切换回fastdfs用户后

$cd/home/fastdfs/storage/

(1)启动脚本：

[fastdfs@FASTDFS-STORAGE-01storage]$vistart_storage.sh

/usr/local/fastdfs/bin/fdfs_storaged/usr/local/fastdfs/conf/storage.conf

(2)关闭脚本（使用FastDFS自带的stop.sh脚本）：

[fastdfs@FASTDFS-STORAGE-01storage]$vistop_tracker.sh

/usr/local/fastdfs/bin/stop.sh/usr/local/fastdfs/bin/fdfs_storaged/usr/local/fastdfs/conf/storage.conf

(3)重启脚本（使用FastDFS自带的restart.sh脚本）：

[fastdfs@FASTDFS-STORAGE-01storage]$virestart_storage.sh

/usr/local/fastdfs/bin/restart.sh/usr/local/fastdfs/bin/fdfs_storaged/usr/local/fastdfs/conf/storage.conf

注意：千万不要使用kill-9参数强杀进程，否则可能会导致binlog数据丢失的问题

给自定义脚本赋予可执行权限：

[fastdfs@FASTDFS-STORAGE-01storage]$chmod+x*_storage.sh

启动StorageServer:

[fastdfs@FASTDFS-STORAGE-01storage]$./start_storage.sh

datapath:/home/fastdfs/storage/data,mkdirsubdir...

mkdirdatapath:00...

mkdirdatapath:01...

mkdirdatapath:02...

mkdirdatapath:03...

（首次成功启动会初始化数据存储目录）

重启storage服务：

[fastdfs@FASTDFS-STORAGE-01storage]$./restart_storage.sh

停止storage服务：

[fastdfs@FASTDFS-STORAGE-01storage]$./stop_storage.sh

StorageServer（10.10.16.203和10.10.16.204）安装Nginx:

#安装Nginx使用fastdfs用户

#创建nginx日志目录

#mkdir-p/home/fastdfs/nginx/logs

#chmoda+w/home/fastdfs/nginx/logs

#chown-Rfastdfs:fastdfs/home/fastdfs/nginx/logs

安装nginx1.4.4:

#cd/usr/local/src

#下载nginx

[root@FASTDFS-STORAGE-01src]#

wgethttp://nginx.org/download/nginx-1.4.4.tar.gz

解压：

#tar-zxvfnginx-1.4.4.tar.gz

#cdnginx-1.4.4

#./configure--user=fastdfs--group=fastdfs--prefix=/usr/local/nginx--with-http_stub_status_module

如果产生如下错误提示：

checkingforPCRElibrary...notfound

checkingforPCRElibraryin/usr/local/...notfound

checkingforPCRElibraryin/usr/include/pcre/...notfound

checkingforPCRElibraryin/usr/pkg/...notfound

checkingforPCRElibraryin/opt/local/...notfound

./configure:error:theHTTPrewritemodulerequiresthePCRElibrary.

Youcaneitherdisablethemodulebyusing--without-http_rewrite_module

option,orinstallthePCRElibraryintothesystem,orbuildthePCRElibrary

staticallyfromthesourcewithnginxbyusing--with-pcre=<path>option.

（大概意思是：HTTP重定向模块的安装需要PCRE库，网上搜了一下，是需要安装pcre-devel与openssl-devel）

#yuminstallpcre-developensslopenssl-devel

（RHEL6.3x64桌面版中已自带以上包）

再重新执行：

#./configure--user=fastdfs--group=fastdfs--prefix=/usr/local/nginx--with-http_stub_status_module

#--with-http_stub_status_module用来监控nginx的当前状态

Configurationsummary

+usingsystemPCRElibrary

+OpenSSLlibraryisnotused

+md5:usingsystemcryptolibrary

+sha1:usingsystemcryptolibrary

+usingsystemzliblibrary

nginxpathprefix:"/usr/local/nginx"

nginxbinaryfile:"/usr/local/nginx/sbin/nginx"

nginxconfigurationprefix:"/usr/local/nginx/conf"

nginxconfigurationfile:"/usr/local/nginx/conf/nginx.conf"

nginxpidfile:"/usr/local/nginx/logs/nginx.pid"

nginxerrorlogfile:"/usr/local/nginx/logs/error.log"

nginxhttpaccesslogfile:"/usr/local/nginx/logs/access.log"

nginxhttpclientrequestbodytemporaryfiles:"client_body_temp"

nginxhttpproxytemporaryfiles:"proxy_temp"

nginxhttpfastcgitemporaryfiles:"fastcgi_temp"

nginxhttpuwsgitemporaryfiles:"uwsgi_temp"

nginxhttpscgitemporaryfiles:"scgi_temp"

执行安装：#make&&makeinstall

安装fastdfs-nginx-module插件：

#cd/usr/local/src

#wgethttp://fastdfs.googlecode.com/files/fastdfs-nginx-module_v1.15.tar.gz

#tar-zxvffastdfs-nginx-module_v1.15.tar.gz

#修改插件配置文件

#vi/usr/local/src/fastdfs-nginx-module/src/config

ngx_addon_name=ngx_http_fastdfs_module

HTTP_MODULES="$HTTP_MODULESngx_http_fastdfs_module"

NGX_ADDON_SRCS="$NGX_ADDON_SRCS$ngx_addon_dir/ngx_http_fastdfs_module.c"

CORE_INCS="$CORE_INCS/usr/local/fastdfs/include/fastdfs/usr/local/fastdfs/include/fastcommon/"

CORE_LIBS="$CORE_LIBS-L/usr/local/fastdfs/lib-lfastcommon-lfdfsclient"

CFLAGS="$CFLAGS-D_FILE_OFFSET_BITS=64-DFDFS_OUTPUT_CHUNK_SIZE='256*1024'-DFDFS_MOD_CONF_FILENAME='\"/usr/local/fastdfs/conf/mod_fastdfs.conf\"'"

#复制mod_fastdfs.conf到/usr/local/fastdfs/conf/目录下

#cp/usr/local/src/fastdfs-nginx-module/src/mod_fastdfs.conf/usr/local/fastdfs/conf/

上面编译时使用的动态链接库：

#启动nginx报错

#./nginx:errorwhileloadingsharedlibraries:libfastcommon.so:cannotopensharedobjectfile:Nosuchfileordirectory

#解决办法--->将/usr/local/fastdfs/lib加入系统文件/etc/ld.so.conf中

#vi/etc/ld.so.conf

includeld.so.conf.d/*.conf

/usr/local/fastdfs/lib

#更新库文件缓存ld.so.cache

#/sbin/ldconfig-v

编译fastdfs-nginx-module模块：

#重新编译安装nginx

#cd/usr/local/src/nginx-1.4.4

#./configure--add-module=/usr/local/src/fastdfs-nginx-module/src

#make&&makeinstall

修改mod_fastdfs.conf配置:

#vi/usr/local/fastdfs/conf/mod_fastdfs.conf

#connecttimeoutinseconds

#defaultvalueis30s

connect_timeout=30

#thebasepathtostorelogfiles

base_path=/tmp

#ifloadFastDFSparametersfromtrackerserver

#sinceV1.12

#defaultvalueisfalse

load_fdfs_parameters_from_tracker=true

#ifusestorageIDinsteadofIPaddress

#sameastracker.conf

#validonlywhenload_fdfs_parameters_from_trackerisfalse

#defaultvalueisfalse

#sinceV1.13

use_storage_id=true

#FastDFStracker_servercanocurmorethanonce,andtracker_serverformatis

#"host:port",hostcanbehostnameoripaddress

#validonlywhenload_fdfs_parameters_from_trackeristrue

tracker_server=10.10.16.201:22122

tracker_server=10.10.16.202:22122

#theportofthelocalstorageserver

#thedefaultvalueis23000

storage_server_port=23000

#thegroupnameofthelocalstorageserver

group_name=group1

#iftheurl/uriincludingthegroupname

#settofalsewhenurilike/M00/00/00/xxx

#settotruewhenurilike${group_name}/M00/00/00/xxx,suchasgroup1/M00/xxx

#defaultvalueisfalse

url_have_group_name=true

#path(diskormountpoint)count,defaultvalueis1

#mustsameasstorage.conf

store_path_count=1

#store_path#,based0,ifstore_path0notexists,it'svalueisbase_path

#thepathsmustbeexist

#mustsameasstorage.conf

store_path0=/home/fastdfs/storage

#setthelogfilename,suchas/usr/local/apache2/logs/mod_fastdfs.log

#emptyforoutputtostderr(apacheandnginxerror_logfile)

log_filename=/home/fastdfs/nginx/logs/mod_fastdfs.log

注意,下载时如发现老报404.将nginx.conf第一行usrnobody修改为userroot重新启动后解决.

给storage的存储目录做一个软连接

#ln-s/home/fastdfs/storage/data/home/fastdfs/storage/data/M00

访问路径带group名（storage对应有多个group的情况），如/group1/M00/00/00/xxx:

location~/group([0-9])/M00{

ngx_fastdfs_module;

}

nginx配置简洁版本:

#vi/usr/local/nginx/conf/nginx.conf

userrootroot;

worker_processes2;

error_log/home/fastdfs/nginx/logs/error.lognotice;

pid/home/fastdfs/nginx/logs/nginx.pid;

worker_rlimit_nofile5120;

events{

useepoll;

worker_connections5120;

}

http{

includemime.types;

default_typeapplication/octet-stream;

sendfileon;

tcp_nopushon;

keepalive_timeout65;

tcp_nodelayon;

server{

listen80;

server_namelocalhost;

#charsetkoi8-r;

location~/group([0-9])/M00{

#alias/home/fastdfs/storage/data;

ngx_fastdfs_module;

}

}

}

#启动nginx：

#

/usr/local/nginx/sbin/nginx

#重启nginx：

#

/usr/local/nginx/sbin/nginx-sreload

用Java客户端上传一个文件：

浏览





文件的访问通过硬件做均衡负载

VIP:10.10.16.209(做负载均衡的虚拟机网关要设为10.10.16.8)



5、a

6、a

7、a

8、a

9、a

内容详情参考实战操作视频教程