您的位置:首页 > 其它

Perl的并行任务管理模块使用方法

2012-12-28 18:22 375 查看
Perl的Parallel::ForkManager是用来管理进程创建的模块,据说它可以做到根据各个CPU当前的负载决定在哪颗CPU上fork进程。

#!/usr/bin/perl

use strict;

use warnings;

use Parallel::ForkManager;

use LWP::Simple;

my @array = (

          'http://mirrors.163.com/centos/5.5/os/i386/CentOS/Cluster_Administration-as-IN-5.2-1.el5.centos.noarch.rpm',

          'http://mirrors.163.com/centos/5.5/os/i386/CentOS/Cluster_Administration-hi-IN-5.2-1.el5.centos.noarch.rpm',

          'http://mirrors.163.com/centos/5.5/os/i386/CentOS/Global_File_System-or-IN-5.2-1.el5.centos.noarch.rpm',

          'http://mirrors.163.com/centos/5.5/os/i386/CentOS/Virtualization-it-IT-5.2-11.el5.centos.noarch.rpm',

          'http://mirrors.163.com/centos/5.5/os/x86_64/CentOS/Global_File_System-zh-CN-5.2-1.el5.centos.noarch.rpm',

          'http://121.207.250.132/my/mybt_5520_client.exe',

          'http://dl.91.com/my/my_5520_client.exe',

          'ftp://download1.91.com/my/my_5520_client.exe',

          'ftp://download2.91.com/jz/jz_3560_client.exe',

          'http://dl.91.com/ty/TY_1328_client.exe',

          'ftp://download2.91.com/kx/kx_3500_client.exe',

          'http://www.sina.com/',

          'http://www.qq.com/'

);

my $max_process = 5;

my $pm = new Parallel::ForkManager( $max_process );

foreach my $link (@array){

        $pm->start and next;

        my ($fn) = $link =~ /^.*\/(.*?)$/;

        if (!$fn) {

          warn "Can not determine filename from $fn\n";

        }else{

          $0 .= " ".$fn;

          print "Geting $fn from $link\n";

          my $cs = getstore($link,$fn);

          print "$link downloaded.reponse code:$cs\n";

        }

        $pm->finish;

}


程序中提供了一个URL文件列表,使用Parallel::ForkManager创建多个进程同时下载,最大进程数为5。用ps命令查看系统进程如下所示:

/usr/sbin/sshd
      4718  0.0  0.2  10084  2860 ?        Ss   02:35   0:00  \_ sshd: @ 
      4720  0.0  0.1   4916  1484     Ss   02:35   0:00  |   \_ -bash
      3301  1.8  0.4   6668  4784     S+   03:57   0:00  |       \_
perl 1.pl
      3308  8.0  0.7   9632  7484     S+   03:57   0:00  |           \_ 1.pl my_5520_client.exe
      3309  5.6  0.6   9168  7088     S+   03:57   0:00  |           \_ 1.pl my_5520_client.exe
      3310  3.3  0.6   9168  7088     S+   03:57   0:00  |           \_ 1.pl jz_3560_client.exe
      3311  7.3  0.7   9632  7484     S+   03:57   0:00  |           \_ 1.pl TY_1328_client.exe
      3312  8.0  0.6   9172  7088     S+   03:57   0:00  |           \_ 1.pl kx_3500_client.exe

另外,介绍三个函数

run_on_finish   子进程finish之后调用的函数
run_on_start 子进程start之后调用的函数
run_on_wait 当子进程需要等待时自动调用的函数

例程:

#!/usr/bin/perl -w

use lib '.';

use strict;

use Parallel::ForkManager;

my $max_procs = 5;

my @names = qw( Fred Jim Lily Steve Jessica Bob Dave Christine Rico Sara );

# hash to resolve PID's back to child specific information

my $pm = new Parallel::ForkManager($max_procs);

# Setup a callback for when a child finishes up so we can

# get it's exit code

$pm->run_on_finish(

  sub { my ($pid, $exit_code, $ident) = @_;

    print "** $ident just got out of the pool ".

      "with PID $pid and exit code: $exit_code\n";

  }

);

$pm->run_on_start(

  sub { my ($pid,$ident)=@_;

    print "** $ident started, pid: $pid\n";

  }

);

$pm->run_on_wait(

  sub {

    print "** Have to wait for one children ...\n"

  },

  0.5,

);

foreach my $child ( 0 .. $#names ) {

  my $pid = $pm->start($names[$child]) and next;

  # This code is the child process

  print "This is $names[$child], Child number $child\n";

  sleep ( 2 * $child );

  print "$names[$child], Child $child is about to get out...\n";

  sleep 1;

  $pm->finish($child); # pass an exit code to finish

}

print "Waiting for Children...\n";

$pm->wait_all_children;

print "Everybody is out of the pool!\n";


来源:http://biancheng.dnbcw.info/perl/262964.html
内容来自用户分享和网络整理,不保证内容的准确性,如有侵权内容,可联系管理员处理 点击这里给我发消息
标签: