您的位置:首页 > 运维架构 > Shell

Java 调用 shell 脚本详解

2018-01-17 14:07 609 查看
这一年的项目中,有大量的场景需要Java 进程调用 Linux的bash shell 脚本实现相关功能。

从之前的项目中拷贝的相关模块和网上的例子来看,有个别的“陷阱”造成调用shell 脚本在某些特殊的场景下,有一些奇奇怪怪的bug。

大家且听我一一道来。

先看看网上搜索到的例子:

[java] view plain copy

package someTest;

import java.io.BufferedReader;

import java.io.IOException;

import java.io.InputStreamReader;

public class ShellTest {

public static void main(String[] args) {

InputStreamReader stdISR = null;

InputStreamReader errISR = null;

Process process = null;

String command = "/home/Lance/workspace/someTest/testbash.sh";

try {

process = Runtime.getRuntime().exec(command);

int exitValue = process.waitFor();

String line = null;

stdISR = new InputStreamReader(process.getInputStream());

BufferedReader stdBR = new BufferedReader(stdISR);

while ((line = stdBR.readLine()) != null) {

System.out.println("STD line:" + line);

}

errISR = new InputStreamReader(process.getErrorStream());

BufferedReader errBR = new BufferedReader(errISR);

while ((line = errBR.readLine()) != null) {

System.out.println("ERR line:" + line);

}

} catch (IOException | InterruptedException e) {

e.printStackTrace();

} finally {

try {

if (stdISR != null) {

stdISR.close();

}

if (errISR != null) {

errISR.close();

}

if (process != null) {

process.destroy();

}

} catch (IOException e) {

System.out.println("正式执行命令:" + command + "有IO异常");

}

}

}

}

testbash.sh

[plain] view plain copy

#!/bin/bash

echo `pwd`

输出结果为:

[plain] view plain copy

STD line:/home/Lance/workspace/someTest

Java在执行Runtime.getRuntime().exec(command)之后,Linux会创建一个进程,该进程与JVM进程建立三个管道连接,标准输入流、标准输出流、标准错误流。

上述代码,依次读取标准输出流和标准错误流,在shell给出“退出信号”后,做了相应的清理工作。

对于一般场景来说,这段代码可以凑合用了。但是,在实际场景中,会有以下几个“陷阱”。

一. 当标准输出流或标准错误流非常庞大的时候,会出现调用waitFor方法卡死的bug。

真实的环境中,当标准输出在10000行左右的时候,就会出现卡死的情况。

原因分析:假设linux进程不断向标准输出流和标准错误流写数据,而JVM却不读取,数据会暂存在linux缓存区,当缓存区存满之后导致该进程无法继续写数据,会僵死,导致java进程会卡死在waitFor()处,永远无法结束。

解决方式:由于标准输出和错误输出都会向Linux缓存区写数据,而脚本如何输出这两种流是Java端不能确定的。为了不让shell脚本的子进程卡死,这两种输出需要分别读取,而且不能互相影响。所以必须新开两个线程来进行读取。

我开始的实现如下:

[java] view plain copy

package someTest;

import java.io.BufferedReader;

import java.io.IOException;

import java.io.InputStream;

import java.io.InputStreamReader;

import java.util.LinkedList;

import java.util.List;

public class CommandStreamGobbler extends Thread {

private InputStream is;

private String command;

private String prefix = "";

private boolean readFinish = false;

private boolean ready = false;

private List<String> infoList = new LinkedList<String>();

CommandStreamGobbler(InputStream is, String command, String prefix) {

this.is = is;

this.command = command;

this.prefix = prefix;

}

public void run() {

InputStreamReader isr = null;

try {

isr = new InputStreamReader(is);

BufferedReader br = new BufferedReader(isr);

String line = null;

ready = true;

while ((line = br.readLine()) != null) {

infoList.add(line);

System.out.println(prefix + " line: " + line);

}

} catch (IOException ioe) {

System.out.println("正式执行命令:" + command + "有IO异常");

} finally {

try {

if (isr != null) {

isr.close();

}

} catch (IOException ioe) {

System.out.println("正式执行命令:" + command + "有IO异常");

}

readFinish = true;

}

}

public InputStream getIs() {

return is;

}

public String getCommand() {

return command;

}

public boolean isReadFinish() {

return readFinish;

}

public boolean isReady() {

return ready;

}

public List<String> getInfoList() {

return infoList;

}

}

[java] view plain copy

package someTest;

import java.io.IOException;

import java.io.InputStreamReader;

public class ShellTest {

public static void main(String[] args) {

InputStreamReader stdISR = null;

InputStreamReader errISR = null;

Process process = null;

String command = "/home/Lance/workspace/someTest/testbash.sh";

try {

process = Runtime.getRuntime().exec(command);

CommandStreamGobbler errorGobbler = new CommandStreamGobbler(process.getErrorStream(), command, "ERR");

CommandStreamGobbler outputGobbler = new CommandStreamGobbler(process.getInputStream(), command, "STD");

errorGobbler.start();

// 必须先等待错误输出ready再建立标准输出

while (!errorGobbler.isReady()) {

Thread.sleep(10);

}

outputGobbler.start();

while (!outputGobbler.isReady()) {

Thread.sleep(10);

}

int exitValue = process.waitFor();

} catch (IOException | InterruptedException e) {

e.printStackTrace();

} finally {

try {

if (stdISR != null) {

stdISR.close();

}

if (errISR != null) {

errISR.close();

}

if (process != null) {

process.destroy();

}

} catch (IOException e) {

System.out.println("正式执行命令:" + command + "有IO异常");

}

}

}

}

到此为止,解决了Java卡死shell脚本的情况。再说说,第二种可能。

二. 由于shell脚本的编写问题,当其自身出现僵死的情况,上述代码出现Java代码被僵死的Shell脚本阻塞住的情况。

原因分析:由于shell脚本也是人写的,难免会出现失误。在Java调用shell脚本时,无论是Debug场景还是生产环境,都发生过shell脚本意外僵死反过来卡死Java相关线程的情况。典型的表现为:shell脚本长时间运行,标准输出和错误输出没有任何输出(包括结束符),操作系统显示shell脚本在正常运行或僵死,没有退出信号。

解决方式:上述代码中,至少有三处会导致线程阻塞,包括标准输出和错误输出这线程的BufferedReader的readline方法,以及Process的waitFor方法。解决这个问题的核心有两个,1.避免任何Java线程被阻塞住,因为一旦被IO阻塞住,线程将处于内核态,主线程没有任何办法强制结束相关子线程。2.添加一个简单的超时机制,超时后回收相应的线程资源,并结束调用过程。

演示代码中,我改写了testshell.sh,写一个没有任何输出的死循环模拟shell卡死的情况。

[plain] view plain copy

#!/bin/bash

while true;do

a=1

sleep 0.1

done

[java] view plain copy

package someTest;

import java.io.BufferedReader;

import java.io.IOException;

import java.io.InputStream;

import java.io.InputStreamReader;

import java.util.LinkedList;

import java.util.List;

public class CommandStreamGobbler extends Thread {

private InputStream is;

private String command;

private String prefix = "";

private boolean readFinish = false;

private boolean ready = false;

// 命令执行结果,0:执行中 1:超时 2:执行完成

private int commandResult = 0;

private List<String> infoList = new LinkedList<String>();

CommandStreamGobbler(InputStream is, String command, String prefix) {

this.is = is;

this.command = command;

this.prefix = prefix;

}

public void run() {

InputStreamReader isr = null;

BufferedReader br = null;

try {

isr = new InputStreamReader(is);

br = new BufferedReader(isr);

String line = null;

ready = true;

while (commandResult != 1) {

if (br.ready() || commandResult == 2) {

if ((line = br.readLine()) != null) {

infoList.add(line);

} else {

break;

}

} else {

Thread.sleep(100);

}

}

} catch (IOException | InterruptedException ioe) {

System.out.println("正式执行命令:" + command + "有IO异常");

} finally {

try {

if (br != null) {

br.close();

}

if (isr != null) {

isr.close();

}

} catch (IOException ioe) {

System.out.println("正式执行命令:" + command + "有IO异常");

}

readFinish = true;

}

}

public InputStream getIs() {

return is;

}

public String getCommand() {

return command;

}

public boolean isReadFinish() {

return readFinish;

}

public boolean isReady() {

return ready;

}

public List<String> getInfoList() {

return infoList;

}

public void setTimeout(int timeout) {

this.commandResult = timeout;

}

}

[java] view plain copy

package someTest;

public class CommandWaitForThread extends Thread {

private Process process;

private boolean finish = false;

private int exitValue = -1;

public CommandWaitForThread(Process process) {

this.process = process;

}

public void run() {

try {

this.exitValue = process.waitFor();

} catch (InterruptedException e) {

e.printStackTrace();

} finally {

finish = true;

}

}

public boolean isFinish() {

return finish;

}

public void setFinish(boolean finish) {

this.finish = finish;

}

public int getExitValue() {

return exitValue;

}

}

[java] view plain copy

package someTest;

import java.io.IOException;

import java.io.InputStreamReader;

import java.util.Date;

public class ShellTest {

public static void main(String[] args) {

InputStreamReader stdISR = null;

InputStreamReader errISR = null;

Process process = null;

String command = "/home/Lance/workspace/someTest/testbash.sh";

long timeout = 10 * 1000;

try {

process = Runtime.getRuntime().exec(command);

CommandStreamGobbler errorGobbler = new CommandStreamGobbler(process.getErrorStream(), command, "ERR");

CommandStreamGobbler outputGobbler = new CommandStreamGobbler(process.getInputStream(), command, "STD");

errorGobbler.start();

// 必须先等待错误输出ready再建立标准输出

while (!errorGobbler.isReady()) {

Thread.sleep(10);

}

outputGobbler.start();

while (!outputGobbler.isReady()) {

Thread.sleep(10);

}

CommandWaitForThread commandThread = new CommandWaitForThread(process);

commandThread.start();

long commandTime = new Date().getTime();

long nowTime = new Date().getTime();

boolean timeoutFlag = false;

while (!commandIsFinish(commandThread, errorGobbler, outputGobbler)) {

if (nowTime - commandTime > timeout) {

timeoutFlag = true;

break;

} else {

Thread.sleep(100);

nowTime = new Date().getTime();

}

}

if (timeoutFlag) {

// 命令超时

errorGobbler.setTimeout(1);

outputGobbler.setTimeout(1);

System.out.println("正式执行命令:" + command + "超时");

}else {

// 命令执行完成

errorGobbler.setTimeout(2);

outputGobbler.setTimeout(2);

}

while (true) {

if (errorGobbler.isReadFinish() && outputGobbler.isReadFinish()) {

break;

}

Thread.sleep(10);

}

} catch (IOException | InterruptedException e) {

e.printStackTrace();

} finally {

if (process != null) {

process.destroy();

}

}

}

private boolean commandIsFinish(CommandWaitForThread commandThread, CommandStreamGobbler errorGobbler, CommandStreamGobbler outputGobbler) {

if (commandThread != null) {

return commandThread.isFinish();

} else {

return (errorGobbler.isReadFinish() && outputGobbler.isReadFinish());

}

}

}


在以上的代码中,为了防止线程被阻塞,要点如下:

1. 在CommandStreamGobbler里,bufferedReader在readLine()之前,先用ready()看一下当前缓冲区的情况,请特别注意ready()描述,这个方法是非阻塞的。

[java] view plain copy

boolean java.io.BufferedReader.ready() throws IOException

Tells whether this stream is ready to be read. A buffered character stream is ready if the buffer is not empty, or if the underlying character stream is ready.

Returns:

True if the next read() is guaranteed not to block for input, false otherwise. Note that returning false does not guarantee that the next read will block.

2.在一个新线程commandThread中,调用process对象的waitFor()从而避免主线程卡死,主线程的最后会执行finally块中的process.destory()保证commandThread正常退出。

以上的两点改进,保证了Java在调用shell脚本过程互不被对方卡死的机制。

三.在执行shell脚本过程中,可能会添加参数,通常在终端中,我们使用“ ”(空格)把参数隔开。

为了区分空格是作为参数分隔符,还是参数的一部分。调用exec方法有特别的注意事项。

[java] view plain copy

String command = "/home/Lance/workspace/someTest/testbash.sh 'hello world'";

process = Runtime.getRuntime().exec(command);

等价于

[java] view plain copy

List<String> commandList = new LinkedList<String>();

commandList.add("/home/Lance/workspace/someTest/testbash.sh");

commandList.add("hello world");

String[] commands = new String[commandList.size()];

for (int i = 0; i < commandList.size(); i++) {

commands[i] = commandList.get(i);

}

process = Runtime.getRuntime().exec(commands);

好了,今天介绍到这里。
内容来自用户分享和网络整理,不保证内容的准确性,如有侵权内容,可联系管理员处理 点击这里给我发消息
标签: