您的位置:首页 > 运维架构 > Shell

shell 脚本实现每秒循环并在指定时间执行代码

2017-04-05 15:19 691 查看
业务场景:有一段sql需要在多个指定时间执行

分析:可以写成crontab,但是由于需要在多个时间点去执行,写在crontab里会显得冗余,考虑使用循环方式去执行

代码实现:

if [ $# -eq 0 ]
then
time_id=`date -d "-1 day" +%Y%m%d`
BEGIN_DATE=`date +%Y-%m-%d`
END_DATE=`date +%Y-%m-%d`
CUR_PARTITIONS=`date -d "-1 day" +%Y%m%d`
STAT_VALUE=`date +%H`
elif [ $# -eq 2 ]
then
format_day=`echo $1|grep -o '[0-9]\{8\}'`
format_hour=`echo $1|grep -o '[0-9]\{2\}$'`
BEGIN_DATE=`date -d "$format_day" +%Y-%m-%d`
END_DATE=`date -d "$format_day" +%Y-%m-%d`
CUR_PARTITIONS=`date -d "$format_day" +%Y%m%d`
STAT_VALUE=$2
else
echo "the args is wrong ,you should give it like '2014092307'"
exit 1;
fi

# work dir
WORK_DIR="/opt/DATA/goldmine/src/kpi/mall_daily_hour_stat"
while true
do
time=`date +%H:%M:%S`
echo "now: ${time}"
if [ '12:01:00' = ${time} ] || [ '14:01:00' = ${time} ] || [ '16:01:00' = ${time} ] || [ '18:01:00' = ${time} ] || [ '20:01:00' = ${time} ] || [ '22:01:00' = ${time} ] || [ '23:50:00' = ${time} ]
then
queryOrderSum="select xxxxxx"
hive -e "${queryOrderSum}">${WORK_DIR}/data/tmpDataDailyHourSum.txt

# format data
cat ${WORK_DIR}/data/tmpDataDailyHourSum.txt | awk -F '\t' '{print $1"^"$2"^"$3"^"$4}' | sed 's/NULL/0/g' >${WORK_DIR}/data/dataDailyHourSum_${BEGIN_DATE}_${STAT_VALUE}.txt

# del data
sh /opt/DATA/goldmine/src/kpi/common_script/handleMysqlDB.sh "delete from dd_mall_common_stat where statdate >= '${BEGIN_DATE}' and statdate <= '${END_DATE}' and stat_type='DAILY_HOUR' and stat_value='${STAT_VALUE}'"

# insertXxx.sh [data.txt]
sh ${WORK_DIR}/insertDailyHour.sh ${WORK_DIR}/data/dataDailyHourSum_${BEGIN_DATE}_${STAT_VALUE}.txt ${STAT_VALUE}
fi
sleep 1
done

----------------------------------------------补充-----------------------------------------------------

在实际的运用场景中,发现了一个问题,如果脚本循环精确到秒的话,很有可能错过这个时间点,并且每秒循环对性能消耗有点多余

考虑每五分钟循环,需要配合crontab

1·:在crontab里设置每5分钟执行一次脚本:

*/5 * * * * /opt/DATA/goldmine/src/kpi/mall_daily_hour_stat/product/mall_product_daily_hour_stat.sh >> /opt/DATA/goldmine/src/kpi/mall_daily_hour_stat/product/mall_product_daily_hour_stat.sh.log 2>&12·:修改脚本:
# work dir
WORK_DIR="/opt/DATA/goldmine/src/kpi/mall_daily_hour_stat/product"

time=`date +%H:%M`
#echo "now: ${time}"
if [ '10:35' = ${time} ] || [ '12:25' = ${time} ] || [ '14:25' = ${time} ] || [ '16:25' = ${time} ] || [ '18:25' = ${time} ] || [ '20:25' = ${time} ] || [ '22:25' = ${time} ] ||[ '23:45' = ${time} ]
then
queryMallProduct="select XXX"

#echo ${queryMallProduct}
hive -e "${queryMallProduct}">${WORK_DIR}/data/tmpDataMallProduct.txt

# format data
cat ${WORK_DIR}/data/tmpDataMallProduct.txt | awk -F '\t' '{print $1"^"$2"^"$3"^"$4"^"$5"^"$6"^"$7"^"$8}' | sed 's/NULL/0/g' >${WORK_DIR}/data/dataMallProduct_${BEGIN_DATE}.txt

# del data
sh /opt/DATA/goldmine/src/kpi/common_script/handleMysqlDB.sh "delete from dd_app_tuan_product_stat where statdate >= '${BEGIN_DATE}' and statdate <= '${END_DATE}'"

# insertXxx.sh [data.txt]
sh ${WORK_DIR}/insertMallProductDailyHourStat.sh ${WORK_DIR}/data/dataMallProduct_${BEGIN_DATE}.txt ${BEGIN_DATE}
echo ${time}
fi
内容来自用户分享和网络整理,不保证内容的准确性,如有侵权内容,可联系管理员处理 点击这里给我发消息
标签: