struct.error: unpack requires a string argument of length 4
2018-02-01 00:00
1186 查看
摘要: 报错如下,总而言之就是三个字:看不懂!!!线下一直运行正常,一到线上就运行错误,最后发现线上少一个openssl库,安装后正常了。。。。。。也不知道为啥
就在刚刚,发现了根本问题,因为爬虫用的jobdir,公司原来用的python2,我后来用的python3,两个程序同时指向了一个jobdir,python2和3编码格式不一样所以导致了问题
Exception:
suara_travel ERROR:
2018-01-31 20:03:16 [twisted] CRITICAL: Unhandled Error
Traceback (most recent call last):
File "/root/airflow_crawl/beritaqu-crawl/wkd/commands/crawl.py", line 65, in run
self.crawler_process.start()
File "/usr/lib64/python2.7/site-packages/scrapy/crawler.py", line 285, in start
reactor.run(installSignalHandlers=False) # blocking call
File "/usr/lib64/python2.7/site-packages/twisted/internet/base.py", line 1243, in run
self.mainLoop()
File "/usr/lib64/python2.7/site-packages/twisted/internet/base.py", line 1252, in mainLoop
self.runUntilCurrent()
--- ---
File "/usr/lib64/python2.7/site-packages/twisted/internet/base.py", line 878, in runUntilCurrent
call.func(*call.args, **call.kw)
File "/usr/lib64/python2.7/site-packages/scrapy/utils/reactor.py", line 41, in __call__
return self._func(*self._a, **self._kw)
File "/usr/lib64/python2.7/site-packages/scrapy/core/engine.py", line 135, in _next_request
self.crawl(request, spider)
File "/usr/lib64/python2.7/site-packages/scrapy/core/engine.py", line 210, in crawl
self.schedule(request, spider)
File "/usr/lib64/python2.7/site-packages/scrapy/core/engine.py", line 216, in schedule
if not self.slot.scheduler.enqueue_request(request):
File "/usr/lib64/python2.7/site-packages/scrapy/core/scheduler.py", line 57, in enqueue_request
dqok = self._dqpush(request)
File "/usr/lib64/python2.7/site-packages/scrapy/core/scheduler.py", line 86, in _dqpush
self.dqs.push(reqd, -request.priority)
File "/usr/lib/python2.7/site-packages/queuelib/pqueue.py", line 33, in push
self.queues[priority] = self.qfactory(priority)
File "/usr/lib64/python2.7/site-packages/scrapy/core/scheduler.py", line 114, in _newdq
return self.dqclass(join(self.dqdir, 'p%s' % priority))
File "/usr/lib/python2.7/site-packages/queuelib/queue.py", line 142, in __init__
self.size, = struct.unpack(self.SIZE_FORMAT, qsize)
struct.error: unpack requires a string argument of length 4
安装步骤参考:
http://blog.csdn.net/shiyong1949/article/details/78212971?locationNum=10&fps=1
就在刚刚,发现了根本问题,因为爬虫用的jobdir,公司原来用的python2,我后来用的python3,两个程序同时指向了一个jobdir,python2和3编码格式不一样所以导致了问题
Exception:
suara_travel ERROR:
2018-01-31 20:03:16 [twisted] CRITICAL: Unhandled Error
Traceback (most recent call last):
File "/root/airflow_crawl/beritaqu-crawl/wkd/commands/crawl.py", line 65, in run
self.crawler_process.start()
File "/usr/lib64/python2.7/site-packages/scrapy/crawler.py", line 285, in start
reactor.run(installSignalHandlers=False) # blocking call
File "/usr/lib64/python2.7/site-packages/twisted/internet/base.py", line 1243, in run
self.mainLoop()
File "/usr/lib64/python2.7/site-packages/twisted/internet/base.py", line 1252, in mainLoop
self.runUntilCurrent()
--- ---
File "/usr/lib64/python2.7/site-packages/twisted/internet/base.py", line 878, in runUntilCurrent
call.func(*call.args, **call.kw)
File "/usr/lib64/python2.7/site-packages/scrapy/utils/reactor.py", line 41, in __call__
return self._func(*self._a, **self._kw)
File "/usr/lib64/python2.7/site-packages/scrapy/core/engine.py", line 135, in _next_request
self.crawl(request, spider)
File "/usr/lib64/python2.7/site-packages/scrapy/core/engine.py", line 210, in crawl
self.schedule(request, spider)
File "/usr/lib64/python2.7/site-packages/scrapy/core/engine.py", line 216, in schedule
if not self.slot.scheduler.enqueue_request(request):
File "/usr/lib64/python2.7/site-packages/scrapy/core/scheduler.py", line 57, in enqueue_request
dqok = self._dqpush(request)
File "/usr/lib64/python2.7/site-packages/scrapy/core/scheduler.py", line 86, in _dqpush
self.dqs.push(reqd, -request.priority)
File "/usr/lib/python2.7/site-packages/queuelib/pqueue.py", line 33, in push
self.queues[priority] = self.qfactory(priority)
File "/usr/lib64/python2.7/site-packages/scrapy/core/scheduler.py", line 114, in _newdq
return self.dqclass(join(self.dqdir, 'p%s' % priority))
File "/usr/lib/python2.7/site-packages/queuelib/queue.py", line 142, in __init__
self.size, = struct.unpack(self.SIZE_FORMAT, qsize)
struct.error: unpack requires a string argument of length 4
安装步骤参考:
http://blog.csdn.net/shiyong1949/article/details/78212971?locationNum=10&fps=1
相关文章推荐
- struct.error: unpack requires a string argument of length 4
- python——unpack问题 ocr_recognize timeout , exception:unpack requires a string argument of length 46
- python二进制解析strut.upack的错误unpack requires a string argument of length 4
- python二进制解析strut.upack的错误unpack requires a string argument of length 4
- logstash nginx 报ArgumentError: comparison of String with 5 failed
- logstash nginx 报ArgumentError: comparison of String with 5 failed
- python:PyInstaller打包发布单个文件错误struct.error: argument for 's' must be a string解决
- logstash ArgumentError: comparison of String with 5 failed
- logstash ArgumentError: comparison of String with 5 failed
- unpack requires a bytes object of length 4
- startIndex can not be larger than length of string. Parameter name: startIndex
- Uncaught TypeError: Cannot read property 'length' of undefined
- ArgumentError: argument --batch_size: conflicting option string: --batch_size
- 解决方法:OTB 中 GDALJP2Box::GDALJP2Box(struct iobuf *),there is the error of unresolved external in debug
- error: argument of type 'void* (My_Thread::)(void*)' does not match 'void *(*)(void*)'
- Invalid layout of java.lang.String at value Error occurred during initialization of VM Invalid layou
- Q10:fatal error:use of unimplemented initializer init(nibName nibNameOrNil: String?, bundle nibBundl
- Cannot find an initializer for type '[(String)]' that accepts an argument list of type '(LazyForward
- No signature of method: java.lang.String.positive() is applicable for argument types: () values: []
- Error 1 Operator '+' cannot be applied to operands of type 'string' and 'method group'