您的位置:首页 > 数据库

scrapy在spider中通过pipeline获取数据库内容

2016-12-27 00:00 330 查看
http://stackoverflow.com/questions/23105590/how-to-get-the-pipeline-object-in-scrapy-spider

# This is my Pipline
class MongoDBPipeline(object):
def __init__(self, mongodb_db=None, mongodb_collection=None):
self.connection = pymongo.Connection(settings['MONGODB_SERVER'], settings['MONGODB_PORT'])
....

def process_item(self, item, spider):
....
def get_date(self):
....

def open_spider(self, spider):
spider.myPipeline = self

class Spider(Spider):
name = "test"

def __init__(self):
self.myPipeline = None

def parse(self, response):
self.myPipeline.get_date()
内容来自用户分享和网络整理,不保证内容的准确性,如有侵权内容,可联系管理员处理 点击这里给我发消息
标签: