大数据爬虫基础（三）Scrapy在ubuntu 16.04下的安装

Scrapy ubuntu下安装
系统：ubuntu 16.04 no gui

依赖包及依赖包的依赖包：

下列的安装步骤假定您已经安装好下列程序:
http://scrapy.org/
Python 2.7

Python Package: pip and setuptools. 现在 pip 依赖 setuptools ，如果未安装，则会自动安装 setuptools 。

lxml. 大多数Linux发行版自带了lxml。如果缺失，请查看http://lxml.de/installation.html

OpenSSL. 除了Windows(请查看 平台安装指南)之外的系统都已经提供。

您可以使用pip来安装Scrapy(推荐使用pip来安装Python package).

使用pip安装:

小写scrapy不是大写，官网是小写
http://scrapy-chs.readthedocs.org/zh_CN/latest/intro/install.html
pip install scrapy

1、pip,easy_install

Ubuntu下安装pip的方法

 http://www.2cto.com/os/201305/213725.html

安装pip的方法：

Install pip and virtualenv for Ubuntu 10.10 Maverick and newer

 

$ sudo apt-get install python-pip python-dev build-essential 

$ sudo pip install --upgrade pip 

$ sudo pip install --upgrade virtualenv 

For older versions of Ubuntu

 

Install Easy Install

$ sudo apt-get install python-setuptools python-dev build-essential 

Install pip

$ sudo easy_install pip 

Install virtualenv

$ sudo pip install --upgrade virtualenv 

sudo apt-get install python-setuptools python-dev build-essential

2、lxml

先安装依赖包，否则装不上，报错：x86_64-linux-gnu-gcc error

装：apt-get install -y libxml2-dev libxslt1-dev zlib1g-dev python3-pip

或者装：apt-get install build-essential autoconf libtool pkg-config python-opengl python-imaging python-pyrex python-pyside.qtopengl idle-python2.7 qt4-dev-tools qt4-designer libqtgui4 libqtcore4 libqt4-xml libqt4-test libqt4-script libqt4-network libqt4-dbus python-qt4
python-qt4-gl libgle3 python-dev

success后

再pip install lxml

success

3、cryptography及其依赖包

直接pip install scrapy会报 cryptography和cffi的错误，安装以下依赖包：
https://cryptography.io/en/latest/installation/#building-cryptography-on-linux
apt-get install build-essential libssl-dev libffi-dev python-dev

success

4、Scrapy

pip install scrapy

success

5、可选包：

pip install pymongo

pip install pillow

pip install pycrypto

6、报错参考
http://stackoverflow.com/questions/22073516/failed-to-install-python-cryptography-package-with-pip-and-setup-Python http://stackoverflow.com/questions/27130286/error-command-x86-64-linux-gnu-gcc-failed-with-exit-status-1-in-virtualenv http://www.cnblogs.com/lyroge/archive/2013/02/22/2922515.html