您的位置:首页 > 数据库 > MySQL

scrapy 安装数据库mysql错误汇总和操作步骤

2016-11-17 10:44 585 查看
 操作步骤:

  1、检查python是否支持mysql
[root@bogon ~]# python
Python 2.7.10 (default, Jun  5 2015, 17:56:24)
[GCC 4.4.4 20100726 (Red Hat 4.4.4-13)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import MySQLdb
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ImportError: No module named MySQLdb


[root@bogon MySQL-python-1.2.5]# python

Python 2.7.12 (default, Oct 18 2016, 02:30:08)

[GCC 4.4.7 20120313 (Red Hat 4.4.7-4)] on linux2

Type "help", "copyright", "credits" or "license" for more information.

>>> import MySQLdb

Traceback (most recent call last):

File "<stdin>", line 1, in <module>

File "MySQLdb/__init__.py", line 19, in <module>

import _mysql

ImportError: No module named _mysql

>>>

  如果出现:ImportError: No module named MySQLdb则说明python尚未支持mysql,需要手工安装,请参考步骤2;如果没有报错,请调到步骤3

2、python安装mysql支持
[root@bogon ~]# pip install mysql-python
Collecting mysql-python
Downloading MySQL-python-1.2.5.zip (108kB)
100% |████████████████████████████████| 110kB 115kB/s
Building wheels for collected packages: mysql-python
Running setup.py bdist_wheel for mysql-python
Stored in directory: /root/.cache/pip/wheels/8c/0d/11/d654cad764b92636ce047897dd2b9e1b0cd76c22f813c5851a
Successfully built mysql-python
Installing collected packages: mysql-python
Successfully installed mysql-python-1.2.5


安装的时候报错:

[root@bogon MySQL-python-1.2.5]# pip install mysql-python

Collecting mysql-python

Using cached MySQL-python-1.2.5.zip

Complete output from command python setup.py egg_info:

sh: mysql_config: command not found

Traceback (most recent call last):

File "<string>", line 1, in <module>

File "/tmp/pip-build-hOhW5F/mysql-python/setup.py", line 17, in <module>

metadata, options = get_config()

File "setup_posix.py", line 43, in get_config

libs = mysql_config("libs_r")

File "setup_posix.py", line 25, in mysql_config

raise EnvironmentError("%s not found" % (mysql_config.path,))

EnvironmentError: mysql_config not found

----------------------------------------

Command "python setup.py egg_info" failed with error code 1 in /tmp/pip-build-hOhW5F/mysql-python/

[root@bogon MySQL-python-1.2.5]#

  安装完以后再次运行步骤1,检查python是否已经支持mysql

  如果还有问题您可以尝试:LC_ALL=C pip install mysql-python

  如果依然报错:error: Python.h: No such file or directory

  您可以尝试先安装python-devel:
yum install python-devel

安装Python-devel 报错:
[root@bogon MySQL-python-1.2.5]# yum install mysql-devel 报错

mysql-5.1.71-1.el6.x86_64: failure: Packages/mysql-5.1.71-1.el6.x86_64.rpm from rhel-iso: [Errno 256] No more mirrors to try.

mysql-devel-5.1.71-1.el6.x86_64: failure: Packages/mysql-devel-5.1.71-1.el6.x86_64.rpm from rhel-iso: [Errno 256] No more mirrors to try.

[root@bogon MySQL-python-1.2.5]# rpm -qa | grep -i mysql

mysql-libs-5.1.71-1.el6.x86_64

下载安装包,手动安装:

[root@bogon MySQL-python-1.2.5]# ll

total 192

drwxr-xr-x. 3 root root 4096 Nov 16 01:41 doc

-rw-rw-rw-. 1 root root 18092 Oct 8 2012 GPL-2.0

-rw-rw-rw-. 1 root root 5687 Jan 1 2013 HISTORY

-rw-rw-rw-. 1 root root 6636 Oct 8 2012 INSTALL

-rw-rw-rw-. 1 root root 309 Jan 2 2014 MANIFEST.in

-rw-rw-rw-. 1 root root 2039 Jan 2 2014 metadata.cfg

-rw-rw-rw-. 1 root root 84709 Jan 2 2014 _mysql.c

drwxr-xr-x. 3 root root 4096 Nov 16 18:50 MySQLdb

-rw-rw-rw-. 1 root root 2352 Oct 8 2012 _mysql_exceptions.py

drwxr-xr-x. 2 root root 4096 Nov 16 01:41 MySQL_python.egg-info

-rw-rw-rw-. 1 root root 1689 Jan 2 2014 PKG-INFO

-rw-rw-rw-. 1 root root 3203 Oct 8 2012 pymemcompat.h

-rw-rw-rw-. 1 root root 1266 Oct 4 2012 README.md

-rw-rw-rw-. 1 root root 398 Jan 2 2014 setup.cfg

-rw-rw-rw-. 1 root root 1088 Oct 8 2012 setup_common.py

-rw-r--r--. 1 root root 1682 Nov 16 01:43 setup_common.pyc

-rw-rw-rw-. 1 root root 3392 Jan 2 2014 setup_posix.py

-rw-r--r--. 1 root root 3274 Nov 16 01:43 setup_posix.pyc

-rw-rw-rw-. 1 root root 609 Jan 2 2014 setup.py

-rw-rw-rw-. 1 root root 1334 Oct 8 2012 setup_windows.py

-rw-rw-rw-. 1 root root 664 Nov 16 01:57 site.cfg

drwxr-xr-x. 2 root root 4096 Nov 16 01:41 tests

[root@bogon MySQL-python-1.2.5]# python setup.py install

sh: /usr/local/bin/mysql_config: No such file or directory

Traceback (most recent call last):

File "setup.py", line 17, in <module>

metadata, options = get_config()

File "/home/kangql/Desktop/MySQL-python-1.2.5/setup_posix.py", line 43, in get_config

libs = mysql_config("libs_r")

File "/home/kangql/Desktop/MySQL-python-1.2.5/setup_posix.py", line 25, in mysql_config

raise EnvironmentError("%s not found" % (mysql_config.path,))

EnvironmentError: /usr/local/bin/mysql_config not found

[root@bogon MySQL-python-1.2.5]#

3. 使用 rpm -qa|grep mysql查看mysql的安装情况

mysql-devel-5.5.16-1.el6_4.x86_64

mysql-client-5.5.16-1.el6_4.x86_64

mysql-server-5.5.16-1.el6_4.x86_64

4. 使用rpm -e mysql-devel-5.5.16-1.el6_4.x86_64移除已有的安装包

5. 再次使用yum install mysql-devel 发现和mysql-server, mysql-client的版本有冲突

6. 去网上寻找相同的版本下载http://downloads.skysql.com/archive/index/p/mysql/v/5.5.16,下载64bit的rpm文件,使用rpm -ivh MySQL-devel-5.5.16-1.linux2.6.x86_64.rpm安装

7. 安装成功后,去/usr/bin/目录下,发现生成了mysql_config文件,继续pip install mysql-python,顺利完成

[root@bogon MySQL-python-1.2.5]# rpm -qa|grep mysql

mysql-libs-5.1.71-1.el6.x86_64

[root@bogon MySQL-python-1.2.5]# rpm -e mysql-libs-5.1.71-1.el6.x86_64

error: Failed dependencies:

libmysqlclient.so.16()(64bit) is needed by (installed) postfix-2:2.6.6-2.2.el6_1.x86_64

libmysqlclient.so.16(libmysqlclient_16)(64bit) is needed by (installed) postfix-2:2.6.6-2.2.el6_1.x86_64

mysql-libs is needed by (installed) postfix-2:2.6.6-2.2.el6_1.x86_64

[root@bogon MySQL-python-1.2.5]#

无法卸载,使用其他命令:

[root@bogon MySQL-python-1.2.5]# rpm -e --nodeps mysql-libs-5.1.71-1.el6.x86_64

[root@bogon MySQL-python-1.2.5]# rpm -qa|grep mysql

[root@bogon MySQL-python-1.2.5]#

yum 安装MySQL

redhat系统需要注册才能更新yum,删除yum更新,结果yum无法使用。。。。。。。

查找各种资料,发现yum在Python2.7下面无法使用

奶奶的,花费了一个下午才解决:

解决方法如下(网上其他不靠谱的方法就不说了):

[root@Linux yum.repos.d]# yum

-bash: /usr/bin/yum: 没有那个文件或目录

[root@Linux yum.repos.d]# rpm -ivh --nodeps http://mirrors.163.com/centos/5/os/x86_64/CentOS//yum-fastestmirror-1.1.16-21.el5.centos.noarch.rpm
Retrieving http://mirrors.163.com/centos/5/os/x86_64/CentOS//yum-fastestmirror-1.1.16-21.el5.centos.noarch.rpm
Preparing... ########################################### [100%]

1:yum-fastestmirror ########################################### [100%]

[root@Linux yum.repos.d]# rpm -ivh --nodeps http://mirrors.163.com/centos/5/os/x86_64/CentOS/yum-fastestmirror-1.1.16-21.el5.centos.noarch.rpm
Retrieving http://mirrors.163.com/centos/5/os/x86_64/CentOS/yum-fastestmirror-1.1.16-21.el5.centos.noarch.rpm
Preparing... ########################################### [100%]

package yum-fastestmirror-1.1.16-21.el5.centos.noarch is already installed

[root@Linux yum.repos.d]# rpm -ivh --nodeps http://mirrors.163.com/centos/5/os/x86_64/CentOS/yum-metadata-parser-1.1.2-4.el5.x86_64.rpm
Retrieving http://mirrors.163.com/centos/5/os/x86_64/CentOS/yum-metadata-parser-1.1.2-4.el5.x86_64.rpm
Preparing... ########################################### [100%]

1:yum-metadata-parser ########################################### [100%]

[root@Linux yum.repos.d]# rpm -ivh --nodeps http://mirrors.163.com/centos/5/os/x86_64/CentOS/yum-3.2.22-40.el5.centos.noarch.rpm
Retrieving http://mirrors.163.com/centos/5/os/x86_64/CentOS/yum-3.2.22-40.el5.centos.noarch.rpm
Preparing... ########################################### [100%]

1:yum ########################################### [100%]

[root@Linux yum.repos.d]# yum
已解决。

[root@bogon Desktop]# yum list

There was a problem importing one of the Python modules

required to run yum. The error leading to this problem was:

No module named yum

Please install a package which provides this module, or

verify that the module is installed correctly.

It's possible that the above module doesn't match the

current version of Python, which is:

2.7.12 (default, Oct 18 2016, 02:30:08)

[GCC 4.4.7 20120313 (Red Hat 4.4.7-4)]

If you cannot solve this problem yourself, please go to

the yum faq at:
http://wiki.linux.duke.edu/YumFaq
[root@bogon Desktop]# rpm -qa |grep yum

yum-fastestmirror-1.1.16-21.el5.centos.noarch

yum-3.2.22-40.el5.centos.noarch

yum-metadata-parser-1.1.2-4.el5.x86_64

[root@bogon Desktop]# whereis python

python: /usr/bin/python2.6-config /usr/bin/python2.6 /usr/bin/python /usr/lib/python2.6 /usr/lib/python2.4 /usr/lib64/python2.6 /usr/lib64/python2.4 /usr/include/python2.6 /usr/share/man/man1/python.1.gz

[root@bogon Desktop]# which yum

/usr/bin/yum

[root@bogon Desktop]# vi /usr/bin/yum

[root@bogon Desktop]#

[root@develop bin]# yum

There was a problem importing one of the Python modules

required to run yum. The error leading to this problem was:

No module named yum

Please install a package which provides this module, or

verify that the module is installed correctly.

It's possible that the above module doesn't match the

current version of Python, which is:

2.6.1 (r261:67515, Aug 7 2010, 11:36:17)

[GCC 4.1.2 20080704 (Red Hat 4.1.2-44)]

If you cannot solve this problem yourself, please go to

the yum faq at:
http://wiki.linux.duke.edu/YumFaq
提示当前版本是2.6.1可能版本不匹配。没有yum依赖的包。

3、查看yum版本

[root@develop local]# rpm -qa |grep yum

yum-3.2.8-9.el5.centos.1

yum-metadata-parser-1.1.2-2.el5

4、查看python版本

[root@develop local]# whereis python

python: /usr/bin/python2.4 /usr/bin/python /usr/lib/python2.4 /usr/local/bin/python2.6 /usr/local/bin/python2.6-config /usr/local/bin/python /usr/local/lib/python2.6 /usr/share/man/man1/python.1.gz

果然装了两个版本python

5、执行python,查看到使用2.6.1的版本

[root@develop local]# python

Python 2.6.1 (r261:67515, Aug 7 2010, 11:36:17)

[GCC 4.1.2 20080704 (Red Hat 4.1.2-44)] on linux2

Type "help", "copyright", "credits" or "license" for more information.

>>>

6、猜测yum调用了高版本的python。

7、解决方法:

查找yum文件,并编辑此py文件

[root@develop local]# which yum

/usr/bin/yum

[root@develop local]# vi /usr/bin/yum



#!/usr/bin/python

改为:

#!/usr/bin/python2.4

然后保存OK.

补充:yum基于python写的。

发现无论怎么修改,yum还是无法使用。

[root@bogon Desktop]# python

Python 2.7.12 (default, Oct 18 2016, 02:30:08)

[GCC 4.4.7 20120313 (Red Hat 4.4.7-4)] on linux2

Type "help", "copyright", "credits" or "license" for more information.

>>> import yum

Traceback (most recent call last):

File "<stdin>", line 1, in <module>

ImportError: No module named yum

>>>

[root@bogon Desktop]# rpm -qa |grep yum

yum-fastestmirror-1.1.16-21.el5.centos.noarch

yum-3.2.22-40.el5.centos.noarch

yum-metadata-parser-1.1.2-4.el5.x86_64

[root@bogon Desktop]# whereis python

python: /usr/bin/python2.6-config /usr/bin/python2.6 /usr/bin/python /usr/lib/python2.3 /usr/lib/python2.6 /usr/lib/python2.4 /usr/lib64/python2.3 /usr/lib64/python2.6 /usr/lib64/python2.4 /usr/include/python2.6 /usr/share/man/man1/python.1.gz

[root@bogon Desktop]#

查看系统使用的python版本

ls /usr/bin |grep python











修改yum文件

#vi /usr/bin/yum

将 #!/usr/bin/python 修改为 #!/usr/bin/python2.6

如果有大神知道怎么处理,请跟我联系或者留言~

ps:yum的安装程序链接http://mirrors.163.com/centos/6/os/x86_64/Packages/

MySQL下载地址:http://dev.mysql.com/downloads/file/?id=466135

MySQL数据库安装好后保存数据,报错误信息:[scrapy] ERROR: (1366, "Incorrect string value: '\\xE4\\xB8\\xAD' for column 'productName' at row 1 原因是数据库字段和Python字段的类型不一致,将数据库字段的类型改为utf-8。参照:http://www.lai18.com/content/9984292.html

另外,爬取出来的数据有[。。。。]这样的方括号,在语句末尾添加[0],这样可以去掉方括号。比如:

sel.xpath('//span[contains(@id,"lblTimeLimit")]/descendant::text()').extract()[0]
内容来自用户分享和网络整理,不保证内容的准确性,如有侵权内容,可联系管理员处理 点击这里给我发消息
标签: