您的位置：首页 > 数据库

using pyodbc on ubuntu to insert a image field on SQL Server

2015-01-12 19:05 2061 查看

using
pyodbc on ubuntu to insert a image field on SQL Server

up
vote6down
votefavorite

8

I am using Ubuntu 9.04

I have installed the following package versions:

unixodbc and unixodbc-dev: 2.2.11-16build3
tdsodbc: 0.82-4
libsybdb5: 0.82-4
freetds-common and freetds-dev: 0.82-4
python2.6-dev

I have configured

/etc/unixodbc.ini

like
this:

[FreeTDS]
Description             = TDS driver (Sybase/MS SQL)
Driver          = /usr/lib/odbc/libtdsodbc.so
Setup           = /usr/lib/odbc/libtdsS.so
CPTimeout               =
CPReuse         =
UsageCount              = 2

I have configured

/etc/freetds/freetds.conf

like
this:

[global]
tds version = 8.0
client charset = UTF-8
text size = 4294967295

I have grabbed pyodbc revision

31e2fae4adbf1b2af1726e5668a3414cf46b454f

from

http://github.com/mkleehammer/pyodbc

and
installed it using "

python
setup.py install

"

I have a windows machine with Microsoft SQL Server 2000 installed on my local network, up and listening on the local ip address 10.32.42.69. I have an
empty database created with name "Common". I have the user "sa" with password "secret" with full privileges.

I am using the following python code to setup the connection:

import pyodbc
odbcstring = "SERVER=10.32.42.69;UID=sa;PWD=secret;DATABASE=Common;DRIVER=FreeTDS"
con = pyodbc.connect(odbcstring)
cur = con.cursor()

cur.execute("""
IF EXISTS(SELECT TABLE_NAME FROM INFORMATION_SCHEMA.TABLES
WHERE TABLE_NAME = 'testing')
DROP TABLE testing
""")
cur.execute('''
CREATE TABLE testing (
id INTEGER NOT NULL IDENTITY(1,1),
myimage IMAGE NULL,
PRIMARY KEY (id)
)
''')
con.commit()

Everything WORKS up to this point. I have used SQLServer's Enterprise Manager on the server and the new table is there. Now I want to insert some data
on the table.

cur = con.cursor()
# using web data for exact reproduction of the error by all.
# I'm actually reading a local file in my real code.
url = 'http://www.forestwander.com/wp-content/original/2009_02/west-virginia-mountains.jpg'
data = urllib2.urlopen(url).read()

sql = "INSERT INTO testing (myimage) VALUES (?)"

Now here on my original question, I was having trouble using

cur.execute(sql,
(data,))

but now I've edited the question, because following Vinay Sajip's answer below (THANKS), I have changed it to:

cur.execute(sql, (pyodbc.Binary(data),))
con.commit()

And insertion is working perfectly. I can confirm the size of the inserted data using the following test code:

cur.execute('SELECT DATALENGTH(myimage) FROM testing WHERE id = 1')
data_inside = cur.fetchone()[0]
assert data_inside == len(data)

Which passes perfectly!!!

Now the problem is on retrieval of the data back.

I am trying the common approach:

cur.execute('SELECT myimage FROM testing WHERE id = 1')
result = cur.fetchone()
returned_data = str(result[0]) # transforming buffer object
print 'Original: %d; Returned: %d' % (len(data), len(returned_data))
assert data == returned_data

However that fails!!

Original: 4744611; Returned: 4096
Traceback (most recent call last):
File "/home/nosklo/devel/teste_mssql_pyodbc_unicode.py", line 53, in <module>
assert data == returned_data
AssertionError

I've put all the code above in a single file here, for
easy testing of anyone that wants to help.

Now for the question:

I want python code to insert an image file into mssql. I want to query the image back and show it to the user.

I don't care about the column type in mssql. I am using the "

IMAGE

"
column type on the example, but any binary/blob type would do, as long as I get the binary data for the file I inserted back unspoiled. Vinay Sajip said below that this is the preferred data type for this in SQL SERVER 2000.

The data is now being inserted without errors, however when I retrieve the data, only 4k are returned. (Data is truncated on 4096).

How can I make that work?

EDITS: Vinay Sajip's answer below gave me a hint to use pyodbc.Binary on the field. I have updated the question accordingly. Thanks Vinay Sajip!

Alex Martelli's comment gave me the idea of using the

DATALENGTH

MS
SQL function to test if the data is fully loaded on the column. Thanks Alex Martelli !

python sql-server image pyodbc freetds

share|improve
this question

edited Dec
2 '09 at 11:20

marc_s

348k53612803

asked Jun 29 '09 at 19:26

nosklo

81.3k22169200

what do you get when you 'SELECT DATALENGTH(myimage) FROM testing'? At least this will tell you if your problem is with storing or with retrieving. – Alex
Martelli Jun
29 '09 at 22:40

add
a comment

4 Answers

active oldest votes

up
vote3down
voteaccepted

Huh, just after offering the bounty, I've found out the solution.

You have to use

SET
TEXTSIZE 2147483647

on the query, in addition of text size configuration option in

/etc/freetds/freetds.conf

.

I have used

cur.execute('SET TEXTSIZE 2147483647 SELECT myimage FROM testing WHERE id = 1')

And everything worked fine.

Strange is what FreeTDS documentation
says about the text size configuration option:

default value of

TEXTSIZE

,
in bytes. For

text

and

image

datatypes,
sets the maximum width of any returned column. Cf.

set
TEXTSIZE

in the

T-SQL

documentation
for your server.

The configuration also says that the maximum value (and the default) is 4,294,967,295. However when trying to use that value in the query I get an error, the max number I could use in the query is 2,147,483,647 (half).

From that explanation I thought that only setting this configuration option would be enough. It turns out that I was wrong, setting TEXTSIZE in the query fixed the issue.

Below is the complete working code:

#!/usr/bin/env python
# -*- coding: utf-8 -*-

import pyodbc
import urllib2

odbcstring = "SERVER=10.32.42.69;UID=sa;PWD=secret;DATABASE=Common;DRIVER=FreeTDS"
con = pyodbc.connect(odbcstring)
cur = con.cursor()

cur.execute("""
IF EXISTS(SELECT TABLE_NAME FROM INFORMATION_SCHEMA.TABLES
WHERE TABLE_NAME = 'testing')
DROP TABLE testing
""")

cur.execute('''
CREATE TABLE testing (
id INTEGER NOT NULL IDENTITY(1,1),
myimage IMAGE NULL,
PRIMARY KEY (id)
)
''')

con.commit()
cur = con.cursor()
url = 'http://www.forestwander.com/wp-content/original/2009_02/west-virginia-mountains.jpg'
data = urllib2.urlopen(url).read()

sql = "INSERT INTO testing (myimage) VALUES (?)"
cur.execute(sql, (pyodbc.Binary(data),))
con.commit()
cur.execute('SELECT DATALENGTH(myimage) FROM testing WHERE id = 1')
data_inside = cur.fetchone()[0]
assert data_inside == len(data)
cur.execute('SET TEXTSIZE 2147483647 SELECT myimage FROM testing WHERE id = 1')result = cur.fetchone()
returned_data = str(result[0])
print 'Original: %d; Returned; %d' % (len(data), len(returned_data))
assert data == returned_data

share|improve
this answer

answered Jul 2 '09 at 11:20

nosklo

81.3k22169200

add
a comment

up
vote2down
vote

I think you should be using a

pyodbc.Binary

instance
to wrap the data:

cur.execute('INSERT INTO testing (myimage) VALUES (?)', (pyodbc.Binary(data),))

Retrieving should be

cur.execute('SELECT myimage FROM testing')
print "image bytes: %r" % str(cur.fetchall()[0][0])

UPDATE: The problem is in insertion. Change your insertion SQL to the following:

"""DECLARE @txtptr varbinary(16)

INSERT INTO testing (myimage) VALUES ('')
SELECT @txtptr = TEXTPTR(myimage) FROM testing
WRITETEXT testing.myimage @txtptr ?
"""

I've also updated the mistake I made in using the value attribute in the retrieval code.

With this change, I'm able to insert and retrieve a 320K JPEG image into the database (retrieved data is identical to inserted data).

N.B. The

image

data
type is deprecated, and is replaced by

varbinary(max)

in
later versions of SQL Server. The same logic for insertion/retrieval should apply, however, for the newer column type.

share|improve
this answer

edited Jun
30 '09 at 6:22

answered Jun 29 '09 at 19:58

Vinay Sajip

43.3k37896

Correct, although I had to use str(cur.fetchall()[0][0]) to get the data. .value returned: AttributeError: 'buffer' object has no attribute
'value' – nosklo Jun
29 '09 at 20:18

I'm still having issues. I've updated the question, please take a look. – nosklo Jun
29 '09 at 20:35

Thans Vinay for the update. Turns out the problem is not in insertion. I've tried the TEXTPTR code and got the exact same result. Thanks
for your time. I've edited the question again to show the current issue. Can you give me the full code you said that worked? Can you give me details about your setup? I still can't make it work. If it is not asking much, can you also please run my code unchanged
on your setup and tell me the results? I've made it available on paste.pocoo.org/show/125955 and
it uses a file on the internet so it will run exactly the same. – nosklo Jun
30 '09 at 17:33

I don't have SQL Server 2000 accessed from Jaunty at the moment; and your code (posted on LodgeIt) works perfectly for me on Windows. So
it might be something to do with the FreeTDS driver or something else in the Linux stack. Sorry I can't be of more help. – Vinay
Sajip Jun
30 '09 at 21:20

add
a comment

up
vote1down
vote

SET TEXTSIZE 2147483647

Many thanks for post your solution. I was two hours searching the internet about this problem. :)

share|improve
this answer

answered Aug 9 '10 at 0:19

stranger

211

add
a comment

up
vote1down
vote

I had a similar

truncation
issue on

TEXT

fields,
which

SET
TEXTSIZE 2147483647

fixed for me, but this also fixed it for me:

import os
os.environ['TDSVER'] = '8.0'

share|improve
this answer

answered Aug 4 '11 at 14:56

Collin Anderson

1,3631419

add
a comment

内容来自用户分享和网络整理，不保证内容的准确性，如有侵权内容，可联系管理员处理

标签：

相关文章推荐

新的分享

章节导航

using pyodbc on ubuntu to insert a image field on SQL Server

usingpyodbc on ubuntu to insert a image field on SQL Server

4 Answers

using
pyodbc on ubuntu to insert a image field on SQL Server