socket接受数据的三种方式
2015-06-02 20:39
344 查看
原位置:http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/408859
Description:
An issue with socket.recv is how to know when you are done receiving data. A TCP stream guarantees the bytes will not arrive out of order or be sent more than once. But you do not know the size of the data that will be sent to you. 100 bytes could be sent
as group of 10 bytes or maybe in one shot. Ultimately, this means you have to use a loop in some fashion until you know it is done.
The basic recv returns an empty string when the socket is disconnected.
From that you can build a simple loop that will work as long as the sender manages to disconnect the socket at the appropriate time. However, there could be situations where a local error will mask as a clean shutdown or maybe a close() is never called.
Three very basic methods are shown below that try to fix that problem. They use either a time-based, end marker, or size of payload method. Since you cannot be sure just what you are going to receive, you have to be careful that you get enough of a message
to determine the size of payload or end marker.
I updated the recv_size method to allocate data in larger chunks if it gets a large stream of data, which can increase performance.
代码如下:
import socket,struct,sys,time
Discussion:
I employ a trivial server, to keep this as simple as possible. Just uncomment the type of receiving server you want to use to see the recv type chosen in action.
The recv_timeout function, which uses non-blocking sockets, will continue trying to get data as long as the client manages to even send a single byte. This is useful for moving data which you know very little about (like encrypted data), so cannot check for
completion in a sane way.
The recv_end function tacks on an end marker understood by both the client and the server. One problem with this is that data cannot look like the marker.
The recv_size function looks for the size of the payload to be prepended to the data. I use a fixed length, 4 bytes. So then I know to always look for that. It is packed to allow a terse representation that can fit sys.maxint. This avoids the problem of data
looking like the marker, however, even if it means a lot of data, you are limited w/the payload being the maximum number that can be packed. An interesting advantage of this method is that it can allocate data in larger chunks since it knows this size of the
data ahead of time. For large streams of data, I saw it increase performace by 10 times.
To test this, in a another process, try using the functions that match with the server type.
send_size(data) #for recv_size
send_end(data) #for recv_end
sock.sendall(data) #for timeout or simple recv(8192)
#do not forget to close if you do a raw sendall
sock.close()
Description:
An issue with socket.recv is how to know when you are done receiving data. A TCP stream guarantees the bytes will not arrive out of order or be sent more than once. But you do not know the size of the data that will be sent to you. 100 bytes could be sent
as group of 10 bytes or maybe in one shot. Ultimately, this means you have to use a loop in some fashion until you know it is done.
The basic recv returns an empty string when the socket is disconnected.
From that you can build a simple loop that will work as long as the sender manages to disconnect the socket at the appropriate time. However, there could be situations where a local error will mask as a clean shutdown or maybe a close() is never called.
Three very basic methods are shown below that try to fix that problem. They use either a time-based, end marker, or size of payload method. Since you cannot be sure just what you are going to receive, you have to be careful that you get enough of a message
to determine the size of payload or end marker.
I updated the recv_size method to allocate data in larger chunks if it gets a large stream of data, which can increase performance.
代码如下:
import socket,struct,sys,time
Port=2222 #assume a socket disconnect (data returned is empty string) means all data was #done being sent. def recv_basic(the_socket): total_data=[] while True: data = the_socket.recv(8192) if not data: break total_data.append(data) return ''.join(total_data) def recv_timeout(the_socket,timeout=2): the_socket.setblocking(0) total_data=[];data='';begin=time.time() while 1: #if you got some data, then break after wait sec if total_data and time.time()-begin>timeout: break #if you got no data at all, wait a little longer elif time.time()-begin>timeout*2: break try: data=the_socket.recv(8192) if data: total_data.append(data) begin=time.time() else: time.sleep(0.1) except: pass return ''.join(total_data) End='something useable as an end marker' def recv_end(the_socket): total_data=[];data='' while True: data=the_socket.recv(8192) if End in data: total_data.append(data[:data.find(End)]) break total_data.append(data) if len(total_data)>1: #check if end_of_data was split last_pair=total_data[-2]+total_data[-1] if End in last_pair: total_data[-2]=last_pair[:last_pair.find(End)] total_data.pop() break return ''.join(total_data) def recv_size(the_socket): #data length is packed into 4 bytes total_len=0;total_data=[];size=sys.maxint size_data=sock_data='';recv_size=8192 while total_len<size: sock_data=the_socket.recv(recv_size) if not total_data: if len(sock_data)>4: size_data+=sock_data size=struct.unpack('>i', size_data[:4])[0] recv_size=size if recv_size>524288:recv_size=524288 total_data.append(size_data[4:]) else: size_data+=sock_data else: total_data.append(sock_data) total_len=sum([len(i) for i in total_data ]) return ''.join(total_data) ############## def start_server(recv_type=''): sock=socket.socket(socket.AF_INET,socket.SOCK_STREAM) sock.bind(('',Port)) sock.listen(5) print 'started on',Port while True: newsock,address=sock.accept() print 'connected' if recv_type=='size': result=recv_size(newsock) elif recv_type=='end': result=recv_end(newsock) elif recv_type=='timeout': result=recv_timeout(newsock) else: result=newsock.recv(8192) print 'got',result if __name__=='__main__': #start_server() #start_server(recv_type='size') #start_server(recv_type='timeout') start_server(recv_type='end') def send_size(data): sock=socket.socket(socket.AF_INET,socket.SOCK_STREAM) sock.connect(('localhost',Port)) sock.sendall(struct.pack('>i', len(data))+data) sock.close() def send_end(data): sock=socket.socket(socket.AF_INET,socket.SOCK_STREAM) sock.connect(('localhost',Port)) sock.sendall(data+End) sock.close()
Discussion:
I employ a trivial server, to keep this as simple as possible. Just uncomment the type of receiving server you want to use to see the recv type chosen in action.
The recv_timeout function, which uses non-blocking sockets, will continue trying to get data as long as the client manages to even send a single byte. This is useful for moving data which you know very little about (like encrypted data), so cannot check for
completion in a sane way.
The recv_end function tacks on an end marker understood by both the client and the server. One problem with this is that data cannot look like the marker.
The recv_size function looks for the size of the payload to be prepended to the data. I use a fixed length, 4 bytes. So then I know to always look for that. It is packed to allow a terse representation that can fit sys.maxint. This avoids the problem of data
looking like the marker, however, even if it means a lot of data, you are limited w/the payload being the maximum number that can be packed. An interesting advantage of this method is that it can allocate data in larger chunks since it knows this size of the
data ahead of time. For large streams of data, I saw it increase performace by 10 times.
To test this, in a another process, try using the functions that match with the server type.
send_size(data) #for recv_size
send_end(data) #for recv_end
sock.sendall(data) #for timeout or simple recv(8192)
#do not forget to close if you do a raw sendall
sock.close()
相关文章推荐
- Android Call requires API level 19 (current min is 11)
- 人人都是产品经理——随记(1)
- linux主机如何添加swap分区
- python第三方库推荐之tablib(用于导出数据到不同的文件格式)
- 贪吃蛇之二维数组实现
- 1025. PAT Ranking (25)
- 《Java编程思想》学习笔记4——集合容器
- 1025. PAT Ranking (25)
- IE开发人员工具手册
- 第二阶段站立会08
- 配置虚拟机支持USB设备的热插拔
- 关于 js 算术运算时精度问题
- Linux文件的默认权限与查找命令详解
- #ifdef 如何实现“与或”
- Linux文件的默认权限与查找命令详解
- SignalTap II应用小实例之触发位置
- IOS 给图片添加水印 打印文字
- python copy & deepcopy 区别
- python socket编程小解
- 如何在Bash Shell脚本中显示对话框