libcurl 使用的几个注意事项
2016-03-25 16:51
375 查看
注:libcurl 入门指南( the tutorial ): http://curl.haxx.se/libcurl/c/libcurl-tutorial.html
原文:
If you specify URL without protocol:// prefix, curl will attempt to guess what protocol you might want. It will then default to HTTP but try other protocols based on often-used host name prefixes. For example, for host names starting with "ftp." curl will assume
you want to speak FTP.
原文:
libcurl offers its own default internal callback that will take care of the data if you don't set the callback with CURLOPT_WRITEFUNCTION. It will then simply output the received data to stdout. You can have the default callback write the data to a different
file handle by passing a 'FILE *' to a file opened for writing with the CURLOPT_WRITEDATA option.
源代码中的实现:
这样,就能够少写一个回调函数了(喂,你是有多懒啊),示比例如以下
使用 CURLOPT_ERRORBUFFER 保存错误, buf_size=CURL_ERROR_SIZE
或使用 curl_easy_strerror(res) (感觉这个简便)
演示样例:
原文:
When using multiple threads you should set the CURLOPT_NOSIGNAL option to 1 for all handles. Everything will or might work fine except that timeouts are not honored during the DNS lookup - which you can work around by building libcurl with c-ares support. c-ares
is a library that provides asynchronous name resolves. On some platforms, libcurl simply will not function properly multi-threaded unless this option is set.
对于 CURLOPT_TIMEOUT(默认0), CURLOPT_CONNECTTIMEOUT(默认300)选项:
In unix-like systems, this might cause signals to be used unless CURLOPT_NOSIGNAL is set.
原文:
There's one golden rule when these things occur: set the CURLOPT_VERBOSE option to 1. It'll cause the library to spew out the entire protocol details it sends, some internal info and some received protocol data as well (especially when using FTP). If you're
using HTTP, adding the headers in the received output to study is also a clever way to get a better understanding why the server behaves the way it does. Include headers in the normal body output with CURLOPT_HEADER set 1.
经试验:
设置 curl_easy_setopt(curl, CURLOPT_HEADER, 1L) 后,回调函数会返回 http头相关信息(原本是直接输出到stdout的)。考虑到还要过滤这些信息,所以还是不要设置这个了
原文:
Using POST with HTTP 1.1 implies the use of a "Expect: 100-continue" header. You can disable this header with CURLOPT_HTTPHEADER as usual.
解释:
当使用libcurl的POST方式时,假设POST数据的大小大于1024个字节,libcurl不会直接发送POST请求,而是会分为两步运行请求:
<1> 发送一个请求,该请求头部包括一个Expect: 100-continue的字段,用来询问server是否愿意接受数据
<2> 当接收到从server返回的100-continue的应答后,它才会真正的发起POST请求。将数据发送给server。
对于“100-continue"这个字段,RFC文档(http://www.w3.org/Protocols/rfc2616/rfc2616-sec8.html#sec8.2.3)是这么解释的:
它能够让client在发送请求数据之前去推断server是否愿意接收该数据。假设server愿意接收,client才会真正发送数据,
这么做的原因是假设client直接发送请求数据,可是server又将该请求拒绝的话。这样的行为将带来非常大的资源开销。
所以为了避免这样的情况,libcurl在发送大于1024字节的POST请求时採用了这样的方法,可是相对的,它会引起请求延迟的加大,
另外并非全部的server都会正确处理而且应答”100-continue“,比方lighttpd,就会返回417”Expectation Failed“,造成请求逻辑出错。
解决的方法:
原因:
Your callback function should return the number of bytes it "took care of". If that is not the exact same amount of bytes that was passed to it, libcurl will abort the operation and return with an error code.
假设回调函数中接收的数据有误,个人感觉能够返回0或者返回你已经处理的数据数。
由于源代码的处理例如以下:
0. 为使用的curl url 加入确定的协议头
原文:If you specify URL without protocol:// prefix, curl will attempt to guess what protocol you might want. It will then default to HTTP but try other protocols based on often-used host name prefixes. For example, for host names starting with "ftp." curl will assume
you want to speak FTP.
1. 把 curl_easy_perform() 回调数据直接写到文件里(FILE *)
原文:libcurl offers its own default internal callback that will take care of the data if you don't set the callback with CURLOPT_WRITEFUNCTION. It will then simply output the received data to stdout. You can have the default callback write the data to a different
file handle by passing a 'FILE *' to a file opened for writing with the CURLOPT_WRITEDATA option.
源代码中的实现:
这样,就能够少写一个回调函数了(喂,你是有多懒啊),示比例如以下
FILE *fp; fp = fopen("/root/test.bmp", "wb"); ... curl_easy_setopt(curl, CURLOPT_WRITEDATA, fp); ... fclose(fp);
2. curl_easy_perform 返回值处理
使用 CURLOPT_ERRORBUFFER 保存错误, buf_size=CURL_ERROR_SIZE或使用 curl_easy_strerror(res) (感觉这个简便)
演示样例:
/* Perform the request, res will get the return code */ res = curl_easy_perform(curl); /* Check for errors */ if(res != CURLE_OK) { printf("%s curl_easy_perform() error! \n", __FUNCTION__); printf("error msg = %s\n", curl_easy_strerror(res)); curl_easy_cleanup(curl); return -1; }
3. 多线程环境配置 CURLOPT_NOSIGNAL
原文:When using multiple threads you should set the CURLOPT_NOSIGNAL option to 1 for all handles. Everything will or might work fine except that timeouts are not honored during the DNS lookup - which you can work around by building libcurl with c-ares support. c-ares
is a library that provides asynchronous name resolves. On some platforms, libcurl simply will not function properly multi-threaded unless this option is set.
对于 CURLOPT_TIMEOUT(默认0), CURLOPT_CONNECTTIMEOUT(默认300)选项:
In unix-like systems, this might cause signals to be used unless CURLOPT_NOSIGNAL is set.
4. 设置 CURLOPT_VERBOSE、CURLOPT_HEADER 的必要性
原文:There's one golden rule when these things occur: set the CURLOPT_VERBOSE option to 1. It'll cause the library to spew out the entire protocol details it sends, some internal info and some received protocol data as well (especially when using FTP). If you're
using HTTP, adding the headers in the received output to study is also a clever way to get a better understanding why the server behaves the way it does. Include headers in the normal body output with CURLOPT_HEADER set 1.
经试验:
设置 curl_easy_setopt(curl, CURLOPT_HEADER, 1L) 后,回调函数会返回 http头相关信息(原本是直接输出到stdout的)。考虑到还要过滤这些信息,所以还是不要设置这个了
5. curl post 注意事项
原文:Using POST with HTTP 1.1 implies the use of a "Expect: 100-continue" header. You can disable this header with CURLOPT_HTTPHEADER as usual.
解释:
当使用libcurl的POST方式时,假设POST数据的大小大于1024个字节,libcurl不会直接发送POST请求,而是会分为两步运行请求:
<1> 发送一个请求,该请求头部包括一个Expect: 100-continue的字段,用来询问server是否愿意接受数据
<2> 当接收到从server返回的100-continue的应答后,它才会真正的发起POST请求。将数据发送给server。
对于“100-continue"这个字段,RFC文档(http://www.w3.org/Protocols/rfc2616/rfc2616-sec8.html#sec8.2.3)是这么解释的:
它能够让client在发送请求数据之前去推断server是否愿意接收该数据。假设server愿意接收,client才会真正发送数据,
这么做的原因是假设client直接发送请求数据,可是server又将该请求拒绝的话。这样的行为将带来非常大的资源开销。
所以为了避免这样的情况,libcurl在发送大于1024字节的POST请求时採用了这样的方法,可是相对的,它会引起请求延迟的加大,
另外并非全部的server都会正确处理而且应答”100-continue“,比方lighttpd,就会返回417”Expectation Failed“,造成请求逻辑出错。
解决的方法:
// POST数据的大于1024个字节 struct curl_slist *headerlist = NULL; static const char buf[] = "Expect:"; headerlist = curl_slist_append(headerlist, buf); /* initalize custom header list */ curl_easy_setopt(curl, CURLOPT_HTTPHEADER, headerlist); /* set header*/ curl_slist_free_all(headerlist); /* free slist */
6. 回调函数的正确返回
return (size * nmemb);
原因:
Your callback function should return the number of bytes it "took care of". If that is not the exact same amount of bytes that was passed to it, libcurl will abort the operation and return with an error code.
假设回调函数中接收的数据有误,个人感觉能够返回0或者返回你已经处理的数据数。
由于源代码的处理例如以下:
/* If the previous block of data ended with CR and this block of data is just a NL, then the length might be zero */ // len 为要发送给回调函数的数据长度 if(len) { wrote = data->set.fwrite_func(ptr, 1, len, data->set.out); } else { wrote = len; } if(wrote != len) { failf(data, "Failed writing body (%zu != %zu)", wrote, len); return CURLE_WRITE_ERROR; }
相关文章推荐
- 第三篇——软件之殇,WE ARE THOUSANDS APART!
- 3. 文件系统管理
- 2. 操作系统引导
- view 上推效果
- .Net使用Redis详解之ServiceStack.Redis(七)
- word2016去除回车符和换行符的方法
- 使用EditText在不同场景下调用软键盘的总结
- MAVEN pom.xml 文件详解,整理学习中··················
- word 文档中嵌入和链接OLE 对象|Packager-对象包装程序
- 静态库的使用方法
- Android 如何找到最匹配资源
- Example of Fixed Asset Service Operations
- 1. 用U盘安装Centos6.5 + Win7 双系统
- VS2008 error PRJ0002 : 错误的结果 31 (从“C:\Program Files\Microsoft SDKs\Windows\v6.0A\bin\rc.exe”返回)。
- linux tasklet和workqueue的应用情景
- Java 加解密技术系列之 PBE RSA AES DH 3DES DES HMAC SHA MD5 BASE64
- PHP7 扩展之自动化测试
- leetcode-24 Swap Nodes in Pairs
- Android 开源组件无限循环ViewPager
- 欢迎使用CSDN-markdown编辑器