您的位置:首页 > 理论基础 > 计算机网络

HttpWebRequest 抓取页面异常处理办法

2015-02-06 11:21 387 查看
抓取页面异常处理办法

public static string GetHtmlTest(string URI)
{
string fullhtml = null;
while (true)
{
try
{
HttpWebRequest req = (HttpWebRequest)WebRequest.Create(URI);
req.Method = "GET";
req.UserAgent = "Opera/9.25 (Windows NT 6.0; U; en)";
req.KeepAlive = true;
req.Timeout = 6000000;
HttpWebResponse resp = (HttpWebResponse)req.GetResponse();
if (resp.StatusCode != HttpStatusCode.OK) //如果服务器未响应,那么继续等待相应
continue;
StreamReader sr = new StreamReader(resp.GetResponseStream(), Encoding.UTF8);
fullhtml = sr.ReadToEnd().Trim();
resp.Close();
sr.Close();
break;
}
catch (WebException e)
{
e.StackTrace.ToString();
System.Diagnostics.Trace.WriteLine(e.Message);
if (true)
continue;
}
}
return fullhtml;
}
内容来自用户分享和网络整理,不保证内容的准确性,如有侵权内容,可联系管理员处理 点击这里给我发消息
标签: