网页数据采集
谁能提供一下网页数据下载采集思路？能否发一些通用代码
[解决办法]
Private Function GetWebData(ByVal strURL As String, ByVal strParam As String, ByVal intTimeout As Integer) As String
Static strRequestURL As String
Static strReturn As String
Dim wr As WebResponse
Dim req As HttpWebRequest
Dim reqStream As Stream
Dim readStream As StreamReader
Dim postBytes As Byte()
Dim byRemoteInfo As Byte()
If strReturn = Nothing OrElse strRequestURL <> strURL & strParam Then
req = HttpWebRequest.Create(strURL)
req.Method = "POST"
req.Timeout = intTimeout
req.ReadWriteTimeout = intTimeout
req.ContentType = "application/x-www-form-urlencoded"
postBytes = Encoding.ASCII.GetBytes(strParam)
req.ContentLength = postBytes.Length
reqStream = req.GetRequestStream()
reqStream.Write(postBytes, 0, postBytes.Length)
wr = req.GetResponse()
reqStream = wr.GetResponseStream
readStream = New StreamReader(reqStream, Encoding.Default)
strReturn = readStream.ReadToEnd()
strRequestURL = strURL & strParam
req.Abort()
reqStream.Close()
readStream.Close()
End If
Return strReturn
End Function
http://www.csharpwin.com/dotnetspace/6476r6408.shtml
http://www.2cto.com/kf/201106/93983.html
http://www.cnblogs.com/hfzsjz/archive/2010/01/26/1656722.html
[解决办法]
没试过这个方法！！我以前是使用vb XMLHTTP 的！！再通过Adodb.stream 转换！！效果不是很好
[解决办法]
底层一点的就用WebRequest，简单实现的就用WebClient，很简单
