gzip网页数据解压之路(分享)
最近玩一个小程序,需要获取网页数据来分析,结果其网站只提供gzip压缩数据。期间尝试把接收的数据去掉包头,写入gz文件,用winrar解压成功,但是没有找到vb调用winrar解压gzip文件的办法。后来下载了一个控件,虽然能解压成功但是只有30天试用期。用网上的vb解压gzip源码,却总是无法解压。
(有事了,待续……)
[最优解释]
End If
End Function
'解压缩数组
Public Function UnCompressByte(ByteArray() As Byte) As Boolean
Dim BufferSize As Long
Dim Buffer() As Byte
Dim lReturn As Long
Dim outUsed As Long
Dim inUsed As Long
'创建解压缩后的缓存
CopyMemory BufferSize, ByteArray(0), OFFSET
BufferSize = BufferSize + (BufferSize * 0.01) + 12
ReDim Buffer(BufferSize) As Byte
'创建解压缩进程
Dim contextHandle As Long: InitDecompression
CreateDecompression contextHandle, 1 '创建
'解压缩数据
lReturn = Decompress(ByVal contextHandle, ByteArray(0), UBound(ByteArray) + 1, Buffer(0), BufferSize, inUsed, outUsed)
DestroyDecompression contextHandle
'删除重复的数据
ReDim Preserve ByteArray(0 To outUsed - 1)
CopyMemory ByteArray(0), Buffer(0), outUsed
End Function
Private Sub Winsock1_DataArrival(ByVal bytesTotal As Long)
Dim MyFile() As Byte, MyLong As Long
ReDim MyFile(bytesTotal - 1)
Winsock1.GetData MyFile
MyLong = UBound(ArrayByte)
ReDim Preserve ArrayByte(MyLong + bytesTotal)
CopyMemory ArrayByte(MyLong + 1), MyFile(0), bytesTotal
End Sub
Private Sub Form_Load()
ReDim ArrayByte(0) '初始化接收数组,winsock1连接前重置更好
End Sub
'还原数据
Private Sub Command1_Click()
Dim i As Long, Stmp As String
'处理包头和包尾
For i = 0 To UBound(ArrayByte)
'gzip流特征:回车,换行,ID1=31(0x1F) ID2=139(0x8B),采用前面3个连续的数据作为判断依据
If ArrayByte(i) = 31 Then
If ArrayByte(i - 2) = 13 And ArrayByte(i - 1) = 10 Then
Exit For
End If
End If
Next
CopyMemory ArrayByte(0), ArrayByte(i), UBound(ArrayByte) - i '去掉包头
ReDim Preserve ArrayByte(UBound(ArrayByte) - i - 8) '末尾8个字节不是gzip流,此处可能根据网页数据要多次尝试,末尾4字节肯定不属于gzip流
UnCompressByte ArrayByte '解压GZIP流
Stmp = Utf8ToUnicode(ArrayByte)
End Sub
[其他解释]
总结:我当时用只去掉包头的字节数组写入文件,通过winrar能打开它,但是调用shell命令失败,用解压字节数组的gzip.dll也失败,后来考虑是不是数据包处理有问题,于是去掉末尾的4字节(结束符:2个回车换行),也失败,不得已,把末尾的字节再去掉4个,居然成功了……
不知道是什么原因,估计还是对网络传输的了解不够吧。
