网页抓取分享

VBA万岁

onlycxb 发表于 2014-10-14 19:16
分享5：阳光私募净值
后期数据处理过程请自学者自行完善。

ie法提取代码如下：

Sub 阳光募捐数据提取()
Cells.ClearContents
Dim p%, tp As Long, i%, j%, t, r As Object

With CreateObject("internetexplorer.application")
    .Visible = True
    .Navigate "http://www.go-goal.com/data/trust"
    Do Until .readyState = 4
        DoEvents
    Loop
   
    t = Timer
    Do While t + 2 > Timer
        DoEvents
    Loop
    tp = Split(Split(.Document.all.tags("span")(17).innerText, "共 ")(2), " 页")(0) '此句显示总页数
    Cells(1, 20) = tp
    For p = 1 To 2
        t = Timer
        Do While t + 2 > Timer
            DoEvents
        Loop
        
        Set r = .Document.all.tags("table")(10).Rows
        For i = 0 To r.Length - 1
            For j = 0 To r(i).Cells.Length - 1
                Cells((p - 1) * 21 + i + 1, j + 1) = r(i).Cells(j).innerText
            Next j
        Next i
   
        .Document.all.tags("a")(118).Click
    Next p
End With
End Sub

onlycxb

VBA万岁 发表于 2014-10-21 15:54
测试通过，多谢分享！
添加引用如59楼，
Mark如附件：

分享暂时到此，我去听课去了，地址。http://club.excelhome.net/thread-1159783-1-1.html

VBA万岁

VBA万岁 发表于 2014-10-21 16:44
ie法提取代码如下：

Sub 阳光募捐数据提取()

附件如下：
阳光募捐数据提取.zip (400.87 KB, 下载次数: 30)

2014-10-21 17:11 上传

点击文件名下载附件

VBA万岁

onlycxb 发表于 2014-10-21 17:10
分享暂时到此，我去听课去了，地址。http://club.excelhome.net/thread-1159783-1-1.html

多谢指点!
我也得进去听听课了......

VBA万岁

onlycxb 发表于 2014-10-14 07:57
分享2：抓取北京地铁1号线站名及换乘区间

onlycxb大侠，我有一个疑问：
我将1、3楼代码中的responseText数据分别复制到了两个Word文档（详见以下附件）中，结果发现，它们全是由英文字符及数字组成，而不像6楼的responseText数据那样直接含有待提取的数据，如“地铁1号线(四惠东-苹果园)”等中文字符串。
想知道你是如何通过 js.eval 而得到最终数据的？

responseText.zip (32.96 KB, 下载次数: 7)

2014-10-31 10:58 上传

点击文件名下载附件

onlycxb

VBA万岁 发表于 2014-10-31 10:56
onlycxb大侠，我有一个疑问：
我将1、3楼代码中的responseText数据分别复制到了两个Word文档中，结果发现 ...

复制的数据是不全的，建议将数据粘贴到记事本中。

onlycxb

分享的代码，一些测试分析过程的代码省略了。这个自己分析并获取才明白。

VBA万岁

onlycxb 发表于 2014-10-31 10:58
复制的数据是不全的，建议将数据粘贴到记事本中。

数据粘贴到记事本中也是一样只见英文字符串而不见中文。

onlycxb

Sub CopyToClipbox(strText As String)
    '文本拷贝到剪贴板
    With CreateObject("new:{1C3B4210-F441-11CE-B9EA-00AA006B1A69}")
        .SetText strText
        .PutInClipboard
    End With
End Sub

onlycxb

VBA万岁 发表于 2014-10-31 11:00
数据粘贴到记事本中也是一样只见英文字符串而不见中文。

debug.print .responsetext
数据如下，经过转码可得到想要的数据：
{"place_info":{"d_activity_gwj_section":"0-+","d_brand_id_section":"0-+","d_business_id":"","d_business_type":"","d_cater_book_pc_section":"0-+","d_cater_book_wap_section":"0-+","d_cater_rating_section":"0-+","d_data_type":"","d_discount2_section":"0-+","d_discount_section":"0-+","d_discount_tm2_section":"0-+","d_discount_tm_section":"0-+","d_dist":"0-+","d_filt_type_section":"0-+","d_free_section":"0-+","d_groupon_section":"0-+","d_groupon_type_section":"0-+","d_health_score_section":"0-+","d_hotel_book_pc_section":"0-+","d_hotel_book_wap_section":"0-+","d_level_section":"0-+","d_lowprice_flag_section":"0-+","d_movie_book_section":"0-+","d_overall_rating_section":"0-+","d_price_section":"0-+","d_query_attr_type":"4","d_rebate_section":"0-+","d_sort_rule":"0","d_sort_type":"","d_spothot_section":"0-+","d_sub_type":"","d_support_imax_section":"0-+","d_tag_filter":"0","d_tag_info_list":"\u4ea4\u901a\u8bbe\u65bd,\u5730\u94c1\u7ad9","d_ticket_book_flag_section":"0-+","d_tonight_sale_flag_section":"0-+","d_total_
score_section":"0-+","d_wise_price_section":"0-+","search_ext":[{"title":"","wd":""}]},"content":[{"acc_flag":1,"area":131,"biz_type":0,"catalogID":905,"cla":[[903,"\u516c\u4ea4\u7ebf\u8def"],[905,"<font color=\"#c60a00\">\u5730\u94c1<\/font>,\u8f7b\u8f68"]],"cp":"bus","detail":0,"dis":-1,"dist2route":0,"dist2start":0,"ext":[{"info":"","price_ext":"","time_ext":"","timetable_ext":"","update_time":"","username":""}],"ext_type":8,"f_flag":5,"father_son":0,"geo":"1|12971354.88,4825923.53;12971354.88,4825923.53|12971354.88,4825923.53;","geo_type":1,"name":"\u5730\u94c11\u53f7\u7ebf(\u56db\u60e0\u4e1c-\u82f9\u679c\u56ed)","poiType":4,"poi_profile":0,"primary_uid":"1557622","route_flag":0,"show_tag":[],"status":1,"storage_src":"index","tag":"","ty":1,"uid":"bce557d6f7fadd4ea5da39b7","view_type":0,"x":1297135488,"y":482592353},{"acc_flag":1,"area":131,"biz_type":0,"catalogID":905,"cla":[[903,"\u516c\u4ea4\u7ebf\u8def"],[905,"<font color=\"#c60a00\">\u5730\u94c1<\/font>,\u8f7b\u8f68"]],"cp":"bus","detail":0,"dis":-1
,"dist2route":0,"dist2start":0,"ext":[{"info":"","price_ext":"","time_ext":"","timetable_ext":"","update_time":"","username":""}],"ext_type":8,"f_flag":5,"father_son":0,"geo":"1|12933692.29,4828438.13;12933692.29,4828438.13|12933692.29,4828438.13;","geo_type":1,"name":"\u5730\u94c11\u53f7\u7ebf(\u82f9\u679c\u56ed-\u56db\u60e0\u4e1c)","poiType":4,"poi_profile":0,"primary_uid":"1557614","route_flag":0,"show_tag":[],"status":1,"storage_src":"index","tag":"","ty":1,"uid":"3e210d94b51b7a184107c1b7","view_type":0,"x":1293369229,"y":482843813},{"acc_flag":0,"addr":"\u56fd\u8d38\u5411\u4e1c\u4e24\u516c\u91cc\u5730\u94c1\u4e00\u53f7\u7ebf\u56db\u60e0\u81f3\u56db\u60e0\u4e1c\u7ad9\u4e4b\u95f4\u5730\u94c1\u4e0a\u65b9\u58f9\u7ebf\u56fd\u9645","alias":["\u9996\u5f00\u58f9\u7ebf\u56fd\u9645\u5546\u4e1a\u5e7f\u573a","\u58f9\u7ebf\u56fd\u9645(\u901a\u60e0\u5bb6\u56ed\u897f\u5357)","\u58f9\u7ebf\u56fd\u9645\u5546\u4e1a\u5e7f\u573a"],"aoi":"\u56db\u60e0","area":2898,"biz_type":0,"catalogID":238,"cla":[[25,"\u5730\u4ea7\u5c0f\
u533a"],[238,"\u5c0f\u533a\/\u697c\u76d8"]],"detail":1,"dis":-1,"dist2route":0,"dist2start":0,"ext"}}