网站后台登陆验证码不对,农村基本制度建设网站,手机百度账号登录个人中心,泸州大浪科技做网站上一篇文章我们已经介绍过使用selenium获取优惠券基本信息的方法
(15条消息) 【java爬虫】使用selenium爬取优惠券_haohulala的博客-CSDN博客
本文将在上一篇文章的基础上更进一步#xff0c;获取每个优惠券的淘口令#xff0c;毕竟我们只有复制淘口令才能在APP里面获取优惠…上一篇文章我们已经介绍过使用selenium获取优惠券基本信息的方法
(15条消息) 【java爬虫】使用selenium爬取优惠券_haohulala的博客-CSDN博客
本文将在上一篇文章的基础上更进一步获取每个优惠券的淘口令毕竟我们只有复制淘口令才能在APP里面获取优惠券嘛
获取淘口令的方法也是非常简单对于首页的每一张优惠券卡片都需要点击进入详细信息页面然后获取淘口令
程序运行起来后的效果如下图所示 先来看一下我们要收集的信息
Data
AllArgsConstructor
NoArgsConstructor
public class GoodItem {// 优惠券标题private String title;// 图片urlprivate String imgUrl;// 券前价格private Double prePrice;// 券后价格private Double postPrice;// 佣金率private Double commissionRate;// 佣金private Double commission;// 口令private String recommend;}
其实相比于上一篇文章本文的重点就是打开一个新的页面然后切换到新的页面然后获取完淘口令后再关闭这个新的页面不断重复这个动作。关闭新打开的页面的代码如下其实selenium就是按照句柄去找页面的句柄就是一串字符串我们首先切换到新打开的页面关闭后需要切换回首页 // 关闭当前窗口public void closeWindow(WebDriver driver) {// 获取所有句柄的集合ListString winHandles new ArrayList(driver.getWindowHandles());driver.switchTo().window((String) winHandles.get(1));driver.close();driver.switchTo().window((String) winHandles.get(0));} 完整的爬虫代码如下
Slf4j
Service
public class SeleniumPlusServiceImpl implements SeleniumPlusService {Overridepublic ListGoodItem startSpider() {// 加载chrome驱动System.setProperty(webdriver.chrome.driver, SeleniumUtils.DRIVER_PATH);ChromeOptions options new ChromeOptions();options.setExperimentalOption(debuggerAddress, 127.0.0.1:9222);// 启动浏览器WebDriver driver new ChromeDriver(options);// 设置最长等待时间driver.manage().timeouts().implicitlyWait(10, TimeUnit.SECONDS);// 实例化一个列表存放数据ListGoodItem rstList new ArrayList();// 开始遍历卡片数据// 遍历100组数据暂停for (int i 0; i 100; ) {WebElement element driver.findElement(By.className(GoodsList__CardList-sc-84so0w-1));ListWebElement divList element.findElements(By.className(union-good-card-wrap));log.info(获取 divList.size() 个优惠券卡片);for (int j 0; j divList.size(); j) {GoodItem item new GoodItem();// 图片urlitem.setImgUrl(divList.get(j).findElement(By.className(union-good-card-good-img-wrap-mediumn)).findElement(By.tagName(a)).findElement(By.tagName(img)).getDomAttribute(src));// 优惠券标题item.setTitle(divList.get(j).findElement(By.className(union-good-card-title)).findElement(By.tagName(span)).getText());// 券前价格item.setPrePrice(getPrice(divList.get(j).findElement(By.className(union-good-card-coupon-reserve-price-mediumn))));// 券后价格item.setPostPrice(getPrice(divList.get(j).findElement(By.className(union-good-card-coupon-final-price))));ListWebElement commissionList divList.get(j).findElements(By.className(union-good-card-commision-info-item));// 佣金率item.setCommissionRate(getPrice(commissionList.get(0)));// 佣金item.setCommission(getPrice(commissionList.get(1)));// 点击图片产生新的页面divList.get(j).findElement(By.tagName(img)).click();// 切换到新的页面ListString handleList new ArrayList(driver.getWindowHandles());driver.switchTo().window(handleList.get(1));// 获取优惠券文字StringBuilder sb new StringBuilder();ListWebElement divsList driver.findElement(By.className(ChatText__ChatTextContainer-sc-hl576v-0)).findElements(By.tagName(div));for(int k0; kdivsList.size(); k) {sb.append(divsList.get(k).getText() \n);}item.setRecommend(sb.toString());log.info(JSON.toJSONString(item));// 关闭页面closeWindow(driver);rstList.add(item);i;if(i 30) {log.info(30条数据获取完毕);return rstList;}}// 切换到下一页driver.findElement(By.className(GoodsList__Pagination-sc-84so0w-2)).findElement(By.className(mux-pagination-icon-next)).click();log.info(进入到下一页);try {Thread.sleep(5000);} catch (InterruptedException e) {e.printStackTrace();}}return rstList;}// 获取券前券后价格private Double getPrice(WebElement element) {StringBuilder sb new StringBuilder();sb.append(element.findElement(By.className(union-number-format-integer)).getText().replaceAll(,, ));sb.append(element.findElement(By.className(union-number-format-pointer)).getText());sb.append(element.findElement(By.className(union-number-format-decimal)).getText());Double price Double.parseDouble(sb.toString());return price;}// 关闭当前窗口public void closeWindow(WebDriver driver) {// 获取所有句柄的集合ListString winHandles new ArrayList(driver.getWindowHandles());driver.switchTo().window((String) winHandles.get(1));driver.close();driver.switchTo().window((String) winHandles.get(0));}}我们写一个接口来触发爬虫并且展示最终获取的数据
Controller
public class HelloController {Autowiredprivate SeleniumPlusService seleniumPlusService;RequestMapping(/start)ResponseBodypublic String startSpider() {ListGoodItem goodItems seleniumPlusService.startSpider();return JSON.toJSONString(goodItems);}}最后程序会获取30条优惠券数据通过json格式返回
[{commission: 0.67,commissionRate: 1.35,imgUrl: //img.alicdn.com/bao/uploaded/O1CN01oDxvso1iw3dVd7jec_!!6000000004476-0-yinhe.jpg,postPrice: 49.9,prePrice: 84.9,recommend: 蒙牛特仑苏纯牛奶250ml*16盒整箱学生早餐奶高端新旧包装混发 82.90元\n库存紧张\n这价位能做到这样真的无可挑剔\nh:/19 CZ0001 dJv9dGcWYdV\n,title: 蒙牛特仑苏纯牛奶250ml*16盒整箱学生早餐奶高端新旧包装混发
}, {commission: 1.6,commissionRate: 1.8,imgUrl: //img.alicdn.com/bao/uploaded/i4/2200828292428/O1CN01zcZrOk1To4VKJIagH_!!0-item_pic.jpg,postPrice: 89.0,prePrice: 118.0,recommend: 逐本清欢晨蜜自在自然植萃卸妆油敏弱肌脸部舒缓深层清洁卸妆水膏 89.00元\n赠运费险\n家人们赶紧动手吧\nh:/08 CZ3457 mH6bdGcdTJA\n,title: 逐本清欢晨蜜自在自然植萃卸妆油敏弱肌脸部舒缓深层清洁卸妆水膏
}, {commission: 0.69,commissionRate: 1.35,imgUrl: //img.alicdn.com/bao/uploaded/O1CN01ry5fh31G8llXLIPuR_!!6000000000578-0-yinhe.jpg,postPrice: 50.9,prePrice: 74.4,recommend: 蒙牛纯牛奶全脂灭菌乳250ml*24盒/1箱学生营养早餐搭配优质乳蛋白 71.40元\n好评过十万\n宝宝们抓紧入手呀\nh:/88 CZ3457 vZtHdGcdZOR\n,title: 蒙牛纯牛奶全脂灭菌乳250ml*24盒/1箱学生营养早餐搭配优质乳蛋白
}, {commission: 2.66,commissionRate: 4.5,imgUrl: //img.alicdn.com/bao/uploaded/i3/2929053804/O1CN01HhUYJz1dyHUbxYY7G_!!0-item_pic.jpg,postPrice: 42.67,prePrice: 79.9,recommend: Mistine小黄帽防晒霜乳SPF50泰版面部防紫外线隔离男女军训蜜丝婷 67.90元\n赠运费险\n超级实惠赶快囤起来\nh:/08 CZ0001 BEXOdGcWnM8\n,title: Mistine小黄帽防晒霜乳SPF50泰版面部防紫外线隔离男女军训蜜丝婷
}, {commission: 1.59,commissionRate: 2.7,imgUrl: //img.alicdn.com/bao/uploaded/i2/4144020062/O1CN0131RjyR1CKRPkpRqQz_!!0-item_pic.jpg,postPrice: 38.9,prePrice: 79.0,recommend: Colorkey珂拉琪水雾唇露唇釉女唇彩口红镜面水光大牌正品不易沾杯 67.00元\n赠运费险\n质量逆天赶紧的闭眼买都不亏\nh:/59 CZ3457 raiDdGcdyLJ\n,title: Colorkey珂拉琪水雾唇露唇釉女唇彩口红镜面水光大牌正品不易沾杯
}, {commission: 1.43,commissionRate: 6.0,imgUrl: //img.alicdn.com/bao/uploaded/i1/2204174570383/O1CN01ntyGTr1EhSVquROow_!!0-item_pic.jpg,postPrice: 23.9,prePrice: 33.9,recommend: 螺满地广西螺蛳粉300g整箱装 柳州正宗速食袋装螺狮粉螺丝粉 23.90元\n今日热销4百\n这价位能做到这样真的无可挑剔\nh:/17 CZ0001 JztJdGcWOHd\n,title: 螺满地广西螺蛳粉300g整箱装 柳州正宗速食袋装螺狮粉螺丝粉
}, {commission: 2.09,commissionRate: 6.0,imgUrl: //img.alicdn.com/bao/uploaded/O1CN014lnCRY1GepVRnaFhB_!!6000000000648-0-yinhe.jpg,postPrice: 34.9,prePrice: 41.0,recommend: Veet薇婷脱毛膏女腋下男女士学生专用非全身非私密不永久去毛神器 39.00元\n今日热销4百\n买它超值\nh:/17 CZ0001 0rh1dGcdmfR\n,title: Veet薇婷脱毛膏女腋下男女士学生专用非全身非私密不永久去毛神器
}, {commission: 2.3,commissionRate: 7.2,imgUrl: //img.alicdn.com/bao/uploaded/O1CN013l8dxo1Ub4qSj8ie8_!!6000000002535-0-yinhe.jpg,postPrice: 31.9,prePrice: 99.9,recommend: 【罗永浩直播专享】Veet/薇婷净纯脱毛膏温和型200ml娇嫩肌肤使用 94.90元\n近期热门\n速速抢购手快有手慢无\nh:/88 CZ0001 JSyndGcWiWS\n,title: 【罗永浩直播专享】Veet/薇婷净纯脱毛膏温和型200ml娇嫩肌肤使用
}, {commission: 9.48,commissionRate: 6.0,imgUrl: //img.alicdn.com/bao/uploaded/O1CN01Mx7Wn51tYf7s8hSiN_!!6000000005914-0-yinhe.jpg,postPrice: 158.0,prePrice: 198.0,recommend: 哈药牌钙铁锌口服液蓝瓶液体钙儿童补钙铁葡萄糖酸锌溶液90支 158.00元\n月销量过千\n别犹豫了新券上次没抢到抓紧\nh:/97 CZ3457 DG61dGcWpPz\n,title: 哈药牌钙铁锌口服液蓝瓶液体钙儿童补钙铁葡萄糖酸锌溶液90支
}, {commission: 0.68,commissionRate: 1.5,imgUrl: //img.alicdn.com/bao/uploaded/i2/2142811280/O1CN01YXCLuF1LKHp3KvAwP_!!0-item_pic.jpg,postPrice: 45.3,prePrice: 55.9,recommend: 心相印茶香抽纸纸巾家用实惠装抽纸整箱100抽24包擦手纸卫生纸 50.90元\n超十万人种草\n喜欢的宝宝们千万不要错过哦~赶紧买起来买起来~\nh:/17 CZ0001 bSyfdGcdnEK\n,title: 心相印茶香抽纸纸巾家用实惠装抽纸整箱100抽24包擦手纸卫生纸
}, {commission: 5.94,commissionRate: 6.0,imgUrl: //img.alicdn.com/bao/uploaded/O1CN01LD9Y1B1IN5HIZtPpq_!!6000000000880-0-yinhe.jpg,postPrice: 99.0,prePrice: 184.0,recommend: 伊利中老年成人高钙低脂冲饮早餐奶粉850g*2罐礼盒装含膳食纤维 164.00元\n月销量过千\n这么大的福利赶紧抢哦~\nh:/59 CZ0001 euTpdGcWJXW\n,title: 伊利中老年成人高钙低脂冲饮早餐奶粉850g*2罐礼盒装含膳食纤维
}, {commission: 6.58,commissionRate: 5.1,imgUrl: //img.alicdn.com/bao/uploaded/i2/6000000004110/O1CN01PgHA3D1gEQghcsWH2_!!6000000004110-0-sm.jpg,postPrice: 129.0,prePrice: 199.0,recommend: 麦富迪猫粮营养森林成猫粮10kg通用型英美短蓝猫全价营养主食20斤 179.00元\n热趋商品\n不要犹豫库存不多抓紧抢\nh:/58 CZ3457 WnabdGcWl43\n,title: 麦富迪猫粮营养森林成猫粮10kg通用型英美短蓝猫全价营养主食20斤
}, {commission: 1.4,commissionRate: 7.06,imgUrl: //img.alicdn.com/bao/uploaded/O1CN01JhbEad1W7QLZwW2vP_!!6000000002741-0-yinhe.jpg,postPrice: 19.9,prePrice: 44.9,recommend: 德国拜耳拜灭士蟑螂药一窝全窝端家用非无毒灭绝杀除驱蟑胶饵神器 24.90元\n热门商品\n现在买就是最划算的别等了想要现在就买\nh:/38 CZ3457 Q44adGcWkKo\n,title: 德国拜耳拜灭士蟑螂药一窝全窝端家用非无毒灭绝杀除驱蟑胶饵神器
}, {commission: 5.66,commissionRate: 5.4,imgUrl: //img.alicdn.com/bao/uploaded/O1CN01tQyn5R1JrbNa55WVu_!!6000000001082-0-yinhe.jpg,postPrice: 104.9,prePrice: 109.9,recommend: 清扬男士去屑清爽控油头皮护理青柠薄荷醇洗发水(500X3200)G 104.90元\n本月上新\n这价位能做到这样真的无可挑剔\nh:/89 CZ3457 Gf7AdGcdpl0\n,title: 清扬男士去屑清爽控油头皮护理青柠薄荷醇洗发水(500X3200)G
}, {commission: 1.74,commissionRate: 5.3,imgUrl: //img.alicdn.com/bao/uploaded/O1CN01nW1OxA1d2ZVMdRCnQ_!!6000000003678-0-yinhe.jpg,postPrice: 32.9,prePrice: 69.9,recommend: 立白Liby除菌洗衣凝珠8G*40颗 2盒装浓缩除螨留香洗衣球 61.90元\n超万人正在疯抢\n超级实惠赶快囤起来\nh:/89 CZ0001 ndp5dGcWieh\n,title: 立白Liby除菌洗衣凝珠8G*40颗 2盒装浓缩除螨留香洗衣球
}, {commission: 5.66,commissionRate: 5.4,imgUrl: //img.alicdn.com/bao/uploaded/O1CN01bfuGxM1U3dxr5hgQb_!!6000000002462-0-yinhe.jpg,postPrice: 104.9,prePrice: 109.9,recommend: 清扬去屑清樱花露水润净屑留香樱花头皮洗发水500g*3 104.90元\n月销量过千\n速速抢购手快有手慢无\nh:/29 CZ0001 9TkMdGcWdCf\n,title: 清扬去屑清樱花露水润净屑留香樱花头皮洗发水500g*3
}, {commission: 1.06,commissionRate: 3.53,imgUrl: //img.alicdn.com/bao/uploaded/O1CN01eHHDyd1QshemXN1t8_!!6000000002032-0-yinhe.jpg,postPrice: 27.9,prePrice: 34.4,recommend: 德佑婴儿隔尿垫一次性防水透气不可洗新生宝宝护理垫大童尺寸夏季 33.40元\n赠运费险\n家人们速度撸起来\nh:/19 CZ3457 JRrDdGcdsIE\n,title: 德佑婴儿隔尿垫一次性防水透气不可洗新生宝宝护理垫大童尺寸夏季
}, {commission: 2.21,commissionRate: 4.5,imgUrl: //img.alicdn.com/bao/uploaded/O1CN01gO1IfQ1ljdhW0a0LT_!!6000000004855-0-yinhe.jpg,postPrice: 49.1,prePrice: 69.0,recommend: 蕉下修容口罩EM320 防晒护眼角开车面罩防紫外线立体夏女透气户外 56.00元\n好评过万\n超级实惠赶快囤起来\nh:/18 CZ3457 8VrgdGcW7ym\n,title: 蕉下修容口罩EM320 防晒护眼角开车面罩防紫外线立体夏女透气户外
}, {commission: 4.69,commissionRate: 5.4,imgUrl: //img.alicdn.com/bao/uploaded/O1CN01DRWnk31Sl2B9AS6Or_!!6000000002286-0-yinhe.jpg,postPrice: 86.9,prePrice: 124.9,recommend: 金纺薰衣草衣物衣服护理剂柔顺护形护衣防静电香氛护理剂4KG*2 121.90元\n同类高佣\n千万不要错过趁着划算囤起来\nh:/58 CZ3457 IKD0dGcdGMv\n,title: 金纺薰衣草衣物衣服护理剂柔顺护形护衣防静电香氛护理剂4KG*2
}, {commission: 2.96,commissionRate: 5.4,imgUrl: //img.alicdn.com/bao/uploaded/O1CN01gxJci41mNwWjs4hRY_!!6000000004943-0-yinhe.jpg,postPrice: 54.9,prePrice: 79.9,recommend: 金纺纯净温和护理剂护形护色少皱褶去除静电 持久留香5KG衣物洗衣 74.90元\n同类热推\n这价格真的得靠抢了捂脸\nh:/59 CZ0001 lP5HdGcdw5n\n,title: 金纺纯净温和护理剂护形护色少皱褶去除静电 持久留香5KG衣物洗衣
}, {commission: 0.54,commissionRate: 1.35,imgUrl: //img.alicdn.com/bao/uploaded/O1CN01xGKGuv1Sym8WcDyk9_!!6000000002316-0-yinhe.jpg,postPrice: 39.9,prePrice: 49.9,recommend: 舒肤佳红石榴山茶花净透泡沫沐浴露持久留香啫喱沐浴乳液品牌男女 44.90元\n好评过万\n家人们速度撸起来\nh:/89 CZ0001 N7B9dGcWthN\n,title: 舒肤佳红石榴山茶花净透泡沫沐浴露持久留香啫喱沐浴乳液品牌男女
}, {commission: 8.99,commissionRate: 15.0,imgUrl: //img.alicdn.com/bao/uploaded/O1CN01hCQXtN1Oc5yax1WHF_!!6000000001725-0-yinhe.jpg,postPrice: 7.15,prePrice: 79.9,recommend: 【零感003】杰士邦避孕套正品官方旗舰店安全套超薄男用裸入持久1 67.90元\n热趋商品\n性价比chao~高\nh:/69 CZ0001 AsxadGcdryJ\n,title: 【零感003】杰士邦避孕套正品官方旗舰店安全套超薄男用裸入持久1
}, {commission: 0.25,commissionRate: 1.5,imgUrl: //img.alicdn.com/bao/uploaded/O1CN01emexE71wD5Cn7WzdN_!!6000000006273-0-yinhe.jpg,postPrice: 16.4,prePrice: 26.0,recommend: 防蚊门帘磁性魔术贴夏季纱门纱窗家用蚊帐隔断自吸磁铁全磁条自粘 19.00元\n近期热门\n快~少量库存赶紧冲拼手速\nh:/39 CZ0001 CTQNdGcWC8w\n,title: 防蚊门帘磁性魔术贴夏季纱门纱窗家用蚊帐隔断自吸磁铁全磁条自粘
}, {commission: 1.96,commissionRate: 5.3,imgUrl: //img.alicdn.com/bao/uploaded/O1CN01Qds2np1iFv82n4XOe_!!6000000004384-0-yinhe.jpg,postPrice: 36.9,prePrice: 39.9,recommend: 臭宝方便速食柳州螺蛳粉浓汤330g*3广西大片腐竹 36.90元\n本周热门\n这价格真的得靠抢了捂脸\nh:/47 CZ3457 ziGbdGcdF3h\n,title: 臭宝方便速食柳州螺蛳粉浓汤330g*3广西大片腐竹
}, {commission: 3.06,commissionRate: 10.5,imgUrl: //img.alicdn.com/bao/uploaded/O1CN01siAhJN1Hwyo2vfWAr_!!6000000000823-0-yinhe.jpg,postPrice: 29.1,prePrice: 49.0,recommend: 猫人抗菌裆男士内裤男冰丝无痕四角裤运动纯棉裆平角大码裤青少年 34.00元\n近期热门\n现在买就是最划算的别等了想要现在就买\nh:/27 CZ0001 HPBTdGcWNiV\n,title: 猫人抗菌裆男士内裤男冰丝无痕四角裤运动纯棉裆平角大码裤青少年
}, {commission: 4.69,commissionRate: 5.4,imgUrl: //img.alicdn.com/bao/uploaded/O1CN01MDqzyl1Lwl0XEKtrp_!!6000000001364-0-yinhe.jpg,postPrice: 86.9,prePrice: 124.9,recommend: 金纺清新柔顺衣物衣服护理剂柔顺剂柔软剂洗衣液伴侣防静电4KG*2 121.90元\n本月上新\n买过的都说好趁着活动多备货\nh:/29 CZ0001 lFT9dGcWLZb\n,title: 金纺清新柔顺衣物衣服护理剂柔顺剂柔软剂洗衣液伴侣防静电4KG*2
}, {commission: 1.58,commissionRate: 5.3,imgUrl: //img.alicdn.com/bao/uploaded/O1CN01ubdFPs1zBCYQ99IsK_!!6000000006675-0-yinhe.jpg,postPrice: 29.9,prePrice: 59.9,recommend: 冷酸灵固齿优护抗敏感清火牙膏100g*3支极地白130g*1支护理套装 44.90元\n库存紧张\n不要犹豫库存不多抓紧抢\nh:/97 CZ3457 VzRxdGcdMyP\n,title: 冷酸灵固齿优护抗敏感清火牙膏100g*3支极地白130g*1支护理套装
}, {commission: 1.19,commissionRate: 7.06,imgUrl: //img.alicdn.com/bao/uploaded/i3/2940753275/O1CN01Hn7Q0K1a3zy8JkB0J_!!0-item_pic.jpg,postPrice: 16.8,prePrice: 24.3,recommend: 德佑湿厕纸家庭实惠装洁厕湿巾私处擦屁屁男女士专用湿厕巾40抽 19.30元\n库存紧张\n别犹豫了新券上次没抢到抓紧\nh:/59 CZ0001 TKr1dGcWao1\n,title: 德佑湿厕纸家庭实惠装洁厕湿巾私处擦屁屁男女士专用湿厕巾40抽
}, {commission: 5.27,commissionRate: 6.0,imgUrl: //img.alicdn.com/bao/uploaded/O1CN01AIr8qk1X1l5Imjz5t_!!6000000002864-0-yinhe.jpg,postPrice: 87.9,prePrice: 89.9,recommend: 多芬/DOVE樱花甜香滋养美肤沐浴乳沐浴露1KGx2190ml 家庭装 87.90元\n月销量过万\n宝宝们抓紧入手呀\nh:/27 CZ0001 GCIidGcWUQO\n,title: 多芬/DOVE樱花甜香滋养美肤沐浴乳沐浴露1KGx2190ml 家庭装
}, {commission: 1.09,commissionRate: 9.0,imgUrl: //img.alicdn.com/bao/uploaded/i4/6000000005476/O1CN018iRuFX1qK3iNgSkCC_!!6000000005476-0-sm.jpg,postPrice: 12.14,prePrice: 75.0,recommend: 2023秋53天天练一二三四五六年级上册语文数学英语人教版同步训练 65.00元\n月销量过万\n喜欢的宝宝们千万不要错过哦~赶紧买起来买起来~\nh:/39 CZ3457 jn5DdGcWkrv\n,title: 2023秋53天天练一二三四五六年级上册语文数学英语人教版同步训练
}]
