良精企业网站系统,网站建设的需要分析,搭建网页代码流程,小制作小发明视频教程/* * Mapreduce步骤 * 1、map#xff1a;获取一行 * 2、按空格分隔 * 3、每个单词统计#xff0c;key,1 输出到 reducer * 4、reducer key,{1,1,1,1,1} 汇总 * 5、输出结果 */ 1、声明一个集合#xff0c;模仿要统计的… /* * Mapreduce步骤 * 1、map获取一行 * 2、按空格分隔 * 3、每个单词统计key,1 输出到 reducer * 4、reducer key,{1,1,1,1,1} 汇总 * 5、输出结果 */ 1、声明一个集合模仿要统计的文档内容一个逗号分隔的集合元素相当于一行
scala val lines List(hello tom hello jerry, hello jerry, hello kitty,beijing shanghai tianjing,xiaoming xiaohua xiaoli xiaozhang) 2、 map获取一行按空格分隔
scala val line2lines.flatMap(_.split( )) line2: List[String] List(hello, tom, hello, jerry, hello, jerry, hello, kitty, beijing, shanghai, tianjing, xiaoming, xiaohua, xiaoli, xiaozhang) 3、每个单词统计key,1 输出到 reducer
scala val line3line2.map((_,1)) line3: List[(String, Int)] List((hello,1), (tom,1), (hello,1), (jerry,1), (hello,1), (jerry,1), (hello,1), (kitty,1), (beijing,1), (shanghai,1), (tianjing,1), (xiaoming,1), (xiaohua,1), (xiaoli,1), (xiaozhang,1)) 4、reducer key,{1,1,1,1,1} 汇总
scala val line4line3.groupBy(_._1) line4: scala.collection.immutable.Map[String,List[(String, Int)]] Map(kitty - List((kitty,1)), xiaoming - List((xiaoming,1)), tianjing - List((tianjing,1)), shanghai - List((shanghai,1)), xiaohua - List((xiaohua,1)), tom - List((tom,1)), beijing - List((beijing,1)), hello - List((hello,1), (hello,1), (hello,1), (hello,1)), jerry - List((jerry,1), (jerry,1)), xiaozhang - List((xiaozhang,1)), xiaoli - List((xiaoli,1))) scala val line5line4.map(t(t._1,t._2.size)) line5: scala.collection.immutable.Map[String,Int] Map(kitty - 1, xiaoming - 1, tianjing - 1, shanghai - 1, xiaohua - 1, tom - 1, beijing - 1, hello - 4, jerry - 2, xiaozhang - 1, xiaoli - 1) 5、输出结果
scala val line6line5.toList line6: List[(String, Int)] List((kitty,1), (xiaoming,1), (tianjing,1), (shanghai,1), (xiaohua,1), (tom,1), (beijing,1), (hello,4), (jerry,2), (xiaozhang,1), (xiaoli,1)) scala val line7line6.sortBy(_._2).reverse line7: List[(String, Int)] List((hello,4), (jerry,2), (xiaoli,1), (xiaozhang,1), (beijing,1), (tom,1), (xiaohua,1), (shanghai,1), (tianjing,1), (xiaoming,1), (kitty,1)) ------------- 更多的JavaAndroid大数据J2EEPython数据库LinuxJava架构师教程视频请访问 http://www.cnblogs.com/zengmiaogen/p/7083694.html
