scala高阶函数实现类似sql查询

需求

输入：

val source = Seq(("a","b","c","d",1,2,3),
("a","b","c","d",1,2,5),
("a","b","c","d",10,2,5),
("a","b","c","d",1,3,1),
("a","b","c","e",10,2,3)
)

输出

Seq(("a", "b", "c", "d", 10, 3, 5), ("a", "b", "c", "e", 10, 2, 3))

类似效果：

select s._1,s._2,s._3,s._4,max(s._5),max(s._6),max(s._7) from s group by s._1,s._2,s._3,s._4

分析

通过groupBy可以对前四项进行分组

val groupByResult =  source.groupBy { k => (k._1, k._2, k._3, k._4) }

结果是一个Map：

以前四项为k，同一个key每条记录组成一个List为value

groupByResult: scala.collection.immutable.Map[(String, String, String, String),Seq[(String, String, String, String, Int, Int, Int)]] = Map((a,b,c,d) -> List((a,b,c,d,1,2,3), (a,b,c,d,1,2,5), (a,b,c,d,10,2,5), (a,b,c,d,1,3,1)), (a,b,c,e) -> List((a,b,c,e,10,2,3), (a,b,c,e,8,1,3)))

再对结果map处理

val result = groupByResult.map {
case (k, v) =>
(k._1, k._2, k._3, k._4, v.maxBy(_._5)._5, v.maxBy(_._6)._6, v.maxBy(_._7)._7)
}.toSeq

其中v是原来的一个元素，通过v.maxBy(._5)找到第5个字段最大的一条记录，再取这条记录的第5个元素，得到max(._5),第6个和第7个类似

完整代码

val result = source.groupBy(r =>(r._1,r._2,r._3,r._4)).map{
case(k,v) => (k._1,k._2,k._3,k._4,v.maxBy(_._5)._5,v.maxBy(_._6)._6,v.maxBy(_._7)._7)
}.toSeq

assert(result == Seq(("a", "b", "c", "d", 10, 3, 5), ("a", "b", "c", "e", 10, 2, 3)))