spark sql 数据量 结果不一致的问题

[HbnKing]

 val w = Window.partitionBy(tmp.col("_id"),tmp.col("pro.fieldID")).orderBy(functions.desc("pro.weight"))
    tmp.where("related = 196 or  related = 2447005 or  u = 196 or  u = 2447005 ").withColumn("rn", functions.row_number().over(w)).show(false)
    println("----------------------")
    tmp.withColumn("rn", functions.row_number().over(w)).where("related = 196 or  related = 2447005 or  u = 196 or  u = 2447005 ").show(false)

同样的数据过来 使用相同的 窗口函数 ，但是 结果 却不一样
data format

root
 |-- _id: struct (nullable = true)
 |    |-- oid: string (nullable = true)
 |-- relatedID: array (nullable = true)
 |    |-- element: integer (containsNull = true)
 |-- related: integer (nullable = true)

+--------------------------+--------------+-------+-------+-------------------------+---+
|_id                       |relatedID     |related|u      |pro                      |rn |
+--------------------------+--------------+-------+-------+-------------------------+---+
|[5c3f2de802353b0d870b05e0]|[196, 2542146]|196    |196    |[age,101,21,0.9]         |1  |
|[5c3f2de802353b0d870b05e0]|[196, 2542146]|196    |196    |[name,100,yyj196,0.8]    |1  |
|[5c3f2dd302353b0d870a7d2f]|[196, 2447005]|196    |196    |[age,101,21,0.9]         |1  |
|[5c3f2dd302353b0d870a7d2f]|[196, 2447005]|2447005|2447005|[age,101,21,0.5]         |2  |
|[5c3f2de802353b0d870b05e0]|[196, 2542146]|196    |196    |[favorite,102,IT,0.7]    |1  |
|[5c3f2dd302353b0d870a7d2f]|[196, 2447005]|196    |196    |[favorite,102,IT,0.7]    |1  |
|[5c3f2dd302353b0d870a7d2f]|[196, 2447005]|2447005|2447005|[favorite,102,iphone,0.5]|2  |
|[5c3f2dd302353b0d870a7d2f]|[196, 2447005]|196    |196    |[name,100,yyj196,0.8]    |1  |
|[5c3f2dd302353b0d870a7d2f]|[196, 2447005]|2447005|2447005|[name,100,yyj2447005,0.5]|2  |
+--------------------------+--------------+-------+-------+-------------------------+---+

----------------------
19/02/01 18:31:11 WARN BaseSessionStateBuilder$$anon$2: Max iterations (100) reached for batch Operator Optimizations
+--------------------------+--------------+-------+-------+-------------------------+---+
|_id                       |relatedID     |related|u      |pro                      |rn |
+--------------------------+--------------+-------+-------+-------------------------+---+
|[5c3f2dd302353b0d870a7d2f]|[196, 2447005]|2447005|2447005|[age,101,21,0.5]         |1  |
|[5c3f2dd302353b0d870a7d2f]|[196, 2447005]|2447005|2447005|[favorite,102,iphone,0.5]|1  |
|[5c3f2dd302353b0d870a7d2f]|[196, 2447005]|2447005|2447005|[name,100,yyj2447005,0.5]|1  |
+--------------------------+--------------+-------+-------+-------------------------+---+