trinodb/trino

EXPLAIN ANALYZE removes top stage

Open

#4,720 opened on Aug 6, 2020

View on GitHub
 (2 comments) (0 reactions) (0 assignees)Java (2,678 forks)batch import
good first issue

Repository metrics

Stars
 (9,113 stars)
PR merge metrics
 (Avg merge 5d 9h) (306 merged PRs in 30d)

Description

SELECT custkey, sum(totalprice)
FROM orders
GROUP BY custkey
ORDER BY 2 DESC

EXPLAIN

                                      Query Plan
---------------------------------------------------------------------------------------
 Fragment 0 [SINGLE]
     Output layout: [custkey, sum]
     Output partitioning: SINGLE []
     Stage Execution Strategy: UNGROUPED_EXECUTION
     Output[custkey, _col1]
     │   Layout: [custkey:bigint, sum:double]
     │   Estimates: {rows: ? (?), cpu: ?, memory: ?, network: ?}
     │   _col1 := sum
     └─ RemoteMerge[1]
            Layout: [custkey:bigint, sum:double]

 Fragment 1 [ROUND_ROBIN]
     Output layout: [custkey, sum]
     Output partitioning: SINGLE []
     Stage Execution Strategy: UNGROUPED_EXECUTION
     LocalMerge[sum DESC_NULLS_LAST]
     │   Layout: [custkey:bigint, sum:double]
     │   Estimates: {rows: ? (?), cpu: ?, memory: ?, network: ?}
     └─ PartialSort[sum DESC_NULLS_LAST]
        │   Layout: [custkey:bigint, sum:double]
        └─ RemoteSource[2]
               Layout: [custkey:bigint, sum:double]

 Fragment 2 [HASH]
     Output layout: [custkey, sum]
     Output partitioning: ROUND_ROBIN []
     Stage Execution Strategy: UNGROUPED_EXECUTION
     Aggregate(FINAL)[custkey]
     │   Layout: [custkey:bigint, sum:double]
     │   Estimates: {rows: ? (?), cpu: ?, memory: ?, network: ?}
     │   sum := sum("sum_0")
     └─ LocalExchange[HASH] ("custkey")
        │   Layout: [custkey:bigint, sum_0:row(bigint, boolean, double, boolean)]
        │   Estimates: {rows: ? (?), cpu: ?, memory: ?, network: ?}
        └─ RemoteSource[3]
               Layout: [custkey:bigint, sum_0:row(bigint, boolean, double, boolean)]

 Fragment 3 [tpch:orders:1500000]
     Output layout: [custkey, sum_0]
     Output partitioning: HASH [custkey]
     Stage Execution Strategy: UNGROUPED_EXECUTION
     Aggregate(PARTIAL)[custkey]
     │   Layout: [custkey:bigint, sum_0:row(bigint, boolean, double, boolean)]
     │   sum_0 := sum("totalprice")
     └─ TableScan[tpch:orders:sf1.0, grouped = false]
            Layout: [custkey:bigint, totalprice:double]
            Estimates: {rows: 1500000 (25.75MB), cpu: 25.75M, memory: 0B, network: 0B}
            custkey := tpch:custkey
            totalprice := tpch:totalprice
            tpch:orderstatus
                :: [[F], [O], [P]]

EXPLAIN ANALYZE

The output of EXPLAIN ANALYZE is missing Fragment 0, which includes the RemoteMerge operation

                                                                   Query Plan
------------------------------------------------------------------------------------------------------------------------------------------------
 Fragment 1 [ROUND_ROBIN]
     CPU: 103.33ms, Scheduled: 158.80ms, Input: 99996 rows (1.72MB); per task: avg.: 33332.00 std.dev.: 6899.92, Output: 99996 rows (1.72MB)
     Output layout: [custkey, sum]
     Output partitioning: SINGLE []
     Stage Execution Strategy: UNGROUPED_EXECUTION
     LocalMerge[sum DESC_NULLS_LAST]
     │   Layout: [custkey:bigint, sum:double]
     │   Estimates: {rows: ? (?), cpu: ?, memory: ?, network: ?}
     │   CPU: 46.00ms (1.00%), Scheduled: 71.00ms (1.15%), Output: 99996 rows (1.72MB)
     │   Input avg.: 8333.00 rows, Input std.dev.: 107.83%
     └─ PartialSort[sum DESC_NULLS_LAST]
        │   Layout: [custkey:bigint, sum:double]
        │   CPU: 55.00ms (1.19%), Scheduled: 80.00ms (1.30%), Output: 99996 rows (1.72MB)
        │   Input avg.: 8333.00 rows, Input std.dev.: 107.83%
        └─ RemoteSource[2]
               Layout: [custkey:bigint, sum:double]
               CPU: 1.00ms (0.02%), Scheduled: 5.00ms (0.08%), Output: 99996 rows (1.72MB)
               Input avg.: 8333.00 rows, Input std.dev.: 107.83%

 Fragment 2 [HASH]
     CPU: 262.03ms, Scheduled: 406.03ms, Input: 1203848 rows (41.33MB); per task: avg.: 401282.67 std.dev.: 623.89, Output: 99996 rows (1.72MB)
     Output layout: [custkey, sum]
     Output partitioning: ROUND_ROBIN []
     Stage Execution Strategy: UNGROUPED_EXECUTION
     Aggregate(FINAL)[custkey]
     │   Layout: [custkey:bigint, sum:double]
     │   Estimates: {rows: ? (?), cpu: ?, memory: ?, network: ?}
     │   CPU: 125.00ms (2.71%), Scheduled: 186.00ms (3.01%), Output: 99996 rows (1.72MB)
     │   Input avg.: 100320.67 rows, Input std.dev.: 1.19%
     │   Collisions avg.: 15354.34 (1066.97% est.), Collisions std.dev.: 141.49%
     │   sum := sum("sum_0")
     └─ LocalExchange[HASH] ("custkey")
        │   Layout: [custkey:bigint, sum_0:row(bigint, boolean, double, boolean)]
        │   Estimates: {rows: ? (?), cpu: ?, memory: ?, network: ?}
        │   CPU: 90.00ms (1.95%), Scheduled: 124.00ms (2.01%), Output: 1203848 rows (41.33MB)
        │   Input avg.: 100320.67 rows, Input std.dev.: 118.61%
        └─ RemoteSource[3]
               Layout: [custkey:bigint, sum_0:row(bigint, boolean, double, boolean)]
               CPU: 40.00ms (0.87%), Scheduled: 54.00ms (0.87%), Output: 1203848 rows (41.33MB)
               Input avg.: 100320.67 rows, Input std.dev.: 118.61%

 Fragment 3 [tpch:orders:1500000]
     CPU: 4.26s, Scheduled: 5.65s, Input: 1500000 rows (25.75MB); per task: avg.: 500000.00 std.dev.: 11.31, Output: 1203848 rows (41.33MB)
     Output layout: [custkey, sum_0]
     Output partitioning: HASH [custkey]
     Stage Execution Strategy: UNGROUPED_EXECUTION
     Aggregate(PARTIAL)[custkey]
     │   Layout: [custkey:bigint, sum_0:row(bigint, boolean, double, boolean)]
     │   CPU: 1.02s (22.06%), Scheduled: 1.53s (24.85%), Output: 1203848 rows (41.33MB)
     │   Input avg.: 41666.67 rows, Input std.dev.: 0.01%
     │   Collisions avg.: 25484.65 (105.71% est.), Collisions std.dev.: 141.47%
     │   sum_0 := sum("totalprice")
     └─ TableScan[tpch:orders:sf1.0, grouped = false]
            Layout: [custkey:bigint, totalprice:double]
            Estimates: {rows: 1500000 (25.75MB), cpu: 25.75M, memory: 0B, network: 0B}
            CPU: 3.24s (70.21%), Scheduled: 4.12s (66.73%), Output: 1500000 rows (25.75MB)
            Input avg.: 41666.67 rows, Input std.dev.: 0.01%
            custkey := tpch:custkey
            totalprice := tpch:totalprice
            tpch:orderstatus
                :: [[F], [O], [P]]

Contributor guide