3.5.7
Jobs
Stages
Storage
Environment
Executors
SQL / DataFrame
SP1
application UI
Details for Stage 16512 (Attempt 0)
Resource Profile Id:
0
Total Time Across All Tasks:
45 s
Locality Level Summary:
Any: 10
Input Size / Records:
36.0 KiB / 10
Shuffle Read Size / Records:
34.9 MiB / 1240521
Shuffle Write Size / Records:
76.7 MiB / 1115956
Associated Job Ids:
8089
DAG Visualization
digraph G { subgraph clusterstage_16512 { label="Stage 16512"; subgraph cluster105822 { label="Exchange"; 45173 [labelType="html" label="ShuffledRowRDD [45173] [Unordered]<br>rdd at PersistJob.java:28"]; } subgraph cluster105818 { label="WholeStageCodegen (22)"; 45174 [labelType="html" label="MapPartitionsRDD [45174] [Unordered]<br>rdd at PersistJob.java:28"]; } subgraph cluster105833 { label="Exchange"; 45175 [labelType="html" label="ShuffledRowRDD [45175] [Unordered]<br>rdd at PersistJob.java:28"]; } subgraph cluster105829 { label="WholeStageCodegen (23)"; 45176 [labelType="html" label="MapPartitionsRDD [45176] [Unordered]<br>rdd at PersistJob.java:28"]; } subgraph cluster105828 { label="Window"; 45177 [labelType="html" label="MapPartitionsRDD [45177] [Unordered]<br>rdd at PersistJob.java:28"]; } subgraph cluster105824 { label="WholeStageCodegen (24)"; 45178 [labelType="html" label="MapPartitionsRDD [45178] [Unordered]<br>rdd at PersistJob.java:28"]; } subgraph cluster105812 { label="WholeStageCodegen (25)"; 45179 [labelType="html" label="ZippedPartitionsRDD2 [45179] [Unordered]<br>rdd at PersistJob.java:28"]; 45180 [labelType="html" label="MapPartitionsRDD [45180] [Unordered]<br>rdd at PersistJob.java:28"]; } subgraph cluster105811 { label="Window"; 45181 [labelType="html" label="MapPartitionsRDD [45181] [Unordered]<br>rdd at PersistJob.java:28"]; } subgraph cluster105808 { label="WholeStageCodegen (26)"; 45182 [labelType="html" label="MapPartitionsRDD [45182] [Unordered]<br>rdd at PersistJob.java:28"]; } subgraph cluster105844 { label="Exchange"; 45183 [labelType="html" label="ShuffledRowRDD [45183] [Unordered]<br>rdd at PersistJob.java:28"]; } subgraph cluster105840 { label="WholeStageCodegen (27)"; 45184 [labelType="html" label="MapPartitionsRDD [45184] [Unordered]<br>rdd at PersistJob.java:28"]; } subgraph cluster105839 { label="Window"; 45185 [labelType="html" label="MapPartitionsRDD [45185] [Unordered]<br>rdd at PersistJob.java:28"]; } subgraph cluster105835 { label="WholeStageCodegen (28)"; 45186 [labelType="html" label="MapPartitionsRDD [45186] [Unordered]<br>rdd at PersistJob.java:28"]; } subgraph cluster105802 { label="WholeStageCodegen (29)"; 45187 [labelType="html" label="ZippedPartitionsRDD2 [45187] [Unordered]<br>rdd at PersistJob.java:28"]; 45188 [labelType="html" label="MapPartitionsRDD [45188] [Unordered]<br>rdd at PersistJob.java:28"]; } subgraph cluster105849 { label="Scan parquet "; 45189 [labelType="html" label="FileScanRDD [45189]<br>rdd at PersistJob.java:28"]; 45190 [labelType="html" label="MapPartitionsRDD [45190]<br>rdd at PersistJob.java:28"]; } subgraph cluster105845 { label="WholeStageCodegen (30)"; 45191 [labelType="html" label="MapPartitionsRDD [45191]<br>rdd at PersistJob.java:28"]; } subgraph cluster105801 { label="CartesianProduct"; 45192 [labelType="html" label="UnsafeCartesianRDD [45192] [Unordered]<br>rdd at PersistJob.java:28"]; 45193 [labelType="html" label="MapPartitionsRDD [45193] [Unordered]<br>rdd at PersistJob.java:28"]; } subgraph cluster105798 { label="WholeStageCodegen (31)"; 45194 [labelType="html" label="MapPartitionsRDD [45194] [Unordered]<br>rdd at PersistJob.java:28"]; } subgraph cluster105797 { label="Exchange"; 45195 [labelType="html" label="MapPartitionsRDD [45195] [Unordered]<br>rdd at PersistJob.java:28"]; } } 45173->45174; 45175->45176; 45176->45177; 45177->45178; 45174->45179; 45178->45179; 45179->45180; 45180->45181; 45181->45182; 45183->45184; 45184->45185; 45185->45186; 45182->45187; 45186->45187; 45187->45188; 45189->45190; 45190->45191; 45188->45192; 45191->45192; 45192->45193; 45193->45194; 45194->45195; }
45108,45173
45172,45175
45124,45183
Event Timeline
Enable zooming
Tasks: 10. 1 Pages. Jump to
. Show
items in a page.
Go
Scheduler Delay
Task Deserialization Time
Shuffle Read Time
Executor Computing Time
Shuffle Write Time
Result Serialization Time
Getting Result Time