3.5.7
Jobs
Stages
Storage
Environment
Executors
SQL / DataFrame
SP1
application UI
Details for Stage 18239 (Attempt 0)
Resource Profile Id:
0
Total Time Across All Tasks:
7 s
Locality Level Summary:
Any: 15
Input Size / Records:
2.6 KiB / 15
Shuffle Read Size / Records:
26.4 MiB / 358829
Shuffle Write Size / Records:
11.4 MiB / 262581
Associated Job Ids:
9132
DAG Visualization
digraph G { subgraph clusterstage_18239 { label="Stage 18239"; subgraph cluster119007 { label="Exchange"; 51117 [labelType="html" label="ShuffledRowRDD [51117] [Unordered]<br>parquet at PersistJob.java:35"]; } subgraph cluster118993 { label="WholeStageCodegen (12)"; 51118 [labelType="html" label="MapPartitionsRDD [51118] [Unordered]<br>parquet at PersistJob.java:35"]; } subgraph cluster118992 { label="Window"; 51119 [labelType="html" label="MapPartitionsRDD [51119] [Unordered]<br>parquet at PersistJob.java:35"]; } subgraph cluster118984 { label="WholeStageCodegen (13)"; 51124 [labelType="html" label="MapPartitionsRDD [51124] [Unordered]<br>parquet at PersistJob.java:35"]; } subgraph cluster119030 { label="AQEShuffleRead"; 51131 [labelType="html" label="ShuffledRowRDD [51131] [Unordered]<br>parquet at PersistJob.java:35"]; } subgraph cluster119027 { label="WholeStageCodegen (14)"; 51132 [labelType="html" label="MapPartitionsRDD [51132] [Unordered]<br>parquet at PersistJob.java:35"]; } subgraph cluster119040 { label="AQEShuffleRead"; 51135 [labelType="html" label="ShuffledRowRDD [51135] [Unordered]<br>parquet at PersistJob.java:35"]; } subgraph cluster119039 { label="ObjectHashAggregate"; 51136 [labelType="html" label="MapPartitionsRDD [51136] [Unordered]<br>parquet at PersistJob.java:35"]; } subgraph cluster119032 { label="WholeStageCodegen (15)"; 51137 [labelType="html" label="MapPartitionsRDD [51137] [Unordered]<br>parquet at PersistJob.java:35"]; } subgraph cluster119009 { label="WholeStageCodegen (16)"; 51138 [labelType="html" label="ZippedPartitionsRDD2 [51138] [Unordered]<br>parquet at PersistJob.java:35"]; 51139 [labelType="html" label="MapPartitionsRDD [51139] [Unordered]<br>parquet at PersistJob.java:35"]; } subgraph cluster118983 { label="Union"; 51140 [labelType="html" label="UnionRDD [51140] [Unordered]<br>parquet at PersistJob.java:35"]; } subgraph cluster119045 { label="Scan csv "; 51141 [labelType="html" label="FileScanRDD [51141]<br>parquet at PersistJob.java:35"]; 51142 [labelType="html" label="MapPartitionsRDD [51142]<br>parquet at PersistJob.java:35"]; } subgraph cluster119041 { label="WholeStageCodegen (17)"; 51143 [labelType="html" label="MapPartitionsRDD [51143]<br>parquet at PersistJob.java:35"]; } subgraph cluster118982 { label="CartesianProduct"; 51144 [labelType="html" label="UnsafeCartesianRDD [51144] [Unordered]<br>parquet at PersistJob.java:35"]; 51145 [labelType="html" label="MapPartitionsRDD [51145] [Unordered]<br>parquet at PersistJob.java:35"]; } subgraph cluster118981 { label="Exchange"; 51146 [labelType="html" label="MapPartitionsRDD [51146] [Unordered]<br>parquet at PersistJob.java:35"]; } } 51117->51118; 51118->51119; 51119->51124; 51131->51132; 51135->51136; 51136->51137; 51132->51138; 51137->51138; 51138->51139; 51124->51140; 51139->51140; 51141->51142; 51142->51143; 51140->51144; 51143->51144; 51144->51145; 51145->51146; }
51074,51117
51009,51131
51067,51135
Event Timeline
Enable zooming
Tasks: 15. 1 Pages. Jump to
. Show
items in a page.
Go
Scheduler Delay
Task Deserialization Time
Shuffle Read Time
Executor Computing Time
Shuffle Write Time
Result Serialization Time
Getting Result Time