3.5.7
Jobs
Stages
Storage
Environment
Executors
SQL / DataFrame
SP1
application UI
Details for Stage 16264 (Attempt 0)
Resource Profile Id:
0
Total Time Across All Tasks:
9 s
Locality Level Summary:
Any: 15
Input Size / Records:
2.6 KiB / 15
Shuffle Read Size / Records:
19.5 MiB / 256861
Shuffle Write Size / Records:
8.3 MiB / 173748
Associated Job Ids:
7943
DAG Visualization
digraph G { subgraph clusterstage_16264 { label="Stage 16264"; subgraph cluster104298 { label="Exchange"; 44544 [labelType="html" label="ShuffledRowRDD [44544] [Unordered]<br>parquet at PersistJob.java:35"]; } subgraph cluster104294 { label="WholeStageCodegen (12)"; 44545 [labelType="html" label="MapPartitionsRDD [44545] [Unordered]<br>parquet at PersistJob.java:35"]; } subgraph cluster104293 { label="Window"; 44546 [labelType="html" label="MapPartitionsRDD [44546] [Unordered]<br>parquet at PersistJob.java:35"]; } subgraph cluster104289 { label="WholeStageCodegen (13)"; 44547 [labelType="html" label="MapPartitionsRDD [44547] [Unordered]<br>parquet at PersistJob.java:35"]; } subgraph cluster104310 { label="AQEShuffleRead"; 44548 [labelType="html" label="ShuffledRowRDD [44548] [Unordered]<br>parquet at PersistJob.java:35"]; } subgraph cluster104307 { label="WholeStageCodegen (14)"; 44549 [labelType="html" label="MapPartitionsRDD [44549] [Unordered]<br>parquet at PersistJob.java:35"]; } subgraph cluster104317 { label="AQEShuffleRead"; 44550 [labelType="html" label="ShuffledRowRDD [44550] [Unordered]<br>parquet at PersistJob.java:35"]; } subgraph cluster104316 { label="ObjectHashAggregate"; 44551 [labelType="html" label="MapPartitionsRDD [44551] [Unordered]<br>parquet at PersistJob.java:35"]; } subgraph cluster104312 { label="WholeStageCodegen (15)"; 44552 [labelType="html" label="MapPartitionsRDD [44552] [Unordered]<br>parquet at PersistJob.java:35"]; } subgraph cluster104299 { label="WholeStageCodegen (16)"; 44553 [labelType="html" label="ZippedPartitionsRDD2 [44553] [Unordered]<br>parquet at PersistJob.java:35"]; 44554 [labelType="html" label="MapPartitionsRDD [44554] [Unordered]<br>parquet at PersistJob.java:35"]; } subgraph cluster104288 { label="Union"; 44555 [labelType="html" label="UnionRDD [44555] [Unordered]<br>parquet at PersistJob.java:35"]; } subgraph cluster104322 { label="Scan csv "; 44556 [labelType="html" label="FileScanRDD [44556]<br>parquet at PersistJob.java:35"]; 44557 [labelType="html" label="MapPartitionsRDD [44557]<br>parquet at PersistJob.java:35"]; } subgraph cluster104318 { label="WholeStageCodegen (17)"; 44558 [labelType="html" label="MapPartitionsRDD [44558]<br>parquet at PersistJob.java:35"]; } subgraph cluster104287 { label="CartesianProduct"; 44559 [labelType="html" label="UnsafeCartesianRDD [44559] [Unordered]<br>parquet at PersistJob.java:35"]; 44560 [labelType="html" label="MapPartitionsRDD [44560] [Unordered]<br>parquet at PersistJob.java:35"]; } subgraph cluster104286 { label="Exchange"; 44561 [labelType="html" label="MapPartitionsRDD [44561] [Unordered]<br>parquet at PersistJob.java:35"]; } } 44544->44545; 44545->44546; 44546->44547; 44548->44549; 44550->44551; 44551->44552; 44549->44553; 44552->44553; 44553->44554; 44547->44555; 44554->44555; 44556->44557; 44557->44558; 44555->44559; 44558->44559; 44559->44560; 44560->44561; }
44543,44544
44526,44548
44536,44550
Event Timeline
Enable zooming
Tasks: 15. 1 Pages. Jump to
. Show
items in a page.
Go
Scheduler Delay
Task Deserialization Time
Shuffle Read Time
Executor Computing Time
Shuffle Write Time
Result Serialization Time
Getting Result Time