3.5.7
Jobs
Stages
Storage
Environment
Executors
SQL / DataFrame
SP1
application UI
Details for Stage 16267 (Attempt 0)
Resource Profile Id:
0
Total Time Across All Tasks:
6 s
Locality Level Summary:
Any: 15
Input Size / Records:
2.6 KiB / 15
Shuffle Read Size / Records:
19.7 MiB / 259086
Shuffle Write Size / Records:
8.4 MiB / 175588
Associated Job Ids:
8501
DAG Visualization
digraph G { subgraph clusterstage_16267 { label="Stage 16267"; subgraph cluster105026 { label="Exchange"; 44413 [labelType="html" label="ShuffledRowRDD [44413] [Unordered]<br>rdd at PersistJob.java:21"]; } subgraph cluster105022 { label="WholeStageCodegen (12)"; 44414 [labelType="html" label="MapPartitionsRDD [44414] [Unordered]<br>rdd at PersistJob.java:21"]; } subgraph cluster105021 { label="Window"; 44415 [labelType="html" label="MapPartitionsRDD [44415] [Unordered]<br>rdd at PersistJob.java:21"]; } subgraph cluster105017 { label="WholeStageCodegen (13)"; 44416 [labelType="html" label="MapPartitionsRDD [44416] [Unordered]<br>rdd at PersistJob.java:21"]; } subgraph cluster105038 { label="AQEShuffleRead"; 44417 [labelType="html" label="ShuffledRowRDD [44417] [Unordered]<br>rdd at PersistJob.java:21"]; } subgraph cluster105035 { label="WholeStageCodegen (14)"; 44418 [labelType="html" label="MapPartitionsRDD [44418] [Unordered]<br>rdd at PersistJob.java:21"]; } subgraph cluster105045 { label="AQEShuffleRead"; 44419 [labelType="html" label="ShuffledRowRDD [44419] [Unordered]<br>rdd at PersistJob.java:21"]; } subgraph cluster105044 { label="ObjectHashAggregate"; 44420 [labelType="html" label="MapPartitionsRDD [44420] [Unordered]<br>rdd at PersistJob.java:21"]; } subgraph cluster105040 { label="WholeStageCodegen (15)"; 44421 [labelType="html" label="MapPartitionsRDD [44421] [Unordered]<br>rdd at PersistJob.java:21"]; } subgraph cluster105027 { label="WholeStageCodegen (16)"; 44422 [labelType="html" label="ZippedPartitionsRDD2 [44422] [Unordered]<br>rdd at PersistJob.java:21"]; 44423 [labelType="html" label="MapPartitionsRDD [44423] [Unordered]<br>rdd at PersistJob.java:21"]; } subgraph cluster105016 { label="Union"; 44424 [labelType="html" label="UnionRDD [44424] [Unordered]<br>rdd at PersistJob.java:21"]; } subgraph cluster105050 { label="Scan csv "; 44425 [labelType="html" label="FileScanRDD [44425]<br>rdd at PersistJob.java:21"]; 44426 [labelType="html" label="MapPartitionsRDD [44426]<br>rdd at PersistJob.java:21"]; } subgraph cluster105046 { label="WholeStageCodegen (17)"; 44427 [labelType="html" label="MapPartitionsRDD [44427]<br>rdd at PersistJob.java:21"]; } subgraph cluster105015 { label="CartesianProduct"; 44428 [labelType="html" label="UnsafeCartesianRDD [44428] [Unordered]<br>rdd at PersistJob.java:21"]; 44429 [labelType="html" label="MapPartitionsRDD [44429] [Unordered]<br>rdd at PersistJob.java:21"]; } subgraph cluster105014 { label="Exchange"; 44430 [labelType="html" label="MapPartitionsRDD [44430] [Unordered]<br>rdd at PersistJob.java:21"]; } } 44413->44414; 44414->44415; 44415->44416; 44417->44418; 44419->44420; 44420->44421; 44418->44422; 44421->44422; 44422->44423; 44416->44424; 44423->44424; 44425->44426; 44426->44427; 44424->44428; 44427->44428; 44428->44429; 44429->44430; }
44406,44413
44395,44417
44412,44419
Event Timeline
Enable zooming
Tasks: 15. 1 Pages. Jump to
. Show
items in a page.
Go
Scheduler Delay
Task Deserialization Time
Shuffle Read Time
Executor Computing Time
Shuffle Write Time
Result Serialization Time
Getting Result Time