3.5.7
Jobs
Stages
Storage
Environment
Executors
SQL / DataFrame
SP1
application UI
Details for Stage 16299 (Attempt 0)
Resource Profile Id:
0
Total Time Across All Tasks:
5 s
Locality Level Summary:
Any: 15
Input Size / Records:
2.6 KiB / 15
Shuffle Read Size / Records:
19.7 MiB / 259086
Shuffle Write Size / Records:
8.4 MiB / 175588
Associated Job Ids:
8510
DAG Visualization
digraph G { subgraph clusterstage_16299 { label="Stage 16299"; subgraph cluster105234 { label="Exchange"; 44488 [labelType="html" label="ShuffledRowRDD [44488] [Unordered]<br>rdd at PersistJob.java:28"]; } subgraph cluster105230 { label="WholeStageCodegen (12)"; 44489 [labelType="html" label="MapPartitionsRDD [44489] [Unordered]<br>rdd at PersistJob.java:28"]; } subgraph cluster105229 { label="Window"; 44490 [labelType="html" label="MapPartitionsRDD [44490] [Unordered]<br>rdd at PersistJob.java:28"]; } subgraph cluster105225 { label="WholeStageCodegen (13)"; 44491 [labelType="html" label="MapPartitionsRDD [44491] [Unordered]<br>rdd at PersistJob.java:28"]; } subgraph cluster105246 { label="AQEShuffleRead"; 44492 [labelType="html" label="ShuffledRowRDD [44492] [Unordered]<br>rdd at PersistJob.java:28"]; } subgraph cluster105243 { label="WholeStageCodegen (14)"; 44493 [labelType="html" label="MapPartitionsRDD [44493] [Unordered]<br>rdd at PersistJob.java:28"]; } subgraph cluster105253 { label="AQEShuffleRead"; 44494 [labelType="html" label="ShuffledRowRDD [44494] [Unordered]<br>rdd at PersistJob.java:28"]; } subgraph cluster105252 { label="ObjectHashAggregate"; 44495 [labelType="html" label="MapPartitionsRDD [44495] [Unordered]<br>rdd at PersistJob.java:28"]; } subgraph cluster105248 { label="WholeStageCodegen (15)"; 44496 [labelType="html" label="MapPartitionsRDD [44496] [Unordered]<br>rdd at PersistJob.java:28"]; } subgraph cluster105235 { label="WholeStageCodegen (16)"; 44497 [labelType="html" label="ZippedPartitionsRDD2 [44497] [Unordered]<br>rdd at PersistJob.java:28"]; 44498 [labelType="html" label="MapPartitionsRDD [44498] [Unordered]<br>rdd at PersistJob.java:28"]; } subgraph cluster105224 { label="Union"; 44499 [labelType="html" label="UnionRDD [44499] [Unordered]<br>rdd at PersistJob.java:28"]; } subgraph cluster105258 { label="Scan csv "; 44500 [labelType="html" label="FileScanRDD [44500]<br>rdd at PersistJob.java:28"]; 44501 [labelType="html" label="MapPartitionsRDD [44501]<br>rdd at PersistJob.java:28"]; } subgraph cluster105254 { label="WholeStageCodegen (17)"; 44502 [labelType="html" label="MapPartitionsRDD [44502]<br>rdd at PersistJob.java:28"]; } subgraph cluster105223 { label="CartesianProduct"; 44503 [labelType="html" label="UnsafeCartesianRDD [44503] [Unordered]<br>rdd at PersistJob.java:28"]; 44504 [labelType="html" label="MapPartitionsRDD [44504] [Unordered]<br>rdd at PersistJob.java:28"]; } subgraph cluster105222 { label="Exchange"; 44505 [labelType="html" label="MapPartitionsRDD [44505] [Unordered]<br>rdd at PersistJob.java:28"]; } } 44488->44489; 44489->44490; 44490->44491; 44492->44493; 44494->44495; 44495->44496; 44493->44497; 44496->44497; 44497->44498; 44491->44499; 44498->44499; 44500->44501; 44501->44502; 44499->44503; 44502->44503; 44503->44504; 44504->44505; }
44487,44488
44470,44492
44480,44494
Event Timeline
Enable zooming
Tasks: 15. 1 Pages. Jump to
. Show
items in a page.
Go
Scheduler Delay
Task Deserialization Time
Shuffle Read Time
Executor Computing Time
Shuffle Write Time
Result Serialization Time
Getting Result Time