3.5.7
Jobs
Stages
Storage
Environment
Executors
SQL / DataFrame
SP1
application UI
Details for Stage 16232 (Attempt 0)
Resource Profile Id:
0
Total Time Across All Tasks:
6 s
Locality Level Summary:
Any: 15
Input Size / Records:
2.6 KiB / 15
Shuffle Read Size / Records:
19.5 MiB / 256861
Shuffle Write Size / Records:
8.3 MiB / 173748
Associated Job Ids:
7934
DAG Visualization
digraph G { subgraph clusterstage_16232 { label="Stage 16232"; subgraph cluster104090 { label="Exchange"; 44468 [labelType="html" label="ShuffledRowRDD [44468] [Unordered]<br>rdd at PersistJob.java:28"]; } subgraph cluster104086 { label="WholeStageCodegen (12)"; 44469 [labelType="html" label="MapPartitionsRDD [44469] [Unordered]<br>rdd at PersistJob.java:28"]; } subgraph cluster104085 { label="Window"; 44470 [labelType="html" label="MapPartitionsRDD [44470] [Unordered]<br>rdd at PersistJob.java:28"]; } subgraph cluster104081 { label="WholeStageCodegen (13)"; 44471 [labelType="html" label="MapPartitionsRDD [44471] [Unordered]<br>rdd at PersistJob.java:28"]; } subgraph cluster104102 { label="AQEShuffleRead"; 44472 [labelType="html" label="ShuffledRowRDD [44472] [Unordered]<br>rdd at PersistJob.java:28"]; } subgraph cluster104099 { label="WholeStageCodegen (14)"; 44473 [labelType="html" label="MapPartitionsRDD [44473] [Unordered]<br>rdd at PersistJob.java:28"]; } subgraph cluster104109 { label="AQEShuffleRead"; 44474 [labelType="html" label="ShuffledRowRDD [44474] [Unordered]<br>rdd at PersistJob.java:28"]; } subgraph cluster104108 { label="ObjectHashAggregate"; 44475 [labelType="html" label="MapPartitionsRDD [44475] [Unordered]<br>rdd at PersistJob.java:28"]; } subgraph cluster104104 { label="WholeStageCodegen (15)"; 44476 [labelType="html" label="MapPartitionsRDD [44476] [Unordered]<br>rdd at PersistJob.java:28"]; } subgraph cluster104091 { label="WholeStageCodegen (16)"; 44477 [labelType="html" label="ZippedPartitionsRDD2 [44477] [Unordered]<br>rdd at PersistJob.java:28"]; 44478 [labelType="html" label="MapPartitionsRDD [44478] [Unordered]<br>rdd at PersistJob.java:28"]; } subgraph cluster104080 { label="Union"; 44479 [labelType="html" label="UnionRDD [44479] [Unordered]<br>rdd at PersistJob.java:28"]; } subgraph cluster104114 { label="Scan csv "; 44480 [labelType="html" label="FileScanRDD [44480]<br>rdd at PersistJob.java:28"]; 44481 [labelType="html" label="MapPartitionsRDD [44481]<br>rdd at PersistJob.java:28"]; } subgraph cluster104110 { label="WholeStageCodegen (17)"; 44482 [labelType="html" label="MapPartitionsRDD [44482]<br>rdd at PersistJob.java:28"]; } subgraph cluster104079 { label="CartesianProduct"; 44483 [labelType="html" label="UnsafeCartesianRDD [44483] [Unordered]<br>rdd at PersistJob.java:28"]; 44484 [labelType="html" label="MapPartitionsRDD [44484] [Unordered]<br>rdd at PersistJob.java:28"]; } subgraph cluster104078 { label="Exchange"; 44485 [labelType="html" label="MapPartitionsRDD [44485] [Unordered]<br>rdd at PersistJob.java:28"]; } } 44468->44469; 44469->44470; 44470->44471; 44472->44473; 44474->44475; 44475->44476; 44473->44477; 44476->44477; 44477->44478; 44471->44479; 44478->44479; 44480->44481; 44481->44482; 44479->44483; 44482->44483; 44483->44484; 44484->44485; }
44467,44468
44450,44472
44460,44474
Event Timeline
Enable zooming
Tasks: 15. 1 Pages. Jump to
. Show
items in a page.
Go
Scheduler Delay
Task Deserialization Time
Shuffle Read Time
Executor Computing Time
Shuffle Write Time
Result Serialization Time
Getting Result Time