3.5.7
Jobs
Stages
Storage
Environment
Executors
SQL / DataFrame
SP1
application UI
Details for Stage 7992 (Attempt 0)
Resource Profile Id:
0
Total Time Across All Tasks:
12 s
Locality Level Summary:
Any: 15
Input Size / Records:
2.6 KiB / 15
Shuffle Read Size / Records:
26.4 MiB / 358608
Shuffle Write Size / Records:
11.4 MiB / 262408
Associated Job Ids:
3899
DAG Visualization
digraph G { subgraph clusterstage_7992 { label="Stage 7992"; subgraph cluster52206 { label="Exchange"; 23036 [labelType="html" label="ShuffledRowRDD [23036] [Unordered]<br>parquet at PersistJob.java:35"]; } subgraph cluster52202 { label="WholeStageCodegen (12)"; 23037 [labelType="html" label="MapPartitionsRDD [23037] [Unordered]<br>parquet at PersistJob.java:35"]; } subgraph cluster52201 { label="Window"; 23038 [labelType="html" label="MapPartitionsRDD [23038] [Unordered]<br>parquet at PersistJob.java:35"]; } subgraph cluster52197 { label="WholeStageCodegen (13)"; 23039 [labelType="html" label="MapPartitionsRDD [23039] [Unordered]<br>parquet at PersistJob.java:35"]; } subgraph cluster52218 { label="AQEShuffleRead"; 23040 [labelType="html" label="ShuffledRowRDD [23040] [Unordered]<br>parquet at PersistJob.java:35"]; } subgraph cluster52215 { label="WholeStageCodegen (14)"; 23041 [labelType="html" label="MapPartitionsRDD [23041] [Unordered]<br>parquet at PersistJob.java:35"]; } subgraph cluster52225 { label="AQEShuffleRead"; 23042 [labelType="html" label="ShuffledRowRDD [23042] [Unordered]<br>parquet at PersistJob.java:35"]; } subgraph cluster52224 { label="ObjectHashAggregate"; 23043 [labelType="html" label="MapPartitionsRDD [23043] [Unordered]<br>parquet at PersistJob.java:35"]; } subgraph cluster52220 { label="WholeStageCodegen (15)"; 23044 [labelType="html" label="MapPartitionsRDD [23044] [Unordered]<br>parquet at PersistJob.java:35"]; } subgraph cluster52207 { label="WholeStageCodegen (16)"; 23045 [labelType="html" label="ZippedPartitionsRDD2 [23045] [Unordered]<br>parquet at PersistJob.java:35"]; 23046 [labelType="html" label="MapPartitionsRDD [23046] [Unordered]<br>parquet at PersistJob.java:35"]; } subgraph cluster52196 { label="Union"; 23047 [labelType="html" label="UnionRDD [23047] [Unordered]<br>parquet at PersistJob.java:35"]; } subgraph cluster52230 { label="Scan csv "; 23048 [labelType="html" label="FileScanRDD [23048]<br>parquet at PersistJob.java:35"]; 23049 [labelType="html" label="MapPartitionsRDD [23049]<br>parquet at PersistJob.java:35"]; } subgraph cluster52226 { label="WholeStageCodegen (17)"; 23050 [labelType="html" label="MapPartitionsRDD [23050]<br>parquet at PersistJob.java:35"]; } subgraph cluster52195 { label="CartesianProduct"; 23051 [labelType="html" label="UnsafeCartesianRDD [23051] [Unordered]<br>parquet at PersistJob.java:35"]; 23052 [labelType="html" label="MapPartitionsRDD [23052] [Unordered]<br>parquet at PersistJob.java:35"]; } subgraph cluster52194 { label="Exchange"; 23053 [labelType="html" label="MapPartitionsRDD [23053] [Unordered]<br>parquet at PersistJob.java:35"]; } } 23036->23037; 23037->23038; 23038->23039; 23040->23041; 23042->23043; 23043->23044; 23041->23045; 23044->23045; 23045->23046; 23039->23047; 23046->23047; 23048->23049; 23049->23050; 23047->23051; 23050->23051; 23051->23052; 23052->23053; }
23029,23036
23018,23040
23035,23042
Event Timeline
Enable zooming
Tasks: 15. 1 Pages. Jump to
. Show
items in a page.
Go
Scheduler Delay
Task Deserialization Time
Shuffle Read Time
Executor Computing Time
Shuffle Write Time
Result Serialization Time
Getting Result Time