3.5.7
Jobs
Stages
Storage
Environment
Executors
SQL / DataFrame
SP1
application UI
Details for Stage 8539 (Attempt 0)
Resource Profile Id:
0
Total Time Across All Tasks:
5 s
Locality Level Summary:
Any: 15
Input Size / Records:
2.6 KiB / 15
Shuffle Read Size / Records:
26.4 MiB / 358608
Shuffle Write Size / Records:
11.4 MiB / 262408
Associated Job Ids:
4230
DAG Visualization
digraph G { subgraph clusterstage_8539 { label="Stage 8539"; subgraph cluster55968 { label="Exchange"; 24540 [labelType="html" label="ShuffledRowRDD [24540] [Unordered]<br>parquet at PersistJob.java:35"]; } subgraph cluster55964 { label="WholeStageCodegen (12)"; 24541 [labelType="html" label="MapPartitionsRDD [24541] [Unordered]<br>parquet at PersistJob.java:35"]; } subgraph cluster55963 { label="Window"; 24542 [labelType="html" label="MapPartitionsRDD [24542] [Unordered]<br>parquet at PersistJob.java:35"]; } subgraph cluster55959 { label="WholeStageCodegen (13)"; 24543 [labelType="html" label="MapPartitionsRDD [24543] [Unordered]<br>parquet at PersistJob.java:35"]; } subgraph cluster55980 { label="AQEShuffleRead"; 24544 [labelType="html" label="ShuffledRowRDD [24544] [Unordered]<br>parquet at PersistJob.java:35"]; } subgraph cluster55977 { label="WholeStageCodegen (14)"; 24545 [labelType="html" label="MapPartitionsRDD [24545] [Unordered]<br>parquet at PersistJob.java:35"]; } subgraph cluster55987 { label="AQEShuffleRead"; 24546 [labelType="html" label="ShuffledRowRDD [24546] [Unordered]<br>parquet at PersistJob.java:35"]; } subgraph cluster55986 { label="ObjectHashAggregate"; 24547 [labelType="html" label="MapPartitionsRDD [24547] [Unordered]<br>parquet at PersistJob.java:35"]; } subgraph cluster55982 { label="WholeStageCodegen (15)"; 24548 [labelType="html" label="MapPartitionsRDD [24548] [Unordered]<br>parquet at PersistJob.java:35"]; } subgraph cluster55969 { label="WholeStageCodegen (16)"; 24549 [labelType="html" label="ZippedPartitionsRDD2 [24549] [Unordered]<br>parquet at PersistJob.java:35"]; 24550 [labelType="html" label="MapPartitionsRDD [24550] [Unordered]<br>parquet at PersistJob.java:35"]; } subgraph cluster55958 { label="Union"; 24551 [labelType="html" label="UnionRDD [24551] [Unordered]<br>parquet at PersistJob.java:35"]; } subgraph cluster55992 { label="Scan csv "; 24552 [labelType="html" label="FileScanRDD [24552]<br>parquet at PersistJob.java:35"]; 24553 [labelType="html" label="MapPartitionsRDD [24553]<br>parquet at PersistJob.java:35"]; } subgraph cluster55988 { label="WholeStageCodegen (17)"; 24554 [labelType="html" label="MapPartitionsRDD [24554]<br>parquet at PersistJob.java:35"]; } subgraph cluster55957 { label="CartesianProduct"; 24555 [labelType="html" label="UnsafeCartesianRDD [24555] [Unordered]<br>parquet at PersistJob.java:35"]; 24556 [labelType="html" label="MapPartitionsRDD [24556] [Unordered]<br>parquet at PersistJob.java:35"]; } subgraph cluster55956 { label="Exchange"; 24557 [labelType="html" label="MapPartitionsRDD [24557] [Unordered]<br>parquet at PersistJob.java:35"]; } } 24540->24541; 24541->24542; 24542->24543; 24544->24545; 24546->24547; 24547->24548; 24545->24549; 24548->24549; 24549->24550; 24543->24551; 24550->24551; 24552->24553; 24553->24554; 24551->24555; 24554->24555; 24555->24556; 24556->24557; }
24533,24540
24522,24544
24539,24546
Event Timeline
Enable zooming
Tasks: 15. 1 Pages. Jump to
. Show
items in a page.
Go
Scheduler Delay
Task Deserialization Time
Shuffle Read Time
Executor Computing Time
Shuffle Write Time
Result Serialization Time
Getting Result Time