3.5.7
Jobs
Stages
Storage
Environment
Executors
SQL / DataFrame
SP1
application UI
Details for Stage 10300 (Attempt 0)
Resource Profile Id:
0
Total Time Across All Tasks:
19 s
Locality Level Summary:
Any: 5
Input Size / Records:
16.9 KiB / 5
Shuffle Read Size / Records:
8.1 MiB / 988926
Shuffle Write Size / Records:
20.6 MiB / 790389
Associated Job Ids:
4537
DAG Visualization
digraph G { subgraph clusterstage_10300 { label="Stage 10300"; subgraph cluster60514 { label="AQEShuffleRead"; 25620 [labelType="html" label="ShuffledRowRDD [25620] [Unordered]<br>rdd at PersistJob.java:28"]; } subgraph cluster60511 { label="WholeStageCodegen (11)"; 25621 [labelType="html" label="MapPartitionsRDD [25621] [Unordered]<br>rdd at PersistJob.java:28"]; } subgraph cluster60524 { label="AQEShuffleRead"; 25622 [labelType="html" label="ShuffledRowRDD [25622] [Unordered]<br>rdd at PersistJob.java:28"]; } subgraph cluster60521 { label="WholeStageCodegen (12)"; 25623 [labelType="html" label="MapPartitionsRDD [25623] [Unordered]<br>rdd at PersistJob.java:28"]; } subgraph cluster60520 { label="Window"; 25624 [labelType="html" label="MapPartitionsRDD [25624] [Unordered]<br>rdd at PersistJob.java:28"]; } subgraph cluster60516 { label="WholeStageCodegen (13)"; 25625 [labelType="html" label="MapPartitionsRDD [25625] [Unordered]<br>rdd at PersistJob.java:28"]; } subgraph cluster60505 { label="WholeStageCodegen (14)"; 25626 [labelType="html" label="ZippedPartitionsRDD2 [25626] [Unordered]<br>rdd at PersistJob.java:28"]; 25627 [labelType="html" label="MapPartitionsRDD [25627] [Unordered]<br>rdd at PersistJob.java:28"]; } subgraph cluster60504 { label="Window"; 25628 [labelType="html" label="MapPartitionsRDD [25628] [Unordered]<br>rdd at PersistJob.java:28"]; } subgraph cluster60501 { label="WholeStageCodegen (15)"; 25629 [labelType="html" label="MapPartitionsRDD [25629] [Unordered]<br>rdd at PersistJob.java:28"]; } subgraph cluster60534 { label="AQEShuffleRead"; 25630 [labelType="html" label="ShuffledRowRDD [25630] [Unordered]<br>rdd at PersistJob.java:28"]; } subgraph cluster60531 { label="WholeStageCodegen (16)"; 25631 [labelType="html" label="MapPartitionsRDD [25631] [Unordered]<br>rdd at PersistJob.java:28"]; } subgraph cluster60530 { label="Window"; 25632 [labelType="html" label="MapPartitionsRDD [25632] [Unordered]<br>rdd at PersistJob.java:28"]; } subgraph cluster60526 { label="WholeStageCodegen (17)"; 25633 [labelType="html" label="MapPartitionsRDD [25633] [Unordered]<br>rdd at PersistJob.java:28"]; } subgraph cluster60495 { label="WholeStageCodegen (18)"; 25634 [labelType="html" label="ZippedPartitionsRDD2 [25634] [Unordered]<br>rdd at PersistJob.java:28"]; 25635 [labelType="html" label="MapPartitionsRDD [25635] [Unordered]<br>rdd at PersistJob.java:28"]; } subgraph cluster60539 { label="Scan parquet "; 25636 [labelType="html" label="FileScanRDD [25636]<br>rdd at PersistJob.java:28"]; 25637 [labelType="html" label="MapPartitionsRDD [25637]<br>rdd at PersistJob.java:28"]; } subgraph cluster60535 { label="WholeStageCodegen (19)"; 25638 [labelType="html" label="MapPartitionsRDD [25638]<br>rdd at PersistJob.java:28"]; } subgraph cluster60494 { label="CartesianProduct"; 25639 [labelType="html" label="UnsafeCartesianRDD [25639] [Unordered]<br>rdd at PersistJob.java:28"]; 25640 [labelType="html" label="MapPartitionsRDD [25640] [Unordered]<br>rdd at PersistJob.java:28"]; } subgraph cluster60491 { label="WholeStageCodegen (20)"; 25641 [labelType="html" label="MapPartitionsRDD [25641] [Unordered]<br>rdd at PersistJob.java:28"]; } subgraph cluster60490 { label="Exchange"; 25642 [labelType="html" label="MapPartitionsRDD [25642] [Unordered]<br>rdd at PersistJob.java:28"]; } } 25620->25621; 25622->25623; 25623->25624; 25624->25625; 25621->25626; 25625->25626; 25626->25627; 25627->25628; 25628->25629; 25630->25631; 25631->25632; 25632->25633; 25629->25634; 25633->25634; 25634->25635; 25636->25637; 25637->25638; 25635->25639; 25638->25639; 25639->25640; 25640->25641; 25641->25642; }
25584,25620
25619,25622
25592,25630
Event Timeline
Enable zooming
Tasks: 5. 1 Pages. Jump to
. Show
items in a page.
Go
Scheduler Delay
Task Deserialization Time
Shuffle Read Time
Executor Computing Time
Shuffle Write Time
Result Serialization Time
Getting Result Time