3.5.7
Jobs
Stages
Storage
Environment
Executors
SQL / DataFrame
SP1
application UI
Details for Stage 32285 (Attempt 0)
Resource Profile Id:
0
Total Time Across All Tasks:
17 s
Locality Level Summary:
Any: 5
Input Size / Records:
18.0 KiB / 5
Shuffle Read Size / Records:
6.7 MiB / 727950
Shuffle Write Size / Records:
15.2 MiB / 591883
Associated Job Ids:
17847
DAG Visualization
digraph G { subgraph clusterstage_32285 { label="Stage 32285"; subgraph cluster205845 { label="AQEShuffleRead"; 89116 [labelType="html" label="ShuffledRowRDD [89116] [Unordered]<br>parquet at PersistJob.java:35"]; } subgraph cluster205842 { label="WholeStageCodegen (25)"; 89117 [labelType="html" label="MapPartitionsRDD [89117] [Unordered]<br>parquet at PersistJob.java:35"]; } subgraph cluster205855 { label="AQEShuffleRead"; 89118 [labelType="html" label="ShuffledRowRDD [89118] [Unordered]<br>parquet at PersistJob.java:35"]; } subgraph cluster205852 { label="WholeStageCodegen (26)"; 89119 [labelType="html" label="MapPartitionsRDD [89119] [Unordered]<br>parquet at PersistJob.java:35"]; } subgraph cluster205851 { label="Window"; 89120 [labelType="html" label="MapPartitionsRDD [89120] [Unordered]<br>parquet at PersistJob.java:35"]; } subgraph cluster205847 { label="WholeStageCodegen (27)"; 89121 [labelType="html" label="MapPartitionsRDD [89121] [Unordered]<br>parquet at PersistJob.java:35"]; } subgraph cluster205836 { label="WholeStageCodegen (28)"; 89122 [labelType="html" label="ZippedPartitionsRDD2 [89122] [Unordered]<br>parquet at PersistJob.java:35"]; 89123 [labelType="html" label="MapPartitionsRDD [89123] [Unordered]<br>parquet at PersistJob.java:35"]; } subgraph cluster205835 { label="Window"; 89124 [labelType="html" label="MapPartitionsRDD [89124] [Unordered]<br>parquet at PersistJob.java:35"]; } subgraph cluster205832 { label="WholeStageCodegen (29)"; 89125 [labelType="html" label="MapPartitionsRDD [89125] [Unordered]<br>parquet at PersistJob.java:35"]; } subgraph cluster205865 { label="AQEShuffleRead"; 89126 [labelType="html" label="ShuffledRowRDD [89126] [Unordered]<br>parquet at PersistJob.java:35"]; } subgraph cluster205862 { label="WholeStageCodegen (30)"; 89127 [labelType="html" label="MapPartitionsRDD [89127] [Unordered]<br>parquet at PersistJob.java:35"]; } subgraph cluster205861 { label="Window"; 89128 [labelType="html" label="MapPartitionsRDD [89128] [Unordered]<br>parquet at PersistJob.java:35"]; } subgraph cluster205857 { label="WholeStageCodegen (31)"; 89129 [labelType="html" label="MapPartitionsRDD [89129] [Unordered]<br>parquet at PersistJob.java:35"]; } subgraph cluster205826 { label="WholeStageCodegen (32)"; 89130 [labelType="html" label="ZippedPartitionsRDD2 [89130] [Unordered]<br>parquet at PersistJob.java:35"]; 89131 [labelType="html" label="MapPartitionsRDD [89131] [Unordered]<br>parquet at PersistJob.java:35"]; } subgraph cluster205870 { label="Scan parquet "; 89132 [labelType="html" label="FileScanRDD [89132]<br>parquet at PersistJob.java:35"]; 89133 [labelType="html" label="MapPartitionsRDD [89133]<br>parquet at PersistJob.java:35"]; } subgraph cluster205866 { label="WholeStageCodegen (33)"; 89134 [labelType="html" label="MapPartitionsRDD [89134]<br>parquet at PersistJob.java:35"]; } subgraph cluster205825 { label="CartesianProduct"; 89135 [labelType="html" label="UnsafeCartesianRDD [89135] [Unordered]<br>parquet at PersistJob.java:35"]; 89136 [labelType="html" label="MapPartitionsRDD [89136] [Unordered]<br>parquet at PersistJob.java:35"]; } subgraph cluster205822 { label="WholeStageCodegen (34)"; 89137 [labelType="html" label="MapPartitionsRDD [89137] [Unordered]<br>parquet at PersistJob.java:35"]; } subgraph cluster205821 { label="Exchange"; 89138 [labelType="html" label="MapPartitionsRDD [89138] [Unordered]<br>parquet at PersistJob.java:35"]; } } 89116->89117; 89118->89119; 89119->89120; 89120->89121; 89117->89122; 89121->89122; 89122->89123; 89123->89124; 89124->89125; 89126->89127; 89127->89128; 89128->89129; 89125->89130; 89129->89130; 89130->89131; 89132->89133; 89133->89134; 89131->89135; 89134->89135; 89135->89136; 89136->89137; 89137->89138; }
89044,89116
89115,89118
89060,89126
Event Timeline
Enable zooming
Tasks: 5. 1 Pages. Jump to
. Show
items in a page.
Go
Scheduler Delay
Task Deserialization Time
Shuffle Read Time
Executor Computing Time
Shuffle Write Time
Result Serialization Time
Getting Result Time