3.5.7
Jobs
Stages
Storage
Environment
Executors
SQL / DataFrame
SP1
application UI
Details for Stage 16331 (Attempt 0)
Resource Profile Id:
0
Total Time Across All Tasks:
5 s
Locality Level Summary:
Any: 15
Input Size / Records:
2.6 KiB / 15
Shuffle Read Size / Records:
19.7 MiB / 259086
Shuffle Write Size / Records:
8.4 MiB / 175588
Associated Job Ids:
8519
DAG Visualization
digraph G { subgraph clusterstage_16331 { label="Stage 16331"; subgraph cluster105442 { label="Exchange"; 44564 [labelType="html" label="ShuffledRowRDD [44564] [Unordered]<br>parquet at PersistJob.java:35"]; } subgraph cluster105438 { label="WholeStageCodegen (12)"; 44565 [labelType="html" label="MapPartitionsRDD [44565] [Unordered]<br>parquet at PersistJob.java:35"]; } subgraph cluster105437 { label="Window"; 44566 [labelType="html" label="MapPartitionsRDD [44566] [Unordered]<br>parquet at PersistJob.java:35"]; } subgraph cluster105433 { label="WholeStageCodegen (13)"; 44567 [labelType="html" label="MapPartitionsRDD [44567] [Unordered]<br>parquet at PersistJob.java:35"]; } subgraph cluster105454 { label="AQEShuffleRead"; 44568 [labelType="html" label="ShuffledRowRDD [44568] [Unordered]<br>parquet at PersistJob.java:35"]; } subgraph cluster105451 { label="WholeStageCodegen (14)"; 44569 [labelType="html" label="MapPartitionsRDD [44569] [Unordered]<br>parquet at PersistJob.java:35"]; } subgraph cluster105461 { label="AQEShuffleRead"; 44570 [labelType="html" label="ShuffledRowRDD [44570] [Unordered]<br>parquet at PersistJob.java:35"]; } subgraph cluster105460 { label="ObjectHashAggregate"; 44571 [labelType="html" label="MapPartitionsRDD [44571] [Unordered]<br>parquet at PersistJob.java:35"]; } subgraph cluster105456 { label="WholeStageCodegen (15)"; 44572 [labelType="html" label="MapPartitionsRDD [44572] [Unordered]<br>parquet at PersistJob.java:35"]; } subgraph cluster105443 { label="WholeStageCodegen (16)"; 44573 [labelType="html" label="ZippedPartitionsRDD2 [44573] [Unordered]<br>parquet at PersistJob.java:35"]; 44574 [labelType="html" label="MapPartitionsRDD [44574] [Unordered]<br>parquet at PersistJob.java:35"]; } subgraph cluster105432 { label="Union"; 44575 [labelType="html" label="UnionRDD [44575] [Unordered]<br>parquet at PersistJob.java:35"]; } subgraph cluster105466 { label="Scan csv "; 44576 [labelType="html" label="FileScanRDD [44576]<br>parquet at PersistJob.java:35"]; 44577 [labelType="html" label="MapPartitionsRDD [44577]<br>parquet at PersistJob.java:35"]; } subgraph cluster105462 { label="WholeStageCodegen (17)"; 44578 [labelType="html" label="MapPartitionsRDD [44578]<br>parquet at PersistJob.java:35"]; } subgraph cluster105431 { label="CartesianProduct"; 44579 [labelType="html" label="UnsafeCartesianRDD [44579] [Unordered]<br>parquet at PersistJob.java:35"]; 44580 [labelType="html" label="MapPartitionsRDD [44580] [Unordered]<br>parquet at PersistJob.java:35"]; } subgraph cluster105430 { label="Exchange"; 44581 [labelType="html" label="MapPartitionsRDD [44581] [Unordered]<br>parquet at PersistJob.java:35"]; } } 44564->44565; 44565->44566; 44566->44567; 44568->44569; 44570->44571; 44571->44572; 44569->44573; 44572->44573; 44573->44574; 44567->44575; 44574->44575; 44576->44577; 44577->44578; 44575->44579; 44578->44579; 44579->44580; 44580->44581; }
44557,44564
44546,44568
44563,44570
Event Timeline
Enable zooming
Tasks: 15. 1 Pages. Jump to
. Show
items in a page.
Go
Scheduler Delay
Task Deserialization Time
Shuffle Read Time
Executor Computing Time
Shuffle Write Time
Result Serialization Time
Getting Result Time