3.5.7
Jobs
Stages
Storage
Environment
Executors
SQL / DataFrame
SP1
application UI
Details for Stage 44983 (Attempt 0)
Resource Profile Id:
0
Total Time Across All Tasks:
7 s
Locality Level Summary:
Any: 15
Input Size / Records:
2.5 KiB / 15
Shuffle Read Size / Records:
38.2 MiB / 519530
Shuffle Write Size / Records:
16.9 MiB / 344297
Associated Job Ids:
23864
DAG Visualization
digraph G { subgraph clusterstage_44983 { label="Stage 44983"; subgraph cluster310390 { label="Exchange"; 135897 [labelType="html" label="ShuffledRowRDD [135897] [Unordered]<br>parquet at PersistJob.java:35"]; } subgraph cluster310386 { label="WholeStageCodegen (12)"; 135898 [labelType="html" label="MapPartitionsRDD [135898] [Unordered]<br>parquet at PersistJob.java:35"]; } subgraph cluster310385 { label="Window"; 135899 [labelType="html" label="MapPartitionsRDD [135899] [Unordered]<br>parquet at PersistJob.java:35"]; } subgraph cluster310381 { label="WholeStageCodegen (13)"; 135900 [labelType="html" label="MapPartitionsRDD [135900] [Unordered]<br>parquet at PersistJob.java:35"]; } subgraph cluster310402 { label="AQEShuffleRead"; 135901 [labelType="html" label="ShuffledRowRDD [135901] [Unordered]<br>parquet at PersistJob.java:35"]; } subgraph cluster310399 { label="WholeStageCodegen (14)"; 135902 [labelType="html" label="MapPartitionsRDD [135902] [Unordered]<br>parquet at PersistJob.java:35"]; } subgraph cluster310409 { label="AQEShuffleRead"; 135903 [labelType="html" label="ShuffledRowRDD [135903] [Unordered]<br>parquet at PersistJob.java:35"]; } subgraph cluster310408 { label="ObjectHashAggregate"; 135904 [labelType="html" label="MapPartitionsRDD [135904] [Unordered]<br>parquet at PersistJob.java:35"]; } subgraph cluster310404 { label="WholeStageCodegen (15)"; 135905 [labelType="html" label="MapPartitionsRDD [135905] [Unordered]<br>parquet at PersistJob.java:35"]; } subgraph cluster310391 { label="WholeStageCodegen (16)"; 135906 [labelType="html" label="ZippedPartitionsRDD2 [135906] [Unordered]<br>parquet at PersistJob.java:35"]; 135907 [labelType="html" label="MapPartitionsRDD [135907] [Unordered]<br>parquet at PersistJob.java:35"]; } subgraph cluster310380 { label="Union"; 135908 [labelType="html" label="UnionRDD [135908] [Unordered]<br>parquet at PersistJob.java:35"]; } subgraph cluster310414 { label="Scan csv "; 135909 [labelType="html" label="FileScanRDD [135909]<br>parquet at PersistJob.java:35"]; 135910 [labelType="html" label="MapPartitionsRDD [135910]<br>parquet at PersistJob.java:35"]; } subgraph cluster310410 { label="WholeStageCodegen (17)"; 135911 [labelType="html" label="MapPartitionsRDD [135911]<br>parquet at PersistJob.java:35"]; } subgraph cluster310379 { label="CartesianProduct"; 135912 [labelType="html" label="UnsafeCartesianRDD [135912] [Unordered]<br>parquet at PersistJob.java:35"]; 135913 [labelType="html" label="MapPartitionsRDD [135913] [Unordered]<br>parquet at PersistJob.java:35"]; } subgraph cluster310378 { label="Exchange"; 135914 [labelType="html" label="MapPartitionsRDD [135914] [Unordered]<br>parquet at PersistJob.java:35"]; } } 135897->135898; 135898->135899; 135899->135900; 135901->135902; 135903->135904; 135904->135905; 135902->135906; 135905->135906; 135906->135907; 135900->135908; 135907->135908; 135909->135910; 135910->135911; 135908->135912; 135911->135912; 135912->135913; 135913->135914; }
135890,135897
135879,135901
135896,135903
Event Timeline
Enable zooming
Tasks: 15. 1 Pages. Jump to
. Show
items in a page.
Go
Scheduler Delay
Task Deserialization Time
Shuffle Read Time
Executor Computing Time
Shuffle Write Time
Result Serialization Time
Getting Result Time