3.5.7
Jobs
Stages
Storage
Environment
Executors
SQL / DataFrame
SP1
application UI
Details for Stage 17139 (Attempt 0)
Resource Profile Id:
0
Total Time Across All Tasks:
6 s
Locality Level Summary:
Any: 15
Input Size / Records:
2.6 KiB / 15
Shuffle Read Size / Records:
26.2 MiB / 357035
Shuffle Write Size / Records:
11.4 MiB / 261221
Associated Job Ids:
8340
DAG Visualization
digraph G { subgraph clusterstage_17139 { label="Stage 17139"; subgraph cluster109895 { label="Exchange"; 46960 [labelType="html" label="ShuffledRowRDD [46960] [Unordered]<br>parquet at PersistJob.java:35"]; } subgraph cluster109891 { label="WholeStageCodegen (12)"; 46961 [labelType="html" label="MapPartitionsRDD [46961] [Unordered]<br>parquet at PersistJob.java:35"]; } subgraph cluster109890 { label="Window"; 46962 [labelType="html" label="MapPartitionsRDD [46962] [Unordered]<br>parquet at PersistJob.java:35"]; } subgraph cluster109886 { label="WholeStageCodegen (13)"; 46963 [labelType="html" label="MapPartitionsRDD [46963] [Unordered]<br>parquet at PersistJob.java:35"]; } subgraph cluster109907 { label="AQEShuffleRead"; 46964 [labelType="html" label="ShuffledRowRDD [46964] [Unordered]<br>parquet at PersistJob.java:35"]; } subgraph cluster109904 { label="WholeStageCodegen (14)"; 46965 [labelType="html" label="MapPartitionsRDD [46965] [Unordered]<br>parquet at PersistJob.java:35"]; } subgraph cluster109914 { label="AQEShuffleRead"; 46966 [labelType="html" label="ShuffledRowRDD [46966] [Unordered]<br>parquet at PersistJob.java:35"]; } subgraph cluster109913 { label="ObjectHashAggregate"; 46967 [labelType="html" label="MapPartitionsRDD [46967] [Unordered]<br>parquet at PersistJob.java:35"]; } subgraph cluster109909 { label="WholeStageCodegen (15)"; 46968 [labelType="html" label="MapPartitionsRDD [46968] [Unordered]<br>parquet at PersistJob.java:35"]; } subgraph cluster109896 { label="WholeStageCodegen (16)"; 46969 [labelType="html" label="ZippedPartitionsRDD2 [46969] [Unordered]<br>parquet at PersistJob.java:35"]; 46970 [labelType="html" label="MapPartitionsRDD [46970] [Unordered]<br>parquet at PersistJob.java:35"]; } subgraph cluster109885 { label="Union"; 46971 [labelType="html" label="UnionRDD [46971] [Unordered]<br>parquet at PersistJob.java:35"]; } subgraph cluster109919 { label="Scan csv "; 46972 [labelType="html" label="FileScanRDD [46972]<br>parquet at PersistJob.java:35"]; 46973 [labelType="html" label="MapPartitionsRDD [46973]<br>parquet at PersistJob.java:35"]; } subgraph cluster109915 { label="WholeStageCodegen (17)"; 46974 [labelType="html" label="MapPartitionsRDD [46974]<br>parquet at PersistJob.java:35"]; } subgraph cluster109884 { label="CartesianProduct"; 46975 [labelType="html" label="UnsafeCartesianRDD [46975] [Unordered]<br>parquet at PersistJob.java:35"]; 46976 [labelType="html" label="MapPartitionsRDD [46976] [Unordered]<br>parquet at PersistJob.java:35"]; } subgraph cluster109883 { label="Exchange"; 46977 [labelType="html" label="MapPartitionsRDD [46977] [Unordered]<br>parquet at PersistJob.java:35"]; } } 46960->46961; 46961->46962; 46962->46963; 46964->46965; 46966->46967; 46967->46968; 46965->46969; 46968->46969; 46969->46970; 46963->46971; 46970->46971; 46972->46973; 46973->46974; 46971->46975; 46974->46975; 46975->46976; 46976->46977; }
46959,46960
46942,46964
46952,46966
Event Timeline
Enable zooming
Tasks: 15. 1 Pages. Jump to
. Show
items in a page.
Go
Scheduler Delay
Task Deserialization Time
Shuffle Read Time
Executor Computing Time
Shuffle Write Time
Result Serialization Time
Getting Result Time