3.5.7
Jobs
Stages
Storage
Environment
Executors
SQL / DataFrame
SP1
application UI
Details for Stage 10379 (Attempt 0)
Resource Profile Id:
0
Total Time Across All Tasks:
20 s
Locality Level Summary:
Any: 5
Input Size / Records:
16.9 KiB / 5
Shuffle Read Size / Records:
8.1 MiB / 988926
Shuffle Write Size / Records:
20.6 MiB / 790389
Associated Job Ids:
4553
DAG Visualization
digraph G { subgraph clusterstage_10379 { label="Stage 10379"; subgraph cluster60758 { label="AQEShuffleRead"; 25740 [labelType="html" label="ShuffledRowRDD [25740] [Unordered]<br>parquet at PersistJob.java:35"]; } subgraph cluster60755 { label="WholeStageCodegen (11)"; 25741 [labelType="html" label="MapPartitionsRDD [25741] [Unordered]<br>parquet at PersistJob.java:35"]; } subgraph cluster60768 { label="AQEShuffleRead"; 25742 [labelType="html" label="ShuffledRowRDD [25742] [Unordered]<br>parquet at PersistJob.java:35"]; } subgraph cluster60765 { label="WholeStageCodegen (12)"; 25743 [labelType="html" label="MapPartitionsRDD [25743] [Unordered]<br>parquet at PersistJob.java:35"]; } subgraph cluster60764 { label="Window"; 25744 [labelType="html" label="MapPartitionsRDD [25744] [Unordered]<br>parquet at PersistJob.java:35"]; } subgraph cluster60760 { label="WholeStageCodegen (13)"; 25745 [labelType="html" label="MapPartitionsRDD [25745] [Unordered]<br>parquet at PersistJob.java:35"]; } subgraph cluster60749 { label="WholeStageCodegen (14)"; 25746 [labelType="html" label="ZippedPartitionsRDD2 [25746] [Unordered]<br>parquet at PersistJob.java:35"]; 25747 [labelType="html" label="MapPartitionsRDD [25747] [Unordered]<br>parquet at PersistJob.java:35"]; } subgraph cluster60748 { label="Window"; 25748 [labelType="html" label="MapPartitionsRDD [25748] [Unordered]<br>parquet at PersistJob.java:35"]; } subgraph cluster60745 { label="WholeStageCodegen (15)"; 25749 [labelType="html" label="MapPartitionsRDD [25749] [Unordered]<br>parquet at PersistJob.java:35"]; } subgraph cluster60778 { label="AQEShuffleRead"; 25750 [labelType="html" label="ShuffledRowRDD [25750] [Unordered]<br>parquet at PersistJob.java:35"]; } subgraph cluster60775 { label="WholeStageCodegen (16)"; 25751 [labelType="html" label="MapPartitionsRDD [25751] [Unordered]<br>parquet at PersistJob.java:35"]; } subgraph cluster60774 { label="Window"; 25752 [labelType="html" label="MapPartitionsRDD [25752] [Unordered]<br>parquet at PersistJob.java:35"]; } subgraph cluster60770 { label="WholeStageCodegen (17)"; 25753 [labelType="html" label="MapPartitionsRDD [25753] [Unordered]<br>parquet at PersistJob.java:35"]; } subgraph cluster60739 { label="WholeStageCodegen (18)"; 25754 [labelType="html" label="ZippedPartitionsRDD2 [25754] [Unordered]<br>parquet at PersistJob.java:35"]; 25755 [labelType="html" label="MapPartitionsRDD [25755] [Unordered]<br>parquet at PersistJob.java:35"]; } subgraph cluster60783 { label="Scan parquet "; 25756 [labelType="html" label="FileScanRDD [25756]<br>parquet at PersistJob.java:35"]; 25757 [labelType="html" label="MapPartitionsRDD [25757]<br>parquet at PersistJob.java:35"]; } subgraph cluster60779 { label="WholeStageCodegen (19)"; 25758 [labelType="html" label="MapPartitionsRDD [25758]<br>parquet at PersistJob.java:35"]; } subgraph cluster60738 { label="CartesianProduct"; 25759 [labelType="html" label="UnsafeCartesianRDD [25759] [Unordered]<br>parquet at PersistJob.java:35"]; 25760 [labelType="html" label="MapPartitionsRDD [25760] [Unordered]<br>parquet at PersistJob.java:35"]; } subgraph cluster60735 { label="WholeStageCodegen (20)"; 25761 [labelType="html" label="MapPartitionsRDD [25761] [Unordered]<br>parquet at PersistJob.java:35"]; } subgraph cluster60734 { label="Exchange"; 25762 [labelType="html" label="MapPartitionsRDD [25762] [Unordered]<br>parquet at PersistJob.java:35"]; } } 25740->25741; 25742->25743; 25743->25744; 25744->25745; 25741->25746; 25745->25746; 25746->25747; 25747->25748; 25748->25749; 25750->25751; 25751->25752; 25752->25753; 25749->25754; 25753->25754; 25754->25755; 25756->25757; 25757->25758; 25755->25759; 25758->25759; 25759->25760; 25760->25761; 25761->25762; }
25704,25740
25739,25742
25712,25750
Event Timeline
Enable zooming
Tasks: 5. 1 Pages. Jump to
. Show
items in a page.
Go
Scheduler Delay
Task Deserialization Time
Shuffle Read Time
Executor Computing Time
Shuffle Write Time
Result Serialization Time
Getting Result Time