3.5.7
Jobs
Stages
Storage
Environment
Executors
SQL / DataFrame
SP1
application UI
Details for Stage 7453 (Attempt 0)
Resource Profile Id:
0
Total Time Across All Tasks:
42 s
Locality Level Summary:
Any: 10
Input Size / Records:
36.0 KiB / 10
Shuffle Read Size / Records:
34.9 MiB / 1240727
Shuffle Write Size / Records:
76.7 MiB / 1115956
Associated Job Ids:
3667
DAG Visualization
digraph G { subgraph clusterstage_7453 { label="Stage 7453"; subgraph cluster48424 { label="Exchange"; 21390 [labelType="html" label="ShuffledRowRDD [21390] [Unordered]<br>parquet at PersistJob.java:35"]; } subgraph cluster48420 { label="WholeStageCodegen (22)"; 21391 [labelType="html" label="MapPartitionsRDD [21391] [Unordered]<br>parquet at PersistJob.java:35"]; } subgraph cluster48435 { label="Exchange"; 21392 [labelType="html" label="ShuffledRowRDD [21392] [Unordered]<br>parquet at PersistJob.java:35"]; } subgraph cluster48431 { label="WholeStageCodegen (23)"; 21393 [labelType="html" label="MapPartitionsRDD [21393] [Unordered]<br>parquet at PersistJob.java:35"]; } subgraph cluster48430 { label="Window"; 21394 [labelType="html" label="MapPartitionsRDD [21394] [Unordered]<br>parquet at PersistJob.java:35"]; } subgraph cluster48426 { label="WholeStageCodegen (24)"; 21395 [labelType="html" label="MapPartitionsRDD [21395] [Unordered]<br>parquet at PersistJob.java:35"]; } subgraph cluster48414 { label="WholeStageCodegen (25)"; 21396 [labelType="html" label="ZippedPartitionsRDD2 [21396] [Unordered]<br>parquet at PersistJob.java:35"]; 21397 [labelType="html" label="MapPartitionsRDD [21397] [Unordered]<br>parquet at PersistJob.java:35"]; } subgraph cluster48413 { label="Window"; 21398 [labelType="html" label="MapPartitionsRDD [21398] [Unordered]<br>parquet at PersistJob.java:35"]; } subgraph cluster48410 { label="WholeStageCodegen (26)"; 21399 [labelType="html" label="MapPartitionsRDD [21399] [Unordered]<br>parquet at PersistJob.java:35"]; } subgraph cluster48446 { label="Exchange"; 21400 [labelType="html" label="ShuffledRowRDD [21400] [Unordered]<br>parquet at PersistJob.java:35"]; } subgraph cluster48442 { label="WholeStageCodegen (27)"; 21401 [labelType="html" label="MapPartitionsRDD [21401] [Unordered]<br>parquet at PersistJob.java:35"]; } subgraph cluster48441 { label="Window"; 21402 [labelType="html" label="MapPartitionsRDD [21402] [Unordered]<br>parquet at PersistJob.java:35"]; } subgraph cluster48437 { label="WholeStageCodegen (28)"; 21403 [labelType="html" label="MapPartitionsRDD [21403] [Unordered]<br>parquet at PersistJob.java:35"]; } subgraph cluster48404 { label="WholeStageCodegen (29)"; 21404 [labelType="html" label="ZippedPartitionsRDD2 [21404] [Unordered]<br>parquet at PersistJob.java:35"]; 21405 [labelType="html" label="MapPartitionsRDD [21405] [Unordered]<br>parquet at PersistJob.java:35"]; } subgraph cluster48451 { label="Scan parquet "; 21406 [labelType="html" label="FileScanRDD [21406]<br>parquet at PersistJob.java:35"]; 21407 [labelType="html" label="MapPartitionsRDD [21407]<br>parquet at PersistJob.java:35"]; } subgraph cluster48447 { label="WholeStageCodegen (30)"; 21408 [labelType="html" label="MapPartitionsRDD [21408]<br>parquet at PersistJob.java:35"]; } subgraph cluster48403 { label="CartesianProduct"; 21409 [labelType="html" label="UnsafeCartesianRDD [21409] [Unordered]<br>parquet at PersistJob.java:35"]; 21410 [labelType="html" label="MapPartitionsRDD [21410] [Unordered]<br>parquet at PersistJob.java:35"]; } subgraph cluster48400 { label="WholeStageCodegen (31)"; 21411 [labelType="html" label="MapPartitionsRDD [21411] [Unordered]<br>parquet at PersistJob.java:35"]; } subgraph cluster48399 { label="Exchange"; 21412 [labelType="html" label="MapPartitionsRDD [21412] [Unordered]<br>parquet at PersistJob.java:35"]; } } 21390->21391; 21392->21393; 21393->21394; 21394->21395; 21391->21396; 21395->21396; 21396->21397; 21397->21398; 21398->21399; 21400->21401; 21401->21402; 21402->21403; 21399->21404; 21403->21404; 21404->21405; 21406->21407; 21407->21408; 21405->21409; 21408->21409; 21409->21410; 21410->21411; 21411->21412; }
21325,21390
21389,21392
21341,21400
Event Timeline
Enable zooming
Tasks: 10. 1 Pages. Jump to
. Show
items in a page.
Go
Scheduler Delay
Task Deserialization Time
Shuffle Read Time
Executor Computing Time
Shuffle Write Time
Result Serialization Time
Getting Result Time