SP1 - Details for Query 3794

Submitted Time: 2026/06/12 15:18:02
Duration: 4 s
Succeeded Jobs: 5887 5888 5889 5890 5891 5892 5893

Show the Stage ID and Task ID that corresponds to the max metric

digraph G { 0 [labelType="html" label=" AdaptiveSparkPlan "]; 1 [labelType="html" label="Execute InsertIntoHadoopFsRelationCommand task commit time total (min, med, max (stageId: taskId)) 0 ms (0 ms, 0 ms, 0 ms (stage 13375.0: task 14725)) number of written files: 5 job commit time: 17 ms number of output rows: 857,416 number of dynamic part: 0 written output: 2.4 MiB"]; 2 [labelType="html" label=" WriteFiles "]; subgraph cluster3 { isCluster="true"; label="WholeStageCodegen (7)\n \nduration: total (min, med, max (stageId: taskId))\n1.5 s (275 ms, 310 ms, 330 ms (stage 13375.0: task 14724))"; 4 [labelType="html" label="Sort sort time total (min, med, max (stageId: taskId)) 105 ms (16 ms, 19 ms, 29 ms (stage 13375.0: task 14724)) peak memory total (min, med, max (stageId: taskId)) 104.0 MiB (20.0 MiB, 20.0 MiB, 24.0 MiB (stage 13375.0: task 14724)) spill size total (min, med, max (stageId: taskId)) 0.0 B (0.0 B, 0.0 B, 0.0 B (stage 13375.0: task 14725))"]; } 5 [labelType="html" label="AQEShuffleRead number of partitions: 5 partition data size total (min, med, max (stageId: taskId)) 8.6 MiB (1685.4 KiB, 1806.3 KiB, 1823.1 KiB (driver)) number of coalesced partitions: 5"]; 6 [labelType="html" label="Exchange shuffle records written: 857,416 local merged chunks fetched: 0 shuffle write time total (min, med, max (stageId: taskId)) 27 ms (2 ms, 3 ms, 13 ms (stage 13370.0: task 14718)) remote merged bytes read: 0.0 B local merged blocks fetched: 0 corrupt merged block chunks: 0 remote merged reqs duration: 0 ms remote merged blocks fetched: 0 records read: 857,416 local bytes read total (min, med, max (stageId: taskId)) 8.2 MiB (1582.7 KiB, 1699.9 KiB, 1734.1 KiB (stage 13375.0: task 14724)) fetch wait time total (min, med, max (stageId: taskId)) 0 ms (0 ms, 0 ms, 0 ms (stage 13375.0: task 14725)) remote bytes read: 0.0 B merged fetch fallback count: 0 local blocks read: 14 remote merged chunks fetched: 0 remote blocks read: 0 data size total (min, med, max (stageId: taskId)) 57.1 MiB (5.9 MiB, 6.4 MiB, 25.8 MiB (stage 13370.0: task 14718)) local merged bytes read: 0.0 B number of partitions: 10 remote reqs duration: 0 ms remote bytes read to disk: 0.0 B shuffle bytes written total (min, med, max (stageId: taskId)) 8.2 MiB (819.3 KiB, 888.4 KiB, 3.9 MiB (stage 13370.0: task 14718))"]; 7 [labelType="html" label=" Union "]; subgraph cluster8 { isCluster="true"; label="WholeStageCodegen (5)\n \nduration: total (min, med, max (stageId: taskId))\n527 ms (233 ms, 294 ms, 294 ms (stage 13370.0: task 14718))"; 9 [labelType="html" label=" Project "]; 10 [labelType="html" label="Generate number of output rows: 802,452"]; 11 [labelType="html" label=" Project "]; 12 [labelType="html" label="Filter number of output rows: 267,484"]; } 13 [labelType="html" label="Window spill size total (min, med, max (stageId: taskId)) 0.0 B (0.0 B, 0.0 B, 0.0 B (stage 13366.0: task 14712))"]; subgraph cluster14 { isCluster="true"; label="WholeStageCodegen (4)\n \nduration: total (min, med, max (stageId: taskId))\n749 ms (355 ms, 394 ms, 394 ms (stage 13370.0: task 14718))"; 15 [labelType="html" label="Sort sort time total (min, med, max (stageId: taskId)) 105 ms (51 ms, 54 ms, 54 ms (stage 13366.0: task 14712)) peak memory total (min, med, max (stageId: taskId)) 40.0 MiB (20.0 MiB, 20.0 MiB, 20.0 MiB (stage 13366.0: task 14712)) spill size total (min, med, max (stageId: taskId)) 0.0 B (0.0 B, 0.0 B, 0.0 B (stage 13366.0: task 14712))"]; } 16 [labelType="html" label="AQEShuffleRead number of partitions: 1 partition data size: 2037.3 KiB number of coalesced partitions: 1"]; 17 [labelType="html" label="Exchange shuffle records written: 133,742 local merged chunks fetched: 0 shuffle write time total (min, med, max (stageId: taskId)) 8 ms (3 ms, 4 ms, 4 ms (stage 13357.0: task 14690)) remote merged bytes read: 0.0 B local merged blocks fetched: 0 corrupt merged block chunks: 0 remote merged reqs duration: 0 ms remote merged blocks fetched: 0 records read: 267,484 local bytes read total (min, med, max (stageId: taskId)) 3.8 MiB (1931.5 KiB, 1931.5 KiB, 1931.5 KiB (stage 13366.0: task 14712)) fetch wait time total (min, med, max (stageId: taskId)) 0 ms (0 ms, 0 ms, 0 ms (stage 13366.0: task 14712)) remote bytes read: 0.0 B merged fetch fallback count: 0 local blocks read: 4 remote merged chunks fetched: 0 remote blocks read: 0 data size total (min, med, max (stageId: taskId)) 10.5 MiB (4.7 MiB, 5.8 MiB, 5.8 MiB (stage 13357.0: task 14690)) local merged bytes read: 0.0 B number of partitions: 10 remote reqs duration: 0 ms remote bytes read to disk: 0.0 B shuffle bytes written total (min, med, max (stageId: taskId)) 1931.5 KiB (857.2 KiB, 1074.3 KiB, 1074.3 KiB (stage 13357.0: task 14690))"]; subgraph cluster18 { isCluster="true"; label="WholeStageCodegen (1)\n \nduration: total (min, med, max (stageId: taskId))\n493 ms (205 ms, 288 ms, 288 ms (stage 13357.0: task 14690))"; 19 [labelType="html" label=" Project "]; 20 [labelType="html" label="Filter number of output rows: 133,742"]; } 21 [labelType="html" label="Scan csv number of output rows: 133,742 number of files read: 1 metadata time: 0 ms size of files read: 7.2 MiB"]; subgraph cluster22 { isCluster="true"; label="WholeStageCodegen (6)\n \nduration: total (min, med, max (stageId: taskId))\n1.4 s (114 ms, 133 ms, 164 ms (stage 13370.0: task 14723))"; 23 [labelType="html" label="Sort sort time total (min, med, max (stageId: taskId)) 485 ms (40 ms, 47 ms, 62 ms (stage 13366.0: task 14716)) peak memory total (min, med, max (stageId: taskId)) 196.0 MiB (18.0 MiB, 20.0 MiB, 20.0 MiB (stage 13366.0: task 14714)) spill size total (min, med, max (stageId: taskId)) 0.0 B (0.0 B, 0.0 B, 0.0 B (stage 13366.0: task 14714))"]; } 24 [labelType="html" label="AQEShuffleRead number of partitions: 5 partition data size total (min, med, max (stageId: taskId)) 6.9 MiB (1338.8 KiB, 1421.0 KiB, 1472.7 KiB (driver)) number of coalesced partitions: 5"]; 25 [labelType="html" label="Exchange shuffle records written: 456,190 local merged chunks fetched: 0 shuffle write time total (min, med, max (stageId: taskId)) 23 ms (4 ms, 4 ms, 4 ms (stage 13362.0: task 14710)) remote merged bytes read: 0.0 B local merged blocks fetched: 0 corrupt merged block chunks: 0 remote merged reqs duration: 0 ms remote merged blocks fetched: 0 records read: 912,380 local bytes read total (min, med, max (stageId: taskId)) 13.3 MiB (1289.4 KiB, 1376.8 KiB, 1441.4 KiB (stage 13366.0: task 14715)) fetch wait time total (min, med, max (stageId: taskId)) 0 ms (0 ms, 0 ms, 0 ms (stage 13366.0: task 14714)) remote bytes read: 0.0 B merged fetch fallback count: 0 local blocks read: 50 remote merged chunks fetched: 0 remote blocks read: 0 data size total (min, med, max (stageId: taskId)) 31.3 MiB (6.2 MiB, 6.3 MiB, 6.3 MiB (stage 13362.0: task 14709)) local merged bytes read: 0.0 B number of partitions: 10 remote reqs duration: 0 ms remote bytes read to disk: 0.0 B shuffle bytes written total (min, med, max (stageId: taskId)) 6.7 MiB (1352.5 KiB, 1365.4 KiB, 1367.0 KiB (stage 13362.0: task 14709))"]; subgraph cluster26 { isCluster="true"; label="WholeStageCodegen (3)\n \nduration: total (min, med, max (stageId: taskId))\n1.1 s (93 ms, 116 ms, 149 ms (stage 13362.0: task 14710))"; 27 [labelType="html" label="HashAggregate spill size: 0.0 B time in aggregation build total (min, med, max (stageId: taskId)) 716 ms (62 ms, 67 ms, 91 ms (stage 13362.0: task 14710)) peak memory total (min, med, max (stageId: taskId)) 170.0 MiB (17.0 MiB, 17.0 MiB, 17.0 MiB (stage 13360.0: task 14703)) number of output rows: 912,380 number of sort fallback tasks: 0 avg hash probes per key (min, med, max (stageId: taskId)): (1.5, 1.5, 1.5 (stage 13360.0: task 14703))"]; } 28 [labelType="html" label="AQEShuffleRead number of partitions: 5 partition data size total (min, med, max (stageId: taskId)) 6.9 MiB (1401.2 KiB, 1414.8 KiB, 1435.2 KiB (driver)) number of coalesced partitions: 5"]; 29 [labelType="html" label="Exchange shuffle records written: 456,260 local merged chunks fetched: 0 shuffle write time total (min, med, max (stageId: taskId)) 32 ms (2 ms, 3 ms, 4 ms (stage 13358.0: task 14700)) remote merged bytes read: 0.0 B local merged blocks fetched: 0 corrupt merged block chunks: 0 remote merged reqs duration: 0 ms remote merged blocks fetched: 0 records read: 912,520 local bytes read total (min, med, max (stageId: taskId)) 13.2 MiB (1342.2 KiB, 1354.5 KiB, 1357.3 KiB (stage 13360.0: task 14702)) fetch wait time total (min, med, max (stageId: taskId)) 0 ms (0 ms, 0 ms, 0 ms (stage 13360.0: task 14703)) remote bytes read: 0.0 B merged fetch fallback count: 0 local blocks read: 100 remote merged chunks fetched: 0 remote blocks read: 0 data size total (min, med, max (stageId: taskId)) 31.3 MiB (3.1 MiB, 3.1 MiB, 3.2 MiB (stage 13358.0: task 14692)) local merged bytes read: 0.0 B number of partitions: 10 remote reqs duration: 0 ms remote bytes read to disk: 0.0 B shuffle bytes written total (min, med, max (stageId: taskId)) 6.6 MiB (663.0 KiB, 676.2 KiB, 687.1 KiB (stage 13358.0: task 14692))"]; subgraph cluster30 { isCluster="true"; label="WholeStageCodegen (2)\n \nduration: total (min, med, max (stageId: taskId))\n24.9 s (2.4 s, 2.5 s, 2.6 s (stage 13358.0: task 14696))"; 31 [labelType="html" label="HashAggregate spill size: 0.0 B time in aggregation build total (min, med, max (stageId: taskId)) 24.6 s (2.3 s, 2.5 s, 2.5 s (stage 13358.0: task 14692)) peak memory total (min, med, max (stageId: taskId)) 162.5 MiB (16.3 MiB, 16.3 MiB, 16.3 MiB (stage 13358.0: task 14701)) number of output rows: 456,260 number of sort fallback tasks: 0 avg hash probes per key (min, med, max (stageId: taskId)): (1, 1, 1 (stage 13358.0: task 14701))"]; 32 [labelType="html" label=" Project "]; 33 [labelType="html" label="Generate number of output rows: 4,904,438"]; 34 [labelType="html" label=" Project "]; 35 [labelType="html" label="ColumnarToRow number of output rows: 700,634 number of input batches: 178"]; } 36 [labelType="html" label="Scan parquet number of files read: 10 scan time total (min, med, max (stageId: taskId)) 67 ms (5 ms, 7 ms, 10 ms (stage 13358.0: task 14692)) metadata time: 0 ms size of files read: 30.7 MiB number of output rows: 700,634"]; 1->0; 2->1; 4->2; 5->4; 6->5; 7->6; 9->7; 10->9; 11->10; 12->11; 13->12; 15->13; 16->15; 17->16; 19->17; 20->19; 21->20; 23->7; 24->23; 25->24; 27->25; 28->27; 29->28; 31->29; 32->31; 33->32; 34->33; 35->34; 36->35; }

AdaptiveSparkPlan isFinalPlan=true

Execute InsertIntoHadoopFsRelationCommand file:/data/output/cache/parquet/datablock/14_B_PASSAGE_PA_217771, false, Parquet, [path=file:///data/output/cache/parquet/datablock/14_B_PASSAGE_PA_217771], Overwrite, [COL_E5ADB260_1BF6_449C_8439_0A94CEE6956C, COL_59774853_4110_4672_B6E2_C0D5DE930914, COL_E58DD17C_E55F_4036_9FA9_A5365B045A1A]

WriteFiles

Sort [COL_E5ADB260_1BF6_449C_8439_0A94CEE6956C#1157261 ASC NULLS FIRST, COL_59774853_4110_4672_B6E2_C0D5DE930914#1157262 ASC NULLS FIRST], true, 0

WholeStageCodegen (7)

AQEShuffleRead coalesced

Exchange rangepartitioning(COL_E5ADB260_1BF6_449C_8439_0A94CEE6956C#1157261 ASC NULLS FIRST, COL_59774853_4110_4672_B6E2_C0D5DE930914#1157262 ASC NULLS FIRST, 10), ENSURE_REQUIREMENTS, [plan_id=695557]

Union

Project [COL_DEB05435_3A84_4F8F_8CD2_6BE209A14FA8#1157221 AS COL_E5ADB260_1BF6_449C_8439_0A94CEE6956C#1157261, col0#1157230 AS COL_59774853_4110_4672_B6E2_C0D5DE930914#1157262, col1#1157231 AS COL_E58DD17C_E55F_4036_9FA9_A5365B045A1A#1157263]

Generate stack(3, 01-UF, COL_48FD2AAE_6F4D_4D43_BEE7_8A161203783C#1157222, 02-Transport, COL_E0699193_FC70_4F64_BAE6_B2F241857F3B#1157223, 03-CCMU, COL_630C2F1A_DB89_4A08_A2D2_179A10CC9FD2#1157224), [COL_DEB05435_3A84_4F8F_8CD2_6BE209A14FA8#1157221], false, [col0#1157230, col1#1157231]

Project [COL_2AF4FE05_74B1_4A24_9BC1_7612D6ED731D#1157125 AS COL_DEB05435_3A84_4F8F_8CD2_6BE209A14FA8#1157221, COL_E82CFB57_01EB_4E90_836D_4D6F4CA131FE#1157126 AS COL_48FD2AAE_6F4D_4D43_BEE7_8A161203783C#1157222, COL_E1010A22_775B_43B4_B8CA_4603C0801D28#1157127 AS COL_E0699193_FC70_4F64_BAE6_B2F241857F3B#1157223, COL_6A6D32F5_2B4E_4DF6_A459_B248882F909D#1157128 AS COL_630C2F1A_DB89_4A08_A2D2_179A10CC9FD2#1157224]

Filter (COL_6B340AEB_6908_4A50_8AD6_FE561D33FBE0#1157141 <=> 1)

WholeStageCodegen (5)

Window [row_number() windowspecdefinition(COL_2AF4FE05_74B1_4A24_9BC1_7612D6ED731D#1157125, 1 DESC NULLS LAST, specifiedwindowframe(RowFrame, unboundedpreceding$(), currentrow$())) AS COL_6B340AEB_6908_4A50_8AD6_FE561D33FBE0#1157141], [COL_2AF4FE05_74B1_4A24_9BC1_7612D6ED731D#1157125], [1 DESC NULLS LAST]

Sort [COL_2AF4FE05_74B1_4A24_9BC1_7612D6ED731D#1157125 ASC NULLS FIRST, 1 DESC NULLS LAST], false, 0

WholeStageCodegen (4)

AQEShuffleRead coalesced

Exchange hashpartitioning(COL_2AF4FE05_74B1_4A24_9BC1_7612D6ED731D#1157125, 10), ENSURE_REQUIREMENTS, [plan_id=695282]

Project [Numéro Séjour#1157007 AS COL_2AF4FE05_74B1_4A24_9BC1_7612D6ED731D#1157125, UF#1157009 AS COL_E82CFB57_01EB_4E90_836D_4D6F4CA131FE#1157126, str_remove_space(Transport admission#1157015, ONLY_ONE) AS COL_E1010A22_775B_43B4_B8CA_4603C0801D28#1157127, str_remove_space(str_list_get(str_extract_by_regex(Niveau CCMU#1157012, [+\d]+), 0), ALL) AS COL_6A6D32F5_2B4E_4DF6_A459_B248882F909D#1157128]

Filter NOT CASE WHEN isnull(StartsWith(Numéro Séjour#1157007, #)) THEN false ELSE StartsWith(Numéro Séjour#1157007, #) END

WholeStageCodegen (1)

FileScan csv [Numéro Séjour#1157007,UF#1157009,Niveau CCMU#1157012,Transport admission#1157015] Batched: false, DataFilters: [NOT CASE WHEN isnull(StartsWith(Numéro Séjour#1157007, #)) THEN false ELSE StartsWith(Numéro Séj..., Format: CSV, Location: InMemoryFileIndex(1 paths)[file:/data/input/depot/csv/tmp/02_PASSAGE_PATI_217095/passage_Patient_..., PartitionFilters: [], PushedFilters: [], ReadSchema: struct<Numéro Séjour:string,UF:string,Niveau CCMU:string,Transport admission:string>

Sort [COL_DC1A3ABF_ADFC_4208_8B24_35ED54CC87DF#1156892 ASC NULLS FIRST, COL_EA9AE57C_DE84_4BBE_9131_053814B337F4#1156893 ASC NULLS FIRST], true, 0

WholeStageCodegen (6)

AQEShuffleRead coalesced

Exchange rangepartitioning(COL_DC1A3ABF_ADFC_4208_8B24_35ED54CC87DF#1156892 ASC NULLS FIRST, COL_EA9AE57C_DE84_4BBE_9131_053814B337F4#1156893 ASC NULLS FIRST, 10), ENSURE_REQUIREMENTS, [plan_id=695455]

HashAggregate(keys=[COL_8D7490B1_ED4B_4A07_9680_ECC66DF49E37#1156886, COL_7F6B8B23_E564_4B4E_8794_0FC7CE59D60B#1156887, COL_B6440F86_8BD4_4AFD_8640_45556688ABB9#1156888], functions=[])

WholeStageCodegen (3)

AQEShuffleRead coalesced

Exchange hashpartitioning(COL_8D7490B1_ED4B_4A07_9680_ECC66DF49E37#1156886, COL_7F6B8B23_E564_4B4E_8794_0FC7CE59D60B#1156887, COL_B6440F86_8BD4_4AFD_8640_45556688ABB9#1156888, 10), ENSURE_REQUIREMENTS, [plan_id=695311]

HashAggregate(keys=[COL_8D7490B1_ED4B_4A07_9680_ECC66DF49E37#1156886, COL_7F6B8B23_E564_4B4E_8794_0FC7CE59D60B#1156887, COL_B6440F86_8BD4_4AFD_8640_45556688ABB9#1156888], functions=[])

Project [COL_BC7CB40E_B2CA_42C4_94A5_B6AA996E14DC#1156840 AS COL_8D7490B1_ED4B_4A07_9680_ECC66DF49E37#1156886, col0#1156857 AS COL_7F6B8B23_E564_4B4E_8794_0FC7CE59D60B#1156887, col1#1156858 AS COL_B6440F86_8BD4_4AFD_8640_45556688ABB9#1156888]

Generate stack(7, 04-Date Naissance, COL_95E23023_63AA_4AB8_A60F_50D45234CFDD#1156841, 05-Date Entree, COL_78B23D10_75E8_415E_A3C0_8AB5C826A837#1156842, 06-Date Sortie, COL_885784F8_1772_40E3_89B6_6ACE1FAB6407#1156843, 07-Fides, COL_32AA235F_1BB1_4047_B8BA_AD2E9197AD1F#1156844, 08-Facturable AM, COL_3BBA8558_C3D5_4921_93AE_D7C9E273B39D#1156845, 09-Justif Exo, COL_35788C71_38EE_4A08_B685_AAC68457246A#1156846, 10-Nature Ass, COL_6EC95BD9_A121_4728_BE4A_49B7EA2976F2#1156847), [COL_BC7CB40E_B2CA_42C4_94A5_B6AA996E14DC#1156840], false, [col0#1156857, col1#1156858]

Project [COL_F76265AB_8748_4114_92B4_00E37DE25D34#1156522 AS COL_BC7CB40E_B2CA_42C4_94A5_B6AA996E14DC#1156840, COL_9262BBED_C393_416D_B38C_8F1458965BA7#1156521 AS COL_95E23023_63AA_4AB8_A60F_50D45234CFDD#1156841, COL_96E2282E_0771_460E_8BDB_D11261BD9020#1156523 AS COL_78B23D10_75E8_415E_A3C0_8AB5C826A837#1156842, COL_12028397_66B1_4E3C_BBDB_64537606E7DE#1156524 AS COL_885784F8_1772_40E3_89B6_6ACE1FAB6407#1156843, COL_0521CAB9_7617_4F31_948E_95DCA16387D7#1156519 AS COL_32AA235F_1BB1_4047_B8BA_AD2E9197AD1F#1156844, COL_1DACB81E_AFA5_4BB8_83D7_57DE8B8062B7#1156520 AS COL_3BBA8558_C3D5_4921_93AE_D7C9E273B39D#1156845, COL_4473F770_A575_4100_97F6_8190F900749A#1156528 AS COL_35788C71_38EE_4A08_B685_AAC68457246A#1156846, COL_715B3479_EAD2_422C_BF5C_7B020197D946#1156529 AS COL_6EC95BD9_A121_4728_BE4A_49B7EA2976F2#1156847]

ColumnarToRow

WholeStageCodegen (2)

FileScan parquet [COL_0521CAB9_7617_4F31_948E_95DCA16387D7#1156519,COL_1DACB81E_AFA5_4BB8_83D7_57DE8B8062B7#1156520,COL_9262BBED_C393_416D_B38C_8F1458965BA7#1156521,COL_F76265AB_8748_4114_92B4_00E37DE25D34#1156522,COL_96E2282E_0771_460E_8BDB_D11261BD9020#1156523,COL_12028397_66B1_4E3C_BBDB_64537606E7DE#1156524,COL_4473F770_A575_4100_97F6_8190F900749A#1156528,COL_715B3479_EAD2_422C_BF5C_7B020197D946#1156529] Batched: true, DataFilters: [], Format: Parquet, Location: InMemoryFileIndex(1 paths)[file:/data/output/cache/parquet/datablock/02_A_ELIGIBLE_217284], PartitionFilters: [], PushedFilters: [], ReadSchema: struct<COL_0521CAB9_7617_4F31_948E_95DCA16387D7:string,COL_1DACB81E_AFA5_4BB8_83D7_57DE8B8062B7:s...

Details

== Physical Plan ==
AdaptiveSparkPlan (56)
+- == Final Plan ==
   Execute InsertIntoHadoopFsRelationCommand (33)
   +- WriteFiles (32)
      +- * Sort (31)
         +- AQEShuffleRead (30)
            +- ShuffleQueryStage (29), Statistics(sizeInBytes=57.1 MiB, rowCount=8.57E+5)
               +- Exchange (28)
                  +- Union (27)
                     :- * Project (12)
                     :  +- * Generate (11)
                     :     +- * Project (10)
                     :        +- * Filter (9)
                     :           +- Window (8)
                     :              +- * Sort (7)
                     :                 +- AQEShuffleRead (6)
                     :                    +- ShuffleQueryStage (5), Statistics(sizeInBytes=10.5 MiB, rowCount=1.34E+5)
                     :                       +- Exchange (4)
                     :                          +- * Project (3)
                     :                             +- * Filter (2)
                     :                                +- Scan csv  (1)
                     +- * Sort (26)
                        +- AQEShuffleRead (25)
                           +- ShuffleQueryStage (24), Statistics(sizeInBytes=31.3 MiB, rowCount=4.56E+5)
                              +- Exchange (23)
                                 +- * HashAggregate (22)
                                    +- AQEShuffleRead (21)
                                       +- ShuffleQueryStage (20), Statistics(sizeInBytes=31.3 MiB, rowCount=4.56E+5)
                                          +- Exchange (19)
                                             +- * HashAggregate (18)
                                                +- * Project (17)
                                                   +- * Generate (16)
                                                      +- * Project (15)
                                                         +- * ColumnarToRow (14)
                                                            +- Scan parquet  (13)
+- == Initial Plan ==
   Execute InsertIntoHadoopFsRelationCommand (55)
   +- WriteFiles (54)
      +- Sort (53)
         +- Exchange (52)
            +- Union (51)
               :- Project (42)
               :  +- Generate (41)
               :     +- Project (40)
               :        +- Filter (39)
               :           +- Window (38)
               :              +- Sort (37)
               :                 +- Exchange (36)
               :                    +- Project (35)
               :                       +- Filter (34)
               :                          +- Scan csv  (1)
               +- Sort (50)
                  +- Exchange (49)
                     +- HashAggregate (48)
                        +- Exchange (47)
                           +- HashAggregate (46)
                              +- Project (45)
                                 +- Generate (44)
                                    +- Project (43)
                                       +- Scan parquet  (13)


(1) Scan csv 
Output [4]: [Numéro Séjour#1157007, UF#1157009, Niveau CCMU#1157012, Transport admission#1157015]
Batched: false
Location: InMemoryFileIndex [file:/data/input/depot/csv/tmp/02_PASSAGE_PATI_217095/passage_Patient_CHSF_M12025.csv]
ReadSchema: struct<Numéro Séjour:string,UF:string,Niveau CCMU:string,Transport admission:string>

(2) Filter [codegen id : 1]
Input [4]: [Numéro Séjour#1157007, UF#1157009, Niveau CCMU#1157012, Transport admission#1157015]
Condition : NOT CASE WHEN isnull(StartsWith(Numéro Séjour#1157007, #)) THEN false ELSE StartsWith(Numéro Séjour#1157007, #) END

(3) Project [codegen id : 1]
Output [4]: [Numéro Séjour#1157007 AS COL_2AF4FE05_74B1_4A24_9BC1_7612D6ED731D#1157125, UF#1157009 AS COL_E82CFB57_01EB_4E90_836D_4D6F4CA131FE#1157126, str_remove_space(Transport admission#1157015, ONLY_ONE) AS COL_E1010A22_775B_43B4_B8CA_4603C0801D28#1157127, str_remove_space(str_list_get(str_extract_by_regex(Niveau CCMU#1157012, [+\d]+), 0), ALL) AS COL_6A6D32F5_2B4E_4DF6_A459_B248882F909D#1157128]
Input [4]: [Numéro Séjour#1157007, UF#1157009, Niveau CCMU#1157012, Transport admission#1157015]

(4) Exchange
Input [4]: [COL_2AF4FE05_74B1_4A24_9BC1_7612D6ED731D#1157125, COL_E82CFB57_01EB_4E90_836D_4D6F4CA131FE#1157126, COL_E1010A22_775B_43B4_B8CA_4603C0801D28#1157127, COL_6A6D32F5_2B4E_4DF6_A459_B248882F909D#1157128]
Arguments: hashpartitioning(COL_2AF4FE05_74B1_4A24_9BC1_7612D6ED731D#1157125, 10), ENSURE_REQUIREMENTS, [plan_id=695282]

(5) ShuffleQueryStage
Output [4]: [COL_2AF4FE05_74B1_4A24_9BC1_7612D6ED731D#1157125, COL_E82CFB57_01EB_4E90_836D_4D6F4CA131FE#1157126, COL_E1010A22_775B_43B4_B8CA_4603C0801D28#1157127, COL_6A6D32F5_2B4E_4DF6_A459_B248882F909D#1157128]
Arguments: 0

(6) AQEShuffleRead
Input [4]: [COL_2AF4FE05_74B1_4A24_9BC1_7612D6ED731D#1157125, COL_E82CFB57_01EB_4E90_836D_4D6F4CA131FE#1157126, COL_E1010A22_775B_43B4_B8CA_4603C0801D28#1157127, COL_6A6D32F5_2B4E_4DF6_A459_B248882F909D#1157128]
Arguments: coalesced

(7) Sort [codegen id : 4]
Input [4]: [COL_2AF4FE05_74B1_4A24_9BC1_7612D6ED731D#1157125, COL_E82CFB57_01EB_4E90_836D_4D6F4CA131FE#1157126, COL_E1010A22_775B_43B4_B8CA_4603C0801D28#1157127, COL_6A6D32F5_2B4E_4DF6_A459_B248882F909D#1157128]
Arguments: [COL_2AF4FE05_74B1_4A24_9BC1_7612D6ED731D#1157125 ASC NULLS FIRST, 1 DESC NULLS LAST], false, 0

(8) Window
Input [4]: [COL_2AF4FE05_74B1_4A24_9BC1_7612D6ED731D#1157125, COL_E82CFB57_01EB_4E90_836D_4D6F4CA131FE#1157126, COL_E1010A22_775B_43B4_B8CA_4603C0801D28#1157127, COL_6A6D32F5_2B4E_4DF6_A459_B248882F909D#1157128]
Arguments: [row_number() windowspecdefinition(COL_2AF4FE05_74B1_4A24_9BC1_7612D6ED731D#1157125, 1 DESC NULLS LAST, specifiedwindowframe(RowFrame, unboundedpreceding$(), currentrow$())) AS COL_6B340AEB_6908_4A50_8AD6_FE561D33FBE0#1157141], [COL_2AF4FE05_74B1_4A24_9BC1_7612D6ED731D#1157125], [1 DESC NULLS LAST]

(9) Filter [codegen id : 5]
Input [5]: [COL_2AF4FE05_74B1_4A24_9BC1_7612D6ED731D#1157125, COL_E82CFB57_01EB_4E90_836D_4D6F4CA131FE#1157126, COL_E1010A22_775B_43B4_B8CA_4603C0801D28#1157127, COL_6A6D32F5_2B4E_4DF6_A459_B248882F909D#1157128, COL_6B340AEB_6908_4A50_8AD6_FE561D33FBE0#1157141]
Condition : (COL_6B340AEB_6908_4A50_8AD6_FE561D33FBE0#1157141 <=> 1)

(10) Project [codegen id : 5]
Output [4]: [COL_2AF4FE05_74B1_4A24_9BC1_7612D6ED731D#1157125 AS COL_DEB05435_3A84_4F8F_8CD2_6BE209A14FA8#1157221, COL_E82CFB57_01EB_4E90_836D_4D6F4CA131FE#1157126 AS COL_48FD2AAE_6F4D_4D43_BEE7_8A161203783C#1157222, COL_E1010A22_775B_43B4_B8CA_4603C0801D28#1157127 AS COL_E0699193_FC70_4F64_BAE6_B2F241857F3B#1157223, COL_6A6D32F5_2B4E_4DF6_A459_B248882F909D#1157128 AS COL_630C2F1A_DB89_4A08_A2D2_179A10CC9FD2#1157224]
Input [5]: [COL_2AF4FE05_74B1_4A24_9BC1_7612D6ED731D#1157125, COL_E82CFB57_01EB_4E90_836D_4D6F4CA131FE#1157126, COL_E1010A22_775B_43B4_B8CA_4603C0801D28#1157127, COL_6A6D32F5_2B4E_4DF6_A459_B248882F909D#1157128, COL_6B340AEB_6908_4A50_8AD6_FE561D33FBE0#1157141]

(11) Generate [codegen id : 5]
Input [4]: [COL_DEB05435_3A84_4F8F_8CD2_6BE209A14FA8#1157221, COL_48FD2AAE_6F4D_4D43_BEE7_8A161203783C#1157222, COL_E0699193_FC70_4F64_BAE6_B2F241857F3B#1157223, COL_630C2F1A_DB89_4A08_A2D2_179A10CC9FD2#1157224]
Arguments: stack(3, 01-UF, COL_48FD2AAE_6F4D_4D43_BEE7_8A161203783C#1157222, 02-Transport, COL_E0699193_FC70_4F64_BAE6_B2F241857F3B#1157223, 03-CCMU, COL_630C2F1A_DB89_4A08_A2D2_179A10CC9FD2#1157224), [COL_DEB05435_3A84_4F8F_8CD2_6BE209A14FA8#1157221], false, [col0#1157230, col1#1157231]

(12) Project [codegen id : 5]
Output [3]: [COL_DEB05435_3A84_4F8F_8CD2_6BE209A14FA8#1157221 AS COL_E5ADB260_1BF6_449C_8439_0A94CEE6956C#1157261, col0#1157230 AS COL_59774853_4110_4672_B6E2_C0D5DE930914#1157262, col1#1157231 AS COL_E58DD17C_E55F_4036_9FA9_A5365B045A1A#1157263]
Input [3]: [COL_DEB05435_3A84_4F8F_8CD2_6BE209A14FA8#1157221, col0#1157230, col1#1157231]

(13) Scan parquet 
Output [8]: [COL_0521CAB9_7617_4F31_948E_95DCA16387D7#1156519, COL_1DACB81E_AFA5_4BB8_83D7_57DE8B8062B7#1156520, COL_9262BBED_C393_416D_B38C_8F1458965BA7#1156521, COL_F76265AB_8748_4114_92B4_00E37DE25D34#1156522, COL_96E2282E_0771_460E_8BDB_D11261BD9020#1156523, COL_12028397_66B1_4E3C_BBDB_64537606E7DE#1156524, COL_4473F770_A575_4100_97F6_8190F900749A#1156528, COL_715B3479_EAD2_422C_BF5C_7B020197D946#1156529]
Batched: true
Location: InMemoryFileIndex [file:/data/output/cache/parquet/datablock/02_A_ELIGIBLE_217284]
ReadSchema: struct<COL_0521CAB9_7617_4F31_948E_95DCA16387D7:string,COL_1DACB81E_AFA5_4BB8_83D7_57DE8B8062B7:string,COL_9262BBED_C393_416D_B38C_8F1458965BA7:string,COL_F76265AB_8748_4114_92B4_00E37DE25D34:string,COL_96E2282E_0771_460E_8BDB_D11261BD9020:string,COL_12028397_66B1_4E3C_BBDB_64537606E7DE:string,COL_4473F770_A575_4100_97F6_8190F900749A:string,COL_715B3479_EAD2_422C_BF5C_7B020197D946:string>

(14) ColumnarToRow [codegen id : 2]
Input [8]: [COL_0521CAB9_7617_4F31_948E_95DCA16387D7#1156519, COL_1DACB81E_AFA5_4BB8_83D7_57DE8B8062B7#1156520, COL_9262BBED_C393_416D_B38C_8F1458965BA7#1156521, COL_F76265AB_8748_4114_92B4_00E37DE25D34#1156522, COL_96E2282E_0771_460E_8BDB_D11261BD9020#1156523, COL_12028397_66B1_4E3C_BBDB_64537606E7DE#1156524, COL_4473F770_A575_4100_97F6_8190F900749A#1156528, COL_715B3479_EAD2_422C_BF5C_7B020197D946#1156529]

(15) Project [codegen id : 2]
Output [8]: [COL_F76265AB_8748_4114_92B4_00E37DE25D34#1156522 AS COL_BC7CB40E_B2CA_42C4_94A5_B6AA996E14DC#1156840, COL_9262BBED_C393_416D_B38C_8F1458965BA7#1156521 AS COL_95E23023_63AA_4AB8_A60F_50D45234CFDD#1156841, COL_96E2282E_0771_460E_8BDB_D11261BD9020#1156523 AS COL_78B23D10_75E8_415E_A3C0_8AB5C826A837#1156842, COL_12028397_66B1_4E3C_BBDB_64537606E7DE#1156524 AS COL_885784F8_1772_40E3_89B6_6ACE1FAB6407#1156843, COL_0521CAB9_7617_4F31_948E_95DCA16387D7#1156519 AS COL_32AA235F_1BB1_4047_B8BA_AD2E9197AD1F#1156844, COL_1DACB81E_AFA5_4BB8_83D7_57DE8B8062B7#1156520 AS COL_3BBA8558_C3D5_4921_93AE_D7C9E273B39D#1156845, COL_4473F770_A575_4100_97F6_8190F900749A#1156528 AS COL_35788C71_38EE_4A08_B685_AAC68457246A#1156846, COL_715B3479_EAD2_422C_BF5C_7B020197D946#1156529 AS COL_6EC95BD9_A121_4728_BE4A_49B7EA2976F2#1156847]
Input [8]: [COL_0521CAB9_7617_4F31_948E_95DCA16387D7#1156519, COL_1DACB81E_AFA5_4BB8_83D7_57DE8B8062B7#1156520, COL_9262BBED_C393_416D_B38C_8F1458965BA7#1156521, COL_F76265AB_8748_4114_92B4_00E37DE25D34#1156522, COL_96E2282E_0771_460E_8BDB_D11261BD9020#1156523, COL_12028397_66B1_4E3C_BBDB_64537606E7DE#1156524, COL_4473F770_A575_4100_97F6_8190F900749A#1156528, COL_715B3479_EAD2_422C_BF5C_7B020197D946#1156529]

(16) Generate [codegen id : 2]
Input [8]: [COL_BC7CB40E_B2CA_42C4_94A5_B6AA996E14DC#1156840, COL_95E23023_63AA_4AB8_A60F_50D45234CFDD#1156841, COL_78B23D10_75E8_415E_A3C0_8AB5C826A837#1156842, COL_885784F8_1772_40E3_89B6_6ACE1FAB6407#1156843, COL_32AA235F_1BB1_4047_B8BA_AD2E9197AD1F#1156844, COL_3BBA8558_C3D5_4921_93AE_D7C9E273B39D#1156845, COL_35788C71_38EE_4A08_B685_AAC68457246A#1156846, COL_6EC95BD9_A121_4728_BE4A_49B7EA2976F2#1156847]
Arguments: stack(7, 04-Date Naissance, COL_95E23023_63AA_4AB8_A60F_50D45234CFDD#1156841, 05-Date Entree, COL_78B23D10_75E8_415E_A3C0_8AB5C826A837#1156842, 06-Date Sortie, COL_885784F8_1772_40E3_89B6_6ACE1FAB6407#1156843, 07-Fides, COL_32AA235F_1BB1_4047_B8BA_AD2E9197AD1F#1156844, 08-Facturable AM, COL_3BBA8558_C3D5_4921_93AE_D7C9E273B39D#1156845, 09-Justif Exo, COL_35788C71_38EE_4A08_B685_AAC68457246A#1156846, 10-Nature Ass, COL_6EC95BD9_A121_4728_BE4A_49B7EA2976F2#1156847), [COL_BC7CB40E_B2CA_42C4_94A5_B6AA996E14DC#1156840], false, [col0#1156857, col1#1156858]

(17) Project [codegen id : 2]
Output [3]: [COL_BC7CB40E_B2CA_42C4_94A5_B6AA996E14DC#1156840 AS COL_8D7490B1_ED4B_4A07_9680_ECC66DF49E37#1156886, col0#1156857 AS COL_7F6B8B23_E564_4B4E_8794_0FC7CE59D60B#1156887, col1#1156858 AS COL_B6440F86_8BD4_4AFD_8640_45556688ABB9#1156888]
Input [3]: [COL_BC7CB40E_B2CA_42C4_94A5_B6AA996E14DC#1156840, col0#1156857, col1#1156858]

(18) HashAggregate [codegen id : 2]
Input [3]: [COL_8D7490B1_ED4B_4A07_9680_ECC66DF49E37#1156886, COL_7F6B8B23_E564_4B4E_8794_0FC7CE59D60B#1156887, COL_B6440F86_8BD4_4AFD_8640_45556688ABB9#1156888]
Keys [3]: [COL_8D7490B1_ED4B_4A07_9680_ECC66DF49E37#1156886, COL_7F6B8B23_E564_4B4E_8794_0FC7CE59D60B#1156887, COL_B6440F86_8BD4_4AFD_8640_45556688ABB9#1156888]
Functions: []
Aggregate Attributes: []
Results [3]: [COL_8D7490B1_ED4B_4A07_9680_ECC66DF49E37#1156886, COL_7F6B8B23_E564_4B4E_8794_0FC7CE59D60B#1156887, COL_B6440F86_8BD4_4AFD_8640_45556688ABB9#1156888]

(19) Exchange
Input [3]: [COL_8D7490B1_ED4B_4A07_9680_ECC66DF49E37#1156886, COL_7F6B8B23_E564_4B4E_8794_0FC7CE59D60B#1156887, COL_B6440F86_8BD4_4AFD_8640_45556688ABB9#1156888]
Arguments: hashpartitioning(COL_8D7490B1_ED4B_4A07_9680_ECC66DF49E37#1156886, COL_7F6B8B23_E564_4B4E_8794_0FC7CE59D60B#1156887, COL_B6440F86_8BD4_4AFD_8640_45556688ABB9#1156888, 10), ENSURE_REQUIREMENTS, [plan_id=695311]

(20) ShuffleQueryStage
Output [3]: [COL_8D7490B1_ED4B_4A07_9680_ECC66DF49E37#1156886, COL_7F6B8B23_E564_4B4E_8794_0FC7CE59D60B#1156887, COL_B6440F86_8BD4_4AFD_8640_45556688ABB9#1156888]
Arguments: 1

(21) AQEShuffleRead
Input [3]: [COL_8D7490B1_ED4B_4A07_9680_ECC66DF49E37#1156886, COL_7F6B8B23_E564_4B4E_8794_0FC7CE59D60B#1156887, COL_B6440F86_8BD4_4AFD_8640_45556688ABB9#1156888]
Arguments: coalesced

(22) HashAggregate [codegen id : 3]
Input [3]: [COL_8D7490B1_ED4B_4A07_9680_ECC66DF49E37#1156886, COL_7F6B8B23_E564_4B4E_8794_0FC7CE59D60B#1156887, COL_B6440F86_8BD4_4AFD_8640_45556688ABB9#1156888]
Keys [3]: [COL_8D7490B1_ED4B_4A07_9680_ECC66DF49E37#1156886, COL_7F6B8B23_E564_4B4E_8794_0FC7CE59D60B#1156887, COL_B6440F86_8BD4_4AFD_8640_45556688ABB9#1156888]
Functions: []
Aggregate Attributes: []
Results [3]: [COL_8D7490B1_ED4B_4A07_9680_ECC66DF49E37#1156886 AS COL_DC1A3ABF_ADFC_4208_8B24_35ED54CC87DF#1156892, COL_7F6B8B23_E564_4B4E_8794_0FC7CE59D60B#1156887 AS COL_EA9AE57C_DE84_4BBE_9131_053814B337F4#1156893, COL_B6440F86_8BD4_4AFD_8640_45556688ABB9#1156888 AS COL_3744BFD9_1F3B_4691_AA97_1C8D69F364B8#1156894]

(23) Exchange
Input [3]: [COL_DC1A3ABF_ADFC_4208_8B24_35ED54CC87DF#1156892, COL_EA9AE57C_DE84_4BBE_9131_053814B337F4#1156893, COL_3744BFD9_1F3B_4691_AA97_1C8D69F364B8#1156894]
Arguments: rangepartitioning(COL_DC1A3ABF_ADFC_4208_8B24_35ED54CC87DF#1156892 ASC NULLS FIRST, COL_EA9AE57C_DE84_4BBE_9131_053814B337F4#1156893 ASC NULLS FIRST, 10), ENSURE_REQUIREMENTS, [plan_id=695455]

(24) ShuffleQueryStage
Output [3]: [COL_DC1A3ABF_ADFC_4208_8B24_35ED54CC87DF#1156892, COL_EA9AE57C_DE84_4BBE_9131_053814B337F4#1156893, COL_3744BFD9_1F3B_4691_AA97_1C8D69F364B8#1156894]
Arguments: 2

(25) AQEShuffleRead
Input [3]: [COL_DC1A3ABF_ADFC_4208_8B24_35ED54CC87DF#1156892, COL_EA9AE57C_DE84_4BBE_9131_053814B337F4#1156893, COL_3744BFD9_1F3B_4691_AA97_1C8D69F364B8#1156894]
Arguments: coalesced

(26) Sort [codegen id : 6]
Input [3]: [COL_DC1A3ABF_ADFC_4208_8B24_35ED54CC87DF#1156892, COL_EA9AE57C_DE84_4BBE_9131_053814B337F4#1156893, COL_3744BFD9_1F3B_4691_AA97_1C8D69F364B8#1156894]
Arguments: [COL_DC1A3ABF_ADFC_4208_8B24_35ED54CC87DF#1156892 ASC NULLS FIRST, COL_EA9AE57C_DE84_4BBE_9131_053814B337F4#1156893 ASC NULLS FIRST], true, 0

(27) Union

(28) Exchange
Input [3]: [COL_E5ADB260_1BF6_449C_8439_0A94CEE6956C#1157261, COL_59774853_4110_4672_B6E2_C0D5DE930914#1157262, COL_E58DD17C_E55F_4036_9FA9_A5365B045A1A#1157263]
Arguments: rangepartitioning(COL_E5ADB260_1BF6_449C_8439_0A94CEE6956C#1157261 ASC NULLS FIRST, COL_59774853_4110_4672_B6E2_C0D5DE930914#1157262 ASC NULLS FIRST, 10), ENSURE_REQUIREMENTS, [plan_id=695557]

(29) ShuffleQueryStage
Output [3]: [COL_E5ADB260_1BF6_449C_8439_0A94CEE6956C#1157261, COL_59774853_4110_4672_B6E2_C0D5DE930914#1157262, COL_E58DD17C_E55F_4036_9FA9_A5365B045A1A#1157263]
Arguments: 3

(30) AQEShuffleRead
Input [3]: [COL_E5ADB260_1BF6_449C_8439_0A94CEE6956C#1157261, COL_59774853_4110_4672_B6E2_C0D5DE930914#1157262, COL_E58DD17C_E55F_4036_9FA9_A5365B045A1A#1157263]
Arguments: coalesced

(31) Sort [codegen id : 7]
Input [3]: [COL_E5ADB260_1BF6_449C_8439_0A94CEE6956C#1157261, COL_59774853_4110_4672_B6E2_C0D5DE930914#1157262, COL_E58DD17C_E55F_4036_9FA9_A5365B045A1A#1157263]
Arguments: [COL_E5ADB260_1BF6_449C_8439_0A94CEE6956C#1157261 ASC NULLS FIRST, COL_59774853_4110_4672_B6E2_C0D5DE930914#1157262 ASC NULLS FIRST], true, 0

(32) WriteFiles
Input [3]: [COL_E5ADB260_1BF6_449C_8439_0A94CEE6956C#1157261, COL_59774853_4110_4672_B6E2_C0D5DE930914#1157262, COL_E58DD17C_E55F_4036_9FA9_A5365B045A1A#1157263]

(33) Execute InsertIntoHadoopFsRelationCommand
Input: []
Arguments: file:/data/output/cache/parquet/datablock/14_B_PASSAGE_PA_217771, false, Parquet, [path=file:///data/output/cache/parquet/datablock/14_B_PASSAGE_PA_217771], Overwrite, [COL_E5ADB260_1BF6_449C_8439_0A94CEE6956C, COL_59774853_4110_4672_B6E2_C0D5DE930914, COL_E58DD17C_E55F_4036_9FA9_A5365B045A1A]

(34) Filter
Input [4]: [Numéro Séjour#1157007, UF#1157009, Niveau CCMU#1157012, Transport admission#1157015]
Condition : NOT CASE WHEN isnull(StartsWith(Numéro Séjour#1157007, #)) THEN false ELSE StartsWith(Numéro Séjour#1157007, #) END

(35) Project
Output [4]: [Numéro Séjour#1157007 AS COL_2AF4FE05_74B1_4A24_9BC1_7612D6ED731D#1157125, UF#1157009 AS COL_E82CFB57_01EB_4E90_836D_4D6F4CA131FE#1157126, str_remove_space(Transport admission#1157015, ONLY_ONE) AS COL_E1010A22_775B_43B4_B8CA_4603C0801D28#1157127, str_remove_space(str_list_get(str_extract_by_regex(Niveau CCMU#1157012, [+\d]+), 0), ALL) AS COL_6A6D32F5_2B4E_4DF6_A459_B248882F909D#1157128]
Input [4]: [Numéro Séjour#1157007, UF#1157009, Niveau CCMU#1157012, Transport admission#1157015]

(36) Exchange
Input [4]: [COL_2AF4FE05_74B1_4A24_9BC1_7612D6ED731D#1157125, COL_E82CFB57_01EB_4E90_836D_4D6F4CA131FE#1157126, COL_E1010A22_775B_43B4_B8CA_4603C0801D28#1157127, COL_6A6D32F5_2B4E_4DF6_A459_B248882F909D#1157128]
Arguments: hashpartitioning(COL_2AF4FE05_74B1_4A24_9BC1_7612D6ED731D#1157125, 10), ENSURE_REQUIREMENTS, [plan_id=695221]

(37) Sort
Input [4]: [COL_2AF4FE05_74B1_4A24_9BC1_7612D6ED731D#1157125, COL_E82CFB57_01EB_4E90_836D_4D6F4CA131FE#1157126, COL_E1010A22_775B_43B4_B8CA_4603C0801D28#1157127, COL_6A6D32F5_2B4E_4DF6_A459_B248882F909D#1157128]
Arguments: [COL_2AF4FE05_74B1_4A24_9BC1_7612D6ED731D#1157125 ASC NULLS FIRST, 1 DESC NULLS LAST], false, 0

(38) Window
Input [4]: [COL_2AF4FE05_74B1_4A24_9BC1_7612D6ED731D#1157125, COL_E82CFB57_01EB_4E90_836D_4D6F4CA131FE#1157126, COL_E1010A22_775B_43B4_B8CA_4603C0801D28#1157127, COL_6A6D32F5_2B4E_4DF6_A459_B248882F909D#1157128]
Arguments: [row_number() windowspecdefinition(COL_2AF4FE05_74B1_4A24_9BC1_7612D6ED731D#1157125, 1 DESC NULLS LAST, specifiedwindowframe(RowFrame, unboundedpreceding$(), currentrow$())) AS COL_6B340AEB_6908_4A50_8AD6_FE561D33FBE0#1157141], [COL_2AF4FE05_74B1_4A24_9BC1_7612D6ED731D#1157125], [1 DESC NULLS LAST]

(39) Filter
Input [5]: [COL_2AF4FE05_74B1_4A24_9BC1_7612D6ED731D#1157125, COL_E82CFB57_01EB_4E90_836D_4D6F4CA131FE#1157126, COL_E1010A22_775B_43B4_B8CA_4603C0801D28#1157127, COL_6A6D32F5_2B4E_4DF6_A459_B248882F909D#1157128, COL_6B340AEB_6908_4A50_8AD6_FE561D33FBE0#1157141]
Condition : (COL_6B340AEB_6908_4A50_8AD6_FE561D33FBE0#1157141 <=> 1)

(40) Project
Output [4]: [COL_2AF4FE05_74B1_4A24_9BC1_7612D6ED731D#1157125 AS COL_DEB05435_3A84_4F8F_8CD2_6BE209A14FA8#1157221, COL_E82CFB57_01EB_4E90_836D_4D6F4CA131FE#1157126 AS COL_48FD2AAE_6F4D_4D43_BEE7_8A161203783C#1157222, COL_E1010A22_775B_43B4_B8CA_4603C0801D28#1157127 AS COL_E0699193_FC70_4F64_BAE6_B2F241857F3B#1157223, COL_6A6D32F5_2B4E_4DF6_A459_B248882F909D#1157128 AS COL_630C2F1A_DB89_4A08_A2D2_179A10CC9FD2#1157224]
Input [5]: [COL_2AF4FE05_74B1_4A24_9BC1_7612D6ED731D#1157125, COL_E82CFB57_01EB_4E90_836D_4D6F4CA131FE#1157126, COL_E1010A22_775B_43B4_B8CA_4603C0801D28#1157127, COL_6A6D32F5_2B4E_4DF6_A459_B248882F909D#1157128, COL_6B340AEB_6908_4A50_8AD6_FE561D33FBE0#1157141]

(41) Generate
Input [4]: [COL_DEB05435_3A84_4F8F_8CD2_6BE209A14FA8#1157221, COL_48FD2AAE_6F4D_4D43_BEE7_8A161203783C#1157222, COL_E0699193_FC70_4F64_BAE6_B2F241857F3B#1157223, COL_630C2F1A_DB89_4A08_A2D2_179A10CC9FD2#1157224]
Arguments: stack(3, 01-UF, COL_48FD2AAE_6F4D_4D43_BEE7_8A161203783C#1157222, 02-Transport, COL_E0699193_FC70_4F64_BAE6_B2F241857F3B#1157223, 03-CCMU, COL_630C2F1A_DB89_4A08_A2D2_179A10CC9FD2#1157224), [COL_DEB05435_3A84_4F8F_8CD2_6BE209A14FA8#1157221], false, [col0#1157230, col1#1157231]

(42) Project
Output [3]: [COL_DEB05435_3A84_4F8F_8CD2_6BE209A14FA8#1157221 AS COL_E5ADB260_1BF6_449C_8439_0A94CEE6956C#1157261, col0#1157230 AS COL_59774853_4110_4672_B6E2_C0D5DE930914#1157262, col1#1157231 AS COL_E58DD17C_E55F_4036_9FA9_A5365B045A1A#1157263]
Input [3]: [COL_DEB05435_3A84_4F8F_8CD2_6BE209A14FA8#1157221, col0#1157230, col1#1157231]

(43) Project
Output [8]: [COL_F76265AB_8748_4114_92B4_00E37DE25D34#1156522 AS COL_BC7CB40E_B2CA_42C4_94A5_B6AA996E14DC#1156840, COL_9262BBED_C393_416D_B38C_8F1458965BA7#1156521 AS COL_95E23023_63AA_4AB8_A60F_50D45234CFDD#1156841, COL_96E2282E_0771_460E_8BDB_D11261BD9020#1156523 AS COL_78B23D10_75E8_415E_A3C0_8AB5C826A837#1156842, COL_12028397_66B1_4E3C_BBDB_64537606E7DE#1156524 AS COL_885784F8_1772_40E3_89B6_6ACE1FAB6407#1156843, COL_0521CAB9_7617_4F31_948E_95DCA16387D7#1156519 AS COL_32AA235F_1BB1_4047_B8BA_AD2E9197AD1F#1156844, COL_1DACB81E_AFA5_4BB8_83D7_57DE8B8062B7#1156520 AS COL_3BBA8558_C3D5_4921_93AE_D7C9E273B39D#1156845, COL_4473F770_A575_4100_97F6_8190F900749A#1156528 AS COL_35788C71_38EE_4A08_B685_AAC68457246A#1156846, COL_715B3479_EAD2_422C_BF5C_7B020197D946#1156529 AS COL_6EC95BD9_A121_4728_BE4A_49B7EA2976F2#1156847]
Input [8]: [COL_0521CAB9_7617_4F31_948E_95DCA16387D7#1156519, COL_1DACB81E_AFA5_4BB8_83D7_57DE8B8062B7#1156520, COL_9262BBED_C393_416D_B38C_8F1458965BA7#1156521, COL_F76265AB_8748_4114_92B4_00E37DE25D34#1156522, COL_96E2282E_0771_460E_8BDB_D11261BD9020#1156523, COL_12028397_66B1_4E3C_BBDB_64537606E7DE#1156524, COL_4473F770_A575_4100_97F6_8190F900749A#1156528, COL_715B3479_EAD2_422C_BF5C_7B020197D946#1156529]

(44) Generate
Input [8]: [COL_BC7CB40E_B2CA_42C4_94A5_B6AA996E14DC#1156840, COL_95E23023_63AA_4AB8_A60F_50D45234CFDD#1156841, COL_78B23D10_75E8_415E_A3C0_8AB5C826A837#1156842, COL_885784F8_1772_40E3_89B6_6ACE1FAB6407#1156843, COL_32AA235F_1BB1_4047_B8BA_AD2E9197AD1F#1156844, COL_3BBA8558_C3D5_4921_93AE_D7C9E273B39D#1156845, COL_35788C71_38EE_4A08_B685_AAC68457246A#1156846, COL_6EC95BD9_A121_4728_BE4A_49B7EA2976F2#1156847]
Arguments: stack(7, 04-Date Naissance, COL_95E23023_63AA_4AB8_A60F_50D45234CFDD#1156841, 05-Date Entree, COL_78B23D10_75E8_415E_A3C0_8AB5C826A837#1156842, 06-Date Sortie, COL_885784F8_1772_40E3_89B6_6ACE1FAB6407#1156843, 07-Fides, COL_32AA235F_1BB1_4047_B8BA_AD2E9197AD1F#1156844, 08-Facturable AM, COL_3BBA8558_C3D5_4921_93AE_D7C9E273B39D#1156845, 09-Justif Exo, COL_35788C71_38EE_4A08_B685_AAC68457246A#1156846, 10-Nature Ass, COL_6EC95BD9_A121_4728_BE4A_49B7EA2976F2#1156847), [COL_BC7CB40E_B2CA_42C4_94A5_B6AA996E14DC#1156840], false, [col0#1156857, col1#1156858]

(45) Project
Output [3]: [COL_BC7CB40E_B2CA_42C4_94A5_B6AA996E14DC#1156840 AS COL_8D7490B1_ED4B_4A07_9680_ECC66DF49E37#1156886, col0#1156857 AS COL_7F6B8B23_E564_4B4E_8794_0FC7CE59D60B#1156887, col1#1156858 AS COL_B6440F86_8BD4_4AFD_8640_45556688ABB9#1156888]
Input [3]: [COL_BC7CB40E_B2CA_42C4_94A5_B6AA996E14DC#1156840, col0#1156857, col1#1156858]

(46) HashAggregate
Input [3]: [COL_8D7490B1_ED4B_4A07_9680_ECC66DF49E37#1156886, COL_7F6B8B23_E564_4B4E_8794_0FC7CE59D60B#1156887, COL_B6440F86_8BD4_4AFD_8640_45556688ABB9#1156888]
Keys [3]: [COL_8D7490B1_ED4B_4A07_9680_ECC66DF49E37#1156886, COL_7F6B8B23_E564_4B4E_8794_0FC7CE59D60B#1156887, COL_B6440F86_8BD4_4AFD_8640_45556688ABB9#1156888]
Functions: []
Aggregate Attributes: []
Results [3]: [COL_8D7490B1_ED4B_4A07_9680_ECC66DF49E37#1156886, COL_7F6B8B23_E564_4B4E_8794_0FC7CE59D60B#1156887, COL_B6440F86_8BD4_4AFD_8640_45556688ABB9#1156888]

(47) Exchange
Input [3]: [COL_8D7490B1_ED4B_4A07_9680_ECC66DF49E37#1156886, COL_7F6B8B23_E564_4B4E_8794_0FC7CE59D60B#1156887, COL_B6440F86_8BD4_4AFD_8640_45556688ABB9#1156888]
Arguments: hashpartitioning(COL_8D7490B1_ED4B_4A07_9680_ECC66DF49E37#1156886, COL_7F6B8B23_E564_4B4E_8794_0FC7CE59D60B#1156887, COL_B6440F86_8BD4_4AFD_8640_45556688ABB9#1156888, 10), ENSURE_REQUIREMENTS, [plan_id=695228]

(48) HashAggregate
Input [3]: [COL_8D7490B1_ED4B_4A07_9680_ECC66DF49E37#1156886, COL_7F6B8B23_E564_4B4E_8794_0FC7CE59D60B#1156887, COL_B6440F86_8BD4_4AFD_8640_45556688ABB9#1156888]
Keys [3]: [COL_8D7490B1_ED4B_4A07_9680_ECC66DF49E37#1156886, COL_7F6B8B23_E564_4B4E_8794_0FC7CE59D60B#1156887, COL_B6440F86_8BD4_4AFD_8640_45556688ABB9#1156888]
Functions: []
Aggregate Attributes: []
Results [3]: [COL_8D7490B1_ED4B_4A07_9680_ECC66DF49E37#1156886 AS COL_DC1A3ABF_ADFC_4208_8B24_35ED54CC87DF#1156892, COL_7F6B8B23_E564_4B4E_8794_0FC7CE59D60B#1156887 AS COL_EA9AE57C_DE84_4BBE_9131_053814B337F4#1156893, COL_B6440F86_8BD4_4AFD_8640_45556688ABB9#1156888 AS COL_3744BFD9_1F3B_4691_AA97_1C8D69F364B8#1156894]

(49) Exchange
Input [3]: [COL_DC1A3ABF_ADFC_4208_8B24_35ED54CC87DF#1156892, COL_EA9AE57C_DE84_4BBE_9131_053814B337F4#1156893, COL_3744BFD9_1F3B_4691_AA97_1C8D69F364B8#1156894]
Arguments: rangepartitioning(COL_DC1A3ABF_ADFC_4208_8B24_35ED54CC87DF#1156892 ASC NULLS FIRST, COL_EA9AE57C_DE84_4BBE_9131_053814B337F4#1156893 ASC NULLS FIRST, 10), ENSURE_REQUIREMENTS, [plan_id=695231]

(50) Sort
Input [3]: [COL_DC1A3ABF_ADFC_4208_8B24_35ED54CC87DF#1156892, COL_EA9AE57C_DE84_4BBE_9131_053814B337F4#1156893, COL_3744BFD9_1F3B_4691_AA97_1C8D69F364B8#1156894]
Arguments: [COL_DC1A3ABF_ADFC_4208_8B24_35ED54CC87DF#1156892 ASC NULLS FIRST, COL_EA9AE57C_DE84_4BBE_9131_053814B337F4#1156893 ASC NULLS FIRST], true, 0

(51) Union

(52) Exchange
Input [3]: [COL_E5ADB260_1BF6_449C_8439_0A94CEE6956C#1157261, COL_59774853_4110_4672_B6E2_C0D5DE930914#1157262, COL_E58DD17C_E55F_4036_9FA9_A5365B045A1A#1157263]
Arguments: rangepartitioning(COL_E5ADB260_1BF6_449C_8439_0A94CEE6956C#1157261 ASC NULLS FIRST, COL_59774853_4110_4672_B6E2_C0D5DE930914#1157262 ASC NULLS FIRST, 10), ENSURE_REQUIREMENTS, [plan_id=695235]

(53) Sort
Input [3]: [COL_E5ADB260_1BF6_449C_8439_0A94CEE6956C#1157261, COL_59774853_4110_4672_B6E2_C0D5DE930914#1157262, COL_E58DD17C_E55F_4036_9FA9_A5365B045A1A#1157263]
Arguments: [COL_E5ADB260_1BF6_449C_8439_0A94CEE6956C#1157261 ASC NULLS FIRST, COL_59774853_4110_4672_B6E2_C0D5DE930914#1157262 ASC NULLS FIRST], true, 0

(54) WriteFiles
Input [3]: [COL_E5ADB260_1BF6_449C_8439_0A94CEE6956C#1157261, COL_59774853_4110_4672_B6E2_C0D5DE930914#1157262, COL_E58DD17C_E55F_4036_9FA9_A5365B045A1A#1157263]

(55) Execute InsertIntoHadoopFsRelationCommand
Input: []
Arguments: file:/data/output/cache/parquet/datablock/14_B_PASSAGE_PA_217771, false, Parquet, [path=file:///data/output/cache/parquet/datablock/14_B_PASSAGE_PA_217771], Overwrite, [COL_E5ADB260_1BF6_449C_8439_0A94CEE6956C, COL_59774853_4110_4672_B6E2_C0D5DE930914, COL_E58DD17C_E55F_4036_9FA9_A5365B045A1A]

(56) AdaptiveSparkPlan
Output: []
Arguments: isFinalPlan=true

SQL / DataFrame Properties

Name	Value
spark.sql.optimizer.nestedPredicatePushdown.supportedFileSources	parquet,orc,geoparquet