SP1 - Details for Query 4128

Submitted Time: 2026/10/19 08:58:41
Duration: 0.1 s
Succeeded Jobs: 6406

Show the Stage ID and Task ID that corresponds to the max metric

digraph G { 0 [labelType="html" label=" CollectLimit "]; subgraph cluster1 { isCluster="true"; label="WholeStageCodegen (1)\n \nduration: 15 ms"; 2 [labelType="html" label=" Project "]; 3 [labelType="html" label="Filter number of output rows: 1"]; 4 [labelType="html" label="ColumnarToRow number of output rows: 209 number of input batches: 1"]; } 5 [labelType="html" label="Scan parquet number of files read: 1 scan time: 2 ms metadata time: 0 ms size of files read: 10.7 KiB number of output rows: 209"]; 2->0; 3->2; 4->3; 5->4; }

CollectLimit 1500

Project [CODE#948902 AS COL_CODE_174097#949064, LIBELLE#948903 AS COL_LIBELLE_174098#949065, Code+REGROUPEMENT#948904 AS COL_CODE_REGROUPEME_174099#949066, str_to_datez(Date_effet#948905, M/d/yy, null, null) AS COL_DATE_EFFET_174100#949067, str_to_datez(Date_r%C3%A9vocation#948906, M/d/yy, null, null) AS COL_DATE_REVOCATION_174101#949068, Responsable+MAJ#948907 AS COL_RESPONSABLE_MAJ_174102#949069, Source#948908 AS COL_SOURCE_174103#949070]

Filter ((((CASE WHEN isnull(Contains(str_remove_accent(upper(CODE#948902)), ACQN001)) THEN false ELSE Contains(str_remove_accent(upper(CODE#948902)), ACQN001) END OR CASE WHEN isnull(Contains(str_remove_accent(upper(LIBELLE#948903)), ACQN001)) THEN false ELSE Contains(str_remove_accent(upper(LIBELLE#948903)), ACQN001) END) OR CASE WHEN isnull(Contains(str_remove_accent(upper(Code+REGROUPEMENT#948904)), ACQN001)) THEN false ELSE Contains(str_remove_accent(upper(Code+REGROUPEMENT#948904)), ACQN001) END) OR CASE WHEN isnull(Contains(str_remove_accent(upper(Responsable+MAJ#948907)), ACQN001)) THEN false ELSE Contains(str_remove_accent(upper(Responsable+MAJ#948907)), ACQN001) END) OR CASE WHEN isnull(Contains(str_remove_accent(upper(Source#948908)), ACQN001)) THEN false ELSE Contains(str_remove_accent(upper(Source#948908)), ACQN001) END)

ColumnarToRow

WholeStageCodegen (1)

FileScan parquet [CODE#948902,LIBELLE#948903,Code+REGROUPEMENT#948904,Date_effet#948905,Date_r%C3%A9vocation#948906,Responsable+MAJ#948907,Source#948908] Batched: true, DataFilters: [((((CASE WHEN isnull(Contains(str_remove_accent(upper(CODE#948902)), ACQN001)) THEN false ELSE C..., Format: Parquet, Location: InMemoryFileIndex(1 paths)[file:/data/input/depot/excel/cache/SIC_SERVEUR_174043_011ecc05c30b12bd..., PartitionFilters: [], PushedFilters: [], ReadSchema: struct<CODE:string,LIBELLE:string,Code+REGROUPEMENT:string,Date_effet:string,Date_r%C3%A9vocation...

Details

== Physical Plan ==
CollectLimit (5)
+- * Project (4)
   +- * Filter (3)
      +- * ColumnarToRow (2)
         +- Scan parquet  (1)


(1) Scan parquet 
Output [7]: [CODE#948902, LIBELLE#948903, Code+REGROUPEMENT#948904, Date_effet#948905, Date_r%C3%A9vocation#948906, Responsable+MAJ#948907, Source#948908]
Batched: true
Location: InMemoryFileIndex [file:/data/input/depot/excel/cache/SIC_SERVEUR_174043_011ecc05c30b12bd525f9ae06b0f0abc/SIC.xlsx_d7e53226963a5de42ec3c8bf43bde627]
ReadSchema: struct<CODE:string,LIBELLE:string,Code+REGROUPEMENT:string,Date_effet:string,Date_r%C3%A9vocation:string,Responsable+MAJ:string,Source:string>

(2) ColumnarToRow [codegen id : 1]
Input [7]: [CODE#948902, LIBELLE#948903, Code+REGROUPEMENT#948904, Date_effet#948905, Date_r%C3%A9vocation#948906, Responsable+MAJ#948907, Source#948908]

(3) Filter [codegen id : 1]
Input [7]: [CODE#948902, LIBELLE#948903, Code+REGROUPEMENT#948904, Date_effet#948905, Date_r%C3%A9vocation#948906, Responsable+MAJ#948907, Source#948908]
Condition : ((((CASE WHEN isnull(Contains(str_remove_accent(upper(CODE#948902)), ACQN001)) THEN false ELSE Contains(str_remove_accent(upper(CODE#948902)), ACQN001) END OR CASE WHEN isnull(Contains(str_remove_accent(upper(LIBELLE#948903)), ACQN001)) THEN false ELSE Contains(str_remove_accent(upper(LIBELLE#948903)), ACQN001) END) OR CASE WHEN isnull(Contains(str_remove_accent(upper(Code+REGROUPEMENT#948904)), ACQN001)) THEN false ELSE Contains(str_remove_accent(upper(Code+REGROUPEMENT#948904)), ACQN001) END) OR CASE WHEN isnull(Contains(str_remove_accent(upper(Responsable+MAJ#948907)), ACQN001)) THEN false ELSE Contains(str_remove_accent(upper(Responsable+MAJ#948907)), ACQN001) END) OR CASE WHEN isnull(Contains(str_remove_accent(upper(Source#948908)), ACQN001)) THEN false ELSE Contains(str_remove_accent(upper(Source#948908)), ACQN001) END)

(4) Project [codegen id : 1]
Output [7]: [CODE#948902 AS COL_CODE_174097#949064, LIBELLE#948903 AS COL_LIBELLE_174098#949065, Code+REGROUPEMENT#948904 AS COL_CODE_REGROUPEME_174099#949066, str_to_datez(Date_effet#948905, M/d/yy, null, null) AS COL_DATE_EFFET_174100#949067, str_to_datez(Date_r%C3%A9vocation#948906, M/d/yy, null, null) AS COL_DATE_REVOCATION_174101#949068, Responsable+MAJ#948907 AS COL_RESPONSABLE_MAJ_174102#949069, Source#948908 AS COL_SOURCE_174103#949070]
Input [7]: [CODE#948902, LIBELLE#948903, Code+REGROUPEMENT#948904, Date_effet#948905, Date_r%C3%A9vocation#948906, Responsable+MAJ#948907, Source#948908]

(5) CollectLimit
Input [7]: [COL_CODE_174097#949064, COL_LIBELLE_174098#949065, COL_CODE_REGROUPEME_174099#949066, COL_DATE_EFFET_174100#949067, COL_DATE_REVOCATION_174101#949068, COL_RESPONSABLE_MAJ_174102#949069, COL_SOURCE_174103#949070]
Arguments: 1500

SQL / DataFrame Properties

Name	Value
spark.sql.optimizer.nestedPredicatePushdown.supportedFileSources	parquet,orc,geoparquet