Please help me with my code, is it because of the TimestamTyoe of data? The QUERY

WHERE and ORDER BY does not work. The WHERE first part without the ORDER BY works.

Consider a warc.csv file related data. An indicative line is:

$2017-03-22$T$22$:$13$:$57$Z$,$ record id$,$ reposnse, $40802,$ ip addres, URL, Server, HTML DOC

Columns in order: first the Warc date, the warc record id$,$ the Warc type $($e$.$g$.$ metadata,

response, etc$),$ the content length, the public IP address, the target URL, the server running

the site $($eg apache, DginX, etc$),$ and finally the overall content of the page with the entire

HTML DOM. For the time range between $2017-03-2222$:$00$ and $2017-03-2223$:$00,$ find the $5$

most used servers. Results to be given in descending order of servers.

My code:

from datetime import datetime

from pyspark, sql import SparkSession

from pyspark.sql$.$types import StructType, StructField, StringType, IntegerType,

FloaType $,$TimestampType.

# Initialize Spark Session

spark $=$ SparkSession.builder.appName$("$WarcAnalysis$").$getOrCreate$()$

# Define the schema

schema $=$ StructType$[$

StructField$["$date$",$ TimestamaType$),$ True$),$

StructField$["$recordid$",$ StringType, True$),$

StructField$("$type$",$ StringType$(),$ True$),$

StructField $["$content$\_$length", IntegerType$(),$ True$),$

StructField$("$public $\_$p$",$ StringType$(),$ True$),$

StructField$("$target$,\_$rl$",$ StringType, True$),$

StructField $("$server$",$ StrionfType$(),$ True$),$

StructField $("$btmldom$",$ StringType$(),$ True$)$

# Load the data into RataFrame

$df=$ spark.read.format$(\text{'}$csv$\text{'})\}$

options$($header$=$'false'$)\backslash$

schema$($schema$)\backslash$

load$("$warc$.$csv$")$

# Register the Bata Frame as a temporary table

df$.$ createOrReplaceTempView$("$warc$")$

##Filter the data using Spark SQL

id quex $=$ "SELECT warc

FROM wars $\backslash$

WHERE warc.date $=\text{'}2017-03-22T22$:$00$:$00Z\text{'}$ AND warc.date $\text{'}2017-03-$

$22$T$22$:$00$:$00$Z$\text{'}$

ORDER BY warc.date ASC $"$

filtered.df $=$ spack.sql$($id$\_$query$)$

# Show the result

filtered df$.$show$(5)$

# Stop Spark Session

spark.stop$()$