Question: Please help me with my code, is it because of the TimestamRTVRe of data? The QUERY WHERE and ORDER BY does not work. The WHERE

Please help me with my code, is it because of the TimestamRTVRe of data? The QUERY
WHERE and ORDER BY does not work. The WHERE first part without the ORDER BY works.
Consider a warc.csv file related data. An indicative line is:
219.155.1.48, http:o000, Apaohe, 'chtml>'
Columns in order: first the WarS date, the warS record id, the WerS type (e.g. metadata,
response, etc), the content length, the public IP address, the target URL, the server running
the site (eg apache, DginX, etc), and finally the overall content of the page with the entire
HTML DOM. For the time range between 2017-03-2222:00 and 2017-03-2223:00, find the 5
most used servers. Results to be given in descending order of servers.
from datetime import datetime
from Ryspark, sol import SoarkSession
from pysparksaltypes import StructType, StructField, StringTvo, IntegerType,
FlgatTxpe TimestamoTyRE.
# Initialize Spark Session
spark = SparkSession.byilderaRRName("Warcnalysis")-getQrCreate()
# Define the schema
schema = StructTXRel [
StructField["date", TimestamaTVRed), True),
StructField["recordid", StringTyRed, True),
StructField("type", StringType(), True),
StructField ["contentalength", IntegerType(), True),
StructField("RublicuiR", StringTvRe(), True),
StructField("target,url", StriggTvag0), True),
StructField ("server", StriogTvRe(), True),
StructField ("btmldom", StriogTvae(), True)
# Load the data into RataFrame
df= sparkread format('csv')}
options(header='false')\
schema(schema)\
load("warc.csv")
# Register the Bata Frame as a temporary table
df.reateOrReglaceTempView("warc")
##Filter the data using Spark SQL
id quex = "SELECT yarcseryer
FROM wars \
WHERE Yarc.date =?12017-03-22T22:00:00Z'' AND warcodate '2017-03-
22T22:00:00Z
ORDER BY warcodate ASC ?n
filteced,df = spack.sgl(id,guery)
# Show the result
filteced df.shou(5)
# Stop Spark Session
sparkst8R()

Step by Step Solution

There are 3 Steps involved in it

1 Expert Approved Answer
Step: 1 Unlock blur-text-image
Question Has Been Solved by an Expert!

Get step-by-step solutions from verified subject matter experts

Step: 2 Unlock
Step: 3 Unlock

Students Have Also Explored These Related Databases Questions!