Data Processing using Python and Pandas

The data is presented in csv format and has the following format:

Attribute$-$ Description

Date $-$ Date in format dd$/$mm$/$yyyy

Time $-$ Time in format hh:mm:ss

Global$\_$active$\_$power $-$ Household global minute$-$averaged active power $($in kilowatt$)$

Global$\_$reactive$\_$power $-$ Household global minute$-$averaged reactive power $($in kilowatt$)$

Voltage$-$ Minute$-$averaged voltage $($in volt$)$

Global$\_$intensity$-$ Household global minute$-$averaged current intensity $($in ampere$)$

Sub$\_$metering$\_1-$ Energy sub$-$metering No$.1($in watt$-$hour of active energy$).$ It corresponds to the kitchen, containing mainly a dishwasher, an oven and a microwave $($hot plates are not electric but gas powered$).$

Sub$\_$metering$\_2-$ Energy sub$-$metering No$.2($in watt$-$hour of active energy$).$ It corresponds to the laundry room, containing a washing$-$machine, a tumble$-$drier, a refrigerator and a light.

Sub$\_$metering$\_3-$ Energy sub$-$metering No$.3($in watt$-$hour of active energy$).$ It corresponds to an electric water$-$heater and an air$-$conditioner.

These tasks are designed to give you an$$opportunity$$to demonstrate the following learning outcomes and to satisfy the assessment criteria:

$$

Clean$$irregularities in the raw data file to convert it into a proper CVS format.

Read$$data from the cleaned CVS file into a Pandas Data frame.

Convert$$between different Date$/$time formats.

Filter$/$restrict the rows and columns in Pandas Data frames to help answer the queries.

Use$$aggregation$$operations $($such as mean, median, sum, max$)$ and to summarize data.

Use$$group by$$to summarize data for various categories.

Create new$$columns$$that are computed based on other existing columns.

Demonstrate appropriate use of a variety of types of$$Plots$$to visualize data $($using Pandas$).$

All plots should have$$meaningful titles, axes labels and user$-$friendly data labels and be scaled large enough to easy see the details required.

Markdown$$headings should be added to clearly separate and explain each of the tasks and markdown should be provided to discuss$/$summarize the key observations.

Don$$t repeat yourself $$ use$$functions$$to avoid duplicating the same logic in multiple places.

Use programming best practice $$ write clear$$simple$$Python code and use$$well$-$chosen$$identified names for all variables and functions.$$

$$

Note that the raw CSV data may require "cleaning" before it can be processed.$$Everything should be included in a single Jupyter notebook $($which you will need to create yourself $$ no skeleton solution for this assignment$).$

$$

Tasks:

Use markdown to document the data cleaning that you performed. Hint: All calendar timestamps are present in the dataset but for some timestamps, the measurement values are missing: a missing value is represented by the absence of value between two consecutive semi$-$colon attribute separators.

$1)$ Read the cleaned CSV file into a Pandas data frame

$2)$ Determine the maximum household global minute$-$averaged active power in kilowatt$$

$3)$ Determine the average household global minute$-$averaged current intensity in ampere

$4)$ Add a column that shows the accumulated reactive power in megawatts

$5)$ Show the global active power and energy sub$-$metering $3$ for the $14$th of February $2008$ in one plot

$6)$ Plot the sub$-$metering $1$ value for every Tuesday in October $2009$ in a graph

$7)$ Add a column that indicates the percentage of sub$-$metering $2$ of the global household active energy

$8)$ Plot a cumulative curve of active power used in the year $2009$

$9)$ Plot the average voltage used in a week during the month of May compared to October

$10)$ Markdown$$headings should be added to clearly separate and explain each of the tasks and markdown should be provided to discuss$/$summarize the key observations.

$11)$ Don$$t repeat yourself $$ use$$functions$$to avoid duplicating the same logic in multiple places.

$12)$ Use programming best practice $$ write clear$$simple$$Python code and use$$well$-$chosen$$identified names for all variables and functions.