Problem Statement: The objective of the problem is to implement an Actor$-$Critic

reinforcement learning algorithm to optimize energy consumption in a building. The agent should learn to adjust the temperature settings dynamically to minimize energy

usage while maintaining comfortable indoor condition.

This dataset contains energy consumption data for a residential building, along

with various environmental and operational factors.

Data Dictionary:

o Appliances: Energy use in Wh

o lights: Energy use of light fixtures in the house in Wh

o T$1-$ T$9$: Temperatures in various rooms and outside

o RH$\_1-$ RH$\_9$: Humidity measurements in various rooms and outside

o Visibility: Visibility in km

o Tdewpoint: Dew point temperature

o Press$\_$mm$\_$hg: Pressure in mm Hg

o Windspeed: Wind speed in m$/$s

State Space:

The state space consists of various features from the dataset that impact energy

consumption and comfort levels.

$$ Current Temperature $($T$1$ to T$9)$: Temperatures in various rooms and

outside.

$$ Current Humidity $($RH$\_1$ to RH$\_9)$: Humidity measurements in different

locations.

$$ Visibility $($Visibility$)$: Visibility in km$.$

$$ Dew Point $($Tdewpoint$)$: Dew point temperature.

$$ Pressure $($Press$\_$mm$\_$hg$)$: Atmospheric pressure in mm Hg$.$

$$ Windspeed $($Windspeed$)$: Wind speed in m$/$s$.$

Total State Vector Dimension: Number of features $=9($temperature$)+9($humidity$)+1$

$($visibility$)+1($dew point$)+1($pressure$)+1($windspeed$)=21$ features

Target Variable: Appliances $($energy consumption in Wh$).$

Action Space:

The action space consists of discrete temperature adjustments:

$$ Action $0$: Decrease temperature by $1\backslash$deg C

$$ Action $1$: Maintain current temperature

$$ Action $2$: Increase temperature by $1\backslash$deg C

Adjustments are clamped within the defined temperature limits $(-10\backslash$deg C to $30\backslash$deg C$).$

If the action is to decrease the temperature by $1\backslash$deg C$,$ you'll adjust each temperature

feature $($T$1$ to T$9)$ down by $1\backslash$deg C$.$ If the action is to increase the temperature by $1\backslash$deg C$,$ you'll

adjust each temperature feature $($T$1$ to T$9)$ up by $1\backslash$deg C$.$ Other features remain

unchanged.

The action space is limited to discrete temperature adjustments $(\backslash$pm $1\backslash$deg C$)$ within a defined range

$(-10\backslash$deg C to $30\backslash$deg C$).$

Policy $($Actor$)$: A neural network that outputs a probability distribution over possible

temperature adjustment.

Value function $($Critic$)$: A neural network that estimates the expected cumulative

reward $($energy savings$)$ from a given state.

Reward function:

The reward function should reflect the overall comfort and energy efficiency based

on all temperature readings. i$.$e$.,$ balance between minimising temperature

deviations and minimizing energy consumption.

$$ Calculate the penalty based on the deviation of each temperature from the

target temperature and then aggregate these penalties.

$$ Measure the change in energy consumption before and after applying the

RL action.

$$ Combine the comfort penalty and energy savings to get the final reward.

The RL framework integrates these adjustments by modifying the temperature

features in the state vector, computing rewards based on energy savings and comfort

penalties, and training the Actor$-$Critic model to find an optimal policy.