#### Review this Rscript provided for the following question. If there is any inconsistency or incomplete part, fix it$.($DONT USE AI$)$

set.seed$(1)$

generate$\_$reward$\_$matrix $<-$ function$($n$\_$cities$)\{$

v $<-$ matrix$(0,$ nrow $=$ n$\_$cities, ncol $=$ n$\_$cities$)$

upper $<-$ runif$($n$\_$cities $*($n$\_$cities $-1)/2)$

v$[$upper$.$tri$($v$)]<-$ upper

v$[$lower$.$tri$($v$)]<-$ t$($v$)[$lower$.$tri$($v$)]$

diag$($v$)<-0$

return$($v$)$

$\}$

total$\_$reward $<-$ function$($x$,$ v$)\{$

sum$($v$[$cbind$($c$(0,$ x$[-$length$($x$)]),$ x$)])$

$\}$

n$\_$iterations $<-10000$

temperature $<-1$

n$\_$cities $<-10$

v $<-$ generate$\_$reward$\_$matrix$($n$\_$cities$)$

x $<-$ sample$(1$:n$\_$cities, n$\_$cities$)$

swap$\_$cities $<-$ function$($x$)\{$

i $<-$ sample$(1$:length$($x$),2)$

x$\_$new $<-$ x

x$\_$new$[$i$]<-$ x$[$i$[2]]$

x$\_$new$[$i$[2]]<-$ x$[$i$[1]]$

return$($x$\_$new$)$

$\}$

itinerary $<-$ matrix$($nrow $=$ n$\_$iterations, ncol $=$ length$($x$))$

itinerary$[1,]<-$ x

parameters $<-$ c$(0\mathrm{.}5,1,10)$

par$($mfrow$=$c$(3,1),$ mar$=$c$(2,2,1,1))$

for $($param in parameters$)\{$

temperature $<-$ param

current$\_$reward $<-$ total$\_$reward$($x$,$ v$)$

for $($i in $2$:n$\_$iterations$)\{$

x$\_$new $<-$ swap$\_$cities$($itinerary$[$i $-1,])$

new$\_$reward $<-$ total$\_$reward$($x$\_$new, v$)$

if $($exp$(($new$\_$reward $-$ current$\_$reward$)/$ temperature$)>=$ runif$(1))\{$

itinerary$[$i$,]<-$ x$\_$new

current$\_$reward $<-$ new$\_$reward

$\}$ else $\{$

itinerary$[$i$,]<-$ itinerary$[$i $-1,]$

$\}$

$\}$

plot$(1$:n$\_$iterations, apply$($itinerary$,1,$ function$($x$)$ total$\_$reward$($x$,$ v$)),$ type $=\text{'}$l$\text{'},$

xlab $=$ 'Iteration', ylab $=$ 'Total Reward', main $=$ paste$(\text{'}$Parameter:$\text{'},$ param$))$

avg$\_$reward $<-$ mean$($apply$($itinerary$,1,$ function$($x$)$ total$\_$reward$($x$,$ v$)))$

cat$("$Average reward for parameter", param, $"$:$",$ avg$\_$reward, $"$

$")$

$\}$

temperature $<-$ initial$\_$temperature

for $($i in $2$:n$\_$iterations$)\{$

x$\_$new $<-$ swap$\_$cities$($itinerary$[$i $-1,])$

new$\_$reward $<-$ total$\_$reward$($x$\_$new, v$)$

current$\_$reward $<-$ total$\_$reward$($itinerary$[$i $-1,],$ v$)$

acceptance$\_$prob $<-$ exp$(($new$\_$reward $-$ current$\_$reward$)/$ temperature$)$

if $($runif$(1)<$ acceptance$\_$prob$)\{$

itinerary$[$i$,]<-$ x$\_$new

$\}$ else $\{$

itinerary$[$i$,]<-$ itinerary$[$i $-1,]$

$\}$

temperature $<-$ temperature $*0\mathrm{.}99$ # Adjust the cooling rate as needed

$\}$

plot$(1$:n$\_$iterations, apply$($itinerary$,1,$ function$($x$)$ total$\_$reward$($x$,$ v$)),$

type $=\text{'}$l$\text{'},$ xlab $=$ 'Iteration', ylab $=$ 'Total Reward',

main $=$ 'Simulated Annealing with Dynamic Temperature'$)$

avg$\_$reward $<-$ mean$($apply$($itinerary$,1,$ function$($x$)$ total$\_$reward$($x$,$ v$)))$

cat$("$Average reward with simulated annealing:", avg$\_$reward, $"$

$")$

Foodys CEO needs to visit cities $1,2,...,$ r with city $0$ being the city she is currently located at$.$ Suppose a non$-$negative reward v$($i$,$ j$)$ is associated with the CEO going from city i to city j$.$ So if the CEO visits the cities in permutation x$1,...,$ xr then the reward of this choice x $=($x$1,...,$ xr$)$ is:

V $($x$)=($sum i$=1$ to r$)$ v$($xi$1,$ xi$),$

where x$0=0.$ Note that there is no reward for coming back to city $0.$ To generate v$($i$,$ j$),$ set seed to $1$ and generate v$($i$,$ j$)$ using Uniform$(0,1)$ random variables starting from i $=0$ and j $=1,...,10,$then v$(1,$ j$)$ for j $=2,...,10,$ v$(2,$ j$)$ for j $=1,3,4,...,10$ and so on$.$

$($i$)$ Use MCMC and r $=10$ to simulate high reward itineraries for the CEO. Define the stationary distribution that we want our Markov Chain to converge in such a way that high valued solutions are given extremely high probability and include a tunable parameter. Find $3$ values of this parameter where the generated Markov Chain behaves differently in the long run for each of these $3$ values and demonstrate this using plots and averages.

$($ii$)$ If you could change this parameter during one simulation of the stationary distribution, how would you change it in order to improve the solutions. Demonstrate that you can get better solutions than the fixed $3$ values that you used in $($i$)$ by running it in R$.$ Present all your results.