We define multi head self attention l$$ke below:

$Y(x)=$ concat $[H1,$dots,$HH]w(0)$

$Hh=$Softmax$[QhKTVhDkh]Vh$

$Qh=xW(q)h,Kh=xW(K)h,Vh=xw(v)h$

It includes some redundancy in consecutive multiplications of matrix $w(v)$ corresponding to every head and also output matrix $w(0).$ Removing this redundancy enables us to write multi head self attention as sum of the effect of every head. Now prove we can write multihead self attention formula as below:

$Y(x)=Hh=1$ Softmax $[QhKT\sqrt[\phantom{2}]{hDkh}]w(h)$

$($Hint: $w(h)$ equals to $w(v)$ Wh$($o$)$ if we devide matrix $w(0)$ in horizontal direction as the number of heads then wh$($o$)$ is for the hth head$)$