Consider the parallelogram whose boundaries are these $4$ lines:

y$=0,$ y$=3,$ x$=$y and $($x$-3)=$y

A multi$-$layer neural network architecture to fire only for points outside the

parallelogram is:

$1.4$ hidden units fire for y $<=0,$ y $>=3,$ x $<=$ y$,$ and $($x$-3)>=$ y$,$ respectively, then

feed their outputs into an AND gate.

$2.4$ hidden units fire for y $>=0,$ y $<=3,$ x $>=$ y$,$ and $($x$-3)<=$ y$,$ respectively, then

feed their outputs into an AND gate.

$3.4$ hidden units fire for y $<=0,$ y $>=3,$ x $<=$ y$,$ and $($x$-3)>=$ y$,$ respectively, then

feed their outputs into an AND gate, then feed the output of that into a NOT

gate.

$4.4$ hidden units fire for y $>=0,$ y $<=3,$ x $>=$ y$,$ and $($x$-3)<=$ y$,$ respectively, then

feed their outputs into an OR gate, then feed the output of that into a NOT

gate.

$5.4$ hidden units fire for y $>=0,$ y $<=3,$ x $>=$ y$,$ and $($x$-3)<=$ y$,$ respectively, then

feed their outputs into an OR gate.

$6.4$ hidden units fire for y $>=0,$ y $<=3,$ x $>=$ y$,$ and $($x$-3)<=$ y$,$ respectively, then

feed their outputs into an AND gate, then feed the output of that into a NOT

gate.