Begin by expanding the expression \(\displaystyle(1-x)(1-y)(1-z).\)
\(\displaystyle (1-x)(1-y)(1-z) = 1-x-y-z+xy+yz+zx-xyz,\)
so that
\(\displaystyle xy+yz+zx-xyz=x+y+z-1+(1-x)(1-y)(1-z), \)
and applying the constraint,
\(xy+yz+zx-xyz=2+(1-x)(1-y)(1-z)\)
and the problem becomes one of finding the maximum value of \(\displaystyle (1-x)(1-y)(1-z),\) subject to the constraint.
Note that \(\displaystyle x=y=z=1\) makes this expression, call it \(\displaystyle S,\) equal to zero.
Moving away from these values, all three cannot be greater than or less than 1 since this violates the constraint, making one of them greater than 1 and the other two less than 1 causes \(\displaystyle S\) to be negative, while making one of them less than 1 and the other two greater than 1 makes \(\displaystyle S\) positive, which is what we want.
Suppose then wlog that \(x<1, y>1 \text{ and }z>1.\)
Substituting for x from the constraint,
\(\displaystyle S = (y+z-2)(1-y)(1-z)=(y+z-2)(y-1)(z-1)\).
For \(S\) to be as big as possible, we would like y and z to be as big as possible and this will be the case when \(y+z=3\), (from the constraint, when \(x=0\) ).
Substituting \(\displaystyle y=3-z\) into the expression for \(\displaystyle S\) produces
\(\displaystyle S = (1)(z-2)(1-z)=-(z^{2}-3z+2)=-\left[(z-3/2)^{2}-1/4\right]\),
from which it follows that \(\displaystyle S\) has a maximum value of 1/4 occurring when z = 3/2 (and y = 3/2).
The final variable could have been chosen as x or y rather than z, so
\(\displaystyle xy+yz+zx-xyz\leq2+1/4\)
occurring when anyone of x, y or z is equal to zero and the other two equal to 3/2.
