If $-5\leq a \leq -1$ and $1 \leq b \leq 3$, what is the least possible value of $\displaystyle\left(\frac{1}{a}+\frac{1}{b}\right)\left(\frac{1}{b}-\frac{1}{a}\right) $? Express your answer as a common fraction.
For the smallest value you want the absolute value of b to be as big as possible, and the absolute value of a to be as small as possible but neither can be 0