The complete graph of $y=f(x),$ which consists of five line segments, is shown in red below. (On this graph, the distance between grid lines is $1.$) Let $a$ and $b$ be the largest negative integer and the smallest positive integer, respectively, such that the functions $g(x)=f(x)+ax$ and $h(x)=f(x)+bx$ are invertible. What is $a^2+b^2?$
