Let $f(x)$ be a polynomial with integer coefficients. There exist distinct integers $p,$ $q,$ $r,$ $s,$ $t$ such that \[f(p) = f(q) = f(r) = f(s) = 1\] and $f(t) > 1.$ What is the smallest possible value of $f(t)?$