Expand the left hand side
abx2+(a2+b2)x+ab=26x2+[]x+26
So we must have
a*b = 26
a and b are integers, so the possibilities are
a = 1 b = 26
a = 2 b = 13
(or the same with a and b switched).
Hence [] must be either 12+262 or 22+132
I'll leave you to decide which is smaler.