Let's have him get his first $4$ pairs from white, red, blue, rainbow. Then we have to pick as many socks as possible without getting another pair. I will take a green sock, a purple sock, and another white, red, blue, and rainbow sock. Now, the next sock I pick, I am guaranteed to get another pair. So far, I have picked $4*2+6=14$ socks. The answer is $14+1=\boxed{15}$ by the pigeonhole principle.