If you connect the radii of each circle to its tangent points with the other circles, you have a square with side lenth 2r.
The diagonal of this square is 2r√2 by the pythagorean theorem.
The diagonal is r+small circle diameter+r = 2r√2
Subtracting both sides by 2r, the diameter of the samll circle should be 2r√2−2r which means the radius should be 2r√2−2r2=√2r−r=r(√2−1).