We tackle the problem in the following manner: First rewrite f(x) = f(f(x)) as the system of equations \(\begin{cases} f(k) = k\\ f(x) = k \end{cases}\).
(This system is obtained by letting k = f(x) and substituting)
Simplifying f(x) gives \(f(x) = x^3 - 6x^2 + 2x\). For f(k) = k, that means:
\(k^3 - 6k^2 +2k = k\\ k^3 - 6k^2 + k = 0\\ k(k^2 - 6k + 1) = 0\\ k = 0 \text{ or }k^2 - 6k + 1 = 0\)
For the remaining part of the problem, we solve it by quadratic formula to get \(k = 3 \pm 2 \sqrt 2\).
For each value of k, we solve the equation f(x) = k. We first start with the easy case k = 0. For k = 0, we have
\(x^3 - 6x^2 + 2x = 0\\ x(x^2 - 6x + 2) = 0\\ x = 0 \text{ or }x^2 - 6x + 2 = 0\\ x = 0\text{ or }x = 3\pm \sqrt 7\)
For k = 3 + 2 sqrt(2), we know that x = 3 + 2 sqrt(2) is a root of f(x) = k because f(k) = k. We have, by long division,
\(x^3 - 6x^2 + 2x = 3 + 2 \sqrt 2\\ (x - (3 + 2 \sqrt 2))(x^2 + (2 \sqrt 2 - 3)x + 1) = 0\)
The quadratic factor has negative discriminant. That means no real root there. The only real root from this case is \(x = 3 + 2 \sqrt 2\).
Similarly, for the case k = 3 - 2 sqrt(2), we have
\((x - (3 - 2 \sqrt 2))(x^2 - (3 + 2 \sqrt 2)x + 1) = 0\)
Again, the quadratic factor has negative discriminant. The only real root in this case is \(x = 3- 2 \sqrt 2\).
Hence, the real roots you are looking for are \(x = 0\), \(x = 3\pm 2 \sqrt 2\), and \(x = 3 \pm \sqrt 7\).
