this case is the range of numbers choose sides of a triangle.
so 6 choose 3 = 20
and we want to choose a set of numbers that when you apply the pythagorean theorem (a^2+b^2+c^2) the result is a whole number
only 3, 4, and 5 works in this case between the range 1 to 6 (proof: 3^2+4^2=9+16=25, 25=5^2)
so 1, 2, and 6 doesn't work
so the probablility is 1/2 if no number can be used more than once.
but if a number can be used more than once in the range 1 to 6 (still only the set 3, 4, and 5 works), then the answer is 1/6 choose 3 = 1/20