It is 3/16.
Let us call the circle's center O . We first note that if A and B are points on the circle, then triangle AOB is isosceles with AO=BO. Therefore, if is an obtuse triangle, then the obtuse angle must be at O. So AOB is an obtuse triangle if and only if minor arc AB has measure of more than 90 degrees.
Now, let the three randomly chosen points be D, E, and F. Let theta be the measure of minor arc DE . Since theta is equally likely to be any value from 0 to pi (in radians), the probability that it is less than pi/2 is 1/2.
Now suppose that theta is less than pi/2. For the problem's condition to hold, it is necessary and sufficient for point F to lie within pi/2 of both D and E along the circumference. This is the same as saying that F must lie along a particular arc of measure pi-theta.
The probability of this occurrence is (pi-theta)/2pi, since F is equally likely to go anywhere on the circle. Since the average value of theta between 0 and pi/2 is pi/4 , it follows that the overall probability for is (1/2) - [(pi/4)/2pi] or 3/8.
Since the probability that theta is less than pi/2 is 1/2, our final probability is 1/2 * 3/8 or 3/16 .
Hope this helps.
~~GF