(a)5/14
(b)(1, 2...(n-1), n) over 2n choose 2
(a) If the rotation is irrelevant, we have 8 choose 2 options for our triangle.
Let's call our origin point 0.
If we number the points clockwise:
Points 1 and 8 have 1 success each.
Points 2 and 7 have 2 successes each.
Points 3 and 6 have 3 successes each.
From here it would be logical to guess that points 4 and 5 have 4 successes each.
That is correct!
(Note: Triangle 0-4-6 is the same triangle as 0-6-4, so we need to divide the total result by 2.)
Adding up our successes, we get 1+2+3+4=10 successes.
Our answer is 10/28 or 5/14.
(b) In part (a) we noticed that there was a pattern in successes.
Therefore, we can say that a regular polygon with a number (2n+1) corners will have (1+2+3...(n-2)+(n-1)+n) over (2n) choose 2.
That is our answer.