why does the great circle distance formula work?

I'm only know a little about this....it involves spherical trigonometry which is quite different from plane trig.

Here are a couple of references that may help you.....

http://en.wikipedia.org/wiki/Great-circle_distance

http://en.wikipedia.org/wiki/Spherical_law_of_cosines

The second site covers the Spherical Law of Cosines......good luck!!! This topic of global navigation is quite detailed and could comprise a whole semester of study on the college level!!  (Maybe it should be an "elective" course !!!)

Maybe they drew a circle.