This is just a way to compute the problem without sine/cosine, but the other way is faster.
To maximize the value of AC, we need to maximize angle ABC, given that it is smaller than 180 degrees. Since A, B, and C are on the sphere, the circumcircle of triangle ABC is a cross section of the sphere. The larger this circle, the greater angle ABC is. So, we want to maximize the size of the circle. Thus, we want the circumcircle to be a great circle of the sphere.
In the diagram below ADB is a 10-13-13 triangle. If we let H be the foot of the perpendicular from D to AB, we see that DHA forms a 5-12-13 right triangle. So, DH = 12 and the area of ADB is
(1/2)·DH · AB = (1/2)·12·10 = 60
We can also write the area of ADB as (DB·AX)/2. Since DB=13, we know that AX = 120/13. Thus AC = 2·AX = 240/13
Ok just use the diagram Melody has because I can't figure out how to upload an image-
Just imagine that H exists.
