The problem setter has made a mistake. Cosine of an angle must be within -1 and 1. -15/3 is -5, which is outside of the range. (Unless the angle is a complex number, of course, but that is out of consideration for, supposedly, a math problem in high school.)