This is a trianglular pyramid
So volume = 1/3 * area of base * perpendicular height
base = 1/2*6*8=24cm squared
$$height=\sqrt{11}*sin\theta$$
$$\\v=\frac{1}{3}*24*\sqrt{11}*sin\theta\\\\
v=8\sqrt{11}sin\theta\\\\$$
Volume is greatest when sin theta is greatest ie theta=90degrees
$$$Max volume $= 8\;\sqrt{11}\; cm^3$$
That is what I think anyway.

In my diagram the right angled vertice is A The going clockwise B is at the top then C then D