I'm assuming that you want help with problem #2.
Call the bottom-left vertex A, the bottom-right vertex B, and the top vertex C.
Label the points where the square hits the triangle P, Q, R, and S in this fashion:
-- along AB, let the points of the base be, in order, A, P, Q, and B
-- along BC, let the point of intersection be R, and
-- along AC, let the point of intersection be S.
Given: AC = 3, CB = 4, and AB = 5
Triangle(APS) is similar to triangle(ACB), so we can call the length of AP = 3x, PS = 4x, and AD = 5x.
Triangle(BQR) is similar to triangle(BCA), so we can call the length of BQ = 4y, BR = 5y, and QR = 3y.
Triangle(CSR) is similar to triangle(CAB), so we can call the length of SR = 5z, CS = 3z, and CR = 4z.
For side(AC), we have AC = AS + SC = 5x + 3z ---> 5x + 3z = 3. (1)
For side(AB), we have AB = AP + PQ + QB = 3x + 5z + 4y ---> 3x + 4y + 5z = 5. (2)
For side(BC), we have BC = BR + RC = 5y + 4z ---> 5y + 4z = 4. (3)
Since we have a square: SP = SR = QR ---> 4x = 5z = 3y
Using 5z = 3y ---> z = 3y/5
Substituting this into equation 3: 5y + 4z = 4 ---> 5y + 4(3y/5) = 4 ---> 25y+ 12y = 20
---> 37y = 20 ---> y = 20/37.
Therefore, since the side of the square is 3y, side = 3(20/37) = 60/37.