Thanks, heureka....here's another way using similar triangles...
Referring to heureka's pic....let the side of the square = s ...let WA = x and let BW = 3 - x
Now....triangles ABC, AXW and WBZ are all similar
Using triangles ABC and AXW, we have that XW / AW = BC/ AC → s / x = 4 / 5 → s = (4/5)x → s = .8x
And using triangles ABC and WBZ, we have that BW / ZW = BA / CA → (3 - x ) / s = 3 / 5 → (3 - x) / (.8x) = .6
Multiply both sides by .8x and we have that
3 - x = .48x add x to both sides
3 = 1.48x divide both sides by 1.48
3 / 1.48 = x = 75 / 37 ..... and s is .8 of this = 600 / 370 = 60/37 units