((x^2+2xy+y^2)/(x^2-y))*((7x^2-3xy-4y^2)/(4x^2-3xy-7y^2))

$$\frac{(x^2+2xy+y^2)}{(x^2-y)}*\frac{(7x^2-3xy-4y^2)}{(4x^2-3xy-7y^2)}\\\\
=\frac{(x+y)(x+y)}{(x^2-y)}*\frac{(x+y)(7x-4y)}{(x+y)(4x-7y)}\\\\
=\frac{(x+y)(x+y)}{(x^2-y)}*\frac{(7x-4y)}{(4x-7y)}\\\\
=\frac{(x+y)^2(7x-4y)}{(x^2-y)(4x-7y)}\\\\$$

I don't know if it can be simplified any further.  

I think that when  7x² - 3xy - 4y²  is factored, the factors are  (x-y)(7x + 4y), giving:

[(x + y)(x + y)]/[(x + y)(x - y)] · [(x - y)(7x + 4y)]/[(x + y)(4x - 7y)]

=  (7x + 4y) / (4x - 7y)