I'm a bit out of my depth here (well, ok, a lot out of my depth!), but I think I might try to get an understanding by generating some specific examples using small matrices; e.g. choosing n = 3 and m = 1, say, so that A is a 3x3 matrix, B is a 1x3 vector, O is just the scalar 0, H is a 4x4 matrix and k would have just two values, 0 and 1. Fill A and B with some (randomly chosen?) test values to see if you can generate values that satisfy each of your three conditions in turn. I don't know if this is feasible, or even if it helps if it is, but it's the best I can come up with!
+2353 
+33616 
