In Ms. Q's deck of cards, every card is one of four colors (red, green, blue, and yellow), and is labeled with one of seven numbers ($1,$ $2,$ $3,$ $4,$ $5,$ $6,$ and $7$). Among all the cards of each color, there is exactly one card labeled with each number. The cards in Ms. Q's deck are shown below.
How many $5$ card hands can Yunseol draw from Ms. Q's deck, so that exactly two of the cards have the same color?