http://www.graficaobscura.com/matrix/index.html describes the general idea. If you look at the RGB Matrix Data Entry dialog, it tells you that
R = *R + *G + *B + 
G = *R + *G + *B + 
B = *R + *G + *B + 
where  is a number that you enter (different number for each ). The key to understanding this is that every color has a red, a green, and a blue component (thus, RGB). This filter processes an input RGB color into an output RGB color. Fully labeling the above equations gives
outputR = *inputR + *inputG + *inputB + 
outputG = *inputR + *inputG + *inputB + 
outputB = *inputR + *inputG + *inputB + 
Why aren't they fully labeled on the dialog? I'm going to claim that I didn't want the dialog to get too wide. Yeah, that's it! I didn't want the dialog to get too wide!
Anyhow, the first row defines the output red component of the color as the weighted sum of the input RGB components. If you want to get rid of all red in your drawing, use 0, 0, 0, 0 for the 4 numbers on the outputR line. If you want the output red to have no relation to the input RGB color, but just be a solid red, then use 0, 0, 0, 1. If you want a perceptually gray color to replace the output, use the numbers 0.3, 0.59, 0.11, 0 for all of the output RGB components (these values are the Gray predefine value from the dialog). The sum of the numbers that you enter across a row should be 1.