Look at the image you are framing, see if you can "pull" a colour from the image. It should not the be most predominant colour in the image (that will cause your image to become lost in the matting) but one of the colour "highlights" in the image. Even better, find two different colours and choose a double mat in order to increase the depth and visual interest of your picture.
Last week, I framed a large group wedding photo. This poster size photograph had a great deal of colour variation and made mat selection very challenging. We finally chose a warm mushroomy-taupe colour for the top mat (picking out the stone from the chapel in the photos background) and a dark flannel gray for the under mat (picking up the mens dark suites). The selection of even relatively neutral colours really brought out the best in the photo and gave it that "custom framed" look.
Finally, If you MUST choose white mats, make sure you choose a white (or even better, and off-white) that has some warmth and character. try not to "match" your images paper colour (it tends to make the images paper, or the mats look "dirty").