I've been wondering about Dumbledore's clothes too. I mean, lots of wizards wear clothing that's flamboyant by the local Muggle standards, but purple velvet seems like it's on the flamboyant side even for them. Fudge might have a lime-green hat, but he also has a pinstriped suit, you know?

On the other hand, it does seem like older wizards in general go for the outrageous robes, so maybe the only reason Dumbledore's wardrobe sticks out is because he's a more major character.