Actually It’s good that you ask, anon, because I’ve been thinking about that matter too and came to the conclusion that yes, even the undies are black.
(All of this is headcanon)
It’s part of the uniform. And I like to imagine that at the beginning of the NW’s establishment they were stricter in members being outfitted properly, so ALL of the clothing had to be black.
But nowadays, with the crumbling force and scant money - and the fact that many of the black brothers are criminals or from poor/peasant families - they eased up a bit on it. I imagine it’s cheaper and easier to maintain the undergarments people brought with them than to dye/buy the right material for clothing no one sees anyway as long as the outward uniform is properly meeting the NW standards (the one you are given (uniform as in “does-not-fit-anyone-properly” unless you’re from a wealthy family or even a lordling so you bring your own blacks made from finer materials (i.e. see Waymar Royce in the prologue in AGOT or the fact that all of the clothes Sam brought with him were useless because they weren’t black).
But then again questions arise: Where/how does the whole washing procedure take place? It’s safe to say it’s a task done by the stewards who also do the sewing/production and maintenance of clothes. But in what dimensions? There is a bathhouse at Castle Black so I assume there also is a laundry? How is the washing organized? (On what weekdays do they gather the clothes from the barracks? When do they wash the undergarments and when the coats? [I have no idea how this works in RL so it’ll be great if someone liked to fill me in]) How do they tell them apart? (name badges on their undies? How many of the brother’s actually can read/write??)Still, any new garment handed out by the NW should be black. So if you’ve lost or tore your undies beyond repair the new ones should be black - operating under the assumption that even the undergarment colour is strictly regulated and the NW has the financial capacities to afford the production.