Vintage, Original PartsThe best part for the bike is the one it came with when built. They fit best, look right, and are original. Sometimes you can't have that. The next best thing is a part from another bike like yours, the same year, same make, same model. Sometimes you have to get one from a different year, but the part is still close in age and vintage itself.
This site is an exchange with a focus on vintage and antique American bicycles. You also sometimes get English parts there.
This site has a focus on post-WWII lightweights, roadsters, and road bikes:
Then there's always Ebay:
Modern ReproductionsThe next alternative is to buy a part that is newly produced, but is otherwise correct. Sometimes they are produced as actual "re"productions to copy a no longer available vintage part. The quality of those parts vary widely. I have a set of reproduction Torrington #10 pedals that I like. However, it turns out there are several types of reproduction Torrington #10 pedals. The type I got were good, but later ones were apparently quite poor in quality and the word "Torrington" was even spelled wrong.
Other times they are new production parts simply made the same way as years ago and continuing in production. They are not "reproductions" because production never stopped. It may have moved from one country to another, but the parts are essentially the same sort. The quality usually is not quite as good, but then you can still get a part that fits.
Here is an example:
I bought several parts for a very old, English bicycle from the above shop. The parts they stock are more or less the same as the 70+ year old originals, but are new production. The reason for that is that in some parts of the world, those old parts and bikes never went out of service. People there are riding new production bicycles that are very much the same as the originals from England. The quality is not quite as good, but then it's an attractive alternative to searching forever for vintage parts.
Modern ReplacementsSometimes an old part is such a headache that you want to switch to a fully modern alternative. One such example is the singletube type rim used in the 1930s and before on American bicycles. These rims took special tires that glued onto the rim, but which are larger and slower than modern sew-up or glue-on road bike tires. They're like a mix of a cruiser tire and a glue-on tire. You can still buy reproductions of those tires, but they are prohibitively expensive. They often run $100+ per tire. What's more, the single tube tires integrate the tube and the tire skin, so punctures are a massive pain to fix. You can't just patch or replace the tube and keep going. You have to fix the whole tire all at once. It's all the more a pain because you're working on a $100 tire and botching a repair will be costly.
So what can you do? You can replace the 28 inch wood and metal single tube rim with a modern 700c Velocity P35 rim. The rim has a very similar shape and size to the original wood and metal clad singletube rim, but will take modern tires and tubes in the 700c size. Effectively, you have a rim that looks almost the same, but now you can use any of a number of modern tires without worrying about a $100 singletube that may pose a problem later. In this case, replacing the original part with a modern one is a good idea (for more on it try here: http://thecabe.com/vbulletin/showthread.php?19356-Mid-20-s-Hawthorne-Flyer&highlight=700c )
That is just one example where you may end up replacing your original part with a fully modern item, in order to maintain a practical and ride ready bicycle. Other items that are candidates are improved brakes, changed gearing, improved chains, and better lights/reflectors.