The average automobile was a toxic chemical factory before catalytic converters (aka "cats") became government-mandated equipment in 1972. Oxides of nitrogen and lead, unburned fuel, sulfur dioxide and huge amounts of carbon monoxide poured forth in abundance, polluting groundwater and atmosphere alike. Catalytic converters work like super-high-temperature ovens, storing exhaust heat and using it to convert many of these dangerous compounds to less dangerous forms.
Removing the Converter
In short: DON'T. If your car came with a catalytic converter you're looking at fines of up to $10,000 for removing it. Sort of. Removing the catalytic converter is illegal, but getting caught without one isn't--most states will simply suspend your smog certification until you get a new one installed.
You can only replace your catalytic converter with one of the exact same make and model as the original, and it has to be in the stock location. The only exception to this rule regards "high-flow" aftermarket cats, which you may use as long as they're EPA-certified for your vehicle. The federal government says you can't touch the converter unless the original is missing or damaged, the vehicle is older than 1996 and has more than 50,000 miles or it's a 1996-and-later vehicle with more than 80,000 miles.
You can utilize a used converter from another vehicle of the same make and model as your own under two conditions. The donor vehicle must have the exact same engine and transmission package as yours, and you must have the converter tested to certify that it still works.
Removal in Excepted States
You'll never get fined if you never get caught, which is entirely possible in many places; some states like South Dakota and Florida don't have any emissions testing. Converter removal is fairly common in states like these, but there's one caveat; don't walk up to any old shop and ask them to remove the converter. Most will either tell you that they don't do exhaust work for undercover EPA agents, or that you have to register your vehicle as "off-road only" before they'll touch it.
Even in smog states, many vehicles are exempt from emissions testing. No emissions testing means no fines for going cat-less. Exceptions include almost any vehicle designated for off-road use only, including race cars, ATVs, and dirt bikes. California residents beware, though--CARB (the California Air Resources Board) passed a number of laws in 2009 regarding boat emissions and cat removal. The new regulations and fines are similar to those for cars.
Don't think that just because the converter is physically present that you're square with the government--gutting the cat is never a good idea. This somewhat morbid term refers to the practice of removing the converter, bashing out the ceramic internals with a crowbar and reinstalling it. Common sense would suggest that this should both improve exhaust flow and keep you legal, but it would be wrong on both counts. Modifying or disabling a converter is just as illegal as removing it, and oftentimes turbulent flow through those broken internals will cause horsepower-sapping back-pressure. And it will send your car's engine control unit into fits.