Statutes and regulations are different animals. It's not just a question of "you say po-tah-to, I say po-tay-to." The words "statute" and "regulation," although used interchangeably by people who don't know better, mean two entirely different things. Violation of either can get you in trouble, though.
Law Makers and Law Breakers
Statutes are laws made by legislatures. At the federal level, this means the United States Congress, the House of Representatives and the Senate. State laws are made by state legislatures. A very specific format is in place that regulates how legislatures make laws, which chamber must introduce the law, whether it must be heard in committee, how it is sent to the other chamber and if, how and when it is sent to the President of the United States or the state governor for signature.
Some statutes regulate individual or corporate behavior. They may forbid certain behavior or mandate certain behavior. If you refuse to follow the law, you can be sanctioned, civilly or criminally. Civil sanctions for failure to obey a statute usually involve having to pay fines. Criminal sanctions include a fine and/or time in jail or prison.
Administrative Fixes
When the legislature enacts a statute, it usually takes a broad-brush approach. It agrees on and creates a framework of laws in a particular area, but doesn't have the time or expertise to fill in the details. The statute usually delegates responsibility to fill in the gaps to government agencies that enact regulations to address the fine points and ambiguities. If the legislature doesn't approve of the way the agency interprets the law, it can enact another, more specific law contradicting the agency's action. At that point, the agency must change its regulations.
It helps to think of this in terms of a specific example, such as taxes. The federal law requiring individuals to pay income tax sets out the general outline of the federal tax structure, but it doesn't get into the picky details, like how to determine whether you can claim an adult child as a dependent. The IRS does this, and the tax statute gives the IRS the duty and authority to do this. It employs accountants and number crunchers who figure out how the law should be employed in different specific situations.