Remove Ignition Key Warning Keeps Going Off

I have a push button start ignition but every time I get out of my car it makes a warning noise that I left the key in the ignition or my headlights are on. What's up with that? Mark explains what's going on and how to stop it. Cheers from Canada!