Plain and simple, when you left click a 'key' icon it sets the key to red. Maybe right click a key icon to remove the key if it exists.??
Offtopic question: Why are key's red when key'd? I would think they should be green, but i feel like the team has debated this topic already amongst themselves.