Are you sure it's not the Dot hearing something from your TV that sounds like "Alexa" or the wake word you chose and then responding to it?
Your TV's audio should not come out of the Dot unless you've: 1. connected it via an audio cable. 2. paired your TV's bluetooth to the Dot's bluetooth. And even if you did do these things, it doesn't just happen sometimes. Audio should either come from your TV or the Dot and it wouldn't just switch back and forth on its own sometimes.