The key to audience etiquette is to know what is considered good manners for the type of performance you are attending. What is appropriate in one context doesn't always readily translate to another.
If you go the theater to watch a play, talking through it will upset members of the audience around you and perhaps even the cast on stage. The same applies to a musical presentation in a concert hall.
However if you go to a large outdoor political rally or rock concert you'll be free to comment to your heart's content.
So what is standard audience etiquette for listening to speakers?
However if you feel he's not performing well, it is not considered good manners to make that public knowledge. Audience etiquette follows the 'do unto others as you would have done unto you' rule.
Like you I can remember many examples of thoughtless behavior.
There was an older woman and her friend sharing a bag of peppermints through a violin concerto. They unwrapped them in loud slow motion oblivious to the glares from those near by.
Yet another occurred at a play. A cell phone rang. It was answered and a conversation began. The lead actress stopped the play, left the stage, and reappeared in the aisle scanning the audience. Having found her man, she escorted him and his phone, amid cheers and clapping, to the exit.
Good audience etiquette let's everyone do what they came to do, listen to the speaker or music, or watch the play.
If you sincerely don't know what's expected for a particular type of public performance, ask someone who's got the experience to tell you. While you're at the event carefully observe the behavior around you. Take the best as a model for your own conduct.