(In reference to swingers) In the meantime, if you wish to declare yourself polyamorous, get used to the fact that the confusion is going to remain as a pejorative. Sure, clear up the misunderstanding as much as you can, but don't put too much effort into setting yourself up as a "good", responsible, community-oriented polyamories by contrasting yourself to the "bad" swingers - they may not be your siblings, but they're definitely your cousins.

Polyamory: Roadmaps for the Clueless and Hopeful: An Introduction on Polyamory