In a 1992 interview with Melody Maker, Siouxsie Sioux explained the meaning behind the song:

“This is based on a programme I saw about a female version of Amnesty, called ‘Les Sentinelles’. They rescue women who are trapped in certain religious climates in the Middle East, religions that view any kind of pre-marital sexual aspersion as punishable by death – either by the hand of the eldest brother in the family, or by public stoning. And there was this instance of a woman whose daughter had developed a tumour, and, of course, gossip abounded that she was pregnant. The doctor who removed the tumour allowed her to take it back to the village to prove that, no, it wasn’t a baby – but they wouldn’t believe her. The woman knew her daughter would have to be stoned to death so she poisoned her, out of kindness, to save her from a worse fate. Now this organisation has all these escape routes for women like her, mainly through the elder brother who pretends to have killed them. But once they’ve been saved, they can never go back. So the song starts, "Kinder than with poison…” I also used the imagery of “He gives birth to swimming horses”, from the fact that male sea horses give birth to the children, so they’re the only species that have a maternal feel for the young. It was, I suppose, an abstract way of linking it all together without being sensationalist. I remember just being really moved by that programme, and wanting to get the sorrow out of me."