I KNOW where Christoph Blocher lives. The billionaire leader of the Schweizerische Volkspartei owns a substantial property – it would probably be fair to call it a castle – that can clearly be seen from the cable car linking Rhäzüns to Feldis, the alpine village where my mother’s ashes have been reburied next to those of her brother and sister.

The SVP, of course, has been the driving force behind the referendum that has seen voters back the call for a constitutional proscription on the construction of minarets on Swiss mosques. Take that as further evidence that Blocher is pushing Swiss politics towards the right, on the back of the growth of a certain ugly xenophobia in a country that has always presented itself as the very model of a successful multinational state.

The Swiss economy has always made use of many seasonal guest workers, and one of my uncles took on Muslims from Turkey and Yugoslavia as well as Catholic Italians at his door factory. Although they had no rights to speak of – they had to return to their country of origin each year – I don’t recall racism being an issue in the 1960s. There was certainly no problem with employees dating the boss’s daughters, for instance. Two of my cousins even married the guys.

Marriage to a Swiss citizen confers the right of residence, and slowly the number of Swiss Muslims began to grow. In 1970 there were just 16,000; today, thanks to a liberalisation of the restrictions on work permits and the arrival fairly sizeable numbers of refugees from Bosnia and Albania, the total stands at 400,000.

As a Swiss national and a fairly regular visitor, I am well aware that the Zwinglian-dominated Germanophone part of the country is naturally conservative on social issues, and unlike some commentators, I am not surprised that the minaret ban won through. But it isn’t necessarily evidence that Switzerland is any more (or any less) racist than anywhere else.

While the decision was clearly the wrong one, the significance may not be that great in practice. There are some 150 mosques and Muslim prayer rooms, all still accorded freedom of worship in accordance with the standards of liberal democracy.

But it is the symbolism is worrying. One day a rightwing populist party, perhaps with mass repatriation as part of its platform, will surely win an election and form a government somewhere in western Europe. That’s when the social fabric will be tested.