Isolationism in Religion

One thing that has struck me as interesting recently is how often isolation or solitude plays a significant role in religion, despite its nominally communal nature.  The Lasceaux cave paintings were deep underground, the anchorites were on top of poles out in the desert, and monks of all strips inhabited isolated monasteries.  

Given that most religions seems to be social experiences it would be interesting to know where this tendency towards hermitism comes from. 

Personally I would guess that it comes about as a result of the intensity of the religious feeling, which sometimes is so overwhelming that the believer cannot devote sufficient to both contemplate it and live in a society at the same time.