Hazrat Shaikh Moulana Muhammad Zakariyya (rahmatullahi ‘alaih) once gave a person the following advice. He said:
As far as possible, always try not to mix with people. To mix too much with people, without a (worldly or deeni) need, is harmful, and does not bring any benefit. As far as mixing with your family is concerned, then this is not harmful, rather this is part of the sunnah.
The people whom I advise you to refrain from mixing with are the general public. Nevertheless, to mix with people for the purpose of tableegh (e.g. gusht, going out in jamaat, etc.), or tazkiyah (i.e. self-rectification through attending islaahi programs, spending time in the khaanqah, etc.) or for ta’leem (to teach people deen) is not that mixing which is discouraged. Rather, this is part of deen.
(Qutbul Aqtaab Hazrat Sheikhul Hadith Moulana Muhammad Zakariyya (rahmatullahi ‘alaih) pg. 457)