The team I have is fairly big as it is, and any time anyone even looks at me funny, they absolutely DEMOLISH them. Problem is, I seem to be addicted to followers now, and I keep wanting to add more. I find myself creating whole backstories and personalities for these characters and imagining how they might interact with each other.
It's certainly changed the game in interesting ways for me, especially from a roleplay standpoint. So far I have Uthgerd the Unbroken, Serana, Derkeethus, Teldryn Sero, and J'Zargo following me. I downloaded the Amazing Follower Tweaks mod which lets you have up to 15 followers or so and built up a nice little team. So I'm done with all the main quests, and now I've decided to spice things up by doing a 'lots-of-followers' roleplay with my character.