I think your individual setup can have a lot to do with whether a secondary is worth your time and effort.
1. Fining - One reason not to rack is that, if you keg, you can brighten up (and fine) your beer in the keg. However, if you don't keg, and you want to reuse the yeast you have in primary, and you want to use post-fermentation fining agents, like gelatin or polyclar, then a secondary is the best time to use them. But, if you can't get your carboy down to refrigerator temp, then you might as well not use a post-fermentation fining agent, anyway.
2. Dry Hops and Other Post-Fermentation Additions - if you keg, you can dry hop in the keg (though I have had mixed results doing this, I still do it fairly often). If you don't keg, your only opportunity to dry hop is in the fermenter. Many people dry hop in primary, but this can be difficult if you want to re-use the yeast, or you want to give your beer an extended primary. The one time I had autolysis issues in a batch was when I left it sit in primary for 6 weeks. Since many (most?) people like to give their dry hops about two weeks, you may want to dry hop in secondary to get the beer off the yeast. The same would apply to oak, spices, fruit, or other post-fermentation additions.
3. Space - If you're impatient like me, then maybe you want to brew and all your buckets and 6.5 gallon carboys are already full of beer! A happy problem, I solve this one by racking one of the beers to make space.
4. Lagering - I have heard of people lagering in primary, but that option would be risky, IMO, if you planned on lagering for an extended period of time. I would be weary of leaving my beer on my yeast for more than 2 months, even at lagering temperatures - and many beers are lagered for even much longer than that. While you can certainly lager in a keg rather than a secondary fermenter, this may not be a good option if you dont keg or you have a limited number of cornies or a lagering fridge that isn't tall enough for a keg, or whatnot.
5. The "If It Ain't Broke" Rule - If you secondary, and are happy with your results, why change it? Likewise, if you do not secondary, and you get bright, beautiful beer, why secondary?
Personally, I do not get bright beautiful beer unless I secondary, cold crash, and lager; or use a post-fermentation fining agent. And I find the flavor of too much suspended yeast to be unpleasant, especially in hoppy beers, which can be biting and needlessly bitter when I don't clear out the yeast. I also like to reuse my yeast several times, and so I don't like putting anything (dry hops or polyclar or gelatin) in my primary vessel. So I frequently do a secondary. When do I not do a secondary? When I am brewing a cloudy style like a witbier, or when I am brewing a low-gravity beer with an English yeast strain, or when I just don't have time.