Having similar thoughts I have tried a moderate amount of lactose (10-15%) to add body in one case, and add sweetness in another to light (colored and bodied) beers. In my opinion, at that rate, I didn't feel like it contributed much of either....of course, I didn't brew side by side comparisons.
Both beers were well loved *by everyone but me*, and I am likely to brew again, upping the ante on lactose somewhat, so "those people" won't drink my good stuff.