I do mine exactly the same as you did and I've yet to have a problem at all.
One thing I like to do (because I'm cheap) is to add a cup or two of sterilized wort back to the growler after pitching. I've found the little bit of yeast left in the growler jumps into action. After a few days, I add another quart or two and in a week I've got another starter ready to go. Of course you have to plan on brewing something that needs that yeast again in the near future but for WLP001 or something that I use a lot, I find this way I can usually have a starter ready to go when I need it. No idea about the yeast number etc but based on the volume of settled yeast, it's gotta be more than a tube or a smack pack. I wouldn't do it for lager where you really want a dense starter but for ales I'm more than happy with the results.