Calcium phosphate, another insoluble will precipitate. This maybe a concern with liquor with a high expected value of calcium - phosphoric acid will reduce calcium ion.
For a while some people were very concerned about phosphoric acid and what it would do to calcium levels in the mash because of this. If you are using just very small doses (ie. the alkalinity of the liquor is very low), then I suspect it makes little difference. If you are using a burtonized profile and have massive amounts of calcium, then phosphate formation will likely impact the calcium levels.
Back to your question at hand. Hardwick in "Handbook of Brewing" references 50 ppm of carbonate and bicarbonate as required for "good brewing liquor" and if I recall, Martin references <60 ppm alkalinity as CaCO3. This effectively neutralizes the buffering capacity of the liquor in the mash, and dramatically reduces the volume of acid necessary to achieve a desired mash pH. I believe one can properly acidify liquor around 120-140 ppm alkalinity without really bumping into flavor contributions from either acid, but you would need to experiment to decide.
I am really hoping Martin steps in and checks my work - I can be expressing this incorrectly.