If the phosphate is a buffer and pH drops during fermentation then wouldn't that mean that the pH of the the finished beer would drop less then the beer with less phosphate buffer... hence calcium rich water in mash could lead to higher pH in finished beer since there will more phosphate buffer to counteract the yeast pumping out those H+ during fermentation?.... I am not correcting you just asking for clarification - still working through the finer points of water chemistry.
The more calcium there is in the mash the more phosphates are precipitated. This reduces the amount of phosphate that buffers the pH during fermentation and as a result there is a larger pH drop during fermentation.
A similar aspect of this topic is the effect of low temperature rests and lightly kilned malts where there is noticeable phosphates activity. Acid rests are a prime example. It has been shown that the resulting increase of phosphate also increased the worts pH buffer capacity. This again results in a lower pH drop during fermentation.
Phosphatase activity is also affected by the mash pH. Low mash pH causes more activity than high mash pH which is why a mash mashed with a high pH (5.7 for example) but with acid additions during the boil (wort acidification) can end up with a lower beer pH than the same beer brewed with a low mash pH (5.3 for example) and not additional wort acidification.
Narziss/Back show some interesting data on the comparison between mash, wort and combined mash/wort acidification:
No acidification: mash pH = 5.75, cast out wort pH = 5.65, beer pH = 4.61
mash acidification: mash pH = 5.52, cast out wort pH = 5.47, beer pH = 4.55
wort acidification: mash pH = 5.74, cast out wort pH = 5.20, beer pH = 4.36
combined mash/wort acidification: mash pH = 5.52, cast out wort pH = 5.20, beer pH = 4.43
You notice that of the beers with the same cast-out wort pH the one with the higher mash pH results in the one with the lowest beer pH.
However, this is pretty far beyond the basic water chemistry for brewers and the idea of lower mash pH -> lower beer pH holds true enough that making this simplification is better than getting into the details of phosphatase activity in the mash.