You will get it to dissolve some as you heat the water up. Make sure to stir it really well before you dump it into your mash so that you get any undissolved solids to carry over with the water.
That's brilliant! Mix it up really well and everything is OK.
Unfortunately, the laws of physics don't agree with you. Chalk is actually dependent upon an acid to dissolve into its ionic components. Although the formula for chalk is CaCO3, it doesn't dissolve to produce calcium and carbonate ions to the water. When it dissolves, its in the form of calcium and bicarbonate (HCO3) ions. That H proton in the bicarbonate has to come from somewhere and its an acid that provides it.
As mentioned above, carbonic acid from dissolved CO2 is one way to get acid into water. Applying a dose of CO2 at higher pressure is a way to get more CO2 dissolved into the water. Unfortunately, the solubility of CO2 decreases markedly as the temperature of the water increases. That is why the boiling treatment of water with high temporary hardness works. The heating of the water drives off the dissolved CO2 and the chalk then precipitates out of the water. If the water has temporary hardness and you boil it, sometimes you can't see the chalk since it starts out at near molecular size. Once the heating is ended, those particles will settle. With enough time, those teeny particles gather together to the point that you might be able to see them. Adding an extra dose of powdered chalk is actually helpful at that stage since that chalk provides nucleation sites for the rest of the teeny chalk particles to attach to and sink to the bottom of the kettle.
In the heating and stirring example above, that finely powdered chalk does seem to go into solution when you mix it up well. But if you stop stirring and wait a while, you would eventually see all of that chalk on the bottom of the pot.
Clarity is a bad measuring stick for saying that something is 'dissolved'. Look at the case of yeast. In White and Zainasheff's book
Yeast, they point out that a yeast suspension with up to 1 million cells per milliliter appears clear. Above that cell density, it looks turbid. But that is a million cells in a mL !! And yeast cells are relatively big, just like pieces of powdered chalk. So it should be no problem to get a few grams of chalk to seemingly disappear in several gallons of water.
I'm surprised with brew1314rw's results regarding the pH ending up fine. I'm curious if those brew's needed extra alkalinity in the first place. If they were relying on Palmer's nomograph, it has long been proven to recommend FAR too much alkalinity for darker brews. So it was probably good that chalk is ineffective at adding alkalinity. I'm sure that they are using a freshly calibrated pH meter to confirm those pH results, but all the evidence that I've gathered says that adding chalk does not provide ANY appreciable alkalinity unless it is properly dissolved with CO2 several days in advance of use. I've even had professional brewers with very thorough testing regimen come to me and we were able to prove that their chalk additions were not doing anything to moderate their mash pH drop.
Don't bother with chalk. It does not work at all. If your water REALLY needs more alkalinity, then using a surer chemical to deliver that alkalinity is recommended. Be sure that you really need the alkalinity in your mash water since producing a too high mash pH is much worse than having a too low mash pH. Either lime or baking soda are sure ways to add alkalinity. Be careful with baking soda since the added sodium can be flavor negative, especially if there is elevated sulfate and/or chloride in the brewing water. Lime is the way to go in most cases, but it REQUIRES a very careful dosing. You must have a accurate scale to use that stuff.
Just say
NO to chalk!