I don't know for sure, but I'll take an educated guess.
The main engineering considerations in building a tank are the weight (water weighs 8 lbs. per gal.) and the pressure it exerts on the exterior wall.
By using a circular design two things are accomplished: The stress is equalized around the exterior wall, and the exterior wall is the minimum length necessary to contain the given quantity of water. (E.g. The relatively small tank at Proctor, Vt. on the Rutland is reported to have had a capacity of 8,000 gallons. 1 cubic foot of water contains 7.48 gallons. So, an 8,000 gallon tank would require 1070 cubic feet. Assuming the tank is 10 feet high, the base needs to be 107 square feet. A square tank would require 41.2 feet of wall, while a circular tank would require only 36.7 feet)
The stress on the exterior wall requires some sort of counterforce pushing inwards. On a circular tank this can be accomplished using bands or cables. The frequency of the bands increases towards the bottom of the tank as the stress is greater. Pictures of square tanks that I have seen appear to have relatively thick braced walls. The square design means that the corners of the tank wall are strong, but the centers are weak, unlike the even strength of a circular tank.
Another consideration is that the joins between the boards had to be carefully milled, or caulked. Again, the circle reduces the number of joins to be milled and/or caulked.
Basically this means the circular design is easier to contstruct (from an engineering standpoint) than the square design.
Hope this helps.
Burr