Ron wrote, "It is the natural shape to contain a mass of liquid that exerts an equal force in all horizontal directions."
That's exactly true, and not only for horizontal force but in all directions as well. Square corners are much more subject to rupture, so it's prudent to avoid those as much as possible. This same principle is what dictates the round shapes of most boilers, engine cylinders, and any other vessels which must hold pressure. Even the relatively low pressures exerted by materials in a non-pressurized container are most safely held in round containers - e.g. tank cars, stationary tanks, silos, etc.