There are several important design and construction factors to consider in high speed rail lines. Among other things, the track must be (as already noted) impeccably maintained. Routes are also designed with extremely gentle curves and absolutely no at-grade road crossings. While curves must be very broad, quite significant grades are fine--with all that horsepower the trains just power over the hills with a minimum loss of speed.
On regular rail routes--lines shared with freight traffic and that have grade crossings--the TGVs run at reduced speed consistent with conventional passenger operation.
The end result is that we won't see true high speed rail in North America unless there is the political will to finance and build brand new rights-of-way.
j