An example of a wartime scrapped grade is the SG Central Pacific grade from Ogden to Corinne, Utah, which has absolutely no metal remnants to indicate railroad use. The only indication of its heritage are bits and pieces of coal scattered among the weeds.
From Corinne to Promontory, the scrappers left quite a lot of spikes and fishplates, but subsequent landowners removed the majority of it. What was left was then dug up and catalogued by the National Park Service when Golden Spike NHS was still new. From current observation there isn't even any ballast to indicate that it was a railroad.
But if you head out west of Promontory past Rozel, it becomes apparent that the grade was once a railroad. Fishplates, tie plates and occasional spikes dot the ROW and dark ballast, still with tie indentations, distinguishes it from the surrounding landscape.
So even on a single stretch of track, conditions can change. It's hard to give a rule of thumb to determine the answer to your question.