1937 profiles of the DRGW between Alamosa and Durango indicate a fair amount of "gravel ballast (in spots)", but gravel ballast is not defined. What non dirt ballast I remember from the 60's was engine cinders and scoria, except of course for the Navajo Reservoir relocation which was rock.
My recollections from the 60's are that the line and surface were pretty good, at least in most places. They were still running big trains and bad track would have been a problem. Maybe toward the very end it got worse. Would be interesting to hear from the folks on the trains. The problem I was told with "native earth" (dirt) ballast was that it did not drain very well and could not be tamped with machines, so it took a lot of manual labor to maintain it. But with enough manual effort it could support pretty good track I guess.
Was it Andy Payne who came up with "Through the weeds not around them". I always thought that was cool.
JBWX