Maybe so. I've been coming up there for 50 years myself and never had a problem either. I think the $5000 fine and the kind of ridiculous (IMO) pic of him standing by the "Keep out " sign is what set people off. And you are right that there are a lot of idiots with 4x4s out there.
I'm very pro law enforcement but i have had a couple of bad experiences while railfanning, but not in Silverton. Both involved being told to "move along" when standing on a public sidewalk taking pics of a train. No reason whatsoever for that and no law was broken. I'm pro law enforcement, not pro-power tripping.