I agree with what you said. Although I got my first HD when I was 18 (got a good price b/c the engine needed a complete rebuild), there were many years when I was raising 4 kids that I couldn't justify spending that much on something I couldn't spend time on, so I made do with some metric bikes (all of them great fun, but not the real thing).
I think that cost is a big reason HD came out with the Street 500 and 750, to try to bring younger people on a tight budget into the fold so they'll buy a full-size model later. Otherwise, HD's customer base is just going to die off and HD sales will go through the floor. That's also the reason for a factory trike, to keep more seniors riding and of course buying parts, service, and accessories.