In one respect, HD has made it much easier to make the tour pak detachable on your bike: HD now sells the same amp used on your bike that's mounted under the fairing, as an in-fairing accessory; here's a link to the installation instructions:
http://www.harley-davidson.com/en_US/Media/downloads/Service/isheets/-J05267.pdfThe accessory amp will allow you retain the same audio that you presently have, though you will need to adapt the wiring; pretty straightforward with the wiring diagrams in the service manuals. Of course, there are better amps available to use in the fairing in place of the HD amp.
HD offers detachable tour pak kits for non-CVO Ultras which would provide most of the parts necessary, and instructions for disconnecting the audio and electrical at the tour pak with the exception of the power lock. Here's the installation instructions:
http://www.harley-davidson.com/en_US/Media/downloads/Service/isheets/-J04744.pdfWith the 2011 HD Touring Service Manual and the SM supplement for your '11 CVO RGU, the project is very doable. Yes, you will spend $$$ for a solo seat, detachable hardware, etc. But given your strong preference for riding the bike w/o the tour pak on the bike, it seems like a worthwhile undertaking--I know I enjoy my '04 w/o the tour pak on the bike, and appreciate the convenience of installing the tour pak for long trips.
Suggestion: Buy the SM's from HD and review the relevant portions dealing with the tour pak's electrical and audio, as well as the instructions provided above. They will probably provide you with a good understanding of 95% of what's involved, and make it easier for you to talk to those who have already done the job.
Personally, a Rio Red CVO RGU set-up w/a solo seat and no tour pak looks killer.
Carl