Ultrafan, you can PM Heatwave or find his thread on making the tour pak detachable using a two-channel amp for the front speakers and no amp for the rear speakers. The OEM wiring sends the outputs for the front and rear speakers to the amp under the tour pak, and sends the amplified signal back to the front speakers and to the rear speakers. If you remove the amp/tour pak, without changing the OEM wiring, you lose the audio signal from the amp, so no sound in the front speakers.
To make the tour pak detachable, you will need to re-route the audio signal from the OEM configuration to the front speakers and keep it inside the fairing; either with a two-channel fairing mounted amp or a direct speaker connection from the head unit to the speakers. An advantage of the J&M amp is it includes a bracket for accommodating the CB and XM modules with their amp.
I don't know if the amp you have under the tour pak can be attached to the HD detachable tour pak rack. If it can be, you can use it to amplify the rear speaker signals only and not connect the amp's output to the front speakers. Or you could add another two-channel amp that will fit under the detachable rack to amplify the rear speaker signal. In any case, you will need to alter the OEM wiring. It's not difficult to do; just depends on how you want the audio to function (amplified front speakers or not/ amplified rear speakers or not) and how much work and or $$$ you want to expend.
It's possible to install the OEM amp in a saddlebag though I'm not sure that's been a pleasing solution to the few have posted on doing it. Check with Heatwave.