Yes, the front mount of the tour pak attaches to the inner chrome looking spacers, not the outside black plastic spots (that's where a detachable backrest would attach).
Best thing I can tell you to do is to get the instructions for the detachable hardware kit. The black spools are oval in shape, so there are "looser" and "tighter" positions. Also, there are chrome washers, spacers, etc that can be placed in different locations.
I had similar trouble and once I read the instructions, which talked about how to get things aligned better, my sissy bar and my tour park go on snugly, but not tight and not difficult and I don't have to push and twist to get things aligned.
If you look up the docking hardware part number on HD's website, the instructions can be downloaded and printed out.