I too am having very similar problems with my 09 SERG. I've already been to my local stealorship for service and the tech found nothing wrong... however, it is a real issue that I can feel when at low speeds when I let go of the bars briefly just to see the reaction. You can see a clear wobble/vibration from the handlebars.
Also at higher speeds when cornering the bike can feel marginally unstable in the turn. Cornering to the right vs. the left yields different results... can't tell you which side is worse but it is obvious when it occurs. Doesn't neccesarily feel unsafe, rather uncomfortable is a better word.
I also notice that when riding on a flat road at any speed, when I briefly let go of the bars the bike veers gently to the left. I've tested this on different roads on different days and the bike veers to the left each time.
Lastly worth noting is that when I purchased the bike, I had the stealor put on a set of Metzelers, front and rear instead of the stock Dunlops.
For me the question becomes: Is it the steering head tightness or lack of proper lubrication..? Or... possibly a tire issue..? Or something worse...
And since my local HD shop can't or won't fix this, I'm heading up to Rossmeyer's Ormond Shop for service on Thursday for another look (Thanks Harry!). I'm hopeful they'll put this to bed once and for all. It's a bit unnerving when this wobble happens at any speed...

And FWIW I have adjusted the tire pressure up to a higher pressure and some of the wobble/vibration does seem to go away to some degree but not completely... and still veers to the left.