So here is what happened.  I took everything apart last night, and checked the side movement of the rockers (while torqued, and on base circle).  Exhaust had almost .021!! Intake was only .016.   When I was finished with shimming, I ended up with about .004 on exhaust and .005 on intake.  Yes I dressed the rockers to use bigger shims.  I then moved on to the rocker boxes.  No new marks for contact.. so that is good.  So I aligned the box, torqued, installed rockers, torqued, checked clearance, and buttoned everything up!
  End result?  Huge difference.  While I still have that clatter, I actually have to listen for it now.  Its very faint and almost sounds like its underneath me and not right out in front of the engine.  I rode it around last night for about an hour.  Highway, in town, etc.  Very different.  While it still has typical Harley sound, It does not have that 2200-3200RPM clatter (faintly) that it used to.  I do however seem to have a tiny bit of a tick like before, and my theory is where I aligned the boxes (ended up moving closer to the pushrods), I may need to clearance the box a little bit more.  Its either that, or one of my new lifters is failing...
We shall see.  
This site is a wealth of knowledge, that you just couldnt teach in a class...
Thanks
GLen