My first guess would be you've been hitting a drive through car wash. The valve stem is blocking the spinning sand grinder (I mean wash wheel, sorry) from reaching that part of the wheel. That, or your washing technique is younkeep folding the stem down against the wheel with your wash mitt instead of going around the stem. Try some type of cleaner wax or polish; the same thing you would use on paint and that should do the trick. Not anything overly aggressive, just a finishing polish. I'd try the cleaner wax first. Depending on your detailing skills, a cleaner wax would be safer. Also, don't be scared to wash the barrel of the wheel too. It'll do wonders for the appearance.
I did it by hand with a brush so I'm pretty sure i covered the area, it was just either stuck on pretty good, or a scuff in the paint. I'll try scrubbing with some of the wax like you said. I think I'll also try to hit with a clay bar.
I was wondering if someone would notice the barrel of the wheel! I wanted to get in there, but the brush I had was too fat to fit. Gonna pick up a set of the wheel woolies from chemical guys soon.