This worn needle rode too...
This worn needle rode too low in the slide, so at small throttle openings, the symptoms mimicked those of an undersized pilot jet.
Sometimes you just have to dig a little deeper, even when the solution to a problem seems evident. Case in point; a while back I converted my enduro bike to a dual-sport and registered it for road use. In the woods the bike ran flawlessly, but on the street it had a noticeable lean surge at small throttle opening, indicative of a too-lean pilot jet. It wasn't particularly bad and the bike ran okay, so I wasn't too concerned, especially since removing the carburetor to replace the jet was such a headache. Besides, I rarely rode the bike on the street, and I'd only registered the thing so I could compete in NETRA (New England Trail Rider Association) Enduros.
Over the next few years my interests shifted from enduros to dual sports, and because a dual sport ride often means riding on a fair amount of asphalt or graded dirt, the surging problem became more of an issue.
Determined to fix it, I pulled the carburetor so I could install a larger pilot jet. The carburetor was spotless internally, so I popped in the new pilot jet and buttoned everything back up and prepared to reinstall the carb. Before I did, I decided to check the needle clip position. I'd changed needles at one point and had tried several different pipes, so I really just wanted to make sure the clip was in the same position my notes said it should be in.
It wasn't. Much to my surprise, the clip had come loose on the needle. Unfortunately it hadn't fallen completely off, which would have caused the bike to run like a bag of manure and clued me to the problem. Rather, it hung on the needle and chattered up and down until it had worn a good-sized groove approximately three needle clip positions long.
This meant the needle was in essentially the second position, or two notches leaner than it should have been. Sure it ran great in the woods; when you're trail riding, you're rarely at a steady throttle. You're either wide open, where the needle has no effect, or coasting. It was only on the road at small throttle openings that the problem surfaced, and because the needle was still in place, it was masking the true issue.
In hindsight, I should have realized that the other symptoms of a lean pilot jet-slow warm up, backfire on deceleration, etc, weren't prevalent, but in my defense the carburetor I'm using has a coast-enrichening circuit to prevent backfiring, and it ran perfectly under the conditions it was most often used.
The moral? Sometimes the problem isn't as intuitive as it might seem. Sometimes you have to dig a little deeper, take a few extra moments, and really think the situation through before you find the right solution. A little luck never hurts either.