Yeah, it happens anytime you have multiple retractions and travels in a short period, resulting in a "undefined" extruder state.
For example, here's a particularly bad case.
It's almost perfect until it gets to a layer with several small top/bottom segments that it has to jump to before jumping to the next layer, which is the start of the outer wall lines, as you can see here:
There are tons of things you can play with to try to mitigate it, but the overall problem is you can't model the extruder state perfectly under arbitrary retraction and travel. Thus we need a strategy to ensure we get the extruder into a known state.
If we could print these small top/bottom segments before the infill, that would often help somewhat, as the extruder state will become predictable while printing these connected infill segments. Generally we can pretty accurately model a single retract/travel/untract sequence, but it quickly becomes intractable as we stack them up, esp. if when dealing with a high-throughput scenario with a volcano nozzle.
Again, I'd love to be able to start these outer wall segments early. Basically the logic would be "N+1 wall lines, except the first wall line (if it exists) is trimmed to the last N millimeters of the line".