Marlin Bug: The Myth of Retraction Speed

[illuminarti 18]

While following up on a posting on the Google Group that Joergen made a week or two back, I discovered that retraction isn't working as it should on current versions of Marlin firmware. Specifically, if you ask for a filament move speed faster than about 12mm/s, you get a lower speed than you ask for - typically half the speed, but sometimes as little as a quarter of the speed.

Joergen was seeing some oddities in how fast the extruder turned at certain requested speeds, and so I investigated further by emptying the extruder drive of filament, and then requesting 100mm moves at various speeds. I'm attaching a pdf file that shows the behavior. Basically, at any speed that requires more than 10,000 steps per second, the motor turns a lot slower than it should.

This happens because internally the stepper code in Marlin is (soft) limited to 40,000 steps per second - any speed faster than that will be capped at 40,000 steps per second (depending on your steps-per-e setting, this is going to be about 48mm/s). But internally, if the requested speed requires more than 10,000 steps per second (around 12mm/s), the firmware actually scales back the stepper clock by a factor of two, or four, and then makes a note to compensate by firing off two or four steps each time.

However, because of how acceleration is implemented, Marlin can forget the number of steps-per-interrupt, and instead use the scale factor from the last acceleration move that it did, while using the correct interrupt timing for the constant speed part of each move. Depending on speed and acceleration settings, this could affect any axis, but in practice it mostly seems to affect the extruder axis because of its high ratio of acceleration to move speed. When it happens, the motors moves the right distance, but take two or four times longer than they should.

The net result? If the extruder is moving below 12mm/s (which it should always be during actual printing) then everything is fine. At speeds between 12 and 24mm/s, it actually moves at about 6 to 12mm/s. At 24 - 27 mm/s requested speed it moves at about 6 or 7 mm/s. Between 27 and 48mm/s, it will move at half the speed you set. (I typically set my retraction speed to 40mm/s, and so am really getting 20mm/s).

With any speed over about 48mm/s, the rate is going to be capped by the 40k/s step limit, and will typically move at 24mm/s. At some higher speed (not sure exactly what) a kink in the acceleration planning means that the interrupt step count gets corrected and then it will start running at a true 48mm/s (but no faster because of the steps-per-second limit). This sudden jump to 48mm/s was what Joergen saw, I think. For me... a true 48mm/s seems to be close to the physical limits, and sometimes the extruder motor will just scream in place without turning. For reference, 48mm/s is roughly two full turns of the big extruder gear per second.

I've corrected the stepper-driver code in Marlin to fix the bug, and will be submitting a pull request. With the new code, the extruder now turns at the expected speed at every setting up to 48mm/s. We probably need to think about how we implement it - some of us will need to reconsider our retraction settings - anyone with a speed setting in the 25-50 mm/s range is probably going to find that retractions happen twice as fast with the corrected firmware. For prints with a lot of quick retractions, at the old speed settings, I can see how this might be a bad thing in terms of potential filament grinding. (Especially as, for me, a true 48mm/s seems marginal in terms of physical operation of the extruder).

On the other hand, it does now give us the option to accurately set a meaningful retraction speed in the full 12-48mm/s range (and I'm not convinced that this bug couldn't have affected the other axes as well in some circumstances - the fact that it didn't was more by luck of acceleration settings, steps-per-mm and typical operating speeds, than by design).

 

[Daid 306]

Small note, if the retraction speed is corrected because of this, the sanest thing to do is limit the E speed in the firmware to 20-25mm/s instead of the current default limit of 45mm/s

 

[illuminarti 18]

Yes... changing the default limit down to 25 would at least make sure that nothing too outrageous happens for existing files or settings. I can't imagine anyone really is running currently at 48mm/s retraction - which requires good luck, and a real setting somewhere over 50 - for me at least it isn't reliable.

If people do see a genuine use case for retraction speeds in the 25-48mm range, then they can override the limit easily with the UltiController, or (less easily) with the equivalent gcodes.

It should be noted that actually any retraction speed over 12 (not 25 as I originally said) will happen twice as fast as before. However, asking for and getting 24mm/s (instead of the 12mm/s you're used to) probably isn't going to be catastrophic.

 

[calinb 11]

I just built the latest V1 from Daid's tree. Unfortunately, netfabb finally fixed their Ultimaker Engine retraction bug and it is now my favorite slicer by far, but I can't get it to work with the latest Marlin V1 containing this Marlin bug fix. I seriously doubt netfabb will "fix" their program again to conform to these significant Marlin changes. What--it took them something like a year to fix their retraction bug?

Given that the push-on and pull-off speeds are identical (the default in netfabb), I simply tried cutting both equal speeds in half. This resulted in what appeared to be a somewhat more aggressive retract (pull-off) than before I uploaded the Marlin fix but, more significantly, there was virtually no push-on. Next I cut the rates to 24% of their defaults. Same thing--retraction speed was reduced, it seemed, but there their was still only an insignificant push-on. It doesn't take long for the printing to stop, because the net retraction exceeds the net push-on.

Can we get a #define USE_OLD_RETRACTION or will I forever have to use test2 Marlin (or figure out the source code myself)?

Thanks,

-Cal

I also tried changing the pull-off rate only (to 24% of its original default value) while leaving the push-on setting at the default. The amount of filament pushed and pulled is set to 2.88 mm, BTW. Nonetheless, Marlin now results in far more pull-off than push-on.

 

[Daid 306]

Keeping in a bug to keep broken features? Sounds like http://xkcd.com/1172/

Seriously, NetFabb didn't get their stuff right with retraction. Just ask Paul Candler.

But the fix is to lower the maximum E speed to 20mm/s in the firmware. Quite easy and you should do that anyhow if you use other software which currently has set the traction speed too high.

 

[calinb 11]

Well, Daid...the company that employs you sold netfabb to me. I'll accept your point of view if they'll refund my money for netfabb!

Nonetheless, there are countless examples of "legacy" modes in the history of software that were retained to cover situations very much like this one.

I'll try to find and change the maximum E speed, but it will limit my maximum forward E-speed as well, right (not terribly desirable)?

 

[calinb 11]

A 20 mm / sec limit does work with netfabb. Retraction seems to be a bit slower, but it should function. 20 mm / sec is not all that fast with a large nozzle, however (for forward extrusion). Maybe netfabb will correct their software again, but they finally did fix their retraction bug (while using bugged Marlin) and it was working well for me.

Thanks,

-Cal

 

[calinb 11]

A timely response from Alexander:

Hi Calin,

thank you for the request. The behaviour of netfabb is not a bug, it is a workaround to enable the combination of movement of the axis with retraction - a feature which Marlin does unfortunatelly not support.

If the behaviour of it has recently been patched, I will have to look at another solution. Probably just do it like the other slicers and not move while retracting.

I will keep you up to date.

Best wishes,

Alexander

 

[Daid 306]

On my home machine the max that the feeder seems to do is 27mm/s, so you can turn it up more then 20. 20 is just a safe start point.

 

[illuminarti 18]

I'm not entirely sure why you'd want to move and retract at the same time (I presume they mean retract at a rate faster than you are moving, so the retraction finishes first. Although with a fast move, I would have thought one could still do both reasonably well). I'm also not clear why the Marlin retraction fix would break anything; unless Netfabb is hard-coding a very high speed into the retraction.

Limiting the extruder speed to something sane - personally, 30 seems to work fine for me - won't affect anything other than retraction speeds. In the normal course of printing - even with a large nozzle - you aren't advancing the extruder more than about 5mm/s ( = 32mm³/s extrusion volume with '3mm' filament).

 

[illuminarti 18]

Re-reading, above with a bit more coffee, I see you saying that you can specify speeds for the retraction - so I'm really not sure why the Marlin retraction speed fix would affect this at all. What speeds are you typically specifying in there?

The fix that was applied doesn't specifically relate to Netfabb's issue with 'move and retract at the same time' (at least to the best I can understand what that might mean) it is merely making sure that the speed at which it moves is the correct one.

Calin - could you perhaps send me some gcode that demonstrates the problem? I'd like to better understand what Netfabb is trying to do.

 

[calinb 11]

Thanks, Daid. I bumped the limit up to 25 mm / sec and it seems to be working on a short test.

Illuminarti, Yes--nf seems to retract at a high speed but I've found that the user "pull-back speed" setting (RPM) doesn't work in nf. Changing the setting does not result in a change in the gcode produced. The F value in the gcode remains 10800, regardless of the setting in the nf "Manage Materials" >> "Speed and jump settings" tab:

 

Disabling this setting may be at least partially what Alexander referred to as a workaround.

G1 X179.4590 Y17.8500 Z0.0400 F1200.0000 E14.3443

(retract -2.88)

G1 X179.7556 Y19.5524 Z0.0400 F10800.0000 E11.4643

G1 X182.1463 Y33.2764 Z0.0400 F10800.0000 E11.4643

(retract 2.88)

G1 X182.1465 Y33.3587 Z0.0400 F1200.0000 E14.3454

 

[illuminarti 18]

Calin -

It's a little difficult to tell without looking at a longer section of code, but this gcode looks to be broken to me - irrespective of the Marlin fix.

What is happening here is that the retract happens as part of the fast move on the third line of your code snippet... F10800 = 180mm/s. So in fact, the move will be limited to be the slowest max speed of any of the axes that is moving - typically the extruder. So it will happen at whatever VmaxE is set to.

Then on the next line it does another fast move... presumably to wherever it wants to start printing again. The extruder position doesn't change; so the move happens at 180mm/s or the lower of VmxX or VmaxY as set in your firmware.

Then on the final line it starts printing again - moving at a slower speed this time - 20mm/s. But what is strange is that it never undoes the retraction before it starts printing. The full amount of retraction gets undone over the course of this first line segment,plus an extra 0.0011mm of filament. Now it's only a short move - 0.08mm - so that net extrusion of 0.0011 mm is the right amount of plastic for the whole of that segment, assuming a 0.4mm nozzle and 0.2mm layer height. But that should be happening over the length of the first segment equally - whereas this gcode is going to dump that little bit of extra plastic at the end of the move (because for most of the move, it's going to be busy unwinding the retraction, and not actually extruding anything.) Since this is a very short move, it probably doesn't make much difference - maybe a slight blob - but if it was a longer first segment, you'd be more likely to get a noticeable under-then-over extrusion pattern

Maybe Netfabb is doing something to ensure that this de-retract move is always very short, but I can't see it's ever going to be good to have gcode that starts printing before fully undoing the retraction.

I'm also not really seeing why the speed fix in Marlin would make a difference, except to the extent that the above code would effectively have been limited to retracting at (probably) half the VmaxE speed you had set in your firmware. With the fix, it's going to operate at the full VmaxE speed - and that may simply be too much for the motor to deliver if your VmaxE is too high.

So, I think there are three takeaways here....

1) With or without the Marlin Retraction Fix, Netfabb is relying on your retraction speed being limited by a reasonable VmaxE setting in the firmware.

2) Netfabb's de-retraction code may be broken (at least the code you provided suggests it may be), because it de-retracts during the first print move, not before it.

3) With the Marlin fix in place, you especially need to make sure that your VmaxE setting is reasonable, because you'll no longer be protected by broken firmware that usually only delivered half (or a quarter) of the VmaxE that you had set.

 

Thanks, Daid. I bumped the limit up to 25 mm / sec and it seems to be working on a short test.

Illuminarti, Yes--nf seems to retract at a high speed but I've found that the user "pull-back speed" setting (RPM) doesn't work in nf. Changing the setting does not result in a change in the gcode produced. The F value in the gcode remains 10800, regardless of the setting in the nf "Manage Materials" >> "Speed and jump settings" tab:

 

Disabling this setting may be at least partially what Alexander referred to as a workaround.

G1 X179.4590 Y17.8500 Z0.0400 F1200.0000 E14.3443

(retract -2.88)

G1 X179.7556 Y19.5524 Z0.0400 F10800.0000 E11.4643

G1 X182.1463 Y33.2764 Z0.0400 F10800.0000 E11.4643

(retract 2.88)

G1 X182.1465 Y33.3587 Z0.0400 F1200.0000 E14.3454

 

[Daid 306]

Well, as I understood. The NetFabb people are pretty much blaming the firmware for not working correctly, and the UM users are blaming NetFabb for not implementing retraction correctly.

The truth is. They are both right, from their own perspective. But nobody really sat down and tried to really understand what the problem is.

NetFabb is trying to retract while moving fast. For this to work, you would need a "proper" "G0" command in the firmware. G0 should be "fast move on all axes" where 1 axis can finish before another. So the retraction could finish before the X/Y move. However the firmware does not implement this and just does a G0 the same as a G1. Now this G0 behavior is not really properly defined anywhere, as wikipedia says that it could also interpolate the move (as Marlin does)

Now, so NetFabb is somewhat right that the G0 is not implemented correctly for their use (just saying "the firmware is bugged" doesn't really help here)

But the users are correct in that NetFabb is trying to do something with is not possible to do properly with the firmware. And the work-around is easy, just retract in place as everyone else does.

The thanks for getting to the truth of this problem should go to Paul, as he explained me what NetFabb was trying to do in GCode. And that the NetFabb people where complaining about a bug in the firmware. With my own knowledge I could fill in the blanks and figure out what the real issue is.

 

[illuminarti 18]

While it's not an ideal solution, I don't think there's anything necessarily wrong with doing 'move and retract at the same time' (even without a proper G0) provided that the speed limit set on all the axes is correct.

But I do think that doing 'print and de-retract at the same time' (as Calin's sample code seems to suggest) is NEVER the right solution. With or without a G0, I can't see how that is ever going to work. The detract needs to happen quickly, while the print part needs to coordinate with the X-Y movement. How can they be combined into one movement?

Finally, since Netfabb are selling (through UM) an Ultimaker-specific version of Netfabb, shouldn't it do whatever Ultimakers do? Saying that UM's do it wrong is totally not a viable defense in that case.

 

Well, as I understood. The NetFabb people are pretty much blaming the firmware for not working correctly, and the UM users are blaming NetFabb for not implementing retraction correctly.

...

NetFabb is trying to retract while moving fast. For this to work, you would need a "proper" "G0" command in the firmware.

...

The thanks for getting to the truth of this problem should go to Paul, as he explained me what NetFabb was trying to do in GCode. And that the NetFabb people where complaining about a bug in the firmware. With my own knowledge I could fill in the blanks and figure out what the real issue is.

 

[calinb 11]

But I do think that doing 'print and de-retract at the same time' (as Calin's sample code seems to suggest) is NEVER the right solution. With or without a G0, I can't see how that is ever going to work. The detract needs to happen quickly, while the print part needs to coordinate with the X-Y movement. How can they be combined into one movement?

 

The last line of the code is for a very short X-Y movement with detract ("push-on" in nf-speak), including a very short positive net E (ending very slightly beyond the filament position before the retract). It is always very short like this. I don't know their motivation for combining the detract with a very short X-Y move, but it doesn't really constitute a print motion. Overall, it actually does work!

 

[calinb 11]

With my own knowledge I could fill in the blanks and figure out what the real issue is.

 

And thanks for explaining the situation, Daid. Your summary seems to fit the landscape! :(

 

[illuminarti 18]

It's good that it's always a short move, because otherwise it would be a disaster. :-) Because it really *is* a print motion - if you do the math, that 'short positive net E' is exactly the right amount of plastic to print that segment at a 0.2mm layer height. So it looks like its intended to be a part of the print. And all of that plastic is going to get dumped at the very end of the move. Now, it probably doesn't matter much in practice, but I can't think of any reason for doing it this way. It doesn't seem to be a work around for anything... I think it's just a bug. It's one thing to make a design decision to retract whilst doing a travel move... but to go in the opposite direction during a print move is only going to impact print quality negatively.

 

The last line of the code is for a very short X-Y movement with detract ("push-on" in nf-speak), including a very short positive net E (ending very slightly beyond the filament position before the retract). It is always very short like this. I don't know their motivation for combining the detract with a very short X-Y move, but it doesn't really constitute a print motion. Overall, it actually does work!

 

 

 

[snowygrouch 1]

Very glad I found this topic, because I updated my firmware from the latest from Github (then modified it so the

2nd extruder worked), but now find that the retraction performance in Kisslicer is now RUBBISH.

I thought I was going nuts an imagining it for a while, spent ages going over print temps, print speeds and

slicer settings....

However, can someone confirm or not if there is anything I can mod in the firmware that will restore my

retraction performance ?

Or do I need to put the old firmware back in, and wait for a fix ?

C.

 

[calinb 11]

I've not used Kisslicer in a while but, as Daid and Illuminarti suggested, I found that changing the following line in Configuration.h from

//#define DEFAULT_MAX_FEEDRATE {500, 500, 5, 45} // (mm/sec) default settings

to

#define DEFAULT_MAX_FEEDRATE {500, 500, 5, 25} // (mm/sec) post retraction bug E limit

limits the default netfabb retraction speeds (which work quite well) to a sane level that approximates nf's performance under the old firmware. While the nf linear retraction user settings ("push-on" and "pull-off") are functional , the nf user settings to change the retraction speed appear to have no effect on the gcode. Therefore it's necessary to limit the speed in the firmware, as it is necessary in order to run old gcode too. I've filed a ticket with netfabb about the apparently non-functional speed settings.

 

Very glad I found this topic, because I updated my firmware from the latest from Github (then modified it so the

2nd extruder worked), but now find that the retraction performance in Kisslicer is now RUBBISH.

I thought I was going nuts an imagining it for a while, spent ages going over print temps, print speeds and

slicer settings....

However, can someone confirm or not if there is anything I can mod in the firmware that will restore my

retraction performance ?

Or do I need to put the old firmware back in, and wait for a fix ?

C.

 

[illuminarti 18]

Yes, you can change the max E speed in the configuration file, or alternatively if you have an ulticontroller, change the 'Vmax E' setting to 25, and then gradually step that up a bit, and see how retraction works. I find that 30mm/s and a retraction distance of 4.5mm works well.

If you don't have an ulticontroller, you can use M203 to set the maximum E speed, as part of your start gcode.

 

[calinb 11]

It will take me awhile to lose my inherent mistrust of all Ulticontroller menu settings, because they have been so buggy in the past. However, your Marlin_Menu_Fixes code tree appears to be the best ever, Illuminarti.

Thanks!

 

Yes, you can change the max E speed in the configuration file, or alternatively if you have an ulticontroller, change the 'Vmax E' setting to 25, and then gradually step that up a bit, and see how retraction works.