Post processing plugin

[nallath 1,120]

A quick update for those of you who want the tweak at z, pause at height and others back. I've been working on a plugin that allows for the loading & execution of such scripts.

The layout is still rubbish, localisation doesn't work correctly and it hasn't been fully tested, but I still want to share this with you guys. You can find the plugin here; https://github.com/nallath/PostProcessingPlugin . See this as an early alpha / beta.

If you want to make your own script, have a look at the examplescript (that does nothing) that is in the scripts folder. The plugin should automatically load all scripts that it finds in that folder.

Edited July 29, 2015 by Guest

[rigs 735]

404 nallath

Edited July 29, 2015 by Guest

[nallath 1,120]

Ugh. It was the dot mistake again. It added the dot to the link. Fixed it now.

Also @dim3nsioneer I believe you were waiting for this plugin

Edited July 29, 2015 by Guest

[rigs 735]

Ok thx

[Dim3nsioneer 557]

Also @dim3nsioneer I believe you were waiting for this plugin

 

Yes, indeed. Thank you. I'll first have to understand it and then do the next steps... it might take some time as I have a few other task to look after...

[nallath 1,120]

All right I've added a pause at height script to it as well. Now it actually does something

[berndjm 16]

Maybe can't see the forest for the trees at the moment

Were should the files and the script-folder be located ? (Ubuntu)

[nallath 1,120]

You need to put the plugin in the plugins folder. The script folder is inside the PostProcessingPlugin folder

[berndjm 16]

You need to put the plugin in the plugins folder. The script folder is inside the PostProcessingPlugin folder

 

O.k. I got it, it should be located in :

/home/username/.local/share/cura/plugins

(the folder "plugins" must be generated by hand - would be nice to establish this folder during installation to give the user a better hint )

But what's that :angry:

 

ERROR - Import error loading module PostProcessingPlugin: No module named 'PostProcessingPlugin.scripts.Test'

 

Commented out the problematic line in "PostProcessingPlugin.py" and it seems to work

O.k. let's not talk about the optic aspects, the usability and the function ... you mentioned in your starting post "See this as an early alpha / beta."

But it gives a good idea how things could work ...

[berndjm 16]

Jetzt grad mal auf deutsch (weil ich echt genervt bin):

diese Forums-Software ist doch einfach für die Katz!

Warum glaubt die Software ausgerechnet in Code-Fragmenten wie meinem obigen rumeditieren zu müßen und da irgendwelche Geschichten reinzuschreiben !!!

Die Fehlermeldung heißt:

ERROR - Import error loading module PostProcessingPlugin: No module named 'PostProcessingPlugin.scripts.Test'

... toll im normalen Text wird nicht "autokorrigiert" ...

[nallath 1,120]

Commented out the problematic line in "PostProcessingPlugin.py" and it seems to work

 

Whoops. I've updated the code. Should work for everyone now. If you feel like changing the GUI, feel free to do so and create a pull request. I'm going to be away next week (holiday in Germany actually) so I won't be able to look at it next week though.

 

Jetzt grad mal auf deutsch (weil ich echt genervt bin):

 

German is so much better for raging

[Labern 774]

@nallath what is the best way to share plugins that we have created?

[ahoeben 1,948]

@nallath responds well the "pull requests" on github. And to cookies.

[nallath 1,120]

@nallath responds well the "pull requests" on github. And to cookies.

Someone is going to get some referral bonus if I get some cookies

[Megahurtz 1]

Good day.  Dredging up an old thread here...

Is a Post Processing Script an appropriate way to have an external binary process a sliced gcode file, before it gets handed off for saving, printing, or sending?

 

There is a new utility binary (ArcWelder: https://github.com/FormerLurker/ArcWelderLib) that takes a .gcode file full of G0/G1 commands and generates G2/G3 LH & RH arc/circle based replacements.  This often has the benefit of improving print quality by making a bunch of G0/G1 commands into a single G2/G3 commands which often serves to decrease gcode size for a given model, removes many potentials for printer stalls due to FW buffer dynamics, and potentially shortens print time.

One can script it using external tools outside of Cura, of course, but that serves to muck up the workflow.  It would be nice to be able to just add a final post processing step and have Cura kickoff the postprocessor (ArcWelder) binary to scan and convert the gcode, prior to the final step of Printing, Saving, Transmitting the slicer generated gcode.

 

Please advise.  Thanks.

[gr5 2,172]

I believe that most printers don't accept G2 and G3 commands.  Right?  Or am I wrong.  I don't think Ultimaker printers do but I'm not 100% certain.  I've looked at much of the Marlin code and even if it does implement these I think it would most likely immediately turn them into line segments anyway.

 

Postprocessor examples are easy to find and they are in python which it seems to me would be easy enough to then just call some executable.  So at least your post processing extension/plugin idea seems reasonable.

[ahoeben 1,948]

4 hours ago, Megahurtz said:

Is a Post Processing Script an appropriate way to have an external binary process a sliced gcode file, before it gets handed off for saving, printing, or sending?

Not really.

 

Though running a post processing process on the gcode is exactly what the Postprocessing Script is for, the architecture of the Post Processing plugin is not ideally suited for how ArcWelder would like to process the gcode. ArcWelder wants to read a file and create a new processed file. At the point the Post Processing plugin runs, there is not yet a file to hand to ArcWelder; only gcode in memory, divided by layer. You could write the gcode into files for postprocessing, and read the result back in, and then finally write the result back out as a final file, but you can see how that is suboptimal.

 

I think this is more suitably handled by a bespoke plugin that runs after all other postprocessing tasks have been run.

 

That has the added benefit of being able to distribute said plugin via the Marketplace, including the required ArcWelder binaries for several platforms (Windows, OSX, Linux, Linux ARM). There is no way to distribute a postprocessing script other than "put this file in exactly this place, and this binary in this other place".

 

I'll try this approach the coming week.

[GregValiant 1,251]

"You can't have your PI and eat it too."

Some musings in regards to gcode and radii.

 

From MegaHurtz -

"This often has the benefit of improving print quality by making a bunch of G0/G1 commands into a single G2/G3 commands which often serves to decrease gcode size for a given model, removes many potentials for printer stalls due to FW buffer dynamics, and potentially shortens print time.

 

From the MarlinFW site regarding G2-G3 ...

"Arc moves actually generate several short straight-line moves, the length of which are determined by the configuration option MM_PER_ARC_SEGMENT (default 1mm). Any change in the Z position is linearly interpolated over the whole arc."

 

I wrote a little Windows App for Accel and Jerk Calibration and it uses G2 and G3 commands for circles since they can be adjusted on the fly (using the X and Y of the bed size as the variables).  As MegaHurtz points out, I only have to transmit a couple of lines of Gcode over the USB instead of whole paragraphs.  Accel and Jerk (and I assume "Junction Deviation") have a large influence on the speed around a circle.  It would seem that any impact on print time would be largely influenced by those other settings vs. just a translation of G1-G0 moves to G2-G3 moves.

 

At the end of the day it's all about PI.  The segments can get shorter.  The path gets closer to being an exact circle.  It can never quite get there.  As the segments get shorter, the number required Gcode moves gets larger whether it's in the Gcode file as a lot of G1 lines or if the PrinterPlanner is doing it on the fly in response to G2-G3.  A large radius circle described by a G2/G3 command takes a long time for the printer to complete.  Gcode "transmission" comes to a halt while the printer deals with that single line of code.  I have to use the USB (the program doesn't "print" it just runs around) so it was necessary to pause transmission to keep from over-running the buffer.

 

MM_PER_ARC_SEGMENT. The default is stated as 1mm in Marlin.  Do printer manufacturers change it for their printers?  Does it mean that a circle of radius 0.63661977 would be a square with 1mm sides?  Maybe not, but this setting would seem to impose its own limit on print resolution by taking it out of the slicer's hands.  When Cura slices a high resolution model file and then it gets translated back to G2-G3 would it lose resolution because of this setting?

 

Archimedes was a very bright man.  Here we are 2300 years later and he is still impacting what we do.

[gr5 2,172]

20 hours ago, GregValiant said:

"Arc moves actually generate several short straight-line moves, the length of which are determined by the configuration option MM_PER_ARC_SEGMENT (default 1mm). Any change in the Z position is linearly interpolated over the whole arc."

Yes.  This is the problem.  They way Marlin works is a hack and it's also very elegant at the same time.  I've been though the path planning code dozens of times to understand all the intricacies.  The people/person who wrote the path planner is really smart and did a reasonable job.  But he/she optimized the code for line segments.  Not arcs.  The handling of arcs is a hack/compromise.  In summary:

 

G02 G03 commands are going to give you worse quality than G0/G1

 

Marlin has a buffer of moves (one move per G1 command but many moves per G2 command).  That buffer only holds 16 moves for most arduino processors.  Some printers out there only have 8 moves in the buffer - yikes.  Marlin is processing new moves/gcodes while it's printing and doesn't know if the next gcode is a sharp corner or a stop (every time you do a retraction, Cura has the printer stop, retract, then move again).  So the path planner can't build up too high of a speed because 16 moves in the future it may have to come to a stop.  The path planner is constantly going backwards through the buffer and figuring out what the max speed is a the end of every move so it can stop within the buffered moves.  Every time it adds a new move at the end of the buffer it re-evaluates to try to move faster on the very next move to print.

 

If you have too many moves in a short arc (e.g. > 16) the printer slows down to a crawl.  It sucks for the quality of the print (ugly bumps and squirts) and it sucks because things slow down.

 

The computer running Cura and CAD (your computer) is thousands of times faster than an Arduino and does a better job of things like converting a curve in CAD into triangles.  The CAD people hopefully did a good job.  Much better than the Marlin people.  Cura turns the triangles into line segments in a slice.  It also sometimes throws out super short lines.  It's also smarter than Marlin. 

 

Best not to have any G02 commands!

 

I've looked at several path planners: Marlin, Repetier, Redeem.  And I've rewritten much of the Redeem path planner.  They are all copied from Marlin.  They are all 99% the same functionality and even much the same code.  I've also looked at the path planner in TinyG - that is completely different.  Written by a different guy who thinks differently.  Very original ideas there.  But pretty much no printers use TinyG - that's mostly for CNC routers and pretty much any XYZ machine that isn't a 3d printer (I guess it's an industry thing?).  I think only printrbot used TinyG and then soon after went out of business.

[GregValiant 1,251]

About the time the Pentium came out I came across an article on the total production numbers of the Intel 286 chip.  They were huge, so being the curious guy I am, I looked around at what they were being installed in (since it wasn't computers anymore).  It turned out that the #1 use of 286 chips was dishwashers.

Jump forward 20 years.

A short time ago I bought a new clothes washer.  Why is there a USB port on the back of my clothes washer?  Does it need an ethernet line? Bluetooth?  Will Alexa wash my clothes?  Does it want to discuss grass stains? mud? the meaning of life in a small room?  Does it run on a 286 or (more likely) some new and improved super chip?  It does exactly the same job that my mother's wringer washer did in the 1950's...just not as well.

 

Lately there has been a movement to 32 bit boards in 3d printers.  Now I do have a fairly wide skill set and I freely admit that this is outside of it but...

Is the problem bandwidth or processing time?  If the architecture of a chipset/mainboard is going to be changed, why not move to a more capable chip?  Is it like GCODE where the installed base is so huge that even small changes lead to tsunamis as the ripple moves through the various industries that utilize it?  I'm sure there is no consensus amongst printer manufacturers and that would seem to be a requirement for a clean break with the past.

 

It just seems that a 3D printer should be more capable than my clothes washer and dishwasher.  What is needed is a Resolution Revolution.  You still couldn't print a real circle whether with G1 or G2-G3 (HA - and you folks thought I went off topic!), but it would seem to be a positive thing if the mental bits of the machine could keep up with the mechanical bits.

[gr5 2,172]

Marlin has a configuration where you can set the look ahead planner buffer size.  Most printers have this set to 8 or 16.  This works for the arduino mega 2560 which has so damn little ram (but lots of eeprom) and is a bit slow.  The calculations a planner is a bit heavy but not too bad.  A few square roots for each path.  These chips are probably something like $1.

 

With a 32 bit 1 GHz $10 chip you can easily do 1000 paths in the lookup path (I know - redeem does on a beaglebone black).  Some printers have gone this route.  Unfortunately most of these 32 bit chips come with Linux which (very rarely) can slow down the cpu enough to mess up a print if some process slows down the computer too much.  I think this is why the S3/S5 ultimaker printers still use a mega2560 for the actual path planner and a second 32 bit linux computer for things like wifi and controlling the touch screen.

 

If you have one of these 32 bit controllers for your printer you can probably perform G2/G3 with 0.1mm resolution just fine at full speed.

 

[GregValiant 1,251]

"With a 32 bit 1 GHz $10 chip you can easily do 1000 paths in the lookup path".  Thanks for that gr5.

 

My buffer holds 8 moves.  At boot-up the machine shows the size as 1232.  I can probably safely assume that it is not terabytes.  Dealing with it reminds me of having to hand load drivers in Windows 3.1 into DOS "high" memory.  There was never enough room.

 

One of my mantras is "Never upgrade an operating system".  It's a gimme that something won't get the memo and it will take days to figure out what the problem is.  I'll stick with the 8 bit system in this 2 bit printer (that's an American colloquialism for "cheap Chinese POS").  I'm coming to understand it and it's limitations.  I'm not trying to run production or build parts for a moon rocket.  If I was to get a chipset like that, I'd want the surrounding hardware to be up to the task.  The word "robust" comes to mind.  Ultimaker printers look robust.

It would absolutely not be the printer that is sitting here in front of me.

[ahoeben 1,948]

This is getting "a bit" offtopic. The realworld benefits of converting multiple G0/G1 movements into fitted G2/G3 arcs are being explored and tested here: https://community.octoprint.org/t/new-plugin-anti-stutter-need-testers/18077

 

The creator of the original plugin for OctoPrint has also created a command-line tool to postprocess gcode. As I have said before, the post processing plugin is not ideally suited to launch external binaries, so I am working on a plugin. That plugin can be discussed here:

 

 

 

[kupfeli 9]

What I would like to see in Cura is a change in the Post-processing plugin, that it checks whether a given script is supported by a given Ultimaker printer. As Cura is an Ultimaker thing, it is perfectly fine to only check for Ultimaker compatibility.
Now the case is that almost none of the Post-processing steps work on my Ultimaker 2+ Connect for instance, so that change would be cool 🙂

 

Also, I would recommend people NOT to use post-processing scripts at all, or only use it when you know what you are doing. Maybe make it an Expert only section for instance, that you can only access when you start Cura in Expert mode?

[GregValiant 1,251]

"Also, I would recommend people NOT to use post-processing scripts at all, or only use it when you know what you are doing. Maybe make it an Expert only section for instance, that you can only access when you start Cura in Expert mode?"

 

Ouch!  Probably 90% of what I've learned about Cura has come from experimenting with things.  It's how I went from not knowing anything to knowing just enough to be dangerous.

There are visibility settings that limit a users view, but not their access, to commands.  A user can select from 4 levels of settings visibility.

I have two post-processors that I almost never shut-off.  By experimenting with the ".py" files I was able to alter them just a bit to make them more in tune with what I need.  My personal PauseAtHeight is configured to fill in all the boxes with MY defaults rather than Cura's generic defaults.

I don't know much about Ultimaker printers but I'm fairly familiar with the Chinese crap (excuse me) printers.  Some don't support G2/G3 and some do.  Some don't support M0, or M1 but some do.  As far as I know most printers do not support a command that would query the printer and inform a user regarding exactly which G and M commands are actually enabled in the firmware.