Need a little help with a post process script

[FredFish 4]

I set out this weekend to generate a post process script in Cura that would allow me to draw with a pen or marker between transparent layers, originally printing embedded traces from a PCB design output as Gerber data code (rs274x type), but it turned into making imbedded logos as well, this first part I think works for me, and is better than exporting editing and importing the code, it’s not the question, I included the script to illustrate the issue.

 

I set up Cura with a custom profile having 3 separate tools, the extruder and 2 pens, and wrote some of the very redundant code. Realizing I was coping over most of the original settings from my printer (Creality Ender 3 Pro) before ever running the script just to get it to re-read all the variables, because the Custom printer selection loads directly from the fdmprinter.py profile, bypassing all the Creality settings.

 

Soooo, I tried to add a section to my script to allow for when the printer only has one print head (the default in my case). The script reads through the data until it finds the correct end of layer section. The pen loads (needed load pen sequence), prints (using converted Gerber data), and unloads (needed unload pen sequence) using the g-code being added by the script. This is where I ran into questions.

 

My question may be with script window formatting under PostProcessingPlugin.qml itself, in order to include the pen load and unload g-code I need to enter several lines of code like is done in the machine setting window to set tool start and end, using the default single entry type line, I end up with hard to enter \n in a long code line, not the newlines that are needed, these are then transposed to be left as is  in the PostProcessingPlugin.py section, any idea on how to get a multi-line window in the script section, or reformat the string for multiple lines without the need for \n?

 

 

ff_InsertGerberAtLayerChange.zip

[ahoeben 1,951]

24 minutes ago, FredFish said:

any idea on how to get a multi-line window in the script section

This is not currently supported in the Post Processing plugin.

 

25 minutes ago, FredFish said:

or reformat the string for multiple lines without the need for \n?

You can use any string you want, and replace it with a newline in your postprocessing code. However it will still be in a single line in the interface.

[FredFish 4]

Thanks for your response. I think I get it now,

 

I may have been overthinking the issue of using the input line as the editor.

 

Sooo, the input window size cannot change without modifying the QML code (To much work for now), but if I cut and paste from an editor (like word or notepad) I can paste multiple lines simultaneously into the input window, I just cannot see the line brakes or do much in the way of editing the lines other than seethe single string without newlines (although the newlines are there when I use the inputted data), so no need to add any extra “\n” at the end of each line, is this correct?

[Slashee_the_Cow 271]

6 hours ago, FredFish said:

Sooo, the input window size cannot change without modifying the QML code (To much work for now), but if I cut and paste from an editor (like word or notepad) I can paste multiple lines simultaneously into the input window, I just cannot see the line brakes or do much in the way of editing the lines other than seethe single string without newlines (although the newlines are there when I use the inputted data), so no need to add any extra “\n” at the end of each line, is this correct?

No. You cannot paste multiline content into a text box designed for single line content. The best workaround I can think of off the top of my head (if you do\n't want to deal with \newlines) is to add extra settings fields to your script.

[FredFish 4]

Down the rabbit hole we must go.

 

In diving into this anomaly with my win7 laptop running Cura4.12 it seems that I can indeed cut from the Machine Settings window, both the start and stop codes, and past the multiple lines into the single input line boxes of the script, although you cannot see the line brakes, the output g-code has the multiple lines of code included. So although a little odd, this seems to work for me but limits any new Cura options.

 

 However when I attempted to do the same on my win10 laptop running Cura5.4 this is not the case, attempting to paste more than a single line into the single input line box, the paste option seems to be blocked, with only visible characters allowed.

 

 Using an external editor to include swappable characters just to reinsert them, seems to defeat the purpose of running the process as a script and not just running the whole thing externally. Is this a case of upgrade into failure just to keep me safe, or is there a way to modify the new input line permissions, or even popup the extruder tool menu like interface window from within the script?

[Slashee_the_Cow 271]

6 hours ago, FredFish said:

Is this a case of upgrade into failure just to keep me safe, or is there a way to modify the new input line permissions, or even popup the extruder tool menu like interface window from within the script?

Could just be a case of "newest version of Qt by the time 5.4 came out disabled that behaviour" for all I know. But no, the post processing plugin interface only allows you to use the default interface elements for each setting type. If you want to use a different interface element, you'll have to create your own plugin, not just a post-processing script.

 

Also worth noting is that by default post processing scripts will not convert \n in a string input to a new line. To split into what will be multiple lines in the g-code, you need to replace any raw \n with a newline character.

string_setting_raw = self.getSettingValueByKey("test_string")
string_setting = string_setting_raw.replace(r"\n", "\n")

[FredFish 4]

I'm standing on the edge looking down, but still haven’t found what I’m looking for,

 

I changed the input to allow for 4 types of input, I find I still have to set up a separate format for each type of input variable. (See updated script)

 

the back end tool setup allows for picking up preset variables from existing stacks when used, but does not replace the variable names from within the strings retrieved, indicating that replacement of these is done after the run processing but before scripting. Any idea  about how to run this after scripting without having to swap each one out one at a time (repeating this procedure)?  Something like reload as g-code but internally, this would also allow you to see all the changes made by the script in the preview window as well.

 

the input file option, allows for automatic conversion of  \r\n newlines to \n, when the file is loaded into a variable, but the input as a string  still won’t allow for multiple lines only \n, and as a side note won’t allow for three consecutive blank lines \n\n\n to be added, what is this restriction for?, is there another input option type in these python scripts that allow for multiline input? I am trying to stay within the boundaries but looking for other options.

 

it just seems like I’m missing something  with all the upgrades, advice would be appreciated.

 

Current script update attached for reference,

to be posted with Pen holder on thingiverse when stable.

 

ff_InsertGerberAtLayerChange.zip

[Slashee_the_Cow 271]

Debugging post-processing scripts sucks because the only way (unless you can make Cura crash entirely) is by going the log file in Cura's configuration folder (in Windows it's %APPDATA%/cura/<version>/cura.log) and search for your script's name.

 

• In my experience I've found that Cura heavily restricts (or blocks entirely) file I/O. Although I was more trying output, trying to write my own log files for diagnostic reasons. Reading might work better.
• As for newlines - did you try my suggestion and replace raw "\n"s with regular "\n"s in whatever strings you're loading?
• Variables in external files won't be mapped to internal variables, it's a safety measure. It might but probably won't work if you run it as an f-string:

  f"do something to {variable_name_here} now"

• The only reliable way to run input from external files or strings and have it processed as code is by running eval() on it - this can be incredibly dangerous if any malicious code somehow gets in, so only use it if you trust your inputs::

  codestring = "print('Slashee is a cow!')"
  
  eval(codestring)
  
  Slashee is a cow!

  However there are limits to eval - it will only run expressions, not statements (and things like assignments are statements):

  codestring = 'cow = "milk"' # Remember to mix different kinds of quotes inside your string to the quotes bordering it, or else you'll have to escape them or make a string literal like """cow = "milk" """ - that space between the "milk" and the """ is important because it won't recognise """" as the end of a string literal
  eval(codestring)
  Traceback (most recent call last):
    File "<stdin>", line 1, in <module>
    File "<string>", line 1
      cow = "milk"
          ^
  SyntaxError: invalid syntax

• For the third time, no, Cura does not have a multiline input option for post-processing scripts. There is no way to get a multi-line input in the post-processing script settings window. You can make a feature request if you want in the Cura Github repo.

 

Edited October 15, 2023 by Slashee_the_Cow

[FredFish 4]

Thanks for your patience with my repetition. I am utterly frustrated with fences with no gates that I run into, I understand it is good practice to keep the nefarious file parsers at bay, but do not know why it should limit everyone else.

 

the terse issue listings in the log file are why I posted the question here, and I also found the need for additional output when debugging (I had added a notes option throughout  the script, to include information to the output file as comments)  

 

The first suggestion using replace did not initially work, because in the newer cura version, once input was converted to a string, changed the string \n to \\\n  (but not  in the  older version that allowed \r\n), I now see going back over it, this was a script sequence error, I needed to do replace before making the input line a string, thanks for that, I get it now (pounding head on fence post).

 

Sorry for wining about the multiline input. I have been searching the Cura files for the place where the input sequences from the machine manager window, swaps out the variants for the values. I could see that allowing this globally with eval() could cause unintended consequences.  What file has the setting that limits this transaction to local() within the machine manager, so I can get an idea of how this is done?

 

The three consecutive newlines being blocked issue is still a mystery, but not relevant here for now, I can live with it, just don’t know why it exists.

 

Finally let me say thank you again for replying to my questions, this project started as a weekend wish list, of just using scriptable options instead of using external programs and jumping back and forth between them, I do appreciate the help.

[Slashee_the_Cow 271]

6 hours ago, FredFish said:

Finally let me say thank you again for replying to my questions, this project started as a weekend wish list, of just using scriptable options instead of using external programs and jumping back and forth between them, I do appreciate the help.

Ah, scope creep. We've all done it. Sorry if I come across as a bit short. I'm a jack of many trades, master of none, but unfortunately "conversational skills" isn't in the many.

 

I'm sure there's a point where post-processing scripts get so complex it's easier to write a plugin for Cura. Unfortunately I only know the first thing about plugins in Cura, and that's how to install them. The resident Grand Master of Plugins here on the forums is @ahoeben, maybe they can offer some insight as to whether you're at that point.

 

Edit: forgot to mention, the best way I've found to get debug info out of post-processing scripts is to add comment lines to the g-code output

Edited October 16, 2023 by Slashee_the_Cow

[FredFish 4]

I had an epiphany regarding the multiline input, and updated the script :)
Almost comical in implementation, it fills the missing void by combining both single lines and slash-n’s. 
Several single lines are in fact a multiline input. Just don’t know how to loop the return definition to call more than one line, so it’s currently limited to 20 lines because it’s just repetitive code for each line, and each line can still have multiple slash-n’s (\n) for newlines.

I had ncluded the variable notes to include comments within the g-code for debugging, but set it not none as the default. 

Thanks for all the assistance with new eyes when my lines of code get blurry, moving this to thingiverse. 

ff_InsertGerberAtLayerChange(update).zip

[Slashee_the_Cow 271]

temp_start_gcode = temp_start_code.replace(str("\r"),"\n")

This - if it works at all - will only work on files created in Windows using a text editor on default settings.

 

Windows - because it has a history of backwards compatibility going well over forty years by now, uses two characters to represent a new line: CR (carriage return) LF (line feed). This harkens back to the days when computers output to automatic typewriters - it needed to both return move the carriage back to the start of the line and feed the next line of paper in.

 

Linux and macOS (as well as... well basically everything which isn't Windows or DOS) only use a LF to represent a new line - it's more efficient and it's still unambiguous as to its meaning.

 

Your code searches for the escape sequence "\r" (and you don't need to explicitly cast it to a string, you literally just created a string literal - pun intended), which is the code for a CR. Python is smart enough that it will use the new line conventions for whatever platform it is running on so a "\n" will be a CRLF on Windows or an LF on other platforms.

 

Are you sure you didn't mean:

temp_start_gcode = temp_start_code.replace(r"\n","\n")

The r preceding the first string makes it treat it as a raw string - characters like newlines are not converted to their relevant escape sequences. This lets you search for the text "\n" and not have it automatically convert it to searching for a new line.

 

That's the first thing that stood out to me - your code is... well, pretty bad. At the very least I'd like to save you a few lines. Instead of:

start_gcode_raw = self.getSettingValueByKey("Load_tool")
# lots of repetitive lines here
if self.getSettingValueByKey("Load_tool_2") != "":
    start_gcode_raw += "\n" + self.getSettingValueByKey("Load_tool_2")
# snip all your repetitive lines
if self.getSettingValueByKey("Load_tool_20") != "":
    start_gcode_raw += "\n" + self.getSettingValueByKey("Load_tool_20")

Use the power of f-strings!:

for i in range(2,21):  # range() is exclusive on the end number, it will go up to the number below that, so this is 2-20
    if self.getSettingValueByKey(f"Load_tool_{i}"):  # Any non-empty string is automatically true
        start_gcode_raw += "\n" + self.getSettingValueByKey(f"Load_tool_{i}")  # Python 3.12 supports nested f-strings so you could do that without the concatenation in the middle, but Cura doesn't use Python 3.12

 

I could go on. But if it works for you, then feel free to ignore me. Although I'm pretty sure I mentioned both having many variables for input and converting raw "\n"s to newlines as possibilities.

Edited October 24, 2023 by Slashee_the_Cow

[FredFish 4]

Thanks for the coding updates, you are correct in that I am using older Windows and an ender 3, the code issues I ran into may be a result of that. And your character replacement had different results with the different versions of my windows and Cura, the code was what worked on both, even though you are correct that it Should work as you described.

 

 i think i mentioned something about it being repetitive, you are correct i should have used a loop, my limitation on lines was arbitrary (just got tired of copying) , using the paradigm how would you make the amount of lines match the capacity of the stack location? is there a limit? All I am doing is collecting the lines and making them into one string anyway.

 

and finally thanks for going down the rabbit hole with me, I could have stuck with the program limitations, but what fun would that be? like you mentioned before scope creep it’ll get you every time.

[Slashee_the_Cow 271]

You're lucky I have way too much free time and the determination to do things just to prove I can:

I haven't tested it because frankly I have no idea what the hell it does (so please look at your gcode before you rely on this) but now you can set an arbitrary number of text fields as the maximum - right now it's set to 50, but there's a variable right at the top you can change if you need more.

 

What a lot of people don't think about is they just use getSettingDataString to return a string literal wrapped in triple quotes. It's a proper function, it can run function things... like for loops that create a bunch of string entry fields... and then return a complete string at the end. I also changed the code that loads them to use loops because... well it should have been in the first place, now it needs to be.

 

If my part doesn't seem to be working feel free to send me a message and hopefully we can work it out.

ff_InsertGerberAtLayerChange.zip

[FredFish 4]

Thanks for posting and updating the code, I can use the example,

I think I should better explain the purpose of the program in more detail.

 

the original intent is to be able to take output from some printed circuit design software (Gerber data a.k.a. RS274X) and using a conductive ink pen, draw out the traces and pads required to connect components that would be loaded into a 3d printed circuit board. Similar technics have been done by using magic markers to colorize the outer walls of prints.

 

I had originally written a standalone program, about 3 years ago, to insert the needed data into a 3d printer g-code file. But it seemed like extra work to save the g-code file from Cura, run the program, and then having to reload the edited g-code file back into Cura to print it. The current insert Gerber at layer script is meant to emulate the program using post processing from within Cura.

 

I do not purposely run into bearers however I do seem to find my share. Python is not my first language so some programming style may seem a bit chaotic as I am taking bits from different examples and pasting them together to replicate previous processes. Your wisdom in this area is very much appreciated.

 

I thank you for the example of looping the input windows, it seems to work well.

 

however for me the script stopped working at the r”\n” line, on both the new and old versions of Cura, the error log shows:

 

2023-10-24 05:26:54,295 - ERROR - [MainThread] UM.Logger.logException [116]:     gcode_list = script.execute(gcode_list)

2023-10-24 05:26:54,295 - ERROR - [MainThread] UM.Logger.logException [116]:   File "D:\Cura4.12.1\Ultimaker Cura 4.12.1\plugins\PostProcessingPlugin\scripts\ff_InsertGerberAtLayerChange.py", line 598, in execute

2023-10-24 05:26:54,295 - ERROR - [MainThread] UM.Logger.logException [116]:     start_gcode = start_gcode_raw.replace(r"\n", "\n")

2023-10-24 05:26:54,311 - ERROR - [MainThread] UM.Logger.logException [116]: AttributeError: 'NoneType' object has no attribute 'replace'

 

any idea what is missing?

[Slashee_the_Cow 271]

Typo on line 596, possibly. Should be:

start_gcode_raw += self.getSettingValueByKey(f"\nLoad_tool_{i}")

(I accidentally did = instead of +=)

 

Hopefully it's that simple.