Using external libraries in my self-written Cura plugin

[Hanabi-san 1]

Hallo

 

I'm writing a plugin for Cura 4.3. I would like to use some external libraries in my plugin. However, when I use those libraries, Cura cannot read my plugin anymore (I added my plugin project folder to the plugin folder in the Cura applications contents folder).

I noticed there is a "lib" folder in Cura's Resources. Is this the right location to add external libraries? Do I have to do it manually, or is there another option? Or if I'm wrong, how do I import or install external libraries into Cura?

 

Thank you in advance!

Edited December 20, 2019 by Hanabi-san

[ghostkeeper 105]

Cura is shipped with its own Python environment (which is CPython 3.5.2). When processing the import command in a Cura plug-in, it will look in three places:

• Cura's own environment, which includes the "cura" and "UM" packages containing Cura's own code.
• Cura's dependencies, which are installed with Cura. You can't add things to this; you'd need administrator access and your users might not have that.
• The plug-in folder itself.

So to add a dependency that's not yet shipped with Cura, you need to supply it in your plug-in folder. There are two types of libraries you can import then: Pure python libraries or compiled binary libraries. I have two examples for this.

 

Pure python libraries are the easiest. You can just put the source code in the plug-ins folder. To see this in action, you can install the Settings Guide plug-in from the Marketplace. If you then go into Help -> Show configuration folder and browse into the plugins folder you'll find the installed plug-in with simply a folder called "Mistune" with a file "mistune.py". Then in the main plug-in code I can just go "from .Mistune import mistune" and it'll import that mistune.py file. Just like any other Python file you need to use a relative import.

 

If your library is pre-compiled it becomes harder. If you're just using the library itself it's still easy. You put the .so file (the actual compiled library) in your plug-in's folder and you can import it similar to Mistune above. However if you want to publish the plug-in, you're going to need to import different libraries depending on whether the user is on Windows, MacOS or Linux. For an example of this, you can look at the X3GWriter plug-in that executes a binary depending on the platform. That's done here: https://github.com/Ghostkeeper/X3GWriter/blob/b55d893ca59b1b50becd372adb9a171ec9e5cf77/X3GWriter.py#L97-L105

Instead of an executable as in that plug-in, you'd be doing an "import something_something" depending on the platform. Very similar.

[Hanabi-san 1]

@ghostkeeper Thank you very much for your quick and detailed answer. I've now added the Python library source code folder to my plugin folder and imported the needed files like you described. It works well for library files that do not import anything from the same library folder. However, e.g. if I want to import a file x.py from the library "abc" and the file x.py includes the import statement "from abc import y", then it does not work anymore. Do you know why this happens and what I can do against it?

Edited December 20, 2019 by Hanabi-san

[ahoeben 2,010]

Here's another example, for when you need to load a module from a specific subfolder in your plugin:

https://github.com/fieldOfView/Cura-OctoPrintPlugin/blob/dece4d6e23b0b99efa585301369fd20e25d8cb96/OctoPrintOutputDevicePlugin.py#L20

This way is a bit more complex, but the path to the module could be made platform-specific if needed.

 

On a general note: it is better not to put your plugins in the program bundle, but instead put it in the plugins folder in the configuration folder. The less you mess with the program bundle, the easier things are to restore if you make a mistake. And as a bonus, your plugin will be carried over to future versions of Cura.

[Hanabi-san 1]

@ahoeben Thank you very much for your fast and detailed reply.

1 hour ago, ahoeben said:

On a general note: it is better not to put your plugins in the program bundle, but instead put it in the plugins folder in the configuration folder. The less you mess with the program bundle, the easier things are to restore if you make a mistake. And as a bonus, your plugin will be carried over to future versions of Cura.

Yeah, that makes sense. I will do so.

 

1 hour ago, ahoeben said:

Here's another example, for when you need to load a module from a specific subfolder in your plugin:

I've also tried this approach in my plugin. However, I encountered the same problem as described above in my previous post: It doesn't work for files that import other files in the same library but works perfectly fine for files that don't. Do I perhaps miss some setup steps?

[Hanabi-san 1]

Quick update: I've just noticed that my problem seems to not have anything to do with the import statements in the library files. For some reason the import does work for some files in libraries and not for others but I couldn't find a pattern yet to what causes this problem.

Edited December 20, 2019 by Hanabi-san

[ahoeben 2,010]

Here's yet another example (of an unreleased project):

https://github.com/fieldOfView/Cura-SerialConnection/blob/master/SerialOutputDevice.py#L36

Here I am manipulating the search path while importing a module; the system search path is used while importing modules. Without that search path manipulation, I ran into similar problems.

[Hanabi-san 1]

@ahoeben Great, thank you very much for the second example. I tried it, but unfortunately the result is still the same: For some files it works but for others not. I'm starting to feel like the problem lies in the kind of libraries I'm trying to use. I mainly want to use encryption libraries like pynacl. Could it be, that such libraries do not work well together with Cura?

[ahoeben 2,010]

I'm sure it depends on the module(s) you are importing and the searchpath(s) you add. Knowing what fails in what way (logs) could help...

 

I've seen some "action" on the Ultimaker github lately that indicates that the next version of Cura will include the cryptographt package: https://pypi.org/project/cryptography/

[ghostkeeper 105]

Here is another example where I'm importing a library that imports among itself: https://github.com/Ghostkeeper/SVGToolpathReader/blob/master/Parser.py#L27-L32

 

I don't think that different Python libraries would collide with each other. And if they do you'd see some very weird error messages like PyCapsule_GetPointer collisions and such (an old bug in SIP).

 

Cryptography is already in the 4.4 build, so you can try it out there. We're going to use it for verifying signing keys of plug-ins installed from the Marketplace (though currently the Marketplace doesn't sign them yet).

[Hanabi-san 1]

Cool, I didn't realize cryptography is already implemented in the 4.4 build. Then I try using the existing libraries of Cura for my plugin. That's even better, because I don't have to include the source code of the library in my own plugin folder.

 

@ghostkeeper and @ahoeben Thank you very much for providing me with so many examples. Even though I couldn't use them for my desired cryptography library, I'm sure I can take advantage of them again when I want to use different kinds of libraries. 

 

[Hanabi-san 1]

On 12/21/2019 at 12:44 PM, ghostkeeper said:

Cryptography is already in the 4.4 build, so you can try it out there. We're going to use it for verifying signing keys of plug-ins installed from the Marketplace (though currently the Marketplace doesn't sign them yet).

I am unable to find the cryptography library already integrated in Cura 4.4. Does anyone know under which name I can find the cryptography library in Cura's lib folder or which import statement I have to use?

[ghostkeeper 105]

Importing "cryptography" should work.

 

We use the library with imports like this: https://github.com/Ultimaker/Uranium/blob/038fb7286ba9418537581ea4b36342e58ac252c8/UM/Trust.py#L9-L14

[Hanabi-san 1]

@ghostkeeper Thank you, I realized however that the cryptography library is not integrated in the downloadable Cura app so far but only on the project on Github. It seems, that it is only going to be integrated in Cura 4.5. Therefore, I'll have to wait until Cura gets updated until I can use the library in my plugin.