Cura stability - the argument for an LTS release cycle

[burtoogle 513]

Hi,

Here's a hot topic... let's see what people think.

As a contributer to Cura, I am pleased that users can try out new features and provide feedback due to the regular releases of Cura. But not everyone wants to try out the latest and greatest features/bugs as they are actually more interested in stability. I use Cura to create parts for my business and so I understand very well people's requirements for a stable version of the software that you know will work (with whatever limitations it may have reliably). I use the development version of Cura and it does get broken but, for me, it is normally no hardship to rewind back to a working version. I appreciate that not everyone is in the position to do that.

So I think that Ultimaker should give serious consideration to changing their Cura release regime so that those people who want stability can have it and those who want to live on the edge can do so.

I can think of two examples of software products that allow users to choose between stability and new features:

Ubuntu Linux - produces feature releases every 6 months and LTS (Long Term Support) releases every 2 years (I think, whatever, it's a long time).

Google chrome browser - has 3 release branches (dev, beta and stable) stable still gets updated fairly frequently but not as much as the other branches.

Personally, I strongly believe that Cura should be released using the LTS model, i.e. once an LTS release has occurred, only critical bug fixes will trigger a new release on that branch. The more frequent feature releases can then be the incubation environment for new features, etc. When new features and other changes have been proven to be sufficiently stable and worthwhile, they could be incorporated into the next LTS release.

My overriding rational here is that the vast majority of users are more interested in stability and basic print quality than having new features.

Please discuss politely.

Cheers,

Mark

Edited November 10, 2017 by burtoogle

[ahoeben 1,948]

I agree with the idea of an LTS release, but it may not be quite as easy as it sounds.

From experience working on Cura 2.1 through Cura 2.3 with the team, and on all the releases since then as an external contributor, I know that solving some bugs that may seem fairly innocent from the outside turn out to require fairly big architectural changes. These bugs would not be solved in the LTS release.

Having different release branches is going to require more man-hours at Ultimaker. It is not just about maintaining two branches, which can diverge quite a bit over the LTS period, but also about testing each release before it goes out. Having LTS releases means more releases in total, so more load on the already overloaded testing team.

Having multiple branches to maintain can also lead to confusion about what fixes are in what branch. As an example I can name a couple of fixes that were made to the development branch of Cura well before Cura 3.0 was released that were simply forgotten to be included in the 3.0 releases (yes, all 5 of them) because the 3.0 branch had already been split off.

I am not saying these issues can not be overcome (and I personally think they should be overcome), but giving a couple of arguments from the outside why this is not going to happen overnight.

[burtoogle 513]

All good points. I don't think for one moment that it would be easy to move to an LTS release cycle. However, something has to be done to avoid the current situation where every time a new Cura release is made there's a possibility of something getting broken or being incompatible with a previous release. It can't be said that hasn't happened in the last couple of releases, because I know for a fact that it has.

If I was an Ultimaker customer that had shelled out for one of their nice machines, I would be very unhappy if I upgraded to the latest feature release only to find that I couldn't achieve the same quality of output that I could previously.

[SandervG 1,521]

Interesting question and thank you for opening it up here @smartavionics. I'm curious to hear how our users think about it. I think it is clearly a topic with many different opinions. (and a difficult one too!) I have the impression that Aldo's feedback was mainly fuelled by technical limitations, which is a good and realistic perspective but not necessarily what would be best from a users perspective. (who probably doesn't care much about those reasons). Following Mark's perspective, they want reliability instead.

There is definitely pros and cons on both sides, and technical/organizational reasons that favor one over the other. For example, with this speed of innovation it is also not easy to create documentation that explain (new) features about Cura. Things can get outdated quickly and change in general.

Personally, I am split between the two. But if I would have to choose I think for now the fast innovations are necessary for Cura to grow and gain more value, and perhaps we have to try to minimize or accept the downsides of this. I realize 'necessary' is different from what you might 'want' as a user, but I guess the necessity outweighs the want-factor in this case (in my opinion). (which is kinda in line with Aldo's rationale).

[ultiarjan 1,223]

Although as a hobbyist a bug here and there doesn't hurt me, I think a split in stable and innovative releases makes a lot of sense. Even for the non business people it's nice to have a stable version for sort off critical work. Assuming one can run the latest versions next to the stable versions one would have the best of both worlds.

I assume many people already try to achieve this currently by keeping older version in use, a "proven" stable version would be helpful.

Ultimately it would be more interesting what ultimaker's business customers think about this...

[UbuntuBirdy 56]

I know you never expected that, but I work with Ubuntu ...

and for almost 10 years now. And that's why I can only agree with the idea with the LTS model! It is extremely comfortable to have a stable LTS version that works reliably for me. but it is also very cool to be able to access a very up-to-date, if not quite as stable version, if I need a newer feature.

[nallath 1,120]

I expect it would mean that we lose about 25-50% of our development time if we were to start supporting a LTS version of Cura.

A user does care about technical perspective, as it simply means that it will come at a cost. It's easy to say that one wants all the features, you want it stable and you want it done yesterday, but that's not how it works (unfortunately). Aldo is making a pretty valid point that it will come at a cost (a pretty high one at that). So in order to make a good decision, people need to ask themselves; Is it worth that much?

[geert_2 557]

If Ultimaker makes *very* sure that all Cura-versions can work parallel, also early developer versions, without influencing each other's settings, then doesn't that solve the problem? When they are installed in totally different directories, based on version number and sub-numbers? I think this is already the case, although I don't know about the very early experimental versions?

Then you can just keep a couple of the most stable older versions on your system for production. And you can freely experiment with all new features in the nightly builds, and give feedback.

A version that is stable enough for my production models, might crash or miss an important feature for someone else. And vice-versa. So anyone can keep a couple of "good-enough" old versions in parallel on his system, as his "personal stable branch".

The advantage is that the newer features get tested well in the field, and that no extra manpower is required to maintain and develop several old branches in parallel.

[GrueMaster 1]

Fortunately (at least for Linux), having multiple versions doesn't seem to be a major issue as they are currently self-contained.  And even if they weren't, it wouldn't take a whole lot to setup Docker containers if needed (mainly drive space - I have a 480G SSD and my Ubuntu 16.04 Desktop takes ~18G with a lot of extras I have installed).

 

Having worked on Ubuntu directly for 3 years, I can tell you that it is really a LOT different to maintain an LTS OS than a single product like Cura, mainly in favor of the OS.  Most of the apps are maintained upstream and a lot now have their own LTS tracks (including several stable kernel branches).  Not to say that Ubuntu doesn't have their work cut out for them as they currently support two LTS releases, a development release, and a 'next' release that receives active development.  Figure ~60-80% workload on new development, and the rest on support.  For a large community, this is not a problem.

 

I don't know how many active + community developers Cura has (I would venture to guess it is less than 30).  Figure it would take 10-15% more to be able to backport bug fixes and then test them for any kind of maintained LTS tree.  The hard part is the testing.  True SW validation takes resources and skills different from most development disciplines.  And even then, bugs get through.

[Dim3nsioneer 557]

+1 for an LTS release cycle.

 

Currently, we seem to have a bug in Cura 3.3.1 for combing which leads to extremely long travels. That significantly increases print time. I don't know if that bug was already in the beta. If it was, it was not detected. That could mean the beta test period was either too short or the wrong people were testing the beta.

However, @smartavionics has already produced a bugfix which is great (thanks, although it is a bit exeptional that a community member fixes bugs implemented by employed devs ?). I hope we will soon get a V3.3.2 including this fix. As I was informed the bug was introduced with many other changes and could maybe not be detected by the Cura devs themselves. The more important it seems to me to have major releases at a much lower pace and bugfixes in between. More thorough internal testing and re-introduction of closed betas could also improve things.

I'm convinced that changes have to be made. The current roll-out system with the extreme pace doesn't work very well. It was ok at the time when Cura needed to catch up to other slicer software but in the meantime it has improved a lot and the strategy should adapt.

Edited May 11, 2018 by Dim3nsioneer

[Anders Olsson 136]

In my opinion there are two main issues with Cura currently:

- There are way too many releases and while there are bug fixes in each of them, there also seems to pop up new bugs in every release.
I would prefer much fewer releases with a minimum of new bugs, more testing before releasing and more focus on removing bugs so that we get to a reasonably bug free version.

- While many of the new features in these frequent releases are very nice and useful the interface of Cura is currently too complex. So the majority of the users will never take advantage of things like Ironing or Spiralize, even though it would improve their prints, simply because they are not aware of the functions or how/when to use them.

Until someone invents a simple way of finding these functions, like a menu where people select the type of object being printed and Cura applies suitable functions for that type of printing (like a one click "Vase mode" for example), it makes little sense to continue adding new features like these. Time would be better spent in fixing essential bugs then, since it will improve the experience for all users.

[thopiekar 46]

I personally don't think an LTS makes sense. The reason why there are LTS versions in Ubuntu is that packages are maintained for a long time while having stability and security in mind. So in my opinion, if someone likes to have an LTS-ish feeling while using Cura, she/he should simply pick the last release with a patch version >= 0. Security-wise Cura is not that much connected to internet services, that you need to worry about breaks there.

From my feeling stability issues are taken always serious at Ultimaker, so it is always a question of time until a fix is there and until then you can always redownload the previous version from the website. If people (or the community itself) think it is not stable enough, then they can also bake their own version and cherry-pick the fixes only. But I doubt that there is a group (large enough), which has time and the capacity for it.

 

@Anders Olsson:

* Fixing bugs is not always as trivial as it sometimes seems. In my opinion, you need to find a red line and skip fixing bugs from a prioritised list, because the fixes, which have been found, need to be shipped to. Therefore I think it is fine just to wait for the next patch version or major release. 

* The default is the simplified view, which does not provide access to all the features. I personally wouldn't like to add more buttons there. Once people will decide to add more options there, then all the other use-cases will come up and the simplified view will get the new advanced view. Comparing Cura to other professional software, I would say that once people want to print more complex things like this, they simply need a training in using Cura. Cura is a professional 3D-slicing software and giving up all the great parameters and options it has would be stupid. I can imagine a lot of effort would be invested into finding relations between the settings, a lot of assumptions would be done on the customer, which maybe don't happen or fit, and, finally, the end-user would be disappointed, because something is not printable.

 

The only thing I agree about is something that @smartavionics  kind of mentioned: longer beta/test cycles. I think it is no secret anymore that Ultimaker creates daily nightly builds. I see no reason why releasing them in public (excluding internal secrets) would be an issue. The sources (cura-build-*) to build Cura are already public. In theory, everyone could build a nightly version on their own, but that is less comfortable, so most potential testers keep using either the last release or go for the early beta.

 

(Guess that is enough from my side for now ?)

[burtoogle 513]

Hi @thopiekar, thanks for your input. I understand what you are saying but I think that a lot of Cura users would be happier if they knew that the major version (3.5, 3.6, etc.) they are using is only going to receive bug fixes (3.5.1, etc.) and, most importantly, will continue to receive those fixes until the next major version that is designated as "LTS" is released. Also, the users should have confidence that the next version that they may choose to upgrade too has received a decently long testing period. Months, not weeks or even days as it is at the moment.

 

As you know, at the present time, bug fixes and new features are all rolled together in each release and bug fixes are never made available for previous releases. i.e. once 3.3 was released, there will never be a 3.2.2 (or whatever).

 

Recent events have proven (once again) that the current release regime is highly unsatisfactory. I feel that unless UM gets to grip with this issue, they will, ultimately, suffer commercially due to lessening customer satisfaction and confidence.

 

[thopiekar 46]

Hmm, I understand the benefits, but it is always a question of time, whether it is worth the effort - just like @nallath mentioned before. Because I know him a better, I'm sure he has a good estimation based of his work experience of how much time in addition it would take.

 

Is your experience of disappointments around Cura based on other people, too?

I personally don't hear much of this kind of feedback. 

(Maybe the question should be asked whether we couldn't make bug reporting easier, so we get more and earlier feedback on found bugs?)

[burtoogle 513]

I don't doubt that the time/effort required to move to an LTS (or other suitable regime) would be substantial but I think the payoff would also be substantial. Think of it as an investment for the future.

 

As I don't use the normal Cura releases and can work around most problems that arise, I am personally not upset by the bugs so, really, my concern is for others who are not able or willing to build the software themselves. And why should they have to build it themselves just to workaround a serious problem that wasn't there in the previous release?

 

Bug reporting in Cura is already fairly easy, either people come here to the forum or the more technical types go to github and open an issue. But getting new releases tested sufficiently is a very real problem because a lot of users obviously don't try the betas and then when the actual release comes along and they find a problem it's a bit late.

 

As a contributor to Cura, I know only too well that the contributions that I make, no matter how much I test them, are possible sources of problems which won't get detected until the community gets their hands on the release. That is why all new code and major fixes that go into Cura need a much longer incubation period then they get right now and those in the community who are willing to try out new stuff should step up and give the betas a good testing because they are probably the most important people in the chain.

Edited May 11, 2018 by burtoogle

[thopiekar 46]

9 minutes ago, smartavionics said:

Bug reporting in Cura is already fairly easy, either people come here to the forum or the more technical types go to github and open an issue. But getting new releases tested sufficiently is a very real problem because a lot of users obviously don't try the betas and then when the actual release comes along and they find a problem it's a bit late.

Good point. The few people are then maybe also not so sensitive to errors.. So when thinking of the workflow when fixing bugs or adding new features. What would you change? Do you already have a strategy in mind?

 

10 minutes ago, smartavionics said:

As a contributor to Cura, I know only too well that the contributions that I make, no matter how much I test them, are possible sources of problems which won't get detected until the community gets their hands on the release. That is why all new code and major fixes that go into Cura need a much longer incubation period then they get right now and those in the community who are willing to try out new stuff should step up and give the betas a good testing because they are probably the most important people in the chain.

I wouldn't differentiate between the community and business/end-users. What actually counts here the most, is that your suggested stable/beta/dev flavours need to be deployed to the users more easily and I think this can be easily done(!).

[burtoogle 513]

27 minutes ago, thopiekar said:

I wouldn't differentiate between the community and business/end-users.

 

Really? To me, they appear as two very distinct groups.

 

38 minutes ago, thopiekar said:

So when thinking of the workflow when fixing bugs or adding new features. What would you change? Do you already have a strategy in mind? 

 

No, I haven't yet given it much thought but off the top of my head I would probably favour LTS releases say, every 6 months. Once released, they would only have essential bugs fixed as sub-releases with no new features. Development releases could continue at whatever rate is manageable but there would need to be a cut-off say, 2 months before the next LTS release where only bug fixes and polishing would be allowed. All users would need to be encouraged to try a test release within, say, 1 month of the LTS release.

A very important part of the testing would be to ensure that profiles, etc. from one LTS release would be acceptable to the next LTS release.

 

There would need to be strict enforcement of cut-off dates and other deadlines but I think once people get used to the regime, it could work very well. Personally, I would much rather any contribution of mine was tested by myself, the Cura devs and members of the community for at least two months before being unleashed on the wider world.

 

A regime such as described above would require, at most, 3 major branches in the repos. The LTS branch and the branches for the next two releases. You would need two because once the next LTS branch is in feature-freeze, any new features/major changes would need to be on another branch.

 

So not much different from what is done now, really. Just a little bit more relaxed!

[Anders Olsson 136]

2 hours ago, thopiekar said:

Is your experience of disappointments around Cura based on other people, too?

I personally don't hear much of this kind of feedback. 

 

That question was probably not ment for me, but just the past few days I had two persons basically asking me what is wrong with the Cura development lately, because of various bugs that leads to odd behavior and stability issues in recent Cura versions.

Personally I have had to roll back to using Ultimaker 2 machines and Cura 15.04 several times lately because all those excessive travels in later versions of Cura ruins the surface quality of objects that I want a nice surface on.

When it comes to Cura features, even experienced users who I talk to are unaware of what are suitable settings for printing a vase or what ironing is for example.
Just a simple function where the user identifies the object from a list and Cura then applies a few suitable settings (like 0% infill, solid bottom and spiralize for a vase, or slow outer shell speed and a minimum of infill for a sculpture) would be nice in my opinion because:

- It would make it much easier to explain for others how to get good printing quality for certain objects.

- The Cura-team could stop trying to find "one size fits all" settings for the printing variables.
Just have a general setting with works decent and then tweak the parameters for certain types of objects instead.

[thopiekar 46]

19 hours ago, smartavionics said:

Really? To me, they appear as two very distinct groups.

I think basically all people are the same. Beginners are quite the same as business customers. Once they get disappointed you are losing them (worst case) forever, because of the disappointment which gets associated  with the  product based on their experience. Actually everyone wants stable software, but most people get tempted by new features. I'm the best example: everytime I f**k up my computer (oh, that's already a while ago) I tell myself: I have to use an LTS or Debian stable. But it takes around 1 week until I begin to upgrade again.

 

Yesterday night I took some time before sleep to think about it. I branch layout I have in mind is even more complex then you suggested. Together with CI I would create for every minor version (eg. 3.2 and 3.3, etc.) a branch with a -next suffix where every change is included, which is not well tested and release. Whenever a serious bug is there, I would create a branch, eg. 3.2-CURA{ticketnumber} or *-GITHUB{issuenumber}. Whenever something is pushed there, a new build is triggered, which builds a new installer for windows or AppImage for Linux. To ease people to report is has been fixed there, we could add an plugin, which adds an button below the toolbox area. This one could have too selections, eg. ? and ?. Culture-wise I doubt there are many people who don't know what it means. As soon as a test build hits more than 10 tests, with a ratio of more than 9:1 successful fixes, the branch could be automatically merged into the brnach of the minor version. The art of all of this is to care about finding the bug at the oldest version. Then all versions upwards can benefit of it.

At this point your suggestion of defining what a LTS is could kick in and define that eg. 3.2 is the current LTS.

 

@Anders Olsson No, the question was not meant for you but still interesting to hear about your impressions how the Cura development does recently.

 

Regarding your quality problems, I would recommend to slice your model and compare the GCode, highlight the changes and differences and report it on GitHub or the forum here, so someone can help.

 

As a conclusion I see understand now, why the development goes too rapidly for you. Currently I don't have much time anymore to help out, but maybe I can help somehow in a different way.

[burtoogle 513]

Hi @thopiekar, yes, having the -next branches would be good because then anyone could see (and, ideally, build) what is coming up in the future. Your scheme for auto-building test releases from bug fix branches that are subsequently merged if the users confirm success is interesting but, perhaps, it doesn't really need that much infrastructure. If fixes are merged into the -next branches and the auto-builds (linux/mac/win) of those branches are always available for testing then it would achieve a similar aim.

 

I'm sure a workable scheme could be devised if UM have the will to do it. There are so many regressions appearing in Cura at the moment, something has to be done!

 

[thopiekar 46]

Well, the solution I had in mind is again a bit over complexed, but if you have the resources, then it is the most elegant way I think. I would personally never merge inthings into -next too early because if something is going wrong you need to get the code out of those -next branches, before they get merged into newer releases and consequently master.

 

Well, the community can either wait for a solution by UM or just kick asses, set such an infrastructure up and prove this kind of CI setup is really needed Of course, you can wait for a solution, but do not complain that nothing is happening (or seems so). . People in the community often underestimate the power they have to influence. Only thing to do is finding people with the same interest and push hard to what you want. However, that is my impression.

[ahoeben 1,948]

Note that just providing more builds will not amount to much; they would all have to be tested, a lot. A new bug introduced by a fix for another bug does not always manifest by testing just for the initial bug. Ultimaker employs a number of testers who do nothing but test Cura all day. Community builds would have to be community-tested.

 

 

[thopiekar 46]

1 minute ago, ahoeben said:

Note that just providing more builds will not amount to much; they would all have to be tested, a lot.

I know. I never said that a lot of effort is needed for it. It really depends on how much the community is willing to do in terms of development and testing.

Ultimaker does a lot to test their software and the people employed are doing very good. However, it seems from the feedback here that this is too less.

 

Building an infrastructure, like @smartavionics and I already brainstormed on, really need more people to keep it up. I can also imagine that UM could jump on and help out with computing resources, but first, it needs a proof of concept and this again needs human resources. This is finally from my experience, what will fail in the community I guess - there are simply too few people, who want to get active and get (big!) things moving. That is, by the way, the reason, why I'm stepping back from community work (very) slowly.

[ahoeben 1,948]

Can we use the name “Curated” for the community edition of Cura?

[thopiekar 46]

Meh, a simple unbranding of Cura-vuild-environment would be enough. On the splash screen it would be enough to call it either simply Cura CE or Cura Community Edition. Would rename CuraVersion.py.in into CuraConstants.py.in and set there the default theme. So within CMake and buildtime we can choose the community theme. From my point of view this is done.

 

Already had unbranding in mind for my PPA, because people might think it is UM driven, but it is not.

 

So how many people do you think could we get together?