Dramatic Layer Shifts

[Yoter 14]

I've had my Ultimaker 2 for about 13 months now and it has around 4700 printing hours.  I don't know if that is a lot or pretty standard, but it does run most days and isn't a new out-of-the box machine anymore.  While it does have a lot of hours, the prints are typically slow.  I rarely exceed 40mm/s linear print speed and typically stay in the 20-30mm/s range.  Many of my models are architectural in nature and I am a strong advocate that the fastest way of doubling print speed and maintaining quality is plugging in a second machine.

In the last week I have been getting intermittent dramatic layer shifts.  It will print fine most times, then suddenly the layers will shift between 3/16" and 1" in a single layer and it will continue printing.  I have seen the shift occur in hour 23 of a 25 hour print, and on the first layer, so it seems like timing is pretty random.  It appears the shift happens along both axes at the same time, but most dramatically along the Y.  The shift seems to be fairly consistently Y- and X-, I have not noticed it ever being any other direction.

I've checked grub screws, belt tension, head motion and all seems in order.  The motion rods are wiped and oiled every Wednesday, or as needed between prints if any sticking is detected and stripped clean with isopropyl and re-oiled each Saturday.  The motion feels smooth and equal in all directions.  I was not expecting to see anything there as when I have had the machine get loose before, I typically see a gradual lean develop into layers shifting, not a sudden clean interruption and shift in the layer.

I thought perhaps the endstop switches were at fault and have checked the mounting, the function of the contact switches, and looked at the wiring.  I'm not a fan of the way the wiring to my Y axis stop passes between the belt and the side of the machine, but I do not see any indication of rubbing or any indication of either harness rubbing against the pulleys so I guess it's fine.   I did not see any indications of damage further down the harness while I had the covers off to pull the motors to check the grub screws, either.

I did try turning down current to the X-Y steppers as I have seen recommended on some of the UMO printers, but it seems to not make a difference.  I didn't expect it would as the problem doesn't seem tied to print time which to me would indicate that it probably isn't a heat issue.

Anything else I need to be looking at?  I am still really thinking there may be something with the endstop switches.  On my printers and other machines alike, when I see something dramatic and clean such as this I always look  at any digital type on-off input that could be interrupting the process, as I typically see mechanical or heat buildup issues cause gradual shifts or changes, but a 1" shift is rather dramatic.  

Once it (hopefully!) finishes the print it is working on, I was thinking I might flip the machine over and check the endstop connections at the board, but I am guessing the endstops are normally open like many of my other machines so I don't think I will find anything there.  Would it be crazy to run a few days with the endstops disconnected from the board once the head is zeroed to test the circuit and switch?  

If it helps, I can run some test prints tomorrow and post pics if it acts up.  I can't really post my daily prints as they are for work and I would have to get permission to share them.

[SandervG 1,521]

Hi @Yoter, thank you for your post. Sounds like you have already done quite some troubleshooting!

Do you have a photo of what such a layer shift looks like in your print?

If it is a very abrupt shift, it could be that one of your end stops is triggering a signal.

If it is slightly more gradually, I would look into your pulleys.

Alternatively, it could be that a motor or stepper driver is overheating, but at that speed, and if it also happens around the first layers, that sounds pretty unlikely.

To rule out pulleys, when you are printing, can you prevent your printhead from moving around? Like stop it in its path?

It should be quite hard to do, and most likely should start skipping steps than your pulley loosing grip / slipping.

I don't think it would be silly to unplug the end stops during a print after it zeroed. You may want to stick around when the print finishes though. But worst case, it will try to move across the frame for like 5 seconds, causing some noise, and then will give up and stop. You could also try flipping the end stops, see if it changes the behaviour of your layer shift.

Alternatively, you could also try to change your X and Y motor.

In any case, some pictures would probably help.

[BrayChristopher 36]

I had some similar random shifting.

https://ultimaker.com/en/community/22908-shifted-print

But it sounds like you checked EVERYTHING already.

I think my problem was a sticky rod, but you wipe your down every week, so that can't be it.

I took the internal motor covers (white sheet metal parts) out of the printer to let the motor "breathe" a little easier.

For long prints, I also aim a desk fan at the electronics underneath printer.

Any chance it could be a slicer problem. I doubt it, but you have ruled out a lot of other causes.

Edited November 3, 2016 by Guest

[Yoter 14]

Hopefully that uploaded the images!

The shift is very abrupt. I set the print back on the bed to show the direction it always seems to take.

I did try to stop the head while it was traveling this morning. I moved it to the far corner and tried to hold it when I directed the machine to home. It did instantly start skipping steps and I didn't see any slippage in the belts/pulleys. Also, once it homes and the servos are powered, I don't feel any backlash.

I'll try a slice in Cura to rule out a Slicer issue. I've never had an issue like that with Simplify, but there is a first time for everything!

While I am waiting for the print, I do believe I am just going to order the stop switches. They're cheap enough I'm willing to swallow my pride as a technician and just throw a pair at it and see what happens.

[tinkergnome 927]

While I am waiting for the print, I do believe I am just going to order the stop switches.   They're cheap enough I'm willing to swallow my pride as a technician and just throw a pair at it and see what happens.

 

I have no idea about your shifted prints, so just out of interest:

Why do you suspect the end stops? AFAIK the end stops are only used during "homing", but completely ignored during a print.

Or do you use a customized firmware?

Are you printing from sd-card or is the USB cable connected?

[SandervG 1,521]

 

While I am waiting for the print, I do believe I am just going to order the stop switches.   They're cheap enough I'm willing to swallow my pride as a technician and just throw a pair at it and see what happens.

 

I have no idea about your shifted prints, so just out of interest:

Why do you suspect the end stops? AFAIK the end stops are only used during "homing", but completely ignored during a print.

Or do you use a customized firmware?

Are you printing from sd-card or is the USB cable connected?

 

As far as I know when you manually trigger an end stop you see immediately in the movement of your printhead during a print. I don't think it is being ignored during a print.

I am not able to visually inspect your end stops, but given the clean cut in your shift, and perhaps the age of your Ultimaker (rubbing of pulleys, maybe a cable broke), I am inclined to think the problem lies with your end stops. Curious to see what your new ones bring to the table!

[Yoter 14]

Honestly, nothing related to the Ultimaker made me suspect end-stops, but I have a plasma table that if the end stop is tripped mid-process, it will quit feeding voltage to the motor but keep running code so it does almost this exact thing, continuing to run once the end-stop is released while skipping movements directed while it was interrupted.

I did just trip an endstop on one of my other printers and you're absolutely right, it ignores the input.  I guess I got frustrated and forgot I have a known-good printer sitting next to it to test my ideas on.  I'll throw the ones I ordered in my parts cabinet, or use them on another project.

I guess its back to square one...  Now I'm not too sure what square one is.  I'll see how it handles the Cura-sliced model it's running today, then move my questions over to the Simplify forum if it takes care of the issue.

Edit: To answer Tinkergnome's question, I do print via SD card. My other printer finally freed up and I am running a couple models that have failed on it, that should show for sure if it is a slicer problem. Just had to wait on a 60 some hour print to finish.

Edited November 3, 2016 by Guest

[tinkergnome 927]

As far as I know when you manually trigger an end stop you see immediately in the movement of your printhead during a print. I don't think it is being ignored during a print.

 

@SandervG

That's one of the (rare) differences between the Marlin configurations of UMO and UM2... - you own both - just try it out!

[Yoter 14]

Well, I guess I ruled out a slicer issue.  This was sliced with Cura.  I may try a fan under the machine to see if it is a heat issue after all.

It did seem to slip back Y+ later in the print, about 1/2# or so for the last six to eight layers.

Edited November 3, 2016 by Guest

[Yoter 14]

Ruling out the limit switches, I went to swap the motors as I was guessing that was my next step one.  Is this amount of discoloring normal? I didn't notice it being this bad when I was in replacing the hotend heater a few months ago.  I was wondering if adding active cooling to the next board may help.  The problem was random so I didn't think I was looking at heat buildup, but now am re-thinking.

Edited November 3, 2016 by Guest

[BrayChristopher 36]

Riser feet for easier "breathing".

https://www.youmagine.com/designs/ultimaker2-big-feet

or these

https://www.youmagine.com/designs/3d-printer-feet-71d19cd6-4955-4abf-a8ae-50564997c580

Active cooling.

https://www.youmagine.com/designs/ultimaker-2-coolboard-duct-for-control-board-cooling

I bought the parts for this, but ending up just using a clip on desk fan. Some day I will probably install it.

Edited November 4, 2016 by Guest

[Yoter 14]

Thanks! I ordered the fan and was getting ready to design a mount, but that takes something off my list! I'll print the feet, as well.

I am 95% sure I have it this time. I swapped the X and Y axis steppers and ran a print and the problem moved with the motor. I have a new pair coming. I'm replacing the (former) Y-axis motor and setting one on the shelf.

[SandervG 1,521]

Hi Yoter,

thank you for your updates. Please let us know if the problem indeed lied with the motors. Are you also going to enable more direct cooling on your electronics, besides replacing the motor?

[Zwakie 21]

Good to hear you nailed it @Yoter!

Nonetheless I would like to make a remark on the dis-colorization around the soldered connections: to me it seems that these soldered connection's integrity degraded significantly (likely due to ingress of moist and oxygen), which significantly increased the resistance of the connection. Physics tell us this will generate HEAT, and from experience I know such connections can get REALLY hot! This certainly is something you need to deal with since this is a breakdown waiting to happen, and what's worse: a breakdown with accompanying fire hazard!

[Yoter 14]

Sorry for the delay! I didn't get a chance to put my parts on until Friday and wanted to run a few days before I updated.

I did add heat sinks and cooling to the board. I found a post in the forums here that seemed to have a pretty good setup. https://ultimaker.com/en/community/16880-ultimaker-2-stepper-temperature

I am printing some feet to lift my printers and give them a little extra breathing space, and adding a fan to my other UM2, as well. I figure they are cheap and I imagine the continual running on our long prints lets the boards build up a lot of heat.

 

I also replaced the X and Y steppers. I was just going to replace the one, but as cheap as they are, I figured replacing the pair at 4700 hours was perfectly reasonable.

I've run three 8 hour prints and one 20 hour print since with no layer shifts at all. I'm gonna call this one fixed! Thanks everyone for your input on this!

[SandervG 1,521]

@Yoter, you can also reduce the current on your motors to 1100, they will get less hot that way. You can do this by going to Maintenance > Advance > Motion Settings

Hopefully this helps too, good to hear your other fixes also paid off!

[Yoter 14]

Well, I know better than to say anything's fixed publicly! Ran a 8 hour print, a 20 hour print and all went well. Set my printer to making feet for itself and went home and came in to this:

I turned the stepper current down to 1100. I am also thinking perhaps I should pause the print about mid-way, and let the fan blow over the board a while to cool the stepper drivers as much as it can, and then re-start? Maybe I am cooling to board too late and it's already toast.

I thought for sure when I swapped the motors and it moved I had found it; but there is always slight X slippage along with the Y, so I wonder if the orientation of my prints (much longer in the Y axis, typically) is showing the offset on the major axis (Y) to an exaggerated degree in relation to the X. By the time I swapped the motors, I was in testing mode running thin walled cubes instead of my normal rectangular prints. This gave a 50/50 chance that if it were a general fault causing both axis to stop, it would display on either axis. I guess I inadvertently changed more variables than I intended.

I'll pause mid-print and let the fan do its thing, if that doesn't work, I'll print something longer down the X axis and see if that accounts for my original thinking that the problem followed the motor.

I was also wondering, how much of a power interruption will these printers tolerate? I don't think input power is an issue, I have three other printers running on the same breaker...but I haven't ruled out a power supply browning out the board with an interruption. If none of this fixes it, I may bring in the power supply from my UM at home and give it a shot.

[SandervG 1,521]

Hi @Yoter, ohhhh you jinxed it!

Never underestimate the power of a jinx.

I must say it catches me too on occasion..

A workflow where you have to pause your Ultimaker after a few hours to let it cool down is not a workflow we wish upon our users. If that is what it resorts too we must have overlooked a better solution.

Did you already try to do a couple of prints with the current down to 1100?

Are your belts still tight, if your print is always kinda off?

It could also indicate a problem with your stepperdrivers / motors. How off are your prints? Is it something you can show in a photo?

Has your printer always printed with the same equipment as where it stands now, and what it uses now?

[Yoter 14]

I have one print running now with the current down to 1100. It is 7.5 hours into a 8 hour print and looking good so far.

The belts are tight. The shift runs anywhere from 20-80 mm in the Y negative direction and a few mm X negative typically.

When I swapped the steppers before, I printed cubes, my typical prints are real long in the Y axis, so I am thinking that's why I thought the problem moved with the stepper. New steppers didn't seem to fix it.

We did change my print area some. Management decided my card tables and metal shelves weren't pretty and had me put together some IKEA cabinets and overheads. They like to run visitors through the print room and show off the models, and they're right, it is much prettier!

We're running the same setup with the printer on a 15 ft 12ga extension cord into a wall outlet that we have always run. The problem was existing before we changed anything and starting to creep up and become a little worse.

Two months ago, we lost about 8% of our prints on that machine to layer shifts. The last four weeks it has been closer to 70%...I don't have as much data in my logs the last couple weeks as I've been doing diagnosis. It does look like week to week, we went from losing 0-1, then 2-3, then 5-6 prints, it has been a pretty steady increase. I can get the numbers closer if needed, I am just looking at my log and doing math in my head.

We log operations and note failures on our shop machinery to help trace faults and patterns, so we follow the same procedure with the printers. Its been a pretty handy tool so far.

[SandervG 1,521]

Lets see how we can bring those numbers down, but first wait to see what happens with your current tuned down to 1100.

Lets see if it finishes a couple of prints successfully and if there is any layershift at all.

[BrayChristopher 36]

Management decided my card tables and metal shelves weren't pretty and had me put together some IKEA cabinets and overheads.

 

HA!

I had a similar situation.

[Yoter 14]

Just an update - with the current turned down to 1100 mA and the board cooling, everything seems to be working well. If I disconnect the fan or turn the amperage back up, not so much.

Is this due to degradation in the connections on the board? If it is and I may see this pop up again with use, do I need to replace the board and keep the amperage down and cooling for the best reliability? I replaced the steppers as they were cheap and the problem persisted, so I am really thinking its the board.

As a rule I print marketing models that marketing gives to customers, and jigs and fixtures for the shop. I also do a lot of prints for the local schools when they overflow their capacity. Typically, none of these are overly time-sensitive but I know how stuff works, and if I see the problem rear its head again, it will surely be the week before a tradeshow when I'm camping at the office to keep all the printers running non-stop!

I know the board is pricey, but if this one is compromised, I am more than happy to throw one in so it doesn't go down when I really need it. I did go ahead and add cooling and turn down amperage on my UM2 at home. It doesn't see nearly the use, but I figure it would be good to do it BEFORE problems pop up!

[SandervG 1,521]

Hi @Yoter,

Good to hear that your prints are looking good!

Since you can replicate the issue by removing your current work around I think we can say with confidence that we are on the right track.

Because the Y stepper driver is located in between the X and Z, and you said your prints usually are longer on Y, it could make sense that this is where most challenges lie. Because it is located in between the X and Z, it can have more difficulties relieving itself from the heat.

I don't think you have to worry about any further degradation now that it is set to 1100, but if you want to be prepared for the future you could consider replacing your pcb. Because you have also replaced the stepper motors, and reducing the current paid off, the problem must lie there.

I don't think there is an immediate threat, so it is totally up to you when you feel comfortable replacing your pcb.

Have a great day!

[Yoter 14]

Definitely the PCB! Came in today and the printer had stalled. With the steppers removed from the axis, they only turn 1/4 turn before stalling out. Honestly, I'm glad it happened now while the printer has been listed on the injured/disabled list than after it had ran great for a week or two and I had told the bosses we were all good...always quicker to say "all fixed" online than to the guy who signs my checks!

I'll get a PCB board coming and I think I'm gonna leave the steppers at 1100mA on the new one. I imagine with them turned down and the cooling on a new board, it'll be bulletproof going forward.

[BrayChristopher 36]

Definitely the PCB!

I'll get a PCB board coming and I think I'm gonna leave the steppers at 1100mA on the new one.  I imagine with them turned down and the cooling on a new board, it'll be bulletproof going forward.

 

This would be a great thing for new owners to know about when they first get their printer. After you cook your PCB, it is a bit too late.

Owners could help prevent this issue right out of gate by:

- Turning down the amperage

- Adding riser feet

- Put the printer on a wire rack shelf

- Add active cooling of your choice

- If you are going to enclose the printer, don't enclose the WHOLE printer. Only enclose the print area.

Ultimaker could help prevent this issue by:

- Making customers aware of it in the set up guide

- Reducing the amperage in the firmware

- Perforating the electronics enclosure more

- Adding some "breather" holes on the side panel where the electronics are

- Add heat sinks to the electronics and/or steppers*

- Add active cooling*

*Some of these ideas add cost and that is a problem. You can't just keep adding cost... then it becomes the UM3. (Does the UM3 have active cooling on the PCB?)

But some of these ideas add next to ZERO cost. Changes could be made gradually as the inventory cycles.

Edited November 22, 2016 by Guest