Jump to content
Ultimaker Community of 3D Printing Experts
  • Sign Up
SmartBlug

Cura not working - No space left on device

Recommended Posts

Posted · Cura not working - No space left on device

I cannot use my printer anymore (Extended 3)

 

On the log :

ultimakersystem-ccbdd300033b flipflop_wsgi.py[1314]: OSError: [Errno 28] No space left on device: '/var/lib/griffin/cluster/preferences.json.new'

 

If I connect in dev mode : 

/dev/mmcblk1p2    963584 828416         0 100% /
devtmpfs          481308      0    481308   0% /dev
tmpfs             515100      0    515100   0% /dev/shm
tmpfs             515100   7840    507260   2% /run
tmpfs               5120      0      5120   0% /run/lock
tmpfs             515100      0    515100   0% /sys/fs/cgroup
tmpfs             515100      0    515100   0% /var/lib/apt
tmpfs             515100      0    515100   0% /tmp
tmpfs             515100      0    515100   0% /var/cache/apt
/dev/mmcblk1p3   2775040  26632   2527224   2% /var/spool/cluster
 

Is there a way to clean the file system ?

 

Thanks,

Share this post


Link to post
Share on other sites
Posted · Cura not working - No space left on device

Have you tried a reboot of the printer? Maybe there are some cleanups during the boot process, but I don't know.

Share this post


Link to post
Share on other sites
Posted · Cura not working - No space left on device

The only idea I have left is that you check /var/log for large logfiles. I would do that at least on a normal Linux system.

But I've never heard of the printer running out of disk space. Maybe someone else knows more, sorry!

Share this post


Link to post
Share on other sites
Posted · Cura not working - No space left on device

I deleted some of the logs and get back 100k...

 

/var/log/journal is big, how to safely delete this journal ?

 

size of main folders :

526231    /
215146    /usr
168876    /var
99980    /usr/lib
91755    /var/lib
87435    /usr/share
83935    /var/lib/connman
83910    /var/lib/connman/ethernet_ccbdd300033b_cable
75247    /var/log
74847    /var/log/journal
74844    /var/log/journal/5e702514274833c0ebe690cc5ccc224d
62364    /opt
53676    /lib
50743    /opt/qt
46852    /usr/share/fonts
42043    /opt/qt/lib
39929    /usr/lib/python3
39926    /usr/lib/python3/dist-packages
37227    /usr/share/fonts/opentype
37224    /usr/share/fonts/opentype/noto
27626    /usr/lib/arm-linux-gnueabihf
26975    /lib/modules
26971    /lib/modules/4.14.32-ultimaker-00309-geaace6d4aede
25823    /lib/modules/4.14.32-ultimaker-00309-geaace6d4aede/kernel
23787    /usr/bin
21829    /usr/lib/python3.4
14635    /lib/modules/4.14.32-ultimaker-00309-geaace6d4aede/kernel/drivers
13239    /usr/lib/python3/dist-packages/numpy
12523    /usr/share/i18n
11618    /opt/pyqt
11568    /lib/udev
11428    /opt/pyqt/PyQt5
9486    /usr/share/i18n/locales
8704    /lib/arm-linux-gnueabihf
8289    /usr/share/okuda
8036    /usr/lib/python3/dist-packages/cluster
7911    /usr/lib/python3/dist-packages/cluster/app
7840    /run
7412    /usr/share/griffin
7267    /usr/lib/python3/dist-packages/sqlalchemy
7237    /usr/share/okuda/resources
6606    /usr/lib/python3/dist-packages/cluster/app/static
6440    /run/log/journal/5e702514274833c0ebe690cc5ccc224d
6440    /run/log/journal
6440    /run/log

 

Share this post


Link to post
Share on other sites
Posted · Cura not working - No space left on device

You mean the directory /var/log/journal ?

It should be safe to delete log files inside the directory, but keep the directory itself. 

 

Are there many log files inside? Because normally there is log rotation, maybe not working or configured on the printer.

If there is only one big logfile, you can also truncate the file with just

> logfilename

instead of delete them. The > is important which means you redirect the input, which is nothing to the file, resulting in a 0 byte file. But just a delete should also be no problem.

 

But I am not sure if this is enough, the directory has 74MB. Maybe there is something else which consumes so much disk space.  

Share this post


Link to post
Share on other sites
Posted · Cura not working - No space left on device

I removed the folder in /var/log/journal and get back 58Mb 😉

It seems it was logs not properly deleted in the past by the software (probably due to bug in old releases)

 

now, only

-rw-r-----  1 root systemd-journal 8388608 Dec 15 07:40 system.journal
-rw-r-----+ 1 root systemd-journal 8388608 Dec 15 07:38 user-1000.journal
 

I'll monitor to see if this folder size continue to increase or not...

Share this post


Link to post
Share on other sites
Posted · Cura not working - No space left on device

I have the same problem.  I suspect it's avahi-daemon logging too much (see below):

 

- I was printing fine

- after some number of prints Cura Connect would break

- reading logs on the touchscreen showed that no space was left on /

- couldn't enable dev mode to ssh in

- would reset cura connect — losing all my stats — printer would work again for a couple of days, then run out of space again

- this time though I am able to SSH in and can verify that /var/log/journal/** is taking up 89MB of 806 available on /

 

root@ultimakersystem-ccbdd30087d3:/var/log/journal/c9c140d1e794462fbc6facaa6f36863f# ls -l
total 90252
-rw-r-----  1 root systemd-journal   53248 Jan  1  1970 system.journal
-rw-r-----  1 root systemd-journal 8388608 Jan  1  1970 system@00000008ccb660c6-1b488519ed25a7ee.journal~
-rw-r-----  1 root systemd-journal 8388608 May 21 17:32 system@8a780352b4d74a25abba2411be124d36-000000000001d96a-0005a62956cc9fe7.journal
-rw-r-----  1 root systemd-journal 8388608 May 21 21:06 system@8a780352b4d74a25abba2411be124d36-0000000000020038-0005a62be43bd14a.journal
-rw-r-----  1 root systemd-journal 8388608 May 22 00:00 system@8a780352b4d74a25abba2411be124d36-000000000002253e-0005a62ee1b94778.journal
-rw-r-----  1 root systemd-journal 8388608 May 22 00:32 system@8a780352b4d74a25abba2411be124d36-0000000000025072-0005a631525ec98c.journal
-rw-r-----  1 root systemd-journal 8388608 May 22 01:04 system@8a780352b4d74a25abba2411be124d36-0000000000027c8b-0005a631c3fec32d.journal
-rw-r-----  1 root systemd-journal 8388608 May 20 23:35 system@ebae5b2b5a4646dabb9d3c59a8d6fe95-0000000000018b9f-0005a61aead30f5f.journal
-rw-r-----  1 root systemd-journal 8388608 May 21 03:07 system@ebae5b2b5a4646dabb9d3c59a8d6fe95-000000000001b68c-0005a61cd98f111f.journal
-rw-r-----+ 1 root systemd-journal 8388608 May 22 01:08 user-1000.journal
-rw-r-----+ 1 root systemd-journal 8388608 May 22 01:04 user-1000@6376e02ffff348eea29dbb59865292ca-0000000000000000-0000000000000000.journal
-rw-r-----+ 1 root systemd-journal 8388608 May 21 03:07 user-1000@6376e02ffff348eea29dbb59865292ca-000000000001d767-0005a61fba8f8837.journal

 

Tailing logs with journalctl -r, I see that avahi-daemon is spamming useless logs to disk:

Jun 01 21:54:15 ultimakersystem-ccbdd30087d3 avahi-daemon[1214]: Invalid response packet from host 192.168.1.137.
Jun 01 21:54:15 ultimakersystem-ccbdd30087d3 avahi-daemon[1214]: Invalid response packet from host 192.168.1.32.
Jun 01 21:54:15 ultimakersystem-ccbdd30087d3 avahi-daemon[1214]: Invalid response packet from host fe80::4da:c6cf:bdab:6026.
Jun 01 21:54:15 ultimakersystem-ccbdd30087d3 avahi-daemon[1214]: Invalid response packet from host fe80::21:efd5:4317:132e.
Jun 01 21:54:15 ultimakersystem-ccbdd30087d3 avahi-daemon[1214]: Invalid response packet from host fe80::4aa6:b8ff:fe83:bbfc.
Jun 01 21:54:15 ultimakersystem-ccbdd30087d3 avahi-daemon[1214]: Invalid response packet from host 192.168.1.173.

Also other systems are logging too much to STDOUT.

Share this post


Link to post
Share on other sites
Posted (edited) · Cura not working - No space left on device

For others who want to debug here are some commands you might find useful:

df -h                      # shows disk usage in human-readable terms
journalctl --disk-usage    # shows how much disk space logs are taking
ls -l /var/log/journal/**/ # list all journald logs
rm /var/log/journal/**/*@* # delete all archived logs (logs containing an @ symbol)

Removing those logs saved me a bunch of space.  I also edited /etc/systemd/journald.conf and halved SystemMaxUse to 32M, but that didn't seem to really work.

Edited by Whitespace
typo
  • Like 1

Share this post


Link to post
Share on other sites
Posted · Cura not working - No space left on device

I suspect that avahi-daemon is logging too much.  The installed version is 0.6.31, and the very next version — 0.6.32 — fixes these warnings from appearing in logs: https://github.com/lathiat/avahi/blob/master/docs/NEWS#L155

 

The installed version is over four years old :(

 

I can't install it via apt-get, so I disabled avahi-daemon by modifying /etc/default/avahi-daemon and stopping it with update-rc.d avahi-daemon stop

  • Like 1

Share this post


Link to post
Share on other sites
Posted · Cura not working - No space left on device

@Whitespace what kind of printer do you have?  Your profile says you have only an S5?  Is this on your S5?

Share this post


Link to post
Share on other sites
Posted · Cura not working - No space left on device

@CarloK - any ideas about this?

Share this post


Link to post
Share on other sites
Posted · Cura not working - No space left on device

I'm currently working on another project, not on the S3/S5 so anything I mention here is from own experience.

 

The bad part is, we are using old software. The good part is, we are using old software.

Many bugs have been solved since the original release and I'd like to see many of our libraries updated to newer versions. The good part is that the software is working on thousands of printers with relative few problems and most of the problems are perhaps annoying, but at least known. 

 

@Whitespace what changed in your setup for these errors to start occuring? Is it a new printer or did you perform a software update?
What firmware version are you running?

Share this post


Link to post
Share on other sites
Posted · Cura not working - No space left on device

Yes this is on an S5. I haven’t had it for a while, so I don’t know how often it’s had this problem. I upgraded it to the latest firmware (5.5?) before the most recent 5.6.

 

It has had space issues since 5.5, but I can’t confirm it had it prior to that.

 

I will say that the network disconnection issues seem to be very related to the space issues.

 

Cura Connect reset works, but if this happens again I’ll try a factory reset.

 

Also I recently upgraded to 5.6 so I will see how that works.

 

Thanks all!

Share this post


Link to post
Share on other sites
Posted · Cura not working - No space left on device

I have this same issue on an S5.  The logs fill up with ~85MB over several days, filling the device entirely.  At that point, the S5 shows as being on the network, but is basically inoperable.

 

There are three ways I've found to get the printer back into an operable state:

 

1) Factory Reset

2) Cura Connect Reset

3) SSH into the printer, and clean the log files directory, then reboot the printer (via the CLI)

 

Once one of those three steps is taken, the printer will be operable again for between 2 and 10 days, at which point the cycle repeats.

 

The root cause appears to be network connectivity issues - the printer appears to have network connectivity issues when under load (the laptop next to the printer does not exhibit any connectivity issues, and the printer is showing full signal).  I see excessive avahi logs, as well as the printer's software having trouble getting firmware version information, or hitting its own API, causing it to log large python stack traces.

 

Long term, it would be great to get a more stable network connection, as this has other issues (Cura will frequently not be able to show the status of the print, despite the printer thinking it has a network connection - this will come and go as a print is operating).  Short term, monitoring the size of the log directory and removing old files, or modifying the rotation settings would be a great help.

  • Like 3

Share this post


Link to post
Share on other sites
Posted · Cura not working - No space left on device

Quick update: after upgrading to 5.6 I have not had any space or network connectivity issues.

Share this post


Link to post
Share on other sites
Posted · Cura not working - No space left on device

Unfortunately, I am on 5.7.2, and continue to see the issue.  I have not seem any changes in the issue in the last 12 months, and I generally upgrade to the latest firmware within a few weeks of its release.

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

  • Our picks

    • Talking additive | The 3D printing podcast
      Why should we be the only ones asking questions? Join us and ask Jabil all your questions on September 22nd 5pm CET
        • Like
      • 2 replies
    • Ultimaker masterclass: Optimizing your Ultimaker Cura workflow
      Save your seat for either broadcast on September 23.
      What will you learn?
      · Best practices for iterative print preparation and every lesson you should learn from each print
      · The right way to use per-object settings and when they are most useful
      · Easy-to-use resources for anyone who wants to develop their own printer definitions, plugins, or print profiles
      · How to optimize print profile settings and whether to “keep” or “discard” changes
      · When is the right time to export your drawing from CAD? (Based on Ultimaker Cura’s surprising power as 3D control software)
      · And a whole lot more tips and tricks!
       
      How can I join?
      This free masterclass will take place twice:
      1. 11am CEST (5pm SGT, 5am EDT)
      2. 5pm CEST (11am EDT, 8am PDT, 11pm SGT)
        • Like
      • 0 replies
    • Do you use 3D printing at work? Let us know
      It doesn't matter if you are using 1 Ultimaker or 10, there is inspiration in everything. We're looking for ...
      • 1 reply
×
×
  • Create New...