Root partition is suddenly 100% full - can't log in

Asked by Jim Hutchinson on 2007-08-20

I have ubuntu set up with a 150GB /home and a 20GB / partition. Until today that / partition was about 25% full. Suddenly it's 100% without a single byte free. I have no idea why or how this happened. I have booted a live CD and am exploring the / partition but I cannot find any large usage of space. Is there a way to search the partition for large files? Any idea how this can happen? I've been using ubuntu for about 2 years and never seen this before.

Thanks.

Question information

Language:
English Edit question
Status:
Answered
For:
Ubuntu Edit question
Assignee:
No assignee Edit question
Last query:
2007-08-20
Last reply:
2007-08-21
Jim Hutchinson (jphutch) said : #1

Okay, I may have solved this. I installed the simple backup tool earlier this week and set it up to write backups to a second HD. The partition was /dev/sdb2 and was supposed to be mounted at /media/sdb2. However, when I ran the backup, I don't think /dev/sdb2 was mounted. So rather than writing the backup to the separate device, I think it wrote the backup to a directory /media/sdb2.

Is that a plausible solution? I booted the recovery mode and then deleted everything in /media/sdb2. After that the / partition was back to 75% empty so I guess that's what happened.

I'm leaving this open for a bit to see if anyone can confirm this and to suggest a better way to run backups.

Thanks.

Cesare Tirabassi (norsetto) said : #2

Couple of observations Jim:

1) it seems strange that a backup of 25% of the disk filled it up completely; perhaps it was a recursive backup?

2) there should have been a reserve set on that HD, so that you could have logged as root and investigated. Did you try that?

Jim Hutchinson (jphutch) said : #3

1) the back up was of my /home partition which is about 60GB in size and should have been written to a second hard drive that seems to have not been mounted at the time of the backup so it defaulted to an actual directory on /media rather than a mounted device. It seems to have written about 16GB to my / partition and filled it up. I was only able to observe this from a recovery console and only after a bit of investigating since the directory "looked" like my mounted backup partition but in fact was actually written to a directory on /. Confusing.

2) I was not at all able to log in. I got the "your session lasted less than 10 seconds" message. Booting the recovery mode I was able to use df and see that indeed the / partition was 100% full. There doesn't seem to have been any "reserve" space. Should that be set up by default or is it something I do manually if I want it? If it should have been automatic - it wasn't, so that suggests a bug.

Thanks for the feedback. I seem to love screwing up my system (but I also love that I'm getting better and figuring it out). This one really had me confused for a while and troubleshooting from CLI was an experience.

Cesare Tirabassi (norsetto) said : #4

The reserve is something you setup, usually during partitioning. If you used gparted, or you partitioned during install, you should have set an amount (default 5% IIRC) aside which is indeed what made it possible for you to login in recovery mode.
You can see that for instance if you do a df, and you will see that available is not total minus used.
You can change the allocation also using a tool, like tune2fs.

peter b (b1pete) said : #5

Hello Cesare, Jim,

Far from me in getting into Jim's problem -it looks that it's already sorted out. Just a comment

I know well from my own past experience the frustrations related to backups/restore operations -they are pretty tricky to say the least, every backup/restore app be it under win or linux has a 'mind' of its own which MUST be 'mastered' well before proceeding (and this goes well back to the time of DOS diskettes, tape drives etc). So, after a couple of 'incidents' I decided to get to the bottom of this and find a reliable and, v important, logical easy to understand and use app that fullfils that important task.

For more than 5 years I'm using to backup/restore partitions/files to/from image files either to CD/DVD or other HD's in the latter case prefferably external HD's. I must say that since I took that decision that 'chapter' called backup/restore is for all tense and purposes closed for me -no more surprises and 'incidents'. Basically, I'm using a utility written under the good old dos by terabyte

http://www.terabyteunlimited.com

that installs beautifully from guess what -a good old 1.44 floppy creating an extended MBR and installing itself in less than 8MB that does marvellous things. I'm not going into the nitty-gritty of it; it is available for trial purposes and if the user likes it it can be had for a nominal amount. Under the same url one can find also Image for DOS and Image for Linux apps. I'm using BootitNG that basically is a 'all-in-one' type of app. Give it a try Gents and tell me here what's your opinion.

Best regards,
peter b

Can you help with this problem?

Provide an answer of your own, or ask Jim Hutchinson for more information if necessary.

To post a message you must log in.