Is dump deprecated?
Startseite -> Computer -> Is dump deprecated?

Dump and restore build for me the most important thing after the OS itself: A well-functional backup with the option to easily restore the system in whole or in part after a failure.

From the MAINTAINERS file:

The dump and restore backup suit was ported to Linux's Second
Extended File System by Rémy Card <card@Linux.EU.Org>

He was the maintainer of the initial versions of dump (up and
including 0.4b4, released in january 1997).

I decided to continue the developpement and release a new version.
Rémy didn't respond to any mail I sent to him, so I have
promoted myself as the new maintainer :)

So, starting with 0.4b5, the new maintainer is me,
Stelian Pop <stelian@popies.net>.
On ->LWN.net, there's an article where Linus Torvalds states why dump is bad and stupid.

Since I'm using dump at home as well as professional, I wanted to get more information on this.
So I wrote a mail to Stelian, starting a small conversation which I will partly reflect here with kind permission from Stelian, to clarify some points.


Date: Wed May 22 22:18:22 2002
To: Stelian Pop <stelian@popies.net>
From: Patrik Schindler <poc@pocnet.net>
Subject: Dump dishonoured by Linus?

Hi there,

please have a look at the following URL:
http://lwn.net/2001/0503/a/lt-dump.php3

Okay, it's a bit old, but a colleague showed me this. Now, I'm looking forward to some kind of statement from you :-)

Perhaps it has something to do with regular syslog messages:
May 22 01:46:54 leela kernel: invalidate: busy buffer
May 22 01:47:32 leela kernel: invalidate: dirty buffer

And so on. Linus told me last year that these messages were not critical. They were invisible before but now there's a printk in certain places which spams my syslog. :-)

:wq! PoC


Date: Sat, 25 May 2002 09:30:10 +0200
From: Stelian Pop <stelian@popies.net>
To: Patrik Schindler <poc@pocnet.net>
Subject: Re: Dump dishonoured by Linus?

On Wed, May 22, 2002 at 10:18:22PM +0200, Patrik Schindler wrote:

> Hi there,
>
> please have a look at the following URL:
> http://lwn.net/2001/0503/a/lt-dump.php3
>
> Okay, it's a bit old, but a colleague showed me this. Now, I'm looking forward to some kind of statement from you :-)

Yes, it's an old one, and I've given statements several times since then :-)

> Perhaps it has something to do with regular syslog messages:
> May 22 01:46:54 leela kernel: invalidate: busy buffer
> May 22 01:47:32 leela kernel: invalidate: dirty buffer

No. Linus just tells everyone something we already know for a long time:
that dumping an active (mounted) filesystem is not safe, because the kernel is modifying the filesystem at the same time as dump is reading it.

When dumping active filesystems, there is a slight possibility that dump will either:

In practice, those things happen rarely, and since backups are supposed to be done every day, it doesn't really matter if one dump has failed (and generally it's more common that it fails due to a bad tape. And you're really supposed to verify your backups anyway).

What Linus also says is that the page->buffer cache change which happened earlier in the 2.4 kernels timeframe will affect in theory dump, which will fail more often. In practice, I didn't see that (this could also be because, when the page->buffer change occured, there was a slight timeframe - 3 or 4 kernels - when the VFS did go through the buffer cache, but reading directly the raw filesystem, as dump do, still did go thgough the page cache. At that time, since there were more pathways and more caches involved, dump had a bigger chance to fail. But this has been changed since.).

> And so on. Linus told me last year that these messages were not critical. They were invisible before but now there's a printk in certain places which spams my syslog. :-)

Some of them were not critical but some of them were caused by genuine kernel bugs which were fixed rather recently. A VFS bug was found for example by Stephen Tweedie some weeks ago. The RedHat 7.3 kernels contains the fix, or go read:

->http://sourceforge.net/mailarchive/forum.php?thread_id=727438&forum_id=7176

Stelian.
--
Stelian Pop <stelian@popies.net>


Date: Sat May 25 13:15:38 2002
To: Stelian Pop <stelian@popies.net>
From: Patrik Schindler <poc@pocnet.net>
Subject: Re: Dump dishonoured by Linus?

Hello,

>Yes, it's an old one, and I've given statements several times since then :-)

Oh. Did I overread some FAQ or so?

>No. Linus just tells everyone something we already know for a long time:
>that dumping an active (mounted) filesystem is not safe, because the
>kernel is modifying the filesystem at the same time as dump is reading
>it.

Ah. He didn't exactly clarify that fact, that he's writing about mounted filesystems in special. On the other hand, if some task is modifying a file, tar would also run into some consistency problems as well.

So what is all the panic about?

I *know* that when I'm dumping, I should take care that it would be a time of day where activity on the machine has ceased to a minimum to ensure consistency.

>In practice, those things happen rarely, and since backups are supposed
>to be done every day, it doesn't really matter if one dump has failed
>(and generally it's more common that it fails due to a bad tape. And
>you're really supposed to verify your backups anyway).

And one has to take care while doing large 0-dumps to cease activity to a minmum for a long time which is sometimes hard to accomplish.

>Some of them were not critical but some of them were caused by genuine
>kernel bugs which were fixed rather recently. A VFS bug was found
>for example by Stephen Tweedie some weeks ago. The RedHat 7.3 kernels
>contains the fix, or go read:
>http://sourceforge.net/mailarchive/forum.php?thread_id=727438&forum_id=7176

Uh, if I read this right, than there's a chance, that thru the massive calling of kernel functions (which do the printk) a FS can get corrupted? This is hard. Okay, it hasn't happened here before. I hope 2.4.19 will be out soon...

Thanks for the pointer, I'll look if I can put this into 2.4.18-rc4 (the *real* 2.4.18) until 2.4.19 is released.

:wq! PoC


I looked at the articles and tried to easily incorporate the fix into 2.4.18. I found that it would take too much time, so I decided to wait for 2.4.19. Fortunately I had not found corruptions on no machine of mine.


Date: Sat, 25 May 2002 13:35:30 +0200
From: Stelian Pop <stelian@popies.net>
To: Patrik Schindler <poc@pocnet.net>
Subject: Re: Dump dishonoured by Linus?

On Sat, May 25, 2002 at 01:15:38PM +0200, Patrik Schindler wrote:

> Oh. Did I overread some FAQ or so?

Not really. It was discussed on the dump-users mailing list and the sourceforge forums IIRC.

> Ah. He didn't exactly clarify that fact, that he's writing about mounted filesystems in special.

He was. Dumping an unmounted filesystem always worked and will always work.

> On the other hand, if some task is modifying a file, tar would also run into some consistency problems as well.

Less problems, since the task modifying the file and tar reading the file will pass through the same kernel layers which will ensure consistency (so tar will see some 'snapshot' version of the file, as it was when it was readed).

> So what is all the panic about?

There is no panic :-)

> I *know* that when I'm dumping, I should take care that it would be a time of day where activity on the machine has ceased to a minimum to ensure consistency.

Yep, it was true before and it is still true.

> And one has to take care while doing large 0-dumps to cease activity to a minmum for a long time which is sometimes hard to accomplish.

I know, but the only way to go in these cases is to use filesystem snapshots (evms, lvm etc). By design, dump cannot do better than it does now...

> Uh, if I read this right, than there's a chance, that thru the massive calling of kernel functions (which do the printk) a FS can get corrupted?

Exactly, yes. I believe that it was pretty easy to reproduce on pretty loaded SMP machines...

> This is hard. Okay, it hasn't happened here before. I hope 2.4.19 will be out soon...

Probably in a month or so... The next scheduled release from Marcelo should be the rc1.

Stelian.
--
Stelian Pop <stelian@popies.net>


Date: Sat May 25 13:48:00 2002
To: Stelian Pop <stelian@popies.net>
From: Patrik Schindler <poc@pocnet.net>
Subject: Re: Dump dishonoured by Linus?

>Less problems, since the task modifying the file and tar reading the file
>will pass through the same kernel layers which will ensure consistency
>(so tar will see some 'snapshot' version of the file, as it was when
>it was readed).

In the tar case the dumped file will be more or less useless, because corrupt. I think it's no difference to the dump case.

>There is no panic :-)

Fine :-)

>I know, but the only way to go in these cases is to use filesystem
>snapshots (evms, lvm etc). By design, dump cannot do better than it does
>now...

Yes. Perhaps I should look forward to some shnapshotting aid...

>Exactly, yes. I believe that it was pretty easy to reproduce on pretty
>loaded SMP machines...

Loaded: Much Disk-I/O? Okay, but as stated, dump dislikes active FSs anyways, so it may be a good rule of thumb to say: Dump in quiet times. :-)

:wq! PoC


Date: Sat, 25 May 2002 15:13:40 +0200
From: Stelian Pop <stelian@popies.net>
To: Patrik Schindler <poc@pocnet.net>
Subject: Re: Dump dishonoured by Linus?

On Sat, May 25, 2002 at 01:48:00PM +0200, Patrik Schindler wrote:

> In the tar case the dumped file will be more or less useless, because corrupt. I think it's no difference to the dump case.

There is a slight difference: tar will save a partial content of the file; dump will probably save just junk.

> Yes. Perhaps I should look forward to some shnapshotting aid...

I heard good things about EVMS. Backed-up by IBM, proved technology (a considerable percentage of the code has already been used for years in AIX systems). Unfortunately I didn't get the time to give it a shot yet...

> Loaded: Much Disk-I/O? Okay, but as stated, dump dislikes active FSs anyways, so it may be a good rule of thumb to say: Dump in quiet times. :-)

Yes. The original redhat bug where this was reported is there:

->http://bugzilla.redhat.com/bugzilla/show_bug.cgi?id=62281

Stelian.
--
Stelian Pop <stelian@popies.net>


Date: Sat May 25 23:09:29 2002
To: Stelian Pop <stelian@popies.net>
From: Patrik Schindler <poc@pocnet.net>
Subject: Re: Dump dishonoured by Linus?

At 15:13 Uhr +0200 25.05.2002, Stelian Pop wrote:

>There is a slight difference: tar will save a partial content of the
>file; dump will probably save just junk.

Hm. In my point of view this is the same. If a file doesn't contain what it is expected to, than it's junk. May it be partially right or total junk.

>I heard good things about EVMS. Backed-up by IBM, proved technology
>(a considerable percentage of the code has already been used for years
>in AIX systems). Unfortunately I didn't get the time to give it a shot
>yet...

Thanks for the pointer! :-)

>Yes. The original redhat bug where this was reported is there:
>->http://bugzilla.redhat.com/bugzilla/show_bug.cgi?id=62281

This is rather ugly. Fortunately it's not that reproducible here.
Perhaps you should mention in the manpage that dumping directories (which aren't mountpoints) is risky if the filesystem is active.

:wq! PoC


Date: Sun, 26 May 2002 19:37:56 +0200
From: Stelian Pop <stelian@popies.net>
To: Patrik Schindler <poc@pocnet.net>
Subject: Re: Dump dishonoured by Linus?

On Sat, May 25, 2002 at 11:09:29PM +0200, Patrik Schindler wrote:

> Hm. In my point of view this is the same. If a file doesn't contain what it is expected to, than it's junk. May it be partially right or total junk.

When you save, let's say, a cleartext file being modified, with tar you'll get a cleartext partial file, with dump you can get random binary data. The cleartext partial file can be useful if this is all that's left from your document.

> This is rather ugly. Fortunately it's not that reproducible here.
> Perhaps you should mention in the manpage that dumping directories (which aren't mountpoints) is risky if the filesystem is active.

??? The above bug report is not related to directories which aren't mountpoints ?

Stelian.
--
Stelian Pop <stelian@popies.net>


Date: Mon May 27 00:39:53 2002
To: Stelian Pop <stelian@popies.net>
From: Patrik Schindler <poc@pocnet.net>
Subject: Re: Dump dishonoured by Linus?

At 19:37 Uhr +0200 26.05.2002, Stelian Pop wrote:

>When you save, let's say, a cleartext file being modified, with tar
>you'll get a cleartext partial file, with dump you can get random
>binary data. The cleartext partial file can be useful if this is
>all that's left from your document.

Okay, maybe a point. Usually I'll have a backup from last day of this textfile :-)

>??? The above bug report is not related to directories which aren't
>mountpoints ?

Sorry, misunderstanding!

In the section which states that one can dump filesystems or subdirectories, it would be meaningful to state that dumping subdirectories is risky. Usually no one unmounts filesystems when doing subdir-backups.

If one dumps a subdirectory which is also a mountpoint, it's no more a subdir, dump uses the whole device directly.

Perhaps it's possible to dump subdirs even if the parent is not mounted (not tested). Than I wrote junk. :-)

:wq! PoC


Date: Mon, 27 May 2002 17:40:32 +0200
From: Stelian Pop <stelian@popies.net>
To: Patrik Schindler <poc@pocnet.net>
Subject: Re: Dump dishonoured by Linus?

On Mon, May 27, 2002 at 12:39:53AM +0200, Patrik Schindler wrote:

> If one dumps a subdirectory which is also a mountpoint, it's no more a subdir, dump uses the whole device directly.

There is exactly no difference between dumping a 'whole device' or dumping only a subdir (from the "risky" point of view).

> Perhaps it's possible to dump subdirs even if the parent is not mounted (not tested). Than I wrote junk. :-)

No, you cannot dump a subdir of an unmounted filesystem.

Stelian.
--
Stelian Pop <stelian@popies.net>


Date: Tue May 28 23:05:55 2002
To: Stelian Pop <stelian@popies.net>
From: Patrik Schindler <poc@pocnet.net>
Subject: Re: Dump dishonoured by Linus?

At 17:40 Uhr +0200 27.05.2002, Stelian Pop wrote:

>There is exactly no difference between dumping a 'whole device' or
>dumping only a subdir (from the "risky" point of view).

A small is: For the whole device one may unmount it. For the subtree not, so subtree is always risky (if not remounted ro).

>No, you cannot dump a subdir of an unmounted filesystem.

I guessed right. :-)

:wq! PoC


Date: Thu, 30 May 2002 13:44:38 +0200
From: Stelian Pop <stelian@popies.net>
To: Patrik Schindler <poc@pocnet.net>
Subject: Re: Dump dishonoured by Linus?

On Tue, May 28, 2002 at 11:05:55PM +0200, Patrik Schindler wrote:

> A small is: For the whole device one may unmount it. For the subtree not, so subtree is always risky (if not remounted ro).

Ok, now I finally see what you meant, and I agree with that ! :-)

Stelian.
--
Stelian Pop <stelian@popies.net>


That's it.

If you want to further use dump or not depends on your opinion. The technical facts should be clear now.

You may also take a look at ->Freshmeat what other backup programs are available. Most of them are rather new, which usually means they're tested only by a limited number of people. I had a look on some of them, some depend on tar or cpio - with the known limitations of these tools. Dump is simple and worked for me many times. I know it's flaws, so I will stay with it.

© by Patrik Schindler <webhamster@pocnet.net>, 1999-2018