Bug#665387: dar: Still getting error: Dates of file's data are not increasing when database's archive number grows

March 23rd, 2012 - 12:50 pm ET by Graham Cobb | Report spam
Package: dar
Version: 2.4.2-1
Severity: important

This is a followup to 637997, which is now archived so opening a new bug.

My earlier assumption that this problem was caused by the bug fixed upstream
in dar 2.4 is incorrect. I have now installed dar 2.4.2.1 (Wheezy) and this
bug is still occuring. In fact it is now much worse because it does not just
affect archives created with old versions of dar but prevents all differential
backups being taken, breaking my whole backup strategy. I will have to
downgrade to an earlier version of dar.

I created a full archive and inserted it into dar_manager. A few days later
I created an incremental archive of the same files but when I try to insert
it into dar_manager I get the following errors:

[Start of errors]
Dates of file's data are not increasing when database's archive number grows.
Concerned file is: usr/share/mime/packages/palapeli-mimetypes.xml
Dates are not increasing for all files when database's archive number grows,
working with this database may lead to improper file's restored version.
Please reorder the archive within the database in the way that the older is
the first archive and so on up to the most recent archive being the last of
the database
Dates of file's data are not increasing when database's archive number grows.
Concerned file is: usr/share/mime/packages/okteta.xml
Dates of file's data are not increasing when database's archive number grows.
Concerned file is: usr/share/doc/kde/HTML/en/ksystemlog/filter-process.png
Dates of file's data are not increasing when database's archive number grows.
Concerned file is: usr/share/doc/kde/HTML/en/ksystemlog/main-screen.png

[... lots more similar messages ...]

Dates of file's data are not increasing when database's archive number grows.
Concerned file is: usr/bin/wcgrep
Dates of file's data are not increasing when database's archive number grows.
Concerned file is: usr/include/kprofilemethod.h
Dates of file's data are not increasing when database's archive number grows.
Concerned file is: usr/lib/libgd.so.2.0.0
Error met while processing operation: Some files do not follow chronological
order when archive index increases withing the database, this can lead
dar_manager to restored a wrong version of these files
Some files do not follow chronological order when archive index increases
withing the database, this can lead dar_manager to restored a wrong version of
these files
DARsystem: Warning: Adding DARsystemDiff01 to DARsystemDataBase failed.
[End of errors]


I note that between the two archives I did do a large "aptitude upgrade",
upgrading many packages. And I note that all the files complained about are
files from packages. Most of them are .png files but there are others
(including many from /usr/bin).

Looking at the "dar -l" output from the two archives for one of the files I
note that the earlier archive shows:

[Saved] [ 52%][ ] -rwxr-xr-x root root 2387 Wed Jan 19 22:07:51 2011 usr/bin/wcgrep

And the later archive shows:

[Saved] [ 52%][ ] -rwxr-xr-x root root 2387 Tue Jan 29 09:18:04 2008 usr/bin/wcgrep

So, the problem appears to be because the later archive has an earlier mtime
for the file (year 2008 instead of 2011). However, I believe it has a later
ctime so dar appears to be using the wrong time field in its checking.

I can reproduce the bug using the following script, in a clean directory:

#!/bin/sh

mkdir a
echo "First version" >a/test
tar cf test.earlier.tar a
sleep 60
echo "Second version" >a/test
tar cf test.later.tar a

rm -rf a
tar xf test.later.tar
ls -l a/test
ls -lc a/test
ls -la a/test
dar -c first -g a
tar xf test.earlier.tar
ls -l a/test
ls -lc a/test
ls -la a/test
dar -c second -g a -A first

dar_manager -C testdb
dar_manager -B testdb -A first
dar_manager -B testdb -A second

This completely breaks dar's incremental backup feature.


Debian Release: wheezy/sid
APT prefers testing
APT policy: (990, 'testing')
Architecture: amd64 (x86_64)

Kernel: Linux 3.2.0-2-amd64 (SMP w/8 CPU cores)
Locale: LANG=en_IE@euro, LC_CTYPE=en_IE@euro (charmap=ISO-8859-15) (ignored: LC_ALL set to en_IE@euro)
Shell: /bin/sh linked to /bin/bash

Versions of packages dar depends on:
ii libattr1 1:2.4.46-5
ii libbz2-1.0 1.0.6-1
ii libc6 2.13-27
ii libdar64-5 2.4.2-1
ii libgcc1 1:4.6.3-1
ii libgcrypt11 1.5.0-3
ii libgpg-error0 1.10-3
ii liblzo2-2 2.06-1
ii libstdc++6 4.6.3-1
ii zlib1g 1:1.2.6.dfsg-2

dar recommends no packages.

Versions of packages dar suggests:
pn dar-docs 2.4.2-1
pn par2 <none>




To UNSUBSCRIBE, email to debian-bugs-dist-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
email Follow the discussionReplies 3 repliesReplies Make a reply

Similar topics

Replies

#1 Brian May
March 26th, 2012 - 06:30 pm ET | Report spam
On 24 March 2012 04:34, Graham Cobb <g+ wrote:
My earlier assumption that this problem was caused by the bug fixed upstream
in dar 2.4 is incorrect.  I have now installed dar 2.4.2.1 (Wheezy) and this
bug is still occuring.  In fact it is now much worse because it does not just
affect archives created with old versions of dar but prevents all differential
backups being taken, breaking my whole backup strategy.  I will have to
downgrade to an earlier version of dar.



Upstream had the following response. Is there any chance you can talk
to upstream directly?
http://sourceforge.net/tracker/?fun...;atidQ1612
Thanks

Hello Brian,

dar_manager expects that the mtime and ctime increases for a given file
from archive to archive in the database. mtime is used to track
modification of data, while ctime is used to track modification of inode
and Extended Attributes.

The provided script set back mtime of a/test to the date of yesterday in
the second archive, which is not a "usual" operation. In a normal system
usage, mtime, ctime and atime are modified to the date of 'now' when data
modification, inode modification or read access occurs, so as system yet
never traveled back to the past (as far as I know) the reported warning
should not show.

This "problem" shows when, the mtime or ctime is manually set to a given
time in the past, either manually by using the 'touch' command, as in the
provided script, or when restoring a file from dar archive or extracting it
from tar, which I suspect is underlying in debian package management
tools.

A workaround is to use dar_manager's -ai option, which appeared in release
2.4.3 (but please directly use 2.4.4 as release 2.4.3 is broken about its
ability to read dar_manager database it generates). This option does not
solve any problem, it only hides the warning:

Why the problem is not solved? As stated above, if the order of dates is
not increasing with archive number for the mtime of a particular file, -w
option of dar_manager may not work as expected for that particular file, as
it may find two or more versions "just before" the date provided with -w
option, and will arbitrarily restore the first one it met.

If we assuming the user did not make any mistake in the order archive got
provided in spite of the non increasing order of mtime dates for a
particular file. A more sophisticated work with dates should be done about
the lookup of file's version at a particular date [upon feature request,
that could be the object of a feature in 2.5.x]. And then if the user did
really made a mistake, there is no way to warn the user as we assumed the
order is correct.

So there is a choice to do between an expected use of dates and user error
possible to detect and on the other hand, an unusal play with dates and the
impossibility to report errors that could leading to restoring the wrong
version of a file.

I just wonder why upgrading a package in Debian leads to have some files
with older mtimes? Couldn't be possible that restoring a package makes the
newly created file get as mtime the current date (the date of the upgrade)?
I suspect this is what would naturally do the operating system if the

Note also, that some particular security tools also expect the mtime not to
be set back in the past.

Regards,
Denis.
Brian May



To UNSUBSCRIBE, email to
with a subject of "unsubscribe". Trouble? Contact
Replies Reply to this message
#2 Graham Cobb
April 16th, 2012 - 05:40 pm ET | Report spam
I have added a comment/suggestion to the upstream bug report.



To UNSUBSCRIBE, email to
with a subject of "unsubscribe". Trouble? Contact
Replies Reply to this message
#3 Graham Cobb
April 22nd, 2012 - 05:10 am ET | Report spam
Upstream has reported that the workround for the problem is a new option ("-
ai") which was introduced in 2.4.3 (although he also reports that 2.4.4 should
be used due to a bug in 2.4.3).

Could the version with the "-ai" option be brought into Testing? Without this
option, anyone who sees the situation of a file's mtime going backwards cannot
get their dar archive inserted into dar_manager at all. Previous releases
worked because they did not report this so-called error so this is a definite
regression. If we cannot go to 2.4.4 for the next Debian release then a small
patch just to disable the error message may be the answer.

Discussion on the problem itself continues upstream. My personal view is
that the -ai option should be the default (or the message removed altogether
as I believe it causes more problems than it fixes).



To UNSUBSCRIBE, email to
with a subject of "unsubscribe". Trouble? Contact
email Follow the discussion Replies Reply to this message
Help Create a new topicReplies Make a reply
Search Make your own search