[Samba] Robocopy from Windows to Samba (3.6.3) with backup flag

On Tue, Mar 13, 2012 at 11:27:07AM +0100, Vincent Miszczak wrote:

Hello,

I need to copy a Windows NAS to a Samba one preserving all stuffs (dates, owner, security, etc...) in an enterprise environment.

I'm used to do that with between two Windows using robocopy and the /b (backup) flag, so I can backup files even if I do not have an ACE for my account as I have the backup and restore privileges.

I need to do the same thing from Windows to Samba but using the backup flag does not work at all : it does not even copy the data. If I copy without the backup flag, it's OK for the files I have access, but as I am in an enterprise, I don't have access to all files.

Here what happens :
(From the Windows NAS) :

robocopy G:\share\XXXXX\04_generique \\samba-nasoot$\xfs\shares\archives\XXXXX\04_generique /V /NS /NC /NDL /NFL /S/E /COPY:DATS /B /NP /XJ /R:0 /W:30

2012/03/13 10:56:18 ERROR 87 (0x00000057) Copying NTFS Security to Destination Directory G:\share\ XXXXX \04_generique\
The parameter is incorrect.


ð No data is copied :/

If do the same without the /B flag, it's OK in this particular case, but I don't have access to all the data and I won't be able to.

I have tested on Centos 6 with Samba 3.5.10 and Samba 3.6.3 and I have the same result.

Is this supposed to work ?

Not in 3.6.x yet (it's being fixed for 4.0 and may get back-ported).
SeBackup/SeRestore require some special case code to ensure this
is done securely with no security holes or races.

Are there "special" parameters in smb.conf for this to work ?
If not, how can I backup from Windows to Samba ?

Currently the best way is to create a custom share, which
uses "force user = root" and is set with the valid users
set to those users who have SeRestore privilege.

This works, but I agree it's a little clunky. I'm working
on it.

Jeremy.
To unsubscribe from this list go to the following URL and read the
instructions: https://lists.samba.org/mailman/options/samba

Thank you for your response.

We are used to have root$ share force root user for admins.
Result is the same with and without force user.

Just using /B flag without /COPY:DATS (ie DAT), result is a bit different (no more "incorrect parameter" but "access denied" with the filename):

87% New File 298099 Generique_Va_GameOverDeath.png
2012/03/13 18:24:20 ERROR 5 (0x00000005) Copying File G:\share\XXXX\04_generique\Generique_Va_GameOverDeath.png
Access is denied.

(This does not happens on all files but it happens a lot and I can't say where is the difference as some files of one folder are copied but others not).

As this point the file data is copied, but date and ACLs are not.

I have even tested with the raw Windows C API BackupRead/BackupWrite (with correct token privileges) and BackupWrite returns false for files failing with robocopy.

I also have other problems with this fu***** closed source Robocopy program.

I have tested all possible configuration, whenever I use it without backup mode, folder ACLs are not copied and files that inherit ACLs get the parent's not copied ACLs, ie ACLs are not copied.
(robocopy options used are /S /E /COPY:DATS)

Using xcopy with /O (ownership and ACLs copy), ACLs are copied on both folders and files with the same smb.conf config but I really need to apply a rsync like (purge, ACL fix, timefix) program because I'm syncing a big bunch of TB with millions of living files. Having tested rsync with Cygwin on Windows in the past, it could not handle path longer than 256 chars and we use larger ones so I can't use it (unless it is fixed).

Doing a debug 10 level does not seem to help.

I'm stuck :/ and any help would be appreciated.

Here is my configuration :

[global]
load printers = no
printing = bsd
printcap name = /dev/null
disable spoolss = yes
log file = /var/log/samba/log.%m
#log level
max log size = 50
load printers = no
netbios name = cifs-anim-arch
server string = %h
workgroup = XXXX
realm = XXXX.LAN
password server = dc-xxx.xxx.lan dc-xxx.xxx.lan
security = ads
use kerberos keytab = yes
disable netbios = yes
smb ports = 445

winbind enum groups = yes
winbind enum users = yes
winbind refresh tickets = true
winbind use default domain = yes
winbind separator = /
winbind cache time = 60
winbind expand groups = 10

domain master = no
client ntlmv2 auth = no
client use spnego = yes

follow symlinks = yes
wide links = yes
unix extensions = no
admin users = "@XXX/admins du domaine"

idmap backend = tdb

idmap uid = 100000-200000
idmap gid = 100000-200000
idmap config XXX : backend = rid
idmap config XXX : range = 100000-150000

acl group control = yes
inherit acls = yes
map acl inherit = yes
ea support = yes
acl map full control = True
force unknown acl user = yes
inherit permissions = yes
nt acl support = yes

vfs objects = acl_xattr

[root$]
path = /
valid users = "@XXX/s_admins"
force user = root
read only = No

The domain is an Active Directory one running on 2k8R2 servers(2k3 level).
The source server is a Windows 2008R2 with robocopy KB979808 patch.

Vincent


De : Jeremy Allison [mailto:]
Envoyé : mardi 13 mars 2012 18:19
À : Vincent Miszczak
Cc :
Objet : Re: [Samba] Robocopy from Windows to Samba (3.6.3) with backup flag

On Tue, Mar 13, 2012 at 11:27:07AM +0100, Vincent Miszczak wrote:

Hello,

I need to copy a Windows NAS to a Samba one preserving all stuffs (dates, owner, security, etc...) in an enterprise environment.

I'm used to do that with between two Windows using robocopy and the /b (backup) flag, so I can backup files even if I do not have an ACE for my account as I have the backup and restore privileges.

I need to do the same thing from Windows to Samba but using the backup flag does not work at all : it does not even copy the data. If I copy without the backup flag, it's OK for the files I have access, but as I am in an enterprise, I don't have access to all files.

Here what happens :
(From the Windows NAS) :

robocopy G:\share\XXXXX\04_generique
\\samba-nasoot$\xfs\shares\archives\XXXXX\04_generique /V /NS /NC
/NDL /NFL /S/E /COPY:DATS /B /NP /XJ /R:0 /W:30

2012/03/13 10:56:18 ERROR 87 (0x00000057) Copying NTFS Security to
Destination Directory G:\share\ XXXXX \04_generique\ The parameter is incorrect.


ð No data is copied :/

If do the same without the /B flag, it's OK in this particular case, but I don't have access to all the data and I won't be able to.

I have tested on Centos 6 with Samba 3.5.10 and Samba 3.6.3 and I have the same result.

Is this supposed to work ?

Not in 3.6.x yet (it's being fixed for 4.0 and may get back-ported).
SeBackup/SeRestore require some special case code to ensure this is done securely with no security holes or races.

Are there "special" parameters in smb.conf for this to work ?
If not, how can I backup from Windows to Samba ?

Currently the best way is to create a custom share, which uses "force user = root" and is set with the valid users set to those users who have SeRestore privilege.

This works, but I agree it's a little clunky. I'm working on it.

Jeremy.

This message has been scanned for viruses and dangerous content by MailScanner, and is believed to be clean.


This message has been scanned for viruses and
dangerous content by MailScanner, and is
believed to be clean.

To unsubscribe from this list go to the following URL and read the
instructions: https://lists.samba.org/mailman/options/samba

I have done further investigations about non backup mode ACL copy failure and I'm pretty sure backup mode failure is related to what I found.

The problem happens on XFS but not on Ext4.

I've been decomposing all ACL attribution with C# client source code, and was able to find exactly where the mess happens.

The test is simple :
Create a new directory on the destination
Foreach ACE in source ACL, copy ACE

Failure occurs at the 31st ACE (in Windows GUI). Whatever the ACE is (user, permission, right) or the tool used (code or Windows GUI) , it is impossible to add a new one. I get "The parameter is incorrect".

At the point of faillure, extended attributes are : security.NTACL=0sAwADAAAAAgAEAAIAAQCFe2P/a3WyjlERw8jwpQr5sD0CT7MhEbSoBinyXSc7gQAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAABAASNZAAAAHQAAAAAAAAAhAAAAAECAAAAAAAWAQAAAAAAAAABAgAAAAAAFgIAAAAAAAAAAgDsAx4AAAAAABgA/wEfAAECAAAAAAAFIAAAACACAAAAAyQAvwETAAEFAAAAAAAFFQAAAHgAbR9SqshoB+U7K94EAAAAAyQAvwETAAEFAAAAAAAFFQAAAHgAbR9SqshoB+U7Kz0FAAAAAyQAvwETAAEFAAAAAAAFFQAAAHgAbR9SqshoB+U7K14FAAAAAyQAqQASAAEFAAAAAAAFFQAAAHgAbR9SqshoB+U7K5AFAAAAAyQAvwETAAEFAAAAAAAFFQAAAHgAbR9SqshoB+U7K9EFAAAAAyQAqQASAAEFAAAAAAAFFQAAAHgAbR9SqshoB+U7K7IIAAAAAyQAvwETAAEFAAAAAAAFFQAAAHgAbR9SqshoB+U7K8cIAAAAAyQAqQASAAEFAAAAAAAFFQAAAHgAbR9SqshoB+U7K+cJAAAAAyQA/wEfAAEFAAAAAAAFFQAAAHgAbR9SqshoB+U7K0YLAAAAAyQAvwETAAEFAAAAAAAFFQAAAHgAbR9SqshoB+U7K2MLAAAAAyQAqQASAAEFAAAAAAAFFQAAAHgAbR9SqshoB+U7K3ELAAAAAyQA/wEfAAEFAAAAAAAFFQAAAHgAbR9SqshoB+U7K48LAAAAAyQAqQASAAEFAAAAAAAFFQAAAHgAbR9SqshoB+U7K0AMAAAAAyQA/wEfAAEFAAAAAAAFFQAAAHgAbR9SqshoB+U7K1cMAAAAAyQAvwETAAEFAAAAAAAFFQAAAHgAbR9SqshoB+U7K4MNAAAAAyQA/wEfAAEFAAAAAAAFFQAAAHgAbR9SqshoB+U7K9AOAAAAAyQA/wEfAAEFAAAAAAAFFQAAAHgAbR9SqshoB+U7K/gOAAAAAyQA/wEfAAEFAAAAAAAFFQAAAHgAbR9SqshoB+U7K/kOAAAAAyQA/wEfAAEFAAAAAAAFFQAAAHgAbR9SqshoB+U7K/sOAAAAAyQA/wEfAAEFAAAAAAAFFQAAAHgAbR9SqshoB+U7K5YPAAAAAyQA/wEfAAEFAAAAAAAFFQAAAHgAbR9SqshoB+U7K94WAAAAAyQA/wEfAAEFAAAAAAAFFQAAAHgAbR9SqshoB+U7K/oWAAAAAyQA/wEfAAEFAAAAAAAFFQAAAHgAbR9SqshoB+U7Kx0XAAAAAyQA/wEfAAEFAAAAAAAFFQAAAHgAbR9SqshoB+U7KwUeAAAAEBgA/wEfAAECAAAAAAAWAQAAAAAAAAAAGxQA/wEfAAEBAAAAAAADAAAAAAAQGACpABIAAQIAAAAAABYCAAAAAAAAAAAbFACpABIAAQEAAAAAAAMBAAAAABMUAKkAEgABAQAAAAAAAQAAAAA
And Linux ACL has 50 entries.

On Ext4, a successful ACL copy gives :

security.NTACL=0sAwADAAAAAgAEAAIAAQDCvZs3JTPb5mk5sCjyLjy5osfckFiBIttFGaZfUShSgQAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAABAASNZAAAAHQAAAAAAAAAhAAAAAECAAAAAAAWAQAAAAAAAAABAgAAAAAAFgIAAAAAAAAAAgCUBzgAAAAAABgA/wEfAAECAAAAAAAFIAAAACACAAAAAyQA/wEfAAEFAAAAAAAFFQAAAHgAbR9SqshoB+U7K/QBAAAAAyQA/wEfAAEFAAAAAAAFFQAAAHgAbR9SqshoB+U7KwcCAAAAAyQAvwETAAEFAAAAAAAFFQAAAHgAbR9SqshoB+U7K94EAAAAAyQAvwETAAEFAAAAAAAFFQAAAHgAbR9SqshoB+U7Kz0FAAAAAyQAvwETAAEFAAAAAAAFFQAAAHgAbR9SqshoB+U7K14FAAAAAyQAqQASAAEFAAAAAAAFFQAAAHgAbR9SqshoB+U7K5AFAAAAAyQAvwETAAEFAAAAAAAFFQAAAHgAbR9SqshoB+U7K6MFAAAAAyQAvwETAAEFAAAAAAAFFQAAAHgAbR9SqshoB+U7K6UFAAAAAyQAvwETAAEFAAAAAAAFFQAAAHgAbR9SqshoB+U7K9EFAAAAAyQAqQASAAEFAAAAAAAFFQAAAHgAbR9SqshoB+U7K7IIAAAAAyQAvwETAAEFAAAAAAAFFQAAAHgAbR9SqshoB+U7K8cIAAAAAyQAvwETAAEFAAAAAAAFFQAAAHgAbR9SqshoB+U7K+IIAAAAAyQAvwETAAEFAAAAAAAFFQAAAHgAbR9SqshoB+U7K3cJAAAAAyQAqQASAAEFAAAAAAAFFQAAAHgAbR9SqshoB+U7K+cJAAAAAyQAqQASAAEFAAAAAAAFFQAAAHgAbR9SqshoB+U7KwULAAAAAyQAqQASAAEFAAAAAAAFFQAAAHgAbR9SqshoB+U7KwYLAAAAAyQAqQASAAEFAAAAAAAFFQAAAHgAbR9SqshoB+U7KwcLAAAAAyQAvwETAAEFAAAAAAAFFQAAAHgAbR9SqshoB+U7KwgLAAAAAyQAqQASAAEFAAAAAAAFFQAAAHgAbR9SqshoB+U7KwkLAAAAAyQAqQASAAEFAAAAAAAFFQAAAHgAbR9SqshoB+U7KwoLAAAAAyQAvwETAAEFAAAAAAAFFQAAAHgAbR9SqshoB+U7KwwLAAAAAyQAqQASAAEFAAAAAAAFFQAAAHgAbR9SqshoB+U7Kw0LAAAAAyQAvwETAAEFAAAAAAAFFQAAAHgAbR9SqshoB+U7KxoLAAAAAyQA/wEfAAEFAAAAAAAFFQAAAHgAbR9SqshoB+U7K0YLAAAAAyQAvwETAAEFAAAAAAAFFQAAAHgAbR9SqshoB+U7K2MLAAAAAyQAqQASAAEFAAAAAAAFFQAAAHgAbR9SqshoB+U7K3ELAAAAAyQA/wEfAAEFAAAAAAAFFQAAAHgAbR9SqshoB+U7K48LAAAAAyQAqQASAAEFAAAAAAAFFQAAAHgAbR9SqshoB+U7K0AMAAAAAyQA/wEfAAEFAAAAAAAFFQAAAHgAbR9SqshoB+U7K1cMAAAAAyQAvwETAAEFAAAAAAAFFQAAAHgAbR9SqshoB+U7K4MNAAAAAyQA/wEfAAEFAAAAAAAFFQAAAHgAbR9SqshoB+U7K9AOAAAAAyQA/wEfAAEFAAAAAAAFFQAAAHgAbR9SqshoB+U7K/gOAAAAAyQA/wEfAAEFAAAAAAAFFQAAAHgAbR9SqshoB+U7K/kOAAAAAyQA/wEfAAEFAAAAAAAFFQAAAHgAbR9SqshoB+U7K/sOAAAAAyQA/wEfAAEFAAAAAAAFFQAAAHgAbR9SqshoB+U7K5YPAAAAAyQAvwETAAEFAAAAAAAFFQAAAHgAbR9SqshoB+U7K8UPAAAAAyQAvwETAAEFAAAAAAAFFQAAAHgAbR9SqshoB+U7K8YPAAAAAyQAqQASAAEFAAAAAAAFFQAAAHgAbR9SqshoB+U7K/oQAAAAAyQAqQASAAEFAAAAAAAFFQAAAHgAbR9SqshoB+U7K/4VAAAAAyQA/wEfAAEFAAAAAAAFFQAAAHgAbR9SqshoB+U7Kx8WAAAAAyQA/wEfAAEFAAAAAAAFFQAAAHgAbR9SqshoB+U7K94WAAAAAyQAvwETAAEFAAAAAAAFFQAAAHgAbR9SqshoB+U7K/kWAAAAAyQA/wEfAAEFAAAAAAAFFQAAAHgAbR9SqshoB+U7K/oWAAAAAyQA/wEfAAEFAAAAAAAFFQAAAHgAbR9SqshoB+U7Kx0XAAAAAyQA/wEfAAEFAAAAAAAFFQAAAHgAbR9SqshoB+U7K20XAAAAAyQAvwETAAEFAAAAAAAFFQAAAHgAbR9SqshoB+U7K88YAAAAAyQAvwETAAEFAAAAAAAFFQAAAHgAbR9SqshoB+U7K6IaAAAAAyQAqQASAAEFAAAAAAAFFQAAAHgAbR9SqshoB+U7K8YdAAAAAyQAqQASAAEFAAAAAAAFFQAAAHgAbR9SqshoB+U7K/gdAAAAAyQA/wEfAAEFAAAAAAAFFQAAAHgAbR9SqshoB+U7KwUeAAAAEBgA/wEfAAECAAAAAAAWAQAAAAAAAAAAGxQA/wEfAAEBAAAAAAADAAAAAAAQGACpABIAAQIAAAAAABYCAAAAAAAAAAAbFACpABIAAQEAAAAAAAMBAAAAABMUAKkAEgABAQAAAAAAAQAAAAA
And there are 102 Linux ACL entries.

I've read about XFS extended attributes limit and it's 64KB, far from what my is.

Am I missing something ?
What can I do ?

Regards.

De : Vincent Miszczak
Envoyé : mardi 13 mars 2012 19:13
À : 'Jeremy Allison'
Cc :
Objet : RE: [Samba] Robocopy from Windows to Samba (3.6.3) with backup flag

Thank you for your response.

We are used to have root$ share force root user for admins.
Result is the same with and without force user.

Just using /B flag without /COPY:DATS (ie DAT), result is a bit different (no more "incorrect parameter" but "access denied" with the filename):

87% New File 298099 Generique_Va_GameOverDeath.png
2012/03/13 18:24:20 ERROR 5 (0x00000005) Copying File G:\share\XXXX\04_generique\Generique_Va_GameOverDeath.png
Access is denied.

(This does not happens on all files but it happens a lot and I can't say where is the difference as some files of one folder are copied but others not).

As this point the file data is copied, but date and ACLs are not.

I have even tested with the raw Windows C API BackupRead/BackupWrite (with correct token privileges) and BackupWrite returns false for files failing with robocopy.

I also have other problems with this fu***** closed source Robocopy program.

I have tested all possible configuration, whenever I use it without backup mode, folder ACLs are not copied and files that inherit ACLs get the parent's not copied ACLs, ie ACLs are not copied.
(robocopy options used are /S /E /COPY:DATS)

Using xcopy with /O (ownership and ACLs copy), ACLs are copied on both folders and files with the same smb.conf config but I really need to apply a rsync like (purge, ACL fix, timefix) program because I'm syncing a big bunch of TB with millions of living files. Having tested rsync with Cygwin on Windows in the past, it could not handle path longer than 256 chars and we use larger ones so I can't use it (unless it is fixed).

Doing a debug 10 level does not seem to help.

I'm stuck :/ and any help would be appreciated.

Here is my configuration :

[global]
load printers = no
printing = bsd
printcap name = /dev/null
disable spoolss = yes
log file = /var/log/samba/log.%m
#log level
max log size = 50
load printers = no
netbios name = cifs-anim-arch
server string = %h
workgroup = XXXX
realm = XXXX.LAN
password server = dc-xxx.xxx.lan dc-xxx.xxx.lan
security = ads
use kerberos keytab = yes
disable netbios = yes
smb ports = 445

winbind enum groups = yes
winbind enum users = yes
winbind refresh tickets = true
winbind use default domain = yes
winbind separator = /
winbind cache time = 60
winbind expand groups = 10

domain master = no
client ntlmv2 auth = no
client use spnego = yes

follow symlinks = yes
wide links = yes
unix extensions = no
admin users = "@XXX/admins du domaine"

idmap backend = tdb

idmap uid = 100000-200000
idmap gid = 100000-200000
idmap config XXX : backend = rid
idmap config XXX : range = 100000-150000

acl group control = yes
inherit acls = yes
map acl inherit = yes
ea support = yes
acl map full control = True
force unknown acl user = yes
inherit permissions = yes
nt acl support = yes

vfs objects = acl_xattr

[root$]
path = /
valid users = "@XXX/s_admins"
force user = root
read only = No

The domain is an Active Directory one running on 2k8R2 servers(2k3 level).
The source server is a Windows 2008R2 with robocopy KB979808 patch.

Vincent


De : Jeremy Allison [mailto:] Envoyé : mardi 13 mars 2012 18:19 À : Vincent Miszczak Cc : Objet : Re: [Samba] Robocopy from Windows to Samba (3.6.3) with backup flag

On Tue, Mar 13, 2012 at 11:27:07AM +0100, Vincent Miszczak wrote:

Hello,

I need to copy a Windows NAS to a Samba one preserving all stuffs (dates, owner, security, etc...) in an enterprise environment.

I'm used to do that with between two Windows using robocopy and the /b (backup) flag, so I can backup files even if I do not have an ACE for my account as I have the backup and restore privileges.

I need to do the same thing from Windows to Samba but using the backup flag does not work at all : it does not even copy the data. If I copy without the backup flag, it's OK for the files I have access, but as I am in an enterprise, I don't have access to all files.

Here what happens :
(From the Windows NAS) :

robocopy G:\share\XXXXX\04_generique
\\samba-nasoot$\xfs\shares\archives\XXXXX\04_generique /V /NS /NC
/NDL /NFL /S/E /COPY:DATS /B /NP /XJ /R:0 /W:30

2012/03/13 10:56:18 ERROR 87 (0x00000057) Copying NTFS Security to
Destination Directory G:\share\ XXXXX \04_generique\ The parameter is incorrect.


ð No data is copied :/

If do the same without the /B flag, it's OK in this particular case, but I don't have access to all the data and I won't be able to.

I have tested on Centos 6 with Samba 3.5.10 and Samba 3.6.3 and I have the same result.

Is this supposed to work ?

Not in 3.6.x yet (it's being fixed for 4.0 and may get back-ported).
SeBackup/SeRestore require some special case code to ensure this is done securely with no security holes or races.

Are there "special" parameters in smb.conf for this to work ?
If not, how can I backup from Windows to Samba ?

Currently the best way is to create a custom share, which uses "force user = root" and is set with the valid users set to those users who have SeRestore privilege.

This works, but I agree it's a little clunky. I'm working on it.

Jeremy.

This message has been scanned for viruses and dangerous content by MailScanner, and is believed to be clean.


This message has been scanned for viruses and
dangerous content by MailScanner, and is
believed to be clean.

To unsubscribe from this list go to the following URL and read the
instructions: https://lists.samba.org/mailman/options/samba