copy chunk preliminary results

[Steve French]

I extended my initial copy chunk kernel client patch to handle files >
1MB (repeating copychunk requests multiple times, 1 chunk at a time, I
realize that this will be even faster when I send more than 1 chunk at
a time)

Preliminary results local loopback mount of current cifs kernel code
(vers=3.0) to Samba (master branch, running on Fedora 19 with btrfs):

File size of test file (vmlinux.o) was 373987044 bytes

Over the network Copychunk (refcopy) was about six times faster than without.
ane even local copy (no reflink) is slower than remote copy with
reflink. Copychunk would look even better except that the server disk
is SSD.


[***@pc-on-right cifs-2.6]$ time cp vmlinux.o ~/trgt-1-local
real 0m0.769s
user 0m0.002s
sys 0m0.502s

(Local copy with reflink is amazingly fast on btrfs)
[***@pc-on-right cifs-2.6]$ time cp --reflink vmlinux.o
~/trgt-2-local-reflink
real 0m0.004s
user 0m0.001s
sys 0m0.002s

(remote copy with reflink to Samba was six times faster than remote
with no reflink, similar results when repeated)
[***@pc-on-right cifs-2.6]$ time cp --reflink
/mnt/cifs-2.6/vmlinux.o /mnt/trgt-3-remote-reflink
real 0m0.416s
user 0m0.000s
sys 0m0.029s


[***@pc-on-right cifs-2.6]$ time cp /mnt/cifs-2.6/vmlinux.o
/mnt/trgt-3-no-reflink
real 0m2.596s
user 0m0.007s
sys 0m0.860s

[Pádraig Brady]

I extended my initial copy chunk kernel client patch to handle files =
1MB (repeating copychunk requests multiple times, 1 chunk at a time, =

I

realize that this will be even faster when I send more than 1 chunk a=

t

a time)
=20
Preliminary results local loopback mount of current cifs kernel code
(vers=3D3.0) to Samba (master branch, running on Fedora 19 with btrfs=
=20
File size of test file (vmlinux.o) was 373987044 bytes
=20
Over the network Copychunk (refcopy) was about six times faster than =

without.

ane even local copy (no reflink) is slower than remote copy with
reflink. Copychunk would look even better except that the server dis=

k

is SSD.
=20
=20
real 0m0.769s
user 0m0.002s
sys 0m0.502s
=20
(Local copy with reflink is amazingly fast on btrfs)
~/trgt-2-local-reflink
real 0m0.004s
user 0m0.001s
sys 0m0.002s
=20
(remote copy with reflink to Samba was six times faster than remote
with no reflink, similar results when repeated)
/mnt/cifs-2.6/vmlinux.o /mnt/trgt-3-remote-reflink
real 0m0.416s
user 0m0.000s
sys 0m0.029s
=20
=20
/mnt/trgt-3-no-reflink
real 0m2.596s
user 0m0.007s
sys 0m0.860s

So cp --reflink has CoW semantics and so probably not an
appropriate interface for this. Unless I'm misunderstanding,
this SMB copy offload does result in a normal copy on the server right?
So that would imply that cp should try to use the "smb_copy_offload"
ioctl unconditionally after a little introspection, or preferentially
this would be encapsulated in the recently mooted copyfile() level
kernel call, which cp etc. could just call to do the operation.

thanks,
P=E1draig.

[David Disseldorp]

On Mon, 04 Nov 2013 10:07:22 +0000

Post by Pádraig Brady
So cp --reflink has CoW semantics and so probably not an
appropriate interface for this. Unless I'm misunderstanding,
this SMB copy offload does result in a normal copy on the server righ=

t?

It depends on how the SMB server interprets the copy-chunk wire request=
=2E
On Btrfs, Samba can translate the request into a BTRFS_IOC_CLONE_RANGE
ioctl, in which case the same CoW semantics are observed[1]. See:

https://wiki.samba.org/index.php/Server-Side_Copy#Btrfs_Enhanced_Server=
-Side_Copy_Offload

By default however, Samba (and Windows) will perform the copy on the
server-side using regular reads/writes. A generic cp --offload or
similar would probably make more sense on the client side.

Cheers, David

[Christoph Hellwig]

It depends on how the SMB server interprets the copy-chunk wire request.
On Btrfs, Samba can translate the request into a BTRFS_IOC_CLONE_RANGE
https://wiki.samba.org/index.php/Server-Side_Copy#Btrfs_Enhanced_Server-Side_Copy_Offload
By default however, Samba (and Windows) will perform the copy on the
server-side using regular reads/writes. A generic cp --offload or
similar would probably make more sense on the client side.

I don't think it really matters what the optimal case is, it matters
what the worst case is. Think about it - a reflink really just is
a smart shortcut for copy + dedup, which a filesystem on the server
could do anyway.

On the other hand a user of cp --reflink expects it to be a quick
operation.

So it's time folks finally get the damn copyfile system call in, use
that for CIFS, NFS and co, as well as letting btrfs optimize it.

[Volker Lendecke]

Post by Christoph Hellwig
So it's time folks finally get the damn copyfile system call in, use
that for CIFS, NFS and co, as well as letting btrfs optimize it.

Has the splice discussion gone anywhere? Sorry for not
following fully, but having splice disk->disk work well
0-copy might have the additional effect that eventually we
get TCP->disk 0-copy.

Volker

[Steve French]

Post by Christoph Hellwig

It depends on how the SMB server interprets the copy-chunk wire request.
On Btrfs, Samba can translate the request into a BTRFS_IOC_CLONE_RANGE
https://wiki.samba.org/index.php/Server-Side_Copy#Btrfs_Enhanced_Server-Side_Copy_Offload
By default however, Samba (and Windows) will perform the copy on the
server-side using regular reads/writes. A generic cp --offload or
similar would probably make more sense on the client side.

On copychunk Windows server is clearly doing the equivalent of reflink
internally,
so this isn't just Samba/btrfs. I was getting performance more than 100 times
better using copychunk to Windows than doing copy with sending
reads/writes over the network.

Post by Christoph Hellwig
I don't think it really matters what the optimal case is, it matters
what the worst case is. Think about it - a reflink really just is
a smart shortcut for copy + dedup, which a filesystem on the server
could do anyway.
On the other hand a user of cp --reflink expects it to be a quick
operation.

My tests show that in every case the copychunk over SMB2/SMB3 mount is
much faster even in the worst case where it is emulated in Samba.

Post by Christoph Hellwig
So it's time folks finally get the damn copyfile system call in, use
that for CIFS, NFS and co, as well as letting btrfs optimize it.

I agree that the copyfile system call would be a big help.