From: Ammar Faizi <[email protected]>
To: "Willy Tarreau" <[email protected]>, "Thomas Weißschuh" <[email protected]>
Cc: Ammar Faizi <[email protected]>,
Nicholas Rosenberg <[email protected]>,
Alviro Iskandar Setiawan <[email protected]>,
Michael William Jonathan <[email protected]>,
GNU/Weeb Mailing List <[email protected]>,
Linux Kernel Mailing List <[email protected]>
Subject: [RFC PATCH v1 1/5] tools/nolibc: x86-64: Use `rep movsb` for `memcpy()` and `memmove()`
Date: Wed, 30 Aug 2023 20:57:22 +0700 [thread overview]
Message-ID: <[email protected]> (raw)
In-Reply-To: <[email protected]>
Simplify memcpy() and memmove() on the x86-64 arch.
The x86-64 arch has a 'rep movsb' instruction, which can perform
memcpy() using only a single instruction, given:
%rdi = destination
%rsi = source
%rcx = length
Additionally, it can also handle the overlapping case by setting DF=1
(backward copy), which can be used as the memmove() implementation.
Before this patch:
```
00000000000010ab <memmove>:
10ab: 48 89 f8 mov %rdi,%rax
10ae: 31 c9 xor %ecx,%ecx
10b0: 48 39 f7 cmp %rsi,%rdi
10b3: 48 83 d1 ff adc $0xffffffffffffffff,%rcx
10b7: 48 85 d2 test %rdx,%rdx
10ba: 74 25 je 10e1 <memmove+0x36>
10bc: 48 83 c9 01 or $0x1,%rcx
10c0: 48 39 f0 cmp %rsi,%rax
10c3: 48 c7 c7 ff ff ff ff mov $0xffffffffffffffff,%rdi
10ca: 48 0f 43 fa cmovae %rdx,%rdi
10ce: 48 01 cf add %rcx,%rdi
10d1: 44 8a 04 3e mov (%rsi,%rdi,1),%r8b
10d5: 44 88 04 38 mov %r8b,(%rax,%rdi,1)
10d9: 48 01 cf add %rcx,%rdi
10dc: 48 ff ca dec %rdx
10df: 75 f0 jne 10d1 <memmove+0x26>
10e1: c3 ret
00000000000010e2 <memcpy>:
10e2: 48 89 f8 mov %rdi,%rax
10e5: 48 85 d2 test %rdx,%rdx
10e8: 74 12 je 10fc <memcpy+0x1a>
10ea: 31 c9 xor %ecx,%ecx
10ec: 40 8a 3c 0e mov (%rsi,%rcx,1),%dil
10f0: 40 88 3c 08 mov %dil,(%rax,%rcx,1)
10f4: 48 ff c1 inc %rcx
10f7: 48 39 ca cmp %rcx,%rdx
10fa: 75 f0 jne 10ec <memcpy+0xa>
10fc: c3 ret
```
After this patch:
```
0000000000001093 <memmove>:
1093: 48 89 f8 mov %rdi,%rax
1096: 48 89 d1 mov %rdx,%rcx
1099: 48 8d 7c 0f ff lea -0x1(%rdi,%rcx,1),%rdi
109e: 48 8d 74 0e ff lea -0x1(%rsi,%rcx,1),%rsi
10a3: fd std
10a4: f3 a4 rep movsb %ds:(%rsi),%es:(%rdi)
10a6: fc cld
10a7: c3 ret
00000000000010a8 <memcpy>:
10a8: 48 89 f8 mov %rdi,%rax
10ab: 48 89 d1 mov %rdx,%rcx
10ae: f3 a4 rep movsb %ds:(%rsi),%es:(%rdi)
10b0: c3 ret
```
Signed-off-by: Ammar Faizi <[email protected]>
---
tools/include/nolibc/arch-x86_64.h | 28 ++++++++++++++++++++++++++++
tools/include/nolibc/string.h | 4 ++++
2 files changed, 32 insertions(+)
diff --git a/tools/include/nolibc/arch-x86_64.h b/tools/include/nolibc/arch-x86_64.h
index e5ccb926c90306b6..c5162170a2ccdff1 100644
--- a/tools/include/nolibc/arch-x86_64.h
+++ b/tools/include/nolibc/arch-x86_64.h
@@ -173,4 +173,32 @@ void __attribute__((weak, noreturn, optimize("Os", "omit-frame-pointer"))) __no_
__builtin_unreachable();
}
+#define NOLIBC_ARCH_HAS_MEMMOVE
+void *memmove(void *dst, const void *src, size_t len);
+
+#define NOLIBC_ARCH_HAS_MEMCPY
+void *memcpy(void *dst, const void *src, size_t len);
+
+__asm__ (
+".section .text.nolibc_memmove\n"
+".weak memmove\n"
+"memmove:\n"
+ "movq %rdi, %rax\n"
+ "movq %rdx, %rcx\n"
+ "leaq -1(%rdi, %rcx), %rdi\n"
+ "leaq -1(%rsi, %rcx), %rsi\n"
+ "std\n"
+ "rep movsb\n"
+ "cld\n"
+ "retq\n"
+
+".section .text.nolibc_memcpy\n"
+".weak memcpy\n"
+"memcpy:\n"
+ "movq %rdi, %rax\n"
+ "movq %rdx, %rcx\n"
+ "rep movsb\n"
+ "retq\n"
+);
+
#endif /* _NOLIBC_ARCH_X86_64_H */
diff --git a/tools/include/nolibc/string.h b/tools/include/nolibc/string.h
index 0c2e06c7c4772bc6..6eca267ec6fa7177 100644
--- a/tools/include/nolibc/string.h
+++ b/tools/include/nolibc/string.h
@@ -49,6 +49,7 @@ void *_nolibc_memcpy_down(void *dst, const void *src, size_t len)
return dst;
}
+#ifndef NOLIBC_ARCH_HAS_MEMMOVE
/* might be ignored by the compiler without -ffreestanding, then found as
* missing.
*/
@@ -72,13 +73,16 @@ void *memmove(void *dst, const void *src, size_t len)
}
return dst;
}
+#endif /* #ifndef NOLIBC_ARCH_HAS_MEMMOVE */
+#ifndef NOLIBC_ARCH_HAS_MEMCPY
/* must be exported, as it's used by libgcc on ARM */
__attribute__((weak,unused,section(".text.nolibc_memcpy")))
void *memcpy(void *dst, const void *src, size_t len)
{
return _nolibc_memcpy_up(dst, src, len);
}
+#endif /* #ifndef NOLIBC_ARCH_HAS_MEMCPY */
/* might be ignored by the compiler without -ffreestanding, then found as
* missing.
--
Ammar Faizi
next prev parent reply other threads:[~2023-08-30 13:57 UTC|newest]
Thread overview: 29+ messages / expand[flat|nested] mbox.gz Atom feed top
2023-08-30 13:57 [RFC PATCH v1 0/5] nolibc x86-64 string functions Ammar Faizi
2023-08-30 13:57 ` Ammar Faizi [this message]
2023-08-30 13:57 ` [RFC PATCH v1 2/5] tools/nolibc: x86-64: Use `rep stosb` for `memset()` Ammar Faizi
2023-08-30 14:08 ` Alviro Iskandar Setiawan
2023-08-30 14:13 ` Ammar Faizi
2023-08-30 14:24 ` Alviro Iskandar Setiawan
2023-08-30 15:09 ` Ammar Faizi
2023-08-30 15:23 ` Willy Tarreau
2023-08-30 15:44 ` Ammar Faizi
2023-08-30 15:51 ` Willy Tarreau
2023-08-30 16:08 ` Ammar Faizi
2023-08-30 16:11 ` Alviro Iskandar Setiawan
2023-08-30 13:57 ` [RFC PATCH v1 3/5] tools/nolibc: x86-64: Use `rep cmpsb` for `memcmp()` Ammar Faizi
2023-08-30 21:26 ` Willy Tarreau
2023-09-01 3:24 ` Ammar Faizi
2023-09-01 3:35 ` Willy Tarreau
2023-09-01 7:27 ` Ammar Faizi
2023-09-04 8:26 ` David Laight
2023-08-30 13:57 ` [RFC PATCH v1 4/5] tools/nolibc: string: Remove the `_nolibc_memcpy_down()` function Ammar Faizi
2023-08-30 21:27 ` Willy Tarreau
2023-08-30 13:57 ` [RFC PATCH v1 5/5] tools/nolibc: string: Remove the `_nolibc_memcpy_up()` function Ammar Faizi
2023-08-30 21:32 ` [RFC PATCH v1 0/5] nolibc x86-64 string functions Willy Tarreau
2023-09-01 11:34 ` David Laight
2023-09-01 11:46 ` Willy Tarreau
2023-09-01 13:06 ` Ammar Faizi
2023-09-01 14:23 ` David Laight
2023-09-01 14:41 ` Ammar Faizi
2023-09-01 14:54 ` David Laight
2023-09-01 15:20 ` Ammar Faizi
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
[email protected] \
[email protected] \
[email protected] \
[email protected] \
[email protected] \
[email protected] \
[email protected] \
[email protected] \
[email protected] \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox