[PATCH v1 1/2] tools/nolibc/stdlib: Support overflow checking for older compiler versions

[Ammar Faizi <ammarfaizi2@gnuweeb.org>]

Previously, we used __builtin_mul_overflow() to check for overflow in
the multiplication operation in the calloc() function. However, older
compiler versions don't support this built-in. This patch changes the
overflow checking mechanism to make it work on any compiler version
by using a division method to check for overflow. No functional change
intended. While in there, remove the unused variable `void *orig`.

Link: https://lore.kernel.org/lkml/20220330024114.GA18892@1wt.eu
Suggested-by: Willy Tarreau <w@1wt.eu>
Cc: Alviro Iskandar Setiawan <alviro.iskandar@gnuweeb.org>
Signed-off-by: Ammar Faizi <ammarfaizi2@gnuweeb.org>
---
 tools/include/nolibc/stdlib.h | 7 +++----
 1 file changed, 3 insertions(+), 4 deletions(-)

diff --git a/tools/include/nolibc/stdlib.h b/tools/include/nolibc/stdlib.h
index 8fd32eaf8037..92378c4b9660 100644
--- a/tools/include/nolibc/stdlib.h
+++ b/tools/include/nolibc/stdlib.h
@@ -128,10 +128,9 @@ void *malloc(size_t len)
 static __attribute__((unused))
 void *calloc(size_t size, size_t nmemb)
 {
-	void *orig;
-	size_t res = 0;
+	size_t x = size * nmemb;
 
-	if (__builtin_expect(__builtin_mul_overflow(nmemb, size, &res), 0)) {
+	if (__builtin_expect(size && ((x / size) != nmemb), 0)) {
 		SET_ERRNO(ENOMEM);
 		return NULL;
 	}
@@ -140,7 +139,7 @@ void *calloc(size_t size, size_t nmemb)
 	 * No need to zero the heap, the MAP_ANONYMOUS in malloc()
 	 * already does it.
 	 */
-	return malloc(res);
+	return malloc(x);
 }
 
 static __attribute__((unused))
-- 
Ammar Faizi