[GIT PULL] nolibc changes for v5.19

[GIT PULL] nolibc changes for v5.19

[Paul E. McKenney]

Hello, Linus,

Once the merge window opens, please pull the latest nolibc changes from:

  git://git.kernel.org/pub/scm/linux/kernel/git/paulmck/linux-rcu.git tags/nolibc.2022.05.20a
  # HEAD 11dbdaeff41d9ec9376476889651fac4838bff99 tools/nolibc/string: Implement `strdup()` and `strndup()`

----------------------------------------------------------------
Pull request for nolibc library for v5.19

This pull request adds a number of library functions and splits this
library into multiple files.

----------------------------------------------------------------
Ammar Faizi (9):
      tools/nolibc: x86-64: Update System V ABI document link
      tools/nolibc: Replace `asm` with `__asm__`
      tools/nolibc: Remove .global _start from the entry point code
      tools/nolibc: i386: Implement syscall with 6 arguments
      tools/nolibc/sys: Implement `mmap()` and `munmap()`
      tools/nolibc/types: Implement `offsetof()` and `container_of()` macro
      tools/nolibc/stdlib: Implement `malloc()`, `calloc()`, `realloc()` and `free()`
      tools/nolibc/string: Implement `strnlen()`
      tools/nolibc/string: Implement `strdup()` and `strndup()`

Willy Tarreau (52):
      tools/nolibc: use pselect6 on RISCV
      tools/nolibc: guard the main file against multiple inclusion
      tools/nolibc/std: move the standard type definitions to std.h
      tools/nolibc/types: split syscall-specific definitions into their own files
      tools/nolibc/arch: split arch-specific code into individual files
      tools/nolibc/sys: split the syscall definitions into their own file
      tools/nolibc/stdlib: extract the stdlib-specific functions to their own file
      tools/nolibc/string: split the string functions into string.h
      tools/nolibc/ctype: split the is* functions to ctype.h
      tools/nolibc/ctype: add the missing is* functions
      tools/nolibc/types: move the FD_* functions to macros in types.h
      tools/nolibc/types: make FD_SETSIZE configurable
      tools/nolibc/types: move makedev to types.h and make it a macro
      tools/nolibc/stdlib: move ltoa() to stdlib.h
      tools/nolibc/stdlib: replace the ltoa() function with more efficient ones
      tools/nolibc/stdlib: add i64toa() and u64toa()
      tools/nolibc/stdlib: add utoh() and u64toh()
      tools/nolibc/stdio: add a minimal set of stdio functions
      tools/nolibc/stdio: add stdin/stdout/stderr and fget*/fput* functions
      tools/nolibc/stdio: add fwrite() to stdio
      tools/nolibc/stdio: add a minimal [vf]printf() implementation
      tools/nolibc/types: define EXIT_SUCCESS and EXIT_FAILURE
      tools/nolibc/stdio: add perror() to report the errno value
      tools/nolibc/sys: make open() take a vararg on the 3rd argument
      tools/nolibc/stdlib: avoid a 64-bit shift in u64toh_r()
      tools/nolibc/stdlib: make raise() use the lower level syscalls only
      tools/nolibc/sys: make getpgrp(), getpid(), gettid() not set errno
      tools/nolibc/string: use unidirectional variants for memcpy()
      tools/nolibc/string: slightly simplify memmove()
      tools/nolibc/string: add strncpy() and strlcpy()
      tools/nolibc/string: add tiny versions of strncat() and strlcat()
      tools/nolibc: move exported functions to their own section
      tools/nolibc/arch: mark the _start symbol as weak
      tools/nolibc/types: define PATH_MAX and MAXPATHLEN
      tools/nolibc/string: export memset() and memmove()
      tools/nolibc/errno: extract errno.h from sys.h
      tools/nolibc/unistd: extract msleep(), sleep(), tcsetpgrp() to unistd.h
      tools/nolibc/unistd: add usleep()
      tools/nolibc/signal: move raise() to signal.h
      tools/nolibc/time: create time.h with time()
      tools/nolibc: also mention how to build by just setting the include path
      tools/nolibc/stdlib: implement abort()
      tools/nolibc/stdio: make printf(%s) accept NULL
      tools/nolibc/stdlib: add a simple getenv() implementation
      tools/nolibc/stdio: add support for '%p' to vfprintf()
      tools/nolibc/string: add strcmp() and strncmp()
      tools/nolibc/sys: add syscall definition for getppid()
      tools/nolibc/types: add poll() and waitpid() flag definitions
      tools/nolibc: add a makefile to install headers
      tools/nolibc: add the nolibc subdir to the common Makefile
      tools/nolibc/string: do not use __builtin_strlen() at -O0
      tools/nolibc/stdlib: only reference the external environ when inlined

 tools/Makefile                      |    4 +
 tools/include/nolibc/Makefile       |   42 +
 tools/include/nolibc/arch-aarch64.h |  199 +++
 tools/include/nolibc/arch-arm.h     |  204 +++
 tools/include/nolibc/arch-i386.h    |  219 +++
 tools/include/nolibc/arch-mips.h    |  215 +++
 tools/include/nolibc/arch-riscv.h   |  204 +++
 tools/include/nolibc/arch-x86_64.h  |  215 +++
 tools/include/nolibc/arch.h         |   32 +
 tools/include/nolibc/ctype.h        |   99 ++
 tools/include/nolibc/errno.h        |   27 +
 tools/include/nolibc/nolibc.h       | 2540 +----------------------------------
 tools/include/nolibc/signal.h       |   22 +
 tools/include/nolibc/std.h          |   49 +
 tools/include/nolibc/stdio.h        |  306 +++++
 tools/include/nolibc/stdlib.h       |  423 ++++++
 tools/include/nolibc/string.h       |  285 ++++
 tools/include/nolibc/sys.h          | 1247 +++++++++++++++++
 tools/include/nolibc/time.h         |   28 +
 tools/include/nolibc/types.h        |  205 +++
 tools/include/nolibc/unistd.h       |   54 +
 21 files changed, 4114 insertions(+), 2505 deletions(-)
 create mode 100644 tools/include/nolibc/Makefile
 create mode 100644 tools/include/nolibc/arch-aarch64.h
 create mode 100644 tools/include/nolibc/arch-arm.h
 create mode 100644 tools/include/nolibc/arch-i386.h
 create mode 100644 tools/include/nolibc/arch-mips.h
 create mode 100644 tools/include/nolibc/arch-riscv.h
 create mode 100644 tools/include/nolibc/arch-x86_64.h
 create mode 100644 tools/include/nolibc/arch.h
 create mode 100644 tools/include/nolibc/ctype.h
 create mode 100644 tools/include/nolibc/errno.h
 create mode 100644 tools/include/nolibc/signal.h
 create mode 100644 tools/include/nolibc/std.h
 create mode 100644 tools/include/nolibc/stdio.h
 create mode 100644 tools/include/nolibc/stdlib.h
 create mode 100644 tools/include/nolibc/string.h
 create mode 100644 tools/include/nolibc/sys.h
 create mode 100644 tools/include/nolibc/time.h
 create mode 100644 tools/include/nolibc/types.h
 create mode 100644 tools/include/nolibc/unistd.h

Re: [GIT PULL] nolibc changes for v5.19

[Linus Torvalds]

On Fri, May 20, 2022 at 11:24 AM Paul E. McKenney <paulmck@kernel.org> wrote:
>
> This pull request adds a number of library functions and splits this
> library into multiple files.

Well, this is annoying.

You add the rule to test and install this, and "make help" will list
"nolibc" as a target, but that is not actually true at all.

So what's the appropriate way to actually test this pull somehow?

I'm guessing it's along the lines of

    make ARCH=x86 nolibc_headers

in the tools directory, but then I got bored and decided I need to
just continue the merge window.

I've pulled this, but it all makes me go "Hmm, I'd have liked to maybe
even build test it".

                Linus

Re: [GIT PULL] nolibc changes for v5.19

[pr-tracker-bot]

The pull request you sent on Fri, 20 May 2022 11:24:28 -0700:

> git://git.kernel.org/pub/scm/linux/kernel/git/paulmck/linux-rcu.git tags/nolibc.2022.05.20a

has been merged into torvalds/linux.git:
https://git.kernel.org/torvalds/c/f814957b018c867a2397bdd3c1816de7502d2c7f

Thank you!

-- 
Deet-doot-dot, I am a bot.
https://korg.docs.kernel.org/prtracker.html

Re: [GIT PULL] nolibc changes for v5.19

[Willy Tarreau]

Hi Linus,

On Mon, May 23, 2022 at 11:42:48AM -0700, Linus Torvalds wrote:
> On Fri, May 20, 2022 at 11:24 AM Paul E. McKenney <paulmck@kernel.org> wrote:
> >
> > This pull request adds a number of library functions and splits this
> > library into multiple files.
> 
> Well, this is annoying.
> 
> You add the rule to test and install this, and "make help" will list
> "nolibc" as a target, but that is not actually true at all.
> 
> So what's the appropriate way to actually test this pull somehow?
> 
> I'm guessing it's along the lines of
> 
>     make ARCH=x86 nolibc_headers
> 
> in the tools directory, but then I got bored and decided I need to
> just continue the merge window.
> 
> I've pulled this, but it all makes me go "Hmm, I'd have liked to maybe
> even build test it".

I did. I must confess I'm embarrassed now because when I added the
entries there, exactly in order to reuse what was in place, I found
it a bit tricky to launch the tests, but after that I felt OK with
it. Now it's been a quite some time now and I don't remember the exact
way to trigger the tests there, so it's likely that I didn't leave
enough info in the commit messages :-( Let me have a look and figure
again how to start the tests.

Sorry about that,
Willy

Re: [GIT PULL] nolibc changes for v5.19

[Willy Tarreau]

On Mon, May 23, 2022 at 09:56:05PM +0200, Willy Tarreau wrote:
> On Mon, May 23, 2022 at 11:42:48AM -0700, Linus Torvalds wrote:
> > On Fri, May 20, 2022 at 11:24 AM Paul E. McKenney <paulmck@kernel.org> wrote:
> > >
> > > This pull request adds a number of library functions and splits this
> > > library into multiple files.
> > 
> > Well, this is annoying.
> > 
> > You add the rule to test and install this, and "make help" will list
> > "nolibc" as a target, but that is not actually true at all.
> > 
> > So what's the appropriate way to actually test this pull somehow?
> > 
> > I'm guessing it's along the lines of
> > 
> >     make ARCH=x86 nolibc_headers
> > 
> > in the tools directory, but then I got bored and decided I need to
> > just continue the merge window.
> > 
> > I've pulled this, but it all makes me go "Hmm, I'd have liked to maybe
> > even build test it".
> 
> I did. I must confess I'm embarrassed now because when I added the
> entries there, exactly in order to reuse what was in place, I found
> it a bit tricky to launch the tests, but after that I felt OK with
> it. Now it's been a quite some time now and I don't remember the exact
> way to trigger the tests there, so it's likely that I didn't leave
> enough info in the commit messages :-( Let me have a look and figure
> again how to start the tests.

So I've figured it again. When you run:

   make tools/help

you get the help of tools/ commands, and:

   make tools/command_<target>

actually runs the <target> target of tools/command.

Here we have:

   make tools/nolibc_headers

which installs only the nolibc headers for the selected architecture
into tools/include/nolibc/sysroot, and:

   make tools/nolibc_headers_standalone

which does the same in addition with a make headers;make headers_install
into that directory so that we get a completely usable sysroot.

Finally:
 
   make tools/nolibc_clean

will clean that directory.

I hadn't found any foo_help target for other commands so I assumed it
was not what users would look like. But if you find it useful I can
easily add:

   make tools/nolibc_help

to enumerate these commands.

Hoping that clarifies the situation.

Regards,
Willy

Re: [GIT PULL] nolibc changes for v5.19

[Linus Torvalds]

On Mon, May 23, 2022 at 1:23 PM Willy Tarreau <w@1wt.eu> wrote:
>
> So I've figured it again. When you run:
>
>    make tools/help
>
> you get the help of tools/ commands, [..]

You speak the words, but you don't actually look at what it does.

Try it.

Yes,  "make tools/help" works. Yes, it lists targets, and talks about
how you can use the "<tool>_install" target.

But none of that then matches the rest of what you write.

You talk about nolibc_headers". That's not something that is actually
listed in the help at all.

So please, can you read your own email message as somebody who doesn't
actually know the code, and try the commands you talk about, and see
the disconnect here?

                  Linus

Re: [GIT PULL] nolibc changes for v5.19

[Willy Tarreau]

On Mon, May 23, 2022 at 01:30:37PM -0700, Linus Torvalds wrote:
> On Mon, May 23, 2022 at 1:23 PM Willy Tarreau <w@1wt.eu> wrote:
> >
> > So I've figured it again. When you run:
> >
> >    make tools/help
> >
> > you get the help of tools/ commands, [..]
> 
> You speak the words, but you don't actually look at what it does.
> 
> Try it.

Why are you saying this ? I've figured the commands by trying
each of them.

> Yes,  "make tools/help" works. Yes, it lists targets, and talks about
> how you can use the "<tool>_install" target.
> 
> But none of that then matches the rest of what you write.
> 
> You talk about nolibc_headers". That's not something that is actually
> listed in the help at all.

I agree. I initially added the few of them there and figured I was
polluting the "make tools/help" output with only nolibc-specific stuff.
I had a look at other comamnds that took arguments and found some that
would also take a few arguments that were not mentioned at the top level
so I concluded that it was the "expected" way to use them there.

> So please, can you read your own email message as somebody who doesn't
> actually know the code, and try the commands you talk about, and see
> the disconnect here?

That's exactly what I'm trying to do and I'm sorry if that doesn't work.
The simple fact that it doesn't work tells me I need to revisit the
approach but I need some hints about it.  Please be more specific.  I
tried to get as close as possible to what I found in order to apply the
principle of least surprise.  Do you want me to create a "noblic_install"
target maybe, even if the equivalent one doesn't exist for half of the
other commands ? If that's it, that can be done as well.

Confused,
Willy

Re: [GIT PULL] nolibc changes for v5.19

[Willy Tarreau]

On Mon, May 23, 2022 at 10:50:43PM +0200, Willy Tarreau wrote:
> On Mon, May 23, 2022 at 01:30:37PM -0700, Linus Torvalds wrote:
> > On Mon, May 23, 2022 at 1:23 PM Willy Tarreau <w@1wt.eu> wrote:
> > >
> > > So I've figured it again. When you run:
> > >
> > >    make tools/help
> > >
> > > you get the help of tools/ commands, [..]
> > 
> > You speak the words, but you don't actually look at what it does.
> > 
> > Try it.
> 
> Why are you saying this ? I've figured the commands by trying
> each of them.

I think I found it:

   make -C tools/ <tool>_<command>

and:

   make tools/<tool>_<command>

work the same on all commands but actually:

   make -C tools/ nolibc_headers

fails while:

   make tools/nolibc_headers

works. I've essentially used the latter by simplicity without noticing
that it doesn't work as suggested with <tool>_install, and I guess
that's what you got.

I'll check why it's like this and will propose a fix.

Thanks,
Willy

Re: [GIT PULL] nolibc changes for v5.19

[Linus Torvalds]

On Mon, May 23, 2022 at 2:04 PM Willy Tarreau <w@1wt.eu> wrote:
>
> On Mon, May 23, 2022 at 10:50:43PM +0200, Willy Tarreau wrote:
> > On Mon, May 23, 2022 at 01:30:37PM -0700, Linus Torvalds wrote:
> > >
> > > You speak the words, but you don't actually look at what it does.
> > >
> > > Try it.
> >
> > Why are you saying this ? I've figured the commands by trying
> > each of them.

Try this:

   make tools/help

and then actually *LOOK* at what it prints out.

Notice how it says 'nolibc' is a target here.

And then think about it. Just for fun, try doing

   make -C tools/ nolibc
   make -C tools/ nolibc_install

like that help message implies you should do.

Does it work? No.

> work the same on all commands but actually:
>
>    make -C tools/ nolibc_headers

Put another way: where did you find that "nolibc_headers"?

THAT is what I'm talking about. You are mentioning all these magical
things that don't match the documentation you yourself added.

                 Linus

Re: [GIT PULL] nolibc changes for v5.19

[Willy Tarreau]

On Mon, May 23, 2022 at 02:13:20PM -0700, Linus Torvalds wrote:
(...)
> Does it work? No.
> 
> > work the same on all commands but actually:
> >
> >    make -C tools/ nolibc_headers
> 
> Put another way: where did you find that "nolibc_headers"?
> 
> THAT is what I'm talking about. You are mentioning all these magical
> things that don't match the documentation you yourself added.

OK thanks so now at least I have all the elements I was looking for,
I'm on it and will shortly provide fixes (in short, the main makefile's
variables not being set when doing make -C tools/ requires that some of
them are set as well under the target makefile).

For the second one I totally agree and as I said I think the problem
is wider, so I'll likely add a suggestion in tools/Makefile to also
try "make -C tools/ foo_help", which will allow different sub-projects
to provide a help target that enumerates their respective commands.
We'll start with nolibc and others could then follow, because clearly
for several other ones it's not obvious either and I think the lack
of standard way to display a tool's help doesn't help here.

thanks,
Willy