[RFC PATCH 00/10] uaccess: Refuse and warn on user accesses when current->mm is NULL

[RFC PATCH 00/10] uaccess: Refuse and warn on user accesses when current->mm is NULL

[James Houghton]

Hi everyone,

This series comes out of a set of bugs that were found in KVM where the
kernel can sometimes write to a random process's user memory[1]. The KVM
fixes at [1] are still required, even with this uaccess series, as noted
by Sean here[2] and below in this cover letter. The beginning of this
series includes fixes for similar bugs found through static analysis
with Claude Opus 5.5. I don't think any of these bugs are more severe
than the KVM bugs[1].

We hit one of the KVM bugs in production, and it was challenging to
debug. This series makes such bugs, where the kernel is doing uaccess
without an mm, much more obvious.

The Cc list is wide because the fixes at the start of the series touch
several subsystems: SCSI (patch 1), btrfs (patch 2), BPF (patch 3),
tracing (patch 4) and perf (patch 5). They're included because the new
warning in patches 6-10 would otherwise fire on those paths. The core
uaccess change is in patches 6-10.

I have a reproducer only for patch 3 (a bpf_wq callback calling
bpf_copy_from_user(), which triggers the new WARN without the patch and
fails with -EFAULT with it); some details are in the notes section of
patch 3.

Patches 1, 2, 4 and 5 are only build-tested (plus syzkaller running on
the final kernel). Let me know if you'd like to see reproducers (or if
any of the patches are incorrect and should be dropped).

== Introduction ==

Accessing user memory from a task that has no mm is always a bug. This
can happen from a kernel thread that is not using kthread_use_mm(), or
from a task that has already gone through exit_mm(), e.g. when the last
reference to a file is dropped at exit and its release handler runs
from exit_task_work(). For example, KVM's nVMX code can write to guest
memory while a vCPU is being freed during VM destruction[1], and other
drivers may have similar bugs.

Such an access does not reliably fail. The user portion of the address
space belongs to whatever mm happens to be lazily loaded, if any. If
the access faults, the fault handler refuses to handle it and the copy
fails, but if the page is already mapped, the access silently reads or
writes another task's memory.

== Fixes ==

Patches 1-5 fix existing places that access current's user memory
without an mm:

 - Patches 1 and 2 fix io_uring passthrough completions in scsi_bsg and
   btrfs that copy to the submitter's user memory even when io_uring
   runs them canceled from a kworker, e.g. after the submitter exited.
   These are real bugs of the kind this series is meant to catch.

 - Patch 3 makes the sleepable BPF user copy helpers fail when
   current->mm is NULL, e.g. for bpf_wq callbacks or lsm.s programs
   running from kernel threads or exiting tasks.

 - Patches 4 and 5 make stack_trace_save_user() (ftrace's
   userstacktrace option) and get_perf_callchain() (bpf_get_task_stack()
   on the current task) use is_user_task(), so that they don't unwind
   the user stack of PID 1 before exec, usermode helpers before exec,
   or tasks that have gone through exit_mm().

== The New WARN ==

The rest of the series makes the generic uaccess helpers WARN_ON_ONCE()
and refuse the access when current->mm is NULL:

 - Patch 6 adds uaccess_should_fail(), which combines the existing
   should_fail_usercopy() fault injection check with the NULL mm check,
   and uses it in place of should_fail_usercopy(). This covers
   copy_{to,from}_user() on all architectures (including the masked
   user access paths, which never call access_ok()), the
   __copy_{to,from}_user() variants used by KVM's guest memory
   accessors, the user-backed iov_iter copy routines and
   strncpy_from_user().

 - Patch 7 makes the user nofault helpers fail quietly when
   current->mm is NULL. They are meant to be callable from any context
   (BPF, kprobes, perf), including from kernel threads, where failing
   is the expected outcome rather than a bug.

 - Patch 8 splits the NULL mm check out into uaccess_without_mm() and
   adds it to __copy_from_user_inatomic(), without also enabling fault
   injection there.

 - Patch 9 adds the check to the remaining generic paths: the scoped
   user access helpers, strnlen_user(), check_zeroed_user(), and the
   iov_iter routines that call into architecture-specific helpers.

 - Patch 10 is included for reference only. It moves the warning out of
   line, which removes ~16 KB of __bug_table growth on arm64 (where
   copy_{to,from}_user() are inlined), at the cost of warning only once
   system-wide and a slight size regression on x86_64. With defconfig +
   KVM, patches 6-9 grow vmlinux by +5,884 B (+0.014%) on x86_64 and
   +40,948 B (+0.10%) on arm64 without it, and by +9,714 B (+0.023%)
   and +24,609 B (+0.06%) with it.

The checks are deliberately limited to generic code. Accesses
implemented entirely by architectures (get_user(), put_user(),
clear_user(), and direct users of user_access_begin() or
masked_user_access_begin()) are not covered. Please let me know if the
new check should be placed elsewhere.

The check also cannot catch a valid mm that isn't the one the caller
expects, e.g. when a file's release or ioctl handlers run in a
different process than the one that created it. Code that keeps user
addresses belonging to a specific mm still has to compare current->mm
against that mm itself. For KVM, Sean's series[1] does exactly that for
guest memory accesses, and so complements this one: it handles the case
where the VM is destroyed from another process, which this series cannot
detect.

Without [1], the nVMX vCPU teardown path described there may trigger
the new warning when the last reference to a VM is dropped by an
exiting task or by the delayed fput kworker. The write to guest memory
is then refused, which is also what [1] does.

For example, running Jim Mattson's selftest for that bug[3] (adapted
slightly) on a kernel with this series but without [1] gives the
following warning, and the write to guest memory is refused. The warning
comes from uaccess_warn_without_mm() because patch 10 was applied;
without it, it points at the inlined copy helper in the caller instead.

  WARNING: lib/usercopy.c:31 at uaccess_warn_without_mm+0xa/0x12, CPU#2: vmx_shadow_vmcs/5299
  Call Trace:
   kvm_write_guest_offset_cached.cold+0xa/0xf
   __nested_vmx_vmexit+0x1974/0x3fd0
   nested_vmx_free_vcpu+0xb3/0xd0
   vmx_vcpu_free+0x96/0x350
   kvm_arch_vcpu_destroy+0x15c/0x400
   kvm_destroy_vcpus+0x270/0x3f0
   kvm_arch_destroy_vm+0xc7/0x230
   kvm_put_kvm+0x560/0xbf0
   kvm_vm_release+0x3c/0x50
   __fput+0x3f8/0xb50
   fput_close_sync+0x113/0x250
   filp_close_sync+0x75/0xa0
   put_files_struct+0x201/0x390
   exit_files+0x97/0xd0
   do_exit+0x872/0x2a00
   do_group_exit+0xd5/0x2a0
   __x64_sys_exit_group+0x3c/0x50

== Testing ==

Build-tested on x86_64 and arm64. Verified that the KVM reproducer hits
the new WARN, and that the BPF reproducer hits it without patch 3.

I'm also fuzzing this series with syzkaller on x86_64 (syzbot's
upstream KASAN config, nested VMX enabled), so far for about a day and
a half. syzkaller hasn't triggered the new warning yet, i.e. it hasn't
found any other path that accesses user memory without an mm. Fuzzing
is ongoing, and I'll report anything it finds in reply to this RFC.

Please let me know what you think. Thank you!

[1] https://lore.kernel.org/kvm/20261001202234.3794060-1-seanjc@google.com
[2] https://lore.kernel.org/kvm/aqMBidJ_96qroogH@google.com/
[3] https://lore.kernel.org/kvm/20260911191011.528460-1-jmattson@google.com/

James Houghton (10):
  scsi: bsg: Don't copy sense data to user memory when canceled
  btrfs: Don't copy encoded read data to user memory when canceled
  bpf: Fail sleepable user copy helpers when current->mm is NULL
  stacktrace: Don't unwind the user stack without an mm
  perf/core: Don't unwind the user stack without an mm
  uaccess: Refuse and warn on user copies when current->mm is NULL
  uaccess: Fail user nofault helpers quietly when current->mm is NULL
  uaccess: Refuse and warn on __copy_from_user_inatomic() without an mm
  uaccess: Refuse and warn on more user accesses without an mm
  uaccess: Move the NULL mm warning out of line

 drivers/scsi/scsi_bsg.c   | 10 +++++++-
 fs/btrfs/ioctl.c          | 10 ++++++++
 include/linux/uaccess.h   | 50 +++++++++++++++++++++++++++++++++++----
 kernel/bpf/helpers.c      | 11 +++++++--
 kernel/events/callchain.c | 12 ++++++----
 kernel/stacktrace.c       |  8 +++++--
 kernel/trace/bpf_trace.c  |  4 ++++
 lib/iov_iter.c            | 12 ++++++++--
 lib/strncpy_from_user.c   |  2 +-
 lib/strnlen_user.c        |  3 +++
 lib/usercopy.c            |  9 +++++++
 mm/maccess.c              | 17 +++++++++++++
 12 files changed, 131 insertions(+), 17 deletions(-)


base-commit: 2a08195e4db5aba84600db6518089a1dbe2d04c7
-- 
2.56.0.307.g5fe7bd680c-goog

[RFC PATCH 01/10] scsi: bsg: Don't copy sense data to user memory when canceled

[James Houghton]

scsi_bsg_uring_task_cb() copies the sense data to the submitter's
response_addr with copy_to_user(). It runs as io_uring task work, but
io_uring may also run it with tw.cancel set from other contexts when the
submitting task can't run it: from a kworker when the submitting task
has exited (io_tctx_fallback_work()), or from io_cancel_local_task_work()
when a DEFER_TASKRUN ring is torn down.

In a kworker, current->mm is NULL, and the copy does not reliably fail:
the user half of the address space belongs to whatever mm happens to be
lazily loaded, if any, so up to max_response_len bytes of sense data may
be written into an unrelated process.

Don't copy the sense data when the task work is canceled, and complete
the command with -ECANCELED.

Fixes: 7b6d3255e7f8 ("scsi: bsg: add io_uring passthrough handler")
Signed-off-by: James Houghton <jthoughton@google.com>
---
 drivers/scsi/scsi_bsg.c | 10 +++++++++-
 1 file changed, 9 insertions(+), 1 deletion(-)

diff --git a/drivers/scsi/scsi_bsg.c b/drivers/scsi/scsi_bsg.c
index 5eec248a77a6..3755e33fcdd2 100644
--- a/drivers/scsi/scsi_bsg.c
+++ b/drivers/scsi/scsi_bsg.c
@@ -43,9 +43,17 @@ static void scsi_bsg_uring_task_cb(struct io_tw_req tw_req, io_tw_token_t tw)
 	if (pdu->bio)
 		blk_rq_unmap_user(pdu->bio);
 
+	/*
+	 * If the request is being canceled, e.g. because the submitting task
+	 * has exited, this may run from a kworker or in a different task, so
+	 * don't copy the sense data to the submitter's address.
+	 */
+	if (unlikely(tw.cancel))
+		ret = -ECANCELED;
+
 	if (scsi_status_is_check_condition(scmd->result)) {
 		driver_status = DRIVER_SENSE;
-		if (pdu->response_addr)
+		if (pdu->response_addr && !tw.cancel)
 			sense_len_wr = min_t(unsigned int, pdu->max_response_len,
 					     scmd->sense_len);
 	}
-- 
2.56.0.307.g5fe7bd680c-goog

[RFC PATCH 02/10] btrfs: Don't copy encoded read data to user memory when canceled

[James Houghton]

btrfs_uring_read_finished() copies the data of an io_uring encoded read
to the submitter's buffers with copy_page_to_iter(). It runs as io_uring
task work, but io_uring may also run it with tw.cancel set from other
contexts when the submitting task can't run it: from a kworker when the
submitting task has exited (io_tctx_fallback_work()), or from
io_cancel_local_task_work() when a DEFER_TASKRUN ring is torn down.

In a kworker, current->mm is NULL, and the copy does not reliably fail:
the user half of the address space belongs to whatever mm happens to be
lazily loaded, if any, so the data may be written into an unrelated
process.

Don't copy the data when the task work is canceled, and complete the
command with -ECANCELED.

Fixes: 34310c442e17 ("btrfs: add io_uring command for encoded reads (ENCODED_READ ioctl)")
Signed-off-by: James Houghton <jthoughton@google.com>
---
 fs/btrfs/ioctl.c | 10 ++++++++++
 1 file changed, 10 insertions(+)

diff --git a/fs/btrfs/ioctl.c b/fs/btrfs/ioctl.c
index f3e2afe221be..d9044fbd9547 100644
--- a/fs/btrfs/ioctl.c
+++ b/fs/btrfs/ioctl.c
@@ -4609,6 +4609,16 @@ static void btrfs_uring_read_finished(struct io_tw_req tw_req, io_tw_token_t tw)
 		goto out;
 	}
 
+	/*
+	 * If the request is being canceled, e.g. because the submitting task
+	 * has exited, this may run from a kworker or in a different task, so
+	 * don't copy the data to the submitter's buffers.
+	 */
+	if (unlikely(tw.cancel)) {
+		ret = -ECANCELED;
+		goto out;
+	}
+
 	if (priv->compressed) {
 		index = 0;
 		page_offset = 0;
-- 
2.56.0.307.g5fe7bd680c-goog

[RFC PATCH 03/10] bpf: Fail sleepable user copy helpers when current->mm is NULL

[James Houghton]

Sleepable BPF programs can run on behalf of a kernel thread, or of a
task that has already gone through exit_mm(). For example, bpf_wq
callbacks always run from a kworker, bpf_task_work callbacks queued
late enough run from exit_task_work(), and lsm.s or fentry.s programs
may be attached to functions called during exit_files(),
exit_task_work() or from kernel threads (e.g. security_file_release()
from a kworker running delayed_fput()).

In such contexts, bpf_copy_from_user(), bpf_copy_from_user_str() and
bpf_copy_from_user{,_str}_dynptr() (when reading from the current task)
call copy_from_user() or strncpy_from_user() without an mm. That access
does not reliably fail: the user half of the address space belongs to
whatever mm happens to be lazily loaded, if any, so the program may read
another task's memory.

As there is no user memory to read from in this case, fail with -EFAULT
without attempting the access.

Fixes: 07be4c4a3e7a ("bpf: Add bpf_copy_from_user() helper.")
Fixes: 65ab5ac4df01 ("bpf: Add bpf_copy_from_user_str kfunc")
Fixes: a498ee7576de ("bpf: Implement dynptr copy kfuncs")
Signed-off-by: James Houghton <jthoughton@google.com>
---
Tested with a small bpf_wq reproducer: a SEC("syscall") program arms a
bpf_wq whose callback (which runs in a kworker) calls
bpf_copy_from_user() on an address in the loader. With the uaccess check
from later in this series but without this patch, that triggers the new
WARN from bpf_copy_from_user(); with this patch, the helper returns
-EFAULT without warning. I can turn this into a selftest under
tools/testing/selftests/bpf if that's useful.
---
 kernel/bpf/helpers.c     | 11 +++++++++--
 kernel/trace/bpf_trace.c |  4 ++++
 2 files changed, 13 insertions(+), 2 deletions(-)

diff --git a/kernel/bpf/helpers.c b/kernel/bpf/helpers.c
index 712dca5a2c5b..c4aa8bcaf2bd 100644
--- a/kernel/bpf/helpers.c
+++ b/kernel/bpf/helpers.c
@@ -659,7 +659,11 @@ const struct bpf_func_proto bpf_event_output_data_proto =  {
 BPF_CALL_3(bpf_copy_from_user, void *, dst, u32, size,
 	   const void __user *, user_ptr)
 {
-	int ret = copy_from_user(dst, user_ptr, size);
+	int ret = -EFAULT;
+
+	/* Sleepable programs may run without an mm, e.g. on kernel threads. */
+	if (likely(current->mm))
+		ret = copy_from_user(dst, user_ptr, size);
 
 	if (unlikely(ret)) {
 		memset(dst, 0, size);
@@ -3642,7 +3646,10 @@ __bpf_kfunc int bpf_copy_from_user_str(void *dst, u32 dst__sz, const void __user
 	if (unlikely(!dst__sz))
 		return 0;
 
-	ret = strncpy_from_user(dst, unsafe_ptr__ign, dst__sz - 1);
+	if (unlikely(!current->mm))
+		ret = -EFAULT;
+	else
+		ret = strncpy_from_user(dst, unsafe_ptr__ign, dst__sz - 1);
 	if (ret < 0) {
 		if (flags & BPF_F_PAD_ZEROS)
 			memset((char *)dst, 0, dst__sz);
diff --git a/kernel/trace/bpf_trace.c b/kernel/trace/bpf_trace.c
index 88c3b5b513ab..793fb3dce5f1 100644
--- a/kernel/trace/bpf_trace.c
+++ b/kernel/trace/bpf_trace.c
@@ -3556,6 +3556,8 @@ static __always_inline int copy_user_data_sleepable(void *dst, const void *unsaf
 	int ret;
 
 	if (!tsk) { /* Read from the current task */
+		if (unlikely(!current->mm))
+			return -EFAULT;
 		ret = copy_from_user(dst, (const void __user *)unsafe_src, size);
 		if (ret)
 			return -EFAULT;
@@ -3590,6 +3592,8 @@ static __always_inline int copy_user_str_sleepable(void *dst, const void *unsafe
 
 	if (tsk) {
 		ret = copy_remote_vm_str(tsk, (unsigned long)unsafe_src, dst, size, 0);
+	} else if (unlikely(!current->mm)) {
+		ret = -EFAULT;
 	} else {
 		ret = strncpy_from_user(dst, (const void __user *)unsafe_src, size - 1);
 		/* strncpy_from_user does not guarantee NUL termination */
-- 
2.56.0.307.g5fe7bd680c-goog

[RFC PATCH 04/10] stacktrace: Don't unwind the user stack without an mm

[James Houghton]

stack_trace_save_user() only refuses to unwind the user stack of tasks
with PF_KTHREAD set. That doesn't cover all tasks without an mm:

 - A user task that has already gone through exit_mm(), e.g. when an
   event fires from exit_files(), exit_task_work() or the final context
   switch in do_exit().
 - PID 1 before it execs init, and usermode helper children before they
   exec, which are created with user_mode_thread() and don't have
   PF_KTHREAD set. E.g. booting with trace_options=userstacktrace and
   an event enabled with trace_event= unwinds PID 1's "user stack"
   during initcalls.

With the ftrace userstacktrace option enabled, the architecture
unwinders then read the user stack without an mm. That access does not
reliably fail: the user half of the address space belongs to whatever
mm happens to be lazily loaded, if any, so the unwinder may read
another task's memory.

Use is_user_task() instead, which also checks current->mm, as perf
already does for the same reason since commit 76ed27608f7d ("perf:
sched: Fix perf crash with new is_user_task() helper"). This also stops
unwinding the user stack of PF_USER_WORKER threads, which never run in
user space, matching perf.

Fixes: 214d8ca6ee85 ("stacktrace: Provide common infrastructure")
Signed-off-by: James Houghton <jthoughton@google.com>
---
 kernel/stacktrace.c | 8 ++++++--
 1 file changed, 6 insertions(+), 2 deletions(-)

diff --git a/kernel/stacktrace.c b/kernel/stacktrace.c
index afb3c116da91..f96a5ec12f6c 100644
--- a/kernel/stacktrace.c
+++ b/kernel/stacktrace.c
@@ -228,8 +228,12 @@ unsigned int stack_trace_save_user(unsigned long *store, unsigned int size)
 		.size	= size,
 	};
 
-	/* Trace user stack if not a kernel thread */
-	if (current->flags & PF_KTHREAD)
+	/*
+	 * Trace user stack only if this is a user task that still has an mm,
+	 * i.e. not a kernel thread or user worker, and not a task that has
+	 * already gone through exit_mm().
+	 */
+	if (!is_user_task(current))
 		return 0;
 
 	arch_stack_walk_user(consume_entry, &c, task_pt_regs(current));
-- 
2.56.0.307.g5fe7bd680c-goog

[RFC PATCH 05/10] perf/core: Don't unwind the user stack without an mm

[James Houghton]

get_perf_callchain() only checks is_user_task(current) when the regs it
is given aren't user mode regs. However, BPF's bpf_get_task_stack() (and
bpf_get_task_stack_sleepable()) pass task_pt_regs(current) when called on
the current task, which are user mode regs for a user task, even if that
task has already gone through exit_mm(), e.g. when called from a program
attached to exit_files() or to the dying task's final sched_switch.

perf_callchain_user() then unwinds the user stack without an mm. That
access does not reliably fail: the user half of the address space
belongs to whatever mm happens to be lazily loaded, if any, so the
unwinder may read another task's memory. This affects e.g. arm64,
riscv, loongarch and csky, whose perf_callchain_user() doesn't check
current->mm itself. perf events themselves are not affected, as
perf_callchain() already clears the user callchain request when
current is not a user task with an mm.

Check is_user_task(current) regardless of the regs' mode.

Fixes: fa28dcb82a38 ("bpf: Introduce helper bpf_get_task_stack()")
Signed-off-by: James Houghton <jthoughton@google.com>
---
 kernel/events/callchain.c | 12 ++++++++----
 1 file changed, 8 insertions(+), 4 deletions(-)

diff --git a/kernel/events/callchain.c b/kernel/events/callchain.c
index 9d24b6e0c91f..a451ca2a5573 100644
--- a/kernel/events/callchain.c
+++ b/kernel/events/callchain.c
@@ -245,11 +245,15 @@ get_perf_callchain(struct pt_regs *regs, bool kernel, bool user,
 	}
 
 	if (user && !crosstask) {
-		if (!user_mode(regs)) {
-			if (!is_user_task(current))
-				goto exit_put;
+		/*
+		 * Even if @regs are user mode regs, e.g. task_pt_regs(current)
+		 * passed in by bpf_get_task_stack(), current may already have
+		 * gone through exit_mm().
+		 */
+		if (!is_user_task(current))
+			goto exit_put;
+		if (!user_mode(regs))
 			regs = task_pt_regs(current);
-		}
 
 		if (defer_cookie) {
 			/*
-- 
2.56.0.307.g5fe7bd680c-goog

[RFC PATCH 06/10] uaccess: Refuse and warn on user copies when current->mm is NULL

[James Houghton]

Accessing user memory from a task that has no mm is always a bug. This
can happen from a kernel thread that is not using kthread_use_mm(), or
from a task that has already gone through exit_mm() (e.g. when the last
reference to a file is dropped from task_work at exit). KVM has had
bugs of exactly this kind, and other drivers may have them too.

Such an access does not reliably fail. The user portion of the address
space belongs to whatever mm happens to be lazily loaded, if any. If
the access faults, the fault handler refuses to handle it and the copy
fails, but if the page is already mapped, the copy silently reads or
writes another task's memory.

Add uaccess_should_fail(), which combines the existing
should_fail_usercopy() fault injection check with a
WARN_ON_ONCE(!current->mm), and use it in place of every
should_fail_usercopy() call. The user copy is then refused as if it had
failed, so no access to user memory takes place.

This covers copy_{to,from}_user() on all architectures, including the
masked user access paths that never call access_ok(), and the
__copy_{to,from}_user() variants used by KVM's guest memory accessors.
It also covers __copy_to_user_inatomic() (and so copy_to_user_nofault()),
the user-backed iov_iter copy routines and strncpy_from_user().

Paths that never called should_fail_usercopy() are not covered, e.g.
__copy_from_user_inatomic(), copy_from_user_nofault(), strnlen_user(),
get_user()/put_user(), clear_user() and unsafe_*() accesses within
user_access_begin() sections.

Signed-off-by: James Houghton <jthoughton@google.com>
---
 include/linux/uaccess.h | 27 ++++++++++++++++++++++-----
 lib/iov_iter.c          |  6 +++---
 lib/strncpy_from_user.c |  2 +-
 3 files changed, 26 insertions(+), 9 deletions(-)

diff --git a/include/linux/uaccess.h b/include/linux/uaccess.h
index eddbbb65ccc4..8ab24e706a22 100644
--- a/include/linux/uaccess.h
+++ b/include/linux/uaccess.h
@@ -50,6 +50,23 @@
  #define mask_user_address(src) (src)
 #endif
 
+/*
+ * Return true if a user copy must not be attempted, either because a failure
+ * is being injected, or because the current task has no mm.
+ *
+ * Accessing user memory without an mm (e.g. from a kernel thread that is not
+ * using kthread_use_mm(), or from a task that has already gone through
+ * exit_mm()) is always a bug: the user portion of the address space belongs to
+ * whatever mm happens to be lazily loaded, if any, so the access could
+ * silently hit another task's memory.
+ */
+static __always_inline bool uaccess_should_fail(void)
+{
+	if (WARN_ON_ONCE(!current->mm))
+		return true;
+	return should_fail_usercopy();
+}
+
 /*
  * Architectures should provide two primitives (raw_copy_{to,from}_user())
  * and get rid of their private instances of copy_{to,from}_user() and
@@ -114,7 +131,7 @@ __copy_from_user(void *to, const void __user *from, unsigned long n)
 
 	might_fault();
 	instrument_copy_from_user_before(to, from, n);
-	if (should_fail_usercopy())
+	if (uaccess_should_fail())
 		return n;
 	check_object_size(to, n, false);
 	res = raw_copy_from_user(to, from, n);
@@ -138,7 +155,7 @@ __copy_from_user(void *to, const void __user *from, unsigned long n)
 static __always_inline __must_check unsigned long
 __copy_to_user_inatomic(void __user *to, const void *from, unsigned long n)
 {
-	if (should_fail_usercopy())
+	if (uaccess_should_fail())
 		return n;
 	instrument_copy_to_user(to, from, n);
 	check_object_size(from, n, true);
@@ -149,7 +166,7 @@ static __always_inline __must_check unsigned long
 __copy_to_user(void __user *to, const void *from, unsigned long n)
 {
 	might_fault();
-	if (should_fail_usercopy())
+	if (uaccess_should_fail())
 		return n;
 	instrument_copy_to_user(to, from, n);
 	check_object_size(from, n, true);
@@ -167,7 +184,7 @@ _inline_copy_from_user(void *to, const void __user *from, unsigned long n)
 {
 	unsigned long res = n;
 	might_fault();
-	if (should_fail_usercopy())
+	if (uaccess_should_fail())
 		goto fail;
 	if (can_do_masked_user_access())
 		from = mask_user_address(from);
@@ -195,7 +212,7 @@ static inline __must_check unsigned long
 _inline_copy_to_user(void __user *to, const void *from, unsigned long n)
 {
 	might_fault();
-	if (should_fail_usercopy())
+	if (uaccess_should_fail())
 		return n;
 	if (access_ok(to, n)) {
 		instrument_copy_to_user(to, from, n);
diff --git a/lib/iov_iter.c b/lib/iov_iter.c
index 2072c04e99d0..3d6ecb50d778 100644
--- a/lib/iov_iter.c
+++ b/lib/iov_iter.c
@@ -17,7 +17,7 @@ static __always_inline
 size_t copy_to_user_iter(void __user *iter_to, size_t progress,
 			 size_t len, void *from, void *priv2)
 {
-	if (should_fail_usercopy())
+	if (uaccess_should_fail())
 		return len;
 	if (access_ok(iter_to, len)) {
 		from += progress;
@@ -33,7 +33,7 @@ size_t copy_to_user_iter_nofault(void __user *iter_to, size_t progress,
 {
 	ssize_t res;
 
-	if (should_fail_usercopy())
+	if (uaccess_should_fail())
 		return len;
 
 	from += progress;
@@ -47,7 +47,7 @@ size_t copy_from_user_iter(void __user *iter_from, size_t progress,
 {
 	size_t res = len;
 
-	if (should_fail_usercopy())
+	if (uaccess_should_fail())
 		return len;
 	if (can_do_masked_user_access()) {
 		iter_from = mask_user_address(iter_from);
diff --git a/lib/strncpy_from_user.c b/lib/strncpy_from_user.c
index 5bb752ff7c61..565b3125e752 100644
--- a/lib/strncpy_from_user.c
+++ b/lib/strncpy_from_user.c
@@ -115,7 +115,7 @@ long strncpy_from_user(char *dst, const char __user *src, long count)
 	unsigned long max_addr, src_addr;
 
 	might_fault();
-	if (should_fail_usercopy())
+	if (uaccess_should_fail())
 		return -EFAULT;
 	if (unlikely(count <= 0))
 		return 0;
-- 
2.56.0.307.g5fe7bd680c-goog

[RFC PATCH 07/10] uaccess: Fail user nofault helpers quietly when current->mm is NULL

[James Houghton]

The user nofault helpers (copy_{from,to}_user_nofault(),
strncpy_from_user_nofault() and strnlen_user_nofault()) are meant to be
callable from any context, e.g. from BPF programs, kprobes or perf, which
may run on behalf of a kernel thread or of a task that has already gone
through exit_mm(). In such a context, there is no user memory to access,
and the right answer is simply to fail.

Since the previous commit, the uaccess helpers they use warn when
current->mm is NULL, e.g. bpf_probe_read_user_str() from a BPF program
attached to a function called by a kernel thread would now trip the
WARN_ON_ONCE() in strncpy_from_user().

Check for a NULL current->mm up front in these helpers, and fail without
warning and without attempting the access.

On x86, copy_from_user_nofault() was already refusing such accesses, as
nmi_uaccess_okay() returns false when current->mm is not the loaded mm.

For the same reason, make copy_to_user_iter_nofault() use
should_fail_usercopy() directly, leaving the NULL mm check to
copy_to_user_nofault().

Signed-off-by: James Houghton <jthoughton@google.com>
---
 lib/iov_iter.c |  2 +-
 mm/maccess.c   | 17 +++++++++++++++++
 2 files changed, 18 insertions(+), 1 deletion(-)

diff --git a/lib/iov_iter.c b/lib/iov_iter.c
index 3d6ecb50d778..21e2ccf1e8f2 100644
--- a/lib/iov_iter.c
+++ b/lib/iov_iter.c
@@ -33,7 +33,7 @@ size_t copy_to_user_iter_nofault(void __user *iter_to, size_t progress,
 {
 	ssize_t res;
 
-	if (uaccess_should_fail())
+	if (should_fail_usercopy())
 		return len;
 
 	from += progress;
diff --git a/mm/maccess.c b/mm/maccess.c
index 486559d68858..678bb6165b39 100644
--- a/mm/maccess.c
+++ b/mm/maccess.c
@@ -121,6 +121,14 @@ long copy_from_user_nofault(void *dst, const void __user *src, size_t size)
 {
 	long ret = -EFAULT;
 
+	/*
+	 * The user nofault helpers may be called from any context, including
+	 * from tasks that have no user memory to access (e.g. kernel threads).
+	 * Fail quietly instead of tripping the warning in the uaccess helpers.
+	 */
+	if (!current->mm)
+		return ret;
+
 	if (!__access_ok(src, size))
 		return ret;
 
@@ -150,6 +158,9 @@ long copy_to_user_nofault(void __user *dst, const void *src, size_t size)
 {
 	long ret = -EFAULT;
 
+	if (!current->mm)
+		return ret;
+
 	if (access_ok(dst, size)) {
 		pagefault_disable();
 		ret = __copy_to_user_inatomic(dst, src, size);
@@ -188,6 +199,9 @@ long strncpy_from_user_nofault(char *dst, const void __user *unsafe_addr,
 	if (unlikely(count <= 0))
 		return 0;
 
+	if (!current->mm)
+		return -EFAULT;
+
 	pagefault_disable();
 	ret = strncpy_from_user(dst, unsafe_addr, count);
 	pagefault_enable();
@@ -222,6 +236,9 @@ long strnlen_user_nofault(const void __user *unsafe_addr, long count)
 {
 	int ret;
 
+	if (!current->mm)
+		return 0;
+
 	pagefault_disable();
 	ret = strnlen_user(unsafe_addr, count);
 	pagefault_enable();
-- 
2.56.0.307.g5fe7bd680c-goog

[RFC PATCH 08/10] uaccess: Refuse and warn on __copy_from_user_inatomic() without an mm

[James Houghton]

__copy_from_user_inatomic() never called should_fail_usercopy(), so the
NULL current->mm check added to uaccess_should_fail() does not cover it.
It is used e.g. by KVM's kvm_read_guest_atomic(), by several
architectures' perf callchain and stack trace code, and by the generic
copy_from_user_inatomic_nontemporal().

Split the NULL mm check out of uaccess_should_fail() into
uaccess_without_mm(), and use it in __copy_from_user_inatomic() to warn
and refuse the copy.

Don't use uaccess_should_fail() there, as that would also start injecting
failures into __copy_from_user_inatomic(), which some callers don't
expect to fail in practice, e.g. __wp_page_copy_user() warns if the copy
fails twice.

copy_from_user_nofault() already fails quietly when current->mm is NULL,
and the generic user stack unwinding entry points (perf_callchain(),
get_perf_callchain() and stack_trace_save_user()) only unwind the user
stack of a user task with an mm, so neither is affected.

Signed-off-by: James Houghton <jthoughton@google.com>
---
 include/linux/uaccess.h | 25 ++++++++++++++++++++-----
 1 file changed, 20 insertions(+), 5 deletions(-)

diff --git a/include/linux/uaccess.h b/include/linux/uaccess.h
index 8ab24e706a22..eae204f7261b 100644
--- a/include/linux/uaccess.h
+++ b/include/linux/uaccess.h
@@ -51,20 +51,33 @@
 #endif
 
 /*
- * Return true if a user copy must not be attempted, either because a failure
- * is being injected, or because the current task has no mm.
+ * Return true, and warn, if the current task has no mm, in which case user
+ * memory must not be accessed.
  *
  * Accessing user memory without an mm (e.g. from a kernel thread that is not
  * using kthread_use_mm(), or from a task that has already gone through
  * exit_mm()) is always a bug: the user portion of the address space belongs to
  * whatever mm happens to be lazily loaded, if any, so the access could
  * silently hit another task's memory.
+ *
+ * Note that this only catches a missing mm. It cannot catch a valid mm that
+ * isn't the one the caller expects, e.g. when a file's release or ioctl
+ * handlers run in a different process than the one that created it. Code
+ * that keeps user addresses belonging to a specific mm must check
+ * current->mm against that mm itself (see e.g. KVM's kvm->mm checks).
+ */
+static __always_inline bool uaccess_without_mm(void)
+{
+	return WARN_ON_ONCE(!current->mm);
+}
+
+/*
+ * Return true if a user copy must not be attempted, either because the
+ * current task has no mm, or because a failure is being injected.
  */
 static __always_inline bool uaccess_should_fail(void)
 {
-	if (WARN_ON_ONCE(!current->mm))
-		return true;
-	return should_fail_usercopy();
+	return uaccess_without_mm() || should_fail_usercopy();
 }
 
 /*
@@ -117,6 +130,8 @@ __copy_from_user_inatomic(void *to, const void __user *from, unsigned long n)
 {
 	unsigned long res;
 
+	if (uaccess_without_mm())
+		return n;
 	instrument_copy_from_user_before(to, from, n);
 	check_object_size(to, n, false);
 	res = raw_copy_from_user(to, from, n);
-- 
2.56.0.307.g5fe7bd680c-goog

[RFC PATCH 09/10] uaccess: Refuse and warn on more user accesses without an mm

[James Houghton]

Several generic user access paths don't go through the copy helpers
that check for a NULL current->mm. Add the uaccess_without_mm() check to
them as well, so that they warn and fail without attempting the access:

 - __scoped_user_access_begin(), covering scoped_user_*_access() and
   get_user_inline()/put_user_inline() on all architectures, including
   when masked user access is used.
 - strnlen_user().
 - check_zeroed_user().
 - The iov_iter user copy routines that call into architecture-specific
   helpers: copy_to_user_iter_mc() (copy_mc_to_user()),
   copy_from_user_iter_nocache() (copy_from_user_inatomic_nontemporal()),
   copy_from_user_iter_flushcache() (copy_from_user_flushcache()) and
   zero_to_user_iter() (clear_user()).

Only check for a NULL mm here rather than using uaccess_should_fail(),
so as not to start injecting failures in these paths.

Paths implemented entirely by architectures, e.g. get_user(),
put_user(), clear_user(), and direct users of user_access_begin() or
masked_user_access_begin(), are still not covered.

Signed-off-by: James Houghton <jthoughton@google.com>
---
 include/linux/uaccess.h | 2 ++
 lib/iov_iter.c          | 8 ++++++++
 lib/strnlen_user.c      | 3 +++
 lib/usercopy.c          | 3 +++
 4 files changed, 16 insertions(+)

diff --git a/include/linux/uaccess.h b/include/linux/uaccess.h
index eae204f7261b..7d26d2de9599 100644
--- a/include/linux/uaccess.h
+++ b/include/linux/uaccess.h
@@ -781,6 +781,8 @@ static __always_inline void __scoped_user_rw_access_end(const void *p)
 ({									\
 	typeof(uptr) __retptr;						\
 									\
+	if (uaccess_without_mm())					\
+		goto elbl;						\
 	if (can_do_masked_user_access()) {				\
 		__retptr = masked_user_##mode##_access_begin(uptr);	\
 	} else {							\
diff --git a/lib/iov_iter.c b/lib/iov_iter.c
index 21e2ccf1e8f2..a0769874241c 100644
--- a/lib/iov_iter.c
+++ b/lib/iov_iter.c
@@ -204,6 +204,8 @@ static __always_inline
 size_t copy_to_user_iter_mc(void __user *iter_to, size_t progress,
 			    size_t len, void *from, void *priv2)
 {
+	if (uaccess_without_mm())
+		return len;
 	if (access_ok(iter_to, len)) {
 		from += progress;
 		instrument_copy_to_user(iter_to, from, len);
@@ -277,6 +279,8 @@ static __always_inline
 size_t copy_from_user_iter_nocache(void __user *iter_from, size_t progress,
 				   size_t len, void *to, void *priv2)
 {
+	if (uaccess_without_mm())
+		return len;
 	return copy_from_user_inatomic_nontemporal(to + progress, iter_from, len);
 }
 
@@ -296,6 +300,8 @@ static __always_inline
 size_t copy_from_user_iter_flushcache(void __user *iter_from, size_t progress,
 				      size_t len, void *to, void *priv2)
 {
+	if (uaccess_without_mm())
+		return len;
 	return copy_from_user_flushcache(to + progress, iter_from, len);
 }
 
@@ -451,6 +457,8 @@ static __always_inline
 size_t zero_to_user_iter(void __user *iter_to, size_t progress,
 			 size_t len, void *priv, void *priv2)
 {
+	if (uaccess_without_mm())
+		return len;
 	return clear_user(iter_to, len);
 }
 
diff --git a/lib/strnlen_user.c b/lib/strnlen_user.c
index 4a6574b67f82..45709798eb8f 100644
--- a/lib/strnlen_user.c
+++ b/lib/strnlen_user.c
@@ -96,6 +96,9 @@ long strnlen_user(const char __user *str, long count)
 	if (unlikely(count <= 0))
 		return 0;
 
+	if (uaccess_without_mm())
+		return 0;
+
 	if (can_do_masked_user_access()) {
 		long retval;
 
diff --git a/lib/usercopy.c b/lib/usercopy.c
index e2f0bf104a59..9146c729c47a 100644
--- a/lib/usercopy.c
+++ b/lib/usercopy.c
@@ -48,6 +48,9 @@ int check_zeroed_user(const void __user *from, size_t size)
 	if (unlikely(size == 0))
 		return 1;
 
+	if (uaccess_without_mm())
+		return -EFAULT;
+
 	from -= align;
 	size += align;
 
-- 
2.56.0.307.g5fe7bd680c-goog

[RFC PATCH 10/10] uaccess: Move the NULL mm warning out of line

[James Houghton]

uaccess_without_mm() is inlined into every user copy, including every
copy_{to,from}_user() call site on architectures that define
INLINE_COPY_USER, such as arm64. Each inlined WARN_ON_ONCE() adds its
own __bug_table entry.

Keep the NULL current->mm check inline, but move the warning into a
single out-of-line, __cold function, uaccess_warn_without_mm().

The trade-off is that only the first offending access in the system
warns, rather than the first one from each call site. The stack trace
still identifies the caller, and the access is refused either way.

Compared to the base of this series, with defconfig + KVM:

                     inline WARN_ON_ONCE()    out-of-line warning
  x86_64 total       +5,884 B (+0.014%)       +9,714 B (+0.023%)
  arm64  total      +40,948 B (+0.10%)       +24,609 B (+0.06%)

On arm64, this removes the ~16 KB of __bug_table growth, while text
size is unchanged, as a call is the same size as the inline brk. On
x86_64, the call is larger than the inline ud2 and causes more .cold
splitting, so this is a slight regression.

Signed-off-by: James Houghton <jthoughton@google.com>
---
NOTE: This patch is included for reference only, to show the cost of the
      inline warning.
---
 include/linux/uaccess.h | 8 +++++++-
 lib/usercopy.c          | 6 ++++++
 2 files changed, 13 insertions(+), 1 deletion(-)

diff --git a/include/linux/uaccess.h b/include/linux/uaccess.h
index 7d26d2de9599..6a5b6c2b3388 100644
--- a/include/linux/uaccess.h
+++ b/include/linux/uaccess.h
@@ -66,9 +66,15 @@
  * that keeps user addresses belonging to a specific mm must check
  * current->mm against that mm itself (see e.g. KVM's kvm->mm checks).
  */
+void __cold uaccess_warn_without_mm(void);
+
 static __always_inline bool uaccess_without_mm(void)
 {
-	return WARN_ON_ONCE(!current->mm);
+	if (likely(current->mm))
+		return false;
+	/* Warn out of line to keep inlined user copies small. */
+	uaccess_warn_without_mm();
+	return true;
 }
 
 /*
diff --git a/lib/usercopy.c b/lib/usercopy.c
index 9146c729c47a..3d547ecb74df 100644
--- a/lib/usercopy.c
+++ b/lib/usercopy.c
@@ -26,6 +26,12 @@ unsigned long _copy_to_user(void __user *to, const void *from, unsigned long n)
 EXPORT_SYMBOL(_copy_to_user);
 #endif
 
+void __cold uaccess_warn_without_mm(void)
+{
+	WARN_ON_ONCE(1);
+}
+EXPORT_SYMBOL(uaccess_warn_without_mm);
+
 /**
  * check_zeroed_user: check if a userspace buffer only contains zero bytes
  * @from: Source address, in userspace.
-- 
2.56.0.307.g5fe7bd680c-goog

Re: [RFC PATCH 04/10] stacktrace: Don't unwind the user stack without an mm

[bot+bpf-ci]

[-- Attachment #1: Type: text/plain, Size: 2649 bytes --]

> stacktrace: Don't unwind the user stack without an mm

TL;DR: Commit message only: the Fixes: tag may point at the wrong commit;
7e8e6816c649, which dropped the !current->mm check, looks like the real
culprit rather than 214d8ca6ee85.

> stacktrace: Don't unwind the user stack without an mm
>
> stack_trace_save_user() only refuses to unwind the user stack of tasks
> with PF_KTHREAD set. That doesn't cover all tasks without an mm:
>  - A user task that has already gone through exit_mm(), e.g. when an
>    event fires from exit_files(), exit_task_work() or the final context
>    switch in do_exit().
>  - PID 1 before it execs init, and usermode helper children before they
>    exec, which are created with user_mode_thread() and don't have
>    PF_KTHREAD set. E.g. booting with trace_options=userstacktrace and
>    an event enabled with trace_event= unwinds PID 1's "user stack"
>    during initcalls.
> With the ftrace userstacktrace option enabled, the architecture
> unwinders then read the user stack without an mm. That access does not
> reliably fail: the user half of the address space belongs to whatever
> mm happens to be lazily loaded, if any, so the unwinder may read
> another task's memory.
> Use is_user_task() instead, which also checks current->mm, as perf
> already does for the same reason since commit 76ed27608f7d ("perf:
> sched: Fix perf crash with new is_user_task() helper"). This also stops
> unwinding the user stack of PF_USER_WORKER threads, which never run in
> user space, matching perf.
> Fixes: 214d8ca6ee85 ("stacktrace: Provide common infrastructure")
> Signed-off-by: James Houghton <jthoughton@google.com>

Is the Fixes: tag pointing at the right commit?

When 214d8ca6ee85 added the ARCH_STACKWALK version of
stack_trace_save_user(), it guarded the unwind with:

	/* Trace user stack if not a kernel thread */
	if (!current->mm)
		return 0;

That check already covers both cases described above (a task past exit_mm(),
and PID 1 or usermode helpers before exec, which have mm == NULL).

The check was replaced by commit 7e8e6816c649 ("stacktrace: Use PF_KTHREAD to
check for kernel threads"), which switched it to:

	if (current->flags & PF_KTHREAD)
		return 0;

That dropped the no-mm protection, which looks like what this patch restores
via is_user_task().

Should this be:

  Fixes: 7e8e6816c649 ("stacktrace: Use PF_KTHREAD to check for kernel threads")


---
AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md

CI run summary: https://github.com/kernel-patches/bpf/actions/runs/40410166379

Re: [RFC PATCH 03/10] bpf: Fail sleepable user copy helpers when current->mm is NULL

[bot+bpf-ci]

[-- Attachment #1: Type: text/plain, Size: 3377 bytes --]

> bpf: Fail sleepable user copy helpers when current->mm is NULL

TL;DR: Commit message only: the security_file_release() example of a sleepable
program running from a kworker looks inaccurate, since such a program appears
not to be attachable there. The fix itself looks justified.

> bpf: Fail sleepable user copy helpers when current->mm is NULL
>
> Sleepable BPF programs can run on behalf of a kernel thread, or of a
> task that has already gone through exit_mm(). For example, bpf_wq
> callbacks always run from a kworker, bpf_task_work callbacks queued
> late enough run from exit_task_work(), and lsm.s or fentry.s programs
> may be attached to functions called during exit_files(),
> exit_task_work() or from kernel threads (e.g. security_file_release()
> from a kworker running delayed_fput()).
> In such contexts, bpf_copy_from_user(), bpf_copy_from_user_str() and
> bpf_copy_from_user{,_str}_dynptr() (when reading from the current task)
> call copy_from_user() or strncpy_from_user() without an mm. That access
> does not reliably fail: the user half of the address space belongs to
> whatever mm happens to be lazily loaded, if any, so the program may read
> another task's memory.
> As there is no user memory to read from in this case, fail with -EFAULT
> without attempting the access.
> Fixes: 07be4c4a3e7a ("bpf: Add bpf_copy_from_user() helper.")
> Fixes: 65ab5ac4df01 ("bpf: Add bpf_copy_from_user_str kfunc")
> Fixes: a498ee7576de ("bpf: Implement dynptr copy kfuncs")
> Signed-off-by: James Houghton <jthoughton@google.com>

The commit message gives security_file_release() as an example of a place
where an lsm.s or fentry.s program can run from a kworker:

> (e.g. security_file_release() from a kworker running delayed_fput())

Is this example accurate for the tree being patched?

bpf_lsm_file_release is not in the sleepable_lsm_hooks set in
kernel/bpf/bpf_lsm.c, so btf_id_allow_sleepable() rejects a sleepable
BPF_PROG_TYPE_LSM program there.  bpf_lsm_file_free_security, the other hook
reached from __fput() through file_free(), is in untrusted_lsm_hooks rather
than the sleepable set.

For a BPF_PROG_TYPE_TRACING program, check_attach_sleepable() requires
within_error_injection_list(addr) (or a syscall prefix when
CONFIG_FUNCTION_ERROR_INJECTION is off), or an fmodret set with
KF_SLEEPABLE.  Neither security_file_release() nor bpf_lsm_file_release()
has ALLOW_ERROR_INJECTION, and the only fmodret sets are in
net/bpf/test_run.c, net/mptcp/bpf.c and bpf_testmod.

That suggests a program attached to security_file_release() is never
sleepable and cannot call bpf_copy_from_user() or the KF_SLEEPABLE copy
kfuncs.

The fix itself still looks justified, since bpf_wq callbacks run from a
kworker and is_async_cb_sleepable() is true for them.  Only the example
appears to be off.

Would an example that is actually reachable keep the changelog accurate?
For instance, bpf_lsm_task_alloc() when kthreadd forks a kernel thread
(current is kthreadd), or bpf_lsm_kernel_read_file() and bpf_lsm_file_open()
when a kworker loads firmware with kernel_read_file().  All three hooks are
in sleepable_lsm_hooks.


---
AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md

CI run summary: https://github.com/kernel-patches/bpf/actions/runs/40410166379

Re: [RFC PATCH 08/10] uaccess: Refuse and warn on __copy_from_user_inatomic() without an mm

[bot+bpf-ci]

[-- Attachment #1: Type: text/plain, Size: 2504 bytes --]

> uaccess: Refuse and warn on __copy_from_user_inatomic() without an mm

TL;DR: Commit message only: it names kvm_read_guest_atomic(), which no longer
exists in this tree; the review suggests kvm_vcpu_read_guest_atomic() instead.

>     uaccess: Refuse and warn on __copy_from_user_inatomic() without an mm
>
>     __copy_from_user_inatomic() never called should_fail_usercopy(), so the
>     NULL current->mm check added to uaccess_should_fail() does not cover it.
>     It is used e.g. by KVM's kvm_read_guest_atomic(), by several
>     architectures' perf callchain and stack trace code, and by the generic
>     copy_from_user_inatomic_nontemporal().
>
>     Split the NULL mm check out of uaccess_should_fail() into
>     uaccess_without_mm(), and use it in __copy_from_user_inatomic() to warn
>     and refuse the copy.
>
>     Don't use uaccess_should_fail() there, as that would also start injecting
>     failures into __copy_from_user_inatomic(), which some callers don't
>     expect to fail in practice, e.g. __wp_page_copy_user() warns if the copy
>     fails twice.
>
>     copy_from_user_nofault() already fails quietly when current->mm is NULL,
>     and the generic user stack unwinding entry points (perf_callchain(),
>     get_perf_callchain() and stack_trace_save_user()) only unwind the user
>     stack of a user task with an mm, so neither is affected.
>
>     Signed-off-by: James Houghton <jthoughton@google.com>

The changelog names kvm_read_guest_atomic() as a user of
__copy_from_user_inatomic(), but that function no longer exists in this tree.
It was removed by commit ef82eddc0e31 ("KVM: Remove kvm_read_guest_atomic()").

The KVM user is now kvm_vcpu_read_guest_atomic() in virt/kvm/kvm_main.c,
which calls the static __kvm_read_guest_atomic():

    static int __kvm_read_guest_atomic(struct kvm_memory_slot *slot, gfn_t gfn,
                                       void *data, int offset, unsigned long len)
    {
        ...
        pagefault_disable();
        r = __copy_from_user_inatomic(data, (void __user *)addr + offset, len);
        pagefault_enable();

A grep for kvm_read_guest_atomic only finds __kvm_read_guest_atomic() and
kvm_vcpu_read_guest_atomic().

Could the changelog say kvm_vcpu_read_guest_atomic() instead?


---
AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md

CI run summary: https://github.com/kernel-patches/bpf/actions/runs/40410166379

Re: [RFC PATCH 08/10] uaccess: Refuse and warn on __copy_from_user_inatomic() without an mm

[James Houghton]

On Mon, Oct 26, 2026 at 3:30 PM <bot+bpf-ci@kernel.org> wrote:
>
> > uaccess: Refuse and warn on __copy_from_user_inatomic() without an mm
>
> TL;DR: Commit message only: it names kvm_read_guest_atomic(), which no longer
> exists in this tree; the review suggests kvm_vcpu_read_guest_atomic() instead.
>
> >     uaccess: Refuse and warn on __copy_from_user_inatomic() without an mm
> >
> >     __copy_from_user_inatomic() never called should_fail_usercopy(), so the
> >     NULL current->mm check added to uaccess_should_fail() does not cover it.
> >     It is used e.g. by KVM's kvm_read_guest_atomic(), by several
> >     architectures' perf callchain and stack trace code, and by the generic
> >     copy_from_user_inatomic_nontemporal().
> >
> >     Split the NULL mm check out of uaccess_should_fail() into
> >     uaccess_without_mm(), and use it in __copy_from_user_inatomic() to warn
> >     and refuse the copy.
> >
> >     Don't use uaccess_should_fail() there, as that would also start injecting
> >     failures into __copy_from_user_inatomic(), which some callers don't
> >     expect to fail in practice, e.g. __wp_page_copy_user() warns if the copy
> >     fails twice.
> >
> >     copy_from_user_nofault() already fails quietly when current->mm is NULL,
> >     and the generic user stack unwinding entry points (perf_callchain(),
> >     get_perf_callchain() and stack_trace_save_user()) only unwind the user
> >     stack of a user task with an mm, so neither is affected.
> >
> >     Signed-off-by: James Houghton <jthoughton@google.com>
>
> The changelog names kvm_read_guest_atomic() as a user of
> __copy_from_user_inatomic(), but that function no longer exists in this tree.
> It was removed by commit ef82eddc0e31 ("KVM: Remove kvm_read_guest_atomic()").
>
> The KVM user is now kvm_vcpu_read_guest_atomic() in virt/kvm/kvm_main.c,
> which calls the static __kvm_read_guest_atomic():
>
>     static int __kvm_read_guest_atomic(struct kvm_memory_slot *slot, gfn_t gfn,
>                                        void *data, int offset, unsigned long len)
>     {
>         ...
>         pagefault_disable();
>         r = __copy_from_user_inatomic(data, (void __user *)addr + offset, len);
>         pagefault_enable();
>
> A grep for kvm_read_guest_atomic only finds __kvm_read_guest_atomic() and
> kvm_vcpu_read_guest_atomic().
>
> Could the changelog say kvm_vcpu_read_guest_atomic() instead?

Yes, that is correct. I've applied this fix in v2.

Re: [RFC PATCH 06/10] uaccess: Refuse and warn on user copies when current->mm is NULL

[bot+bpf-ci]

[-- Attachment #1: Type: text/plain, Size: 7160 bytes --]

> uaccess: Refuse and warn on user copies when current->mm is NULL

TL;DR:
- Possible spurious -EFAULT on valid kernel addresses on xtensa/nios2 from
  mm-less tasks (kworkers), e.g. d_path
- nofault helpers may hit the new WARN_ON_ONCE (BPF/tracing); fix is later in
  series, breaking bisection

> diff --git a/include/linux/uaccess.h b/include/linux/uaccess.h
> index eddbbb65ccc4..e54a6d804bf9 100644
> --- a/include/linux/uaccess.h
> +++ b/include/linux/uaccess.h
> @@ -50,6 +50,23 @@

[ ... ]

> +static __always_inline bool uaccess_should_fail(void)
> +{
> +	if (WARN_ON_ONCE(!current->mm))
> +		return true;
> +	return should_fail_usercopy();
> +}

[ ... ]

> @@ -114,7 +131,7 @@ __copy_from_user(void *to, const void __user *from, unsigned long n)
>  
>  	might_fault();
>  	instrument_copy_from_user_before(to, from, n);
> -	if (should_fail_usercopy())
> +	if (uaccess_should_fail())
>  		return n;
>  	check_object_size(to, n, false);
>  	res = raw_copy_from_user(to, from, n);

Can this refuse legitimate kernel address copies on xtensa and nios2?

On both architectures __copy_from_user() and __copy_to_user() also sit under
the generic kernel nofault accessors.  Neither architecture defines
__get_kernel_nofault or arch_get_kernel_nofault, so both use the fallback in
include/linux/uaccess.h, which goes through __get_user()/__put_user() on the
kernel address:

    #define __get_kernel_nofault(dst, src, type, label)    \
    ...
            if (__get_user(data, p))                       \
                    goto label;                            \

On xtensa, 8-byte __get_user()/__put_user() are implemented with
__copy_from_user()/__copy_to_user() (arch/xtensa/include/asm/uaccess.h):

    case 8: {
            u64 __x;
            if (unlikely(__copy_from_user(&__x, ptr, 8))) {
    ...
            retval = __copy_to_user(ptr, &__v64, 8) ? -EFAULT : 0;

On nios2, __put_user_common() sends any size other than 1, 2 or 4 to
__copy_to_user().

Neither architecture selects HAVE_EFFICIENT_UNALIGNED_ACCESS, so
copy_from_kernel_nofault() and copy_to_kernel_nofault() (mm/maccess.c) use
the u64 loop whenever dst and src are both 8-byte aligned and at least 8
bytes remain:

    if (!(align & 7))
            copy_from_kernel_nofault_loop(dst, src, size, u64, Efault);

With this patch, every such access from a task with current->mm == NULL
(kworkers, khungtaskd, kernel_init before exec, the idle task in IRQ, kgdb
entered from idle) hits WARN_ON_ONCE(!current->mm), and __copy_from_user()
returns n.  copy_from_kernel_nofault() then returns -EFAULT on valid kernel
memory.

Concrete generic callers include:

  d_path() -> prepend() -> prepend_copy() (fs/d_path.c) fills the component
  with 'x' and stops the walk.  This is reachable from LSM/IMA path hooks
  run by kworkers, for example async firmware loading.

  print_worker_info() (kernel/workqueue.c), called from
  dump_stack_print_info() and sched_show_task() for kworkers, which copies
  wq->name and worker->desc.

  copy_to_kernel_nofault() from kgdb_hex2mem()/kgdb_ebin2mem() on nios2,
  which selects HAVE_ARCH_KGDB.

Before this patch these accesses worked on kernel addresses, since
raw_copy_{from,to}_user() on these architectures is a plain fixup-protected
copy and access_ok() is not involved.  Only opt-in fault injection could
make them fail.

The commit message rationale ("Accessing user memory from a task that has no
mm is always a bug") does not apply to kernel addresses.  The commit message
also says get_user()/put_user() are not covered, which does not hold for
8-byte accesses on these two architectures.

No later commit in this series appears to touch these paths.

[ ... ]

> diff --git a/lib/strncpy_from_user.c b/lib/strncpy_from_user.c
> index 5bb752ff7c61..565b3125e752 100644
> --- a/lib/strncpy_from_user.c
> +++ b/lib/strncpy_from_user.c
> @@ -115,7 +115,7 @@ long strncpy_from_user(char *dst, const char __user *src, long count)
>  	unsigned long max_addr, src_addr;
>  
>  	might_fault();
> -	if (should_fail_usercopy())
> +	if (uaccess_should_fail())
>  		return -EFAULT;
>  	if (unlikely(count <= 0))
>  		return 0;

Can strncpy_from_user_nofault() now trip this WARN_ON_ONCE() from contexts
where current->mm is NULL by design?

In this tree strncpy_from_user_nofault() (mm/maccess.c) has no current->mm
check and no nmi_uaccess_okay() check.  It goes straight to
strncpy_from_user():

    pagefault_disable();
    ret = strncpy_from_user(dst, unsafe_addr, count);
    pagefault_enable();

and strncpy_from_user() now calls uaccess_should_fail(), which does:

    if (WARN_ON_ONCE(!current->mm))
            return true;

The callers of strncpy_from_user_nofault() are tracing helpers that run in
whatever context the probe fires in, including kernel threads and IRQs taken
while the idle task runs:

  bpf_probe_read_user_str() via bpf_probe_read_user_str_common()
  (kernel/trace/bpf_trace.c), and bpf_probe_read_str() for addresses below
  TASK_SIZE.

  bpf_trace_printk()/bpf_snprintf() through bpf_trace_copy_string()
  (kernel/bpf/helpers.c).  With CONFIG_ARCH_HAS_NON_OVERLAPPING_ADDRESS_SPACE
  (x86, arm64, ...), any '%s' argument below TASK_SIZE takes this path,
  including a NULL pointer.  So bpf_printk("%s", ptr) with ptr == NULL in a
  kprobe that fires in a kworker now triggers the warning.

  kprobe/fprobe/eprobe ustring fetch args (fetch_store_string_user() in
  kernel/trace/trace_probe_kernel.h).

  The .ustring event filters (test_ustring() in
  kernel/trace/trace_events_filter.c).

  Synthetic event string fields (trace_string() in
  kernel/trace/trace_events_synth.c).

  The bpf_copy_from_user_str_dynptr family (copy_user_str_nofault()).

Before this patch these calls either failed quietly with -EFAULT or read the
lazily loaded mm.  Now a tracing user (CAP_BPF plus CAP_PERFMON, or tracefs
access) can trigger a warning splat, and a panic with panic_on_warn, just by
tracing a function that kernel threads run.

The same applies to copy_to_user_iter_nofault() in lib/iov_iter.c, which this
patch also switches to uaccess_should_fail().  The commit message lists
copy_to_user_nofault() as intentionally covered, but it does not account for
the nofault helpers being used from mm-less contexts by design.

A fix appears later in the series, e8bf23b5b30d ("uaccess: Fail user nofault
helpers quietly when current->mm is NULL").  It adds an early !current->mm
return to copy_{from,to}_user_nofault(), strncpy_from_user_nofault() and
strnlen_user_nofault(), and switches copy_to_user_iter_nofault() back to
should_fail_usercopy().  Its message describes this exact problem
(bpf_probe_read_user_str() from a BPF program attached to a function called
by a kernel thread would now trip the WARN_ON_ONCE() in
strncpy_from_user()).

This commit on its own still introduces the regression, which breaks
bisection.  Could that fix be folded into this patch, or ordered before it?


---
AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md

CI run summary: https://github.com/kernel-patches/bpf/actions/runs/40410166379

Re: [RFC PATCH 04/10] stacktrace: Don't unwind the user stack without an mm

[James Houghton]

On Mon, Oct 26, 2026 at 3:30 PM <bot+bpf-ci@kernel.org> wrote:
>
> > stacktrace: Don't unwind the user stack without an mm
>
> TL;DR: Commit message only: the Fixes: tag may point at the wrong commit;
> 7e8e6816c649, which dropped the !current->mm check, looks like the real
> culprit rather than 214d8ca6ee85.
>
> > stacktrace: Don't unwind the user stack without an mm
> >
> > stack_trace_save_user() only refuses to unwind the user stack of tasks
> > with PF_KTHREAD set. That doesn't cover all tasks without an mm:
> >  - A user task that has already gone through exit_mm(), e.g. when an
> >    event fires from exit_files(), exit_task_work() or the final context
> >    switch in do_exit().
> >  - PID 1 before it execs init, and usermode helper children before they
> >    exec, which are created with user_mode_thread() and don't have
> >    PF_KTHREAD set. E.g. booting with trace_options=userstacktrace and
> >    an event enabled with trace_event= unwinds PID 1's "user stack"
> >    during initcalls.
> > With the ftrace userstacktrace option enabled, the architecture
> > unwinders then read the user stack without an mm. That access does not
> > reliably fail: the user half of the address space belongs to whatever
> > mm happens to be lazily loaded, if any, so the unwinder may read
> > another task's memory.
> > Use is_user_task() instead, which also checks current->mm, as perf
> > already does for the same reason since commit 76ed27608f7d ("perf:
> > sched: Fix perf crash with new is_user_task() helper"). This also stops
> > unwinding the user stack of PF_USER_WORKER threads, which never run in
> > user space, matching perf.
> > Fixes: 214d8ca6ee85 ("stacktrace: Provide common infrastructure")
> > Signed-off-by: James Houghton <jthoughton@google.com>
>
> Is the Fixes: tag pointing at the right commit?
>
> When 214d8ca6ee85 added the ARCH_STACKWALK version of
> stack_trace_save_user(), it guarded the unwind with:
>
>         /* Trace user stack if not a kernel thread */
>         if (!current->mm)
>                 return 0;
>
> That check already covers both cases described above (a task past exit_mm(),
> and PID 1 or usermode helpers before exec, which have mm == NULL).
>
> The check was replaced by commit 7e8e6816c649 ("stacktrace: Use PF_KTHREAD to
> check for kernel threads"), which switched it to:
>
>         if (current->flags & PF_KTHREAD)
>                 return 0;
>
> That dropped the no-mm protection, which looks like what this patch restores
> via is_user_task().
>
> Should this be:
>
>   Fixes: 7e8e6816c649 ("stacktrace: Use PF_KTHREAD to check for kernel threads")

Thank you, yes that appears to be correct. Updated in v2.

Re: [RFC PATCH 10/10] uaccess: Move the NULL mm warning out of line

[Arnd Bergmann]

On Mon, Oct 26, 2026, at 22:52, James Houghton wrote:
> uaccess_without_mm() is inlined into every user copy, including every
> copy_{to,from}_user() call site on architectures that define
> INLINE_COPY_USER, such as arm64. Each inlined WARN_ON_ONCE() adds its
> own __bug_table entry.
>
> Keep the NULL current->mm check inline, but move the warning into a
> single out-of-line, __cold function, uaccess_warn_without_mm().
>
> The trade-off is that only the first offending access in the system
> warns, rather than the first one from each call site. The stack trace
> still identifies the caller, and the access is refused either way.
>
> Compared to the base of this series, with defconfig + KVM:
>
>                      inline WARN_ON_ONCE()    out-of-line warning
>   x86_64 total       +5,884 B (+0.014%)       +9,714 B (+0.023%)
>   arm64  total      +40,948 B (+0.10%)       +24,609 B (+0.06%)
>
> On arm64, this removes the ~16 KB of __bug_table growth, while text
> size is unchanged, as a call is the same size as the inline brk. On
> x86_64, the call is larger than the inline ud2 and causes more .cold
> splitting, so this is a slight regression.
>
> Signed-off-by: James Houghton <jthoughton@google.com>
> ---
> NOTE: This patch is included for reference only, to show the cost of the
>       inline warning.

I would suspect that the growth is larger on architectures that
don't implement __WARN_FLAGS() but rely on the generic WARN_ON().

This would mainly affect 32-bit arm and mips machines, some of which
are very memory limited, most of the 64-bit architectures have it.

      Arnd

Re: [RFC PATCH 06/10] uaccess: Refuse and warn on user copies when current->mm is NULL

[James Houghton]

On Mon, Oct 26, 2026 at 3:42 PM <bot+bpf-ci@kernel.org> wrote:
>
> > uaccess: Refuse and warn on user copies when current->mm is NULL
>
> TL;DR:
> - Possible spurious -EFAULT on valid kernel addresses on xtensa/nios2 from
>   mm-less tasks (kworkers), e.g. d_path
> - nofault helpers may hit the new WARN_ON_ONCE (BPF/tracing); fix is later in
>   series, breaking bisection
>
> > diff --git a/include/linux/uaccess.h b/include/linux/uaccess.h
> > index eddbbb65ccc4..e54a6d804bf9 100644
> > --- a/include/linux/uaccess.h
> > +++ b/include/linux/uaccess.h
> > @@ -50,6 +50,23 @@
>
> [ ... ]
>
> > +static __always_inline bool uaccess_should_fail(void)
> > +{
> > +     if (WARN_ON_ONCE(!current->mm))
> > +             return true;
> > +     return should_fail_usercopy();
> > +}
>
> [ ... ]
>
> > @@ -114,7 +131,7 @@ __copy_from_user(void *to, const void __user *from, unsigned long n)
> >
> >       might_fault();
> >       instrument_copy_from_user_before(to, from, n);
> > -     if (should_fail_usercopy())
> > +     if (uaccess_should_fail())
> >               return n;
> >       check_object_size(to, n, false);
> >       res = raw_copy_from_user(to, from, n);
>
> Can this refuse legitimate kernel address copies on xtensa and nios2?
>
> On both architectures __copy_from_user() and __copy_to_user() also sit under
> the generic kernel nofault accessors.  Neither architecture defines
> __get_kernel_nofault or arch_get_kernel_nofault, so both use the fallback in
> include/linux/uaccess.h, which goes through __get_user()/__put_user() on the
> kernel address:
>
>     #define __get_kernel_nofault(dst, src, type, label)    \
>     ...
>             if (__get_user(data, p))                       \
>                     goto label;                            \
>
> On xtensa, 8-byte __get_user()/__put_user() are implemented with
> __copy_from_user()/__copy_to_user() (arch/xtensa/include/asm/uaccess.h):
>
>     case 8: {
>             u64 __x;
>             if (unlikely(__copy_from_user(&__x, ptr, 8))) {
>     ...
>             retval = __copy_to_user(ptr, &__v64, 8) ? -EFAULT : 0;
>
> On nios2, __put_user_common() sends any size other than 1, 2 or 4 to
> __copy_to_user().
>
> Neither architecture selects HAVE_EFFICIENT_UNALIGNED_ACCESS, so
> copy_from_kernel_nofault() and copy_to_kernel_nofault() (mm/maccess.c) use
> the u64 loop whenever dst and src are both 8-byte aligned and at least 8
> bytes remain:
>
>     if (!(align & 7))
>             copy_from_kernel_nofault_loop(dst, src, size, u64, Efault);
>
> With this patch, every such access from a task with current->mm == NULL
> (kworkers, khungtaskd, kernel_init before exec, the idle task in IRQ, kgdb
> entered from idle) hits WARN_ON_ONCE(!current->mm), and __copy_from_user()
> returns n.  copy_from_kernel_nofault() then returns -EFAULT on valid kernel
> memory.
>
> Concrete generic callers include:
>
>   d_path() -> prepend() -> prepend_copy() (fs/d_path.c) fills the component
>   with 'x' and stops the walk.  This is reachable from LSM/IMA path hooks
>   run by kworkers, for example async firmware loading.
>
>   print_worker_info() (kernel/workqueue.c), called from
>   dump_stack_print_info() and sched_show_task() for kworkers, which copies
>   wq->name and worker->desc.
>
>   copy_to_kernel_nofault() from kgdb_hex2mem()/kgdb_ebin2mem() on nios2,
>   which selects HAVE_ARCH_KGDB.
>
> Before this patch these accesses worked on kernel addresses, since
> raw_copy_{from,to}_user() on these architectures is a plain fixup-protected
> copy and access_ok() is not involved.  Only opt-in fault injection could
> make them fail.
>
> The commit message rationale ("Accessing user memory from a task that has no
> mm is always a bug") does not apply to kernel addresses.  The commit message
> also says get_user()/put_user() are not covered, which does not hold for
> 8-byte accesses on these two architectures.
>
> No later commit in this series appears to touch these paths.

This analysis is correct. Implementing copy-from-kernel functions with calls to
uaccess functions is a little hard to make sense of. :/

I've included a patch at the front of my v2 for xtensa and nios2. I don't think
any other architecture has this problem. I've placed the patch at the bottom of
this reply. I've also adjusted the wording for patch 6.

>
> [ ... ]
>
> > diff --git a/lib/strncpy_from_user.c b/lib/strncpy_from_user.c
> > index 5bb752ff7c61..565b3125e752 100644
> > --- a/lib/strncpy_from_user.c
> > +++ b/lib/strncpy_from_user.c
> > @@ -115,7 +115,7 @@ long strncpy_from_user(char *dst, const char __user *src, long count)
> >       unsigned long max_addr, src_addr;
> >
> >       might_fault();
> > -     if (should_fail_usercopy())
> > +     if (uaccess_should_fail())
> >               return -EFAULT;
> >       if (unlikely(count <= 0))
> >               return 0;
>
> Can strncpy_from_user_nofault() now trip this WARN_ON_ONCE() from contexts
> where current->mm is NULL by design?
>
> In this tree strncpy_from_user_nofault() (mm/maccess.c) has no current->mm
> check and no nmi_uaccess_okay() check.  It goes straight to
> strncpy_from_user():
>
>     pagefault_disable();
>     ret = strncpy_from_user(dst, unsafe_addr, count);
>     pagefault_enable();
>
> and strncpy_from_user() now calls uaccess_should_fail(), which does:
>
>     if (WARN_ON_ONCE(!current->mm))
>             return true;
>
> The callers of strncpy_from_user_nofault() are tracing helpers that run in
> whatever context the probe fires in, including kernel threads and IRQs taken
> while the idle task runs:
>
>   bpf_probe_read_user_str() via bpf_probe_read_user_str_common()
>   (kernel/trace/bpf_trace.c), and bpf_probe_read_str() for addresses below
>   TASK_SIZE.
>
>   bpf_trace_printk()/bpf_snprintf() through bpf_trace_copy_string()
>   (kernel/bpf/helpers.c).  With CONFIG_ARCH_HAS_NON_OVERLAPPING_ADDRESS_SPACE
>   (x86, arm64, ...), any '%s' argument below TASK_SIZE takes this path,
>   including a NULL pointer.  So bpf_printk("%s", ptr) with ptr == NULL in a
>   kprobe that fires in a kworker now triggers the warning.
>
>   kprobe/fprobe/eprobe ustring fetch args (fetch_store_string_user() in
>   kernel/trace/trace_probe_kernel.h).
>
>   The .ustring event filters (test_ustring() in
>   kernel/trace/trace_events_filter.c).
>
>   Synthetic event string fields (trace_string() in
>   kernel/trace/trace_events_synth.c).
>
>   The bpf_copy_from_user_str_dynptr family (copy_user_str_nofault()).
>
> Before this patch these calls either failed quietly with -EFAULT or read the
> lazily loaded mm.  Now a tracing user (CAP_BPF plus CAP_PERFMON, or tracefs
> access) can trigger a warning splat, and a panic with panic_on_warn, just by
> tracing a function that kernel threads run.
>
> The same applies to copy_to_user_iter_nofault() in lib/iov_iter.c, which this
> patch also switches to uaccess_should_fail().  The commit message lists
> copy_to_user_nofault() as intentionally covered, but it does not account for
> the nofault helpers being used from mm-less contexts by design.
>
> A fix appears later in the series, e8bf23b5b30d ("uaccess: Fail user nofault
> helpers quietly when current->mm is NULL").  It adds an early !current->mm
> return to copy_{from,to}_user_nofault(), strncpy_from_user_nofault() and
> strnlen_user_nofault(), and switches copy_to_user_iter_nofault() back to
> should_fail_usercopy().  Its message describes this exact problem
> (bpf_probe_read_user_str() from a BPF program attached to a function called
> by a kernel thread would now trip the WARN_ON_ONCE() in
> strncpy_from_user()).
>
> This commit on its own still introduces the regression, which breaks
> bisection.  Could that fix be folded into this patch, or ordered before it?

Ah, yes. I'll flip the order of patch 6 and 7 in the v2. Thanks.

Okay, the xtensa/nios2 patch is here:

-------

    xtensa, nios2: Don't use __copy_{from,to}_user() in __{get,put}_user()
    
    Architectures that don't define __get_kernel_nofault() or
    arch_get_kernel_nofault() use the generic fallback in
    include/linux/uaccess.h, which accesses *kernel* addresses with
    __get_user() and __put_user(). That relies on __get_user() and
    __put_user() being raw accessors, i.e. plain fixup-protected loads and
    stores with no extra checks that only make sense for user memory.
    
    xtensa and nios2 use this fallback, but don't quite meet that: on
    xtensa, 8-byte __get_user() and __put_user() are implemented with
    __copy_from_user() and __copy_to_user(), and on nios2, __put_user() of
    any size other than 1, 2 or 4 bytes (i.e. 8-byte puts) uses
    __copy_to_user(). __copy_{from,to}_user() do more than a plain copy: they
    inject failures with CONFIG_FAULT_INJECTION_USERCOPY, check the kernel
    object size with CONFIG_HARDENED_USERCOPY, and, with a later patch in
    this series, refuse and warn on user accesses when current->mm is NULL.
    
    As neither architecture selects HAVE_EFFICIENT_UNALIGNED_ACCESS,
    copy_from_kernel_nofault() and copy_to_kernel_nofault() use 8-byte
    accesses when both pointers are 8-byte aligned, so kernel nofault
    accesses (used e.g. by d_path() and print_worker_info()) end up in
    __copy_{from,to}_user() on kernel addresses. With the NULL mm check,
    copy_from_kernel_nofault() would then fail on valid kernel memory when
    called from kernel threads.
    
    Use raw_copy_{from,to}_user() instead, as asm-generic's
    __get_user_fn()/__put_user_fn() do, and as nios2 already does for 8-byte
    gets (__get_user_8()). This only drops the extra checks and KASAN/KCSAN
    instrumentation for 8-byte accesses, making them consistent with the 1,
    2 and 4-byte cases, which are plain inline asm.
    
    The fallback itself is fine and is still needed: it's used by alpha,
    arc, csky, hexagon, microblaze, nios2, openrisc, sh, sparc and xtensa.
    Of those, only xtensa and nios2 have __get_user()/__put_user() that
    aren't raw. The others use either asm-generic's, which call
    raw_copy_{from,to}_user(), or inline asm for all sizes. The remaining
    architectures (arm, arm64, loongarch, m68k, mips, parisc, powerpc,
    riscv, s390, um and x86) define their own __get_kernel_nofault() or
    arch_get_kernel_nofault(), which don't use __get_user()/__put_user() or
    __copy_{from,to}_user().

    Signed-off-by: James Houghton <jthoughton@google.com>
    
diff --git a/arch/nios2/include/asm/uaccess.h b/arch/nios2/include/asm/uaccess.h
index 5e6e05cc6efc..f20f11bd21dc 100644
--- a/arch/nios2/include/asm/uaccess.h
+++ b/arch/nios2/include/asm/uaccess.h
@@ -161,7 +161,7 @@ do {									\
 	default:							\
 		/* XXX: This looks wrong... */				\
 		__pu_err = 0;						\
-		if (__copy_to_user(__pu_ptr, &(__pu_val),		\
+		if (raw_copy_to_user(__pu_ptr, &(__pu_val),		\
 			sizeof(*__pu_ptr)))				\
 			__pu_err = -EFAULT;				\
 		break;							\
diff --git a/arch/xtensa/include/asm/uaccess.h b/arch/xtensa/include/asm/uaccess.h
index 6538a29a2bbd..30c8805d1c83 100644
--- a/arch/xtensa/include/asm/uaccess.h
+++ b/arch/xtensa/include/asm/uaccess.h
@@ -76,7 +76,7 @@ do {									\
 	case 4: __put_user_asm(x, ptr, retval, 4, "s32i", __cb); break;	\
 	case 8: {							\
 		     __typeof__(*ptr) __v64 = x;			\
-		     retval = __copy_to_user(ptr, &__v64, 8) ? -EFAULT : 0;	\
+		     retval = raw_copy_to_user(ptr, &__v64, 8) ? -EFAULT : 0;	\
 		     break;						\
 	        }							\
 	default: __put_user_bad();					\
@@ -176,7 +176,7 @@ do {									\
 	case 4: __get_user_asm(x, ptr, retval, 4, "l32i", __cb);  break;\
 	case 8: {							\
 		u64 __x;						\
-		if (unlikely(__copy_from_user(&__x, ptr, 8))) {		\
+		if (unlikely(raw_copy_from_user(&__x, ptr, 8))) {	\
 			retval = -EFAULT;				\
 			(x) = (__typeof__(*(ptr)))0;			\
 		} else {						\

Re: [RFC PATCH 10/10] uaccess: Move the NULL mm warning out of line

[James Houghton]

On Mon, Oct 26, 2026 at 4:03 PM Arnd Bergmann <arnd@arndb.de> wrote:
>
> On Mon, Oct 26, 2026, at 22:52, James Houghton wrote:
> > uaccess_without_mm() is inlined into every user copy, including every
> > copy_{to,from}_user() call site on architectures that define
> > INLINE_COPY_USER, such as arm64. Each inlined WARN_ON_ONCE() adds its
> > own __bug_table entry.
> >
> > Keep the NULL current->mm check inline, but move the warning into a
> > single out-of-line, __cold function, uaccess_warn_without_mm().
> >
> > The trade-off is that only the first offending access in the system
> > warns, rather than the first one from each call site. The stack trace
> > still identifies the caller, and the access is refused either way.
> >
> > Compared to the base of this series, with defconfig + KVM:
> >
> >                      inline WARN_ON_ONCE()    out-of-line warning
> >   x86_64 total       +5,884 B (+0.014%)       +9,714 B (+0.023%)
> >   arm64  total      +40,948 B (+0.10%)       +24,609 B (+0.06%)
> >
> > On arm64, this removes the ~16 KB of __bug_table growth, while text
> > size is unchanged, as a call is the same size as the inline brk. On
> > x86_64, the call is larger than the inline ud2 and causes more .cold
> > splitting, so this is a slight regression.
> >
> > Signed-off-by: James Houghton <jthoughton@google.com>
> > ---
> > NOTE: This patch is included for reference only, to show the cost of the
> >       inline warning.
>
> I would suspect that the growth is larger on architectures that
> don't implement __WARN_FLAGS() but rely on the generic WARN_ON().
>
> This would mainly affect 32-bit arm and mips machines, some of which
> are very memory limited, most of the 64-bit architectures have it.

Thanks, Arnd! I ran some build tests on 32-bit arm and mips:

arm (multi_v7_defconfig):
 - base: 31,913,726 B
 - patches 1-9: +64,816 B (+0.20%)
 - patches 1-10: +23,193 B (+0.07%)

mips (malta_defconfig):
 - base: 11,830,177 B
 - patches 1-9: +61,168 B (+0.52%)
 - patches 1-10: +25,881 B (+0.22%)

I think it makes sense to include patch 10 with these results; cutting
>50% of the size increase for arm and mips seems worth the trade-off
of only WARNing on the violating callstack. Let me know what you
think.

Is the increase even with patch 10 too much for mips or arm? I can add
a Kconfig in that case.

Re: [RFC PATCH 00/10] uaccess: Refuse and warn on user accesses when current->mm is NULL

[Arnd Bergmann]

On Mon, Oct 26, 2026, at 22:52, James Houghton wrote:
>
> This series comes out of a set of bugs that were found in KVM where the
> kernel can sometimes write to a random process's user memory[1]. The KVM
> fixes at [1] are still required, even with this uaccess series, as noted
> by Sean here[2] and below in this cover letter. The beginning of this
> series includes fixes for similar bugs found through static analysis
> with Claude Opus 5.5. I don't think any of these bugs are more severe
> than the KVM bugs[1].
>
> We hit one of the KVM bugs in production, and it was challenging to
> debug. This series makes such bugs, where the kernel is doing uaccess
> without an mm, much more obvious.

Nice work!

> Such an access does not reliably fail. The user portion of the address
> space belongs to whatever mm happens to be lazily loaded, if any. If
> the access faults, the fault handler refuses to handle it and the copy
> fails, but if the page is already mapped, the access silently reads or
> writes another task's memory.

This sounds like the problem depends a lot on the architecture,
and to what degree they implement lazy mm switching. If an architecture
makes all user pointers invalid for a kernel thread, this does not
actually happen, right?

E.g. on arm64 with CONFIG_ARM64_SW_TTBR0_PAN, the page table base for
the current process is only set in user_access_begin() and would
not point to any task here. Without ttbr0_pan, it should be
covered by cpu_set_reserved_ttbr0() turning off all user space access
for kernel threads.

Should this perhaps be addressed in architecture specific code
instead of in a common place?

> == The New WARN ==
>
> The rest of the series makes the generic uaccess helpers WARN_ON_ONCE()
> and refuse the access when current->mm is NULL:
>
>  - Patch 6 adds uaccess_should_fail(), which combines the existing
>    should_fail_usercopy() fault injection check with the NULL mm check,
>    and uses it in place of should_fail_usercopy(). This covers
>    copy_{to,from}_user() on all architectures (including the masked
>    user access paths, which never call access_ok()), the
>    __copy_{to,from}_user() variants used by KVM's guest memory
>    accessors, the user-backed iov_iter copy routines and
>    strncpy_from_user().
...
> The checks are deliberately limited to generic code. Accesses
> implemented entirely by architectures (get_user(), put_user(),
> clear_user(), and direct users of user_access_begin() or
> masked_user_access_begin()) are not covered. Please let me know if the
> new check should be placed elsewhere.

I would think that user_access_begin() is a more natural place
for the added checks, as this should be called exactly once for
any function that accesses user space and also stays out of the
way of the copy_from_kernel_nofault() etc.

       Arnd

Re: [RFC PATCH 01/10] scsi: bsg: Don't copy sense data to user memory when canceled

[Jens Axboe]

Reviewed-by: Jens Axboe <axboe@kernel.dk>

-- 
Jens Axboe

Re: [RFC PATCH 02/10] btrfs: Don't copy encoded read data to user memory when canceled

[Jens Axboe]

Reviewed-by: Jens Axboe <axboe@kernel.dk>

-- 
Jens Axboe

Re: [RFC PATCH 00/10] uaccess: Refuse and warn on user accesses when current->mm is NULL

[Arnd Bergmann]

On Tue, Oct 27, 2026, at 01:12, Arnd Bergmann wrote:
>
> E.g. on arm64 with CONFIG_ARM64_SW_TTBR0_PAN, the page table base for
> the current process is only set in user_access_begin() and would
> not point to any task here. Without ttbr0_pan, it should be
> covered by cpu_set_reserved_ttbr0() turning off all user space access
> for kernel threads.

correction: for non-sw_ttbr0_pan, enter_lazy_tlb() does nothing
on arm64, so it is the same as x86 and the others.

     Arnd

Re: [RFC PATCH 05/10] perf/core: Don't unwind the user stack without an mm

[Peter Zijlstra]

On Mon, Oct 26, 2026 at 09:52:21PM +0000, James Houghton wrote:
> get_perf_callchain() only checks is_user_task(current) when the regs it
> is given aren't user mode regs. However, BPF's bpf_get_task_stack() (and
> bpf_get_task_stack_sleepable()) pass task_pt_regs(current) when called on
> the current task, which are user mode regs for a user task, even if that
> task has already gone through exit_mm(), e.g. when called from a program
> attached to exit_files() or to the dying task's final sched_switch.
> 
> perf_callchain_user() then unwinds the user stack without an mm. That
> access does not reliably fail: the user half of the address space
> belongs to whatever mm happens to be lazily loaded, if any, so the
> unwinder may read another task's memory. This affects e.g. arm64,
> riscv, loongarch and csky, whose perf_callchain_user() doesn't check
> current->mm itself. perf events themselves are not affected, as
> perf_callchain() already clears the user callchain request when
> current is not a user task with an mm.
> 
> Check is_user_task(current) regardless of the regs' mode.
> 
> Fixes: fa28dcb82a38 ("bpf: Introduce helper bpf_get_task_stack()")
> Signed-off-by: James Houghton <jthoughton@google.com>
> ---
>  kernel/events/callchain.c | 12 ++++++++----
>  1 file changed, 8 insertions(+), 4 deletions(-)
> 
> diff --git a/kernel/events/callchain.c b/kernel/events/callchain.c
> index 9d24b6e0c91f..a451ca2a5573 100644
> --- a/kernel/events/callchain.c
> +++ b/kernel/events/callchain.c
> @@ -245,11 +245,15 @@ get_perf_callchain(struct pt_regs *regs, bool kernel, bool user,
>  	}
>  
>  	if (user && !crosstask) {
> -		if (!user_mode(regs)) {
> -			if (!is_user_task(current))
> -				goto exit_put;
> +		/*
> +		 * Even if @regs are user mode regs, e.g. task_pt_regs(current)
> +		 * passed in by bpf_get_task_stack(), current may already have
> +		 * gone through exit_mm().
> +		 */
> +		if (!is_user_task(current))
> +			goto exit_put;
> +		if (!user_mode(regs))
>  			regs = task_pt_regs(current);
> -		}
>  
>  		if (defer_cookie) {
>  			/*

We have perf_event_exit_task() before exit_mm().

Still, it doesn't hurt to be cautious. So no real objection, but perhaps
adjust the Changelog?