kernel_samsung_a53x/kernel
Johannes Weiner 90c0c9aa4a cgroup: rstat: punt root-level optimization to individual controllers
Current users of the rstat code can source root-level statistics from
the native counters of their respective subsystem, allowing them to
forego aggregation at the root level.  This optimization is currently
implemented inside the generic rstat code, which doesn't track the root
cgroup and doesn't invoke the subsystem flush callbacks on it.

However, the memory controller cannot do this optimization, because
cgroup1 breaks out memory specifically for the local level, including at
the root level.  In preparation for the memory controller switching to
rstat, move the optimization from rstat core to the controllers.

Afterwards, rstat will always track the root cgroup for changes and
invoke the subsystem callbacks on it; and it's up to the subsystem to
special-case and skip aggregation of the root cgroup if it can source
this information through other, cheaper means.

This is the case for the io controller and the cgroup base stats.  In
their respective flush callbacks, check whether the parent is the root
cgroup, and if so, skip the unnecessary upward propagation.

The extra cost of tracking the root cgroup is negligible: on stat
changes, we actually remove a branch that checks for the root.  The
queueing for a flush touches only per-cpu data, and only the first stat
change since a flush requires a (per-cpu) lock.

Link: https://lkml.kernel.org/r/20210209163304.77088-6-hannes@cmpxchg.org
Signed-off-by: Johannes Weiner <hannes@cmpxchg.org>
Acked-by: Tejun Heo <tj@kernel.org>
Cc: Michal Hocko <mhocko@suse.com>
Cc: Michal Koutný <mkoutny@suse.com>
Cc: Roman Gushchin <guro@fb.com>
Cc: Shakeel Butt <shakeelb@google.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
(cherry picked from commit dc26532aed0ab25c0801a34640d1f3b9b9098a48)
(cherry picked from commit 69da183fcd0112af130879a1c93113a941e2241b)
(cherry picked from commit ddf1013871482b246147e71a04c865c1be5cf74d)
(cherry picked from commit 30fcd52e18dd1d508b1b22f7c660ac22de734f67)
(cherry picked from commit 19c9a1b9d9ae9a4f359deaf89101f9013254f43d)
(cherry picked from commit 0b4286aea9bb0a6ea6acb723f8396e476044190b)
2024-11-19 17:40:21 +01:00
..
bpf bpf: Fix overrunning reservations in ringbuf 2024-11-19 14:19:51 +01:00
cgroup cgroup: rstat: punt root-level optimization to individual controllers 2024-11-19 17:40:21 +01:00
configs Import A536BXXU9EXDC 2024-06-15 16:02:09 -03:00
debug kdb: Use format-specifiers rather than memset() for padding in kdb_read() 2024-11-19 12:27:19 +01:00
dma dma-mapping: clear dev->dma_mem to NULL after freeing it 2024-11-18 12:12:37 +01:00
entry Import A536BXXU9EXDC 2024-06-15 16:02:09 -03:00
events perf/core: Fix missing wakeup when waiting for context reference 2024-11-19 14:19:30 +01:00
futex futex: Don't include process MM in futex key on no-MMU 2024-11-18 11:42:47 +01:00
gcov gcov: add support for GCC 14 2024-11-19 14:19:10 +01:00
irq genirq/cpuhotplug, x86/vector: Prevent vector leak during CPU offline 2024-11-19 12:27:11 +01:00
kcsan Import A536BXXU9EXDC 2024-06-15 16:02:09 -03:00
livepatch kallsyms: refactor {,module_}kallsyms_on_each_symbol 2024-11-19 12:27:32 +01:00
locking lockdep: Fix block chain corruption 2024-11-18 12:10:56 +01:00
power PM: suspend: Set mem_sleep_current during kernel command line setup 2024-11-19 09:22:17 +01:00
printk printk: Update @console_may_schedule in console_trylock_spinning() 2024-11-19 09:22:40 +01:00
rcu rcu: Avoid unnecessary softirq when system is idle 2024-11-19 17:34:20 +01:00
sched sched/fair: Always update CPU capacity when load balancing 2024-11-19 17:34:49 +01:00
time tick/nohz_full: Don't abuse smp_call_function_single() in tick_setup_device() 2024-11-19 14:19:05 +01:00
trace tracing: Add MODULE_DESCRIPTION() to preemptirq_delay_test 2024-11-19 14:19:29 +01:00
acct.c Import A536BXXU9EXDC 2024-06-15 16:02:09 -03:00
async.c async: Introduce async_schedule_dev_nocall() 2024-11-18 12:12:56 +01:00
audit.c audit: Send netlink ACK before setting connection in auditd_set 2024-11-18 12:13:09 +01:00
audit.h Import A536BXXU9EXDC 2024-06-15 16:02:09 -03:00
audit_fsnotify.c fsnotify: make allow_dups a property of the group 2024-11-19 12:27:55 +01:00
audit_tree.c fsnotify: pass flags argument to fsnotify_alloc_group() 2024-11-19 12:27:55 +01:00
audit_watch.c fsnotify: pass flags argument to fsnotify_alloc_group() 2024-11-19 12:27:55 +01:00
auditfilter.c ima: Avoid blocking in RCU read-side critical section 2024-11-19 14:19:42 +01:00
auditsc.c Import A536BXXU9EXDC 2024-06-15 16:02:09 -03:00
backtracetest.c Import A536BXXU9EXDC 2024-06-15 16:02:09 -03:00
bounds.c bounds: Use the right number of bits for power-of-two CONFIG_NR_CPUS 2024-11-19 11:32:40 +01:00
capability.c Import A536BXXU9EXDC 2024-06-15 16:02:09 -03:00
cfi.c Import A536BXXU9EXDC 2024-06-15 16:02:09 -03:00
compat.c Import A536BXXU9EXDC 2024-06-15 16:02:09 -03:00
configs.c Import A536BXXU9EXDC 2024-06-15 16:02:09 -03:00
context_tracking.c Import A536BXXU9EXDC 2024-06-15 16:02:09 -03:00
cpu.c cpu: Re-enable CPU mitigations by default for !X86 architectures 2024-11-19 11:32:38 +01:00
cpu_pm.c Import A536BXXU9EXDC 2024-06-15 16:02:09 -03:00
crash_core.c Import A536BXXU9EXDC 2024-06-15 16:02:09 -03:00
crash_dump.c Import A536BXXU9EXDC 2024-06-15 16:02:09 -03:00
cred.c Import A536BXXU9EXDC 2024-06-15 16:02:09 -03:00
delayacct.c Import A536BXXU9EXDC 2024-06-15 16:02:09 -03:00
dma.c Import A536BXXU9EXDC 2024-06-15 16:02:09 -03:00
exec_domain.c Import A536BXXU9EXDC 2024-06-15 16:02:09 -03:00
exit.c mm: optimize the redundant loop of mm_update_owner_next() 2024-11-19 14:19:41 +01:00
extable.c Import A536BXXU9EXDC 2024-06-15 16:02:09 -03:00
fail_function.c Import A536BXXU9EXDC 2024-06-15 16:02:09 -03:00
fork.c exec: Simplify unshare_files 2024-11-19 12:27:27 +01:00
freezer.c Import A536BXXU9EXDC 2024-06-15 16:02:09 -03:00
gen_kheaders.sh kheaders: explicitly define file modes for archived headers 2024-11-19 14:19:30 +01:00
groups.c Import A536BXXU9EXDC 2024-06-15 16:02:09 -03:00
hung_task.c Import A536BXXU9EXDC 2024-06-15 16:02:09 -03:00
iomem.c Import A536BXXU9EXDC 2024-06-15 16:02:09 -03:00
irq_work.c Import A536BXXU9EXDC 2024-06-15 16:02:09 -03:00
jump_label.c Import A536BXXU9EXDC 2024-06-15 16:02:09 -03:00
kallsyms.c kallsyms: only build {,module_}kallsyms_on_each_symbol when required 2024-11-19 12:27:33 +01:00
kcmp.c kcmp: In get_file_raw_ptr use task_lookup_fd_rcu 2024-11-19 12:27:27 +01:00
Kconfig.freezer Import A536BXXU9EXDC 2024-06-15 16:02:09 -03:00
Kconfig.hz Import A536BXXU9EXDC 2024-06-15 16:02:09 -03:00
Kconfig.locks Import A536BXXU9EXDC 2024-06-15 16:02:09 -03:00
Kconfig.preempt Import A536BXXU9EXDC 2024-06-15 16:02:09 -03:00
kcov.c kcov: don't lose track of remote references during softirqs 2024-11-19 14:19:10 +01:00
kexec.c Import A536BXXU9EXDC 2024-06-15 16:02:09 -03:00
kexec_core.c Import A536BXXU9EXDC 2024-06-15 16:02:09 -03:00
kexec_elf.c Import A536BXXU9EXDC 2024-06-15 16:02:09 -03:00
kexec_file.c Import A536BXXU9EXDC 2024-06-15 16:02:09 -03:00
kexec_internal.h Import A536BXXU9EXDC 2024-06-15 16:02:09 -03:00
kheaders.c Import A536BXXU9EXDC 2024-06-15 16:02:09 -03:00
kmod.c Import A536BXXU9EXDC 2024-06-15 16:02:09 -03:00
kprobes.c kprobes: Fix possible use-after-free issue on kprobe registration 2024-11-19 11:32:21 +01:00
ksysfs.c Import A536BXXU9EXDC 2024-06-15 16:02:09 -03:00
kthread.c exit: Implement kthread_exit 2024-11-19 12:27:49 +01:00
latencytop.c Import A536BXXU9EXDC 2024-06-15 16:02:09 -03:00
Makefile kernel: Use the stock defconfig for /proc/config.gz 2024-06-15 16:20:14 -03:00
module-internal.h Import A536BXXU9EXDC 2024-06-15 16:02:09 -03:00
module.c NFSD: Remove svc_serv_ops::svo_module 2024-11-19 12:27:54 +01:00
module_signature.c Import A536BXXU9EXDC 2024-06-15 16:02:09 -03:00
module_signing.c Import A536BXXU9EXDC 2024-06-15 16:02:09 -03:00
notifier.c Import A536BXXU9EXDC 2024-06-15 16:02:09 -03:00
nsproxy.c Import A536BXXU9EXDC 2024-06-15 16:02:09 -03:00
padata.c padata: Disable BH when taking works lock on MT path 2024-11-19 14:19:06 +01:00
panic.c panic: Flush kernel log buffer at the end 2024-11-19 09:23:11 +01:00
params.c params: lift param_set_uint_minmax to common code 2024-11-19 12:27:09 +01:00
pid.c kernel/pid.c: implement additional checks upon pidfd_create() parameters 2024-11-19 12:27:43 +01:00
pid_namespace.c zap_pid_ns_processes: clear TIF_NOTIFY_SIGNAL along with TIF_SIGPENDING 2024-11-19 14:19:05 +01:00
profile.c Import A536BXXU9EXDC 2024-06-15 16:02:09 -03:00
ptrace.c Import A536BXXU9EXDC 2024-06-15 16:02:09 -03:00
range.c Import A536BXXU9EXDC 2024-06-15 16:02:09 -03:00
reboot.c kernel/reboot: emergency_restart: Set correct system_state 2024-11-18 11:43:25 +01:00
regset.c Import A536BXXU9EXDC 2024-06-15 16:02:09 -03:00
relay.c Import A536BXXU9EXDC 2024-06-15 16:02:09 -03:00
resource.c Import A536BXXU9EXDC 2024-06-15 16:02:09 -03:00
rseq.c Import A536BXXU9EXDC 2024-06-15 16:02:09 -03:00
scftorture.c Import A536BXXU9EXDC 2024-06-15 16:02:09 -03:00
scs.c Import A536BXXU9EXDC 2024-06-15 16:02:09 -03:00
seccomp.c seccomp: Invalidate seccomp mode to catch death failures 2024-11-18 22:25:35 +01:00
signal.c Import A536BXXU9EXDC 2024-06-15 16:02:09 -03:00
smp.c Import A536BXXU9EXDC 2024-06-15 16:02:09 -03:00
smpboot.c Import A536BXXU9EXDC 2024-06-15 16:02:09 -03:00
smpboot.h Import A536BXXU9EXDC 2024-06-15 16:02:09 -03:00
softirq.c Import A536BXXU9EXDC 2024-06-15 16:02:09 -03:00
stackleak.c Import A536BXXU9EXDC 2024-06-15 16:02:09 -03:00
stacktrace.c Import A536BXXU9EXDC 2024-06-15 16:02:09 -03:00
static_call.c Import A536BXXU9EXDC 2024-06-15 16:02:09 -03:00
stop_machine.c Import A536BXXU9EXDC 2024-06-15 16:02:09 -03:00
sys.c Revert "fs: add file and path permissions helpers" 2024-11-19 13:30:21 +01:00
sys_ni.c syscalls: fix compat_sys_io_pgetevents_time64 usage 2024-11-19 14:19:34 +01:00
sysctl-test.c Import A536BXXU9EXDC 2024-06-15 16:02:09 -03:00
sysctl.c kernel: sysctl: add init protection to common mm-related nodes 2024-11-19 17:39:17 +01:00
task_work.c Import A536BXXU9EXDC 2024-06-15 16:02:09 -03:00
taskstats.c Import A536BXXU9EXDC 2024-06-15 16:02:09 -03:00
test_kprobes.c Import A536BXXU9EXDC 2024-06-15 16:02:09 -03:00
torture.c Import A536BXXU9EXDC 2024-06-15 16:02:09 -03:00
tracepoint.c Import A536BXXU9EXDC 2024-06-15 16:02:09 -03:00
tsacct.c Import A536BXXU9EXDC 2024-06-15 16:02:09 -03:00
ucount.c fanotify: configurable limits via sysfs 2024-11-19 12:27:37 +01:00
uid16.c Import A536BXXU9EXDC 2024-06-15 16:02:09 -03:00
uid16.h Import A536BXXU9EXDC 2024-06-15 16:02:09 -03:00
umh.c security: samsung: defex_lsm: nuke 2024-06-15 16:20:49 -03:00
up.c Import A536BXXU9EXDC 2024-06-15 16:02:09 -03:00
user-return-notifier.c Import A536BXXU9EXDC 2024-06-15 16:02:09 -03:00
user.c Import A536BXXU9EXDC 2024-06-15 16:02:09 -03:00
user_namespace.c Import A536BXXU9EXDC 2024-06-15 16:02:09 -03:00
usermode_driver.c Import A536BXXU9EXDC 2024-06-15 16:02:09 -03:00
utsname.c Import A536BXXU9EXDC 2024-06-15 16:02:09 -03:00
utsname_sysctl.c Import A536BXXU9EXDC 2024-06-15 16:02:09 -03:00
watch_queue.c Import A536BXXU9EXDC 2024-06-15 16:02:09 -03:00
watchdog.c watchdog: move softlockup_panic back to early_param 2024-11-18 11:43:21 +01:00
watchdog_hld.c Import A536BXXU9EXDC 2024-06-15 16:02:09 -03:00
workqueue.c workqueue: Make queue_rcu_work() use call_rcu_flush() 2024-11-19 17:37:56 +01:00
workqueue_internal.h Import A536BXXU9EXDC 2024-06-15 16:02:09 -03:00