FireAsf 🔥
Find a file
zhenwei pi d4b65d3072 virtio-crypto: wait ctrl queue instead of busy polling
[ Upstream commit 977231e8d45657871a86fe3c7bed94921d04e447 ]

Originally, after submitting request into virtio crypto control
queue, the guest side polls the result from the virt queue. This
works like following:
    CPU0   CPU1               ...             CPUx  CPUy
     |      |                                  |     |
     \      \                                  /     /
      \--------spin_lock(&vcrypto->ctrl_lock)-------/
                           |
                 virtqueue add & kick
                           |
                  busy poll virtqueue
                           |
              spin_unlock(&vcrypto->ctrl_lock)
                          ...

There are two problems:
1, The queue depth is always 1, the performance of a virtio crypto
   device gets limited. Multi user processes share a single control
   queue, and hit spin lock race from control queue. Test on Intel
   Platinum 8260, a single worker gets ~35K/s create/close session
   operations, and 8 workers get ~40K/s operations with 800% CPU
   utilization.
2, The control request is supposed to get handled immediately, but
   in the current implementation of QEMU(v6.2), the vCPU thread kicks
   another thread to do this work, the latency also gets unstable.
   Tracking latency of virtio_crypto_alg_akcipher_close_session in 5s:
        usecs               : count     distribution
         0 -> 1          : 0        |                        |
         2 -> 3          : 7        |                        |
         4 -> 7          : 72       |                        |
         8 -> 15         : 186485   |************************|
        16 -> 31         : 687      |                        |
        32 -> 63         : 5        |                        |
        64 -> 127        : 3        |                        |
       128 -> 255        : 1        |                        |
       256 -> 511        : 0        |                        |
       512 -> 1023       : 0        |                        |
      1024 -> 2047       : 0        |                        |
      2048 -> 4095       : 0        |                        |
      4096 -> 8191       : 0        |                        |
      8192 -> 16383      : 2        |                        |
This means that a CPU may hold vcrypto->ctrl_lock as long as 8192~16383us.

To improve the performance of control queue, a request on control queue
waits completion instead of busy polling to reduce lock racing, and gets
completed by control queue callback.
    CPU0   CPU1               ...             CPUx  CPUy
     |      |                                  |     |
     \      \                                  /     /
      \--------spin_lock(&vcrypto->ctrl_lock)-------/
                           |
                 virtqueue add & kick
                           |
      ---------spin_unlock(&vcrypto->ctrl_lock)------
     /      /                                  \     \
     |      |                                  |     |
    wait   wait                               wait  wait

Test this patch, the guest side get ~200K/s operations with 300% CPU
utilization.

Cc: Michael S. Tsirkin <mst@redhat.com>
Cc: Jason Wang <jasowang@redhat.com>
Cc: Gonglei <arei.gonglei@huawei.com>
Reviewed-by: Gonglei <arei.gonglei@huawei.com>
Signed-off-by: zhenwei pi <pizhenwei@bytedance.com>
Message-Id: <20220506131627.180784-4-pizhenwei@bytedance.com>
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
Stable-dep-of: fed93fb62e05 ("crypto: virtio - Handle dataq logic with tasklet")
Signed-off-by: Sasha Levin <sashal@kernel.org>
2024-11-18 12:12:26 +01:00
.github/workflows Add build stuff 2024-06-15 16:48:05 -03:00
android Import A536BXXU9EXDC 2024-06-15 16:02:09 -03:00
arch powerpc/imc-pmu: Add a null pointer check in update_events_in_group() 2024-11-18 12:12:20 +01:00
block blk-throttle: fix lockdep warning of "cgroup_mutex or RCU read lock required!" 2024-11-18 12:11:56 +01:00
certs Import A536BXXU9EXDC 2024-06-15 16:02:09 -03:00
crypto crypto: pcrypt - Fix hungtask for PADATA_RESET 2024-11-18 11:43:13 +01:00
Documentation dt-bindings: nvmem: mxs-ocotp: Document fsl,ocotp 2024-11-18 12:12:01 +01:00
drivers virtio-crypto: wait ctrl queue instead of busy polling 2024-11-18 12:12:26 +01:00
firmware/tsp_goodix Import A536BXXU9EXDC 2024-06-15 16:02:09 -03:00
fs efivarfs: force RO when remounting if SetVariable is not supported 2024-11-18 12:12:25 +01:00
gki Import A536BXXU9EXDC 2024-06-15 16:02:09 -03:00
include virtio-crypto: introduce akcipher service 2024-11-18 12:12:25 +01:00
init kernel: sched: Configuring PELT half-life via Kconfig 2024-11-17 17:41:11 +01:00
io_uring io_uring/af_unix: disable sending io_uring over sockets 2024-11-18 12:11:46 +01:00
ipc Import A536BXXU9EXDC 2024-06-15 16:02:09 -03:00
kernel ring-buffer: Do not record in NMI if the arch does not support cmpxchg in NMI 2024-11-18 12:12:16 +01:00
kernel_build kernel_build: build.sh: Fixed kernel version on zips/tars 2024-11-17 22:12:20 +01:00
kunitconfigs Import A536BXXU9EXDC 2024-06-15 16:02:09 -03:00
lib ida: Fix crash in ida_free when the bitmap is empty 2024-11-18 12:12:17 +01:00
LICENSES Import A536BXXU9EXDC 2024-06-15 16:02:09 -03:00
mm mm: fix unmap_mapping_range high bits shift bug 2024-11-18 12:12:08 +01:00
net calipso: fix memory leak in netlbl_calipso_add_pass() 2024-11-18 12:12:25 +01:00
samples Import A536BXXU9EXDC 2024-06-15 16:02:09 -03:00
scripts sign-file: Fix incorrect return values check 2024-11-18 12:11:49 +01:00
security selinux: Fix error priority for bind with AF_UNSPEC on PF_INET6 socket 2024-11-18 12:12:25 +01:00
sound ASoC: da7219: Support low DC impedance headset 2024-11-18 12:12:15 +01:00
test Import A536BXXU9EXDC 2024-06-15 16:02:09 -03:00
tools selftests/powerpc: Fix error handling in FPU/VMX preemption tests 2024-11-18 12:12:20 +01:00
usr Import A536BXXU9EXDC 2024-06-15 16:02:09 -03:00
virt Import A536BXXU9EXDC 2024-06-15 16:02:09 -03:00
build.config.aarch64 Import A536BXXU9EXDC 2024-06-15 16:02:09 -03:00
build.config.allmodconfig Import A536BXXU9EXDC 2024-06-15 16:02:09 -03:00
build.config.allmodconfig.aarch64 Import A536BXXU9EXDC 2024-06-15 16:02:09 -03:00
build.config.allmodconfig.arm Import A536BXXU9EXDC 2024-06-15 16:02:09 -03:00
build.config.allmodconfig.x86_64 Import A536BXXU9EXDC 2024-06-15 16:02:09 -03:00
build.config.amlogic Import A536BXXU9EXDC 2024-06-15 16:02:09 -03:00
build.config.arm Import A536BXXU9EXDC 2024-06-15 16:02:09 -03:00
build.config.common Import A536BXXU9EXDC 2024-06-15 16:02:09 -03:00
build.config.db845c Import A536BXXU9EXDC 2024-06-15 16:02:09 -03:00
build.config.erd8825_a25_s Import A536BXXU9EXDC 2024-06-15 16:02:09 -03:00
build.config.erd8825_s Import A536BXXU9EXDC 2024-06-15 16:02:09 -03:00
build.config.erd9925_evt0_s Import A536BXXU9EXDC 2024-06-15 16:02:09 -03:00
build.config.erd9925_evt0_s5300_s Import A536BXXU9EXDC 2024-06-15 16:02:09 -03:00
build.config.erd9925_s Import A536BXXU9EXDC 2024-06-15 16:02:09 -03:00
build.config.gki Import A536BXXU9EXDC 2024-06-15 16:02:09 -03:00
build.config.gki-debug.aarch64 Import A536BXXU9EXDC 2024-06-15 16:02:09 -03:00
build.config.gki-debug.x86_64 Import A536BXXU9EXDC 2024-06-15 16:02:09 -03:00
build.config.gki.aarch64 Import A536BXXU9EXDC 2024-06-15 16:02:09 -03:00
build.config.gki.aarch64.fips140 Import A536BXXU9EXDC 2024-06-15 16:02:09 -03:00
build.config.gki.aarch64.fips140_eval_testing Import A536BXXU9EXDC 2024-06-15 16:02:09 -03:00
build.config.gki.x86_64 Import A536BXXU9EXDC 2024-06-15 16:02:09 -03:00
build.config.gki_kasan Import A536BXXU9EXDC 2024-06-15 16:02:09 -03:00
build.config.gki_kasan.aarch64 Import A536BXXU9EXDC 2024-06-15 16:02:09 -03:00
build.config.gki_kasan.x86_64 Import A536BXXU9EXDC 2024-06-15 16:02:09 -03:00
build.config.gki_kprobes Import A536BXXU9EXDC 2024-06-15 16:02:09 -03:00
build.config.gki_kprobes.aarch64 Import A536BXXU9EXDC 2024-06-15 16:02:09 -03:00
build.config.gki_kprobes.x86_64 Import A536BXXU9EXDC 2024-06-15 16:02:09 -03:00
build.config.hikey960 Import A536BXXU9EXDC 2024-06-15 16:02:09 -03:00
build.config.khwasan Import A536BXXU9EXDC 2024-06-15 16:02:09 -03:00
build.config.mcd Import A536BXXU9EXDC 2024-06-15 16:02:09 -03:00
build.config.rockchip Import A536BXXU9EXDC 2024-06-15 16:02:09 -03:00
build.config.universal2100_s Import A536BXXU9EXDC 2024-06-15 16:02:09 -03:00
build.config.universal8825_s Import A536BXXU9EXDC 2024-06-15 16:02:09 -03:00
build.config.universal9925_evt0_s Import A536BXXU9EXDC 2024-06-15 16:02:09 -03:00
build.config.universal9925_s Import A536BXXU9EXDC 2024-06-15 16:02:09 -03:00
build.config.x86_64 Import A536BXXU9EXDC 2024-06-15 16:02:09 -03:00
build.sh build.sh: Disabled temporary build without debug 2024-11-17 22:32:40 +01:00
COPYING Import A536BXXU9EXDC 2024-06-15 16:02:09 -03:00
CREDITS Import A536BXXU9EXDC 2024-06-15 16:02:09 -03:00
Kbuild Import A536BXXU9EXDC 2024-06-15 16:02:09 -03:00
Kconfig Import A536BXXU9EXDC 2024-06-15 16:02:09 -03:00
linux-stable.sh linux-stable.sh: Added for upstream 2024-11-08 11:11:32 +01:00
MAINTAINERS Import A536BXXU9EXDC 2024-06-15 16:02:09 -03:00
Makefile Linux 5.10.208 2024-11-18 12:12:11 +01:00
OWNERS Import A536BXXU9EXDC 2024-06-15 16:02:09 -03:00
README Import A536BXXU9EXDC 2024-06-15 16:02:09 -03:00
README.md Import A536BXXU9EXDC 2024-06-15 16:02:09 -03:00
vendor_boot_module_order_exynos2100.cfg Import A536BXXU9EXDC 2024-06-15 16:02:09 -03:00
vendor_boot_module_order_s5e8825.cfg Import A536BXXU9EXDC 2024-06-15 16:02:09 -03:00
vendor_boot_module_order_s5e9925.cfg Import A536BXXU9EXDC 2024-06-15 16:02:09 -03:00
vendor_module_list_s5e8825.cfg Import A536BXXU9EXDC 2024-06-15 16:02:09 -03:00
vendor_module_list_s5e9925.cfg Import A536BXXU9EXDC 2024-06-15 16:02:09 -03:00
vendor_module_list_s5e9925_b0s.cfg Import A536BXXU9EXDC 2024-06-15 16:02:09 -03:00
vendor_module_list_s5e9925_g0s.cfg Import A536BXXU9EXDC 2024-06-15 16:02:09 -03:00
vendor_module_list_s5e9925_r0s.cfg Import A536BXXU9EXDC 2024-06-15 16:02:09 -03:00

How do I submit patches to Android Common Kernels

  1. BEST: Make all of your changes to upstream Linux. If appropriate, backport to the stable releases. These patches will be merged automatically in the corresponding common kernels. If the patch is already in upstream Linux, post a backport of the patch that conforms to the patch requirements below.

    • Do not send patches upstream that contain only symbol exports. To be considered for upstream Linux, additions of EXPORT_SYMBOL_GPL() require an in-tree modular driver that uses the symbol -- so include the new driver or changes to an existing driver in the same patchset as the export.
    • When sending patches upstream, the commit message must contain a clear case for why the patch is needed and beneficial to the community. Enabling out-of-tree drivers or functionality is not not a persuasive case.
  2. LESS GOOD: Develop your patches out-of-tree (from an upstream Linux point-of-view). Unless these are fixing an Android-specific bug, these are very unlikely to be accepted unless they have been coordinated with kernel-team@android.com. If you want to proceed, post a patch that conforms to the patch requirements below.

Common Kernel patch requirements

  • All patches must conform to the Linux kernel coding standards and pass script/checkpatch.pl
  • Patches shall not break gki_defconfig or allmodconfig builds for arm, arm64, x86, x86_64 architectures (see https://source.android.com/setup/build/building-kernels)
  • If the patch is not merged from an upstream branch, the subject must be tagged with the type of patch: UPSTREAM:, BACKPORT:, FROMGIT:, FROMLIST:, or ANDROID:.
  • All patches must have a Change-Id: tag (see https://gerrit-review.googlesource.com/Documentation/user-changeid.html)
  • If an Android bug has been assigned, there must be a Bug: tag.
  • All patches must have a Signed-off-by: tag by the author and the submitter

Additional requirements are listed below based on patch type

Requirements for backports from mainline Linux: UPSTREAM:, BACKPORT:

  • If the patch is a cherry-pick from Linux mainline with no changes at all
    • tag the patch subject with UPSTREAM:.
    • add upstream commit information with a (cherry picked from commit ...) line
    • Example:
      • if the upstream commit message is
        important patch from upstream

        This is the detailed description of the important patch

        Signed-off-by: Fred Jones <fred.jones@foo.org>
  • then Joe Smith would upload the patch for the common kernel as
        UPSTREAM: important patch from upstream

        This is the detailed description of the important patch

        Signed-off-by: Fred Jones <fred.jones@foo.org>

        Bug: 135791357
        Change-Id: I4caaaa566ea080fa148c5e768bb1a0b6f7201c01
        (cherry picked from commit c31e73121f4c1ec41143423ac6ce3ce6dafdcec1)
        Signed-off-by: Joe Smith <joe.smith@foo.org>
  • If the patch requires any changes from the upstream version, tag the patch with BACKPORT: instead of UPSTREAM:.
    • use the same tags as UPSTREAM:
    • add comments about the changes under the (cherry picked from commit ...) line
    • Example:
        BACKPORT: important patch from upstream

        This is the detailed description of the important patch

        Signed-off-by: Fred Jones <fred.jones@foo.org>

        Bug: 135791357
        Change-Id: I4caaaa566ea080fa148c5e768bb1a0b6f7201c01
        (cherry picked from commit c31e73121f4c1ec41143423ac6ce3ce6dafdcec1)
        [joe: Resolved minor conflict in drivers/foo/bar.c ]
        Signed-off-by: Joe Smith <joe.smith@foo.org>

Requirements for other backports: FROMGIT:, FROMLIST:,

  • If the patch has been merged into an upstream maintainer tree, but has not yet been merged into Linux mainline
    • tag the patch subject with FROMGIT:
    • add info on where the patch came from as (cherry picked from commit <sha1> <repo> <branch>). This must be a stable maintainer branch (not rebased, so don't use linux-next for example).
    • if changes were required, use BACKPORT: FROMGIT:
    • Example:
      • if the commit message in the maintainer tree is
        important patch from upstream

        This is the detailed description of the important patch

        Signed-off-by: Fred Jones <fred.jones@foo.org>
  • then Joe Smith would upload the patch for the common kernel as
        FROMGIT: important patch from upstream

        This is the detailed description of the important patch

        Signed-off-by: Fred Jones <fred.jones@foo.org>

        Bug: 135791357
        (cherry picked from commit 878a2fd9de10b03d11d2f622250285c7e63deace
         https://git.kernel.org/pub/scm/linux/kernel/git/foo/bar.git test-branch)
        Change-Id: I4caaaa566ea080fa148c5e768bb1a0b6f7201c01
        Signed-off-by: Joe Smith <joe.smith@foo.org>
  • If the patch has been submitted to LKML, but not accepted into any maintainer tree
    • tag the patch subject with FROMLIST:
    • add a Link: tag with a link to the submittal on lore.kernel.org
    • add a Bug: tag with the Android bug (required for patches not accepted into a maintainer tree)
    • if changes were required, use BACKPORT: FROMLIST:
    • Example:
        FROMLIST: important patch from upstream

        This is the detailed description of the important patch

        Signed-off-by: Fred Jones <fred.jones@foo.org>

        Bug: 135791357
        Link: https://lore.kernel.org/lkml/20190619171517.GA17557@someone.com/
        Change-Id: I4caaaa566ea080fa148c5e768bb1a0b6f7201c01
        Signed-off-by: Joe Smith <joe.smith@foo.org>

Requirements for Android-specific patches: ANDROID:

  • If the patch is fixing a bug to Android-specific code
    • tag the patch subject with ANDROID:
    • add a Fixes: tag that cites the patch with the bug
    • Example:
        ANDROID: fix android-specific bug in foobar.c

        This is the detailed description of the important fix

        Fixes: 1234abcd2468 ("foobar: add cool feature")
        Change-Id: I4caaaa566ea080fa148c5e768bb1a0b6f7201c01
        Signed-off-by: Joe Smith <joe.smith@foo.org>
  • If the patch is a new feature
    • tag the patch subject with ANDROID:
    • add a Bug: tag with the Android bug (required for android-specific features)