[Azure] WARNING: CPU: 0 PID: 499 at include/linux/dma-mapping.h:555 netvsc_probe+0x3c9/0x3e0

Bug #1975717 reported by Tim Gardner
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
linux-azure (Ubuntu)
Fix Released
Undecided
Unassigned
Impish
Fix Released
Medium
Tim Gardner
Jammy
Fix Released
Medium
Tim Gardner

Bug Description

SRU Justification

[Impact]

When I hot-add a NIC to a Ubuntu 20.04 VM (“5.13.0-1023-azure #27~20.04.1-Ubuntu”), I get the the below call-trace.
Please include the March-2022 fix: “Drivers: hv: vmbus: Fix initialization of device object in vmbus_device_register()” (https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=3a5469582c241abca22500f36a9cb8e9331969cf)

BTW, this call-trace should be harmless to a non-CVM VM.

[ 364.275105] hv_utils: Heartbeat IC version 3.0
[ 364.275137] hv_utils: KVP IC version 4.0
[ 364.275146] hv_utils: Shutdown IC version 3.2
[ 364.275153] hv_utils: TimeSync IC version 4.0
[ 365.281376] ------------[ cut here ]------------
[ 365.281380] WARNING: CPU: 0 PID: 499 at include/linux/dma-mapping.h:555 netvsc_probe+0x3c9/0x3e0 [hv_netvsc]
[ 365.281392] Modules linked in: udf crc_itu_t iptable_mangle iptable_filter iptable_raw xt_LOG nf_log_syslog bpfilter nls_iso8859_1 dm_multipath scsi_dh_rdac scsi_dh_emc scsi_dh_alua kvm_intel kvm joydev hid_generic crct10dif_pclmul crc32_pclmul ghash_clmulni_intel aesni_intel crypto_simd cryptd serio_raw hv_netvsc pata_acpi hyperv_keyboard hid_hyperv hv_utils hv_balloon hyperv_fb hid sch_fq_codel ipmi_devintf drm ipmi_msghandler msr i2c_core ip_tables x_tables autofs4
[ 365.281422] CPU: 0 PID: 499 Comm: kworker/0:3 Not tainted 5.13.0-1023-azure #27~20.04.1-Ubuntu
[ 365.281424] Hardware name: Microsoft Corporation Virtual Machine/Virtual Machine, BIOS 090008 12/07/2018
[ 365.281427] Workqueue: hv_pri_chan vmbus_add_channel_work
[ 365.281434] RIP: 0010:netvsc_probe+0x3c9/0x3e0 [hv_netvsc]
[ 365.281440] Code: 37 c0 4c 89 e6 48 c7 c7 98 f2 37 c0 e8 70 b1 c4 da e9 e9 fc ff ff 49 c7 84 24 88 0b 00 00 00 00 00 00 41 bf f4 ff ff ff eb b0 <0f> 0b e9 19 fe ff ff 41 bf f4 ff ff ff e9 51 ff ff ff 0f 1f 44 00
[ 365.281442] RSP: 0018:ffff99270075bbd0 EFLAGS: 00010246
[ 365.281444] RAX: 0000000000000000 RBX: ffff890840b21000 RCX: 0000000000000002
[ 365.281446] RDX: 0000000000000002 RSI: 0000000000000000 RDI: 0000000000000002
[ 365.281447] RBP: ffff99270075bbf8 R08: 0000000000000000 R09: 0000000000000000
[ 365.281448] R10: ffffffff9c74e820 R11: 0000000000000394 R12: ffff890844dc6000
[ 365.281449] R13: 0000000000000002 R14: 0000301d0800ae28 R15: 000000000000000f
[ 365.281451] FS: 0000000000000000(0000) GS:ffff8909f7c00000(0000) knlGS:0000000000000000
[ 365.281453] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 365.281454] CR2: 00005619b3504d38 CR3: 00000001091f8005 CR4: 00000000003706f0
[ 365.281459] Call Trace:
[ 365.281461] <TASK>
[ 365.281465] vmbus_probe+0x67/0x80
[ 365.281468] really_probe+0x1dc/0x440
[ 365.281472] driver_probe_device+0xf0/0x160
[ 365.281476] __device_attach_driver+0x79/0xe0
[ 365.281479] ? driver_allows_async_probing+0x50/0x50
[ 365.281482] bus_for_each_drv+0x84/0xd0
[ 365.281485] __device_attach+0xed/0x170
[ 365.281488] device_initial_probe+0x13/0x20
[ 365.281491] bus_probe_device+0x8f/0xa0
[ 365.281494] device_add+0x3f4/0x8e0
[ 365.281496] ? hrtimer_init+0x2b/0x70
[ 365.281501] device_register+0x1b/0x20
[ 365.281503] vmbus_device_register+0x5e/0xe0
[ 365.281505] vmbus_add_channel_work+0x12d/0x190
[ 365.281507] process_one_work+0x21a/0x3b0
[ 365.281511] worker_thread+0x4d/0x3e0
[ 365.281515] ? process_one_work+0x3b0/0x3b0
[ 365.281517] kthread+0x12b/0x150
[ 365.281521] ? set_kthread_struct+0x40/0x40
[ 365.281523] ret_from_fork+0x22/0x30
[ 365.281528] </TASK>
[ 365.281529] ---[ end trace 80a393e06f0ee58d ]---
[ 365.514764] hv_balloon: Max. dynamic memory size: 8192 MB
[ 367.937000] IPv6: ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready

[Test Case]

Microsoft tested

[Where things could go wrong]

CVM VNIC instances may not initialize correctly.

[Other Info]

SF: #00337569

Tim Gardner (timg-tpi)
affects: linux (Ubuntu) → linux-azure (Ubuntu)
Changed in linux-azure (Ubuntu):
status: New → Invalid
status: Invalid → Fix Released
Changed in linux-azure (Ubuntu Impish):
assignee: nobody → Tim Gardner (timg-tpi)
importance: Undecided → Medium
status: New → In Progress
Changed in linux-azure (Ubuntu Jammy):
assignee: nobody → Tim Gardner (timg-tpi)
importance: Undecided → Medium
status: New → In Progress
Tim Gardner (timg-tpi)
Changed in linux-azure (Ubuntu Impish):
status: In Progress → Fix Committed
Changed in linux-azure (Ubuntu Jammy):
status: In Progress → Fix Committed
Revision history for this message
Ubuntu Kernel Bot (ubuntu-kernel-bot) wrote :

This bug is awaiting verification that the linux-azure/5.15.0-1008.9 kernel in -proposed solves the problem. Please test the kernel and update this bug with the results. If the problem is solved, change the tag 'verification-needed-jammy' to 'verification-done-jammy'. If the problem still exists, change the tag 'verification-needed-jammy' to 'verification-failed-jammy'.

If verification is not done by 5 working days from today, this fix will be dropped from the source code, and this bug will be closed.

See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Thank you!

tags: added: verification-needed-jammy
tags: added: verification-needed-impish
Revision history for this message
Ubuntu Kernel Bot (ubuntu-kernel-bot) wrote :

This bug is awaiting verification that the linux-azure/5.13.0-1026.30 kernel in -proposed solves the problem. Please test the kernel and update this bug with the results. If the problem is solved, change the tag 'verification-needed-impish' to 'verification-done-impish'. If the problem still exists, change the tag 'verification-needed-impish' to 'verification-failed-impish'.

If verification is not done by 5 working days from today, this fix will be dropped from the source code, and this bug will be closed.

See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Thank you!

Revision history for this message
Tim Gardner (timg-tpi) wrote :

Microsoft tested. Marking verification done.

tags: added: verification-done-impish verification-done-jammy
removed: verification-needed-impish verification-needed-jammy
Revision history for this message
Launchpad Janitor (janitor) wrote :
Download full text (12.2 KiB)

This bug was fixed in the package linux-azure - 5.13.0-1026.30

---------------
linux-azure (5.13.0-1026.30) impish; urgency=medium

  * impish/linux-azure: 5.13.0-1026.30 -proposed tracker (LP: #1974334)

  * [Azure] WARNING: CPU: 0 PID: 499 at include/linux/dma-mapping.h:555
    netvsc_probe+0x3c9/0x3e0 (LP: #1975717)
    - Drivers: hv: vmbus: Rework use of DMA_BIT_MASK(64)
    - Drivers: hv: vmbus: Fix initialization of device object in
      vmbus_device_register()

  * [Azure] hv_netvsc: Add support for XDP_REDIRECT (LP: #1972832)
    - hv_netvsc: Add comment of netvsc_xdp_xmit()
    - hv_netvsc: Add support for XDP_REDIRECT

  * linux-azure: Patch Set for ARM64 Images 20.04 and 18.04 (LP: #1970468)
    - Drivers: hv: vmbus: Replace smp_store_mb() with virt_store_mb()
    - Drivers: hv: balloon: Support status report for larger page sizes
    - Drivers: hv: balloon: Disable balloon and hot-add accordingly

  * [Azure] PCI: hv: Do not set PCI_COMMAND_MEMORY to reduce VM boot time
    (LP: #1972662)
    - PCI: hv: Do not set PCI_COMMAND_MEMORY to reduce VM boot time

  * config CONFIG_HISI_PMU for kunpeng920 (LP: #1956086)
    - [Config] azure: CONFIG_HISI_PMU=m

  [ Ubuntu: 5.13.0-46.51 ]

  * CVE-2022-21499
    - SAUCE: debug: Lock down kgdb

  [ Ubuntu: 5.13.0-45.50 ]

  * impish/linux: 5.13.0-45.50 -proposed tracker (LP: #1974347)
  * CVE-2022-1158
    - KVM: x86/mmu: do compare-and-exchange of gPTE via the user address
  * config CONFIG_HISI_PMU for kunpeng920 (LP: #1956086)
    - [Config] CONFIG_HISI_PMU=m
  * re-apply missing overlayfs SAUCE patch (LP: #1967924)
    - SAUCE: overlayfs: fix incorrect mnt_id of files opened from map_files
  * Impish update: upstream stable patchset 2022-04-20 (LP: #1969666)
    - mac80211_hwsim: report NOACK frames in tx_status
    - mac80211_hwsim: initialize ieee80211_tx_info at hw_scan_work
    - i2c: bcm2835: Avoid clock stretching timeouts
    - ASoC: rt5668: do not block workqueue if card is unbound
    - ASoC: rt5682: do not block workqueue if card is unbound
    - regulator: core: fix false positive in regulator_late_cleanup()
    - KVM: arm64: vgic: Read HW interrupt pending state from the HW
    - tipc: fix a bit overflow in tipc_crypto_key_rcv()
    - cifs: fix double free race when mount fails in cifs_get_root()
    - selftests/seccomp: Fix seccomp failure by adding missing headers
    - i2c: cadence: allow COMPILE_TEST
    - i2c: qup: allow COMPILE_TEST
    - net: usb: cdc_mbim: avoid altsetting toggling for Telit FN990
    - usb: gadget: don't release an existing dev->buf
    - usb: gadget: clear related members when goto fail
    - exfat: reuse exfat_inode_info variable instead of calling EXFAT_I()
    - exfat: fix i_blocks for files truncated over 4 GiB
    - tracing: Add test for user space strings when filtering on string pointers
    - serial: stm32: prevent TDR register overwrite when sending x_char
    - ata: pata_hpt37x: fix PCI clock detection
    - drm/amdgpu: check vm ready by amdgpu_vm->evicting flag
    - tracing: Add ustring operation to filtering string pointers
    - ALSA: intel_hdmi: Fix reference to PCM buffer address
    - riscv/efi_stub: Fix get_boot_hartid_from_fdt...

Changed in linux-azure (Ubuntu Impish):
status: Fix Committed → Fix Released
Revision history for this message
Launchpad Janitor (janitor) wrote :
Download full text (105.5 KiB)

This bug was fixed in the package linux-azure - 5.15.0-1008.9

---------------
linux-azure (5.15.0-1008.9) jammy; urgency=medium

  * jammy/linux-azure: 5.15.0-1008.9 -proposed tracker (LP: #1974294)

  * Packaging resync (LP: #1786013)
    - debian/dkms-versions -- update from kernel-versions (main/2022.04.18)

  * [Azure] WARNING: CPU: 0 PID: 499 at include/linux/dma-mapping.h:555
    netvsc_probe+0x3c9/0x3e0 (LP: #1975717)
    - Drivers: hv: vmbus: Rework use of DMA_BIT_MASK(64)
    - Drivers: hv: vmbus: Fix initialization of device object in
      vmbus_device_register()

  * config CONFIG_HISI_PMU for kunpeng920 (LP: #1956086)
    - [Config] azure: CONFIG_HISI_PMU=m

  * linux: CONFIG_SERIAL_8250_MID=y (LP: #1967338)
    - [Config] azure: CONFIG_SERIAL_8250_MID=y

  * Support AMD P-State cpufreq control mechanism (LP: #1956509) // Enable
    speakup kernel modules to allow the speakup screen reader to function
    (LP: #1967702)
    - [Config] azure: Update configs after rebase

  * Azure: swiotlb patch needed for CVM (LP: #1971701) // [Azure][CVM] Fix
    swiotlb_max_mapping_size() for potential bounce buffer allocation failure in
    storvsc (LP: #1973169)
    - SAUCE: swiotlb: Max mapping size takes min align mask into account

  * Azure: swiotlb patch needed for CVM (LP: #1971701)
    - SAUCE: treewide: Replace the use of mem_encrypt_active() with
      cc_platform_has()
    - SAUCE: swiotlb: use bitmap to track free slots
    - SAUCE: swiotlb: allocate memory in a cache-friendly way
    - SAUCE: swiotlb: Split up single swiotlb lock

  * jammy/linux-azure: Update cifs to 5.15 backport (LP: #1970977)
    - improve error message when mount options conflict with posix
    - cifs: call cifs_reconnect when a connection is marked
    - cifs: call helper functions for marking channels for reconnect
    - cifs: mark sessions for reconnection in helper function
    - treewide: Replace zero-length arrays with flexible-array members
    - smb3: fix incorrect session setup check for multiuser mounts
    - cifs: truncate the inode and mapping when we simulate fcollapse
    - cifs: use a different reconnect helper for non-cifsd threads
    - cifs: do not skip link targets when an I/O fails
    - cifs: convert the path to utf16 in smb2_query_info_compound
    - cifs: change smb2_query_info_compound to use a cached fid, if available
    - cifs: fix bad fids sent over wire
    - cifs: fix incorrect use of list iterator after the loop
    - move more common protocol header definitions to smbfs_common
    - smb3: move defines for ioctl protocol header and SMB2 sizes to smbfs_common
    - smb3: move defines for query info and query fsinfo to smbfs_common
    - smb3: cleanup and clarify status of tree connections
    - smb3: fix ksmbd bigendian bug in oplock break, and move its struct to
      smbfs_common
    - fs: Remove ->readpages address space operation
    - cifs: fix potential race with cifsd thread
    - cifs: remove check of list iterator against head past the loop body
    - cifs: force new session setup and tcon for dfs
    - cifs: update internal module number
    - cifs: Check the IOCB_DIRECT flag, not O_DIRECT
    - cifs: Split the smb3_a...

Changed in linux-azure (Ubuntu Jammy):
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.