General protection fault while trying to discover a nvmeof target

Asked by Pradeep

nvme discover command times out and results in general protection fault in kernel.

# nvme discover -t rdma -a 10.1.1.6 -s 4420
Failed to write to /dev/nvme-fabrics: Input/output error^M
#
# [16900.248129] nvme nvme0: Connect command failed, error wo/DNR bit: 7
[16900.248182] nvme nvme0: failed to connect queue: 0 ret=7
[16900.253138] nvme nvme0: Reconnecting in 10 seconds...
[16910.300029] general protection fault: 0000 [#1] SMP PTI
[16910.300069] Modules linked in: mlx4_en mlx4_ib mlx4_core nvme_rdma nvme_fabrics nvme_core rdma_ucm ib_umad ib_ucm ib_uverbs xt_conntrack ipt_MASQUERADE nf_nat_masquerade_ipv4 nfnetlink xfrm_user xfrm_algo iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack xt_addrtype overlay aufs joydev input_leds ipmi_ssif intel_rapl skx_edac x86_pkg_temp_thermal intel_powerclamp coretemp kvm_intel kvm irqbypass intel_cstate intel_rapl_perf mei_me mei ioatdma lpc_ich shpchp sch_fq_codel ipmi_si ipmi_devintf ipmi_msghandler acpi_pad mac_hid acpi_power_meter iptable_filter ip6table_filter ip6_tables br_netfilter bridge stp llc arp_tables ib_iser rdma_cm iw_cm ib_cm iscsi_tcp libiscsi_tcp libiscsi scsi_transport_iscsi ip_tables x_tables autofs4 btrfs zstd_compress raid10 raid456 async_raid6_recov
[16910.300534] async_memcpy async_pq async_xor async_tx xor raid6_pq libcrc32c raid1 raid0 multipath linear mlx5_ib ib_core hid_generic usbhid hid crct10dif_pclmul crc32_pclmul ghash_clmulni_intel pcbc aesni_intel aes_x86_64 crypto_simd glue_helper cryptd mlx5_core ast mlxfw i2c_algo_bit devlink ttm drm_kms_helper ixgbe syscopyarea sysfillrect sysimgblt fb_sys_fops dca ptp drm pps_core mdio ahci libahci wmi [last unloaded: nvme_core]
[16910.300794] CPU: 33 PID: 3173 Comm: kworker/u146:1 Not tainted 4.15.0-33-generic #36-Ubuntu
[16910.300845] Hardware name: Supermicro SYS-6019U-TN4RT/X11DPU, BIOS 2.0b 02/24/2018
[16910.300898] Workqueue: nvme-wq nvme_rdma_reconnect_ctrl_work [nvme_rdma]
[16910.300945] RIP: 0010:nvme_rdma_alloc_queue+0x3c/0x190 [nvme_rdma]
[16910.300984] RSP: 0018:ffff9ba8a037be08 EFLAGS: 00010206
[16910.301019] RAX: 0000000000000000 RBX: 52404d65f5994178 RCX: ffff8a0f78002820
[16910.301063] RDX: ffffffffc0b58600 RSI: ffffffffc0b573ab RDI: ffff8a0f70aec000
[16910.301108] RBP: ffff9ba8a037be28 R08: 0000000000000334 R09: 0000000000000000
[16910.301152] R10: 0000000000000000 R11: 0000000000000337 R12: ffff8a0f70aec000
[16910.301195] R13: 0000000000000000 R14: 0000000000000020 R15: 0000000000000000
[16910.301240] FS: 0000000000000000(0000) GS:ffff8a0f7efc0000(0000) knlGS:0000000000000000
[16910.301289] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[16910.301326] CR2: 00007ffc4e5cadb8 CR3: 000000123ba0a003 CR4: 00000000007606e0
[16910.301370] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[16910.301414] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
[16910.301458] PKRU: 55555554
[16910.301477] Call Trace:
[16910.301504] nvme_rdma_configure_admin_queue+0x22/0x2d0 [nvme_rdma]
[16910.301548] nvme_rdma_reconnect_ctrl_work+0x27/0xd0 [nvme_rdma]
[16910.301591] process_one_work+0x1de/0x410
[16910.301620] worker_thread+0x32/0x410
[16910.301651] kthread+0x121/0x140
[16910.301676] ? process_one_work+0x410/0x410
[16910.301707] ? kthread_create_worker_on_cpu+0x70/0x70
[16910.301744] ? do_syscall_64+0x73/0x130
[16910.301774] ? SyS_exit_group+0x14/0x20
[16910.301805] ret_from_fork+0x35/0x40
[16910.301831] Code: 89 e5 41 56 41 55 41 54 53 48 8d 1c c5 00 00 00 00 49 89 fc 49 89 c5 49 89 d6 48 29 c3 48 c7 c2 00 86 b5 c0 48 c1 e3 04 48 03 1f <48> 89 7b 18 48 8d 7b 58 c7 43 50 00 00 00 00 e8 20 d4 57 ef 45
[16910.305367] RIP: nvme_rdma_alloc_queue+0x3c/0x190 [nvme_rdma] RSP: ffff9ba8a037be08
[16910.307146] ---[ end trace 3ae215b72b2ac870 ]---

#
# lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu 18.04.1 LTS
Release: 18.04
Codename: bionic
#
# uname -a
Linux esdev005 4.15.0-33-generic #36-Ubuntu SMP Wed Aug 15 16:00:05 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux
#

Question information

Language:
English Edit question
Status:
Open
For:
Ubuntu Edit question
Assignee:
No assignee Edit question
Last query:
Last reply:

Can you help with this problem?

Provide an answer of your own, or ask Pradeep for more information if necessary.

To post a message you must log in.