snmptrapd Segmentation Faults When Calling my_load_defaults()

Bug #1979933 reported by Matthew Ruffell
12
This bug affects 1 person
Affects Status Importance Assigned to Milestone
net-snmp (Ubuntu)
Fix Released
Undecided
Unassigned
Focal
Fix Released
Medium
Matthew Ruffell

Bug Description

[Impact]

When starting snmptrapd configured to connect to a mysql server, we segmentation fault when calling my_load_defaults():

$ sudo /usr/sbin/snmptrapd -LOw -f
Segmentation fault (core dumped)

(gdb) bt
#0 0x00007efd8d9f15b0 in MEM_ROOT::Alloc (length=56, this=this@entry=0x0) at ./include/my_alloc.h:157
#1 init_default_directories (alloc=alloc@entry=0x0) at ./mysys/my_default.cc:1632
#2 0x00007efd8d9f817a in my_load_defaults (conf_file=0x7efd8e751f64 "my", groups=0x7efd8e7550c8, argc=0x7ffd8941b674, argv=0x7ffd8941b678, alloc=0x0, default_directories=0x555a8abccc00) at ./mysys/my_default.cc:692
#3 0x00007efd8e74fe1b in netsnmp_mysql_init () from /lib/x86_64-linux-gnu/libnetsnmptrapd.so.35
#4 0x0000555a8a9e3873 in main (argc=<optimized out>, argv=<optimized out>) at snmptrapd.c:1196

What happens is that we are calling my_load_defaults() even though we have mysql_options(), and the arguments we pass into my_load_defaults() are NULL, which eventually get de-referenced.

The fix is to change the configure script to only call my_load_defaults() if we don't have mysql_options().

[Testcase]

$ sudo apt update
$ sudo apt install snmp snmpd snmptrapd snmp-mibs-downloader

Edit /etc/snmp/snmptrapd.conf and add the following entries:

disableAuthorization yes
traphandle default /usr/bin/logger
sqlMaxQueue 1
sqlSaveInterval 9

Save and exit.

It is easier to reproduce if you stop and disable all services:

$ sudo systemctl stop snmptrapd.service
$ sudo systemctl stop snmpd.service

Then try running:

$ sudo /usr/sbin/snmptrapd -LOw -f
Segmentation fault (core dumped)

There is a test package available in the following ppa:

https://launchpad.net/~mruffell/+archive/ubuntu/sf339433-test

When running this test package, you will instead see:

$ sudo /usr/sbin/snmptrapd -LOw -f
mysql_real_connect() failed
Error 2002 (HY000): Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2)

(I don't have a mysql server configured, but we don't segfault anymore).

[Where problems can occur]

We are changing how snmptrapd initialises and begins connections to a mysql server, and if a regression were to occur, it would be limited to users of snmptrapd with the mysql backend. Other database backends would not be affected.

Other binaries produced would also not be affected.

[Other Info]

The issue was fixed upstream by the following commit:

commit 011342d8e453b9e0585bf77f659d80c648df8c9f
Author: Bart Van Assche <email address hidden>
Date: Sat Aug 18 09:28:14 2018 -0700
Subject: snmptrapd: Let configure check for mysql_options()
Link: https://github.com/net-snmp/net-snmp/commit/011342d8e453b9e0585bf77f659d80c648df8c9f

Changed in net-snmp (Ubuntu):
status: New → Fix Released
Changed in net-snmp (Ubuntu Focal):
status: New → In Progress
importance: Undecided → Medium
assignee: nobody → Matthew Ruffell (mruffell)
tags: added: focal sts
description: updated
Revision history for this message
Matthew Ruffell (mruffell) wrote :

Attached is a patch that resolves the issue on Focal.

description: updated
tags: added: sts-sponsor
tags: added: sts-sponsor-mfo
removed: sts-sponsor
Revision history for this message
Mauricio Faria de Oliveira (mfo) wrote :

Hi Matthew,

Thanks for the debdiff; sponsored to focal!

I just fixed the Description: field in the patch DEP3 headers,
which isn't as free-form as Subject: line + unstructured text.
(Please see sample in DEP-3 [1] and multiline fields/paragraph
separators in Debian Policy [2] for details, if you'd like it.)

Everything else looks excellent, and I'll just document checks.

cheers,
Mauricio

- Built successfully on all architectures with focal-proposed [0].

- Upstream source code remains the same as with this change,
as of commit 4d43fbd64a07 ("Makefile.in: Only build the fuzzing tests in regular mode").

- Focal's MySQL source code, of course, defines the relied upon
function mysql_options():

    int STDCALL mysql_options(MYSQL *mysql, enum mysql_option option,
                              const void *arg);

- Supported releases after Focal already have the fix:

 $ git describe --contains 011342d8e453b9e0585bf77f659d80c648df8c9f
 v5.8.1.pre1~91

 $ rmadison --arch=source net-snmp
 ...
  net-snmp | 5.8+dfsg-2ubuntu2 | focal | source
  net-snmp | 5.8+dfsg-2ubuntu2.3 | focal-security | source
  net-snmp | 5.8+dfsg-2ubuntu2.3 | focal-updates | source
  net-snmp | 5.9.1+dfsg-1ubuntu2 | jammy | source
  net-snmp | 5.9.1+dfsg-1ubuntu2.1 | jammy-updates | source
  net-snmp | 5.9.1+dfsg-4ubuntu1 | kinetic | source

- The backport delta (thanks for noting it in DEP-3's Origin:)
is 2 context lines:

 - /* Define to 1 if you have the <my_global.h> header file. */
 - #undef HAVE_MY_GLOBAL_H
 + /* Define if MY_INIT() is availabe in libmysqlclient */
 + #undef HAVE_MY_INIT

- The patch seems to no longer accept command line arguments to
specify connection parameters to MySQL, but it looks like that
was never the purpose (just some flexibility, I guess) per:

* the original code commit

* the comment "load .my.cnf values" in the failing / now disabled calls:

     /** load .my.cnf values */
 #if HAVE_MY_LOAD_DEFAULTS
     my_load_defaults ("my", _sql.groups, &not_argc, &not_argv, 0);
 #elif defined(HAVE_LOAD_DEFAULTS)
     load_defaults ("my", _sql.groups, &not_argc, &not_argv);

* the usage of mysql_options(MYSQL_READ_DEFAULT_GROUP) [3]:

   MYSQL_READ_DEFAULT_GROUP (argument type: char *)

   Read options from the named group from my.cnf or the file specified with MYSQL_READ_DEFAULT_FILE.

* Most importantly, the project's README.sql file:

 net-snmp-5.8+dfsg$ head -n15 README.sql
 snmptrapd MySQL Logging
 -----------------------

 A trap handler for logging traps to a MySQL database was added
 in release 5.5.0.

 The MySQL database location and password must be configured in
 /root/.my.cnf:

  [snmptrapd]
  host=localhost
  password=sql

 User may also be configured, if using a MySQL user besides root.

[0] https://launchpad.net/~mfo/+archive/ubuntu/lp1979933
[1] https://dep-team.pages.debian.net/deps/dep3/
[2] https://www.debian.org/doc/debian-policy/ch-controlfields.html#syntax-of-control-files
[3] https://dev.mysql.com/doc/c-api/8.0/en/mysql-options.html

Revision history for this message
Matthew Ruffell (mruffell) wrote :

The new upload in the -unapproved upload queue has been bumped by a security update, and so, attaching a new debdiff rebased to the current package in -updates and -security.

Mauricio, could you please sponsor the new debdiff?

I took the feedback on Description field formatting into account, should be fixed on this debdiff.

Revision history for this message
Mauricio Faria de Oliveira (mfo) wrote :

Hey Matthew,

Sure; uploaded to Focal.

I've reviewed the differences between the debdiffs (all good) and that the security update isn't related to the change content (no expected/incremental behavior change; all good), and build tested it.

Thanks for the prompt reaction/update on top of the security pocket, and the DEP3 change.

cheers,
Mauricio

Revision history for this message
Robie Basak (racb) wrote :

In this upload you're patching the generated configure script directly, but not configure.ac. This may cause problems with future updates, including security updates, if the configure script needs to be regenerated. Did you attempt to use dh-autoreconf?

Changed in net-snmp (Ubuntu Focal):
status: In Progress → Incomplete
Revision history for this message
Robie Basak (racb) wrote :

Oh, you did indirectly patch configure.ac by patching configure.d/, sorry.

I asked in #ubuntu-devel about this, and the conclusion is that this is OK:

https://irclogs.ubuntu.com/2022/08/03//%23ubuntu-devel.html#t11:56

Changed in net-snmp (Ubuntu Focal):
status: Incomplete → Fix Committed
tags: added: verification-needed verification-needed-focal
Revision history for this message
Robie Basak (racb) wrote : Please test proposed package

Hello Matthew, or anyone else affected,

Accepted net-snmp into focal-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/net-snmp/5.8+dfsg-2ubuntu2.5 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-focal to verification-done-focal. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-focal. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

Revision history for this message
Matthew Ruffell (mruffell) wrote :

Performing verification for Focal

I install snmptrapd 5.8+dfsg-2ubuntu2.4 from updates, and edited /etc/snmp/snmptrapd.conf to have the following contents:

disableAuthorization yes
traphandle default /usr/bin/logger
sqlMaxQueue 1
sqlSaveInterval 9

I disabled all snmptrapd services:

$ sudo systemctl stop snmptrapd.service
$ sudo systemctl stop snmpd.service

and then tried to start snmptrapd, but received a segmentation fault:

$ sudo /usr/sbin/snmptrapd -LOw -f
Segmentation fault

From syslog:
Aug 4 00:47:45 focal-snmp kernel: [ 1615.972207] snmptrapd[5293]: segfault at 8 ip 00007f4879003680 sp 00007ffe914993f0 error 4 in libmysqlclient.so.21.2.30[7f4878f93000+130000]
Aug 4 00:47:45 focal-snmp kernel: [ 1615.972229] Code: 5c 41 5d 41 5e 5d c3 e8 0e 17 f9 ff 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 40 00 55 48 89 e5 41 55 49 89 fd 41 54 53 48 83 ec 08 <4c> 8b 67 08 48 8b 47 10 4c 29 e0 48 83 f8 37 0f 86 cb 00 00 00 49

I then enabled -proposed and installed snmptrapd 5.8+dfsg-2ubuntu2.5.

Again, I stopped the services:

$ sudo systemctl stop snmptrapd.service
$ sudo systemctl stop snmpd.service

and tried to start snmptrapd:

$ sudo /usr/sbin/snmptrapd -LOw -f
mysql_real_connect() failed
Error 2002 (HY000): Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2)

This time we no longer segmentation fault, and snmptrapd keeps running.

The package in -proposed fixes the issue, happy to mark verified.

tags: added: verification-done-focal
removed: verification-needed verification-needed-focal
Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package net-snmp - 5.8+dfsg-2ubuntu2.5

---------------
net-snmp (5.8+dfsg-2ubuntu2.5) focal; urgency=medium

  * Fix segmentation fault when attempting to initialise mysql
    connections due to incorrectly calling my_load_defaults()
    (LP: #1979933).
    - d/p/lp1979933-snmptrapd-Let-configure-check-for-mysql_options.patch

 -- Matthew Ruffell <email address hidden> Tue, 02 Aug 2022 15:55:28 +1200

Changed in net-snmp (Ubuntu Focal):
status: Fix Committed → Fix Released
Revision history for this message
Brian Murray (brian-murray) wrote : Update Released

The verification of the Stable Release Update for net-snmp has completed successfully and the package is now being released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.