Ubuntu
systemtap package

dtrace predictable temp file causes race

Bug #1962785 reported by dann frazier
8
This bug affects 1 person
Affects Status Importance Assigned to Milestone
systemtap (Debian)
Fix Released
Unknown
systemtap (Ubuntu)
Fix Released
Undecided
Unassigned
Focal
Fix Released
Undecided
dann frazier
Impish
Fix Released
Undecided
dann frazier
Jammy
Fix Released
Undecided
Unassigned

Bug Description

[Impact]
dtrace generates .c code in a predictable temporary file which makes it susceptible to crashes. I've seen this happen in practice when rebuilding libvirt/focal on a system with 48 cores. The race window is wide enough that the crash is nearly 100% reproducible.

[Test Case]
user@host:~$ cat foo/Makefile
all: ../foo/foo.out foo.out

%.out:
    dtrace -o $@ -G -s /dev/null

clean:
    rm -f foo.out
user@host:~$ cd foo
user@host:~/foo$ make
dtrace -o ../foo/foo.out -G -s /dev/null
user@host:~/foo$ make clean
rm -f foo.out
user@host:~/foo$ make -j2
dtrace -o ../foo/foo.out -G -s /dev/null
dtrace -o foo.out -G -s /dev/null
Traceback (most recent call last):
  File "/usr/bin/dtrace", line 445, in <module>
    sys.exit(main())
  File "/usr/bin/dtrace", line 432, in main
    os.remove(fname)
FileNotFoundError: [Errno 2] No such file or directory: 'foo.out.dtrace-temp.c'
make: *** [Makefile:4: foo.out] Error 1

[Fix]
https://sourceware.org/git/?p=systemtap.git;a=commit;h=0de9020c970bceda73e32bbd169c12e7579f21ec

[What could go wrong]
It is possible that a race still exists, but the fix should be an overall improvement. Perhaps it is possible that a funky filename string could cause UTF-8 encoding to fail resulting in a crash - though it would seem that such input wouldn't be a valid path name and dtrace would crash anyway.

See original description
Revision history for this message
Frank Ch. Eigler (fche) wrote :

Suggest adding commit cfabd38cfdd75e also, which makes it more robust toward identical concurrent jobs.

Bug Watch Updater (bug-watch-updater)
Changed in systemtap (Debian):
status: Unknown → New
Revision history for this message
dann frazier (dannf) wrote :

Ah, thanks Frank - and I see the follow-up fix 7cf95304f8e9 as well. Since 22.04 is nearing release, I think I'll upload the bare minimum 0de9020c970b, as it is sufficient to fix the problem we're seeing. Once 22.10 opens up for devel let's reconsider applying the others (or go back to syncing with Debian if they've addressed it by then).

Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package systemtap - 4.6-1ubuntu1

---------------
systemtap (4.6-1ubuntu1) jammy; urgency=medium

  * d/p/PR28923-dtrace-Use-hash-based-scheme-for-predictable.patch:
    Cherry-pick fix from upstream to avoid a race condition due to
    use of a predictable temporary file name. LP: #1962785.

 -- dann frazier <email address hidden> Mon, 07 Mar 2022 13:50:07 -0700

Changed in systemtap (Ubuntu):
status: New → Fix Released
Bug Watch Updater (bug-watch-updater)
Changed in systemtap (Debian):
status: New → Fix Released
dann frazier (dannf)
Changed in systemtap (Ubuntu Impish):
status: New → In Progress
Changed in systemtap (Ubuntu Focal):
status: New → In Progress
assignee: nobody → dann frazier (dannf)
Changed in systemtap (Ubuntu Impish):
assignee: nobody → dann frazier (dannf)
dann frazier (dannf)
description: updated
description: updated
Revision history for this message
Brian Murray (brian-murray) wrote : Please test proposed package

Hello dann, or anyone else affected,

Accepted systemtap into impish-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/systemtap/4.4-2ubuntu0.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-impish to verification-done-impish. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-impish. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

Changed in systemtap (Ubuntu Impish):
status: In Progress → Fix Committed
tags: added: verification-needed verification-needed-impish
Changed in systemtap (Ubuntu Focal):
status: In Progress → Fix Committed
tags: added: verification-needed-focal
Revision history for this message
Brian Murray (brian-murray) wrote :

Hello dann, or anyone else affected,

Accepted systemtap into focal-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/systemtap/4.2-3ubuntu0.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-focal to verification-done-focal. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-focal. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

Revision history for this message
dann frazier (dannf) wrote :

= impish verification =
Setting up systemtap-sdt-dev (4.4-2ubuntu0.1) ...
Processing triggers for man-db (2.9.4-2) ...
ubuntu@dannf-lp1962785-impish:~/foo$ make -j2
dtrace -o ../foo/foo.out -G -s /dev/null
dtrace -o foo.out -G -s /dev/null

= focal verification =
Setting up systemtap-sdt-dev (4.2-3ubuntu0.1) ...
Processing triggers for man-db (2.9.1-1) ...
ubuntu@dannf-lp1962785-focal:~/foo$ make -j2
dtrace -o ../foo/foo.out -G -s /dev/null
dtrace -o foo.out -G -s /dev/null

tags: added: verification-done verification-done-focal verification-done-impish
removed: verification-needed verification-needed-focal verification-needed-impish
Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package systemtap - 4.4-2ubuntu0.1

---------------
systemtap (4.4-2ubuntu0.1) impish; urgency=medium

  * d/p/dtrace-predictable-tmp-file.patch:
    Cherry-pick fix from upstream to avoid a race condition due to
    use of a predictable temporary file name. (LP: #1962785).

 -- dann frazier <email address hidden> Wed, 16 Mar 2022 13:47:08 -0600

Changed in systemtap (Ubuntu Impish):
status: Fix Committed → Fix Released
Revision history for this message
Robie Basak (racb) wrote : Update Released

The verification of the Stable Release Update for systemtap has completed successfully and the package is now being released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package systemtap - 4.2-3ubuntu0.1

---------------
systemtap (4.2-3ubuntu0.1) focal; urgency=medium

  * d/p/dtrace-predictable-tmp-file.patch:
    Cherry-pick fix from upstream to avoid a race condition due to
    use of a predictable temporary file name. (LP: #1962785).

 -- dann frazier <email address hidden> Wed, 16 Mar 2022 14:04:27 -0600

Changed in systemtap (Ubuntu Focal):
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.
Subscribing...

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.