ruby2.7: backport upstream fix to "hash modified during iteration"

I have rebuilt Ruby after applying the upstream-provided patch and no longer have issues with sonic-pi.

Thanks for reporting this bug and trying to make it better.

This bug seems valid and the fix is straightforward. Could you please provide a reproducer for this bug? Detailed steps on how to reproduce the crash locally. This is required if we want to release an update to Ubuntu Focal.

As shown on Ruby's issue tracker [1], the following is enough to reproduce Ruby's issue:

h = {a:1, b:2, c:3, d:4, e:5, f:6, g:7, h:8}
h.each{|k,v| GC.start; h.delete(k)}

Saved in `test.rb` and launched via command line `ruby test.rb` this should crash with the following:

Traceback (most recent call last):
 1: from test.rb:2:in `<main>'
test.rb:2:in `each': ret: 2, hash modified during iteration (RuntimeError)

After patching Ruby I no longer experience a RuntimeError.

I could also provide a reproducer for sonic-pi crash, but it would have to be interactive, as sonic-pi in Ubuntu Focal does not yet have a CLI.

[1] https://bugs.ruby-lang.org/issues/16503

Thanks, Andrius. I was able to reproduce the bug locally.

It took a bit of digging to confirm that this is fixed in Jammy ruby3.0 - this is the relevant upstream commit: https://github.com/ruby/ruby/commit/350dafd56a9cff58d36303aeb7515ab41c5dbbb3

I also found https://github.com/ruby/ruby/commit/ab6f78bc926f6fc12dc8d7846056fc9c04d63ead and https://github.com/ruby/ruby/commit/897d4e31b0e5b29405480ed3236bc8c5c6cac3fa upstream. They look like they relate to a similar area. Has anyone checked if these interact with the proposed fix here?

I would also like the Test Plan to incorporate how we are verifying that other typical Ruby use cases aren't being broken by this change. Maybe there's a test suite that covers that? If so that's fine and no further manual steps are needed, but please make sure to check and document the answer.

Apart from that this is fine for SRU accept.

Thanks for taking a look Robie. The upstream patches you mentioned in comment #6 are indeed touching the same code, however, they are fixing another issue, not the one I am trying to address here. This is the upstream bug they are fixing:

https://bugs.ruby-lang.org/issues/16676

So yes, it would be good to also have this fix but no user reported this issue to us yet. Ideally, we should update ruby2.7 version 2.7.0 that we have in Focal to version 2.7.8 (latest upstream release of this track) which contains those fixes and more, but we do not do microrelease updates to ruby2.7. As I am trying to fix this single issue reported by our user I believe the single patch proposed in this SRU is enough.

About testing (comment #7), as you imagined, it is covered by the upstream test suite. There is test/ruby/test_hash.rb which contains multiple tests covering hash usage. All the tests passed during build and autopkgtest time for me. I do no think we need to do any extra manual test.

Hello Andrius, or anyone else affected,

Accepted ruby2.7 into focal-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/ruby2.7/2.7.0-5ubuntu1.11 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-focal to verification-done-focal. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-focal. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

Thanks a lot for taking care of this issue!

I have enabled the focal-proposed and installed libruby2.7 and ruby2.7 2.7.0-5ubuntu1.11 from there on my Ubuntu 20.04 machine. I tested both the issue with sonic-pi (which made me discover this bug initially) and the reproducer provided in comment #3. Both tests pass.

In sonic-pi, I can stop the playing loops successfully now. First of all, no exceptions are being thrown. Second, I notice that loops stop (a message about stopped loop appears, previously it did not). Third, sound does not disappear anymore - previously the program used to possibly run out of usable threads and new loops were not playing anymore.

The reproducer from comment #3 no longer fails with exception.

All autopkgtests for the newly accepted ruby2.7 (2.7.0-5ubuntu1.11) for focal have finished running.
The following regressions have been reported in tests triggered by the package:

pcs/0.10.4-3 (armhf)
ruby-fast-stemmer/1.0.2-2build1 (armhf)

Please visit the excuses page listed below and investigate the failures, proceeding afterwards as per the StableReleaseUpdates policy regarding autopkgtest regressions [1].

https://people.canonical.com/~ubuntu-archive/proposed-migration/focal/update_excuses.html#ruby2.7

[1] https://wiki.ubuntu.com/StableReleaseUpdates#Autopkgtest_Regressions

Thank you!

The reverse dependencies' tests got fixed after a retry.

This bug was fixed in the package ruby2.7 - 2.7.0-5ubuntu1.11

---------------
ruby2.7 (2.7.0-5ubuntu1.11) focal; urgency=medium

  * d/p/0026-reload-AR-table-body-for-transient-heap.patch: Fix hash iteration
    (LP: #2018215).

 -- Lucas Kanashiro <email address hidden> Wed, 03 May 2023 04:51:06 -0300

The verification of the Stable Release Update for ruby2.7 has completed successfully and the package is now being released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.