Comment 19 for bug 1862053

Ok, after debugging, this looks to be a bug in rs6000_legitimate_address_p().
At the beginning of LRA, we have the following insn:

(insn 520 67 71 5 (set (mem:V16QI (and:DI (reg/f:DI 110 sfp)
                (const_int -16 [0xfffffffffffffff0])) [0 MEM <vector(16) char> [(char *)&a + 16B]+0 S16 A128])
        (reg:V16QI 303 [ vect__2.13 ])) "bug.i":10:10 1103 {vsx_movv16qi_64bit}
     (expr_list:REG_DEAD (reg:V16QI 303 [ vect__2.13 ])
        (nil)))

lra_eliminate() then converts that to:

(insn 520 67 71 5 (set (mem:V16QI (and:DI (plus:DI (reg/f:DI 110 sfp)
                    (const_int 112 [0x70]))
                (const_int -16 [0xfffffffffffffff0])) [0 MEM <vector(16) char> [(char *)&a + 16B]+0 S16 A128])
        (reg:V16QI 303 [ vect__2.13 ])) "bug.i":10:10 1103 {vsx_movv16qi_64bit}
     (expr_list:REG_DEAD (reg:V16QI 303 [ vect__2.13 ])
        (nil)))

which isn't valid and needs fixing...which is fine. process_address() then gives us:

(insn 520 67 71 5 (set (mem:V16QI (and:DI (reg:DI 633)
                (const_int -16 [0xfffffffffffffff0])) [0 MEM <vector(16) char> [(char *)&a + 16B]+0 S16 A128])
        (reg:V16QI 303 [ vect__2.13 ])) "bug.i":10:10 1103 {vsx_movv16qi_64bit}
     (expr_list:REG_DEAD (reg:V16QI 303 [ vect__2.13 ])
        (nil)))

...with pseudo 633 being set to plus:DI (reg/f:DI 110 sfp) (const_int 112 [0x70])
which is also good. The bug comes when we now pass insn 520 to rs6000_legitimate_address_p() and we hit the following code:

  /* If this is an unaligned stvx/ldvx type address, discard the outer AND. */
  if (VECTOR_MEM_ALTIVEC_P (mode)
      && GET_CODE (x) == AND
      && CONST_INT_P (XEXP (x, 1))
      && INTVAL (XEXP (x, 1)) == -16)
    x = XEXP (x, 0);

Our mode is V16QI, so we should execute this code, but we don't. The problem is that VECTOR_MEM_ALTIVEC_P(mode) checks for "rs6000_vector_unit[(MODE)] == VECTOR_ALTIVEC" and V16QI mode returns VECTOR_VSX, so we fail to fall into the code above and end up telling LRA this isn't a valid address. LRA then attempts to spill this again and again and...

The following patch fixes the bug for me and I'm regtesting it now:

--- a/gcc/config/rs6000/rs6000.c
+++ b/gcc/config/rs6000/rs6000.c
@@ -8808,7 +8808,7 @@ rs6000_legitimate_address_p (machine_mode mode, rtx x, bool reg_ok_strict)
   bool quad_offset_p = mode_supports_dq_form (mode);

   /* If this is an unaligned stvx/ldvx type address, discard the outer AND. */
- if (VECTOR_MEM_ALTIVEC_P (mode)
+ if (VECTOR_MEM_ALTIVEC_OR_VSX_P (mode)
       && GET_CODE (x) == AND
       && CONST_INT_P (XEXP (x, 1))
       && INTVAL (XEXP (x, 1)) == -16)