[AIE2P] Support wide vector postinc 2D/3D, and offset load/store #323

abnikant · 2025-01-31T07:20:20Z

Enable combine_load_store_increment, combine_offset_load_store_ptradd and combine_offset_load_store_share_ptradd
Support wide vector POSTINC, POSTINC_2D, POSTINC_3D load and store.
Fixup , offset opcode.
Enable fifo combined load/store incr and offset load/store for 512-bits.
Add tests for combining and load/store.

andcarminati · 2025-01-31T08:08:52Z

llvm/test/CodeGen/AIE/aie2p/combine-loads-stores.mir

+# See https://llvm.org/LICENSE.txt for license information.
+# SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+#
+# (c) Copyright 2023-2024 Advanced Micro Devices, Inc. or its affiliates


andcarminati · 2025-01-31T08:09:10Z

llvm/lib/Target/AIE/AIECombinerHelper.cpp

@@ -58,7 +58,10 @@ MachineInstr *findPreIncMatch(MachineInstr &MemI, MachineRegisterInfo &MRI,
                              const AIEBaseInstrInfo &TII) {
  // This is currently done with patterns in instruction selection.
  // No need to do it here.
-  if (MRI.getType(MemI.getOperand(0).getReg()).getSizeInBits() >= 1024)
+  MachineFunction &MF = *MemI.getMF();
+  bool isAIE2 = MF.getTarget().getTargetTriple().isAIE2();


nit: const

andcarminati · 2025-01-31T08:09:25Z

llvm/lib/Target/AIE/AIECombinerHelper.cpp

@@ -320,9 +323,11 @@ MachineInstr *findPostIncMatch(MachineInstr &MemI, MachineRegisterInfo &MRI,
                               const AIEBaseInstrInfo &TII) {
  if (!EnablePostIncCombine)
    return nullptr;
-  if (MRI.getType(MemI.getOperand(0).getReg()).getSizeInBits() >= 1024)
+  MachineFunction &MF = *MemI.getMF();
+  bool isAIE2 = MF.getTarget().getTargetTriple().isAIE2();


nit: const

andcarminati · 2025-01-31T08:19:41Z

llvm/test/CodeGen/AIE/aie2p/combine-loads-stores.mir

+  ; CHECK-NEXT: bb.1:
+  ; CHECK-NEXT:   successors: %bb.1(0x80000000)
+  ; CHECK-NEXT: {{  $}}
+  ; CHECK-NEXT:   [[PHI:%[0-9]+]]:_(p0) = G_PHI [[COPY]](p0), %bb.0, %2(p0), %bb.1


Interesting, here we have just %2 instead of the regex.

khallouh · 2025-01-31T10:49:18Z

llvm/lib/Target/AIE/aie2p/AIE2PInstructionSelector.cpp

+                                        : AIE2P::VLDA_dmx_lda_fifohl_idx;
+        return {ISelOpcode, FitsImmediateRange,
+                /*OffsetOpcode=*/AIE2P::VLDA_dmx_lda_fifohl_idx_imm};
+      }
      llvm_unreachable("512-bit vector type must be in AccRegBank or VRegBank");


Nit: ...or FifoRegBank

khallouh · 2025-01-31T10:49:40Z

llvm/lib/Target/AIE/aie2p/AIE2PInstrInfo.cpp

@@ -252,6 +252,11 @@ unsigned AIE2PInstrInfo::getOffsetMemOpcode(unsigned BaseMemOpcode) const {
  llvm_unreachable("not a generic load/store");
 }

+bool AIE2PInstrInfo::isOffsetMemOpcode(unsigned Opcode) const {


Nit: isGenericMemOffsetOpcode

updated the name.

martien-de-jong · 2025-01-31T12:58:51Z

llvm/lib/Target/AIE/AIECombinerHelper.cpp

+  MachineFunction &MF = *MemI.getMF();
+  const bool isAIE2 = MF.getTarget().getTargetTriple().isAIE2();
+  if (isAIE2 &&
+      MRI.getType(MemI.getOperand(0).getReg()).getSizeInBits() >= 1024)


Shouldn't there be a corresponding size check for AIE2P?

yes, I have added a check but I don't expect the vector size to be greater that 2048-bits.

martien-de-jong · 2025-01-31T13:09:11Z

llvm/lib/Target/AIE/aie2p/AIE2PInstrInfo.cpp

@@ -253,6 +253,11 @@ unsigned AIE2PInstrInfo::getOffsetMemOpcode(unsigned BaseMemOpcode) const {
  llvm_unreachable("not a generic load/store");
 }

+bool AIE2PInstrInfo::isGenericOffsetMemOpcode(unsigned Opcode) const {
+  return ((Opcode == AIE2P::G_AIE_OFFSET_STORE) ||
+          (Opcode == AIE2P::G_AIE_OFFSET_LOAD));


What about G_AIE_OFFSET_SEXTLOAD / G_AIE_OFFSET_ZEXTLOAD ?

F-Stuckmann · 2025-01-31T15:41:44Z

llvm/test/CodeGen/AIE/aie2p/fifo-loads.ll

-; CHECK-NEXT:    vst lfl0, [p1, #0] // Delay Slot 4
-; CHECK-NEXT:    vst lfh0, [p1, #64] // Delay Slot 3
+; CHECK-NEXT:    nop
+; CHECK-NEXT:    nop


why can the vst not work on the fifo registers as before?

not sure I understood the question but if your question is why the test is updated ? since MIR is updated after the load/store combiner is enabled, it uses G_AIE_POSTINC_STORE.

do we have a vst post increment that works on fifo regs? Because that would be preferable in this situation

andcarminati · 2025-01-31T16:44:52Z

llvm/test/CodeGen/AIE/aie2p/ldst-fifo-stores.ll

-; CHECK-NEXT:    vst sfl, [p1, #0] // Delay Slot 4
-; CHECK-NEXT:    vst sfh, [p1, #64] // Delay Slot 3
+; CHECK-NEXT:    nop
+; CHECK-NEXT:    vmov x1, sfh


I guess @F-Stuckmann is referring to this case. Now we are storing using vector registers.

we do have the post-increment for fifo-regs , refer to the test case inst-select-vector-pre-post-increment.mir. The problem with the above case is register bank assignment not assigning fiforegbank.

%7:ptrregbank(p0), %8:fiforegbank(<32 x s32>), %9:gprregbank(s32) = G_INTRINSIC_W_SIDE_EFFECTS intrinsic(@llvm.aie2p.fifo.ld.fill), %4(p0), %5(<32 x s32>), %6(s32)
%10:vregbank(<32 x s32>) = COPY %8(<32 x s32>)
%3:ptrregbank(p0) = G_AIE_POSTINC_STORE %10(<32 x s32>), %1, %2(s20) :: (store (<32 x s32>) into %ir.s)

if I use %8 in G_AIE_POSTINC_STORE , then it generates post-inc with fifo. We are missing G_AIE_POSTINC_STORE opcode in regbank-select.

abnikant requested review from abhinay-anubola, andcarminati, F-Stuckmann, gbossu, katerynamuts, khallouh, konstantinschwarz, martien-de-jong, niwinanto, SagarMaheshwari99 and stephenneuendorffer as code owners January 31, 2025 07:20

andcarminati reviewed Jan 31, 2025

View reviewed changes

khallouh reviewed Jan 31, 2025

View reviewed changes

abnikant force-pushed the aie2p.wide.ld.st.incr.offset branch 2 times, most recently from 9d70b1d to a9baa70 Compare January 31, 2025 11:36

martien-de-jong reviewed Jan 31, 2025

View reviewed changes

abnikant force-pushed the aie2p.wide.ld.st.incr.offset branch from a9baa70 to 48a5ce8 Compare January 31, 2025 14:46

F-Stuckmann reviewed Jan 31, 2025

View reviewed changes

Abnikant Singh added 2 commits January 31, 2025 21:47

[AIE2P] Enable post-pre incr and offset load combine

f1b9aa2

[AIE2P] Support postinc 2D/3D, and offset load/store

c82ccd3

abnikant force-pushed the aie2p.wide.ld.st.incr.offset branch from 48a5ce8 to c82ccd3 Compare January 31, 2025 16:21

andcarminati reviewed Jan 31, 2025

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[AIE2P] Support wide vector postinc 2D/3D, and offset load/store #323

[AIE2P] Support wide vector postinc 2D/3D, and offset load/store #323

abnikant commented Jan 31, 2025

andcarminati Jan 31, 2025

andcarminati Jan 31, 2025

abnikant Jan 31, 2025

andcarminati Jan 31, 2025

andcarminati Jan 31, 2025

khallouh Jan 31, 2025

abnikant Jan 31, 2025

khallouh Jan 31, 2025

abnikant Jan 31, 2025 •

edited

Loading

martien-de-jong Jan 31, 2025

abnikant Jan 31, 2025

martien-de-jong Jan 31, 2025

abnikant Jan 31, 2025

F-Stuckmann Jan 31, 2025

abnikant Jan 31, 2025

F-Stuckmann Jan 31, 2025

andcarminati Jan 31, 2025 •

edited

Loading

abnikant Jan 31, 2025 •

edited

Loading

[AIE2P] Support wide vector postinc 2D/3D, and offset load/store #323

Are you sure you want to change the base?

[AIE2P] Support wide vector postinc 2D/3D, and offset load/store #323

Conversation

abnikant commented Jan 31, 2025

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

abnikant Jan 31, 2025 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

andcarminati Jan 31, 2025 • edited Loading

Choose a reason for hiding this comment

abnikant Jan 31, 2025 • edited Loading

Choose a reason for hiding this comment

abnikant Jan 31, 2025 •

edited

Loading

andcarminati Jan 31, 2025 •

edited

Loading

abnikant Jan 31, 2025 •

edited

Loading