[v4,2/7] gdb: Start supporting AVX instruction

Message ID 20240923151541.616723-4-guinevere@redhat.com
State New
Headers
Series Support for recording some AVX instructions |

Checks

Context Check Description
linaro-tcwg-bot/tcwg_gdb_build--master-aarch64 success Build passed
linaro-tcwg-bot/tcwg_gdb_build--master-arm success Build passed
linaro-tcwg-bot/tcwg_gdb_check--master-aarch64 success Test passed
linaro-tcwg-bot/tcwg_gdb_check--master-arm success Test passed

Commit Message

Guinevere Larsen Sept. 23, 2024, 3:15 p.m. UTC
  This patch introduces the information needed to properly identify the
VEX prefix, used to signal an AVX and AVX2 instruction, and introduces
a helper function to handle all AVX instruction, instead of adding to
the 3000 line long recording function.

This new function will temporarily set the current thread as "not
executing" so that it can read from pseudo registers as we record, since
most AVX/AVX2 instructions would benefit from recording ymm registers.

The new helper also handles unsupported instructions so that the largest
part of the i386_process_record doesn't have to be shifted by 2 spaces,
which made an unreadably big patch file.

The only expected difference to the end user added by this patch is a
small change to the unsupported message. This patch also updates the
test gdb.reverse/step-precsave.exp, by recognizing the new output.

As a note for the future, we don't handle xmm16-31 and ymm16-31 because
those require the EVEX prefix, meaning avx512 support.
---
 gdb/amd64-tdep.c                            |   3 +-
 gdb/i386-tdep.c                             | 103 +++++++++++++++++++-
 gdb/i386-tdep.h                             |   1 +
 gdb/testsuite/gdb.reverse/step-precsave.exp |   2 +-
 4 files changed, 105 insertions(+), 4 deletions(-)
  

Comments

Tom Tromey Oct. 4, 2024, 7:56 p.m. UTC | #1
>>>>> "Guinevere" == Guinevere Larsen <guinevere@redhat.com> writes:

Guinevere> +  /* These are used by VEX and XOP prefixes.  */
Guinevere> +  uint8_t map_select;
Guinevere> +  uint8_t vvvv;
Guinevere> +  uint8_t pp;
Guinevere> +  uint8_t l;

I assume these are standard names for these things.
Otherwise they should probably be more clear.

Guinevere> +  /* Since we are reading pseudo registers, we need to tell GDB that it is
Guinevere> +     safe to do so, by saying we aren't _really_ running the inferior right
Guinevere> +     now.  */
Guinevere> +  SCOPE_EXIT { inferior_thread ()->set_executing (true); };
Guinevere> +  inferior_thread () -> set_executing (false);

Is this needed given patch #1, or vice versa?

Tom
  
Guinevere Larsen Oct. 7, 2024, 7:08 p.m. UTC | #2
On 10/4/24 4:56 PM, Tom Tromey wrote:
>>>>>> "Guinevere" == Guinevere Larsen <guinevere@redhat.com> writes:
> Guinevere> +  /* These are used by VEX and XOP prefixes.  */
> Guinevere> +  uint8_t map_select;
> Guinevere> +  uint8_t vvvv;
> Guinevere> +  uint8_t pp;
> Guinevere> +  uint8_t l;
>
> I assume these are standard names for these things.
> Otherwise they should probably be more clear.
yeah, that's how the manual shows the fields of the prefix. While they 
have more descriptive names at the start of the manual, every place 
where they're relevant, it's always using these unreadable 
abbreviations, so I figured it would be easiest to debug/support them if 
I copied directly from the manual.
>
> Guinevere> +  /* Since we are reading pseudo registers, we need to tell GDB that it is
> Guinevere> +     safe to do so, by saying we aren't _really_ running the inferior right
> Guinevere> +     now.  */
> Guinevere> +  SCOPE_EXIT { inferior_thread ()->set_executing (true); };
> Guinevere> +  inferior_thread () -> set_executing (false);
>
> Is this needed given patch #1, or vice versa?

Yes, they are both needed. The change for patch 1 relates to replaying 
the inferior, this is related to recording. We can't have the general 
approach like patch 1 because there are times when recording is actually 
executing the inferior instead of fiddling with registers/memory, so I 
figured marking that for the span of this specific function we aren't 
executing was the next best thing - since at this point we are 
guaranteed to have the inferior stopped.
  

Patch

diff --git a/gdb/amd64-tdep.c b/gdb/amd64-tdep.c
index df6b882a3fb..532337c8f84 100644
--- a/gdb/amd64-tdep.c
+++ b/gdb/amd64-tdep.c
@@ -3134,7 +3134,8 @@  static const int amd64_record_regmap[] =
   AMD64_R8_REGNUM, AMD64_R9_REGNUM, AMD64_R10_REGNUM, AMD64_R11_REGNUM,
   AMD64_R12_REGNUM, AMD64_R13_REGNUM, AMD64_R14_REGNUM, AMD64_R15_REGNUM,
   AMD64_RIP_REGNUM, AMD64_EFLAGS_REGNUM, AMD64_CS_REGNUM, AMD64_SS_REGNUM,
-  AMD64_DS_REGNUM, AMD64_ES_REGNUM, AMD64_FS_REGNUM, AMD64_GS_REGNUM
+  AMD64_DS_REGNUM, AMD64_ES_REGNUM, AMD64_FS_REGNUM, AMD64_GS_REGNUM,
+  AMD64_XMM0_REGNUM
 };
 
 /* Implement the "in_indirect_branch_thunk" gdbarch function.  */
diff --git a/gdb/i386-tdep.c b/gdb/i386-tdep.c
index 625ffe5a306..11fc2f81107 100644
--- a/gdb/i386-tdep.c
+++ b/gdb/i386-tdep.c
@@ -4636,6 +4636,12 @@  struct i386_record_s
   int rip_offset;
   int popl_esp_hack;
   const int *regmap;
+
+  /* These are used by VEX and XOP prefixes.  */
+  uint8_t map_select;
+  uint8_t vvvv;
+  uint8_t pp;
+  uint8_t l;
 };
 
 /* Parse the "modrm" part of the memory address irp->addr points at.
@@ -4974,6 +4980,43 @@  static int i386_record_floats (struct gdbarch *gdbarch,
   return 0;
 }
 
+/* i386_process_record helper to deal with instructions that start
+   with VEX prefix.  */
+
+static int
+i386_record_vex (struct i386_record_s *ir, uint8_t vex_w, uint8_t vex_r,
+		 int opcode, struct gdbarch *gdbarch)
+{
+  /* We need this to find YMM (and once AVX-512 is supported, ZMM) registers.
+     We should always save the largest available register, since an
+     instruction that handles a smaller reg may zero out the higher bits,
+     so we must have them saved.  */
+  i386_gdbarch_tdep *tdep = gdbarch_tdep<i386_gdbarch_tdep> (gdbarch);
+
+  /* Since we are reading pseudo registers, we need to tell GDB that it is
+     safe to do so, by saying we aren't _really_ running the inferior right
+     now.  */
+  SCOPE_EXIT { inferior_thread ()->set_executing (true); };
+  inferior_thread () -> set_executing (false);
+
+  switch (opcode)
+    {
+    default:
+      gdb_printf (gdb_stderr,
+		  _("Process record does not support VEX instruction 0x%02x "
+		    "at address %s.\n"),
+		  (unsigned int) (opcode),
+		  paddress (gdbarch, ir->orig_addr));
+      return -1;
+    }
+
+  record_full_arch_list_add_reg (ir->regcache, ir->regmap[X86_RECORD_REIP_REGNUM]);
+  if (record_full_arch_list_add_end ())
+    return -1;
+
+  return 0;
+}
+
 /* Parse the current instruction, and record the values of the
    registers and memory that will be changed by the current
    instruction.  Returns -1 if something goes wrong, 0 otherwise.  */
@@ -4996,6 +5039,7 @@  i386_process_record (struct gdbarch *gdbarch, struct regcache *regcache,
   i386_gdbarch_tdep *tdep = gdbarch_tdep<i386_gdbarch_tdep> (gdbarch);
   uint8_t rex_w = -1;
   uint8_t rex_r = 0;
+  bool vex_prefix = false;
 
   memset (&ir, 0, sizeof (struct i386_record_s));
   ir.regcache = regcache;
@@ -5081,6 +5125,53 @@  i386_process_record (struct gdbarch *gdbarch, struct regcache *regcache,
 	  else					/* 32 bit target */
 	    goto out_prefixes;
 	  break;
+	case 0xc4:	/* 3-byte VEX prefixes (for AVX/AVX2 instructions).  */
+	  {
+	    /* The first byte just identifies the VEX prefix.  Data is stored
+	       on the following 2 bytes.  */
+	    uint8_t byte;
+	    if (record_read_memory (gdbarch, ir.addr, &byte, 1))
+	      return -1;
+	    ir.addr++;
+
+	    rex_r = !((byte >> 7) & 0x1);
+	    ir.rex_x = !((byte >> 6) & 0x1);
+	    ir.rex_b = !((byte >> 5) & 0x1);
+	    ir.map_select = byte & 0x1f;
+	    /* Collect the last byte of the prefix.  */
+	    if (record_read_memory (gdbarch, ir.addr, &byte, 1))
+	      return -1;
+	    ir.addr++;
+	    rex_w = (byte >> 7) & 0x1;
+	    ir.vvvv = (~(byte >> 3) & 0xf);
+	    ir.l = (byte >> 2) & 0x1;
+	    ir.pp = byte & 0x3;
+	    vex_prefix = true;
+
+	    break;
+	  }
+	case 0xc5:	/* 2-byte VEX prefix for AVX/AVX2 instructions.  */
+	  {
+	    /* The first byte just identifies the VEX prefix.  Data is stored
+	       on the following 2 bytes.  */
+	    uint8_t byte;
+	    if (record_read_memory (gdbarch, ir.addr, &byte, 1))
+	      return -1;
+	    ir.addr++;
+
+	    /* On the 2-byte versions, these are pre-defined.  */
+	    ir.rex_x = 0;
+	    ir.rex_b = 0;
+	    rex_w = 0;
+	    ir.map_select = 1;
+
+	    rex_r = !((byte >> 7) & 0x1);
+	    ir.vvvv = (~(byte >> 3) & 0xf);
+	    ir.l = (byte >> 2) & 0x1;
+	    ir.pp = byte & 0x3;
+	    vex_prefix = true;
+	    break;
+	  }
 	default:
 	  goto out_prefixes;
 	  break;
@@ -5103,6 +5194,12 @@  i386_process_record (struct gdbarch *gdbarch, struct regcache *regcache,
 
   /* Now check op code.  */
   opcode = (uint32_t) opcode8;
+  if (vex_prefix)
+    {
+      /* If we found the VEX prefix, i386 will either record or warn that
+	 the instruction isn't supported, so we can return the VEX result.  */
+      return i386_record_vex (&ir, rex_w, rex_r, opcode, gdbarch);
+    }
  reswitch:
   switch (opcode)
     {
@@ -8087,9 +8184,11 @@  static const int i386_record_regmap[] =
 {
   I386_EAX_REGNUM, I386_ECX_REGNUM, I386_EDX_REGNUM, I386_EBX_REGNUM,
   I386_ESP_REGNUM, I386_EBP_REGNUM, I386_ESI_REGNUM, I386_EDI_REGNUM,
-  0, 0, 0, 0, 0, 0, 0, 0,
+  0, 0, 0, 0,
+  0, 0, 0, 0,
   I386_EIP_REGNUM, I386_EFLAGS_REGNUM, I386_CS_REGNUM, I386_SS_REGNUM,
-  I386_DS_REGNUM, I386_ES_REGNUM, I386_FS_REGNUM, I386_GS_REGNUM
+  I386_DS_REGNUM, I386_ES_REGNUM, I386_FS_REGNUM, I386_GS_REGNUM,
+  /* xmm0_regnum */ 0
 };
 
 /* Check that the given address appears suitable for a fast
diff --git a/gdb/i386-tdep.h b/gdb/i386-tdep.h
index a85e0a984a0..ad3951bde05 100644
--- a/gdb/i386-tdep.h
+++ b/gdb/i386-tdep.h
@@ -339,6 +339,7 @@  enum record_i386_regnum
   X86_RECORD_ES_REGNUM,
   X86_RECORD_FS_REGNUM,
   X86_RECORD_GS_REGNUM,
+  X86_RECORD_XMM0_REGNUM,
 };
 
 #define I386_NUM_GREGS	16
diff --git a/gdb/testsuite/gdb.reverse/step-precsave.exp b/gdb/testsuite/gdb.reverse/step-precsave.exp
index 9ae67b37f3f..9d5ff988faa 100644
--- a/gdb/testsuite/gdb.reverse/step-precsave.exp
+++ b/gdb/testsuite/gdb.reverse/step-precsave.exp
@@ -46,7 +46,7 @@  with_timeout_factor 20 {
 	-re -wrap "Breakpoint .* end of main .*" {
 	    pass $gdb_test_name
 	}
-	-re -wrap "Process record does not support instruction 0xc5 at.*" {
+	-re -wrap "Process record does not support VEX instruction.*" {
 	    kfail "record/23188" $gdb_test_name
 	}
 	-re -wrap "Process record does not support instruction 0xfae64 at.*" {