[v4,2/3] Do not mistreat instructions as cmpxchg8b

Message ID 20181006001539.32414-2-pierre.marsais@lse.epita.fr
State New, archived
Headers

Commit Message

Pierre Marsais Oct. 6, 2018, 12:15 a.m. UTC
  All x86 instructions starting with opcode 0x0f7c where considered as
cmpxchg8b if ir.mod == 3, regardless of ir.reg. However, there are some
instructions (such as xsavec) sharing the same opcode, but with
different ir.reg.

This change throws an error when recording on unsupported instructions
instead of considering them as cmpxchg8b.

gdb/ChangeLog:

2018-10-05  Pierre Marsais <pierre.marsais@lse.epita.fr>

	* i386-tdep.c: (i386_process_record): Improve decoding of
	instructions starting with 0x0f7c.
---
 gdb/i386-tdep.c | 76 ++++++++++++++++++++++++++++---------------------
 1 file changed, 43 insertions(+), 33 deletions(-)
  

Comments

Metzger, Markus T Oct. 11, 2018, 11:56 a.m. UTC | #1
Hello Pierre,

> +      switch (ir.reg) {
> +        case 1: /* cmpxchg8b */
> +          if (ir.mod == 3)
> +            {
> +              ir.addr -= 2;
> +              opcode = opcode << 8 | ir.modrm;
> +              goto no_support;
> +            }

We'd also need to check that neither 66 nor f3 prefixes are used.


> +        case 6: /* rdrand */
> +        case 7: /* rdseed */
> +          if (ir.mod != 3)
> +            {
> +              /* We don't handle this particular instruction yet.  */
> +              ir.addr -= 2;
> +              opcode = opcode << 8 | ir.modrm;
> +              goto no_support;
> +            }

We'd also need to check that f3 prefix is not used.


Regards,
Markus.

Intel Deutschland GmbH
Registered Address: Am Campeon 10-12, 85579 Neubiberg, Germany
Tel: +49 89 99 8853-0, www.intel.de
Managing Directors: Christin Eisenschmid, Christian Lamprechter
Chairperson of the Supervisory Board: Nicole Lau
Registered Office: Munich
Commercial Register: Amtsgericht Muenchen HRB 186928
  

Patch

diff --git a/gdb/i386-tdep.c b/gdb/i386-tdep.c
index a9fe290307..90c78e0bbc 100644
--- a/gdb/i386-tdep.c
+++ b/gdb/i386-tdep.c
@@ -5477,39 +5477,49 @@  i386_process_record (struct gdbarch *gdbarch, struct regcache *regcache,
     case 0x0fc7:    /* cmpxchg8b / rdrand / rdseed */
       if (i386_record_modrm (&ir))
 	return -1;
-      if (ir.mod == 3)
-	{
-	  /* rdrand and rdseed use the 3 bits of the REG field of ModR/M as
-	     an extended opcode.  rdrand has bits 110 (/6) and rdseed
-	     has bits 111 (/7).  */
-	  if (ir.reg == 6 || ir.reg == 7)
-	    {
-	      /* The storage register is described by the 3 R/M bits, but the
-		 REX.B prefix may be used to give access to registers
-		 R8~R15.  In this case ir.rex_b + R/M will give us the register
-		 in the range R8~R15.
-
-		 REX.W may also be used to access 64-bit registers, but we
-		 already record entire registers and not just partial bits
-		 of them.  */
-	      I386_RECORD_FULL_ARCH_LIST_ADD_REG (ir.rex_b + ir.rm);
-	      /* These instructions also set conditional bits.  */
-	      I386_RECORD_FULL_ARCH_LIST_ADD_REG (X86_RECORD_EFLAGS_REGNUM);
-	      break;
-	    }
-	  else
-	    {
-	      /* We don't handle this particular instruction yet.  */
-	      ir.addr -= 2;
-	      opcode = opcode << 8 | ir.modrm;
-	      goto no_support;
-	    }
-	}
-      I386_RECORD_FULL_ARCH_LIST_ADD_REG (X86_RECORD_REAX_REGNUM);
-      I386_RECORD_FULL_ARCH_LIST_ADD_REG (X86_RECORD_REDX_REGNUM);
-      if (i386_record_lea_modrm (&ir))
-	return -1;
-      I386_RECORD_FULL_ARCH_LIST_ADD_REG (X86_RECORD_EFLAGS_REGNUM);
+      switch (ir.reg) {
+        case 1: /* cmpxchg8b */
+          if (ir.mod == 3)
+            {
+              ir.addr -= 2;
+              opcode = opcode << 8 | ir.modrm;
+              goto no_support;
+            }
+          I386_RECORD_FULL_ARCH_LIST_ADD_REG (X86_RECORD_REAX_REGNUM);
+          I386_RECORD_FULL_ARCH_LIST_ADD_REG (X86_RECORD_REDX_REGNUM);
+          if (i386_record_lea_modrm (&ir))
+            return -1;
+          I386_RECORD_FULL_ARCH_LIST_ADD_REG (X86_RECORD_EFLAGS_REGNUM);
+          break;
+        case 6: /* rdrand */
+        case 7: /* rdseed */
+          if (ir.mod != 3)
+            {
+              /* We don't handle this particular instruction yet.  */
+              ir.addr -= 2;
+              opcode = opcode << 8 | ir.modrm;
+              goto no_support;
+            }
+          /* rdrand and rdseed use the 3 bits of the REG field of ModR/M as
+             an extended opcode.  rdrand has bits 110 (/6) and rdseed
+             has bits 111 (/7).  */
+          /* The storage register is described by the 3 R/M bits, but the
+             REX.B prefix may be used to give access to registers
+             R8~R15.  In this case ir.rex_b + R/M will give us the register
+             in the range R8~R15.
+
+             REX.W may also be used to access 64-bit registers, but we
+             already record entire registers and not just partial bits
+             of them.  */
+          I386_RECORD_FULL_ARCH_LIST_ADD_REG (ir.rex_b + ir.rm);
+          /* These instructions also set conditional bits.  */
+          I386_RECORD_FULL_ARCH_LIST_ADD_REG (X86_RECORD_EFLAGS_REGNUM);
+          break;
+        default:
+          ir.addr -= 2;
+          opcode = opcode << 8 | ir.modrm;
+          goto no_support;
+      }
       break;
 
     case 0x50:    /* push */