x86: restrict prefix use with .insn VEX/XOP/EVEX

Message ID e1c1d593-4588-08e0-d3f7-0cd42b0dc6e4@suse.com
State New
Headers
Series x86: restrict prefix use with .insn VEX/XOP/EVEX |

Checks

Context Check Description
linaro-tcwg-bot/tcwg_binutils_check--master-aarch64 success Testing passed
linaro-tcwg-bot/tcwg_binutils_check--master-arm success Testing passed
linaro-tcwg-bot/tcwg_binutils_build--master-arm success Testing passed
linaro-tcwg-bot/tcwg_binutils_build--master-aarch64 success Testing passed

Commit Message

Jan Beulich Sept. 1, 2023, 12:30 p.m. UTC
  Avoid triggering the respective abort() in output_insn().
  

Patch

--- a/gas/config/tc-i386.c
+++ b/gas/config/tc-i386.c
@@ -10806,6 +10806,29 @@  s_insn (int dummy ATTRIBUTE_UNUSED)
   if (line > end && i.vec_encoding == vex_encoding_default)
     i.vec_encoding = evex ? vex_encoding_evex : vex_encoding_vex;
 
+  if (i.vec_encoding != vex_encoding_default)
+    {
+      /* Only address size and segment override prefixes are permitted with
+         VEX/XOP/EVEX encodings.  */
+      const unsigned char *p = i.prefix;
+
+      for (j = 0; j < ARRAY_SIZE (i.prefix); ++j, ++p)
+	{
+	  if (!*p)
+	    continue;
+
+	  switch (j)
+	    {
+	    case SEG_PREFIX:
+	    case ADDR_PREFIX:
+	      break;
+	    default:
+		  as_bad (_("illegal prefix used with VEX/XOP/EVEX"));
+		  goto bad;
+	    }
+	}
+    }
+
   if (line > end && *line == '.')
     {
       /* Length specifier (VEX.L, XOP.L, EVEX.L'L).  */