[1/3] diagnostics: Enable escape sequence processing on windows consoles

Message ID 20240509144808.18468-1-peter0x44@disroot.org
State New
Headers
Series [1/3] diagnostics: Enable escape sequence processing on windows consoles |

Checks

Context Check Description
linaro-tcwg-bot/tcwg_gcc_build--master-arm success Testing passed
linaro-tcwg-bot/tcwg_gcc_build--master-aarch64 success Testing passed
linaro-tcwg-bot/tcwg_gcc_check--master-arm success Testing passed
linaro-tcwg-bot/tcwg_gcc_check--master-aarch64 success Testing passed

Commit Message

Peter0x44 May 9, 2024, 2:48 p.m. UTC
  Since windows 10 release v1511, the windows console has had support for VT100
escape sequences. We should try to enable this, and utilize it where possible.

gcc/ChangeLog:
	* diagnostic-color.cc (should_colorize): Enable processing of VT100
	escape sequences on windows consoles

Signed-off-by: Peter Damianov <peter0x44@disroot.org>
---
 gcc/diagnostic-color.cc | 20 +++++++++++++++-----
 1 file changed, 15 insertions(+), 5 deletions(-)
  

Comments

LIU Hao May 9, 2024, 3:55 p.m. UTC | #1
在 2024-05-09 22:48, Peter Damianov 写道:
> -  h = GetStdHandle (STD_ERROR_HANDLE);
> -  return (h != INVALID_HANDLE_VALUE) && (h != NULL)
> -	  && GetConsoleMode (h, &m);
> +  handle = GetStdHandle (STD_ERROR_HANDLE);
> +  isconsole = GetConsoleMode (handle, &mode);

Shouldn't `GetConsoleMode()` be called only when `handle` is valid? I think you may initialize 
`isconsole` to `false`; then only if the handle is valid, should it be set accordingly; and this 
function just returns `isconsole`.

The other two patches look good to me.


-- 
Best regards,
LIU Hao
  

Patch

diff --git a/gcc/diagnostic-color.cc b/gcc/diagnostic-color.cc
index f01a0fc2e37..0a4845406f5 100644
--- a/gcc/diagnostic-color.cc
+++ b/gcc/diagnostic-color.cc
@@ -213,12 +213,22 @@  should_colorize (void)
      pp_write_text_to_stream() in pretty-print.cc calls fputs() on
      that stream.  However, the code below for non-Windows doesn't seem
      to care about it either...  */
-  HANDLE h;
-  DWORD m;
+  HANDLE handle;
+  DWORD mode;
+  BOOL isconsole;
 
-  h = GetStdHandle (STD_ERROR_HANDLE);
-  return (h != INVALID_HANDLE_VALUE) && (h != NULL)
-	  && GetConsoleMode (h, &m);
+  handle = GetStdHandle (STD_ERROR_HANDLE);
+  isconsole = GetConsoleMode (handle, &mode);
+
+  if (isconsole)
+    {
+      /* Try to enable processing of VT100 escape sequences */
+      mode |= ENABLE_PROCESSED_OUTPUT | ENABLE_VIRTUAL_TERMINAL_PROCESSING;
+      SetConsoleMode (handle, mode);
+    }
+
+  return (handle != INVALID_HANDLE_VALUE) && (handle != NULL)
+	  && isconsole;
 #else
   char const *t = getenv ("TERM");
   /* emacs M-x shell sets TERM="dumb".  */