From patchwork Thu Jun 22 22:44:54 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Zack Weinberg X-Patchwork-Id: 21214 Received: (qmail 74548 invoked by alias); 22 Jun 2017 22:45:05 -0000 Mailing-List: contact libc-alpha-help@sourceware.org; run by ezmlm Precedence: bulk List-Id: List-Unsubscribe: List-Subscribe: List-Archive: List-Post: List-Help: , Sender: libc-alpha-owner@sourceware.org Delivered-To: mailing list libc-alpha@sourceware.org Received: (qmail 74344 invoked by uid 89); 22 Jun 2017 22:45:02 -0000 Authentication-Results: sourceware.org; auth=none X-Virus-Found: No X-Spam-SWARE-Status: No, score=-25.6 required=5.0 tests=AWL, BAYES_00, GIT_PATCH_0, GIT_PATCH_1, GIT_PATCH_2, GIT_PATCH_3, SPF_PASS, T_RP_MATCHES_RCVD autolearn=ham version=3.3.2 spammy=expire, timely, dialogue, clumsy X-Spam-User: qpsmtpd, 2 recipients X-HELO: mailbackend.panix.com From: Zack Weinberg To: libc-alpha@sourceware.org, gdb@sourceware.org Cc: joseph@codesourcery.com, fweimer@redhat.com, tom@tromey.com, siddhesh@gotplt.org Subject: [PATCH 1/3] Improve testing of GDB pretty-printers. Date: Thu, 22 Jun 2017 18:44:54 -0400 Message-Id: <20170622224456.1358-2-zackw@panix.com> In-Reply-To: <20170622224456.1358-1-zackw@panix.com> References: <20170622224456.1358-1-zackw@panix.com> MIME-Version: 1.0 The C programs used to test GDB pretty-printers were being compiled with -DMODULE_NAME=libc (!) which causes many problems, such as failure to link if they refer to errno. Now they are compiled with -DMODULE_NAME=testsuite instead. test_printers_common.py was testing for expected output in a clumsy way which meant the pexpect timeout had to expire before it could report a failure, even if the regexp was never going to match. This slows down debugging a test quite a bit. Rewrote that logic so it doesn't do that anymore. Note that as a side effect, test() fails the test by calling exit() rather than throwing an exception -- that could change if people think it's a bad idea. Add an 'unsupported_pattern' argument to test(); if the normal 'pattern' fails to match, but an 'unsupported_pattern' was supplied and it matches, then the test fails as unsupported, not as a normal failure. This feature is used in part 2. Tighten up the code to honor TIMEOUTFACTOR, and add another environment variable TEST_PRINTERS_LOG; if this is set to a pathname, all of the dialogue with the gdb subprocess will be logged to that file. * Rules: Set MODULE_NAME=testsuite for everything in tests-printers. * scripts/test_printers_common.py (TIMEOUTFACTOR): Tighten up handling. (TEST_PRINTERS_LOG): New env variable; if set, pexpect will log all dialogue with the gdb subprocess to the file it names. (send_command): New function broken out of test. (test): Add 'unsupported_pattern' argument and improve handling of 'pattern' argument; match failures no longer have to wait for the timeout. --- Rules | 4 ++ scripts/test_printers_common.py | 84 ++++++++++++++++++++++------------------- 2 files changed, 50 insertions(+), 38 deletions(-) diff --git a/Rules b/Rules index 168cf508d7..85b77a00e8 100644 --- a/Rules +++ b/Rules @@ -265,6 +265,10 @@ endif # tests ifdef PYTHON ifneq "$(strip $(tests-printers))" "" +cpp-srcs-left := $(tests-printers) +lib := testsuite +include $(patsubst %,$(..)libof-iterator.mk,$(cpp-srcs-left)) + # Static pattern rule for building the test programs for the pretty printers. $(tests-printers-programs): %: %.o $(tests-printers-libs) \ $(sort $(filter $(common-objpfx)lib%,$(link-libc-static-tests))) \ diff --git a/scripts/test_printers_common.py b/scripts/test_printers_common.py index fe88f36366..aabb9da83e 100644 --- a/scripts/test_printers_common.py +++ b/scripts/test_printers_common.py @@ -54,11 +54,7 @@ if not pexpect.which(gdb_bin): print('gdb 7.8 or newer must be installed to test the pretty printers.') exit(UNSUPPORTED) -timeout = 5 -TIMEOUTFACTOR = os.environ.get('TIMEOUTFACTOR') - -if TIMEOUTFACTOR: - timeout = int(TIMEOUTFACTOR) +timeout = int(os.environ.get('TIMEOUTFACTOR', '5')) try: # Check the gdb version. @@ -93,15 +89,39 @@ try: # If everything's ok, spawn the gdb process we'll use for testing. gdb = pexpect.spawn(gdb_invocation, echo=False, timeout=timeout, encoding=encoding) - gdb_prompt = u'\(gdb\)' + logfile = os.environ.get("TEST_PRINTERS_LOG") + if logfile is not None: + gdb.logfile = open(logfile, "wt") + + gdb_prompt = u'(?:\A|\r\n)\(gdb\) ' gdb.expect(gdb_prompt) except pexpect.ExceptionPexpect as exception: print('Error: {0}'.format(exception)) exit(FAIL) -def test(command, pattern=None): - """Sends 'command' to gdb and expects the given 'pattern'. +def send_command(command): + """Sends 'command' to gdb, and returns all output up to but not + including the next gdb prompt. If a gdb prompt is not detected + in a timely fashion, raises pexpect.TIMEOUT. + + Args: + command (string): The command we'll send to gdb. + """ + + gdb.sendline(command) + + # PExpect does a non-greedy match for '+' and '*', since it can't + # look ahead on the gdb output stream. Therefore, we must include + # the gdb prompt in the match to ensure that all of the output of + # the command is captured. + gdb.expect(u'(.*?){}'.format(gdb_prompt)) + return gdb.match.group(1) + + +def test(command, pattern=None, unsupported_pattern=None): + """Sends 'command' to gdb and expects the given 'pattern'. If + the match fails, the test fails. If 'pattern' is None, simply consumes everything up to and including the gdb prompt. @@ -109,43 +129,31 @@ def test(command, pattern=None): Args: command (string): The command we'll send to gdb. pattern (raw string): A pattern the gdb output should match. + unsupported_pattern (raw string): If the gdb output fails to + match 'pattern', but it _does_ match this, then the test + is marked unsupported rather than failing outright. Returns: string: The string that matched 'pattern', or an empty string if 'pattern' was None. """ + output = send_command(command) + if pattern is None: + return None - match = '' + match = re.search(pattern, output, re.DOTALL) + if not match: + if (unsupported_pattern is not None + and re.search(unsupported_pattern, output, re.DOTALL)): + exit(UNSUPPORTED) + else: + print('Response does not match the expected pattern.\n' + 'Command: {0}\n' + 'Expected pattern: {1}\n' + 'Response: {2}'.format(command, pattern, output)) + exit(FAIL) - gdb.sendline(command) - - if pattern: - # PExpect does a non-greedy match for '+' and '*'. Since it can't look - # ahead on the gdb output stream, if 'pattern' ends with a '+' or a '*' - # we may end up matching only part of the required output. - # To avoid this, we'll consume 'pattern' and anything that follows it - # up to and including the gdb prompt, then extract 'pattern' later. - index = gdb.expect([u'{0}.+{1}'.format(pattern, gdb_prompt), - pexpect.TIMEOUT]) - - if index == 0: - # gdb.after now contains the whole match. Extract the text that - # matches 'pattern'. - match = re.match(pattern, gdb.after, re.DOTALL).group() - elif index == 1: - # We got a timeout exception. Print information on what caused it - # and bail out. - error = ('Response does not match the expected pattern.\n' - 'Command: {0}\n' - 'Expected pattern: {1}\n' - 'Response: {2}'.format(command, pattern, gdb.before)) - - raise pexpect.TIMEOUT(error) - else: - # Consume just the the gdb prompt. - gdb.expect(gdb_prompt) - - return match + return match.group(0) def init_test(test_bin, printer_files, printer_names): """Loads the test binary file and the required pretty printers to gdb.