diff mbox

Allow tests-special to be marked as UNSUPPORTED.

Message ID 55E49599.2050200@redhat.com
State Dropped
Headers show

Commit Message

Carlos O'Donell Aug. 31, 2015, 5:57 p.m. UTC
Joseph, Roland,

Is this the way we want to solve marking special tests as unsupported?

This patch adds a new target, similar to tests-unsupported, but allows
tests-special to be marked as unsupported.

When the test is unsupported the rule to make the test should be removed,
and the test added to tests-special-unsupported. The test should always
be added to tests-special to trigger the running of the test.

I immediately use this to mark c++-types-test.sh as unsupported if a
C++ compiler is not present, as might be the case during a bootstrap.
At present this generates a FAIL for c++-types-test during Fedora
stage1 bootstrap because there is no C++ compiler yet, and because the
check is `ifneq ($(CXX),no)` which is always true since CXX is
set to empty when no compiler present or a valid path to a compiler
when it is present.

Tested on x86_64 without a C++ compiler installed to verify that all the
C++ related tests become UNSUPPORTED, including c++-types-check.

Note that due to the tests-special in Makefile we can't put this target
into Rules, but we add to Rules a comment.

OK?

Cheers,
Carlos.

2015-08-28  Carlos O'Donell  <carlos@redhat.com>

	* Makefile [ifeq (,$(CXX)] (tests-special-unsupported):
	Add $(objpfx)c++-types-check.out.
	[ifneq "$(strip $(tests-special-unsupported))" ""]
	($(tests-special-unsupported)): New target.
	* Rules: Add comment that $(tests-special-unsupported) target
	is in Makefile.

---

Cheers,
Carlos.

Comments

Joseph Myers Aug. 31, 2015, 7:55 p.m. UTC | #1
On Mon, 31 Aug 2015, Carlos O'Donell wrote:

> +# The tests-special-unsupported target lives in Makefile because there are
> +# top-level tests in Makefile, and one or more of those top-level tests should
> +# be unsupported in some configurations, and Rules can't be included here due
> +# to ordering.  A better solution would be to move all tests-special tests out
> +# of the top-level and into subdirs and move this rule into Rules with the
> +# matching tests-unsupported target.

As far as I know c++-types-check.out is the only top-level test there 
might be a reason to make conditional like this, and moving it to a 
subdirectory (misc/ unless there's anything obviously better) would 
probably be a patch smaller than the comments you're adding about the 
issue.

(check-local-headers.out, however, might be trickier to move - it would be 
necessary to consider the unresolved issues with build ordering.  
begin-end-check would be in between.)
Roland McGrath Sept. 3, 2015, 11:32 p.m. UTC | #2
I haven't worked through all the details, but I agree with Joseph that if
moving tests from top-level into some subdir reduces the amount of change
you need to do, that is the thing to do.
Carlos O'Donell Sept. 4, 2015, 1:51 a.m. UTC | #3
On 09/03/2015 07:32 PM, Roland McGrath wrote:
> I haven't worked through all the details, but I agree with Joseph that if
> moving tests from top-level into some subdir reduces the amount of change
> you need to do, that is the thing to do.
> 

Yeah, I'll move it to misc then, if I can.

c.
Florian Weimer Oct. 29, 2015, 12:30 p.m. UTC | #4
On 08/31/2015 09:55 PM, Joseph Myers wrote:
> On Mon, 31 Aug 2015, Carlos O'Donell wrote:
> 
>> +# The tests-special-unsupported target lives in Makefile because there are
>> +# top-level tests in Makefile, and one or more of those top-level tests should
>> +# be unsupported in some configurations, and Rules can't be included here due
>> +# to ordering.  A better solution would be to move all tests-special tests out
>> +# of the top-level and into subdirs and move this rule into Rules with the
>> +# matching tests-unsupported target.
> 
> As far as I know c++-types-check.out is the only top-level test there 
> might be a reason to make conditional like this, and moving it to a 
> subdirectory (misc/ unless there's anything obviously better) would 
> probably be a patch smaller than the comments you're adding about the 
> issue.

I tried this.  Even in subdirectories, I couldn't get tests-special to
work with tests-unsupported.  I couldn't find any examples which used
tests-special in conjunction with tests-unsupported, either.

So it seems that Carlos' original patch is in fact the simplest approach.

Florian
Joseph Myers Oct. 29, 2015, 12:57 p.m. UTC | #5
On Thu, 29 Oct 2015, Florian Weimer wrote:

> I tried this.  Even in subdirectories, I couldn't get tests-special to
> work with tests-unsupported.  I couldn't find any examples which used
> tests-special in conjunction with tests-unsupported, either.
> 
> So it seems that Carlos' original patch is in fact the simplest approach.

I think there are two separate issues:

(a) Marking tests-special unsupported.

(b) Marking top-level tests unsupported.

For (a), we may need something like Carlos's patch, but with the rules in 
the correct places for subdirectory tests, rather than having long 
comments to justify them being in the wrong place.  (I have not reviewed 
the substance of Carlos's patch to see if, with things moved, it would be 
the right approach.)

For (b), the tests in question should be moved to subdirectories so that 
(a) works for them.

What I don't want is any additional infrastructure related to top-level 
tests, when top-level tests themselves are deprecated.  If any extra 
infrastructure is needed for a top-level test, that test should be moved 
to a subdirectory, and then any infrastructure added in the subdirectory 
context.
diff mbox

Patch

diff --git a/Makefile b/Makefile
index 658ccfa..585100a 100644
--- a/Makefile
+++ b/Makefile
@@ -251,8 +251,11 @@  tests-clean:
 	@$(MAKE) subdir_testclean no_deps=t
 
 tests-special += $(objpfx)c++-types-check.out $(objpfx)check-local-headers.out
-ifneq ($(CXX),no)
 
+ifeq (,$(CXX))
+# This test requires a C++ compiler to verify the types.
+tests-special-unsupported += $(objpfx)c++-types-check.out
+else
 vpath c++-types.data $(+sysdep_dirs)
 
 $(objpfx)c++-types-check.out: c++-types.data scripts/check-c++-types.sh
@@ -338,6 +341,24 @@  xtests:
 	  > $(objpfx)xtests.sum
 	$(call summarize-tests,xtests.sum, for extra tests)
 
+# tests-special-unsupported lists tests that we will not try to run at all in
+# this configuration.  Note this runs every time because it does not actually
+# create its target.  The dependency on Makefile is meant to ensure that it
+# runs after a Makefile change to add a tests to the list when it previously
+# ran and produced a .out file.
+# 
+# The tests-special-unsupported target lives in Makefile because there are
+# top-level tests in Makefile, and one or more of those top-level tests should
+# be unsupported in some configurations, and Rules can't be included here due
+# to ordering.  A better solution would be to move all tests-special tests out
+# of the top-level and into subdirs and move this rule into Rules with the
+# matching tests-unsupported target.
+ifneq "$(strip $(tests-special-unsupported))" ""
+$(tests-special-unsupported): Makefile
+	$(..)scripts/evaluate-test.sh $(patsubst $(common-objpfx)%.out,%,$@) \
+				      77 false false > $(@:.out=.test-result)
+endif
+
 # The realclean target is just like distclean for the parent, but we want
 # the subdirs to know the difference in case they care.
 realclean distclean: parent-clean
diff --git a/Rules b/Rules
index e237d03..7706fd1 100644
--- a/Rules
+++ b/Rules
@@ -210,6 +210,9 @@  $(tests-unsupported:%=$(objpfx)%.out): $(objpfx)%.out: Makefile
 				      77 false false > $(@:.out=.test-result)
 endif
 
+# NOTE: There is a tests-special-unsupported target, which should go here, but
+# for now lives in the top-level Makefile.
+
 endif	# tests