diff mbox series

[AutoFDO,IPA-CP] Handle zero-caller locals after AutoFDO early inlining

Message ID 20260804003733.89961-1-kvivekananda@nvidia.com
State New
Headers
Series [AutoFDO,IPA-CP] Handle zero-caller locals after AutoFDO early inlining |

Checks

Context Check Description
linaro-tcwg-bot/tcwg_gcc_build--master-arm success Build passed
linaro-tcwg-bot/tcwg_simplebootstrap_build--master-arm-bootstrap success Build passed
linaro-tcwg-bot/tcwg_gcc_check--master-arm success Test passed

Commit Message

Kugan Vivekanandarajah Aug. 4, 2026, 12:37 a.m. UTC 
  AutoFDO early inlining can inline all direct calls to a local
function while speculative call promotion keeps the function
address-taken.  The function consequently remains in the call graph
with no direct callers, triggering the -ftoplevel-reorder checking
assertion in IPA-CP.

Ordinary IPA devirtualization does not normally produce this state at
IPA-CP: it creates speculative edges before IPA-CP, but those edges are
inlined only by the later IPA inliner.  IPA-CP therefore still sees a
direct caller.  AutoFDO instead performs speculative inlining early, so
only its IPA_REF_ADDR may remain by the time IPA-CP runs.

When AutoFDO inlining is enabled, recognize zero-caller functions that
cannot be removed after their direct calls disappear and initialize
their lattices as variable.  Preserve the assertion for removable
locals and non-AutoFDO compilation.

gcc/ChangeLog:

	* ipa-cp.cc (initialize_node_lattices): Handle zero-caller local
	functions left after AutoFDO early inlining.

Regression tested on aarch64-linux-gnu with no new regressions

Signed-off-by: Kugan Vivekanandarajah <kvivekananda@nvidia.com>
---
 gcc/ipa-cp.cc | 13 ++++++++++++-
 1 file changed, 12 insertions(+), 1 deletion(-)

Comments

Jan Hubička Aug. 4, 2026, 6:49 a.m. UTC | #1 
Hi,

On Tue, Aug 4, 2026 at 2:38 AM Kugan Vivekanandarajah <
kvivekananda@nvidia.com> wrote:

> AutoFDO early inlining can inline all direct calls to a local
> function while speculative call promotion keeps the function
> address-taken.  The function consequently remains in the call graph
> with no direct callers, triggering the -ftoplevel-reorder checking
> assertion in IPA-CP.
>
> Ordinary IPA devirtualization does not normally produce this state at
> IPA-CP: it creates speculative edges before IPA-CP, but those edges are
> inlined only by the later IPA inliner.  IPA-CP therefore still sees a
> direct caller.  AutoFDO instead performs speculative inlining early, so
> only its IPA_REF_ADDR may remain by the time IPA-CP runs.
>
> When AutoFDO inlining is enabled, recognize zero-caller functions that
> cannot be removed after their direct calls disappear and initialize
> their lattices as variable.  Preserve the assertion for removable
> locals and non-AutoFDO compilation.
>
> gcc/ChangeLog:
>
>         * ipa-cp.cc (initialize_node_lattices): Handle zero-caller local
>         functions left after AutoFDO early inlining.
>

I think the problem here is earlier.  If function is local, it can not be
called
indirectly and thus it makes no sense to add speculative call to it.
I am not sure how the auto-FDO profile data got to a conclussion that
the function can be called indirectly - I can see it can be triggered when
source code changes between train run and compilation. Can you perhaps
check what is happening and if something like this works?

diff --git a/gcc/auto-profile.cc b/gcc/auto-profile.cc
index 6aa5166c748..63ca8938146 100644
--- a/gcc/auto-profile.cc
+++ b/gcc/auto-profile.cc
@@ -3326,6 +3333,14 @@ afdo_indirect_call (gcall *stmt, const
icall_target_map &map,
                 afdo_string_table->get_symbol_name (max_iter->first));
       return false;
     }
+  if (direct_call->local)
+    {
+      if (dump_file)
+       fprintf (dump_file, "AFDO Indirect call target %s "
+                "is local and thus can not be called indirectly\n",
+                afdo_string_table->get_symbol_name (max_iter->first));
+      return false;
+    }

   callee = gimple_call_fn (stmt);



>
> Regression tested on aarch64-linux-gnu with no new regressions
>
> Signed-off-by: Kugan Vivekanandarajah <kvivekananda@nvidia.com>
> ---
>  gcc/ipa-cp.cc | 13 ++++++++++++-
>  1 file changed, 12 insertions(+), 1 deletion(-)
>
> diff --git a/gcc/ipa-cp.cc b/gcc/ipa-cp.cc
> index c1b1f6201fd3..d68952ec8711 100644
> --- a/gcc/ipa-cp.cc
> +++ b/gcc/ipa-cp.cc
> @@ -1443,7 +1443,18 @@ initialize_node_lattices (struct cgraph_node *node)
>                                                   NULL, true);
>        else if (caller_count == 0)
>         {
> -         gcc_checking_assert (!opt_for_fn (node->decl,
> flag_toplevel_reorder));
> +         /* With -ftoplevel-reorder, a local body with no direct callers
> +            should already have been removed.  AutoFDO early-inline of a
> +            speculative call expands via gimple_ic (&target), so after all
> +            direct call edges are inlined the local can remain
> address-taken
> +            with unknown indirect uses; treat that as VARIABLE.  Ordinary
> +            IPA devirtualization does not need this exception: it creates
> +            speculative edges before IPA-CP, but inlines them only later,
> so
> +            IPA-CP still sees a direct caller.  */
> +         if (!(flag_auto_profile && flag_auto_profile_inlining
> +               && !node->can_remove_if_no_direct_calls_p (false)))
> +           gcc_checking_assert
> +             (!opt_for_fn (node->decl, flag_toplevel_reorder));
>           variable = true;
>         }
>      }
> --
> 2.34.1
>
>
diff mbox series

Patch

diff --git a/gcc/ipa-cp.cc b/gcc/ipa-cp.cc
index c1b1f6201fd3..d68952ec8711 100644
--- a/gcc/ipa-cp.cc
+++ b/gcc/ipa-cp.cc
@@ -1443,7 +1443,18 @@  initialize_node_lattices (struct cgraph_node *node)
 						  NULL, true);
       else if (caller_count == 0)
 	{
-	  gcc_checking_assert (!opt_for_fn (node->decl, flag_toplevel_reorder));
+	  /* With -ftoplevel-reorder, a local body with no direct callers
+	     should already have been removed.  AutoFDO early-inline of a
+	     speculative call expands via gimple_ic (&target), so after all
+	     direct call edges are inlined the local can remain address-taken
+	     with unknown indirect uses; treat that as VARIABLE.  Ordinary
+	     IPA devirtualization does not need this exception: it creates
+	     speculative edges before IPA-CP, but inlines them only later, so
+	     IPA-CP still sees a direct caller.  */
+	  if (!(flag_auto_profile && flag_auto_profile_inlining
+		&& !node->can_remove_if_no_direct_calls_p (false)))
+	    gcc_checking_assert
+	      (!opt_for_fn (node->decl, flag_toplevel_reorder));
 	  variable = true;
 	}
     }