From patchwork Sat Feb 3 10:50:09 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Lehua Ding X-Patchwork-Id: 85263 Return-Path: X-Original-To: patchwork@sourceware.org Delivered-To: patchwork@sourceware.org Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id 5759C3857C62 for ; Sat, 3 Feb 2024 10:51:07 +0000 (GMT) X-Original-To: gcc-patches@gcc.gnu.org Delivered-To: gcc-patches@gcc.gnu.org Received: from smtpbgjp3.qq.com (smtpbgjp3.qq.com [54.92.39.34]) by sourceware.org (Postfix) with ESMTPS id F168C3858403 for ; Sat, 3 Feb 2024 10:50:21 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org F168C3858403 Authentication-Results: sourceware.org; dmarc=none (p=none dis=none) header.from=rivai.ai Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=rivai.ai ARC-Filter: OpenARC Filter v1.0.0 sourceware.org F168C3858403 Authentication-Results: server2.sourceware.org; arc=none smtp.remote-ip=54.92.39.34 ARC-Seal: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1706957424; cv=none; b=pWlolcIQKcE2Yh+e8G4IXowfaqoIkmSbpnzxAMwjhdgCoAAGwJpIJyJB4sbSANE0dC0OrhzX/Vk6KbF3vkgN9gLW2j1i+L5ihxJ5pw9C9xbhWY6lVDAl5SfqDQuCjTJ+7wS2WVAHTWUaxNjvYBtJyY0FBmhiSX7lx1XdSdMcywE= ARC-Message-Signature: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1706957424; c=relaxed/simple; bh=gk70wZBQRx91wKjuU84bFgf7tWLi/W6LgZ8GPXb7InA=; h=From:To:Subject:Date:Message-Id:MIME-Version; b=PQxmYL6aZ1fMyH/HrvjfM7yFXBQeQMZxkLV8WfJkR3KzX/bndmbQy7TKsg0fuVeHC0Gl31peLUm3Y1tnBFcLLCAOJNQk6ULUF/6mKA02D09Vd78PzQCkRNhdTpErRUedZ1AtOd9teR4mZHGhToPt14BseAE3AWDw+Vd494IknJA= ARC-Authentication-Results: i=1; server2.sourceware.org X-QQ-mid: bizesmtp72t1706957417tkl52ulr X-QQ-Originating-IP: WUXRpBymlBijIfL9TpKIl21vviooYT9XoAxSnlDhX0I= Received: from rios-cad5.localdomain ( [58.60.1.25]) by bizesmtp.qq.com (ESMTP) with id ; Sat, 03 Feb 2024 18:50:16 +0800 (CST) X-QQ-SSF: 01400000000000C0F000000A0000000 X-QQ-FEAT: 90EFqYDyPxBsGiBNwLQOtLB4gJlavPy8D0ryBqrNbAtgBjPbHxJAhTFwm6Fty Pe4ne/J7kqJjEe28OvNihqE7kWAkpLb0HdDmiewS1Gn9KuyRYjOkExUTrl8cruPeGoIJB3L C26jzZs/wG5BdJo3Iy8Ert0mziiyLYHe7bihqrSPgHTxyi7wB8P1DZYMpPTK0unfXOv7/fp xTZzA1P//nRCmcbzZwxtOxyLuaR84ZDOoz+TgQQwrzuyWnUiZvYqePb263ESMwTEVJ+HynT +q6ar6H/+QGdunnDzNN1j0TNeG4H7nzd+rLTJ+gdpG17827VY9nme4XVqG+FQ51W6F3mwUZ RAURX6QsSTAK4PSAO99WppeeHbDFyQKCPr+3zwIiRd5jeiZkpxjDchruOsHFg== X-QQ-GoodBg: 2 X-BIZMAIL-ID: 15639308851980038823 From: Lehua Ding To: gcc-patches@gcc.gnu.org Cc: vmakarov@redhat.com, richard.sandiford@arm.com, juzhe.zhong@rivai.ai, lehua.ding@rivai.ai Subject: [PATCH 1/4] df: Add -ftrack-subreg-liveness option Date: Sat, 3 Feb 2024 18:50:09 +0800 Message-Id: <20240203105012.208998-2-lehua.ding@rivai.ai> X-Mailer: git-send-email 2.36.3 In-Reply-To: <20240203105012.208998-1-lehua.ding@rivai.ai> References: <20240203105012.208998-1-lehua.ding@rivai.ai> MIME-Version: 1.0 X-QQ-SENDSIZE: 520 Feedback-ID: bizesmtp:rivai.ai:qybglogicsvrgz:qybglogicsvrgz5a-3 X-Spam-Status: No, score=-9.4 required=5.0 tests=BAYES_00, GIT_PATCH_0, KAM_DMARC_STATUS, RCVD_IN_BARRACUDACENTRAL, RCVD_IN_DNSWL_NONE, RCVD_IN_MSPIKE_H4, RCVD_IN_MSPIKE_WL, SPF_HELO_PASS, SPF_PASS, TXREP, T_SCC_BODY_TEXT_LINE, URIBL_CSS_A autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org X-BeenThere: gcc-patches@gcc.gnu.org X-Mailman-Version: 2.1.30 Precedence: list List-Id: Gcc-patches mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: gcc-patches-bounces+patchwork=sourceware.org@gcc.gnu.org Add new flag -ftrack-subreg-liveness to enable track-subreg-liveness. This flag is enabled at -O3/fast. gcc/ChangeLog: * common.opt: add -ftrack-subreg-liveness option. * opts.cc: auto aneble -ftrack-subreg-liveness in -O3/fast --- gcc/common.opt | 4 ++++ gcc/opts.cc | 1 + 2 files changed, 5 insertions(+) diff --git a/gcc/common.opt b/gcc/common.opt index 51c4a17da83..d4592c6426a 100644 --- a/gcc/common.opt +++ b/gcc/common.opt @@ -2156,6 +2156,10 @@ fira-verbose= Common RejectNegative Joined UInteger Var(flag_ira_verbose) Init(5) -fira-verbose= Control IRA's level of diagnostic messages. +ftrack-subreg-liveness +Common Var(flag_track_subreg_liveness) Init(0) Optimization +Track subreg liveness information for IRA and LRA, enabled at -O3. + fivopts Common Var(flag_ivopts) Init(1) Optimization Optimize induction variables on trees. diff --git a/gcc/opts.cc b/gcc/opts.cc index 3333600e0ea..50c0b62c5af 100644 --- a/gcc/opts.cc +++ b/gcc/opts.cc @@ -689,6 +689,7 @@ static const struct default_options default_options_table[] = { OPT_LEVELS_3_PLUS, OPT_funswitch_loops, NULL, 1 }, { OPT_LEVELS_3_PLUS, OPT_fvect_cost_model_, NULL, VECT_COST_MODEL_DYNAMIC }, { OPT_LEVELS_3_PLUS, OPT_fversion_loops_for_strides, NULL, 1 }, + { OPT_LEVELS_3_PLUS, OPT_ftrack_subreg_liveness, NULL, 1 }, /* -O3 parameters. */ { OPT_LEVELS_3_PLUS, OPT__param_max_inline_insns_auto_, NULL, 30 }, From patchwork Sat Feb 3 10:50:10 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Lehua Ding X-Patchwork-Id: 85264 Return-Path: X-Original-To: patchwork@sourceware.org Delivered-To: patchwork@sourceware.org Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id 90D893857BA0 for ; Sat, 3 Feb 2024 10:51:30 +0000 (GMT) X-Original-To: gcc-patches@gcc.gnu.org Delivered-To: gcc-patches@gcc.gnu.org Received: from smtpbg150.qq.com (smtpbg150.qq.com [18.132.163.193]) by sourceware.org (Postfix) with ESMTPS id D55353858403 for ; Sat, 3 Feb 2024 10:50:25 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org D55353858403 Authentication-Results: sourceware.org; dmarc=none (p=none dis=none) header.from=rivai.ai Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=rivai.ai ARC-Filter: OpenARC Filter v1.0.0 sourceware.org D55353858403 Authentication-Results: server2.sourceware.org; arc=none smtp.remote-ip=18.132.163.193 ARC-Seal: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1706957434; cv=none; b=mLXPpkEt+TCFSHcuwwkPaj7PM+2yyZ/ksDBhw2UZgq/aVXsvcwlIcgoOSx7Nz7g11sQgwFeNai2jsJOim/yUOYMThAr0FMM18xSWHpVXw3IvKBXjX6tXVi/hTUBTuKdMJJSPG3V3c6DGdKtoyBxonse3DNdfTPCJmrDX/F+hAUg= ARC-Message-Signature: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1706957434; c=relaxed/simple; bh=ev6qNb5tgVsQyUIG5bIYruzQaLmoRWzLVJtBFhQZlBg=; h=From:To:Subject:Date:Message-Id:MIME-Version; b=U9qGxcs/ktm9UILqwJZgEFqOOx4fIfROQbzteg2JhGIZ5YbHtSQUakSGmlGFc9mgMR261EriGmF0hMBKCUeDwAHeyFhVN6G2M7kFfD+3GBLPQPryIVNSdn91ntlhYbZ1dzQK0emkYw6+2Qwasv4bmkC8VzRg8G57WNpn24SN5aQ= ARC-Authentication-Results: i=1; server2.sourceware.org X-QQ-mid: bizesmtp72t1706957420tyqdzd1q X-QQ-Originating-IP: ybtPR720aPD5FB53dojWtxUUC8JRgj2/JO0sOeFxhEU= Received: from rios-cad5.localdomain ( [58.60.1.25]) by bizesmtp.qq.com (ESMTP) with id ; Sat, 03 Feb 2024 18:50:19 +0800 (CST) X-QQ-SSF: 01400000000000C0F000000A0000000 X-QQ-FEAT: znfcQSa1hKakNAZxXGcJO2LPD66ABXWFtc3Y3c40Fn8o/2dbTIm41MXYAFhJH IIW5gI/f04c4NEXqcKW90dkMoIV0bbg7mhbtKajZNmnio6AOLFn7vsHOunVCJXg2t6rky6Y syhd9dm6wY4HSpZbRt0qygD8BTdrqBNZhnLnUVb5aAgrEz4lH9T0zChRkwzx5IgVGNzts8B Wx1YYRg2rQKtdhSRHqTnJKsog5IZORW6H9II5zqmY0TlRQXiIViREHZyPIVsTJBpp6VrIKG fIsGIjEixoHAqIqbK6NIM9P9fK7giQOdLEFTDnWdcxQV9fS/k1szOr/f8egoPx0Xh71pV/X XKSUKNIBqHcxhPN10ncHvTsMzpOxTFY/22vE1/JQ0QvGGML2iIkGZmMMCjanfkb7ecfLgiR X-QQ-GoodBg: 2 X-BIZMAIL-ID: 1229324787317692073 From: Lehua Ding To: gcc-patches@gcc.gnu.org Cc: vmakarov@redhat.com, richard.sandiford@arm.com, juzhe.zhong@rivai.ai, lehua.ding@rivai.ai Subject: [PATCH 2/4] df: Add DF_LIVE_SUBREG problem Date: Sat, 3 Feb 2024 18:50:10 +0800 Message-Id: <20240203105012.208998-3-lehua.ding@rivai.ai> X-Mailer: git-send-email 2.36.3 In-Reply-To: <20240203105012.208998-1-lehua.ding@rivai.ai> References: <20240203105012.208998-1-lehua.ding@rivai.ai> MIME-Version: 1.0 X-QQ-SENDSIZE: 520 Feedback-ID: bizesmtp:rivai.ai:qybglogicsvrgz:qybglogicsvrgz5a-3 X-Spam-Status: No, score=-9.1 required=5.0 tests=BAYES_00, GIT_PATCH_0, KAM_ASCII_DIVIDERS, KAM_DMARC_STATUS, KAM_SHORT, RCVD_IN_BARRACUDACENTRAL, RCVD_IN_DNSWL_NONE, RCVD_IN_MSPIKE_H3, RCVD_IN_MSPIKE_WL, SPF_HELO_PASS, SPF_PASS, TXREP, T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org X-BeenThere: gcc-patches@gcc.gnu.org X-Mailman-Version: 2.1.30 Precedence: list List-Id: Gcc-patches mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: gcc-patches-bounces+patchwork=sourceware.org@gcc.gnu.org This patch add a new DF problem, named DF_LIVE_SUBREG. This problem is extended from the DF_LR problem and support track the subreg liveness of multireg pseudo if these pseudo satisfy the following conditions: 1. the mode size greater than it's REGMODE_NATURAL_SIZE. 2. the reg is used in insns via subreg pattern. The main methods are as follows: 1. split bitmap in/out/def/use fileds to full_in/out/def/use and partial_in/out/def/use. If a pseudo need to be tracked it's subreg liveness, then it is recorded in partial_in/out/def/use fileds. Meantimes, there are range_in/out/def/use fileds which records the live range of the tracked pseudo. 2. in the df_live_subreg_finalize function, we move the tracked pseudo from the partial_in/out/def/use to full_in/out/def/use if the pseudo's live range is full. gcc/ChangeLog: * Makefile.in: Add subreg-live-range object file. * df-problems.cc (struct df_live_subreg_problem_data): Private struct for DF_LIVE_SUBREG problem. (df_live_subreg_get_bb_info): getting bb regs in/out data. (get_live_subreg_local_bb_info): getting bb regs use/def data. (multireg_p): checking is the regno a pseudo multireg. (need_track_subreg_p): checking is the regno need to be tracked. (init_range): getting the range of subreg rtx. (remove_subreg_range): removing use data for the reg/subreg rtx. (add_subreg_range): adding def/use data for the reg/subreg rtx. (df_live_subreg_free_bb_info): Free basic block df data. (df_live_subreg_alloc): Allocate and init df data. (df_live_subreg_reset): Reset the live in/out df data. (df_live_subreg_bb_local_compute): Compute basic block df data. (df_live_subreg_local_compute): Compute all basic blocks df data. (df_live_subreg_init): Init the in/out df data. (df_live_subreg_check_result): Assert the full and partial df data. (df_live_subreg_confluence_0): Confluence function for infinite loops. (df_live_subreg_confluence_n): Confluence function for normal edge. (df_live_subreg_transfer_function): Transfer function. (df_live_subreg_finalize): Finalize the all_in/all_out df data. (df_live_subreg_free): Free the df data. (df_live_subreg_top_dump): Dump top df data. (df_live_subreg_bottom_dump): Dump bottom df data. (df_live_subreg_add_problem): Add the DF_LIVE_SUBREG problem. * df.h (enum df_problem_id): Add DF_LIVE_SUBREG. (class subregs_live): Simple decalare. (class df_live_subreg_local_bb_info): New class for full/partial def/use df data. (class df_live_subreg_bb_info): New class for full/partial in/out df data. (df_live_subreg): getting the df_live_subreg data. (df_live_subreg_add_problem): Exported. (df_live_subreg_finalize): Ditto. (df_live_subreg_check_result): Ditto. (multireg_p): Ditto. (init_range): Ditto. (add_subreg_range): Ditto. (remove_subreg_range): Ditto. (df_get_subreg_live_in): Accessor the all_in df data. (df_get_subreg_live_out): Accessor the all_out df data. (df_get_subreg_live_full_in): Accessor the full_in df data. (df_get_subreg_live_full_out): Accessor the full_out df data. (df_get_subreg_live_partial_in): Accessor the partial_in df data. (df_get_subreg_live_partial_out): Accessor the partial_out df data. (df_get_subreg_live_range_in): Accessor the range_in df data. (df_get_subreg_live_range_out): Accessor the range_out df data. * regs.h (get_nblocks): Get the blocks of mode. * sbitmap.cc (bitmap_full_p): sbitmap predicator. (bitmap_same_p): sbitmap predicator. (test_full): test bitmap_full_p. (test_same): test bitmap_same_p. (sbitmap_cc_tests): Add test_full and test_same. * sbitmap.h (bitmap_full_p): Exported. (bitmap_same_p): Ditto. * timevar.def (TV_DF_LIVE_SUBREG): add DF_LIVE_SUBREG timevar. * subreg-live-range.cc: New file. * subreg-live-range.h: New file. --- gcc/Makefile.in | 1 + gcc/df-problems.cc | 855 ++++++++++++++++++++++++++++++++++++++- gcc/df.h | 155 +++++++ gcc/regs.h | 5 + gcc/sbitmap.cc | 98 +++++ gcc/sbitmap.h | 2 + gcc/subreg-live-range.cc | 53 +++ gcc/subreg-live-range.h | 206 ++++++++++ gcc/timevar.def | 1 + 9 files changed, 1375 insertions(+), 1 deletion(-) create mode 100644 gcc/subreg-live-range.cc create mode 100644 gcc/subreg-live-range.h diff --git a/gcc/Makefile.in b/gcc/Makefile.in index 95caa54a52b..c30ea333736 100644 --- a/gcc/Makefile.in +++ b/gcc/Makefile.in @@ -1684,6 +1684,7 @@ OBJS = \ store-motion.o \ streamer-hooks.o \ stringpool.o \ + subreg-live-range.o \ substring-locations.o \ target-globals.o \ targhooks.o \ diff --git a/gcc/df-problems.cc b/gcc/df-problems.cc index 88ee0dd67fc..703c7f914a4 100644 --- a/gcc/df-problems.cc +++ b/gcc/df-problems.cc @@ -28,6 +28,7 @@ along with GCC; see the file COPYING3. If not see #include "target.h" #include "rtl.h" #include "df.h" +#include "subreg-live-range.h" #include "memmodel.h" #include "tm_p.h" #include "insn-config.h" @@ -1344,8 +1345,860 @@ df_lr_verify_transfer_functions (void) bitmap_clear (&all_blocks); } +/*---------------------------------------------------------------------------- + REGISTER AND SUBREGS LIVES + Like DF_LR, but include tracking subreg liveness. Currently used to provide + subreg liveness related information to the register allocator. The subreg + information is currently tracked for registers that satisfy the following + conditions: + 1. REG is a pseudo register + 2. MODE_SIZE > UNIT_SIZE + 3. MODE_SIZE is a multiple of UNIT_SIZE + 4. REG is used via subreg pattern + Assuming: MODE = the machine mode of the REG + MODE_SIZE = GET_MODE_SIZE (MODE) + UNIT_SIZE = REGMODE_NATURAL_SIZE (MODE) + Condition 3 is currently strict, maybe it can be removed in the future, but + for now it is sufficient. + ----------------------------------------------------------------------------*/ + +/* These two empty data are used as default data in case the user does not turn + * on the track-subreg-liveness feature. */ +bitmap_head empty_bitmap; +subregs_live empty_live; + +/* Private data for live_subreg problem. */ +struct df_live_subreg_problem_data +{ + /* Record registers that need to track subreg liveness. */ + bitmap_head tracked_regs; + /* An obstack for the bitmaps we need for this problem. */ + bitmap_obstack live_subreg_bitmaps; +}; + +/* Helper functions */ + +static df_live_subreg_bb_info * +df_live_subreg_get_bb_info (unsigned int index) +{ + if (index < df_live_subreg->block_info_size) + return &( + (class df_live_subreg_bb_info *) df_live_subreg->block_info)[index]; + else + return NULL; +} + +static df_live_subreg_local_bb_info * +get_live_subreg_local_bb_info (unsigned int bb_index) +{ + return df_live_subreg_get_bb_info (bb_index); +} + +/* Return true if regno is a multireg. */ +bool +multireg_p (int regno) +{ + if (regno < FIRST_PSEUDO_REGISTER) + return false; + rtx regno_rtx = regno_reg_rtx[regno]; + machine_mode reg_mode = GET_MODE (regno_rtx); + poly_int64 total_size = GET_MODE_SIZE (reg_mode); + poly_int64 natural_size = REGMODE_NATURAL_SIZE (reg_mode); + return maybe_gt (total_size, natural_size) + && multiple_p (total_size, natural_size); +} + +/* Return true if the REGNO need be track with subreg liveness. */ + +static bool +need_track_subreg_p (unsigned regno) +{ + struct df_live_subreg_problem_data *problem_data + = (struct df_live_subreg_problem_data *) df_live_subreg->problem_data; + return bitmap_bit_p (&problem_data->tracked_regs, regno); +} + +/* Return the subreg_range of REF. */ +void +init_range (rtx op, sbitmap range) +{ + rtx reg = SUBREG_P (op) ? SUBREG_REG (op) : op; + machine_mode reg_mode = GET_MODE (reg); + + if (!read_modify_subreg_p (op)) + { + bitmap_set_range (range, 0, get_nblocks (reg_mode)); + return; + } + + rtx subreg = op; + machine_mode subreg_mode = GET_MODE (subreg); + poly_int64 offset = SUBREG_BYTE (subreg); + int nblocks = get_nblocks (reg_mode); + poly_int64 unit_size = REGMODE_NATURAL_SIZE (reg_mode); + poly_int64 subreg_size = GET_MODE_SIZE (subreg_mode); + poly_int64 left = offset + subreg_size; + + int subreg_start = -1; + int subreg_nblocks = -1; + for (int i = 0; i < nblocks; i += 1) + { + poly_int64 right = unit_size * (i + 1); + if (subreg_start < 0 && maybe_lt (offset, right)) + subreg_start = i; + if (subreg_nblocks < 0 && maybe_le (left, right)) + { + subreg_nblocks = i + 1 - subreg_start; + break; + } + } + gcc_assert (subreg_start >= 0 && subreg_nblocks > 0); + + bitmap_set_range (range, subreg_start, subreg_nblocks); +} + +/* Remove RANGE from BB_INFO's use data. */ +void +remove_subreg_range (df_live_subreg_local_bb_info *bb_info, unsigned int regno, + const_sbitmap range) +{ + bitmap partial = &bb_info->partial_use; + + bb_info->range_use->remove_range (regno, range); + if (bb_info->range_use->empty_p (regno)) + bitmap_clear_bit (partial, regno); +} + +/* Remove RANGE of REF from BB_INFO's use data. */ +static void +remove_subreg_range (df_live_subreg_local_bb_info *bb_info, df_ref ref) +{ + unsigned int regno = DF_REF_REGNO (ref); + machine_mode reg_mode = GET_MODE (DF_REF_REAL_REG (ref)); + + if (need_track_subreg_p (regno)) + { + auto_sbitmap range (get_nblocks (reg_mode)); + init_range (DF_REF_REG (ref), range); + remove_subreg_range (bb_info, regno, range); + } + else + { + bitmap_clear_bit (&bb_info->full_use, regno); + gcc_assert (!bitmap_bit_p (&bb_info->partial_use, regno)); + gcc_assert (!bitmap_bit_p (&bb_info->partial_def, regno)); + } +} + +/* add RANGE to BB_INFO's def/use. If is_def is true, means for BB_INFO's def, + otherwise for BB_INFO's use. */ +void +add_subreg_range (df_live_subreg_local_bb_info *bb_info, unsigned int regno, + const_sbitmap range, bool is_def) +{ + bitmap partial = is_def ? &bb_info->partial_def : &bb_info->partial_use; + subregs_live *range_live = is_def ? bb_info->range_def : bb_info->range_use; + + bitmap_set_bit (partial, regno); + range_live->add_range (regno, range); +} + +/* add RANGE of REF to BB_INFO def/use. If is_def is true, means for BB_INFO's + def, otherwise for BB_INFO's use. */ +static void +add_subreg_range (df_live_subreg_local_bb_info *bb_info, df_ref ref, + bool is_def) +{ + unsigned int regno = DF_REF_REGNO (ref); + machine_mode reg_mode = GET_MODE (DF_REF_REAL_REG (ref)); + + if (need_track_subreg_p (regno)) + { + auto_sbitmap range (get_nblocks (reg_mode)); + init_range (DF_REF_REG (ref), range); + add_subreg_range (bb_info, regno, range, is_def); + } + else + { + bitmap full = is_def ? &bb_info->full_def : &bb_info->full_use; + bitmap partial = is_def ? &bb_info->partial_def : &bb_info->partial_use; + bitmap_set_bit (full, regno); + gcc_assert (!bitmap_bit_p (partial, regno)); + } +} + +/* Free basic block info. */ + +static void +df_live_subreg_free_bb_info (basic_block bb ATTRIBUTE_UNUSED, void *vbb_info) +{ + df_live_subreg_bb_info *bb_info = (df_live_subreg_bb_info *) vbb_info; + if (bb_info) + { + delete bb_info->range_in; + bb_info->range_in = NULL; + delete bb_info->range_out; + bb_info->range_out = NULL; + + bitmap_clear (&bb_info->all_in); + bitmap_clear (&bb_info->full_in); + bitmap_clear (&bb_info->partial_in); + bitmap_clear (&bb_info->all_out); + bitmap_clear (&bb_info->full_out); + bitmap_clear (&bb_info->partial_out); + } +} + +/* Allocate or reset bitmaps for DF_LIVE_SUBREG blocks. The solution bits are + not touched unless the block is new. */ + +static void +df_live_subreg_alloc (bitmap all_blocks ATTRIBUTE_UNUSED) +{ + struct df_live_subreg_problem_data *problem_data; + df_grow_bb_info (df_live_subreg); + if (df_live_subreg->problem_data) + problem_data + = (struct df_live_subreg_problem_data *) df_live_subreg->problem_data; + else + { + problem_data = XNEW (struct df_live_subreg_problem_data); + df_live_subreg->problem_data = problem_data; + + bitmap_obstack_initialize (&problem_data->live_subreg_bitmaps); + bitmap_initialize (&problem_data->tracked_regs, + &problem_data->live_subreg_bitmaps); + } + + bitmap_clear (&problem_data->tracked_regs); + + basic_block bb; + FOR_EACH_BB_FN (bb, cfun) + bitmap_set_bit (df_live_subreg->out_of_date_transfer_functions, bb->index); + + bitmap_set_bit (df_live_subreg->out_of_date_transfer_functions, ENTRY_BLOCK); + bitmap_set_bit (df_live_subreg->out_of_date_transfer_functions, EXIT_BLOCK); + + unsigned int bb_index; + bitmap_iterator bi; + EXECUTE_IF_SET_IN_BITMAP (df_live_subreg->out_of_date_transfer_functions, 0, + bb_index, bi) + { + /* Find the regs which we need to track it's subreg liveness. */ + rtx_insn *insn; + df_ref use; + FOR_BB_INSNS (bb, insn) + { + if (!NONDEBUG_INSN_P (insn)) + continue; + + df_insn_info *insn_info = DF_INSN_INFO_GET (insn); + + FOR_EACH_INSN_INFO_USE (use, insn_info) + { + unsigned int regno = DF_REF_REGNO (use); + /* A multireg which is used via subreg pattern. */ + if (multireg_p (regno) + && DF_REF_FLAGS (use) & (DF_REF_SUBREG)) + bitmap_set_bit (&problem_data->tracked_regs, regno); + } + } + } + + if (dump_file) + { + fprintf (dump_file, ";; regs need to be tracked subreg liveness: "); + df_print_regset (dump_file, &problem_data->tracked_regs); + } + + size_t n = bitmap_count_bits (&problem_data->tracked_regs); + + EXECUTE_IF_SET_IN_BITMAP (df_live_subreg->out_of_date_transfer_functions, 0, + bb_index, bi) + { + /* Clean global infos. */ + df_live_subreg_bb_info *bb_info = df_live_subreg_get_bb_info (bb_index); + + /* When bitmaps are already initialized, just clear them. */ + if (bb_info->all_in.obstack) + { + bitmap_clear (&bb_info->all_in); + bitmap_clear (&bb_info->full_in); + bitmap_clear (&bb_info->partial_in); + bitmap_clear (&bb_info->all_out); + bitmap_clear (&bb_info->full_out); + bitmap_clear (&bb_info->partial_out); + } + else + { + bitmap_initialize (&bb_info->all_in, + &problem_data->live_subreg_bitmaps); + bitmap_initialize (&bb_info->full_in, + &problem_data->live_subreg_bitmaps); + bitmap_initialize (&bb_info->partial_in, + &problem_data->live_subreg_bitmaps); + bitmap_initialize (&bb_info->all_out, + &problem_data->live_subreg_bitmaps); + bitmap_initialize (&bb_info->full_out, + &problem_data->live_subreg_bitmaps); + bitmap_initialize (&bb_info->partial_out, + &problem_data->live_subreg_bitmaps); + } + + if (bb_info->range_in) + { + bb_info->range_in->clear (n); + bb_info->range_out->clear (n); + } + else + { + bb_info->range_in = new subregs_live (n); + bb_info->range_out = new subregs_live (n); + } + + /* Clean local infos. */ + df_live_subreg_local_bb_info *local_bb_info + = get_live_subreg_local_bb_info (bb_index); + + /* When bitmaps are already initialized, just clear them. */ + if (local_bb_info->full_use.obstack) + { + bitmap_clear (&local_bb_info->full_def); + bitmap_clear (&local_bb_info->partial_def); + bitmap_clear (&local_bb_info->full_use); + bitmap_clear (&local_bb_info->partial_use); + } + else + { + bitmap_initialize (&local_bb_info->full_def, + &problem_data->live_subreg_bitmaps); + bitmap_initialize (&local_bb_info->partial_def, + &problem_data->live_subreg_bitmaps); + bitmap_initialize (&local_bb_info->full_use, + &problem_data->live_subreg_bitmaps); + bitmap_initialize (&local_bb_info->partial_use, + &problem_data->live_subreg_bitmaps); + } + + if (local_bb_info->range_def) + { + local_bb_info->range_def->clear (n); + local_bb_info->range_use->clear (n); + } + else + { + local_bb_info->range_def = new subregs_live (n); + local_bb_info->range_use = new subregs_live (n); + } + } + + df_live_subreg->optional_p = true; +} + +/* Reset the global solution for recalculation. */ + +static void +df_live_subreg_reset (bitmap all_blocks) +{ + unsigned int bb_index; + bitmap_iterator bi; + + EXECUTE_IF_SET_IN_BITMAP (all_blocks, 0, bb_index, bi) + { + df_live_subreg_bb_info *bb_info = df_live_subreg_get_bb_info (bb_index); + gcc_assert (bb_info); + bitmap_clear (&bb_info->all_in); + bitmap_clear (&bb_info->full_in); + bitmap_clear (&bb_info->partial_in); + bitmap_clear (&bb_info->all_out); + bitmap_clear (&bb_info->full_out); + bitmap_clear (&bb_info->partial_out); + bb_info->range_in->clear (); + bb_info->range_out->clear (); + } +} + +/* Compute local live register info for basic block BB. */ + +static void +df_live_subreg_bb_local_compute (unsigned int bb_index) +{ + basic_block bb = BASIC_BLOCK_FOR_FN (cfun, bb_index); + df_live_subreg_local_bb_info *local_bb_info + = get_live_subreg_local_bb_info (bb_index); + + rtx_insn *insn; + df_ref def, use; + + /* Process the registers set in an exception handler. */ + FOR_EACH_ARTIFICIAL_DEF (def, bb_index) + if ((DF_REF_FLAGS (def) & DF_REF_AT_TOP) == 0) + { + add_subreg_range (local_bb_info, def, true); + remove_subreg_range (local_bb_info, def); + } + + /* Process the hardware registers that are always live. */ + FOR_EACH_ARTIFICIAL_USE (use, bb_index) + if ((DF_REF_FLAGS (use) & DF_REF_AT_TOP) == 0) + add_subreg_range (local_bb_info, use, false); + + FOR_BB_INSNS_REVERSE (bb, insn) + { + if (!NONDEBUG_INSN_P (insn)) + continue; + + df_insn_info *insn_info = DF_INSN_INFO_GET (insn); + FOR_EACH_INSN_INFO_DEF (def, insn_info) + { + remove_subreg_range (local_bb_info, def); + add_subreg_range (local_bb_info, def, true); + } + + FOR_EACH_INSN_INFO_USE (use, insn_info) + { + unsigned int regno = DF_REF_REGNO (use); + /* Ignore the use of subreg which is used as dest operand. */ + if (need_track_subreg_p (regno) + && DF_REF_FLAGS (use) & (DF_REF_READ_WRITE | DF_REF_SUBREG)) + continue; + add_subreg_range (local_bb_info, use, false); + } + } + + /* Process the registers set in an exception handler or the hard + frame pointer if this block is the target of a non local + goto. */ + FOR_EACH_ARTIFICIAL_DEF (def, bb_index) + if (DF_REF_FLAGS (def) & DF_REF_AT_TOP) + { + add_subreg_range (local_bb_info, def, true); + remove_subreg_range (local_bb_info, def); + } + +#ifdef EH_USES + /* Process the uses that are live into an exception handler. */ + FOR_EACH_ARTIFICIAL_USE (use, bb_index) + /* Add use to set of uses in this BB. */ + if (DF_REF_FLAGS (use) & DF_REF_AT_TOP) + add_subreg_range (local_bb_info, use, false); +#endif +} + +/* Compute local live register info for each basic block within BLOCKS. */ + +static void +df_live_subreg_local_compute (bitmap all_blocks ATTRIBUTE_UNUSED) +{ + unsigned int bb_index, i; + bitmap_iterator bi; + + bitmap_clear (&df->hardware_regs_used); + + /* The all-important stack pointer must always be live. */ + bitmap_set_bit (&df->hardware_regs_used, STACK_POINTER_REGNUM); + + /* Global regs are always live, too. */ + for (i = 0; i < FIRST_PSEUDO_REGISTER; i++) + if (global_regs[i]) + bitmap_set_bit (&df->hardware_regs_used, i); + + /* Before reload, there are a few registers that must be forced + live everywhere -- which might not already be the case for + blocks within infinite loops. */ + if (!reload_completed) + { + unsigned int pic_offset_table_regnum = PIC_OFFSET_TABLE_REGNUM; + /* Any reference to any pseudo before reload is a potential + reference of the frame pointer. */ + bitmap_set_bit (&df->hardware_regs_used, FRAME_POINTER_REGNUM); + + /* Pseudos with argument area equivalences may require + reloading via the argument pointer. */ + if (FRAME_POINTER_REGNUM != ARG_POINTER_REGNUM + && fixed_regs[ARG_POINTER_REGNUM]) + bitmap_set_bit (&df->hardware_regs_used, ARG_POINTER_REGNUM); + + /* Any constant, or pseudo with constant equivalences, may + require reloading from memory using the pic register. */ + if (pic_offset_table_regnum != INVALID_REGNUM + && fixed_regs[pic_offset_table_regnum]) + bitmap_set_bit (&df->hardware_regs_used, pic_offset_table_regnum); + } + + EXECUTE_IF_SET_IN_BITMAP (df_live_subreg->out_of_date_transfer_functions, 0, + bb_index, bi) + { + if (bb_index == EXIT_BLOCK) + { + /* The exit block is special for this problem and its bits are + computed from thin air. */ + df_live_subreg_local_bb_info *local_bb_info + = get_live_subreg_local_bb_info (EXIT_BLOCK); + bitmap_copy (&local_bb_info->full_use, df->exit_block_uses); + } + else + df_live_subreg_bb_local_compute (bb_index); + } + + bitmap_clear (df_live_subreg->out_of_date_transfer_functions); +} + +/* Initialize the solution vectors. */ + +static void +df_live_subreg_init (bitmap all_blocks) +{ + unsigned int bb_index; + bitmap_iterator bi; + + EXECUTE_IF_SET_IN_BITMAP (all_blocks, 0, bb_index, bi) + { + df_live_subreg_bb_info *bb_info = df_live_subreg_get_bb_info (bb_index); + df_live_subreg_local_bb_info *local_bb_info + = get_live_subreg_local_bb_info (bb_index); + bitmap_copy (&bb_info->full_in, &local_bb_info->full_use); + bitmap_copy (&bb_info->partial_in, &local_bb_info->partial_use); + bb_info->range_in->copy_lives (*local_bb_info->range_use); + bitmap_clear (&bb_info->full_out); + bitmap_clear (&bb_info->partial_out); + bb_info->range_out->clear (); + } +} + +/* Check that the status of the final result is correct. */ + +void +df_live_subreg_check_result (bitmap full, bitmap partial, + subregs_live *partial_live) +{ + unsigned int regno; + bitmap_iterator bi; + /* Make sure that full and partial do not overlap. */ + gcc_assert (!bitmap_intersect_p (full, partial)); + /* Ensure that registers belonging to full are not partially live. */ + EXECUTE_IF_SET_IN_BITMAP (full, 0, regno, bi) + gcc_assert (partial_live->empty_p (regno)); + /* Ensure that registers belonging to partial are partially live. */ + EXECUTE_IF_SET_IN_BITMAP (partial, 0, regno, bi) + gcc_assert (!partial_live->empty_p (regno)); +} + +/* Confluence function that processes infinite loops. This might be a + noreturn function that throws. And even if it isn't, getting the + unwind info right helps debugging. */ +static void +df_live_subreg_confluence_0 (basic_block bb) +{ + bitmap full_out = &df_live_subreg_get_bb_info (bb->index)->full_out; + if (bb != EXIT_BLOCK_PTR_FOR_FN (cfun)) + bitmap_copy (full_out, &df->hardware_regs_used); +} + +/* Confluence function that ignores fake edges. */ + +static bool +df_live_subreg_confluence_n (edge e) +{ + class df_live_subreg_bb_info *src_bb_info + = df_live_subreg_get_bb_info (e->src->index); + class df_live_subreg_bb_info *dest_bb_info + = df_live_subreg_get_bb_info (e->dest->index); + + bool changed = false; + + /* Call-clobbered registers die across exception and call edges. + Conservatively treat partially-clobbered registers as surviving + across the edges; they might or might not, depending on what + mode they have. */ + /* ??? Abnormal call edges ignored for the moment, as this gets + confused by sibling call edges, which crashes reg-stack. */ + if (e->flags & EDGE_EH) + { + bitmap_view eh_kills (eh_edge_abi.full_reg_clobbers ()); + changed = bitmap_ior_and_compl_into (&src_bb_info->full_out, + &dest_bb_info->full_in, eh_kills); + } + else + changed = bitmap_ior_into (&src_bb_info->full_out, &dest_bb_info->full_in); + + changed |= bitmap_ior_into (&src_bb_info->full_out, &df->hardware_regs_used); + + /* Handle partial live case. */ + if (!bitmap_empty_p (&dest_bb_info->partial_in)) + { + unsigned int regno; + bitmap_iterator bi; + EXECUTE_IF_SET_IN_BITMAP (&dest_bb_info->partial_in, + FIRST_PSEUDO_REGISTER, regno, bi) + { + sbitmap dest_range = dest_bb_info->range_in->get_range (regno); + changed |= src_bb_info->range_out->add_range (regno, dest_range); + } + changed |= bitmap_ior_into (&src_bb_info->partial_out, + &dest_bb_info->partial_in); + + df_live_subreg_check_result (&src_bb_info->full_out, + &src_bb_info->partial_out, + src_bb_info->range_out); + } + + return changed; +} + +/* Transfer function. IN = USE | (OUT & ~DEF). */ + +static bool +df_live_subreg_transfer_function (int bb_index) +{ + class df_live_subreg_bb_info *bb_info = df_live_subreg_get_bb_info (bb_index); + class df_live_subreg_local_bb_info *local_bb_info + = get_live_subreg_local_bb_info (bb_index); + + bool changed = false; + + changed + |= bitmap_ior_and_compl (&bb_info->full_in, &local_bb_info->full_use, + &bb_info->full_out, &local_bb_info->full_def); + + /* Handle partial live case. */ + if (!bitmap_empty_p (&bb_info->partial_out) + || !bitmap_empty_p (&local_bb_info->partial_use)) + { + unsigned int regno; + bitmap_iterator bi; + bitmap_head temp_partial_out; + subregs_live temp_range_out; + + /* TEMP = (OUT & ~DEF) */ + bitmap_initialize (&temp_partial_out, &bitmap_default_obstack); + EXECUTE_IF_SET_IN_BITMAP (&bb_info->partial_out, FIRST_PSEUDO_REGISTER, + regno, bi) + { + sbitmap out_range = bb_info->range_out->get_range (regno); + temp_range_out.add_range (regno, out_range); + if (bitmap_bit_p (&local_bb_info->partial_def, regno)) + { + sbitmap def_range = local_bb_info->range_def->get_range (regno); + temp_range_out.remove_range (regno, def_range); + if (!temp_range_out.empty_p (regno)) + bitmap_set_bit (&temp_partial_out, regno); + } + else + bitmap_set_bit (&temp_partial_out, regno); + } + + /* TEMP = USE | TEMP */ + EXECUTE_IF_SET_IN_BITMAP (&local_bb_info->partial_use, + FIRST_PSEUDO_REGISTER, regno, bi) + { + sbitmap use_range = local_bb_info->range_use->get_range (regno); + temp_range_out.add_range (regno, use_range); + } + bitmap_ior_into (&temp_partial_out, &local_bb_info->partial_use); + + /* IN = TEMP */ + changed |= bb_info->range_in->copy_lives (temp_range_out); + bitmap_copy (&bb_info->partial_in, &temp_partial_out); + df_live_subreg_check_result (&bb_info->full_in, &bb_info->partial_in, + bb_info->range_in); + } + else if (!bitmap_empty_p (&bb_info->partial_in)) + { + changed = true; + bitmap_clear (&bb_info->partial_in); + bb_info->range_in->clear (); + } + + return changed; +} + +/* Calculating ALL_IN from FULL_IN and PARTIAL_IN + and ALL_OUT from FULL_OUT and PARTIAL_OUT. */ + +void +df_live_subreg_finalize (bitmap all_blocks) +{ + unsigned int bb_index; + bitmap_iterator bi; + EXECUTE_IF_SET_IN_BITMAP (all_blocks, 0, bb_index, bi) + { + class df_live_subreg_bb_info *bb_info + = df_live_subreg_get_bb_info (bb_index); + gcc_assert (bb_info); + bitmap_ior (&bb_info->all_in, &bb_info->full_in, &bb_info->partial_in); + bitmap_ior (&bb_info->all_out, &bb_info->full_out, &bb_info->partial_out); + + /* Move full live reg in partial_in/partial_out to full_in/full_out. */ + unsigned int regno; + bitmap_iterator ri; + bool changed = false; + EXECUTE_IF_SET_IN_BITMAP (&bb_info->partial_in, FIRST_PSEUDO_REGISTER, + regno, ri) + { + if (bb_info->range_in->full_p (regno)) + { + bitmap_set_bit (&bb_info->full_in, regno); + bb_info->range_in->remove_range (regno); + changed = true; + } + } + if (changed) + bitmap_and_compl_into (&bb_info->partial_in, &bb_info->full_in); + + changed = false; + EXECUTE_IF_SET_IN_BITMAP (&bb_info->partial_out, FIRST_PSEUDO_REGISTER, + regno, ri) + { + if (bb_info->range_out->full_p (regno)) + { + bitmap_set_bit (&bb_info->full_out, regno); + bb_info->range_out->remove_range (regno); + changed = true; + } + } + if (changed) + bitmap_and_compl_into (&bb_info->partial_out, &bb_info->full_out); + } +} + +/* Free all storage associated with the problem. */ + +static void +df_live_subreg_free (void) +{ + df_live_subreg_problem_data *problem_data + = (df_live_subreg_problem_data *) df_live_subreg->problem_data; + if (df_live_subreg->block_info) + { + bitmap_obstack_release (&problem_data->live_subreg_bitmaps); + basic_block bb; + FOR_ALL_BB_FN (bb, cfun) + { + df_live_subreg_bb_info *bb_info + = df_live_subreg_get_bb_info (bb->index); + df_live_subreg_local_bb_info *local_bb_info + = get_live_subreg_local_bb_info (bb->index); + delete bb_info->range_in; + delete bb_info->range_out; + delete local_bb_info->range_def; + delete local_bb_info->range_use; + } + free (problem_data); + df_live_subreg->problem_data = NULL; + + df_live_subreg->block_info_size = 0; + free (df_live_subreg->block_info); + df_live_subreg->block_info = NULL; + } + + BITMAP_FREE (df_live_subreg->out_of_date_transfer_functions); + free (df_live_subreg); +} + +/* Debugging info at top of bb. */ + +static void +df_live_subreg_top_dump (basic_block bb, FILE *file) +{ + df_live_subreg_bb_info *bb_info = df_live_subreg_get_bb_info (bb->index); + df_live_subreg_local_bb_info *local_bb_info + = get_live_subreg_local_bb_info (bb->index); + + if (!bb_info) + return; + + fprintf (file, ";; subreg live all in \t"); + df_print_regset (file, &bb_info->all_in); + fprintf (file, ";; subreg live full in \t"); + df_print_regset (file, &bb_info->full_in); + fprintf (file, ";; subreg live partial in \t"); + df_print_regset (file, &bb_info->partial_in); + fprintf (file, ";; subreg live range in \t"); + bb_info->range_in->dump (file, ""); + + fprintf (file, "\n;; subreg live full use \t"); + df_print_regset (file, &local_bb_info->full_use); + fprintf (file, ";; subreg live partial use \t"); + df_print_regset (file, &local_bb_info->partial_use); + fprintf (file, ";; subreg live range use \t"); + local_bb_info->range_use->dump (file, ""); + + fprintf (file, "\n;; subreg live full def \t"); + df_print_regset (file, &local_bb_info->full_def); + fprintf (file, ";; subreg live partial def \t"); + df_print_regset (file, &local_bb_info->partial_def); + fprintf (file, ";; subreg live range def \t"); + local_bb_info->range_def->dump (file, ""); +} + +/* Debugging info at bottom of bb. */ + +static void +df_live_subreg_bottom_dump (basic_block bb, FILE *file) +{ + df_live_subreg_bb_info *bb_info = df_live_subreg_get_bb_info (bb->index); + if (!bb_info) + return; + + fprintf (file, ";; subreg live all out \t"); + df_print_regset (file, &bb_info->all_out); + fprintf (file, ";; subreg live full out \t"); + df_print_regset (file, &bb_info->full_out); + fprintf (file, ";; subreg live partial out \t"); + df_print_regset (file, &bb_info->partial_out); + fprintf (file, ";; subreg live range out \t"); + bb_info->range_out->dump (file, ""); +} + +/* All of the information associated with every instance of the problem. */ + +static const struct df_problem problem_LIVE_SUBREG = { + DF_LIVE_SUBREG, /* Problem id. */ + DF_BACKWARD, /* Direction. */ + df_live_subreg_alloc, /* Allocate the problem specific data. */ + df_live_subreg_reset, /* Reset global information. */ + df_live_subreg_free_bb_info, /* Free basic block info. */ + df_live_subreg_local_compute, /* Local compute function. */ + df_live_subreg_init, /* Init the solution specific data. */ + df_worklist_dataflow, /* Worklist solver. */ + df_live_subreg_confluence_0, /* Confluence operator 0. */ + df_live_subreg_confluence_n, /* Confluence operator n. */ + df_live_subreg_transfer_function, /* Transfer function. */ + df_live_subreg_finalize, /* Finalize function. */ + df_live_subreg_free, /* Free all of the problem information. */ + df_live_subreg_free, /* Remove this problem from the stack of dataflow + problems. */ + NULL, /* Debugging. */ + df_live_subreg_top_dump, /* Debugging start block. */ + df_live_subreg_bottom_dump, /* Debugging end block. */ + NULL, /* Debugging start insn. */ + NULL, /* Debugging end insn. */ + NULL, /* Incremental solution verify start. */ + NULL, /* Incremental solution verify end. */ + &problem_LR, /* Dependent problem. */ + sizeof (df_live_subreg_bb_info), /* Size of entry of block_info array. */ + TV_DF_LIVE_SUBREG, /* Timing variable. */ + false /* Reset blocks on dropping out of blocks_to_analyze. */ +}; + +/* Create a new DATAFLOW instance and add it to an existing instance + of DF. The returned structure is what is used to get at the + solution. */ + +void +df_live_subreg_add_problem (void) +{ + gcc_assert (flag_track_subreg_liveness); + + df_add_problem (&problem_LIVE_SUBREG); + + /* These will be initialized when df_scan_blocks processes each + block. */ + df_live_subreg->out_of_date_transfer_functions + = BITMAP_ALLOC (&df_bitmap_obstack); +} - /*---------------------------------------------------------------------------- LIVE AND MAY-INITIALIZED REGISTERS. diff --git a/gcc/df.h b/gcc/df.h index 84e5aa8b524..78c57273cf1 100644 --- a/gcc/df.h +++ b/gcc/df.h @@ -47,6 +47,7 @@ enum df_problem_id { DF_SCAN, DF_LR, /* Live Registers backward. */ + DF_LIVE_SUBREG, /* Live Regs and Subregs */ DF_LIVE, /* Live Registers & Uninitialized Registers */ DF_RD, /* Reaching Defs. */ DF_CHAIN, /* Def-Use and/or Use-Def Chains. */ @@ -878,6 +879,33 @@ public: bitmap_head out; /* At the bottom of the block. */ }; +class subregs_live; /* Defined in subreg-range.{h,cc} */ + +/* Local live info in basic block. Use by live_subreg problem and LRA pass. */ +class df_live_subreg_local_bb_info +{ +public: + bitmap_head full_def; + bitmap_head full_use; + /* Only for pseudo registers. */ + bitmap_head partial_def; + bitmap_head partial_use; + subregs_live *range_def = nullptr; + subregs_live *range_use = nullptr; +}; + +/* Live in/out infos of each basic. */ +class df_live_subreg_bb_info : public df_live_subreg_local_bb_info +{ +public: + bitmap_head all_in, full_in; + bitmap_head all_out, full_out; + /* Only for pseudo registers. */ + bitmap_head partial_in; + bitmap_head partial_out; + subregs_live *range_in = nullptr; + subregs_live *range_out = nullptr; +}; /* Uninitialized registers. All bitmaps are referenced by the register number. Anded results of the forwards and backward live @@ -940,6 +968,7 @@ extern class df_d *df; #define df_scan (df->problems_by_index[DF_SCAN]) #define df_rd (df->problems_by_index[DF_RD]) #define df_lr (df->problems_by_index[DF_LR]) +#define df_live_subreg (df->problems_by_index[DF_LIVE_SUBREG]) #define df_live (df->problems_by_index[DF_LIVE]) #define df_chain (df->problems_by_index[DF_CHAIN]) #define df_word_lr (df->problems_by_index[DF_WORD_LR]) @@ -1031,6 +1060,8 @@ extern void df_lr_add_problem (void); extern void df_lr_verify_transfer_functions (void); extern void df_live_verify_transfer_functions (void); extern void df_live_add_problem (void); +extern void df_live_subreg_add_problem (); +extern void df_live_subreg_finalize (bitmap all_blocks); extern void df_live_set_all_dirty (void); extern void df_chain_add_problem (unsigned int); extern void df_word_lr_add_problem (void); @@ -1059,6 +1090,16 @@ extern bool can_move_insns_across (rtx_insn *, rtx_insn *, rtx_insn *, rtx_insn *, basic_block, regset, regset, rtx_insn **); +extern void +df_live_subreg_check_result (bitmap, bitmap, subregs_live *); +extern bool multireg_p (int); +extern void init_range (rtx, sbitmap); +extern void +add_subreg_range (df_live_subreg_local_bb_info *, unsigned int, const_sbitmap, + bool); +extern void +remove_subreg_range (df_live_subreg_local_bb_info *, unsigned int, + const_sbitmap); /* Functions defined in df-scan.cc. */ extern void df_scan_alloc (bitmap); @@ -1192,6 +1233,120 @@ df_get_live_in (basic_block bb) return DF_LR_IN (bb); } +/* Get the subreg live at in set for BB. The live set include full and partial + * live. we only track and use subreg liveness when -ftrack-subreg-liveness, + * otherwise use DF_LR_IN. This function is used by the register allocators. */ + +inline bitmap +df_get_subreg_live_in (basic_block bb) +{ + if (flag_track_subreg_liveness) + { + df_live_subreg_bb_info *bb_info = &( + (class df_live_subreg_bb_info *) df_live_subreg->block_info)[bb->index]; + return &(bb_info->all_in); + } + return DF_LR_IN (bb); +} + +/* Get the subreg live at out set for BB. The live set include full and + * partial live. we only track and use subreg liveness when + * -ftrack-subreg-liveness, otherwise use DF_LR_OUT. This function is used by + * the register allocators. */ + +inline bitmap +df_get_subreg_live_out (basic_block bb) +{ + if (flag_track_subreg_liveness) + { + df_live_subreg_bb_info *bb_info = &( + (class df_live_subreg_bb_info *) df_live_subreg->block_info)[bb->index]; + return &(bb_info->all_out); + } + return DF_LR_OUT (bb); +} + +/* Get the subreg live at in set for BB. The live set only include full and + * partial live. we only track and use subreg liveness when + * -ftrack-subreg-liveness, otherwise use DF_LR_OUT. This function is used by + * the register allocators. */ + +inline bitmap +df_get_subreg_live_full_in (basic_block bb) +{ + if (flag_track_subreg_liveness) + { + df_live_subreg_bb_info *bb_info = &( + (class df_live_subreg_bb_info *) df_live_subreg->block_info)[bb->index]; + return &(bb_info->full_in); + } + return DF_LR_IN (bb); +} + +inline bitmap +df_get_subreg_live_full_out (basic_block bb) +{ + if (flag_track_subreg_liveness) + { + df_live_subreg_bb_info *bb_info = &( + (class df_live_subreg_bb_info *) df_live_subreg->block_info)[bb->index]; + return &(bb_info->full_out); + } + return DF_LR_OUT (bb); +} + +/* Define in df-problems.cc, used when disable track-subreg-liveness. */ +extern bitmap_head empty_bitmap; +extern subregs_live empty_live; + +inline bitmap +df_get_subreg_live_partial_in (basic_block bb) +{ + if (flag_track_subreg_liveness) + { + df_live_subreg_bb_info *bb_info = &( + (class df_live_subreg_bb_info *) df_live_subreg->block_info)[bb->index]; + return &(bb_info->partial_in); + } + return &empty_bitmap; +} + +inline bitmap +df_get_subreg_live_partial_out (basic_block bb) +{ + if (flag_track_subreg_liveness) + { + df_live_subreg_bb_info *bb_info = &( + (class df_live_subreg_bb_info *) df_live_subreg->block_info)[bb->index]; + return &(bb_info->partial_out); + } + return &empty_bitmap; +} + +inline subregs_live * +df_get_subreg_live_range_in (basic_block bb) +{ + if (flag_track_subreg_liveness) + { + df_live_subreg_bb_info *bb_info = &( + (class df_live_subreg_bb_info *) df_live_subreg->block_info)[bb->index]; + return bb_info->range_in; + } + return &empty_live; +} + +inline subregs_live * +df_get_subreg_live_range_out (basic_block bb) +{ + if (flag_track_subreg_liveness) + { + df_live_subreg_bb_info *bb_info = &( + (class df_live_subreg_bb_info *) df_live_subreg->block_info)[bb->index]; + return bb_info->range_out; + } + return &empty_live; +} + /* Get basic block info. */ /* Get the artificial defs for a basic block. */ diff --git a/gcc/regs.h b/gcc/regs.h index 16189c58fd1..3b524096533 100644 --- a/gcc/regs.h +++ b/gcc/regs.h @@ -389,4 +389,9 @@ range_in_hard_reg_set_p (const_hard_reg_set set, unsigned regno, int nregs) return true; } +/* Return the number of blocks the MODE overlap. One block equal to mode's + natural size. */ +#define get_nblocks(mode) \ + (exact_div (GET_MODE_SIZE (mode), REGMODE_NATURAL_SIZE (mode)).to_constant ()) + #endif /* GCC_REGS_H */ diff --git a/gcc/sbitmap.cc b/gcc/sbitmap.cc index 03bb2c6d44b..5d960adcabe 100644 --- a/gcc/sbitmap.cc +++ b/gcc/sbitmap.cc @@ -208,6 +208,29 @@ bitmap_empty_p (const_sbitmap bmap) return true; } +/* Return true if the bitmap is full, i.e. all bits are set. */ + +bool +bitmap_full_p (const_sbitmap bmap) +{ + unsigned int end_word = bmap->n_bits / SBITMAP_ELT_BITS; + unsigned int i = 0; + for (; i < end_word; i++) + if (bmap->elms[i] != (SBITMAP_ELT_TYPE) -1) + return false; + + unsigned int end_bitno = bmap->n_bits % SBITMAP_ELT_BITS; + + if (end_bitno == 0) + return true; + + gcc_checking_assert (i + 1 == bmap->size); + + SBITMAP_ELT_TYPE mask = ((SBITMAP_ELT_TYPE) 1 << end_bitno) - 1; + /* Make sure the tail bits are set. */ + return (bmap->elms[i] & mask) == mask; +} + /* Clear COUNT bits from START in BMAP. */ void @@ -662,6 +685,36 @@ bitmap_subset_p (const_sbitmap a, const_sbitmap b) return true; } +/* Return true if the bits in A and B are set the same and the number of bits is + the same. */ + +bool +bitmap_same_p (const_sbitmap a, const_sbitmap b) +{ + if (a->n_bits != b->n_bits) + return false; + + gcc_checking_assert (a->size == b->size); + + unsigned int end_word = a->n_bits / SBITMAP_ELT_BITS; + unsigned int i = 0; + for (; i < end_word; i++) + if (a->elms[i] != b->elms[i]) + return false; + + unsigned int end_bitno = a->n_bits % SBITMAP_ELT_BITS; + + if (end_bitno == 0) + return true; + + gcc_checking_assert (i + 1 == a->size); + + SBITMAP_ELT_TYPE mask = ((SBITMAP_ELT_TYPE) 1 << end_bitno) - 1; + + /* Make sure the tail bits are same. */ + return (a->elms[i] & mask) == (b->elms[i] & mask); +} + /* Set DST to be (A or (B and C)). Return nonzero if any change is made. */ @@ -994,6 +1047,49 @@ test_bit_in_range () sbitmap_free (s); } +/* Verify bitmap_full_p functions for sbitmap. */ + +static void +test_full () +{ + sbitmap s = sbitmap_alloc (193); + + bitmap_clear (s); + ASSERT_FALSE (bitmap_full_p (s)); + + bitmap_ones (s); + ASSERT_TRUE (bitmap_full_p (s)); + + bitmap_clear_bit (s, 192); + ASSERT_FALSE (bitmap_full_p (s)); + + bitmap_ones (s); + bitmap_clear_bit (s, 17); + ASSERT_FALSE (bitmap_full_p (s)); +} + +/* Verify bitmap_same_p functions for sbitmap. */ + +static void +test_same () +{ + sbitmap s1 = sbitmap_alloc (193); + sbitmap s2 = sbitmap_alloc (193); + sbitmap s3 = sbitmap_alloc (192); + + ASSERT_FALSE (bitmap_same_p (s1, s3)); + + bitmap_clear (s1); + bitmap_clear (s2); + ASSERT_TRUE (bitmap_same_p (s1, s2)); + + bitmap_set_bit (s2, 192); + ASSERT_FALSE (bitmap_same_p (s1, s2)); + + bitmap_set_bit (s1, 192); + ASSERT_TRUE (bitmap_same_p (s1, s2)); +} + /* Run all of the selftests within this file. */ void @@ -1001,6 +1097,8 @@ sbitmap_cc_tests () { test_set_range (); test_bit_in_range (); + test_full (); + test_same (); } } // namespace selftest diff --git a/gcc/sbitmap.h b/gcc/sbitmap.h index da6116ce925..71cfded9fb2 100644 --- a/gcc/sbitmap.h +++ b/gcc/sbitmap.h @@ -267,6 +267,7 @@ extern void bitmap_copy (sbitmap, const_sbitmap); extern bool bitmap_equal_p (const_sbitmap, const_sbitmap); extern unsigned int bitmap_count_bits (const_sbitmap); extern bool bitmap_empty_p (const_sbitmap); +extern bool bitmap_full_p (const_sbitmap); extern void bitmap_clear (sbitmap); extern void bitmap_clear_range (sbitmap, unsigned, unsigned); extern void bitmap_set_range (sbitmap, unsigned, unsigned); @@ -287,6 +288,7 @@ extern bool bitmap_and (sbitmap, const_sbitmap, const_sbitmap); extern bool bitmap_ior (sbitmap, const_sbitmap, const_sbitmap); extern bool bitmap_xor (sbitmap, const_sbitmap, const_sbitmap); extern bool bitmap_subset_p (const_sbitmap, const_sbitmap); +extern bool bitmap_same_p (const_sbitmap, const_sbitmap); extern bool bitmap_bit_in_range_p (const_sbitmap, unsigned int, unsigned int); extern int bitmap_first_set_bit (const_sbitmap); diff --git a/gcc/subreg-live-range.cc b/gcc/subreg-live-range.cc new file mode 100644 index 00000000000..fe8d4210eb6 --- /dev/null +++ b/gcc/subreg-live-range.cc @@ -0,0 +1,53 @@ +/* SUBREG liveness tracking classes for DF & IRA & LRA. + Copyright (C) 2023 Free Software Foundation, Inc. + Contributed by Lehua Ding (lehua.ding@rivai.ai), RiVAI Technologies Ltd. + + This file is part of GCC. + + GCC is free software; you can redistribute it and/or modify it + under the terms of the GNU General Public License as published by + the Free Software Foundation; either version 3, or (at your option) + any later version. + + GCC is distributed in the hope that it will be useful, but + WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + General Public License for more details. + + You should have received a copy of the GNU General Public License + along with GCC; see the file COPYING3. If not see + . */ + +#include "subreg-live-range.h" + +void +subregs_live::dump (FILE *file, const char *indent) const +{ + if (lives.empty ()) + { + fprintf (file, "%sempty\n", indent); + return; + } + fprintf (file, "%s", indent); + for (auto &kv : lives) + { + const_sbitmap range = kv.second; + if (bitmap_empty_p (range)) + continue; + fprintf (file, "%d: ", kv.first); + if (!bitmap_full_p (range)) + { + dump_bitmap_file (file, range); + fprintf (file, ", "); + } + else + fprintf (file, "full, "); + } + fprintf (file, "\n"); +} + +DEBUG_FUNCTION void +debug (const subregs_live &l) +{ + l.dump (stderr, ""); +} diff --git a/gcc/subreg-live-range.h b/gcc/subreg-live-range.h new file mode 100644 index 00000000000..c0b88071858 --- /dev/null +++ b/gcc/subreg-live-range.h @@ -0,0 +1,206 @@ +/* SUBREG liveness tracking classes for DF & IRA & LRA. + Copyright (C) 2023 Free Software Foundation, Inc. + Contributed by Lehua Ding (lehua.ding@rivai.ai), RiVAI Technologies Ltd. + + This file is part of GCC. + + GCC is free software; you can redistribute it and/or modify it + under the terms of the GNU General Public License as published by + the Free Software Foundation; either version 3, or (at your option) + any later version. + + GCC is distributed in the hope that it will be useful, but + WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + General Public License for more details. + + You should have received a copy of the GNU General Public License + along with GCC; see the file COPYING3. If not see + . */ + +#ifndef GCC_SUBREG_LIVE_RANGE_H +#define GCC_SUBREG_LIVE_RANGE_H + +#include "config.h" +#include "system.h" +#include "coretypes.h" +#include +#include "sbitmap.h" + +/* class subregs_live record the live subreg_ranges of registers. */ +class subregs_live +{ +public: + /* The key is usually the register's regno. */ + std::unordered_map lives; + + subregs_live () : lives () {} + + subregs_live (size_t n) : lives (n) {} + + ~subregs_live () + { + for (auto &kv : lives) + sbitmap_free (kv.second); + } + + void clear () + { + for (auto &kv : lives) + sbitmap_free (kv.second); + lives.clear (); + } + + void clear (size_t n) + { + clear (); + lives.rehash (n); + } + + bool find_p (unsigned int regno) const + { + return lives.find (regno) != lives.end (); + } + + sbitmap get_range (unsigned int regno) + { + gcc_assert (find_p (regno)); + return lives.at (regno); + } + + const_sbitmap get_range (unsigned int regno) const + { + gcc_assert (find_p (regno)); + return lives.at (regno); + } + + /* Added RANGE to regno's ranges. Return true if leads to a change. */ + bool add_range (unsigned int regno, const_sbitmap range) + { + if (find_p (regno)) + { + sbitmap curr = get_range (regno); + gcc_assert (range->n_bits == curr->n_bits); + return bitmap_ior (curr, curr, range); + } + else + { + sbitmap a = sbitmap_alloc (range->n_bits); + lives.insert ({regno, a}); + sbitmap curr = get_range (regno); + bitmap_copy (curr, range); + return !bitmap_empty_p (range); + } + } + /* Removed RANGE from regno's ranges. Return true if leads to a change. */ + bool remove_range (unsigned int regno, const_sbitmap range) + { + if (find_p (regno)) + { + sbitmap curr = get_range (regno); + bitmap_check_sizes (curr, range); + if (bitmap_subset_p (curr, range)) + return remove_range (regno); + + auto_sbitmap a (range->n_bits); + bitmap_not (a, range); + return bitmap_and (curr, curr, a); + } + return false; + } + /* Removed the whole range of REGNO. Return true if leads to a change. */ + bool remove_range (unsigned int regno) + { + if (find_p (regno)) + { + sbitmap curr = get_range (regno); + bool changed = !bitmap_empty_p (curr); + sbitmap_free (curr); + lives.erase (regno); + return changed; + } + return false; + } + /* Replace the range of REGNO with RANGE. Return true if leads to a change. + */ + bool replace_range (unsigned int regno, const_sbitmap range) + { + if (find_p (regno)) + { + sbitmap curr = get_range (regno); + if (!bitmap_same_p (curr, range)) + { + bitmap_copy (curr, range); + return true; + } + else + return false; + } + else + return add_range (regno, range); + } + /* Copy subregs_live SL. Return true if leads to a change. */ + bool copy_lives (const subregs_live &sl) + { + bool changed = false; + for (auto &kv : sl.lives) + { + unsigned int regno = kv.first; + const_sbitmap range = kv.second; + if (bitmap_empty_p (range)) + continue; + if (find_p (regno)) + changed |= replace_range (regno, range); + else + changed |= add_range (regno, range); + } + + for (auto it = lives.cbegin (); it != lives.cend ();) + { + unsigned int regno = it->first; + auto prev_it = it; + it++; + if (sl.empty_p (regno)) + { + changed |= bitmap_empty_p (it->second); + lives.erase (prev_it); + } + } + + return changed; + } + /* Added subregs_live SL. Return true if leads to a change. */ + bool add_lives (const subregs_live &sl) + { + bool changed = false; + for (auto &kv : sl.lives) + { + unsigned int regno = kv.first; + const_sbitmap range = kv.second; + if (find_p (regno)) + { + sbitmap curr = get_range (regno); + changed |= bitmap_ior (curr, curr, range); + } + else + changed |= add_range (regno, range); + } + return changed; + } + + /* Return true if regno's live range is full. */ + bool full_p (unsigned int regno) const + { + return find_p (regno) && bitmap_full_p (get_range (regno)); + } + /* Return true if regno's live range is empty. */ + bool empty_p (unsigned int regno) const + { + return !find_p (regno) || bitmap_empty_p (get_range (regno)); + } + + /* Debug helper. */ + void dump (FILE *file, const char *indent = ";; ") const; +}; + +#endif /* GCC_SUBREG_LIVE_RANGE_H */ diff --git a/gcc/timevar.def b/gcc/timevar.def index 8e2168e0817..337a2b7225b 100644 --- a/gcc/timevar.def +++ b/gcc/timevar.def @@ -120,6 +120,7 @@ DEFTIMEVAR (TV_DF_SCAN , "df scan insns") DEFTIMEVAR (TV_DF_MD , "df multiple defs") DEFTIMEVAR (TV_DF_RD , "df reaching defs") DEFTIMEVAR (TV_DF_LR , "df live regs") +DEFTIMEVAR (TV_DF_LIVE_SUBREG , "df live subregs") DEFTIMEVAR (TV_DF_LIVE , "df live&initialized regs") DEFTIMEVAR (TV_DF_MIR , "df must-initialized regs") DEFTIMEVAR (TV_DF_CHAIN , "df use-def / def-use chains") From patchwork Sat Feb 3 10:50:11 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Lehua Ding X-Patchwork-Id: 85266 Return-Path: X-Original-To: patchwork@sourceware.org Delivered-To: patchwork@sourceware.org Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id 841CB3857B8A for ; Sat, 3 Feb 2024 10:52:23 +0000 (GMT) X-Original-To: gcc-patches@gcc.gnu.org Delivered-To: gcc-patches@gcc.gnu.org Received: from smtpbgsg1.qq.com (smtpbgsg1.qq.com [54.254.200.92]) by sourceware.org (Postfix) with ESMTPS id 5B15B385840E for ; Sat, 3 Feb 2024 10:50:29 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org 5B15B385840E Authentication-Results: sourceware.org; dmarc=none (p=none dis=none) header.from=rivai.ai Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=rivai.ai ARC-Filter: OpenARC Filter v1.0.0 sourceware.org 5B15B385840E Authentication-Results: server2.sourceware.org; arc=none smtp.remote-ip=54.254.200.92 ARC-Seal: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1706957434; cv=none; b=W+3L8df9T5Ka3tLxdPbsCPtoQIaU+LdKXZYt8L/5+25hHF7TmTYP+qF5eHt6D2h/VEY/x4LOJ4CJhMUcvCZtSKXqcrU4YGWlMD8BZZqalVOHguE3DfMrESoWqohtbDwVzDq+aOnJiIejnssIN29pbsB0C3NN5kD/TNzWAYM2nzM= ARC-Message-Signature: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1706957434; c=relaxed/simple; bh=ScelwjQCZK419kT1x9cvM1hZaYLoEiKvvFMJuB2lgP8=; h=From:To:Subject:Date:Message-Id:MIME-Version; b=ZPEymjEe/VZj4PdI06y7E/Vc88kd+ueRX4l8I/pxQJ0/djBbM2/WOFJeDi3YGkD7Ulbpbg09Qdc+GcPkV0IvhtA/M9uI74VPj/4LOABSaaCrQ6nmQUDqvzGNn4SJi9I6xJcfWaUbygxQLcOlkXOjWuD4swO1mQw6CVOaWgRBrHA= ARC-Authentication-Results: i=1; server2.sourceware.org X-QQ-mid: bizesmtp72t1706957424t0z2nlfl X-QQ-Originating-IP: IOoiM1SdS1IqYIV371JKtIC1YLJ0A1STj0xx63IqX6w= Received: from rios-cad5.localdomain ( [58.60.1.25]) by bizesmtp.qq.com (ESMTP) with id ; Sat, 03 Feb 2024 18:50:23 +0800 (CST) X-QQ-SSF: 01400000000000C0F000000A0000000 X-QQ-FEAT: q+EIYT+FhZrD6pxJosXlsuioC10LmlnaQkED6pc933ZLdsfTGhxYVqKFUl+RK AjRxseNy8CwUYfODdvTmRELP0jX1SwKgqJ9104Cy8Eo2XC/bEOLmWObEywlHiEn9gzlOFEe NQdffNEanj3RMe3Shvp1qk2RfpEjFWOjYdvfACYApfdmJ0SFJSBNe09a97a/L4t2ocD8mxo e+cIw/ifmrLgRo3w1IxIXM2TyrUmB2NkWejv766GyIgesIiOn591yb8KS//uAAuELVXBtrg 8NVbHlJZGFTbQLRabplv6IY4FmtNLQ+qd5NyYq8XUrcqJeFHXIm7S4AjNEqFiKzwYdnxqwj LvPVaIAVptC4XJd8NU+O4bP5Vak6ync5LKhcExXE/6VtTRkaoxBIio2e+5gSQ== X-QQ-GoodBg: 2 X-BIZMAIL-ID: 14457489961731901332 From: Lehua Ding To: gcc-patches@gcc.gnu.org Cc: vmakarov@redhat.com, richard.sandiford@arm.com, juzhe.zhong@rivai.ai, lehua.ding@rivai.ai Subject: [PATCH 3/4] ira: Apply DF_LIVE_SUBREG data Date: Sat, 3 Feb 2024 18:50:11 +0800 Message-Id: <20240203105012.208998-4-lehua.ding@rivai.ai> X-Mailer: git-send-email 2.36.3 In-Reply-To: <20240203105012.208998-1-lehua.ding@rivai.ai> References: <20240203105012.208998-1-lehua.ding@rivai.ai> MIME-Version: 1.0 X-QQ-SENDSIZE: 520 Feedback-ID: bizesmtp:rivai.ai:qybglogicsvrgz:qybglogicsvrgz5a-3 X-Spam-Status: No, score=-9.5 required=5.0 tests=BAYES_00, GIT_PATCH_0, KAM_DMARC_STATUS, RCVD_IN_BARRACUDACENTRAL, RCVD_IN_DNSWL_NONE, RCVD_IN_MSPIKE_H2, SPF_HELO_PASS, SPF_PASS, TXREP, T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org X-BeenThere: gcc-patches@gcc.gnu.org X-Mailman-Version: 2.1.30 Precedence: list List-Id: Gcc-patches mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: gcc-patches-bounces+patchwork=sourceware.org@gcc.gnu.org This patch simple replace df_get_live_in to df_get_subreg_live_in and replace df_get_live_out to df_get_subreg_live_out. gcc/ChangeLog: * ira-build.cc (create_bb_allocnos): Switch to DF_LIVE_SUBREG df data. (create_loop_allocnos): Ditto. * ira-color.cc (ira_loop_edge_freq): Ditto. * ira-emit.cc (generate_edge_moves): Ditto. (add_ranges_and_copies): Ditto. * ira-lives.cc (process_out_of_region_eh_regs): Ditto. (add_conflict_from_region_landing_pads): Ditto. (process_bb_node_lives): Ditto. * ira.cc (find_moveable_pseudos): Ditto. (interesting_dest_for_shprep_1): Ditto. (allocate_initial_values): Ditto. (ira): Ditto. --- gcc/ira-build.cc | 7 ++++--- gcc/ira-color.cc | 8 ++++---- gcc/ira-emit.cc | 12 ++++++------ gcc/ira-lives.cc | 7 ++++--- gcc/ira.cc | 19 ++++++++++++------- 5 files changed, 30 insertions(+), 23 deletions(-) diff --git a/gcc/ira-build.cc b/gcc/ira-build.cc index ea593d5a087..283ff36d3dd 100644 --- a/gcc/ira-build.cc +++ b/gcc/ira-build.cc @@ -1921,7 +1921,8 @@ create_bb_allocnos (ira_loop_tree_node_t bb_node) create_insn_allocnos (PATTERN (insn), NULL, false); /* It might be a allocno living through from one subloop to another. */ - EXECUTE_IF_SET_IN_REG_SET (df_get_live_in (bb), FIRST_PSEUDO_REGISTER, i, bi) + EXECUTE_IF_SET_IN_REG_SET (df_get_subreg_live_in (bb), FIRST_PSEUDO_REGISTER, + i, bi) if (ira_curr_regno_allocno_map[i] == NULL) ira_create_allocno (i, false, ira_curr_loop_tree_node); } @@ -1937,9 +1938,9 @@ create_loop_allocnos (edge e) bitmap_iterator bi; ira_loop_tree_node_t parent; - live_in_regs = df_get_live_in (e->dest); + live_in_regs = df_get_subreg_live_in (e->dest); border_allocnos = ira_curr_loop_tree_node->border_allocnos; - EXECUTE_IF_SET_IN_REG_SET (df_get_live_out (e->src), + EXECUTE_IF_SET_IN_REG_SET (df_get_subreg_live_out (e->src), FIRST_PSEUDO_REGISTER, i, bi) if (bitmap_bit_p (live_in_regs, i)) { diff --git a/gcc/ira-color.cc b/gcc/ira-color.cc index b9ae32d1b4d..bfebc48ef83 100644 --- a/gcc/ira-color.cc +++ b/gcc/ira-color.cc @@ -2786,8 +2786,8 @@ ira_loop_edge_freq (ira_loop_tree_node_t loop_node, int regno, bool exit_p) FOR_EACH_EDGE (e, ei, loop_node->loop->header->preds) if (e->src != loop_node->loop->latch && (regno < 0 - || (bitmap_bit_p (df_get_live_out (e->src), regno) - && bitmap_bit_p (df_get_live_in (e->dest), regno)))) + || (bitmap_bit_p (df_get_subreg_live_out (e->src), regno) + && bitmap_bit_p (df_get_subreg_live_in (e->dest), regno)))) freq += EDGE_FREQUENCY (e); } else @@ -2795,8 +2795,8 @@ ira_loop_edge_freq (ira_loop_tree_node_t loop_node, int regno, bool exit_p) auto_vec edges = get_loop_exit_edges (loop_node->loop); FOR_EACH_VEC_ELT (edges, i, e) if (regno < 0 - || (bitmap_bit_p (df_get_live_out (e->src), regno) - && bitmap_bit_p (df_get_live_in (e->dest), regno))) + || (bitmap_bit_p (df_get_subreg_live_out (e->src), regno) + && bitmap_bit_p (df_get_subreg_live_in (e->dest), regno))) freq += EDGE_FREQUENCY (e); } diff --git a/gcc/ira-emit.cc b/gcc/ira-emit.cc index d347f11fa02..8075b082e36 100644 --- a/gcc/ira-emit.cc +++ b/gcc/ira-emit.cc @@ -510,8 +510,8 @@ generate_edge_moves (edge e) return; src_map = src_loop_node->regno_allocno_map; dest_map = dest_loop_node->regno_allocno_map; - regs_live_in_dest = df_get_live_in (e->dest); - regs_live_out_src = df_get_live_out (e->src); + regs_live_in_dest = df_get_subreg_live_in (e->dest); + regs_live_out_src = df_get_subreg_live_out (e->src); EXECUTE_IF_SET_IN_REG_SET (regs_live_in_dest, FIRST_PSEUDO_REGISTER, regno, bi) if (bitmap_bit_p (regs_live_out_src, regno)) @@ -1229,16 +1229,16 @@ add_ranges_and_copies (void) destination block) to use for searching allocnos by their regnos because of subsequent IR flattening. */ node = IRA_BB_NODE (bb)->parent; - bitmap_copy (live_through, df_get_live_in (bb)); + bitmap_copy (live_through, df_get_subreg_live_in (bb)); add_range_and_copies_from_move_list (at_bb_start[bb->index], node, live_through, REG_FREQ_FROM_BB (bb)); - bitmap_copy (live_through, df_get_live_out (bb)); + bitmap_copy (live_through, df_get_subreg_live_out (bb)); add_range_and_copies_from_move_list (at_bb_end[bb->index], node, live_through, REG_FREQ_FROM_BB (bb)); FOR_EACH_EDGE (e, ei, bb->succs) { - bitmap_and (live_through, - df_get_live_in (e->dest), df_get_live_out (bb)); + bitmap_and (live_through, df_get_subreg_live_in (e->dest), + df_get_subreg_live_out (bb)); add_range_and_copies_from_move_list ((move_t) e->aux, node, live_through, REG_FREQ_FROM_EDGE_FREQ (EDGE_FREQUENCY (e))); diff --git a/gcc/ira-lives.cc b/gcc/ira-lives.cc index e07d3dc3e89..7641184069d 100644 --- a/gcc/ira-lives.cc +++ b/gcc/ira-lives.cc @@ -1254,7 +1254,8 @@ process_out_of_region_eh_regs (basic_block bb) if (! eh_p) return; - EXECUTE_IF_SET_IN_BITMAP (df_get_live_out (bb), FIRST_PSEUDO_REGISTER, i, bi) + EXECUTE_IF_SET_IN_BITMAP (df_get_subreg_live_out (bb), FIRST_PSEUDO_REGISTER, + i, bi) { ira_allocno_t a = ira_curr_regno_allocno_map[i]; for (int n = ALLOCNO_NUM_OBJECTS (a) - 1; n >= 0; n--) @@ -1288,7 +1289,7 @@ add_conflict_from_region_landing_pads (eh_region region, ira_object_t obj, if ((landing_label = lp->landing_pad) != NULL && (landing_bb = BLOCK_FOR_INSN (landing_label)) != NULL && (region->type != ERT_CLEANUP - || bitmap_bit_p (df_get_live_in (landing_bb), + || bitmap_bit_p (df_get_subreg_live_in (landing_bb), ALLOCNO_REGNO (a)))) { HARD_REG_SET new_conflict_regs @@ -1325,7 +1326,7 @@ process_bb_node_lives (ira_loop_tree_node_t loop_tree_node) high_pressure_start_point[ira_pressure_classes[i]] = -1; } curr_bb_node = loop_tree_node; - reg_live_out = df_get_live_out (bb); + reg_live_out = df_get_subreg_live_out (bb); sparseset_clear (objects_live); REG_SET_TO_HARD_REG_SET (hard_regs_live, reg_live_out); hard_regs_live &= ~(eliminable_regset | ira_no_alloc_regs); diff --git a/gcc/ira.cc b/gcc/ira.cc index 5642aea3caa..bebf6bb3e9b 100644 --- a/gcc/ira.cc +++ b/gcc/ira.cc @@ -4735,8 +4735,8 @@ find_moveable_pseudos (void) bitmap_initialize (local, 0); bitmap_initialize (transp, 0); bitmap_initialize (moveable, 0); - bitmap_copy (live, df_get_live_out (bb)); - bitmap_and_into (live, df_get_live_in (bb)); + bitmap_copy (live, df_get_subreg_live_out (bb)); + bitmap_and_into (live, df_get_subreg_live_in (bb)); bitmap_copy (transp, live); bitmap_clear (moveable); bitmap_clear (live); @@ -5036,7 +5036,8 @@ interesting_dest_for_shprep_1 (rtx set, basic_block call_dom) rtx dest = SET_DEST (set); if (!REG_P (src) || !HARD_REGISTER_P (src) || !REG_P (dest) || HARD_REGISTER_P (dest) - || (call_dom && !bitmap_bit_p (df_get_live_in (call_dom), REGNO (dest)))) + || (call_dom + && !bitmap_bit_p (df_get_subreg_live_in (call_dom), REGNO (dest)))) return NULL; return dest; } @@ -5514,10 +5515,12 @@ allocate_initial_values (void) /* Update global register liveness information. */ FOR_EACH_BB_FN (bb, cfun) { - if (REGNO_REG_SET_P (df_get_live_in (bb), regno)) - SET_REGNO_REG_SET (df_get_live_in (bb), new_regno); - if (REGNO_REG_SET_P (df_get_live_out (bb), regno)) - SET_REGNO_REG_SET (df_get_live_out (bb), new_regno); + if (REGNO_REG_SET_P (df_get_subreg_live_in (bb), regno)) + SET_REGNO_REG_SET (df_get_subreg_live_in (bb), + new_regno); + if (REGNO_REG_SET_P (df_get_subreg_live_out (bb), regno)) + SET_REGNO_REG_SET (df_get_subreg_live_out (bb), + new_regno); } } } @@ -5679,6 +5682,8 @@ ira (FILE *f) if (optimize > 1) df_remove_problem (df_live); gcc_checking_assert (df_live == NULL); + if (flag_track_subreg_liveness) + df_live_subreg_add_problem (); if (flag_checking) df->changeable_flags |= DF_VERIFY_SCHEDULED; From patchwork Sat Feb 3 10:50:12 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Lehua Ding X-Patchwork-Id: 85265 Return-Path: X-Original-To: patchwork@sourceware.org Delivered-To: patchwork@sourceware.org Received: from server2.sourceware.org (localhost [IPv6:::1]) by sourceware.org (Postfix) with ESMTP id A12F63857B92 for ; Sat, 3 Feb 2024 10:51:53 +0000 (GMT) X-Original-To: gcc-patches@gcc.gnu.org Delivered-To: gcc-patches@gcc.gnu.org Received: from smtpbgbr2.qq.com (smtpbgbr2.qq.com [54.207.22.56]) by sourceware.org (Postfix) with ESMTPS id 7BE113858289 for ; Sat, 3 Feb 2024 10:50:35 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org 7BE113858289 Authentication-Results: sourceware.org; dmarc=none (p=none dis=none) header.from=rivai.ai Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=rivai.ai ARC-Filter: OpenARC Filter v1.0.0 sourceware.org 7BE113858289 Authentication-Results: server2.sourceware.org; arc=none smtp.remote-ip=54.207.22.56 ARC-Seal: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1706957441; cv=none; b=H/RHz/G59vjh4HZjpD1wv7l26etO6fXDgZ14fV2yR2xwTSKzE/VeALywZBqswF9DR+AsZpn/T68SAfDeS4Sey53Uctiok0M54NBIL3wrPat+d/Vbub0Jsssc5MzcA0sxq6iBg9V1stML1UijFBlHL8UD+SxpOCQGu6ghce9SjKs= ARC-Message-Signature: i=1; a=rsa-sha256; d=sourceware.org; s=key; t=1706957441; c=relaxed/simple; bh=XJ+5GLBfB5kXviuDAXYPAzoqhs3er06qleaSS3IA7EE=; h=From:To:Subject:Date:Message-Id:MIME-Version; b=wTUBkuqqw/kmARwtT7TSurvNIUndYizEMYtyOiFectBekxCRdIqB/i8nnYCIkSfMg7vzo5wyYYWGKl5qt0epD0lnmqlYVWu3hgf9cDO12jaxtZbyVYgFSdaVpii6GlapvrLKubnnVWtSpVm7ykBproAA5QnxVz+n1fFb1rqn3Tk= ARC-Authentication-Results: i=1; server2.sourceware.org X-QQ-mid: bizesmtp72t1706957427t4fhb6xg X-QQ-Originating-IP: 5x5BkfjicudGiOEOqxY4XBEjgjUTxE7gwZHEc1hFEtE= Received: from rios-cad5.localdomain ( [58.60.1.25]) by bizesmtp.qq.com (ESMTP) with id ; Sat, 03 Feb 2024 18:50:26 +0800 (CST) X-QQ-SSF: 01400000000000C0F000000A0000000 X-QQ-FEAT: +ynUkgUhZJlq9vRSwPu3aF3w/Bbjf/BRg0WdacLMrrJAe3KPRmrrBWaX26fF7 BiHH57Kmdaw+UN6CmT3UOiOxClgglGQrkhDOGqVHZdkoh6WKAF3vKQOpNmh2Xskbvh+YRcj KozHybFWQZtoNLaC6BGDbNRjiXtmDWrcAu7BQ90cVC67jbzVx21Wc9ZmPvXvHcx3g5nz3g5 56hhRhNejLlqYmDCuIQ0BIWr+AOCldxmpHB4fkkUpP4wY8ND8X40g3UCBAq4Yf/Z5X5GLmg TOKm/Hk88nSFjutdO3cNPuQGdGLUBj7Sx4DTiUYSIBJNEGJgBHv3r+CizVM+wu0K/rJFYea rWGz0n5c80BsrIh2qxU7pXQyQ1Abkgr2xBzIe0DrMz/j+CQ0Xv+rh09yKGjhdgmEfMWUsKW X-QQ-GoodBg: 2 X-BIZMAIL-ID: 7222153971514329300 From: Lehua Ding To: gcc-patches@gcc.gnu.org Cc: vmakarov@redhat.com, richard.sandiford@arm.com, juzhe.zhong@rivai.ai, lehua.ding@rivai.ai Subject: [PATCH 4/4] lra: Apply DF_LIVE_SUBREG data Date: Sat, 3 Feb 2024 18:50:12 +0800 Message-Id: <20240203105012.208998-5-lehua.ding@rivai.ai> X-Mailer: git-send-email 2.36.3 In-Reply-To: <20240203105012.208998-1-lehua.ding@rivai.ai> References: <20240203105012.208998-1-lehua.ding@rivai.ai> MIME-Version: 1.0 X-QQ-SENDSIZE: 520 Feedback-ID: bizesmtp:rivai.ai:qybglogicsvrgz:qybglogicsvrgz5a-3 X-Spam-Status: No, score=-9.5 required=5.0 tests=BAYES_00, GIT_PATCH_0, KAM_DMARC_STATUS, RCVD_IN_BARRACUDACENTRAL, RCVD_IN_DNSWL_NONE, RCVD_IN_MSPIKE_H3, RCVD_IN_MSPIKE_WL, SPF_HELO_PASS, SPF_PASS, TXREP, T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org X-BeenThere: gcc-patches@gcc.gnu.org X-Mailman-Version: 2.1.30 Precedence: list List-Id: Gcc-patches mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: gcc-patches-bounces+patchwork=sourceware.org@gcc.gnu.org This patch apply the DF_LIVE_SUBREG to LRA pass. More changes were made to the LRA than the IRA since the LRA will modify the DF data directly. The main big changes are centered on the lra-lives.cc file. gcc/ChangeLog: * lra-coalesce.cc (update_live_info): Extend to DF_LIVE_SUBREG. (lra_coalesce): Ditto. * lra-constraints.cc (update_ebb_live_info): Ditto. (get_live_on_other_edges): Ditto. (inherit_in_ebb): Ditto. (lra_inheritance): Ditto. (fix_bb_live_info): Ditto. (remove_inheritance_pseudos): Ditto. * lra-int.h (GCC_LRA_INT_H): include subreg-live-range.h (struct lra_insn_reg): Add op filed to record the corresponding rtx. * lra-lives.cc (class bb_data_pseudos): Extend the bb_data_pseudos to include new partial_def/use and range_def/use fileds for DF_LIVE_SUBREG problem. (need_track_subreg_p): checking is the regno need to be tracked. (make_hard_regno_live): switch to live_subreg filed. (make_hard_regno_dead): Ditto. (mark_regno_live): Support record subreg liveness. (mark_regno_dead): Ditto. (live_trans_fun): Adjust transfer function to support subreg liveness. (live_con_fun_0): Adjust Confluence function to support subreg liveness. (live_con_fun_n): Ditto. (initiate_live_solver): Ditto. (finish_live_solver): Ditto. (process_bb_lives): Ditto. (lra_create_live_ranges_1): Dump subreg liveness. * lra-remat.cc (dump_candidates_and_remat_bb_data): Switch to DF_LIVE_SUBREG df data. (calculate_livein_cands): Ditto. (do_remat): Ditto. * lra-spills.cc (spill_pseudos): Ditto. * lra.cc (new_insn_reg): New argument op. (add_regs_to_insn_regno_info): Add new argument op. --- gcc/lra-coalesce.cc | 27 +++- gcc/lra-constraints.cc | 109 ++++++++++--- gcc/lra-int.h | 4 + gcc/lra-lives.cc | 357 ++++++++++++++++++++++++++++++++--------- gcc/lra-remat.cc | 8 +- gcc/lra-spills.cc | 27 +++- gcc/lra.cc | 10 +- 7 files changed, 430 insertions(+), 112 deletions(-) diff --git a/gcc/lra-coalesce.cc b/gcc/lra-coalesce.cc index a9b5b51cb3f..9416775a009 100644 --- a/gcc/lra-coalesce.cc +++ b/gcc/lra-coalesce.cc @@ -186,19 +186,28 @@ static bitmap_head used_pseudos_bitmap; /* Set up USED_PSEUDOS_BITMAP, and update LR_BITMAP (a BB live info bitmap). */ static void -update_live_info (bitmap lr_bitmap) +update_live_info (bitmap all, bitmap full, bitmap partial) { unsigned int j; bitmap_iterator bi; bitmap_clear (&used_pseudos_bitmap); - EXECUTE_IF_AND_IN_BITMAP (&coalesced_pseudos_bitmap, lr_bitmap, + EXECUTE_IF_AND_IN_BITMAP (&coalesced_pseudos_bitmap, all, FIRST_PSEUDO_REGISTER, j, bi) bitmap_set_bit (&used_pseudos_bitmap, first_coalesced_pseudo[j]); - if (! bitmap_empty_p (&used_pseudos_bitmap)) + if (!bitmap_empty_p (&used_pseudos_bitmap)) { - bitmap_and_compl_into (lr_bitmap, &coalesced_pseudos_bitmap); - bitmap_ior_into (lr_bitmap, &used_pseudos_bitmap); + bitmap_and_compl_into (all, &coalesced_pseudos_bitmap); + bitmap_ior_into (all, &used_pseudos_bitmap); + + if (flag_track_subreg_liveness) + { + bitmap_and_compl_into (full, &coalesced_pseudos_bitmap); + bitmap_ior_and_compl_into (full, &used_pseudos_bitmap, partial); + + bitmap_and_compl_into (partial, &coalesced_pseudos_bitmap); + bitmap_ior_and_compl_into (partial, &used_pseudos_bitmap, full); + } } } @@ -301,8 +310,12 @@ lra_coalesce (void) bitmap_initialize (&used_pseudos_bitmap, ®_obstack); FOR_EACH_BB_FN (bb, cfun) { - update_live_info (df_get_live_in (bb)); - update_live_info (df_get_live_out (bb)); + update_live_info (df_get_subreg_live_in (bb), + df_get_subreg_live_full_in (bb), + df_get_subreg_live_partial_in (bb)); + update_live_info (df_get_subreg_live_out (bb), + df_get_subreg_live_full_out (bb), + df_get_subreg_live_partial_out (bb)); FOR_BB_INSNS_SAFE (bb, insn, next) if (INSN_P (insn) && bitmap_bit_p (&involved_insns_bitmap, INSN_UID (insn))) diff --git a/gcc/lra-constraints.cc b/gcc/lra-constraints.cc index 0ae81c1ff9c..d1316620f51 100644 --- a/gcc/lra-constraints.cc +++ b/gcc/lra-constraints.cc @@ -6505,34 +6505,86 @@ update_ebb_live_info (rtx_insn *head, rtx_insn *tail) { if (prev_bb != NULL) { - /* Update df_get_live_in (prev_bb): */ + /* Update subreg live (prev_bb): */ + bitmap subreg_all_in = df_get_subreg_live_in (prev_bb); + bitmap subreg_full_in = df_get_subreg_live_full_in (prev_bb); + bitmap subreg_partial_in = df_get_subreg_live_partial_in (prev_bb); + subregs_live *range_in = df_get_subreg_live_range_in (prev_bb); EXECUTE_IF_SET_IN_BITMAP (&check_only_regs, 0, j, bi) if (bitmap_bit_p (&live_regs, j)) - bitmap_set_bit (df_get_live_in (prev_bb), j); - else - bitmap_clear_bit (df_get_live_in (prev_bb), j); + { + bitmap_set_bit (subreg_all_in, j); + if (flag_track_subreg_liveness) + { + bitmap_set_bit (subreg_full_in, j); + if (bitmap_bit_p (subreg_partial_in, j)) + { + bitmap_clear_bit (subreg_partial_in, j); + range_in->remove_range (j); + } + } + } + else if (bitmap_bit_p (subreg_all_in, j)) + { + bitmap_clear_bit (subreg_all_in, j); + if (flag_track_subreg_liveness) + { + bitmap_clear_bit (subreg_full_in, j); + if (bitmap_bit_p (subreg_partial_in, j)) + { + bitmap_clear_bit (subreg_partial_in, j); + range_in->remove_range (j); + } + } + } } + bitmap subreg_all_out = df_get_subreg_live_out (curr_bb); if (curr_bb != last_bb) { - /* Update df_get_live_out (curr_bb): */ + /* Update subreg live (curr_bb): */ + bitmap subreg_full_out = df_get_subreg_live_full_out (curr_bb); + bitmap subreg_partial_out = df_get_subreg_live_partial_out (curr_bb); + subregs_live *range_out = df_get_subreg_live_range_out (curr_bb); EXECUTE_IF_SET_IN_BITMAP (&check_only_regs, 0, j, bi) { live_p = bitmap_bit_p (&live_regs, j); if (! live_p) FOR_EACH_EDGE (e, ei, curr_bb->succs) - if (bitmap_bit_p (df_get_live_in (e->dest), j)) + if (bitmap_bit_p (df_get_subreg_live_in (e->dest), j)) { live_p = true; break; } if (live_p) - bitmap_set_bit (df_get_live_out (curr_bb), j); - else - bitmap_clear_bit (df_get_live_out (curr_bb), j); + { + bitmap_set_bit (subreg_all_out, j); + if (flag_track_subreg_liveness) + { + bitmap_set_bit (subreg_full_out, j); + if (bitmap_bit_p (subreg_partial_out, j)) + { + bitmap_clear_bit (subreg_partial_out, j); + range_out->remove_range (j); + } + } + } + else if (bitmap_bit_p (subreg_all_out, j)) + { + bitmap_clear_bit (subreg_all_out, j); + if (flag_track_subreg_liveness) + { + bitmap_clear_bit (subreg_full_out, j); + if (bitmap_bit_p (subreg_partial_out, j)) + { + bitmap_clear_bit (subreg_partial_out, j); + range_out->remove_range (j); + } + } + } } } prev_bb = curr_bb; - bitmap_and (&live_regs, &check_only_regs, df_get_live_out (curr_bb)); + bitmap_and (&live_regs, &check_only_regs, subreg_all_out); } if (! NONDEBUG_INSN_P (curr_insn)) continue; @@ -6649,7 +6701,7 @@ get_live_on_other_edges (basic_block from, basic_block to, bitmap res) bitmap_clear (res); FOR_EACH_EDGE (e, ei, from->succs) if (e->dest != to) - bitmap_ior_into (res, df_get_live_in (e->dest)); + bitmap_ior_into (res, df_get_subreg_live_in (e->dest)); last = get_last_insertion_point (from); if (! JUMP_P (last)) return; @@ -6721,7 +6773,7 @@ inherit_in_ebb (rtx_insn *head, rtx_insn *tail) { /* We are at the end of BB. Add qualified living pseudos for potential splitting. */ - to_process = df_get_live_out (curr_bb); + to_process = df_get_subreg_live_out (curr_bb); if (last_processed_bb != NULL) { /* We are somewhere in the middle of EBB. */ @@ -7093,7 +7145,7 @@ inherit_in_ebb (rtx_insn *head, rtx_insn *tail) { /* We reached the beginning of the current block -- do rest of spliting in the current BB. */ - to_process = df_get_live_in (curr_bb); + to_process = df_get_subreg_live_in (curr_bb); if (BLOCK_FOR_INSN (head) != curr_bb) { /* We are somewhere in the middle of EBB. */ @@ -7170,7 +7222,7 @@ lra_inheritance (void) fprintf (lra_dump_file, "EBB"); /* Form a EBB starting with BB. */ bitmap_clear (&ebb_global_regs); - bitmap_ior_into (&ebb_global_regs, df_get_live_in (bb)); + bitmap_ior_into (&ebb_global_regs, df_get_subreg_live_in (bb)); for (;;) { if (lra_dump_file != NULL) @@ -7186,7 +7238,7 @@ lra_inheritance (void) break; bb = bb->next_bb; } - bitmap_ior_into (&ebb_global_regs, df_get_live_out (bb)); + bitmap_ior_into (&ebb_global_regs, df_get_subreg_live_out (bb)); if (lra_dump_file != NULL) fprintf (lra_dump_file, "\n"); if (inherit_in_ebb (BB_HEAD (start_bb), BB_END (bb))) @@ -7215,15 +7267,26 @@ int lra_undo_inheritance_iter; /* Fix BB live info LIVE after removing pseudos created on pass doing inheritance/split which are REMOVED_PSEUDOS. */ static void -fix_bb_live_info (bitmap live, bitmap removed_pseudos) +fix_bb_live_info (bitmap all, bitmap full, bitmap partial, + bitmap removed_pseudos) { unsigned int regno; bitmap_iterator bi; EXECUTE_IF_SET_IN_BITMAP (removed_pseudos, 0, regno, bi) - if (bitmap_clear_bit (live, regno) - && REG_P (lra_reg_info[regno].restore_rtx)) - bitmap_set_bit (live, REGNO (lra_reg_info[regno].restore_rtx)); + { + if (bitmap_clear_bit (all, regno) + && REG_P (lra_reg_info[regno].restore_rtx)) + { + bitmap_set_bit (all, REGNO (lra_reg_info[regno].restore_rtx)); + if (flag_track_subreg_liveness) + { + bitmap_clear_bit (full, regno); + bitmap_set_bit (full, REGNO (lra_reg_info[regno].restore_rtx)); + gcc_assert (!bitmap_bit_p (partial, regno)); + } + } + } } /* Return regno of the (subreg of) REG. Otherwise, return a negative @@ -7289,8 +7352,12 @@ remove_inheritance_pseudos (bitmap remove_pseudos) constraint pass. */ FOR_EACH_BB_FN (bb, cfun) { - fix_bb_live_info (df_get_live_in (bb), remove_pseudos); - fix_bb_live_info (df_get_live_out (bb), remove_pseudos); + fix_bb_live_info (df_get_subreg_live_in (bb), + df_get_subreg_live_full_in (bb), + df_get_subreg_live_partial_in (bb), remove_pseudos); + fix_bb_live_info (df_get_subreg_live_out (bb), + df_get_subreg_live_full_out (bb), + df_get_subreg_live_partial_out (bb), remove_pseudos); FOR_BB_INSNS_REVERSE (bb, curr_insn) { if (! INSN_P (curr_insn)) diff --git a/gcc/lra-int.h b/gcc/lra-int.h index 5f605c3ae41..d73ff4e31b5 100644 --- a/gcc/lra-int.h +++ b/gcc/lra-int.h @@ -21,6 +21,8 @@ along with GCC; see the file COPYING3. If not see #ifndef GCC_LRA_INT_H #define GCC_LRA_INT_H +#include "subreg-live-range.h" + #define lra_assert(c) gcc_checking_assert (c) /* The parameter used to prevent infinite reloading for an insn. Each @@ -161,6 +163,8 @@ struct lra_insn_reg int regno; /* Next reg info of the same insn. */ struct lra_insn_reg *next; + /* The corresponding reg or subreg RTX. */ + rtx op; }; /* Static part (common info for insns with the same ICODE) of LRA diff --git a/gcc/lra-lives.cc b/gcc/lra-lives.cc index 66c6577e5d6..291f9420e06 100644 --- a/gcc/lra-lives.cc +++ b/gcc/lra-lives.cc @@ -272,8 +272,34 @@ update_pseudo_point (int regno, int point, enum point_type type) } } -/* The corresponding bitmaps of BB currently being processed. */ -static bitmap bb_killed_pseudos, bb_gen_pseudos; +/* Structure describing local BB data used for pseudo + live-analysis. */ +class bb_data_pseudos : public df_live_subreg_local_bb_info +{ +public: + /* Basic block about which the below data are. */ + basic_block bb; +}; + +/* Array for all BB data. Indexed by the corresponding BB index. */ +typedef class bb_data_pseudos *bb_data_t; + +/* All basic block data are referred through the following array. */ +static bb_data_t bb_data; + +/* The corresponding df local data of BB currently being processed. */ +static bb_data_t curr_bb_info; + +/* The regs which need to be tracked it's subreg liveness. */ +static bitmap_head tracked_regs; + +/* Return true if the REGNO need be track with subreg liveness. */ + +static bool +need_track_subreg_p (unsigned regno) +{ + return bitmap_bit_p (&tracked_regs, regno); +} /* Record hard register REGNO as now being live. It updates living hard regs and START_LIVING. */ @@ -287,7 +313,7 @@ make_hard_regno_live (int regno) SET_HARD_REG_BIT (hard_regs_live, regno); sparseset_set_bit (start_living, regno); if (fixed_regs[regno] || TEST_HARD_REG_BIT (hard_regs_spilled_into, regno)) - bitmap_set_bit (bb_gen_pseudos, regno); + bitmap_set_bit (&curr_bb_info->full_use, regno); } /* Process the definition of hard register REGNO. This updates @@ -310,8 +336,8 @@ make_hard_regno_dead (int regno) sparseset_set_bit (start_dying, regno); if (fixed_regs[regno] || TEST_HARD_REG_BIT (hard_regs_spilled_into, regno)) { - bitmap_clear_bit (bb_gen_pseudos, regno); - bitmap_set_bit (bb_killed_pseudos, regno); + bitmap_clear_bit (&curr_bb_info->full_use, regno); + bitmap_set_bit (&curr_bb_info->full_def, regno); } } @@ -343,7 +369,7 @@ mark_pseudo_dead (int regno) /* Mark register REGNO (pseudo or hard register) in MODE as being live and update BB_GEN_PSEUDOS. */ static void -mark_regno_live (int regno, machine_mode mode) +mark_regno_live (int regno, machine_mode mode, struct lra_insn_reg *reg) { int last; @@ -355,15 +381,23 @@ mark_regno_live (int regno, machine_mode mode) else { mark_pseudo_live (regno); - bitmap_set_bit (bb_gen_pseudos, regno); + if (!need_track_subreg_p (regno)) + bitmap_set_bit (&curr_bb_info->full_use, regno); + else + { + machine_mode reg_mode + = GET_MODE (SUBREG_P (reg->op) ? SUBREG_REG (reg->op) : reg->op); + auto_sbitmap range (get_nblocks (reg_mode)); + init_range (reg->op, range); + add_subreg_range (curr_bb_info, regno, range, false); + } } } - /* Mark register REGNO (pseudo or hard register) in MODE as being dead and update BB_GEN_PSEUDOS and BB_KILLED_PSEUDOS. */ static void -mark_regno_dead (int regno, machine_mode mode) +mark_regno_dead (int regno, machine_mode mode, struct lra_insn_reg *reg) { int last; @@ -375,8 +409,20 @@ mark_regno_dead (int regno, machine_mode mode) else { mark_pseudo_dead (regno); - bitmap_clear_bit (bb_gen_pseudos, regno); - bitmap_set_bit (bb_killed_pseudos, regno); + if (!need_track_subreg_p (regno)) + { + bitmap_clear_bit (&curr_bb_info->full_use, regno); + bitmap_set_bit (&curr_bb_info->full_def, regno); + } + else + { + machine_mode reg_mode + = GET_MODE (SUBREG_P (reg->op) ? SUBREG_REG (reg->op) : reg->op); + auto_sbitmap range (get_nblocks (reg_mode)); + init_range (reg->op, range); + remove_subreg_range (curr_bb_info, regno, range); + add_subreg_range (curr_bb_info, regno, range, true); + } } } @@ -387,23 +433,6 @@ mark_regno_dead (int regno, machine_mode mode) border. That might be a consequence of some global transformations in LRA, e.g. PIC pseudo reuse or rematerialization. */ -/* Structure describing local BB data used for pseudo - live-analysis. */ -class bb_data_pseudos -{ -public: - /* Basic block about which the below data are. */ - basic_block bb; - bitmap_head killed_pseudos; /* pseudos killed in the BB. */ - bitmap_head gen_pseudos; /* pseudos generated in the BB. */ -}; - -/* Array for all BB data. Indexed by the corresponding BB index. */ -typedef class bb_data_pseudos *bb_data_t; - -/* All basic block data are referred through the following array. */ -static bb_data_t bb_data; - /* Two small functions for access to the bb data. */ static inline bb_data_t get_bb_data (basic_block bb) @@ -429,14 +458,73 @@ static bitmap_head all_hard_regs_bitmap; static bool live_trans_fun (int bb_index) { - basic_block bb = get_bb_data_by_index (bb_index)->bb; - bitmap bb_liveout = df_get_live_out (bb); - bitmap bb_livein = df_get_live_in (bb); - bb_data_t bb_info = get_bb_data (bb); - - bitmap_and_compl (&temp_bitmap, bb_liveout, &all_hard_regs_bitmap); - return bitmap_ior_and_compl (bb_livein, &bb_info->gen_pseudos, - &temp_bitmap, &bb_info->killed_pseudos); + bb_data_t local_bb_info = get_bb_data_by_index (bb_index); + bitmap full_in = df_get_subreg_live_full_in (local_bb_info->bb); + bitmap full_out = df_get_subreg_live_full_out (local_bb_info->bb); + + bitmap_and_compl (&temp_bitmap, full_out, &all_hard_regs_bitmap); + bool changed = bitmap_ior_and_compl (full_in, &local_bb_info->full_use, + &temp_bitmap, &local_bb_info->full_def); + + /* Handle partial live case. */ + if (flag_track_subreg_liveness) + { + bitmap partial_in = df_get_subreg_live_partial_in (local_bb_info->bb); + bitmap partial_out = df_get_subreg_live_partial_out (local_bb_info->bb); + subregs_live *range_in = df_get_subreg_live_range_in (local_bb_info->bb); + subregs_live *range_out + = df_get_subreg_live_range_out (local_bb_info->bb); + + if (!bitmap_empty_p (partial_out) + || !bitmap_empty_p (&local_bb_info->partial_use)) + { + unsigned int regno; + bitmap_iterator bi; + bitmap_head temp_partial_out; + subregs_live temp_range_out; + + /* TEMP = (OUT & ~DEF) */ + bitmap_initialize (&temp_partial_out, &bitmap_default_obstack); + EXECUTE_IF_SET_IN_BITMAP (partial_out, FIRST_PSEUDO_REGISTER, regno, + bi) + { + sbitmap out_range = range_out->get_range (regno); + temp_range_out.add_range (regno, out_range); + if (bitmap_bit_p (&local_bb_info->partial_def, regno)) + { + sbitmap def_range + = local_bb_info->range_def->get_range (regno); + temp_range_out.remove_range (regno, def_range); + if (!temp_range_out.empty_p (regno)) + bitmap_set_bit (&temp_partial_out, regno); + } + else + bitmap_set_bit (&temp_partial_out, regno); + } + + /* TEMP = USE | TEMP */ + EXECUTE_IF_SET_IN_BITMAP (&local_bb_info->partial_use, + FIRST_PSEUDO_REGISTER, regno, bi) + { + sbitmap use_range = local_bb_info->range_use->get_range (regno); + temp_range_out.add_range (regno, use_range); + } + bitmap_ior_into (&temp_partial_out, &local_bb_info->partial_use); + + /* IN = TEMP */ + changed |= range_in->copy_lives (temp_range_out); + bitmap_copy (partial_in, &temp_partial_out); + df_live_subreg_check_result (full_in, partial_in, range_in); + } + else if (!bitmap_empty_p (partial_in)) + { + changed = true; + bitmap_clear (partial_in); + range_in->clear (); + } + } + + return changed; } /* The confluence function used by the DF equation solver to set up @@ -444,7 +532,9 @@ live_trans_fun (int bb_index) static void live_con_fun_0 (basic_block bb) { - bitmap_and_into (df_get_live_out (bb), &all_hard_regs_bitmap); + bitmap_and_into (df_get_subreg_live_out (bb), &all_hard_regs_bitmap); + if (flag_track_subreg_liveness) + bitmap_and_into (df_get_subreg_live_full_out (bb), &all_hard_regs_bitmap); } /* The confluence function used by the DF equation solver to propagate @@ -456,13 +546,37 @@ live_con_fun_0 (basic_block bb) static bool live_con_fun_n (edge e) { - basic_block bb = e->src; - basic_block dest = e->dest; - bitmap bb_liveout = df_get_live_out (bb); - bitmap dest_livein = df_get_live_in (dest); + bitmap src_full_out = df_get_subreg_live_full_out (e->src); + bitmap dest_full_in = df_get_subreg_live_full_in (e->dest); + + bool changed = bitmap_ior_and_compl_into (src_full_out, dest_full_in, + &all_hard_regs_bitmap); + /* Handle partial live case. */ + if (flag_track_subreg_liveness) + { + bitmap src_partial_out = df_get_subreg_live_partial_out (e->src); + subregs_live *src_range_out = df_get_subreg_live_range_out (e->src); + bitmap dest_partial_in = df_get_subreg_live_partial_in (e->dest); + subregs_live *dest_range_in = df_get_subreg_live_range_in (e->dest); + + if (bitmap_empty_p (dest_partial_in)) + return changed; + + unsigned int regno; + bitmap_iterator bi; + EXECUTE_IF_SET_IN_BITMAP (dest_partial_in, FIRST_PSEUDO_REGISTER, regno, + bi) + { + sbitmap dest_range = dest_range_in->get_range (regno); + changed |= src_range_out->add_range (regno, dest_range); + } + changed |= bitmap_ior_into (src_partial_out, dest_partial_in); - return bitmap_ior_and_compl_into (bb_liveout, - dest_livein, &all_hard_regs_bitmap); + df_live_subreg_check_result (src_full_out, src_partial_out, + src_range_out); + } + + return changed; } /* Indexes of all function blocks. */ @@ -479,12 +593,47 @@ initiate_live_solver (void) bitmap_initialize (&all_blocks, ®_obstack); basic_block bb; + if (flag_track_subreg_liveness) + { + bitmap_initialize (&tracked_regs, ®_obstack); + FOR_ALL_BB_FN (bb, cfun) + { + rtx_insn *insn; + df_ref use; + FOR_BB_INSNS (bb, insn) + { + if (!NONDEBUG_INSN_P (insn)) + continue; + + df_insn_info *insn_info = DF_INSN_INFO_GET (insn); + + FOR_EACH_INSN_INFO_USE (use, insn_info) + { + unsigned int regno = DF_REF_REGNO (use); + /* A multireg which is used via subreg pattern. */ + if (multireg_p (regno) + && DF_REF_FLAGS (use) & (DF_REF_SUBREG)) + bitmap_set_bit (&tracked_regs, regno); + } + } + } + } + + FOR_ALL_BB_FN (bb, cfun) { bb_data_t bb_info = get_bb_data (bb); bb_info->bb = bb; - bitmap_initialize (&bb_info->killed_pseudos, ®_obstack); - bitmap_initialize (&bb_info->gen_pseudos, ®_obstack); + bitmap_initialize (&bb_info->full_def, ®_obstack); + bitmap_initialize (&bb_info->full_use, ®_obstack); + if (flag_track_subreg_liveness) + { + bitmap_initialize (&bb_info->partial_def, ®_obstack); + bitmap_initialize (&bb_info->partial_use, ®_obstack); + size_t num_regs = bitmap_count_bits (&tracked_regs); + bb_info->range_def = new subregs_live (num_regs); + bb_info->range_use = new subregs_live (num_regs); + } bitmap_set_bit (&all_blocks, bb->index); } } @@ -496,11 +645,19 @@ finish_live_solver (void) basic_block bb; bitmap_clear (&all_blocks); + bitmap_clear (&tracked_regs); FOR_ALL_BB_FN (bb, cfun) { bb_data_t bb_info = get_bb_data (bb); - bitmap_clear (&bb_info->killed_pseudos); - bitmap_clear (&bb_info->gen_pseudos); + bitmap_clear (&bb_info->full_def); + bitmap_clear (&bb_info->full_use); + if (flag_track_subreg_liveness) + { + bitmap_clear (&bb_info->partial_def); + bitmap_clear (&bb_info->partial_use); + delete bb_info->range_def; + delete bb_info->range_use; + } } free (bb_data); bitmap_clear (&all_hard_regs_bitmap); @@ -663,7 +820,7 @@ process_bb_lives (basic_block bb, int &curr_point, bool dead_insn_p) /* Only has a meaningful value once we've seen a call. */ function_abi last_call_abi = default_function_abi; - reg_live_out = df_get_live_out (bb); + reg_live_out = df_get_subreg_live_out (bb); sparseset_clear (pseudos_live); sparseset_clear (pseudos_live_through_calls); sparseset_clear (pseudos_live_through_setjumps); @@ -675,10 +832,16 @@ process_bb_lives (basic_block bb, int &curr_point, bool dead_insn_p) mark_pseudo_live (j); } - bb_gen_pseudos = &get_bb_data (bb)->gen_pseudos; - bb_killed_pseudos = &get_bb_data (bb)->killed_pseudos; - bitmap_clear (bb_gen_pseudos); - bitmap_clear (bb_killed_pseudos); + curr_bb_info = get_bb_data (bb); + bitmap_clear (&curr_bb_info->full_use); + bitmap_clear (&curr_bb_info->full_def); + if (flag_track_subreg_liveness) + { + bitmap_clear (&curr_bb_info->partial_use); + bitmap_clear (&curr_bb_info->partial_def); + curr_bb_info->range_use->clear (); + curr_bb_info->range_def->clear (); + } freq = REG_FREQ_FROM_BB (bb); if (lra_dump_file != NULL) @@ -860,7 +1023,7 @@ process_bb_lives (basic_block bb, int &curr_point, bool dead_insn_p) if (reg->type != OP_IN) { update_pseudo_point (reg->regno, curr_point, USE_POINT); - mark_regno_live (reg->regno, reg->biggest_mode); + mark_regno_live (reg->regno, reg->biggest_mode, reg); /* ??? Should be a no-op for unused registers. */ check_pseudos_live_through_calls (reg->regno, last_call_abi); } @@ -886,7 +1049,7 @@ process_bb_lives (basic_block bb, int &curr_point, bool dead_insn_p) { if (reg->type == OP_OUT) update_pseudo_point (reg->regno, curr_point, DEF_POINT); - mark_regno_dead (reg->regno, reg->biggest_mode); + mark_regno_dead (reg->regno, reg->biggest_mode, reg); } for (reg = curr_static_id->hard_regs; reg != NULL; reg = reg->next) @@ -945,8 +1108,12 @@ process_bb_lives (basic_block bb, int &curr_point, bool dead_insn_p) { if (reg->type == OP_IN) update_pseudo_point (reg->regno, curr_point, USE_POINT); - mark_regno_live (reg->regno, reg->biggest_mode); check_pseudos_live_through_calls (reg->regno, last_call_abi); + /* Ignore the use of subreg which is used as dest operand. */ + if (need_track_subreg_p (reg->regno) && reg->subreg_p + && reg->type == OP_INOUT) + continue; + mark_regno_live (reg->regno, reg->biggest_mode, reg); } for (reg = curr_static_id->hard_regs; reg != NULL; reg = reg->next) @@ -970,12 +1137,12 @@ process_bb_lives (basic_block bb, int &curr_point, bool dead_insn_p) { if (reg->type == OP_OUT) update_pseudo_point (reg->regno, curr_point, DEF_POINT); - mark_regno_dead (reg->regno, reg->biggest_mode); + mark_regno_dead (reg->regno, reg->biggest_mode, reg); /* We're done processing inputs, so make sure early clobber operands that are both inputs and outputs are still live. */ if (reg->type == OP_INOUT) - mark_regno_live (reg->regno, reg->biggest_mode); + mark_regno_live (reg->regno, reg->biggest_mode, reg); } for (reg = curr_static_id->hard_regs; reg != NULL; reg = reg->next) @@ -1099,8 +1266,8 @@ process_bb_lives (basic_block bb, int &curr_point, bool dead_insn_p) bool live_change_p = false; /* Check if bb border live info was changed. */ unsigned int live_pseudos_num = 0; - EXECUTE_IF_SET_IN_BITMAP (df_get_live_in (bb), - FIRST_PSEUDO_REGISTER, j, bi) + EXECUTE_IF_SET_IN_BITMAP (df_get_subreg_live_in (bb), FIRST_PSEUDO_REGISTER, + j, bi) { live_pseudos_num++; if (! sparseset_bit_p (pseudos_live, j)) @@ -1118,7 +1285,7 @@ process_bb_lives (basic_block bb, int &curr_point, bool dead_insn_p) live_change_p = true; if (lra_dump_file != NULL) EXECUTE_IF_SET_IN_SPARSESET (pseudos_live, j) - if (! bitmap_bit_p (df_get_live_in (bb), j)) + if (! bitmap_bit_p (df_get_subreg_live_in (bb), j)) fprintf (lra_dump_file, " r%d is added to live at bb%d start\n", j, bb->index); } @@ -1133,7 +1300,8 @@ process_bb_lives (basic_block bb, int &curr_point, bool dead_insn_p) mark_pseudo_dead (i); } - EXECUTE_IF_SET_IN_BITMAP (df_get_live_in (bb), FIRST_PSEUDO_REGISTER, j, bi) + EXECUTE_IF_SET_IN_BITMAP (df_get_subreg_live_in (bb), FIRST_PSEUDO_REGISTER, + j, bi) { if (sparseset_cardinality (pseudos_live_through_calls) == 0) break; @@ -1149,7 +1317,7 @@ process_bb_lives (basic_block bb, int &curr_point, bool dead_insn_p) if (!TEST_HARD_REG_BIT (hard_regs_spilled_into, i)) continue; - if (bitmap_bit_p (df_get_live_in (bb), i)) + if (bitmap_bit_p (df_get_subreg_live_in (bb), i)) continue; live_change_p = true; @@ -1157,7 +1325,9 @@ process_bb_lives (basic_block bb, int &curr_point, bool dead_insn_p) fprintf (lra_dump_file, " hard reg r%d is added to live at bb%d start\n", i, bb->index); - bitmap_set_bit (df_get_live_in (bb), i); + bitmap_set_bit (df_get_subreg_live_in (bb), i); + if (flag_track_subreg_liveness) + bitmap_set_bit (df_get_subreg_live_full_in (bb), i); } if (need_curr_point_incr) @@ -1421,12 +1591,30 @@ lra_create_live_ranges_1 (bool all_p, bool dead_insn_p) { /* We need to clear pseudo live info as some pseudos can disappear, e.g. pseudos with used equivalences. */ - FOR_EACH_BB_FN (bb, cfun) + FOR_BB_BETWEEN (bb, ENTRY_BLOCK_PTR_FOR_FN (cfun), + EXIT_BLOCK_PTR_FOR_FN (cfun), next_bb) { - bitmap_clear_range (df_get_live_in (bb), FIRST_PSEUDO_REGISTER, + bitmap_clear_range (df_get_subreg_live_in (bb), FIRST_PSEUDO_REGISTER, max_regno - FIRST_PSEUDO_REGISTER); - bitmap_clear_range (df_get_live_out (bb), FIRST_PSEUDO_REGISTER, + bitmap_clear_range (df_get_subreg_live_out (bb), FIRST_PSEUDO_REGISTER, max_regno - FIRST_PSEUDO_REGISTER); + if (flag_track_subreg_liveness) + { + bitmap_clear_range (df_get_subreg_live_full_in (bb), + FIRST_PSEUDO_REGISTER, + max_regno - FIRST_PSEUDO_REGISTER); + bitmap_clear_range (df_get_subreg_live_partial_in (bb), + FIRST_PSEUDO_REGISTER, + max_regno - FIRST_PSEUDO_REGISTER); + bitmap_clear_range (df_get_subreg_live_full_out (bb), + FIRST_PSEUDO_REGISTER, + max_regno - FIRST_PSEUDO_REGISTER); + bitmap_clear_range (df_get_subreg_live_partial_out (bb), + FIRST_PSEUDO_REGISTER, + max_regno - FIRST_PSEUDO_REGISTER); + df_get_subreg_live_range_in (bb)->clear (); + df_get_subreg_live_range_out (bb)->clear (); + } } /* As we did not change CFG since LRA start we can use DF-infrastructure solver to solve live data flow problem. */ @@ -1439,6 +1627,10 @@ lra_create_live_ranges_1 (bool all_p, bool dead_insn_p) (DF_BACKWARD, NULL, live_con_fun_0, live_con_fun_n, live_trans_fun, &all_blocks, df_get_postorder (DF_BACKWARD), df_get_n_blocks (DF_BACKWARD)); + + if (flag_track_subreg_liveness) + df_live_subreg_finalize (&all_blocks); + if (lra_dump_file != NULL) { fprintf (lra_dump_file, @@ -1447,16 +1639,33 @@ lra_create_live_ranges_1 (bool all_p, bool dead_insn_p) FOR_EACH_BB_FN (bb, cfun) { bb_data_t bb_info = get_bb_data (bb); - bitmap bb_livein = df_get_live_in (bb); - bitmap bb_liveout = df_get_live_out (bb); fprintf (lra_dump_file, "\nBB %d:\n", bb->index); - lra_dump_bitmap_with_title (" gen:", - &bb_info->gen_pseudos, bb->index); - lra_dump_bitmap_with_title (" killed:", - &bb_info->killed_pseudos, bb->index); - lra_dump_bitmap_with_title (" livein:", bb_livein, bb->index); - lra_dump_bitmap_with_title (" liveout:", bb_liveout, bb->index); + lra_dump_bitmap_with_title (" full use", &bb_info->full_use, + bb->index); + lra_dump_bitmap_with_title (" full def", &bb_info->full_def, + bb->index); + lra_dump_bitmap_with_title (" live in full", + df_get_subreg_live_full_in (bb), + bb->index); + lra_dump_bitmap_with_title (" live out full", + df_get_subreg_live_full_out (bb), + bb->index); + if (flag_track_subreg_liveness) + { + lra_dump_bitmap_with_title (" partial use", + &bb_info->partial_use, bb->index); + lra_dump_bitmap_with_title (" partial def", + &bb_info->partial_def, bb->index); + lra_dump_bitmap_with_title (" live in partial", + df_get_subreg_live_partial_in ( + bb), + bb->index); + lra_dump_bitmap_with_title (" live out partial", + df_get_subreg_live_partial_out ( + bb), + bb->index); + } } } } diff --git a/gcc/lra-remat.cc b/gcc/lra-remat.cc index c84bf3c9938..ef0157513b0 100644 --- a/gcc/lra-remat.cc +++ b/gcc/lra-remat.cc @@ -556,11 +556,11 @@ dump_candidates_and_remat_bb_data (void) fprintf (lra_dump_file, "\nBB %d:\n", bb->index); /* Livein */ fprintf (lra_dump_file, " register live in:"); - dump_regset (df_get_live_in (bb), lra_dump_file); + dump_regset (df_get_subreg_live_in (bb), lra_dump_file); putc ('\n', lra_dump_file); /* Liveout */ fprintf (lra_dump_file, " register live out:"); - dump_regset (df_get_live_out (bb), lra_dump_file); + dump_regset (df_get_subreg_live_out (bb), lra_dump_file); putc ('\n', lra_dump_file); /* Changed/dead regs: */ fprintf (lra_dump_file, " changed regs:"); @@ -727,7 +727,7 @@ calculate_livein_cands (void) FOR_EACH_BB_FN (bb, cfun) { - bitmap livein_regs = df_get_live_in (bb); + bitmap livein_regs = df_get_subreg_live_in (bb); bitmap livein_cands = &get_remat_bb_data (bb)->livein_cands; for (unsigned int i = 0; i < cands_num; i++) { @@ -1064,7 +1064,7 @@ do_remat (void) FOR_EACH_BB_FN (bb, cfun) { CLEAR_HARD_REG_SET (live_hard_regs); - EXECUTE_IF_SET_IN_BITMAP (df_get_live_in (bb), 0, regno, bi) + EXECUTE_IF_SET_IN_BITMAP (df_get_subreg_live_in (bb), 0, regno, bi) { int hard_regno = regno < FIRST_PSEUDO_REGISTER ? regno diff --git a/gcc/lra-spills.cc b/gcc/lra-spills.cc index 8b1a945d632..08ec247bc00 100644 --- a/gcc/lra-spills.cc +++ b/gcc/lra-spills.cc @@ -566,8 +566,31 @@ spill_pseudos (void) "Debug insn #%u is reset because it referenced " "removed pseudo\n", INSN_UID (insn)); } - bitmap_and_compl_into (df_get_live_in (bb), spilled_pseudos); - bitmap_and_compl_into (df_get_live_out (bb), spilled_pseudos); + bitmap_and_compl_into (df_get_subreg_live_in (bb), spilled_pseudos); + bitmap_and_compl_into (df_get_subreg_live_out (bb), spilled_pseudos); + + if (flag_track_subreg_liveness) + { + bitmap_and_compl_into (df_get_subreg_live_full_in (bb), + spilled_pseudos); + bitmap partial_in = df_get_subreg_live_partial_in (bb); + subregs_live *range_in = df_get_subreg_live_range_in (bb); + unsigned int regno; + bitmap_iterator bi; + EXECUTE_IF_AND_IN_BITMAP (partial_in, spilled_pseudos, + FIRST_PSEUDO_REGISTER, regno, bi) + range_in->remove_range (regno); + bitmap_and_compl_into (partial_in, spilled_pseudos); + + bitmap_and_compl_into (df_get_subreg_live_full_out (bb), + spilled_pseudos); + bitmap partial_out = df_get_subreg_live_partial_out (bb); + subregs_live *range_out = df_get_subreg_live_range_out (bb); + EXECUTE_IF_AND_IN_BITMAP (partial_out, spilled_pseudos, + FIRST_PSEUDO_REGISTER, regno, bi) + range_out->remove_range (regno); + bitmap_and_compl_into (partial_out, spilled_pseudos); + } } } } diff --git a/gcc/lra.cc b/gcc/lra.cc index fb32e134004..a6fabe7d118 100644 --- a/gcc/lra.cc +++ b/gcc/lra.cc @@ -574,7 +574,7 @@ object_allocator lra_insn_reg_pool ("insn regs"); EARLY_CLOBBER_ALTS. */ static struct lra_insn_reg * new_insn_reg (rtx_insn *insn, int regno, enum op_type type, - machine_mode mode, bool subreg_p, + machine_mode mode, bool subreg_p, rtx op, alternative_mask early_clobber_alts, struct lra_insn_reg *next) { @@ -586,6 +586,7 @@ new_insn_reg (rtx_insn *insn, int regno, enum op_type type, ir->subreg_p = subreg_p; ir->early_clobber_alts = early_clobber_alts; ir->regno = regno; + ir->op = op; ir->next = next; return ir; } @@ -926,7 +927,7 @@ collect_non_operand_hard_regs (rtx_insn *insn, rtx *x, && ! (FIRST_STACK_REG <= regno && regno <= LAST_STACK_REG)); #endif - list = new_insn_reg (data->insn, regno, type, mode, subreg_p, + list = new_insn_reg (data->insn, regno, type, mode, subreg_p, *x, early_clobber ? ALL_ALTERNATIVES : 0, list); } } @@ -1484,6 +1485,7 @@ add_regs_to_insn_regno_info (lra_insn_recog_data_t data, rtx x, code = GET_CODE (x); mode = GET_MODE (x); subreg_p = false; + rtx op = x; if (GET_CODE (x) == SUBREG) { mode = wider_subreg_mode (x); @@ -1501,7 +1503,7 @@ add_regs_to_insn_regno_info (lra_insn_recog_data_t data, rtx x, if (bitmap_set_bit (&lra_reg_info[regno].insn_bitmap, INSN_UID (insn))) { data->regs = new_insn_reg (data->insn, regno, type, mode, subreg_p, - early_clobber_alts, data->regs); + op, early_clobber_alts, data->regs); return; } else @@ -1513,7 +1515,7 @@ add_regs_to_insn_regno_info (lra_insn_recog_data_t data, rtx x, /* The info cannot be integrated into the found structure. */ data->regs = new_insn_reg (data->insn, regno, type, mode, - subreg_p, early_clobber_alts, + subreg_p, op, early_clobber_alts, data->regs); else {