From patchwork Fri Oct 20 19:22:39 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tulio Magno Quites Machado Filho X-Patchwork-Id: 23739 Received: (qmail 62924 invoked by alias); 20 Oct 2017 19:24:40 -0000 Mailing-List: contact libc-alpha-help@sourceware.org; run by ezmlm Precedence: bulk List-Id: List-Unsubscribe: List-Subscribe: List-Archive: List-Post: List-Help: , Sender: libc-alpha-owner@sourceware.org Delivered-To: mailing list libc-alpha@sourceware.org Received: (qmail 62791 invoked by uid 89); 20 Oct 2017 19:24:27 -0000 Authentication-Results: sourceware.org; auth=none X-Virus-Found: No X-Spam-SWARE-Status: No, score=-24.1 required=5.0 tests=AWL, BAYES_00, GIT_PATCH_0, GIT_PATCH_1, GIT_PATCH_2, GIT_PATCH_3, KAM_LAZY_DOMAIN_SECURITY, RCVD_IN_DNSWL_LOW autolearn=ham version=3.3.2 spammy= X-HELO: mx0a-001b2d01.pphosted.com From: "Tulio Magno Quites Machado Filho" To: Adhemerval Zanella , "Carlos O'Donell" , libc-alpha@sourceware.org Cc: raji@linux.vnet.ibm.com, fweimer@redhat.com Subject: [PATCHv3] powerpc: Replace lxvd2x/stxvd2x with lvx/stvx in P7's memcpy/memmove Date: Fri, 20 Oct 2017 17:22:39 -0200 In-Reply-To: <2f3d25f5-bbd6-b409-4658-50bc172c258b@linaro.org> References: <2f3d25f5-bbd6-b409-4658-50bc172c258b@linaro.org> X-TM-AS-GCONF: 00 x-cbid: 17102019-0024-0000-0000-000002E5ADF4 X-IBM-SpamModules-Scores: X-IBM-SpamModules-Versions: BY=3.00007925; HX=3.00000241; KW=3.00000007; PH=3.00000004; SC=3.00000239; SDB=6.00933987; UDB=6.00470486; IPR=6.00714276; BA=6.00005651; NDR=6.00000001; ZLA=6.00000005; ZF=6.00000009; ZB=6.00000000; ZP=6.00000000; ZH=6.00000000; ZU=6.00000002; MB=3.00017631; XFM=3.00000015; UTC=2017-10-20 19:24:04 X-IBM-AV-DETECTION: SAVI=unused REMOTE=unused XFE=unused x-cbparentid: 17102019-0025-0000-0000-000045CAB403 Message-Id: <20171020192239.24706-1-tuliom@linux.vnet.ibm.com> X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:, , definitions=2017-10-20_09:, , signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 spamscore=0 suspectscore=0 malwarescore=0 phishscore=0 adultscore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1707230000 definitions=main-1710200270 From: Rajalakshmi Srinivasaraghavan POWER9 DD2.1 and earlier has an issue where some cache inhibited vector load traps to the kernel, causing a performance degradation. To handle this in memcpy and memmove, lvx/stvx is used for aligned addresses instead of lxvd2x/stxvd2x. Reference: https://patchwork.ozlabs.org/patch/814059/ 2017-10-20 Rajalakshmi Srinivasaraghavan * sysdeps/powerpc/powerpc64/power7/memcpy.S: Replace lxvd2x/stxvd2x with lvx/stvx. * sysdeps/powerpc/powerpc64/power7/memmove.S: Likewise. Reviewed-by: Tulio Magno Quites Machado Filho --- sysdeps/powerpc/powerpc64/power7/memcpy.S | 64 +++++++-------- sysdeps/powerpc/powerpc64/power7/memmove.S | 128 ++++++++++++++--------------- 2 files changed, 96 insertions(+), 96 deletions(-) diff --git a/sysdeps/powerpc/powerpc64/power7/memcpy.S b/sysdeps/powerpc/powerpc64/power7/memcpy.S index 1ccbc2e..a7cdf8b 100644 --- a/sysdeps/powerpc/powerpc64/power7/memcpy.S +++ b/sysdeps/powerpc/powerpc64/power7/memcpy.S @@ -91,63 +91,63 @@ L(aligned_copy): srdi 12,cnt,7 cmpdi 12,0 beq L(aligned_tail) - lxvd2x 6,0,src - lxvd2x 7,src,6 + lvx 6,0,src + lvx 7,src,6 mtctr 12 b L(aligned_128loop) .align 4 L(aligned_128head): /* for the 2nd + iteration of this loop. */ - lxvd2x 6,0,src - lxvd2x 7,src,6 + lvx 6,0,src + lvx 7,src,6 L(aligned_128loop): - lxvd2x 8,src,7 - lxvd2x 9,src,8 - stxvd2x 6,0,dst + lvx 8,src,7 + lvx 9,src,8 + stvx 6,0,dst addi src,src,64 - stxvd2x 7,dst,6 - stxvd2x 8,dst,7 - stxvd2x 9,dst,8 - lxvd2x 6,0,src - lxvd2x 7,src,6 + stvx 7,dst,6 + stvx 8,dst,7 + stvx 9,dst,8 + lvx 6,0,src + lvx 7,src,6 addi dst,dst,64 - lxvd2x 8,src,7 - lxvd2x 9,src,8 + lvx 8,src,7 + lvx 9,src,8 addi src,src,64 - stxvd2x 6,0,dst - stxvd2x 7,dst,6 - stxvd2x 8,dst,7 - stxvd2x 9,dst,8 + stvx 6,0,dst + stvx 7,dst,6 + stvx 8,dst,7 + stvx 9,dst,8 addi dst,dst,64 bdnz L(aligned_128head) L(aligned_tail): mtocrf 0x01,cnt bf 25,32f - lxvd2x 6,0,src - lxvd2x 7,src,6 - lxvd2x 8,src,7 - lxvd2x 9,src,8 + lvx 6,0,src + lvx 7,src,6 + lvx 8,src,7 + lvx 9,src,8 addi src,src,64 - stxvd2x 6,0,dst - stxvd2x 7,dst,6 - stxvd2x 8,dst,7 - stxvd2x 9,dst,8 + stvx 6,0,dst + stvx 7,dst,6 + stvx 8,dst,7 + stvx 9,dst,8 addi dst,dst,64 32: bf 26,16f - lxvd2x 6,0,src - lxvd2x 7,src,6 + lvx 6,0,src + lvx 7,src,6 addi src,src,32 - stxvd2x 6,0,dst - stxvd2x 7,dst,6 + stvx 6,0,dst + stvx 7,dst,6 addi dst,dst,32 16: bf 27,8f - lxvd2x 6,0,src + lvx 6,0,src addi src,src,16 - stxvd2x 6,0,dst + stvx 6,0,dst addi dst,dst,16 8: bf 28,4f diff --git a/sysdeps/powerpc/powerpc64/power7/memmove.S b/sysdeps/powerpc/powerpc64/power7/memmove.S index 93baa69..667c6e2 100644 --- a/sysdeps/powerpc/powerpc64/power7/memmove.S +++ b/sysdeps/powerpc/powerpc64/power7/memmove.S @@ -92,63 +92,63 @@ L(aligned_copy): srdi 12,r5,7 cmpdi 12,0 beq L(aligned_tail) - lxvd2x 6,0,r4 - lxvd2x 7,r4,6 + lvx 6,0,r4 + lvx 7,r4,6 mtctr 12 b L(aligned_128loop) .align 4 L(aligned_128head): /* for the 2nd + iteration of this loop. */ - lxvd2x 6,0,r4 - lxvd2x 7,r4,6 + lvx 6,0,r4 + lvx 7,r4,6 L(aligned_128loop): - lxvd2x 8,r4,7 - lxvd2x 9,r4,8 - stxvd2x 6,0,r11 + lvx 8,r4,7 + lvx 9,r4,8 + stvx 6,0,r11 addi r4,r4,64 - stxvd2x 7,r11,6 - stxvd2x 8,r11,7 - stxvd2x 9,r11,8 - lxvd2x 6,0,r4 - lxvd2x 7,r4,6 + stvx 7,r11,6 + stvx 8,r11,7 + stvx 9,r11,8 + lvx 6,0,r4 + lvx 7,r4,6 addi r11,r11,64 - lxvd2x 8,r4,7 - lxvd2x 9,r4,8 + lvx 8,r4,7 + lvx 9,r4,8 addi r4,r4,64 - stxvd2x 6,0,r11 - stxvd2x 7,r11,6 - stxvd2x 8,r11,7 - stxvd2x 9,r11,8 + stvx 6,0,r11 + stvx 7,r11,6 + stvx 8,r11,7 + stvx 9,r11,8 addi r11,r11,64 bdnz L(aligned_128head) L(aligned_tail): mtocrf 0x01,r5 bf 25,32f - lxvd2x 6,0,r4 - lxvd2x 7,r4,6 - lxvd2x 8,r4,7 - lxvd2x 9,r4,8 + lvx 6,0,r4 + lvx 7,r4,6 + lvx 8,r4,7 + lvx 9,r4,8 addi r4,r4,64 - stxvd2x 6,0,r11 - stxvd2x 7,r11,6 - stxvd2x 8,r11,7 - stxvd2x 9,r11,8 + stvx 6,0,r11 + stvx 7,r11,6 + stvx 8,r11,7 + stvx 9,r11,8 addi r11,r11,64 32: bf 26,16f - lxvd2x 6,0,r4 - lxvd2x 7,r4,6 + lvx 6,0,r4 + lvx 7,r4,6 addi r4,r4,32 - stxvd2x 6,0,r11 - stxvd2x 7,r11,6 + stvx 6,0,r11 + stvx 7,r11,6 addi r11,r11,32 16: bf 27,8f - lxvd2x 6,0,r4 + lvx 6,0,r4 addi r4,r4,16 - stxvd2x 6,0,r11 + stvx 6,0,r11 addi r11,r11,16 8: bf 28,4f @@ -488,63 +488,63 @@ L(aligned_copy_bwd): srdi r12,r5,7 cmpdi r12,0 beq L(aligned_tail_bwd) - lxvd2x v6,r4,r6 - lxvd2x v7,r4,r7 + lvx v6,r4,r6 + lvx v7,r4,r7 mtctr 12 b L(aligned_128loop_bwd) .align 4 L(aligned_128head_bwd): /* for the 2nd + iteration of this loop. */ - lxvd2x v6,r4,r6 - lxvd2x v7,r4,r7 + lvx v6,r4,r6 + lvx v7,r4,r7 L(aligned_128loop_bwd): - lxvd2x v8,r4,r8 - lxvd2x v9,r4,r9 - stxvd2x v6,r11,r6 + lvx v8,r4,r8 + lvx v9,r4,r9 + stvx v6,r11,r6 subi r4,r4,64 - stxvd2x v7,r11,r7 - stxvd2x v8,r11,r8 - stxvd2x v9,r11,r9 - lxvd2x v6,r4,r6 - lxvd2x v7,r4,7 + stvx v7,r11,r7 + stvx v8,r11,r8 + stvx v9,r11,r9 + lvx v6,r4,r6 + lvx v7,r4,7 subi r11,r11,64 - lxvd2x v8,r4,r8 - lxvd2x v9,r4,r9 + lvx v8,r4,r8 + lvx v9,r4,r9 subi r4,r4,64 - stxvd2x v6,r11,r6 - stxvd2x v7,r11,r7 - stxvd2x v8,r11,r8 - stxvd2x v9,r11,r9 + stvx v6,r11,r6 + stvx v7,r11,r7 + stvx v8,r11,r8 + stvx v9,r11,r9 subi r11,r11,64 bdnz L(aligned_128head_bwd) L(aligned_tail_bwd): mtocrf 0x01,r5 bf 25,32f - lxvd2x v6,r4,r6 - lxvd2x v7,r4,r7 - lxvd2x v8,r4,r8 - lxvd2x v9,r4,r9 + lvx v6,r4,r6 + lvx v7,r4,r7 + lvx v8,r4,r8 + lvx v9,r4,r9 subi r4,r4,64 - stxvd2x v6,r11,r6 - stxvd2x v7,r11,r7 - stxvd2x v8,r11,r8 - stxvd2x v9,r11,r9 + stvx v6,r11,r6 + stvx v7,r11,r7 + stvx v8,r11,r8 + stvx v9,r11,r9 subi r11,r11,64 32: bf 26,16f - lxvd2x v6,r4,r6 - lxvd2x v7,r4,r7 + lvx v6,r4,r6 + lvx v7,r4,r7 subi r4,r4,32 - stxvd2x v6,r11,r6 - stxvd2x v7,r11,r7 + stvx v6,r11,r6 + stvx v7,r11,r7 subi r11,r11,32 16: bf 27,8f - lxvd2x v6,r4,r6 + lvx v6,r4,r6 subi r4,r4,16 - stxvd2x v6,r11,r6 + stvx v6,r11,r6 subi r11,r11,16 8: bf 28,4f