diff mbox series

[v2] x86: Set rep_movsb_threshold to 2112 on processors with FSRM

Message ID CAMe9rOr1Sjcsuh1WdoDGNf5rFCK61B7aTAgTzsyA81vep2Khwg@mail.gmail.com
State Committed
Headers
Series [v2] x86: Set rep_movsb_threshold to 2112 on processors with FSRM |

Commit Message

H.J. Lu May 1, 2021, 2:08 p.m. UTC 
  On Fri, Apr 30, 2021 at 6:26 PM H.J. Lu <hjl.tools@gmail.com> wrote:
>
> On Fri, Apr 30, 2021 at 5:17 PM Noah Goldstein <goldstein.w.n@gmail.com> wrote:
> >
> > On Fri, Apr 30, 2021 at 4:51 PM H.J. Lu <hjl.tools@gmail.com> wrote:
> > >
> > > On Fri, Apr 30, 2021 at 3:59 PM Noah Goldstein <goldstein.w.n@gmail.com> wrote:
> > > >
> > > > On Fri, Apr 30, 2021 at 4:43 PM H.J. Lu via Libc-alpha
> > > > <libc-alpha@sourceware.org> wrote:
> > > > >
> > > > > The glibc memcpy benchmark on Intel Core i7-1065G7 (Ice Lake) showed
> > > > > that REP MOVSB became faster after 2112 bytes:
> > > > >
> > > > >                                     Vector + REP MOVSB  REP MOVSB
> > > > > length=2112, align1=0, align2=0:        24.20             24.40
> > > > > length=2112, align1=1, align2=0:        26.07             23.13
> > > > > length=2112, align1=0, align2=1:        27.18             28.13
> > > > > length=2112, align1=1, align2=1:        26.23             25.16
> > > > > length=2176, align1=0, align2=0:        23.18             22.52
> > > > > length=2176, align1=2, align2=0:        25.45             22.52
> > > > > length=2176, align1=0, align2=2:        27.14             27.82
> > > > > length=2176, align1=2, align2=2:        22.73             25.56
> > > > > length=2240, align1=0, align2=0:        24.62             24.25
> > > > > length=2240, align1=3, align2=0:        29.77             27.15
> > > > > length=2240, align1=0, align2=3:        35.55             29.93
> > > > > length=2240, align1=3, align2=3:        34.49             25.15
> > > > > length=2304, align1=0, align2=0:        34.75             26.64
> > > > > length=2304, align1=4, align2=0:        32.09             22.63
> > > > > length=2304, align1=0, align2=4:        28.43             31.24
> > > > >
> > > >
> > > > Do you know what is happening at:
> > > > length=2304, align1=0, align2=4:        28.43             31.24
> > > >
> > > > Seems align2 > align1 gets worse for larger sizes as well.
> > > >
> > > > I.e from my Icelake (1 run):
> > > >                                                               w/o
> > > > Patch      w/ Patch
> > > > length=3648, align1=0, align2=25:        72.91             83.99
> > > > length=3712, align1=0, align2=26:        72.52             84.92
> > > > length=3776, align1=0, align2=27:        76.16             84.02
> > > > length=3840, align1=0, align2=28:        75.44             90.13
> > > > length=3904, align1=0, align2=29:        81.62             84.43
> > > > length=3968, align1=0, align2=30:        76.82             93.39
> > > > length=4032, align1=0, align2=31:        80.01             89.89
> > > > length=4096, align1=0, align2=32:        72.89             97.50
> > >
> > > On Intel Core i7-1165G7 (Tiger Lake), I got
> >
> > Where the original numbers icl or tgl? But was just wondering
>
> Here are numbers on Core i7-1065G7 (Ice Lake):
>
>                                     __memcpy_evex_unaligned_erms __memcpy_erms
>      length=2048, align1=0, align2=0:        21.51        22.52
>      length=2048, align1=0, align2=0:        22.11        21.50
>      length=2048, align1=0, align2=0:        21.96        22.10
>      length=2048, align1=0, align2=0:        22.58        21.51
>      length=2112, align1=0, align2=0:        24.20        24.40
>      length=2112, align1=1, align2=0:        26.07        23.13
>      length=2112, align1=0, align2=1:        27.18        28.13
>      length=2112, align1=1, align2=1:        26.23        25.16
>      length=2176, align1=0, align2=0:        23.18        22.52
>      length=2176, align1=2, align2=0:        25.45        22.52
>      length=2176, align1=0, align2=2:        27.14        35.82
>      length=2176, align1=2, align2=2:        22.73        25.56
>      length=2240, align1=0, align2=0:        24.62        24.25
>      length=2240, align1=3, align2=0:        29.77        27.15
>      length=2240, align1=0, align2=3:        35.55        29.93
>      length=2240, align1=3, align2=3:        34.49        25.15
>      length=2304, align1=0, align2=0:        34.75        26.64
>      length=2304, align1=4, align2=0:        32.09        22.63
>      length=2304, align1=0, align2=4:        28.43        31.24
>      length=2304, align1=4, align2=4:        23.95        25.56
>      length=2368, align1=0, align2=0:        25.57        24.34
>      length=2368, align1=5, align2=0:        30.57        23.94
>      length=2368, align1=0, align2=5:        31.29        31.24
>      length=2368, align1=5, align2=5:        25.57        26.69
>      length=2432, align1=0, align2=0:        25.16        23.77
>      length=2432, align1=6, align2=0:        29.27        23.13
>      length=2432, align1=0, align2=6:        30.40        33.77
>      length=2432, align1=6, align2=6:        25.16        26.37
>      length=2496, align1=0, align2=0:        26.85        24.34
>      length=2496, align1=7, align2=0:        32.04        24.75
>      length=2496, align1=0, align2=7:        32.70        33.06
>      length=2496, align1=7, align2=7:        26.79        25.97
>      length=2560, align1=0, align2=0:        61.84        23.53
>      length=2560, align1=8, align2=0:        30.74        23.53
>      length=2560, align1=0, align2=8:        33.35        35.44
>      length=2560, align1=8, align2=8:        27.94        27.52
>      length=2624, align1=0, align2=0:        27.60        24.75
>      length=2624, align1=9, align2=0:        33.28        25.16
>      length=2624, align1=0, align2=9:        34.14        34.48
>      length=2624, align1=9, align2=9:        28.00        26.78
>      length=2688, align1=0, align2=0:        38.94        24.61
>     length=2688, align1=10, align2=0:        31.23        23.94
>     length=2688, align1=0, align2=10:        33.89        37.11
>    length=2688, align1=10, align2=10:        28.40        27.18
>      length=2752, align1=0, align2=0:        36.17        25.56
>     length=2752, align1=11, align2=0:        35.88        25.57
>     length=2752, align1=0, align2=11:        35.40        36.11
>    length=2752, align1=11, align2=11:        28.82        27.94
>      length=2816, align1=0, align2=0:        33.78        24.54
>     length=2816, align1=12, align2=0:        33.75        25.96
>     length=2816, align1=0, align2=12:        34.24        37.73
>    length=2816, align1=12, align2=12:        29.22        27.59
>      length=2880, align1=0, align2=0:        39.76        25.97
>     length=2880, align1=13, align2=0:        41.26        26.66
>     length=2880, align1=0, align2=13:        40.72        37.73
>    length=2880, align1=13, align2=13:        35.13        27.59
>      length=2944, align1=0, align2=0:        32.09        24.75
>     length=2944, align1=14, align2=0:        40.27        26.78
>     length=2944, align1=0, align2=14:        45.59        39.55
>    length=2944, align1=14, align2=14:        40.94        28.00
>      length=3008, align1=0, align2=0:        42.17        25.97
>     length=3008, align1=15, align2=0:        41.81        27.07
>     length=3008, align1=0, align2=15:        42.67        39.35
>    length=3008, align1=15, align2=15:        42.18        28.00
>      length=3072, align1=0, align2=0:        34.71        25.16
>     length=3072, align1=16, align2=0:        43.31        26.37
>     length=3072, align1=0, align2=16:        43.41        40.98
>    length=3072, align1=16, align2=16:        36.62        27.99
>      length=3136, align1=0, align2=0:        45.75        26.78
>     length=3136, align1=17, align2=0:        43.29        27.59
>     length=3136, align1=0, align2=17:        44.44        41.16
>    length=3136, align1=17, align2=17:        34.99        28.59
>      length=3200, align1=0, align2=0:        43.12        25.16
>     length=3200, align1=18, align2=0:        43.04        27.59
>     length=3200, align1=0, align2=18:        45.08        42.59
>    length=3200, align1=18, align2=18:        36.00        28.81
>      length=3264, align1=0, align2=0:        43.01        27.48
>     length=3264, align1=19, align2=0:        47.72        28.61
>     length=3264, align1=0, align2=19:        47.00        42.60
>    length=3264, align1=19, align2=19:        44.44        28.81
>      length=3328, align1=0, align2=0:        35.50        25.75
>     length=3328, align1=20, align2=0:        44.84        27.19
>     length=3328, align1=0, align2=20:        46.38        44.41
>    length=3328, align1=20, align2=20:        33.68        28.81
>      length=3392, align1=0, align2=0:        43.82        27.18
>     length=3392, align1=21, align2=0:        45.91        28.81
>     length=3392, align1=0, align2=21:        47.61        43.06
>    length=3392, align1=21, align2=21:        43.82        29.21
>      length=3456, align1=0, align2=0:        43.83        26.64
>     length=3456, align1=22, align2=0:        44.77        29.49
>     length=3456, align1=0, align2=22:        47.57        46.01
>    length=3456, align1=22, align2=22:        44.11        29.62
>      length=3520, align1=0, align2=0:        45.04        27.99
>     length=3520, align1=23, align2=0:        46.59        30.55
>     length=3520, align1=0, align2=23:        50.58        44.80
>    length=3520, align1=23, align2=23:        43.85        29.21
>      length=3584, align1=0, align2=0:        45.04        26.88
>     length=3584, align1=24, align2=0:        47.22        29.21
>     length=3584, align1=0, align2=24:        50.12        47.47
>    length=3584, align1=24, align2=24:        45.04        29.80
>      length=3648, align1=0, align2=0:        47.13        28.40
>     length=3648, align1=25, align2=0:        48.78        30.94
>     length=3648, align1=0, align2=25:        49.25        61.42
>    length=3648, align1=25, align2=25:        45.85        30.02
>      length=3712, align1=0, align2=0:        47.24        27.19
>     length=3712, align1=26, align2=0:        48.41        30.79
>     length=3712, align1=0, align2=26:        49.65        51.57
>    length=3712, align1=26, align2=26:        48.76        30.33
>      length=3776, align1=0, align2=0:        47.35        34.61
>     length=3776, align1=27, align2=0:        49.16        31.99
>     length=3776, align1=0, align2=27:        52.06        49.38
>    length=3776, align1=27, align2=27:        47.36        30.43
>      length=3840, align1=0, align2=0:        49.70        28.35
>     length=3840, align1=28, align2=0:        50.29        31.81
>     length=3840, align1=0, align2=28:        51.15        50.71
>    length=3840, align1=28, align2=28:        47.07        31.65
>      length=3904, align1=0, align2=0:        48.28        29.21
>     length=3904, align1=29, align2=0:        51.96        32.99
>     length=3904, align1=0, align2=29:        53.24        50.71
>    length=3904, align1=29, align2=29:        47.88        31.64
>      length=3968, align1=0, align2=0:        47.89        29.19
>     length=3968, align1=30, align2=0:        51.52        32.57
>     length=3968, align1=0, align2=30:        54.03        52.33
>    length=3968, align1=30, align2=30:        46.62        33.15
>      length=4032, align1=0, align2=0:        49.77        29.62
>     length=4032, align1=31, align2=0:        53.62        34.27
>     length=4032, align1=0, align2=31:        56.44        52.33
>    length=4032, align1=31, align2=31:        48.94        33.16
>      length=4096, align1=0, align2=0:        49.37        29.21
>     length=4096, align1=32, align2=0:        50.46        33.00
>     length=4096, align1=0, align2=32:        49.31        55.14
>    length=4096, align1=32, align2=32:        49.28        33.05
>
> > if the align2 != 0, align1 = 0 case on Icelake warranted more investigation.
> > That being said the change is a clear winner on Icelake as well
> > for any other alignment schema.
>
> True, REP MOVSB is more efficient.
>

This is the v2 patch I am checking in next week.

Thanks.

Comments

Noah Goldstein May 1, 2021, 6:45 p.m. UTC | #1 
On Sat, May 1, 2021 at 10:09 AM H.J. Lu <hjl.tools@gmail.com> wrote:
>
> On Fri, Apr 30, 2021 at 6:26 PM H.J. Lu <hjl.tools@gmail.com> wrote:
> >
> > On Fri, Apr 30, 2021 at 5:17 PM Noah Goldstein <goldstein.w.n@gmail.com> wrote:
> > >
> > > On Fri, Apr 30, 2021 at 4:51 PM H.J. Lu <hjl.tools@gmail.com> wrote:
> > > >
> > > > On Fri, Apr 30, 2021 at 3:59 PM Noah Goldstein <goldstein.w.n@gmail.com> wrote:
> > > > >
> > > > > On Fri, Apr 30, 2021 at 4:43 PM H.J. Lu via Libc-alpha
> > > > > <libc-alpha@sourceware.org> wrote:
> > > > > >
> > > > > > The glibc memcpy benchmark on Intel Core i7-1065G7 (Ice Lake) showed
> > > > > > that REP MOVSB became faster after 2112 bytes:
> > > > > >
> > > > > >                                     Vector + REP MOVSB  REP MOVSB
> > > > > > length=2112, align1=0, align2=0:        24.20             24.40
> > > > > > length=2112, align1=1, align2=0:        26.07             23.13
> > > > > > length=2112, align1=0, align2=1:        27.18             28.13
> > > > > > length=2112, align1=1, align2=1:        26.23             25.16
> > > > > > length=2176, align1=0, align2=0:        23.18             22.52
> > > > > > length=2176, align1=2, align2=0:        25.45             22.52
> > > > > > length=2176, align1=0, align2=2:        27.14             27.82
> > > > > > length=2176, align1=2, align2=2:        22.73             25.56
> > > > > > length=2240, align1=0, align2=0:        24.62             24.25
> > > > > > length=2240, align1=3, align2=0:        29.77             27.15
> > > > > > length=2240, align1=0, align2=3:        35.55             29.93
> > > > > > length=2240, align1=3, align2=3:        34.49             25.15
> > > > > > length=2304, align1=0, align2=0:        34.75             26.64
> > > > > > length=2304, align1=4, align2=0:        32.09             22.63
> > > > > > length=2304, align1=0, align2=4:        28.43             31.24
> > > > > >
> > > > >
> > > > > Do you know what is happening at:
> > > > > length=2304, align1=0, align2=4:        28.43             31.24
> > > > >
> > > > > Seems align2 > align1 gets worse for larger sizes as well.
> > > > >
> > > > > I.e from my Icelake (1 run):
> > > > >                                                               w/o
> > > > > Patch      w/ Patch
> > > > > length=3648, align1=0, align2=25:        72.91             83.99
> > > > > length=3712, align1=0, align2=26:        72.52             84.92
> > > > > length=3776, align1=0, align2=27:        76.16             84.02
> > > > > length=3840, align1=0, align2=28:        75.44             90.13
> > > > > length=3904, align1=0, align2=29:        81.62             84.43
> > > > > length=3968, align1=0, align2=30:        76.82             93.39
> > > > > length=4032, align1=0, align2=31:        80.01             89.89
> > > > > length=4096, align1=0, align2=32:        72.89             97.50
> > > >
> > > > On Intel Core i7-1165G7 (Tiger Lake), I got
> > >
> > > Where the original numbers icl or tgl? But was just wondering
> >
> > Here are numbers on Core i7-1065G7 (Ice Lake):
> >
> >                                     __memcpy_evex_unaligned_erms __memcpy_erms
> >      length=2048, align1=0, align2=0:        21.51        22.52
> >      length=2048, align1=0, align2=0:        22.11        21.50
> >      length=2048, align1=0, align2=0:        21.96        22.10
> >      length=2048, align1=0, align2=0:        22.58        21.51
> >      length=2112, align1=0, align2=0:        24.20        24.40
> >      length=2112, align1=1, align2=0:        26.07        23.13
> >      length=2112, align1=0, align2=1:        27.18        28.13
> >      length=2112, align1=1, align2=1:        26.23        25.16
> >      length=2176, align1=0, align2=0:        23.18        22.52
> >      length=2176, align1=2, align2=0:        25.45        22.52
> >      length=2176, align1=0, align2=2:        27.14        35.82
> >      length=2176, align1=2, align2=2:        22.73        25.56
> >      length=2240, align1=0, align2=0:        24.62        24.25
> >      length=2240, align1=3, align2=0:        29.77        27.15
> >      length=2240, align1=0, align2=3:        35.55        29.93
> >      length=2240, align1=3, align2=3:        34.49        25.15
> >      length=2304, align1=0, align2=0:        34.75        26.64
> >      length=2304, align1=4, align2=0:        32.09        22.63
> >      length=2304, align1=0, align2=4:        28.43        31.24
> >      length=2304, align1=4, align2=4:        23.95        25.56
> >      length=2368, align1=0, align2=0:        25.57        24.34
> >      length=2368, align1=5, align2=0:        30.57        23.94
> >      length=2368, align1=0, align2=5:        31.29        31.24
> >      length=2368, align1=5, align2=5:        25.57        26.69
> >      length=2432, align1=0, align2=0:        25.16        23.77
> >      length=2432, align1=6, align2=0:        29.27        23.13
> >      length=2432, align1=0, align2=6:        30.40        33.77
> >      length=2432, align1=6, align2=6:        25.16        26.37
> >      length=2496, align1=0, align2=0:        26.85        24.34
> >      length=2496, align1=7, align2=0:        32.04        24.75
> >      length=2496, align1=0, align2=7:        32.70        33.06
> >      length=2496, align1=7, align2=7:        26.79        25.97
> >      length=2560, align1=0, align2=0:        61.84        23.53
> >      length=2560, align1=8, align2=0:        30.74        23.53
> >      length=2560, align1=0, align2=8:        33.35        35.44
> >      length=2560, align1=8, align2=8:        27.94        27.52
> >      length=2624, align1=0, align2=0:        27.60        24.75
> >      length=2624, align1=9, align2=0:        33.28        25.16
> >      length=2624, align1=0, align2=9:        34.14        34.48
> >      length=2624, align1=9, align2=9:        28.00        26.78
> >      length=2688, align1=0, align2=0:        38.94        24.61
> >     length=2688, align1=10, align2=0:        31.23        23.94
> >     length=2688, align1=0, align2=10:        33.89        37.11
> >    length=2688, align1=10, align2=10:        28.40        27.18
> >      length=2752, align1=0, align2=0:        36.17        25.56
> >     length=2752, align1=11, align2=0:        35.88        25.57
> >     length=2752, align1=0, align2=11:        35.40        36.11
> >    length=2752, align1=11, align2=11:        28.82        27.94
> >      length=2816, align1=0, align2=0:        33.78        24.54
> >     length=2816, align1=12, align2=0:        33.75        25.96
> >     length=2816, align1=0, align2=12:        34.24        37.73
> >    length=2816, align1=12, align2=12:        29.22        27.59
> >      length=2880, align1=0, align2=0:        39.76        25.97
> >     length=2880, align1=13, align2=0:        41.26        26.66
> >     length=2880, align1=0, align2=13:        40.72        37.73
> >    length=2880, align1=13, align2=13:        35.13        27.59
> >      length=2944, align1=0, align2=0:        32.09        24.75
> >     length=2944, align1=14, align2=0:        40.27        26.78
> >     length=2944, align1=0, align2=14:        45.59        39.55
> >    length=2944, align1=14, align2=14:        40.94        28.00
> >      length=3008, align1=0, align2=0:        42.17        25.97
> >     length=3008, align1=15, align2=0:        41.81        27.07
> >     length=3008, align1=0, align2=15:        42.67        39.35
> >    length=3008, align1=15, align2=15:        42.18        28.00
> >      length=3072, align1=0, align2=0:        34.71        25.16
> >     length=3072, align1=16, align2=0:        43.31        26.37
> >     length=3072, align1=0, align2=16:        43.41        40.98
> >    length=3072, align1=16, align2=16:        36.62        27.99
> >      length=3136, align1=0, align2=0:        45.75        26.78
> >     length=3136, align1=17, align2=0:        43.29        27.59
> >     length=3136, align1=0, align2=17:        44.44        41.16
> >    length=3136, align1=17, align2=17:        34.99        28.59
> >      length=3200, align1=0, align2=0:        43.12        25.16
> >     length=3200, align1=18, align2=0:        43.04        27.59
> >     length=3200, align1=0, align2=18:        45.08        42.59
> >    length=3200, align1=18, align2=18:        36.00        28.81
> >      length=3264, align1=0, align2=0:        43.01        27.48
> >     length=3264, align1=19, align2=0:        47.72        28.61
> >     length=3264, align1=0, align2=19:        47.00        42.60
> >    length=3264, align1=19, align2=19:        44.44        28.81
> >      length=3328, align1=0, align2=0:        35.50        25.75
> >     length=3328, align1=20, align2=0:        44.84        27.19
> >     length=3328, align1=0, align2=20:        46.38        44.41
> >    length=3328, align1=20, align2=20:        33.68        28.81
> >      length=3392, align1=0, align2=0:        43.82        27.18
> >     length=3392, align1=21, align2=0:        45.91        28.81
> >     length=3392, align1=0, align2=21:        47.61        43.06
> >    length=3392, align1=21, align2=21:        43.82        29.21
> >      length=3456, align1=0, align2=0:        43.83        26.64
> >     length=3456, align1=22, align2=0:        44.77        29.49
> >     length=3456, align1=0, align2=22:        47.57        46.01
> >    length=3456, align1=22, align2=22:        44.11        29.62
> >      length=3520, align1=0, align2=0:        45.04        27.99
> >     length=3520, align1=23, align2=0:        46.59        30.55
> >     length=3520, align1=0, align2=23:        50.58        44.80
> >    length=3520, align1=23, align2=23:        43.85        29.21
> >      length=3584, align1=0, align2=0:        45.04        26.88
> >     length=3584, align1=24, align2=0:        47.22        29.21
> >     length=3584, align1=0, align2=24:        50.12        47.47
> >    length=3584, align1=24, align2=24:        45.04        29.80
> >      length=3648, align1=0, align2=0:        47.13        28.40
> >     length=3648, align1=25, align2=0:        48.78        30.94
> >     length=3648, align1=0, align2=25:        49.25        61.42
> >    length=3648, align1=25, align2=25:        45.85        30.02
> >      length=3712, align1=0, align2=0:        47.24        27.19
> >     length=3712, align1=26, align2=0:        48.41        30.79
> >     length=3712, align1=0, align2=26:        49.65        51.57
> >    length=3712, align1=26, align2=26:        48.76        30.33
> >      length=3776, align1=0, align2=0:        47.35        34.61
> >     length=3776, align1=27, align2=0:        49.16        31.99
> >     length=3776, align1=0, align2=27:        52.06        49.38
> >    length=3776, align1=27, align2=27:        47.36        30.43
> >      length=3840, align1=0, align2=0:        49.70        28.35
> >     length=3840, align1=28, align2=0:        50.29        31.81
> >     length=3840, align1=0, align2=28:        51.15        50.71
> >    length=3840, align1=28, align2=28:        47.07        31.65
> >      length=3904, align1=0, align2=0:        48.28        29.21
> >     length=3904, align1=29, align2=0:        51.96        32.99
> >     length=3904, align1=0, align2=29:        53.24        50.71
> >    length=3904, align1=29, align2=29:        47.88        31.64
> >      length=3968, align1=0, align2=0:        47.89        29.19
> >     length=3968, align1=30, align2=0:        51.52        32.57
> >     length=3968, align1=0, align2=30:        54.03        52.33
> >    length=3968, align1=30, align2=30:        46.62        33.15
> >      length=4032, align1=0, align2=0:        49.77        29.62
> >     length=4032, align1=31, align2=0:        53.62        34.27
> >     length=4032, align1=0, align2=31:        56.44        52.33
> >    length=4032, align1=31, align2=31:        48.94        33.16
> >      length=4096, align1=0, align2=0:        49.37        29.21
> >     length=4096, align1=32, align2=0:        50.46        33.00
> >     length=4096, align1=0, align2=32:        49.31        55.14
> >    length=4096, align1=32, align2=32:        49.28        33.05
> >
> > > if the align2 != 0, align1 = 0 case on Icelake warranted more investigation.
> > > That being said the change is a clear winner on Icelake as well
> > > for any other alignment schema.
> >
> > True, REP MOVSB is more efficient.
> >
>
> This is the v2 patch I am checking in next week.

Sorry! LGTM. Change is a big net gain :)

>
> Thanks.
>
> --
> H.J.
Sunil Pandey April 27, 2022, 11:56 p.m. UTC | #2 
On Sat, May 1, 2021 at 12:47 PM Noah Goldstein via Libc-alpha
<libc-alpha@sourceware.org> wrote:
>
> On Sat, May 1, 2021 at 10:09 AM H.J. Lu <hjl.tools@gmail.com> wrote:
> >
> > On Fri, Apr 30, 2021 at 6:26 PM H.J. Lu <hjl.tools@gmail.com> wrote:
> > >
> > > On Fri, Apr 30, 2021 at 5:17 PM Noah Goldstein <goldstein.w.n@gmail.com> wrote:
> > > >
> > > > On Fri, Apr 30, 2021 at 4:51 PM H.J. Lu <hjl.tools@gmail.com> wrote:
> > > > >
> > > > > On Fri, Apr 30, 2021 at 3:59 PM Noah Goldstein <goldstein.w.n@gmail.com> wrote:
> > > > > >
> > > > > > On Fri, Apr 30, 2021 at 4:43 PM H.J. Lu via Libc-alpha
> > > > > > <libc-alpha@sourceware.org> wrote:
> > > > > > >
> > > > > > > The glibc memcpy benchmark on Intel Core i7-1065G7 (Ice Lake) showed
> > > > > > > that REP MOVSB became faster after 2112 bytes:
> > > > > > >
> > > > > > >                                     Vector + REP MOVSB  REP MOVSB
> > > > > > > length=2112, align1=0, align2=0:        24.20             24.40
> > > > > > > length=2112, align1=1, align2=0:        26.07             23.13
> > > > > > > length=2112, align1=0, align2=1:        27.18             28.13
> > > > > > > length=2112, align1=1, align2=1:        26.23             25.16
> > > > > > > length=2176, align1=0, align2=0:        23.18             22.52
> > > > > > > length=2176, align1=2, align2=0:        25.45             22.52
> > > > > > > length=2176, align1=0, align2=2:        27.14             27.82
> > > > > > > length=2176, align1=2, align2=2:        22.73             25.56
> > > > > > > length=2240, align1=0, align2=0:        24.62             24.25
> > > > > > > length=2240, align1=3, align2=0:        29.77             27.15
> > > > > > > length=2240, align1=0, align2=3:        35.55             29.93
> > > > > > > length=2240, align1=3, align2=3:        34.49             25.15
> > > > > > > length=2304, align1=0, align2=0:        34.75             26.64
> > > > > > > length=2304, align1=4, align2=0:        32.09             22.63
> > > > > > > length=2304, align1=0, align2=4:        28.43             31.24
> > > > > > >
> > > > > >
> > > > > > Do you know what is happening at:
> > > > > > length=2304, align1=0, align2=4:        28.43             31.24
> > > > > >
> > > > > > Seems align2 > align1 gets worse for larger sizes as well.
> > > > > >
> > > > > > I.e from my Icelake (1 run):
> > > > > >                                                               w/o
> > > > > > Patch      w/ Patch
> > > > > > length=3648, align1=0, align2=25:        72.91             83.99
> > > > > > length=3712, align1=0, align2=26:        72.52             84.92
> > > > > > length=3776, align1=0, align2=27:        76.16             84.02
> > > > > > length=3840, align1=0, align2=28:        75.44             90.13
> > > > > > length=3904, align1=0, align2=29:        81.62             84.43
> > > > > > length=3968, align1=0, align2=30:        76.82             93.39
> > > > > > length=4032, align1=0, align2=31:        80.01             89.89
> > > > > > length=4096, align1=0, align2=32:        72.89             97.50
> > > > >
> > > > > On Intel Core i7-1165G7 (Tiger Lake), I got
> > > >
> > > > Where the original numbers icl or tgl? But was just wondering
> > >
> > > Here are numbers on Core i7-1065G7 (Ice Lake):
> > >
> > >                                     __memcpy_evex_unaligned_erms __memcpy_erms
> > >      length=2048, align1=0, align2=0:        21.51        22.52
> > >      length=2048, align1=0, align2=0:        22.11        21.50
> > >      length=2048, align1=0, align2=0:        21.96        22.10
> > >      length=2048, align1=0, align2=0:        22.58        21.51
> > >      length=2112, align1=0, align2=0:        24.20        24.40
> > >      length=2112, align1=1, align2=0:        26.07        23.13
> > >      length=2112, align1=0, align2=1:        27.18        28.13
> > >      length=2112, align1=1, align2=1:        26.23        25.16
> > >      length=2176, align1=0, align2=0:        23.18        22.52
> > >      length=2176, align1=2, align2=0:        25.45        22.52
> > >      length=2176, align1=0, align2=2:        27.14        35.82
> > >      length=2176, align1=2, align2=2:        22.73        25.56
> > >      length=2240, align1=0, align2=0:        24.62        24.25
> > >      length=2240, align1=3, align2=0:        29.77        27.15
> > >      length=2240, align1=0, align2=3:        35.55        29.93
> > >      length=2240, align1=3, align2=3:        34.49        25.15
> > >      length=2304, align1=0, align2=0:        34.75        26.64
> > >      length=2304, align1=4, align2=0:        32.09        22.63
> > >      length=2304, align1=0, align2=4:        28.43        31.24
> > >      length=2304, align1=4, align2=4:        23.95        25.56
> > >      length=2368, align1=0, align2=0:        25.57        24.34
> > >      length=2368, align1=5, align2=0:        30.57        23.94
> > >      length=2368, align1=0, align2=5:        31.29        31.24
> > >      length=2368, align1=5, align2=5:        25.57        26.69
> > >      length=2432, align1=0, align2=0:        25.16        23.77
> > >      length=2432, align1=6, align2=0:        29.27        23.13
> > >      length=2432, align1=0, align2=6:        30.40        33.77
> > >      length=2432, align1=6, align2=6:        25.16        26.37
> > >      length=2496, align1=0, align2=0:        26.85        24.34
> > >      length=2496, align1=7, align2=0:        32.04        24.75
> > >      length=2496, align1=0, align2=7:        32.70        33.06
> > >      length=2496, align1=7, align2=7:        26.79        25.97
> > >      length=2560, align1=0, align2=0:        61.84        23.53
> > >      length=2560, align1=8, align2=0:        30.74        23.53
> > >      length=2560, align1=0, align2=8:        33.35        35.44
> > >      length=2560, align1=8, align2=8:        27.94        27.52
> > >      length=2624, align1=0, align2=0:        27.60        24.75
> > >      length=2624, align1=9, align2=0:        33.28        25.16
> > >      length=2624, align1=0, align2=9:        34.14        34.48
> > >      length=2624, align1=9, align2=9:        28.00        26.78
> > >      length=2688, align1=0, align2=0:        38.94        24.61
> > >     length=2688, align1=10, align2=0:        31.23        23.94
> > >     length=2688, align1=0, align2=10:        33.89        37.11
> > >    length=2688, align1=10, align2=10:        28.40        27.18
> > >      length=2752, align1=0, align2=0:        36.17        25.56
> > >     length=2752, align1=11, align2=0:        35.88        25.57
> > >     length=2752, align1=0, align2=11:        35.40        36.11
> > >    length=2752, align1=11, align2=11:        28.82        27.94
> > >      length=2816, align1=0, align2=0:        33.78        24.54
> > >     length=2816, align1=12, align2=0:        33.75        25.96
> > >     length=2816, align1=0, align2=12:        34.24        37.73
> > >    length=2816, align1=12, align2=12:        29.22        27.59
> > >      length=2880, align1=0, align2=0:        39.76        25.97
> > >     length=2880, align1=13, align2=0:        41.26        26.66
> > >     length=2880, align1=0, align2=13:        40.72        37.73
> > >    length=2880, align1=13, align2=13:        35.13        27.59
> > >      length=2944, align1=0, align2=0:        32.09        24.75
> > >     length=2944, align1=14, align2=0:        40.27        26.78
> > >     length=2944, align1=0, align2=14:        45.59        39.55
> > >    length=2944, align1=14, align2=14:        40.94        28.00
> > >      length=3008, align1=0, align2=0:        42.17        25.97
> > >     length=3008, align1=15, align2=0:        41.81        27.07
> > >     length=3008, align1=0, align2=15:        42.67        39.35
> > >    length=3008, align1=15, align2=15:        42.18        28.00
> > >      length=3072, align1=0, align2=0:        34.71        25.16
> > >     length=3072, align1=16, align2=0:        43.31        26.37
> > >     length=3072, align1=0, align2=16:        43.41        40.98
> > >    length=3072, align1=16, align2=16:        36.62        27.99
> > >      length=3136, align1=0, align2=0:        45.75        26.78
> > >     length=3136, align1=17, align2=0:        43.29        27.59
> > >     length=3136, align1=0, align2=17:        44.44        41.16
> > >    length=3136, align1=17, align2=17:        34.99        28.59
> > >      length=3200, align1=0, align2=0:        43.12        25.16
> > >     length=3200, align1=18, align2=0:        43.04        27.59
> > >     length=3200, align1=0, align2=18:        45.08        42.59
> > >    length=3200, align1=18, align2=18:        36.00        28.81
> > >      length=3264, align1=0, align2=0:        43.01        27.48
> > >     length=3264, align1=19, align2=0:        47.72        28.61
> > >     length=3264, align1=0, align2=19:        47.00        42.60
> > >    length=3264, align1=19, align2=19:        44.44        28.81
> > >      length=3328, align1=0, align2=0:        35.50        25.75
> > >     length=3328, align1=20, align2=0:        44.84        27.19
> > >     length=3328, align1=0, align2=20:        46.38        44.41
> > >    length=3328, align1=20, align2=20:        33.68        28.81
> > >      length=3392, align1=0, align2=0:        43.82        27.18
> > >     length=3392, align1=21, align2=0:        45.91        28.81
> > >     length=3392, align1=0, align2=21:        47.61        43.06
> > >    length=3392, align1=21, align2=21:        43.82        29.21
> > >      length=3456, align1=0, align2=0:        43.83        26.64
> > >     length=3456, align1=22, align2=0:        44.77        29.49
> > >     length=3456, align1=0, align2=22:        47.57        46.01
> > >    length=3456, align1=22, align2=22:        44.11        29.62
> > >      length=3520, align1=0, align2=0:        45.04        27.99
> > >     length=3520, align1=23, align2=0:        46.59        30.55
> > >     length=3520, align1=0, align2=23:        50.58        44.80
> > >    length=3520, align1=23, align2=23:        43.85        29.21
> > >      length=3584, align1=0, align2=0:        45.04        26.88
> > >     length=3584, align1=24, align2=0:        47.22        29.21
> > >     length=3584, align1=0, align2=24:        50.12        47.47
> > >    length=3584, align1=24, align2=24:        45.04        29.80
> > >      length=3648, align1=0, align2=0:        47.13        28.40
> > >     length=3648, align1=25, align2=0:        48.78        30.94
> > >     length=3648, align1=0, align2=25:        49.25        61.42
> > >    length=3648, align1=25, align2=25:        45.85        30.02
> > >      length=3712, align1=0, align2=0:        47.24        27.19
> > >     length=3712, align1=26, align2=0:        48.41        30.79
> > >     length=3712, align1=0, align2=26:        49.65        51.57
> > >    length=3712, align1=26, align2=26:        48.76        30.33
> > >      length=3776, align1=0, align2=0:        47.35        34.61
> > >     length=3776, align1=27, align2=0:        49.16        31.99
> > >     length=3776, align1=0, align2=27:        52.06        49.38
> > >    length=3776, align1=27, align2=27:        47.36        30.43
> > >      length=3840, align1=0, align2=0:        49.70        28.35
> > >     length=3840, align1=28, align2=0:        50.29        31.81
> > >     length=3840, align1=0, align2=28:        51.15        50.71
> > >    length=3840, align1=28, align2=28:        47.07        31.65
> > >      length=3904, align1=0, align2=0:        48.28        29.21
> > >     length=3904, align1=29, align2=0:        51.96        32.99
> > >     length=3904, align1=0, align2=29:        53.24        50.71
> > >    length=3904, align1=29, align2=29:        47.88        31.64
> > >      length=3968, align1=0, align2=0:        47.89        29.19
> > >     length=3968, align1=30, align2=0:        51.52        32.57
> > >     length=3968, align1=0, align2=30:        54.03        52.33
> > >    length=3968, align1=30, align2=30:        46.62        33.15
> > >      length=4032, align1=0, align2=0:        49.77        29.62
> > >     length=4032, align1=31, align2=0:        53.62        34.27
> > >     length=4032, align1=0, align2=31:        56.44        52.33
> > >    length=4032, align1=31, align2=31:        48.94        33.16
> > >      length=4096, align1=0, align2=0:        49.37        29.21
> > >     length=4096, align1=32, align2=0:        50.46        33.00
> > >     length=4096, align1=0, align2=32:        49.31        55.14
> > >    length=4096, align1=32, align2=32:        49.28        33.05
> > >
> > > > if the align2 != 0, align1 = 0 case on Icelake warranted more investigation.
> > > > That being said the change is a clear winner on Icelake as well
> > > > for any other alignment schema.
> > >
> > > True, REP MOVSB is more efficient.
> > >
> >
> > This is the v2 patch I am checking in next week.
>
> Sorry! LGTM. Change is a big net gain :)
>
> >
> > Thanks.
> >
> > --
> > H.J.

I would like to backport this patch to release branches.
Any comments or objections?

--Sunil
diff mbox series

Patch

From e269371e0f8b8426c4880dd4991e527c03d3a735 Mon Sep 17 00:00:00 2001
From: "H.J. Lu" <hjl.tools@gmail.com>
Date: Fri, 30 Apr 2021 05:58:59 -0700
Subject: [PATCH v2] x86: Set rep_movsb_threshold to 2112 on processors with
 FSRM

The glibc memcpy benchmark on Intel Core i7-1065G7 (Ice Lake) showed
that REP MOVSB became faster after 2112 bytes:

                                      Vector Move       REP MOVSB
length=2112, align1=0, align2=0:        24.20             24.40
length=2112, align1=1, align2=0:        26.07             23.13
length=2112, align1=0, align2=1:        27.18             28.13
length=2112, align1=1, align2=1:        26.23             25.16
length=2176, align1=0, align2=0:        23.18             22.52
length=2176, align1=2, align2=0:        25.45             22.52
length=2176, align1=0, align2=2:        27.14             27.82
length=2176, align1=2, align2=2:        22.73             25.56
length=2240, align1=0, align2=0:        24.62             24.25
length=2240, align1=3, align2=0:        29.77             27.15
length=2240, align1=0, align2=3:        35.55             29.93
length=2240, align1=3, align2=3:        34.49             25.15
length=2304, align1=0, align2=0:        34.75             26.64
length=2304, align1=4, align2=0:        32.09             22.63
length=2304, align1=0, align2=4:        28.43             31.24

Use REP MOVSB for data size > 2112 bytes in memcpy on processors with
fast short REP MOVSB (FSRM).

	* sysdeps/x86/dl-cacheinfo.h (dl_init_cacheinfo): Set
	rep_movsb_threshold to 2112 on processors with fast short REP
	MOVSB (FSRM).
---
 sysdeps/x86/dl-cacheinfo.h | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/sysdeps/x86/dl-cacheinfo.h b/sysdeps/x86/dl-cacheinfo.h
index d9944250fc..e6c94dfd02 100644
--- a/sysdeps/x86/dl-cacheinfo.h
+++ b/sysdeps/x86/dl-cacheinfo.h
@@ -891,6 +891,10 @@  dl_init_cacheinfo (struct cpu_features *cpu_features)
       minimum_rep_movsb_threshold = 16 * 8;
 #endif
     }
+  /* NB: The default REP MOVSB threshold is 2112 on processors with fast
+     short REP MOVSB (FSRM).  */
+  if (CPU_FEATURE_USABLE_P (cpu_features, FSRM))
+    rep_movsb_threshold = 2112;
 
   unsigned long int rep_movsb_stop_threshold;
   /* ERMS feature is implemented from AMD Zen3 architecture and it is
-- 
2.31.1