[v5,00/16] Support RISC-V Control Flow Integrifty (CFI)

Message ID 20260628070241.88310-1-jesse.huang@sifive.com (mailing list archive)
Headers
Series Support RISC-V Control Flow Integrifty (CFI) |

Message

Jesse Huang June 28, 2026, 7:02 a.m. UTC
  This patch series adds support for the new RISC-V Control Flow Integrity (CFI)
extensions, i.e. Zicfilp and Zicfiss, as described in the following sections
of the RISC-V Instruction Set Manual:

    - Volume I, Chapter 33
    - Volume II, Chapter 22

Our implementation largely refers to the existing x86 CET code, and we would
like to thank the developers for their work.

Summary of Changes
------------------

1) New Option for the Build System

    A new '--enable-cfi' configure option is introduced to control whether
    CFI-related features are enabled. It appends the '-fcf-protection=full'
    compiler flag to all source files.

2) Adjustment to Assembly Code

    While compiler automatically do the job for C source files, assembly files
    and routines are requiring manual modifications
    - Insert GNU property notes and landing pad, label setting instructions
      into assembly files and routines
    - Replace indirect branches with software-guarded branches where applicable
    - Extend setjmp/longjmp to support for the shadow stack by storing the
      shadow stack pointer (SSP) to a new union __ssp_sigset_arch_t, that wraps
      the original sigset_t and a new __ssp_sigset_t that shrinks the mask size
      to make spaces for storing SSP and SSP base, and handling the
      save/restore/unwinding logic

3) Parse GNU Property Notes and Setup the Environment

    The loader scans the GNU_PROPERTY_RISCV_FEATURE_1_AND note and parses the
    bits specified by the binary to determine the required CFI features, it
    then performs checks on all dependencies, and uses new prctl operations
    to call to the kernel to do the setup work.

4) Add Tunables for Overriding Runtime Behavior

    Two new tunables are introduced:
      - glibc.cpu.riscv_cfi_lp
      - glibc.cpu.riscv_cfi_ss

    These control landing pad and shadow stack behavior at runtime,
    respectively. Each accepts 'on|permissive|off' for its value, which is
    same as x86.

5. Store shadow stack information in TLS to support ucontext

    We make the following structure changes to support the ucontext library
    - A new helper function `__allocate_shadow_stack` to map a new shadow stack
      Shadow stack size for the new context is based on an estimation from the
      runtime stack size, which is same to the implementation of x86.
    - Store ssp and shadow stack base marker in TLS
    - Change the type of uc_sigmask in ucontext_t to __ssp_sigset_arch_t, in that
      we can make use of the space for storing ssp and shadow stack base
      that is also used by jmp_buf

Changes from V1
--------------------
- The new option '--enable-cfi' gets a descrition in install.texi and
  an entry in NEWS
- Update prctl numbers of landing pad
- Re-implement ucontext using shadow stack restore token techniques
- Change makecontext to use the `map_shadow_stack` syscall

Changes from V2
--------------------
- Use indirect jump instead of return in __longjmp if CFI is enabled
- Add __INDIRECT_RETURN attribute to swapcontext, which instructs the
  compiler to insert a LPAD after the function callsite, so that we can
  use CFI-guarded indirect jump for context swtiching. Compiler would
  need to be updated to support this attribute for it to work
- Remove SSP from jmp_buf, stored it into a new union __ssp_sigset_arch_t
  that holds a original sigset_t and the shrinked sigset_t with ssp and
  ssp_base, so that we won't affect the size of jmp_buf or sigset_t
- SSP field in ucontext_t is also moved into the uc_sigmask that is of
  type __ssp_sigset_arch_t

Changes from V3
--------------------
- Fixed the offset used for accessing ssp in TLS that was incorrect
- Make setjmp/longjmp capable of handling inter-ucontext jumps
- Switch to the new prctl option numbers that was introduced in Linux
  7.0

Changes from V4
--------------------
- Fixed several typos and commit messages that does not reflect the
  newest version of the patch
- The patch for updating prctl option numbers is squashed
- ENTRY() and LEAF() macros now inserts LPAD so no longer need to
  manually modify the assembly functions defined with these macros and
  their derivatives

Jesse Huang (16):
  riscv: Add --enable-cfi option for controlling CFI features
  riscv/cfi: Set up necessary options for --enable-cfi
  riscv: Add GNU property definitions for RISC-V CFI
  riscv: Adjust assembly routines to support landing pad
  riscv: Introduce feature variables for RISC-V GNU properties
  riscv/cfi: Add prctl definitions for RISC-V CFI
  riscv/cfi: Enable CFI on static binaries
  riscv/cfi: Enable CFI on dynamic binaries
  riscv/cfi: Introduce tunables for CFI features
  riscv/cfi: Adjust setjmp/longjmp for shadow stack to work
  riscv/cfi: Support locking/disabling CFI and move OS dependent code
  riscv/cfi: Store shadow stack information in TLS
  riscv/cfi: Add internal sigset_t union and use it for both
    ucontext/jmpbuf
  riscv/cfi: Add __allocate_shadow_stack for mapping new shadow stack
  riscv/cfi: Support ucontext under CFI
  riscv/cfi: Add __INDIRECT_RETURN attribute to swapcontext

 INSTALL                                       |  13 +
 NEWS                                          |   3 +
 configure                                     |  12 +
 configure.ac                                  |   6 +
 elf/elf.h                                     |   5 +
 manual/install.texi                           |  12 +
 manual/tunables.texi                          |  22 ++
 sysdeps/riscv/Makefile                        |  17 +-
 sysdeps/riscv/__longjmp.S                     |  54 +++
 sysdeps/riscv/bits/indirect-return.h          |  36 ++
 sysdeps/riscv/cpu-features.c                  |  46 +++
 sysdeps/riscv/cpu-tunables.c                  |  50 +++
 sysdeps/riscv/crti.S                          |   4 +
 sysdeps/riscv/crtn.S                          |   4 +
 sysdeps/riscv/dl-cfi.c                        | 317 ++++++++++++++++++
 sysdeps/riscv/dl-get-cpu-features.c           |  27 ++
 sysdeps/riscv/dl-machine.h                    |  35 ++
 sysdeps/riscv/dl-procruntime.c                |  77 +++++
 sysdeps/riscv/dl-prop.h                       |  74 ++++
 sysdeps/riscv/dl-trampoline.S                 |  41 +--
 sysdeps/riscv/dl-tunables.list                |  27 ++
 sysdeps/riscv/feature-control.h               |  42 +++
 sysdeps/riscv/features-offsets.sym            |   5 +
 sysdeps/riscv/ldsodefs.h                      |   1 +
 sysdeps/riscv/libc-start.c                    |  31 ++
 sysdeps/riscv/libc-start.h                    |  95 ++++++
 sysdeps/riscv/link_map.h                      |  22 ++
 sysdeps/riscv/nptl/Makefile                   |   1 +
 sysdeps/riscv/nptl/tcb-offsets.sym            |   5 +
 sysdeps/riscv/nptl/tls.h                      |   2 +
 sysdeps/riscv/preconfigure                    |   2 +
 sysdeps/riscv/preconfigure.ac                 |   1 +
 sysdeps/riscv/setjmp.S                        |  22 ++
 sysdeps/riscv/start.S                         |   9 +
 sysdeps/riscv/sys/asm.h                       |  12 +-
 sysdeps/unix/sysv/linux/riscv/Makefile        |   1 +
 .../sysv/linux/riscv/allocate-shadow-stack.c  |  59 ++++
 .../sysv/linux/riscv/allocate-shadow-stack.h  |  31 ++
 sysdeps/unix/sysv/linux/riscv/bits/mman.h     |  30 ++
 .../sysv/linux/riscv/bits/types/__sigset_t.h  |  43 +++
 sysdeps/unix/sysv/linux/riscv/clone.S         |   1 +
 sysdeps/unix/sysv/linux/riscv/dl-cfi.h        | 116 +++++++
 sysdeps/unix/sysv/linux/riscv/getcontext.S    |  20 ++
 .../unix/sysv/linux/riscv/include/asm/prctl.h |  43 +++
 sysdeps/unix/sysv/linux/riscv/jmp_buf-ssp.sym |   7 +
 sysdeps/unix/sysv/linux/riscv/makecontext.c   |  19 ++
 sysdeps/unix/sysv/linux/riscv/setcontext.S    |  67 ++++
 sysdeps/unix/sysv/linux/riscv/setjmpP.h       |  49 +++
 sysdeps/unix/sysv/linux/riscv/swapcontext.S   |  81 ++++-
 sysdeps/unix/sysv/linux/riscv/sys/ucontext.h  |  14 +-
 sysdeps/unix/sysv/linux/riscv/sysdep.h        |  71 ++++
 sysdeps/unix/sysv/linux/riscv/ucontext_i.sym  |   4 +-
 52 files changed, 1763 insertions(+), 25 deletions(-)
 create mode 100644 sysdeps/riscv/bits/indirect-return.h
 create mode 100644 sysdeps/riscv/cpu-features.c
 create mode 100644 sysdeps/riscv/cpu-tunables.c
 create mode 100644 sysdeps/riscv/crti.S
 create mode 100644 sysdeps/riscv/crtn.S
 create mode 100644 sysdeps/riscv/dl-cfi.c
 create mode 100644 sysdeps/riscv/dl-get-cpu-features.c
 create mode 100644 sysdeps/riscv/dl-procruntime.c
 create mode 100644 sysdeps/riscv/dl-prop.h
 create mode 100644 sysdeps/riscv/dl-tunables.list
 create mode 100644 sysdeps/riscv/feature-control.h
 create mode 100644 sysdeps/riscv/features-offsets.sym
 create mode 100644 sysdeps/riscv/libc-start.c
 create mode 100644 sysdeps/riscv/libc-start.h
 create mode 100644 sysdeps/riscv/link_map.h
 create mode 100644 sysdeps/riscv/nptl/Makefile
 create mode 100644 sysdeps/riscv/nptl/tcb-offsets.sym
 create mode 100644 sysdeps/unix/sysv/linux/riscv/allocate-shadow-stack.c
 create mode 100644 sysdeps/unix/sysv/linux/riscv/allocate-shadow-stack.h
 create mode 100644 sysdeps/unix/sysv/linux/riscv/bits/mman.h
 create mode 100644 sysdeps/unix/sysv/linux/riscv/bits/types/__sigset_t.h
 create mode 100644 sysdeps/unix/sysv/linux/riscv/dl-cfi.h
 create mode 100644 sysdeps/unix/sysv/linux/riscv/include/asm/prctl.h
 create mode 100644 sysdeps/unix/sysv/linux/riscv/jmp_buf-ssp.sym
 create mode 100644 sysdeps/unix/sysv/linux/riscv/setjmpP.h
  

Comments

Andreas Schwab June 29, 2026, 8:22 a.m. UTC | #1
On Jun 28 2026, Jesse Huang wrote:

> This patch series adds support for the new RISC-V Control Flow Integrity (CFI)
> extensions, i.e. Zicfilp and Zicfiss, as described in the following sections
> of the RISC-V Instruction Set Manual:
>
>     - Volume I, Chapter 33
>     - Volume II, Chapter 22

Do I understand corectly that this is incompatible with implementations
lacking the Zimop/Zcmop extensions?
  
Florian Weimer June 29, 2026, 10:07 a.m. UTC | #2
* Andreas Schwab:

> On Jun 28 2026, Jesse Huang wrote:
>
>> This patch series adds support for the new RISC-V Control Flow Integrity (CFI)
>> extensions, i.e. Zicfilp and Zicfiss, as described in the following sections
>> of the RISC-V Instruction Set Manual:
>>
>>     - Volume I, Chapter 33
>>     - Volume II, Chapter 22
>
> Do I understand corectly that this is incompatible with implementations
> lacking the Zimop/Zcmop extensions?

Yes, I believe that's accurate.  It's not possible to maintain
compatibility with the baseline with this approach.

Thanks,
Florian
  
Jesse Huang June 29, 2026, 4:03 p.m. UTC | #3
On Mon, Jun 29, 2026 at 6:08 PM Florian Weimer <fweimer@redhat.com> wrote:
>
> * Andreas Schwab:
>
> > On Jun 28 2026, Jesse Huang wrote:
> >
> >> This patch series adds support for the new RISC-V Control Flow Integrity (CFI)
> >> extensions, i.e. Zicfilp and Zicfiss, as described in the following sections
> >> of the RISC-V Instruction Set Manual:
> >>
> >>     - Volume I, Chapter 33
> >>     - Volume II, Chapter 22
> >
> > Do I understand corectly that this is incompatible with implementations
> > lacking the Zimop/Zcmop extensions?
>
> Yes, I believe that's accurate.  It's not possible to maintain
> compatibility with the baseline with this approach.
>
> Thanks,
> Florian
>

Regarding the extensions, Zicfilp uses the existing auipc space to
encode lpad, so it actually
compatible with cores that lacking support of Zimop and Zcmop.
However, from a security standpoint I don't think it makes sense to
enable only one of the two,
so there is only a single --enable-cfi option that turns on both.

Thanks,
Jesse
  
Andreas K. Huettel June 30, 2026, 1:15 p.m. UTC | #4
Am Sonntag, 28. Juni 2026, 16:02:25 Japanische Normalzeit schrieb Jesse Huang:
> This patch series adds support for the new RISC-V Control Flow Integrity (CFI)
> extensions, i.e. Zicfilp and Zicfiss, as described in the following sections
> of the RISC-V Instruction Set Manual:
> 
>     - Volume I, Chapter 33
>     - Volume II, Chapter 22
> 

Let's figure this out after the release please.

>
> Our implementation largely refers to the existing x86 CET code, and we would
> like to thank the developers for their work.
> 
> Summary of Changes
> ------------------
> 
> 1) New Option for the Build System
> 
>     A new '--enable-cfi' configure option is introduced to control whether
>     CFI-related features are enabled. It appends the '-fcf-protection=full'
>     compiler flag to all source files.
> 
> 2) Adjustment to Assembly Code
> 
>     While compiler automatically do the job for C source files, assembly files
>     and routines are requiring manual modifications
>     - Insert GNU property notes and landing pad, label setting instructions
>       into assembly files and routines
>     - Replace indirect branches with software-guarded branches where applicable
>     - Extend setjmp/longjmp to support for the shadow stack by storing the
>       shadow stack pointer (SSP) to a new union __ssp_sigset_arch_t, that wraps
>       the original sigset_t and a new __ssp_sigset_t that shrinks the mask size
>       to make spaces for storing SSP and SSP base, and handling the
>       save/restore/unwinding logic
> 
> 3) Parse GNU Property Notes and Setup the Environment
> 
>     The loader scans the GNU_PROPERTY_RISCV_FEATURE_1_AND note and parses the
>     bits specified by the binary to determine the required CFI features, it
>     then performs checks on all dependencies, and uses new prctl operations
>     to call to the kernel to do the setup work.
> 
> 4) Add Tunables for Overriding Runtime Behavior
> 
>     Two new tunables are introduced:
>       - glibc.cpu.riscv_cfi_lp
>       - glibc.cpu.riscv_cfi_ss
> 
>     These control landing pad and shadow stack behavior at runtime,
>     respectively. Each accepts 'on|permissive|off' for its value, which is
>     same as x86.
> 
> 5. Store shadow stack information in TLS to support ucontext
> 
>     We make the following structure changes to support the ucontext library
>     - A new helper function `__allocate_shadow_stack` to map a new shadow stack
>       Shadow stack size for the new context is based on an estimation from the
>       runtime stack size, which is same to the implementation of x86.
>     - Store ssp and shadow stack base marker in TLS
>     - Change the type of uc_sigmask in ucontext_t to __ssp_sigset_arch_t, in that
>       we can make use of the space for storing ssp and shadow stack base
>       that is also used by jmp_buf
> 
> Changes from V1
> --------------------
> - The new option '--enable-cfi' gets a descrition in install.texi and
>   an entry in NEWS
> - Update prctl numbers of landing pad
> - Re-implement ucontext using shadow stack restore token techniques
> - Change makecontext to use the `map_shadow_stack` syscall
> 
> Changes from V2
> --------------------
> - Use indirect jump instead of return in __longjmp if CFI is enabled
> - Add __INDIRECT_RETURN attribute to swapcontext, which instructs the
>   compiler to insert a LPAD after the function callsite, so that we can
>   use CFI-guarded indirect jump for context swtiching. Compiler would
>   need to be updated to support this attribute for it to work
> - Remove SSP from jmp_buf, stored it into a new union __ssp_sigset_arch_t
>   that holds a original sigset_t and the shrinked sigset_t with ssp and
>   ssp_base, so that we won't affect the size of jmp_buf or sigset_t
> - SSP field in ucontext_t is also moved into the uc_sigmask that is of
>   type __ssp_sigset_arch_t
> 
> Changes from V3
> --------------------
> - Fixed the offset used for accessing ssp in TLS that was incorrect
> - Make setjmp/longjmp capable of handling inter-ucontext jumps
> - Switch to the new prctl option numbers that was introduced in Linux
>   7.0
> 
> Changes from V4
> --------------------
> - Fixed several typos and commit messages that does not reflect the
>   newest version of the patch
> - The patch for updating prctl option numbers is squashed
> - ENTRY() and LEAF() macros now inserts LPAD so no longer need to
>   manually modify the assembly functions defined with these macros and
>   their derivatives
> 
> Jesse Huang (16):
>   riscv: Add --enable-cfi option for controlling CFI features
>   riscv/cfi: Set up necessary options for --enable-cfi
>   riscv: Add GNU property definitions for RISC-V CFI
>   riscv: Adjust assembly routines to support landing pad
>   riscv: Introduce feature variables for RISC-V GNU properties
>   riscv/cfi: Add prctl definitions for RISC-V CFI
>   riscv/cfi: Enable CFI on static binaries
>   riscv/cfi: Enable CFI on dynamic binaries
>   riscv/cfi: Introduce tunables for CFI features
>   riscv/cfi: Adjust setjmp/longjmp for shadow stack to work
>   riscv/cfi: Support locking/disabling CFI and move OS dependent code
>   riscv/cfi: Store shadow stack information in TLS
>   riscv/cfi: Add internal sigset_t union and use it for both
>     ucontext/jmpbuf
>   riscv/cfi: Add __allocate_shadow_stack for mapping new shadow stack
>   riscv/cfi: Support ucontext under CFI
>   riscv/cfi: Add __INDIRECT_RETURN attribute to swapcontext
> 
>  INSTALL                                       |  13 +
>  NEWS                                          |   3 +
>  configure                                     |  12 +
>  configure.ac                                  |   6 +
>  elf/elf.h                                     |   5 +
>  manual/install.texi                           |  12 +
>  manual/tunables.texi                          |  22 ++
>  sysdeps/riscv/Makefile                        |  17 +-
>  sysdeps/riscv/__longjmp.S                     |  54 +++
>  sysdeps/riscv/bits/indirect-return.h          |  36 ++
>  sysdeps/riscv/cpu-features.c                  |  46 +++
>  sysdeps/riscv/cpu-tunables.c                  |  50 +++
>  sysdeps/riscv/crti.S                          |   4 +
>  sysdeps/riscv/crtn.S                          |   4 +
>  sysdeps/riscv/dl-cfi.c                        | 317 ++++++++++++++++++
>  sysdeps/riscv/dl-get-cpu-features.c           |  27 ++
>  sysdeps/riscv/dl-machine.h                    |  35 ++
>  sysdeps/riscv/dl-procruntime.c                |  77 +++++
>  sysdeps/riscv/dl-prop.h                       |  74 ++++
>  sysdeps/riscv/dl-trampoline.S                 |  41 +--
>  sysdeps/riscv/dl-tunables.list                |  27 ++
>  sysdeps/riscv/feature-control.h               |  42 +++
>  sysdeps/riscv/features-offsets.sym            |   5 +
>  sysdeps/riscv/ldsodefs.h                      |   1 +
>  sysdeps/riscv/libc-start.c                    |  31 ++
>  sysdeps/riscv/libc-start.h                    |  95 ++++++
>  sysdeps/riscv/link_map.h                      |  22 ++
>  sysdeps/riscv/nptl/Makefile                   |   1 +
>  sysdeps/riscv/nptl/tcb-offsets.sym            |   5 +
>  sysdeps/riscv/nptl/tls.h                      |   2 +
>  sysdeps/riscv/preconfigure                    |   2 +
>  sysdeps/riscv/preconfigure.ac                 |   1 +
>  sysdeps/riscv/setjmp.S                        |  22 ++
>  sysdeps/riscv/start.S                         |   9 +
>  sysdeps/riscv/sys/asm.h                       |  12 +-
>  sysdeps/unix/sysv/linux/riscv/Makefile        |   1 +
>  .../sysv/linux/riscv/allocate-shadow-stack.c  |  59 ++++
>  .../sysv/linux/riscv/allocate-shadow-stack.h  |  31 ++
>  sysdeps/unix/sysv/linux/riscv/bits/mman.h     |  30 ++
>  .../sysv/linux/riscv/bits/types/__sigset_t.h  |  43 +++
>  sysdeps/unix/sysv/linux/riscv/clone.S         |   1 +
>  sysdeps/unix/sysv/linux/riscv/dl-cfi.h        | 116 +++++++
>  sysdeps/unix/sysv/linux/riscv/getcontext.S    |  20 ++
>  .../unix/sysv/linux/riscv/include/asm/prctl.h |  43 +++
>  sysdeps/unix/sysv/linux/riscv/jmp_buf-ssp.sym |   7 +
>  sysdeps/unix/sysv/linux/riscv/makecontext.c   |  19 ++
>  sysdeps/unix/sysv/linux/riscv/setcontext.S    |  67 ++++
>  sysdeps/unix/sysv/linux/riscv/setjmpP.h       |  49 +++
>  sysdeps/unix/sysv/linux/riscv/swapcontext.S   |  81 ++++-
>  sysdeps/unix/sysv/linux/riscv/sys/ucontext.h  |  14 +-
>  sysdeps/unix/sysv/linux/riscv/sysdep.h        |  71 ++++
>  sysdeps/unix/sysv/linux/riscv/ucontext_i.sym  |   4 +-
>  52 files changed, 1763 insertions(+), 25 deletions(-)
>  create mode 100644 sysdeps/riscv/bits/indirect-return.h
>  create mode 100644 sysdeps/riscv/cpu-features.c
>  create mode 100644 sysdeps/riscv/cpu-tunables.c
>  create mode 100644 sysdeps/riscv/crti.S
>  create mode 100644 sysdeps/riscv/crtn.S
>  create mode 100644 sysdeps/riscv/dl-cfi.c
>  create mode 100644 sysdeps/riscv/dl-get-cpu-features.c
>  create mode 100644 sysdeps/riscv/dl-procruntime.c
>  create mode 100644 sysdeps/riscv/dl-prop.h
>  create mode 100644 sysdeps/riscv/dl-tunables.list
>  create mode 100644 sysdeps/riscv/feature-control.h
>  create mode 100644 sysdeps/riscv/features-offsets.sym
>  create mode 100644 sysdeps/riscv/libc-start.c
>  create mode 100644 sysdeps/riscv/libc-start.h
>  create mode 100644 sysdeps/riscv/link_map.h
>  create mode 100644 sysdeps/riscv/nptl/Makefile
>  create mode 100644 sysdeps/riscv/nptl/tcb-offsets.sym
>  create mode 100644 sysdeps/unix/sysv/linux/riscv/allocate-shadow-stack.c
>  create mode 100644 sysdeps/unix/sysv/linux/riscv/allocate-shadow-stack.h
>  create mode 100644 sysdeps/unix/sysv/linux/riscv/bits/mman.h
>  create mode 100644 sysdeps/unix/sysv/linux/riscv/bits/types/__sigset_t.h
>  create mode 100644 sysdeps/unix/sysv/linux/riscv/dl-cfi.h
>  create mode 100644 sysdeps/unix/sysv/linux/riscv/include/asm/prctl.h
>  create mode 100644 sysdeps/unix/sysv/linux/riscv/jmp_buf-ssp.sym
>  create mode 100644 sysdeps/unix/sysv/linux/riscv/setjmpP.h
> 
>
  
Jeffrey Law June 30, 2026, 5:36 p.m. UTC | #5
On 6/30/2026 7:15 AM, Andreas K. Huettel wrote:
> Am Sonntag, 28. Juni 2026, 16:02:25 Japanische Normalzeit schrieb Jesse Huang:
>> This patch series adds support for the new RISC-V Control Flow Integrity (CFI)
>> extensions, i.e. Zicfilp and Zicfiss, as described in the following sections
>> of the RISC-V Instruction Set Manual:
>>
>>      - Volume I, Chapter 33
>>      - Volume II, Chapter 22
>>
> Let's figure this out after the release please.
We kicked this around a bit at the gcc patchwork meeting.  WIthin that 
group (included Peter B, Robin and myself) we agreed this can be 
reasonably deferred.  Seems like it's just landing fairly late in the 
cycle with some concerns...   So, yea, let's figure this out after the 
release.

Jeff