[0/7] introduce get_runtime_path

  Hi. I tried to run a testsuite using a remote host test setup on QEMU
(RISC-V) with Linux.
Some tests that passed on local machine are failed due to hardcoded pathes in tests.
For example, see gdb.base/print-file-var.exp. This test has SHLIB_NAME
that is an absolute path on local machine, and of course this file does not
exist on remote machine. To fix this, a function get_runtime_path was created.

These patches introduce this function and some tests that could be fixed
by changing an absolute path on local machine to a runtime path.

Ivan Tetyushkin (7):
  [gdb/testsuite] Adding function to find runtime path to support remote
    execution for testsuite
  [gdb/testsuite] fix test gdb.base/print-file-var.exp for remote
    execution
  [gdb/testsuite] fix test gdb.base/jit-reader-exec.exp for remote
    execution
  [gdb/testsuite] fix test gdb.base/solib-vanish.exp for remote
    execution
  [gdb/testsuite] fix test gdb.base/infcall-exec.exp for remote
    execution
  [gdb/testsuite] fix test gdb.base/info-shared.exp for remote execution
  [gdb/testsuite] fix test gdb.base/jit-elf-so.exp for remote execution

 gdb/testsuite/gdb.base/infcall-exec.exp    |  7 +++++--
 gdb/testsuite/gdb.base/info-shared.exp     |  9 +++++++--
 gdb/testsuite/gdb.base/jit-elf-so.exp      |  6 ++++--
 gdb/testsuite/gdb.base/jit-reader-exec.exp |  5 ++++-
 gdb/testsuite/gdb.base/print-file-var.exp  |  4 +++-
 gdb/testsuite/gdb.base/solib-vanish.exp    |  7 ++++++-
 gdb/testsuite/lib/gdb.exp                  | 12 ++++++++++++
 7 files changed, 41 insertions(+), 9 deletions(-)
  

Ivan Tetyushkin <ivan.tetyushkin@syntacore.com> writes:

> Hi. I tried to run a testsuite using a remote host test setup on QEMU
> (RISC-V) with Linux.

Thanks for looking at these problems.  I suspect the setup you are using
is not one that is used very often.

Could you give more details about your setup so that others might be
able to reproduce the issues you are seeing, and validate your fixes?

Thanks,
Andrew


> Some tests that passed on local machine are failed due to hardcoded pathes in tests.
> For example, see gdb.base/print-file-var.exp. This test has SHLIB_NAME
> that is an absolute path on local machine, and of course this file does not
> exist on remote machine. To fix this, a function get_runtime_path was created.
>
> These patches introduce this function and some tests that could be fixed
> by changing an absolute path on local machine to a runtime path.
>
> Ivan Tetyushkin (7):
>   [gdb/testsuite] Adding function to find runtime path to support remote
>     execution for testsuite
>   [gdb/testsuite] fix test gdb.base/print-file-var.exp for remote
>     execution
>   [gdb/testsuite] fix test gdb.base/jit-reader-exec.exp for remote
>     execution
>   [gdb/testsuite] fix test gdb.base/solib-vanish.exp for remote
>     execution
>   [gdb/testsuite] fix test gdb.base/infcall-exec.exp for remote
>     execution
>   [gdb/testsuite] fix test gdb.base/info-shared.exp for remote execution
>   [gdb/testsuite] fix test gdb.base/jit-elf-so.exp for remote execution
>
>  gdb/testsuite/gdb.base/infcall-exec.exp    |  7 +++++--
>  gdb/testsuite/gdb.base/info-shared.exp     |  9 +++++++--
>  gdb/testsuite/gdb.base/jit-elf-so.exp      |  6 ++++--
>  gdb/testsuite/gdb.base/jit-reader-exec.exp |  5 ++++-
>  gdb/testsuite/gdb.base/print-file-var.exp  |  4 +++-
>  gdb/testsuite/gdb.base/solib-vanish.exp    |  7 ++++++-
>  gdb/testsuite/lib/gdb.exp                  | 12 ++++++++++++
>  7 files changed, 41 insertions(+), 9 deletions(-)
>
> -- 
> 2.38.1
  

Hi

> Could you give more details about your setup so that others might be
> able to reproduce the issues you are seeing, and validate your fixes?
To reproduce my setup please:

0) Setup environment
0.1) Determine QEMU-related variables:
# QEMU_SSH_PORT - port is used for no-password access with ssh.
# QEMU_WORK_PORT - port used by gdb to send commands to gdbserver.

export QEMU_WORK_PORT=1111
export QEMU_SSH_PORT=2222

# HOSTNAME - name, used for connection with ssh.
export HOSTNAME=debian

# QEMU_ROOT_USER, QEMU_ROOT_PASS, QEMU_USER, QEMU_PASS - name and password for use in QEMU
# In suggested to download image:
export QEMU_ROOT_USER=root
export QEMU_ROOT_PASS=root
export QEMU_USER=debian
export QEMU_PASS=debian

0.2) Determine paths:
# for easier setup, you may want to use:
# BASEDIR - default parent directory for all paths here
export BASEDIR=~/BASEDIR

# QEMU_DIR - full path to folder where you want to have QEMU-related files.
export QEMU_DIR=$BASEDIR/qemu_dir
# BUILDDIR - full path to place where you want configure & build to be done
export BUILDDIR=$BASEDIR/gdb-build
# INSTALL_DIR - full path to install gdb
export INSTALL_DIR=$BASEDIR/gdb-install
# gdb_source - full path to binutils-gdb repo with configure
export gdb_source=$BASEDIR/binutils-gdb

# Optional
# dejagnu_dir - full path where you want to have dejagnu, if you do not have it already
export dejagnu_dir=$BASEDIR

0.3) Make sure that all directories exist.
mkdir --parents $BASEDIR
mkdir --parents $QEMU_DIR
mkdir --parents $BUILDDIR

0.4) Make sure you have required apps in your path.
apt update
apt-get install -y gcc-riscv64-linux-gnu
apt-get install -y g++-riscv64-linux-gnu
apt-get install -y qemu-system-riscv64
apt-get install -y u-boot-qemu opensbi


1) Prepare QEMU with Linux.
1.1) Download Debian-based image for RISCV QEMU:
url: https://gitlab.com/api/v4/projects/giomasce%2Fdqib/jobs/artifacts/master/download?job=convert_riscv64-virt

wget "https://gitlab.com/api/v4/projects/giomasce%2Fdqib/jobs/artifacts/master/download?job=convert_riscv64-virt" -O "$QEMU_DIR/debian-rv64.zip"

1.2) Unpack & prepare QEMU files for work:

unzip "$QEMU_DIR/debian-rv64.zip" -d "$QEMU_DIR/debian_tmp"
mv "$QEMU_DIR/debian_tmp/artifacts" "$QEMU_DIR/debian-rv64"

1.3) Create image overlay file (overlay.qcow2) for QEMU:

qemu-img create -o backing_file="$QEMU_DIR/debian-rv64/image.qcow2",backing_fmt=qcow2 -f qcow2 "$QEMU_DIR/overlay.qcow2"

1.4) Run QEMU:

qemu-system-riscv64 \
-machine virt \
-smp 8 \
-cpu rv64 \
-m 8G \
-device virtio-blk-device,drive=hd \
-drive file=$QEMU_DIR/overlay.qcow2,if=none,id=hd \
-device virtio-net-device,netdev=net \
-netdev user,id=net,hostfwd=tcp::$QEMU_SSH_PORT-:22,hostfwd=tcp::$QEMU_WORK_PORT-:1111 \
-bios /usr/lib/riscv64-linux-gnu/opensbi/generic/fw_jump.elf \
-kernel /usr/lib/u-boot/qemu-riscv64_smode/uboot.elf \
-object rng-random,filename=/dev/urandom,id=rng \
-device virtio-rng-device,rng=rng \
-daemonize

1.5) Add ssh config to be able to connect without password:

To your .ssh/config write (if you changed earlier environment variables, please, change there too):
Host debian
  HostName 127.0.0.1
  User debian
  Port 2222

check that ssh connection works:
# adding to the list of known hosts (may need to be done twice due to reset of connection).
ssh $HOSTNAME

If you do not have ssh keys, you need to generate them:
ssh-keygen -t rsa -N '' -f ~/.ssh/id_rsa <<< y;

Copy you ssh public keys to QEMU:
sshpass -p $QEMU_PASS ssh $HOSTNAME "mkdir --parents /home/$QEMU_USER/.ssh"
sshpass -p $QEMU_PASS scp ~/.ssh/id_rsa.pub $HOSTNAME:/home/$QEMU_USER/.ssh/authorized_keys

Now you should be able to connect to QEMU without password:
ssh $HOSTNAME

1.6) Install gdbserver on QEMU:

# adding to the list of known hosts. No need to put password
ssh $QEMU_ROOT_USER@localhost -p $QEMU_SSH_PORT

sshpass -p $QEMU_ROOT_PASS ssh $QEMU_ROOT_USER@localhost -p $QEMU_SSH_PORT
# in QEMU
apt update
apt install gdbserver
exit

2) Prepare binutils-gdb

2.0) If you want clean repo and used $BASEDIR structure, you may want to clone it from the sourceware repo to $BASEDIR.
cd $BASEDIR; git clone https://sourceware.org/git/binutils-gdb.git

2.1) Create board settings file. Setup usage riscv64-linux-gnu-gcc and remote execution.

Create new file NEW_BOARD.exp in $gdb_source/gdb/testsuite/boards (if you changed HOSTNAME or QEMU_PORT, change here as well):

load_generic_config "gdbserver"
load_board_description "gdbserver-base"

process_multilib_options ""

set_board_info compiler riscv64-linux-gnu-gcc
set_board_info c++compiler riscv64-linux-gnu-g++

set GDBSERVER "/usr/bin/gdbserver"
set_board_info gdb_server_prog "/usr/bin/gdbserver"

set_board_info hostname debian

set_board_info gdb,socketport 1111
set_board_info portnum 1111


set_board_info rsh_prog /usr/bin/ssh
set_board_info rcp_prog /usr/bin/scp
set_board_info file_transfer ftp
set_board_info protocol standard

set_board_info gdb_protocol "remote"

set_board_info use_gdb_stub 1
set_board_info gdb,do_reload_on_run 1
set_board_info noargs 1
set_board_info gdb,noinferiorio 1
set_board_info gdb,no_hardware_watchpoints 1

2.2) call binutils-gdb configure

cd $BUILDDIR; $gdb_source/configure \
--prefix=$INSTALL_DIR \
--with-expat=yes \
--target=riscv64-linux-gnu

2.3) build gdb
cd $BUILDDIR; make all-gdb


3) Download and run tests to reproduce the problem.
3.1) If you do not have dejagnu in the path, you need to download it and add to the path:
cd $dejagnu_dir
git clone  https://git.savannah.gnu.org/git/dejagnu.git
export PATH=$PATH:$(realpath ./)/dejagnu

3.2) Run test and observe failure without my patches
cd $BUILDDIR/gdb
make check RUNTESTFLAGS="--target_board=NEW_BOARD gdb.base/print-file-var.exp"

--
Ivan
  

On 10/29/22 11:20, Andrew Burgess via Gdb-patches wrote:
> Ivan Tetyushkin <ivan.tetyushkin@syntacore.com> writes:
> 
>> Hi. I tried to run a testsuite using a remote host test setup on QEMU
>> (RISC-V) with Linux.
> 
> Thanks for looking at these problems.  I suspect the setup you are using
> is not one that is used very often.
> 
> Could you give more details about your setup so that others might be
> able to reproduce the issues you are seeing, and validate your fixes?
> 

Hi,

I found a fairly easy way to reproduce this (after reading the response 
to Andrews question and realizing that given the use of --target_board 
rather than --host_board, from the point of view of dejagnu, this is not 
a remote host, but a remote target setup).

Add a local test account remote-target, and put a remote-target entry in 
the .ssh/config:
...
Host remote-target
      HostName  127.0.0.1
      User remote-target
      IdentityFile <some rsa file>
      IdentitiesOnly yes
      PreferredAuthentications publickey
...
such that we can access it without password:
...
$ ssh remote-target
Last login: Sat Nov  5 15:12:34 2022 from 127.0.0.1
Have a lot of fun...
remote-target@127.0.0.1:~>
...
and likewise for remote-target@remote-target.

Edit remote-gdbserver-on-localhost.exp to use the remote-target account:
...
  set_board_info rcp_prog "/usr/bin/scp"
  set_board_info rsh_prog "/usr/bin/ssh"
  set_board_info protocol standard
-set_board_info username $env(USER)
-set_board_info hostname localhost
+set_board_info username remote-target
+set_board_info hostname remote-target

  proc ${board}_spawn { board cmd } {
      global board_info
...

Run test-case gdb.base/print-file-var.exp with target board 
remote-gdbserver-on-localhost.exp, and watch it pass.

Now make sure the remote-target account cannot access files from the 
$USER account (well, as far as the testsuite output dir is concerned):
...
$ chmod go-rx build/gdb/testsuite/outputs/
...
mimicking an actual remote target machine not being able to access the 
files on the build/host machine.

Run the test again, and watch it fail.

Now apply the patch series, and watch it pass again.

Thanks,
- Tom