I never had a Hetzner server but I guess the rescue mode “logic” is to reboot the server and boot from a minimalist operating system to access to your server and troubleshoot it, as explained here :
The Hetzner Rescue System is a Linux live environment that allows you to have administrative access to your server. The environment starts from the network (PXE boot) and runs in the memory of the server. This makes it possible to carry out repairs to the installed system, to check file systems or to install a new operating system.
So executing QuickBox installation commands from the rescue mode won’t work.
That’s why I advised you to find the issue with rtorrent instead of reinstalling it (much less pain).
If you absolutely want to reinstall it, you will need to copy your data somewhere, reinstall your server and copy back you data.