An update on the state of upgrading

overhaul
eco-system
update

#1

Well, I have promised to deliver an easy (or even a method at all) method of upgrading QuickBox. This has found itself swimming in a sea of numerous delays… even my very typing of this announcement produces yet more delay… but, it’s the community that divulges my ethic and transparency, hence this update.

There was going to be a sort of history chart put together wherein several commits and their histories would be compiled in accordance to the version a user was on, this did turn into an epic task that after 4 days of supporting QuickBox here in the forums, maintaining and doing maintenance to the forums (we have a lot of lurkers round here :wink:), building in new features to QuickBox, general stability testing, RL job life and home life… I simply gave up on it. No, not on QuickBox! The update procedure I was going to use.

I found it to be incredibly bulky and just generally sloppy. Haphazardly following the ghost of commits past and marking them for archive. It looked like a half baked product and the last thing anyone wants is their product to be half-baked… not saying anything about the people themselves.

With that being said, all the work I was putting into the process pulled a tear from eye and kicked me in the feels a bit, but me wanting QuickBox to reign supreme at what it does, felt that dusting the shoulder off and starting once more from scratch on a more desirable and easier method.

I am happy to announce, that there is in fact, going to be a truly easy method of upgrading. This is going to take me a bit to really nail down, but it makes so much sense and it also adds a much easier availability (as per the code) for those that are wanting to contribute.

QuickBox Repo Structure

Granted, anyone with an established install may not like this news, but here it goes.

The Repo structure is going to change entirely. Obviously you will still find the Swizards/QuickBox Repo, but this will be used to address the functionality and the installation procedures of the script itself.

QuickBox will now be divided into it’s relevant parts that will add in making the update process a breeze… and I mean really simple. It will essentially clone certain parts of QuickBox into basic pieces on your server for ease of use.

Here are the Repos:

  • quickbox_setup - this is the repo used for the initial setup of QuickBox on the users server. This can be installed by doing the following:
git clone https://github.com/QuickBox/quickbox_setup quickbox_setup
cd quickbox_setup
bash quickbox-setup
  • quickbox_rutorrent - this is the repo used for ruTorrent. All ruTorrent relevant plugins and theme adjustments, fixes, additions and extra enhancements will be included here.

Much like the quickbox_setup repo, the script will be pulled via the following:

git clone https://github.com/QuickBox/quickbox_rutorrent quickbox_rutorrent

This is actually handled during the initial install with the quickbox_setup repo. The quickbox_rutorrent directory is then copied over to your /srv/ directory where it maintains it’s git-like qualities for easy updating later on.

  • quickbox_dashboard - this is the repo used for the QuickBox UI. All QuickBox Dashboard relevant widgets and theme adjustments, fixes, additions, extra enhancements and future language files will be included here.

Much like the other repo, the script will be pulled via the following:

git clone https://github.com/QuickBox/quickbox_dashboard quickbox_dashboard
cd  quickbox_dashboard
mkdir -p /srv/rutorrent/home
cp -r home/. /srv/rutorrent/home

Again, this is just an excerpt and this function is handled by the script. Just as with any of the other features, navigate to /srv/rutorrent/home and run git pull to push updates to your server.

  • quickbox_packages - this is the repo used for the installers and uninstallers. All needed files/commands for packages installers/removers as well as plugin installers/removers will be included here.
    Much like the other repo, the script will be pulled via the following:
git clone https://github.com/QuickBox/quickbox_packages quickbox_packages
cd  quickbox_packages
cp -r quickbox_packages/. /usr/local/bin/

Again, this is just an excerpt and this function is handled by the script. Just as with any of the other features, navigate to /usr/local/bin/quickbox and run git pull to push updates to your server.

  • club-Swizards - this is the custom ruTorrent theme created and designed by Swizards. Any adjustments you would like to push can be made here. Updating the template… again, as simple as git pull from within /srv/rutorrent/plugins/theme/themes/club-Swizards. The theme is independent of the update for the rutorrent directory which handles plugins etc.

But how does this make updating easy?

I don’t want to have to request that anyone ever reinstall or reboot to solve a problem that, simply put, is not proper administration or good advice as in most instances… it rarely solves a problem. However, in this case it is such a necessity to reinstall and there really is no way around it (unless you’re server savvy then you can just pull the needed repos and move/remove files as needed :wink: ). I do however promise that this revision does usher in complete simplicity for the future. All updates will be simply handled via a git pull management system.

Here’s an example:

Let’s say that @liara thinks the padding is off in a popup modal for the autodl plugin. Now, she can submit a pull request (or make the addition herself) and push it to the quickbox_rutorrent repo. Anyone curious about any changes can view the complete changelog via the commit history of the repo. Then, seeing these changes as something they want, they do the following:

cd /srv/rutorrent
git pull

It will now automagically update via the power of git. The output will show the following (or similar)

[[email protected]]:(776.9kb)~# cd /srv/rutorrent
[[email protected]]:(15.2kb)~/srv/rutorrent# git pull
remote: Counting objects: 3, done.
remote: Compressing objects: 100% (1/1), done.
remote: Total 3 (delta 2), reused 3 (delta 2), pack-reused 0
Unpacking objects: 100% (3/3), done.
From https://github.com/QuickBox/quickbox_rutorrent
   81021ae..38bbd0e  master     -> origin/master
Updating 81021ae..38bbd0e
Fast-forward
 index.html | 72 ++++++++++++++++++++++++++++++++++++------------------------------------
 1 file changed, 36 insertions(+), 36 deletions(-)
[[email protected]]:(15.2kb)~/srv/rutorrent#

Just like that, you have updated to match all changes made on the master branch of a given repo. You can also check and update these as all repos are first pulled to your root directory and then moved from there.

This same approach will be used for the dashboard as well as the package installer/uninstallers currently located at /usr/local/bin. This will ensure if any new features/packages are added in the future, or bugs are found in the present, they can be squashed, announced and a simple git pull in the proper directory will update with little to no hassle.

Moreover, one the biggest concerns has been doing updates without overwriting any changes users may have made to their dashboards. With this git pull method, we can rest assured knowing that the only files that receive changes are those that are included in the repo itself. Any files containing the network interface adjustments which are done on script install will be left alone and only changed, added or removed files will be updated.

I’ll share more as this continues to move forward. I am pushing to get this released within the next 3 days, therefore, before this weekend everyone has time (if willing and able to do a reinstall) to seed their current files to avoid H&R.

Please, feel free to share your thoughts. I know many of you will not like the approach of having to reinstall from scratch, however, this is a major version release that ensures the longevity and future capabilities and growth of the QuickBox Ecosystem.

Greetz!


#2

I think this sounds more doable than having to write an update script with each major point revision – people can keep their systems up to date rolling release style. For those with a bit of savvy it should technically be possible to update your current installs to the new system without a reinstallation.

I am just starting to get a systemd fork to stability for swizards use so I am not certain if I will adopt the new structure – we can’t really provide public access to the private repo as it is, so the installer is most important for my uses. That said, this should make it a bit easier for pulling features and bug fixes between the two. I’m also guessing you’d like to be able to cherry pick some commits as well, which is something I would like to pass back to you given all your work here.


#3

I literally got the ball rolling on your comment about Docker installations. I have been working on setting up docker structures for an install… this new method just seems to make absolute sense, like you said “rolling release style”. I have been building this over an old 2.1.0 install so it is 100% doable without reinstall… I can only hear the requests for how-to’s now :stuck_out_tongue_closed_eyes:

That systemd is something I have kept as an aside as I am certain @kc1 wants that magic kept on Swizards entirely… so absolutely that stays private. Not to mention, you have put in an incredible amount of work on that setup so it is entirely a paid-for solution. Unless it is expressed that the public QuickBox ecosystem can hop on the dispatch array train.

But yes, this way it will be much easier to :cherries: pick between any and all repos we so chose.

Also, adding you to the QuickBox repos… just for fun.


#4

I agree as well. I just wish some of these hosts will give me iLOM/iDRAC so I/we can reinstall ourselves.

I do not mind the reinstall, I just don’t want to be “that customer” that asks for a reinstall every 5 days, but, the way this project is heading it seems you’ve perfectly resolved the issues as far as updating goes. I have absolutely no issues with asking one last time for a reinstall.

Thank you for all the hard work you do!

~v/r
denellum


#5

@denellum i did find that SYS is great for self reinstallations, there is also a really cheap vps bhost.net which is great for testing idk that i would use long term for seeding and or uploading but cheap for install testing.


#6

@dtech thanks, but im pretty set in stone with my provider :

SO X3440 Intel X3440 4 x 2.53GHz 8 GB 4 x 2 TB 100 TB € 76 / month

at seedhost.eu

Cant really beat that price :slight_smile:

Thank you though! I am fine 100% with asking them to reimage, just didnt want to have to do this until i knew the next update would have built in help with that :stuck_out_tongue:


#7

For testing I just use VirtualBox setups on a bridged home network. Easy peezy.

Although, it would be great if more of these host offered an automated means of reinstall… I mean, in the age of the cloud… it just makes sense. It’s a lot of work to convert an infrastructure, so I assume this holds a lot of the other players back on that arrangement.


#8

same, for testing i use ESXI, here is my current home “lab” that i run the tests on :


#9

I could hear my wife nagging me now.


#10

Nice setup! I’ve been using scaleways as they are nice and cheap; however with the atom processors an installation that compiles rtorrent/deluge/ffmpeg can take 30-45 minutes


#11

@denellum I hate you just a little :stuck_out_tongue: I have a old pc that i have also just use vms on my desktop. but i would love a setup like that. but being broke sucks lol.


#12

Here’s some info on the status of the update.

The installer is now 100% stable and now begins some final testing on Ubuntu 16.04 as well as well Debian 8. I am currently running trials against root mounts and will follow this up with home mounts shortly. Thus far, everything is going swimmingly and a lot of the bugs have been sorted out.

I am also going to be trialing a method of upgrading from previous installs to the new eco-system. It should actually be a surprisingly easy process without the need to reinstall ones machine. Now, many of the commands that require changes on the initial install are being handled via cron.d, this will allow users on current installs as well as future installs to overwrite and have the data replaced automatically. These commands will/are also included with the new system… one such example is the updating of your dashboards network interface adapter. This can easily be handled via set_interface… however, this will additionally be handled by the cron set in place.

I’ll post more as it comes along. I can say with certainty that by this time tomorrow (wherever you are in the world) the new update and method of install will be released. Then we can focus on rolling release updates as they are committed to the repos.


#13

Pumped, can’t wait to try it out :slight_smile:


#14

I have tested this out as thoroughly as I possibly can and all systems are a go.

You can run the update if you are on Ubuntu 16.04 (I only say this as it is the only one I have had the time to fully test) with the following snappy bits:

git clone https://github.com/QuickBox/quickbox_packages quickbox_packages
cd quickbox_packages/system
bash updateQuickBox

This will fully update your sudoers files, your master database files (that is new, yes), relocate your command code strucutre for QuickBox v2.3.0 as well as add in the rolling update features that are so welcomed here and on into the future.

Here is a quick bit of info on how updating works once you are on v.2.3.0 – and it couldn’t be easier. I will have a more thorough Wiki on this later as well as proper ways to contribute to the code from here on out.


If you would like to test new commits not yet placed within the master branch, you can do so by navigating to the repo you’d like to test and typing git checkout development. This will switch your pulls from branch to branch and thus track the changes from that particular repo. The output will be similar to that of below…

[[email protected]]:(0b)~/quickbox_packages# git checkout development
Branch development set up to track remote branch development from origin.
Switched to a new branch 'development'

Once you have successfully switched branches, you can now run git pull to pull all known commits within the branch of a particular QuickBox repo. Again, these repos are located at /root/quickbox_[repo-here]. Running git pull on the branch will show similar to that of below (if any additions have been made prior to your initial pull).

[[email protected]]:(0b)~/quickbox_packages# git pull
remote: Counting objects: 4, done.
remote: Compressing objects: 100% (4/4), done.
remote: Total 4 (delta 3), reused 1 (delta 0), pack-reused 0
Unpacking objects: 100% (4/4), done.
From https://github.com/QuickBox/quickbox_packages
   01009fe..0e7efe2  development -> origin/development
Updating 01009fe..0e7efe2
Fast-forward
 system/updateQuickBox | 93 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++----------------
 1 file changed, 77 insertions(+), 16 deletions(-)
[[email protected]]:(0b)~/quickbox_packages#

You can switch back to the master branch (stable) at any time by navigating back to your desired repo and typing in git checkout master followed by git pull


#15

does this mean if i run removepackage-cron it will update the system to the new auto update without messing everything up if so thats pretty cool