Fixing gitlab 12.0 after OS updates

Unfortunately I'm currently stuck with gitlab 12.0.x as I want to use mysql, my server is still 32bit and only has 8G of RAM for all services it's running. I do plan to move gitlab to a docker container on a x86_64 server but at this time I'm stuck with what I have.

Now most of the ruby packages for gitlab are installed for the gitlab user, however some are shared system and even then some of the local ones are linked against system libs if they've a native component.

On occasion gitlab won't work correctly and needs a recompile of components. The below runs as the git user after stopping gitlab. There's some slight changes to the official docs to ensure the ruby components are rebuilt and not reusing the potentially broken already installed ones.

I'm not sure if the backup would work prior to doing this but it's advisable to try before stopping the service.

Let's get to it.

Backup - as git user.

$ bundle exec rake gitlab:backup:create RAILS_ENV=production

Stop the service - probably best as root

$ service gitlab stop

Reinstall all the gitlab components - as git user.

$ cd /home/git/gitlab-shell/
$ bin/compile 

$ cd /home/git/gitlab-workhorse
$ make clean
$ make

$ cd /home/git/gitaly/
$ make clean
$ cd  /home/git/gitaly/ruby && bundle install --redownload --deployment
$ cd ..
$ make

$ cd /home/git/gitlab-pages/
$ make

$ cd /home/git/gitlab
$ bundle install --redownload --deployment --without development test postgres aws kerberos
$ bundle clean
$ bundle exec rake db:migrate RAILS_ENV=production
$ bundle exec rake gettext:compile RAILS_ENV=production
$ bundle exec rake yarn:install gitlab:assets:clean gitlab:assets:compile RAILS_ENV=production NODE_ENV=production NO_SOURCEMAPS=true
$ bundle exec rake cache:clear RAILS_ENV=production

Start the gitlab service back up - again probably as root

service gitlab start

Check gitlab is ok - as git user

$ cd /home/git/gitlab
$ bundle exec rake gitlab:env:info RAILS_ENV=production
System information
System:         Gentoo 2.6
Current User:   git
Using RVM:      no
Ruby Version:   2.5.7p206
Gem Version:    3.1.2
Bundler Version:2.1.4
Rake Version:   12.3.2
Redis Version:  5.0.7
Git Version:    2.24.1
Sidekiq Version:5.2.7
Go Version:     go1.12.15 linux/386

GitLab information
Version:        12.0.12
Revision:       ce125b970e4
Directory:      /home/git/gitlab
DB Adapter:     MySQL
DB Version:     10.2.29-MariaDB-log
SSH Clone URL:
Using LDAP:     no
Using Omniauth: no

GitLab Shell
Version:        9.3.0
Repository storage paths:
- default:      /home/git/repositories
GitLab Shell path:              /home/git/gitlab-shell
Git:            /usr/bin/git
$ cd /home/git/gitlab
$ bundle exec rake gitlab:check RAILS_ENV=production
Checking GitLab subtasks ...

Checking GitLab Shell ...

GitLab Shell: ... GitLab Shell version >= 9.3.0 ? ... OK (9.3.0)
Running /home/git/gitlab-shell/bin/check
Check GitLab API access: OK
Redis available via internal API: OK

Access to /home/git/.ssh/authorized_keys: OK
gitlab-shell self-check successful

Checking GitLab Shell ... Finished

Checking Gitaly ...

Gitaly: ... default ... OK

Checking Gitaly ... Finished

Checking Sidekiq ...

Sidekiq: ... Running? ... yes
Number of Sidekiq processes ... 1

Checking Sidekiq ... Finished

Checking Incoming Email ...

Incoming Email: ... Reply by email is disabled in config/gitlab.yml

Checking Incoming Email ... Finished

Checking LDAP ...

LDAP: ... LDAP is disabled in config/gitlab.yml

Checking LDAP ... Finished

Checking GitLab App ...

Git configured correctly? ... yes
Database config exists? ... yes
All migrations up? ... yes
Database contains orphaned GroupMembers? ... no
GitLab config exists? ... yes
GitLab config up to date? ... yes
Log directory writable? ... yes
Tmp directory writable? ... yes
Uploads directory exists? ... yes
Uploads directory has correct permissions? ... yes
Uploads directory tmp has correct permissions? ... skipped (no tmp uploads folder yet)
Init script exists? ... yes
Init script up-to-date? ... yes
Projects have namespace: ... 
( redacted multiple - "Group / Project ... yes" )
Redis version >= 2.8.0? ... yes
Ruby version >= 2.5.3 ? ... yes (2.5.7)
Git version >= 2.21.0 ? ... yes (2.24.1)
Git user has default SSH configuration? ... yes
Active users: ... 2

Checking GitLab App ... Finished

Checking GitLab subtasks ... Finished

Opening the website login and other functionality should be working again.