[gitlab] Mise-à-jour de gitlab de la version 13 à 14

La mise à jour de gitlab de la version 13 à la version 14 ne s’est pas déroulée comme prévue, sinon je n’aurai pas écrit ce blog.

Ci-dessous les différentes étapes que j’ai suivies, pour arriver à mes fins.

Ne copier pas bêtement mes codes sans lire les messages d’erreurs durant la mise-à-jour sur votre machine. Lisez bien ce qui est marqué dans ces messages afin de pouvoir prendre les bonnes mesures pour réparer ce qui ne va pas.

Le problème que j’ai rencontré à commencé par une simple màj du système.

$ sudo apt upgrade
...
The following packages will be upgraded:
  gitlab-ee
1 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.
Need to get 0 B/1,013 MB of archives.
After this operation, 70.9 MB of additional disk space will be used.
Do you want to continue? [Y/n] 
(Reading database ... 115405 files and directories currently installed.)
Preparing to unpack .../gitlab-ee_14.2.3-ee.0_amd64.deb ...
gitlab preinstall: It seems you are upgrading from major version 13 to major version 14.
gitlab preinstall: It is required to upgrade to the latest 14.0.x version first before proceeding.
gitlab preinstall: Please follow the upgrade documentation at https://docs.gitlab.com/ee/update/index.html#upgrade-paths
dpkg: error processing archive /var/cache/apt/archives/gitlab-ee_14.2.3-ee.0_amd64.deb (--unpack):
 new gitlab-ee package pre-installation script subprocess returned error exit status 1

Pour vérifier la version de gitlab,

$ grep gitlab /opt/gitlab/version-manifest.txt |head -n1
gitlab-ee 13.11.3

Le message d’erreur me dit qu’il faudrait installer la dernière version en 14.0.x  (gitlab-ee_14.0.9-ee.0_arm64.deb) dans un premier temps.

L’installation de ce packet se fait, en précisant la version à installer après le signe égale en relation à notre paquet.

$ sudo apt install gitlab-ee=14.0.9-ee.0
Reading package lists... Done
Building dependency tree       
Reading state information... Done
The following packages will be upgraded:
  gitlab-ee
1 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.
Need to get 1,001 MB of archives.
After this operation, 77.9 MB of additional disk space will be used.
Get:1 https://packages.gitlab.com/gitlab/gitlab-ee/debian buster/main amd64 gitlab-ee amd64 14.0.9-ee.0 [1,001 MB]
Fetched 1,001 MB in 44s (22.8 MB/s)                                                                                                                                                     
(Reading database ... 115405 files and directories currently installed.)
Preparing to unpack .../gitlab-ee_14.0.9-ee.0_amd64.deb ...
gitlab preinstall: It seems you are upgrading from major version 13 to major version 14.
gitlab preinstall: It is required to upgrade to the latest 13.12.x version first before proceeding.
gitlab preinstall: Please follow the upgrade documentation at https://docs.gitlab.com/ee/update/index.html#upgrade-paths
dpkg: error processing archive /var/cache/apt/archives/gitlab-ee_14.0.9-ee.0_amd64.deb (--unpack):
 new gitlab-ee package pre-installation script subprocess returned error exit status 1
Errors were encountered while processing:
 /var/cache/apt/archives/gitlab-ee_14.0.9-ee.0_amd64.deb
E: Sub-process /usr/bin/dpkg returned an error code (1)

Même problème que précédement. Le système me demande d’installer la dernière version en 13.12.x, qui est gitlab-ee_13.12.8-ee.0_amd64.deb.

$ sudo apt install gitlab-ee=13.12.9-ee.0
...
Upgrade complete! If your GitLab server is misbehaving try running
  sudo gitlab-ctl restart
before anything else.
If you need to roll back to the previous version you can use the database
backup made during the upgrade (scroll up for the filename).

Puis,

$ sudo apt install gitlab-ee=14.0.9-ee.0
Reading package lists... Done
Building dependency tree       
Reading state information... Done
The following packages will be upgraded:
  gitlab-ee
1 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.
Need to get 0 B/1,001 MB of archives.
After this operation, 53.0 MB of additional disk space will be used.
(Reading database ... 116886 files and directories currently installed.)
Preparing to unpack .../gitlab-ee_14.0.9-ee.0_amd64.deb ...
* unicorn['worker_timeout'] has been deprecated since 13.10 and was removed in 14.0. Starting with GitLab 14.0, Unicorn is no longer supported and users must switch to Puma, following https://docs.gitlab.com/ee/administration/operations/puma.html.
* unicorn['worker_processes'] has been deprecated since 13.10 and was removed in 14.0. Starting with GitLab 14.0, Unicorn is no longer supported and users must switch to Puma, following
 https://docs.gitlab.com/ee/administration/operations/puma.html.
Deprecations found. Please correct them and try again.
dpkg: error processing archive /var/cache/apt/archives/gitlab-ee_14.0.9-ee.0_amd64.deb (--unpack):
 new gitlab-ee package pre-installation script subprocess returned error exit status 1
Errors were encountered while processing:
 /var/cache/apt/archives/gitlab-ee_14.0.9-ee.0_amd64.deb
E: Sub-process /usr/bin/dpkg returned an error code (1)

La màj vers la version 14.0.9 n’est pas possible, puisque l’utilisation de l’ancien serveur Web de gitlab (unicorn) est obsolète.

Pour désactiver l’ancien mode de serveur,

$ sudo cat /etc/gitlab/gitlab.rb | grep -e ^unicorn
unicorn['worker_timeout'] = 60
unicorn['worker_processes'] = 4

$ sudo sed -i "s/^unicorn/# unicorn/g" gitlab.rb

$ sudo gitlab-ctl reconfigure

$ sudo apt install gitlab-ee=14.0.9-ee.0

Et voilà, la dernière version de gitlab en 14.0.x est installée. Essayons de mettre à jour le système une nouvelle fois.

$ sudo apt upgrade 
...
Warnings:
The version of the running redis service is different than what is installed.
Please restart redis to start the new version.

sudo gitlab-ctl restart redis

The version of the running postgresql service is different than what is installed.
Please restart postgresql to start the new version.

sudo gitlab-ctl restart postgresql

Running handlers complete
Chef Infra Client failed. 19 resources updated in 01 minutes 46 seconds

Ben, faisons ce qu’on nous demande de faire

$ sudo gitlab-ctl restart redis
ok: run: redis: (pid 865) 1s

$ sudo gitlab-ctl restart postgresql
ok: run: postgresql: (pid 877) 0s

Puis,

$ sudo gitlab-ctl reconfigure
..
Running handlers:
There was an error running gitlab-ctl reconfigure:

rails_migration[gitlab-rails] (gitlab::database_migrations line 51) had an error: Mixlib::ShellOut::ShellCommandFailed: bash[migrate gitlab-rails database] (/opt/gitlab/embedded/cookbooks/cache/cookbooks/gitlab/resources/rails_migration.rb line 16) had an error: Mixlib::ShellOut::ShellCommandFailed: Expected process to exit with [0], but received '1'
---- Begin output of "bash"  "/tmp/chef-script20210902-890-17x6l1n" ----
STDOUT: rake aborted!
StandardError: An error has occurred, all later migrations canceled:

Expected batched background migration for the given configuration to be marked as 'finished', but it is 'active':	{:job_class_name=>"CopyColumnUsingBackgroundMigrationJob", :table_name=>"ci_stages", :column_name=>"id", :job_arguments=>[["id"], ["id_convert_to_bigint"]]}

Finalize it manualy by running

	sudo gitlab-rake gitlab:background_migrations:finalize[CopyColumnUsingBackgroundMigrationJob,ci_stages,id,'[["id"]\, ["id_convert_to_bigint"]]']

For more information, check the documentation

	https://docs.gitlab.com/ee/user/admin_area/monitoring/background_migrations.html#database-migrations-failing-because-of-batched-background-migration-not-finished
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/migration_helpers.rb:1129:in `ensure_batched_background_migration_is_finished'
/opt/gitlab/embedded/service/gitlab-rails/db/post_migrate/20210707210916_finalize_ci_stages_bigint_conversion.rb:13:in `up'
/opt/gitlab/embedded/service/gitlab-rails/lib/tasks/gitlab/db.rake:61:in `block (3 levels) in '
/opt/gitlab/embedded/bin/bundle:23:in `load'
/opt/gitlab/embedded/bin/bundle:23:in `

Encore une fois, faisons ce qu’on nous demande de faire.

$ sudo gitlab-rake gitlab:background_migrations:finalize[CopyColumnUsingBackgroundMigrationJob,ci_stages,id,'[["id"]\, ["id_convert_to_bigint"]]']
Done.

En lançant de nouveau la commande « gitlab-ctl reconfigure », on me demande de lancer la commande,

$ sudo gitlab-rake gitlab:background_migrations:finalize[CopyColumnUsingBackgroundMigrationJob,push_event_payloads,event_id,'[["event_id"]\, ["event_id_convert_to_bigint"]]']
Done.

$ sudo gitlab-ctl reconfigure
...
Running handlers:
Running handlers complete
Chef Infra Client finished, 9/585 resources updated in 01 minutes 20 seconds
gitlab Reconfigured

Et voilà, gitlab est maintenant à jour.

$ grep gitlab /opt/gitlab/version-manifest.txt
gitlab-ee 14.2.3
gitlab-config-template         14.2.3                                     
gitlab-cookbooks               14.2.3                                     
gitlab-ctl                     14.2.3                                     
gitlab-ctl-ee                  14.2.3                                     
gitlab-elasticsearch-indexer   v2.13.0                                    git:31d8f69fc3155cfb266d2d8828722bf25dc8f22e                              
gitlab-exporter                11.2.0                                     
gitlab-geo-psql                bc339ea21bdbb329263ad93788aaa428           
gitlab-healthcheck             90e6447aeead4a29ac9fffc15945ce6b           
gitlab-kas                     v14.2.2                                    git:2b22fa677a333d86c52cbc01d059b0f5a7d1f311                              
gitlab-pages                   v1.42.0                                    git:118a1252ada382e00e53d7ee7e57f059f0e31aba                              
gitlab-pg-ctl                  bcf1e38894e7acf21c429648b0176666           
gitlab-psql                    7e11364159031db0eb55867ad3d5713b           
gitlab-rails                   v14.2.3-ee                                 git:b5eea856ecaa112a6ca45a0a39adc7e370a0fc90                              
gitlab-redis-cli               8866b9c12e308342bc148a794f5cc166           
gitlab-scripts                 14.2.3                                     
gitlab-selinux                 14.2.3                                     
gitlab-shell                   v13.19.1                                   git:757aa7c0e0b8733c75c744f188b0136c1fe1830f                              
registry                       v3.7.0-gitlab                              git:af5f507f08f18178694282570330b9a4a75c8151

Sources

Upgrading from 13.12 to 14.0