Migrating from a docker container with SQLite to the TrueNAS SCALE TrueCharts enterprise-train app #3805
Lockszmith-GH
started this conversation in
Show and tell
Migrating from a docker container with SQLite to the TrueNAS SCALE TrueCharts enterprise-train app
#3805
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
-
Migrating from a docker container with SQLite to the TrueNAS SCALE TrueCharts enterprise-train app
Why is this posted here?
I'll cross post this to the TrueCharts community as well, but they don't have a forum or discussion board like this, only discord - this seems like `more stable` choice.I've finally successfully completed this migration which I've been putting off for a very long time, and am documenting it here.
There were a few trial-and-error iterations, and I hoped to capture this to share with anyone else who was struggling with this type of migration.
Goal
Migrate from a docker container (docker-compose) based install of vaultwarden with SQLite (running on Linux - In my case an Ubuntu 20.04 host where the
vaultwarden/server:latestimage, is currently at 1.29.1), to a TrueNAS SCALE server, running the TrueCharts helm chart for Vaultwarden.Challange
In TrueNAS SCALE, the concept of Apps is implemented as deployment of specialized Helm charts on a k3s cluster. These are deployed and managed via the TrueNAS SCALE's Web UI.
TrueCharts - a community repository of TrueNAS SCALE Apps, specifically their Enterprise train, introduce very robust implementations relying (among other things) of CloudNative PostgreSQL databases, with very robust and stable database deployments, even when running on a single node.
One of those apps, is Vaultwarden.
When migrating data from external sources, this presents challanges, as external access to these databases isn't straigh forward, and mainpulating k3s directly, isn't officially supported.
The migration process below was adapted from the steps published in the migration guide in the wiki while addressing the requirements of this envronment without breaking any rules that would deem the installation unsupported.
Migration process
flowchart LR subgraph ubuntu subgraph docker["- Docker Daemon -"] subgraph old-vw["vaultwaren (old)"] sqlite end subgraph tmp-pg["temp-postgresql"] tmp-vw-db end end end subgraph trueNAS subgraph k3s["k3s ix-apps cluster"] subgraph vw["vaultwarden (new TrueCharts app)"] cpng["cpng<br/>CloudNative PostgreSQL db"] end end end sqlite -->|'pgloader'| tmp-vw-db --> |'pgAdmin'| cpngOverview of the process:
vaultwardenapp on TrueNAS SCALE.Full details are provided below.
Tools / Prerequisites
The prerequisites for a successful process:
pgloaderinstalledheavyscriptinstalled - this is used to start and stop TrueChart apps.(back to overview)
Step 1: Install a fresh
vaultwardenapp on TrueNAS SCALEtcdbinfo.shscript to extract the db credentials. Here is an adapted version I made for myself, it is based on TrueChart's tcdbinfo.sh.* When using
--forcewith my version, you can use the superadmin user (named postgres), instead of just app user.cnpggoing forward) and perform a Backup of thevaultwardendatabase (namedcnpg-blank-backup-dbin this doc), as well as the server's globals* (namedcnpg-backup-globals) (the globals define the roles assigned to thecnpg).* will require connecting with
postgresuser (see # 2 above).(back to overview)
Step 2: Create a temporary PostgreSQL and initialize it
Start a PostgreSQL container (name in doc:
temp-pg) with the following command:* NOTE:
cpgnruns PostgreSQL 15.3, that's why I've chosenpostgres:15.3From pgAdmin, connect to
temp-pg.[optional] Restore the roles* from the globals backup - this is a manual process of loading the backup file into the Query Tool when connected to the postgres database on
temp-pg.* Will require
postgressuser, see # 2.* NOTE: This will reset the password of the users, and you'll need to disconnect and reconnect using the
cpngcredentials from this point forward.Created a new
tmp-vw-dbdb, assigned to thevaultwardenrole. (role assignment is optional)Restore
cnpg-blank-backupintotmp-vw-db, using the following options:Privileges + Comments + Subscriptions + Table access methods
At this point, You should have an initialized vaultwarden PostgreSQL database that is accessible from the old host.
(back to overview)
Step 3: Migrate the SQLite db into the temporary PostgreSQL database
Per the migration wiki, it time to use
pgloader.Prep a
migration.pgloaderfile with the following content:* NOTE: change the
fromandintoto match your deployment.Stop the source vaultwarden docker container.
Run
pgloader ./migration.pgloader. (See Appendix section below for example output)(back to overview)
Step 4: Migrate the data from the temporary PostgreSQL db into the app's database
With the data in the
tmp-vw-dbPostgreSQL db, it's time to transfer everything to thevaultwardendb oncnpg.This involved using
pgAdminagain.tmp-vw-db, named ittmp-vw-db-backup__diesel_schema_migrationsfromtmp-vw-db(per migration wiki we don't load it anyways, and if it is preset during restore, the restore breaks).tmp-vw-db, named itvw-migrationvaultwardenoncpngfromvw-migration, using the following options:Privileges + Comments + Subscriptions + Table access methods
(back to overview)
Step 5: Finalize migration
(back to overview)
Appendix
example pgloader output
(click to expand)Beta Was this translation helpful? Give feedback.
All reactions