SQL: Add 'migration' file

[awilfox]

Allows for the database to be created without running Prisma. The file is based on the output of pg_dump --schema-only willa in a developer container. I cleaned up the syntax a bit and grouped related elements.

Adds a bin/dbinit script that can be run to create the initial database structure.

Reorder the Dockerfile a bit to ensure files that are more likely to change are in the more outer layers.

Ref: AP-412

---

I'd prefer to land #74 first, but this should be the last actual requirement to a production deployment.

Tagging @danschmidt5189 for usability for the Docker build and the changes to the Dockerfile.

[anarchivist]

looks great. thank you! could we add directions in the readme?

[anarchivist]

looking at this again, i'm seeing the following on running from a freshly built image on a new container:

GDKJFQ1GXT:willa matienzo$ docker compose exec app bin/dbinit
createdb: error: database creation failed: ERROR:  database "willa" already exists
SET
SET
SET
SET
SET
 set_config
------------

(1 row)

SET
SET
SET
SET
SET
psql:sql/willa.sql:18: ERROR:  role "willa" does not exist
psql:sql/willa.sql:21: ERROR:  current transaction is aborted, commands ignored until end of transaction block
psql:sql/willa.sql:22: ERROR:  current transaction is aborted, commands ignored until end of transaction block
psql:sql/willa.sql:36: ERROR:  current transaction is aborted, commands ignored until end of transaction block
psql:sql/willa.sql:39: ERROR:  current transaction is aborted, commands ignored until end of transaction block
psql:sql/willa.sql:40: ERROR:  current transaction is aborted, commands ignored until end of transaction block
psql:sql/willa.sql:60: ERROR:  current transaction is aborted, commands ignored until end of transaction block
psql:sql/willa.sql:71: ERROR:  current transaction is aborted, commands ignored until end of transaction block
psql:sql/willa.sql:89: ERROR:  current transaction is aborted, commands ignored until end of transaction block
psql:sql/willa.sql:101: ERROR:  current transaction is aborted, commands ignored until end of transaction block
psql:sql/willa.sql:110: ERROR:  current transaction is aborted, commands ignored until end of transaction block
psql:sql/willa.sql:122: ERROR:  current transaction is aborted, commands ignored until end of transaction block
psql:sql/willa.sql:126: ERROR:  current transaction is aborted, commands ignored until end of transaction block
psql:sql/willa.sql:129: ERROR:  current transaction is aborted, commands ignored until end of transaction block
psql:sql/willa.sql:132: ERROR:  current transaction is aborted, commands ignored until end of transaction block
psql:sql/willa.sql:135: ERROR:  current transaction is aborted, commands ignored until end of transaction block
psql:sql/willa.sql:138: ERROR:  current transaction is aborted, commands ignored until end of transaction block
psql:sql/willa.sql:141: ERROR:  current transaction is aborted, commands ignored until end of transaction block
psql:sql/willa.sql:144: ERROR:  current transaction is aborted, commands ignored until end of transaction block
psql:sql/willa.sql:146: ERROR:  current transaction is aborted, commands ignored until end of transaction block
psql:sql/willa.sql:148: ERROR:  current transaction is aborted, commands ignored until end of transaction block
psql:sql/willa.sql:150: ERROR:  current transaction is aborted, commands ignored until end of transaction block
psql:sql/willa.sql:152: ERROR:  current transaction is aborted, commands ignored until end of transaction block
psql:sql/willa.sql:154: ERROR:  current transaction is aborted, commands ignored until end of transaction block
psql:sql/willa.sql:156: ERROR:  current transaction is aborted, commands ignored until end of transaction block
psql:sql/willa.sql:158: ERROR:  current transaction is aborted, commands ignored until end of transaction block
psql:sql/willa.sql:160: ERROR:  current transaction is aborted, commands ignored until end of transaction block
psql:sql/willa.sql:162: ERROR:  current transaction is aborted, commands ignored until end of transaction block
psql:sql/willa.sql:164: ERROR:  current transaction is aborted, commands ignored until end of transaction block
psql:sql/willa.sql:166: ERROR:  current transaction is aborted, commands ignored until end of transaction block
psql:sql/willa.sql:168: ERROR:  current transaction is aborted, commands ignored until end of transaction block
psql:sql/willa.sql:170: ERROR:  current transaction is aborted, commands ignored until end of transaction block
psql:sql/willa.sql:172: ERROR:  current transaction is aborted, commands ignored until end of transaction block
psql:sql/willa.sql:174: ERROR:  current transaction is aborted, commands ignored until end of transaction block
psql:sql/willa.sql:176: ERROR:  current transaction is aborted, commands ignored until end of transaction block
psql:sql/willa.sql:178: ERROR:  current transaction is aborted, commands ignored until end of transaction block
psql:sql/willa.sql:181: ERROR:  current transaction is aborted, commands ignored until end of transaction block
psql:sql/willa.sql:185: ERROR:  current transaction is aborted, commands ignored until end of transaction block
psql:sql/willa.sql:188: ERROR:  current transaction is aborted, commands ignored until end of transaction block
psql:sql/willa.sql:191: ERROR:  current transaction is aborted, commands ignored until end of transaction block
psql:sql/willa.sql:194: ERROR:  current transaction is aborted, commands ignored until end of transaction block
psql:sql/willa.sql:197: ERROR:  current transaction is aborted, commands ignored until end of transaction block
psql:sql/willa.sql:200: ERROR:  current transaction is aborted, commands ignored until end of transaction block

this is happening also if the containers are down and i run docker compose run app bin/dbinit instead.

[awilfox]

Changes in v2:

• Use CURRENT_USER to avoid hardcoding the username willa.

[awilfox]

Changes in v3:

• Modified README.

[awilfox]

Rebased atop main with #74 applied.

[danschmidt5189]

I wasn't able to get this running, and it also includes a number of statements (e.g. the grants) which go beyond the scope of just establishing the schema.

What about cat-ing together the migration files that ship with prisma (in prisma/migrations/*/migration.sql)? That would be identical to what it seems prisma is doing under the hood.

[awilfox]

The GRANT was needed in my testing because the database created by default in Docker doesn't allow the created role to create tables in the public schema. (It wasn't part of the original pg_dump.)

[awilfox]

I removed some of the superfluous sets that are already defaults. What failure were you seeing?

[danschmidt5189]

That GRANT line is not necessary in my testing. Steps to reproduce:

1. Hard reset to the current code version.
2. Remove or comment out the GRANT line.
3. Down the stack (remove orphans and volumes)
4. Delete .data/postgres¹: rm -rf .data/postgres
5. Start the stack: docker compose up --build -d app db
6. Run dbinit: docker compose exec app bin/dbinit²

¹ Should we use a named volume here instead? I found this confusing as it means docker compose down -v doesn't actually delete the data, which has to be rm'd manually. (Not necessary for this PR IMO, just raising the Q.)

² Or, equivalently, use docker compose run.

[awilfox]

Following those exact steps, I get:

psql:sql/willa.sql:10: ERROR:  no schema has been selected to create in

And this is because the public schema does not allow CREATE to willa:

willa=# \dn+
                                       List of schemas
  Name  |       Owner       |           Access privileges            |      Description       
--------+-------------------+----------------------------------------+------------------------
 public | pg_database_owner | pg_database_owner=UC/pg_database_owner+| standard public schema
        |                   | =U/pg_database_owner                   | 
(1 row)

willa=# GRANT CREATE ON SCHEMA public TO willa;
GRANT
willa=# \dn+
                                       List of schemas
  Name  |       Owner       |           Access privileges            |      Description       
--------+-------------------+----------------------------------------+------------------------
 public | pg_database_owner | pg_database_owner=UC/pg_database_owner+| standard public schema
        |                   | =U/pg_database_owner                  +| 
        |                   | willa=C/pg_database_owner              | 
(1 row)

Now the real question is why willa isn't inheriting it, because as far as I can tell, willa is the owner:

willa=# \l willa
                                               List of databases
 Name  | Owner | Encoding | Locale Provider |  Collate   |   Ctype    | Locale | ICU Rules | Access privileges 
-------+-------+----------+-----------------+------------+------------+--------+-----------+-------------------
 willa | willa | UTF8     | libc            | en_US.utf8 | en_US.utf8 |        |           | 

[awilfox]

Created AP-472 for the named volumes issue.

[awilfox]

Changes in v5:

• Rebased against main again.
• Added more logic to control when createdb is executed.
• DRY environment variables for PostgreSQL connection details.
• Remove unnecessary set commands from opening stanza.

[danschmidt5189]

Softly requesting two changes but they're not critical:

1. PGPASSWORD doesn't need to be set multiple times in dbinit.
2. The GRANT in the schema dump does not appear to be necessary based on my testing with a fresh instance.

Besides that it looks good, let's get this in!

[awilfox]

Changes in v6:

• Don't keep setting PGPASSWORD everywhere.
• Remove GRANT from schema.