New image planning

[stuartpb]

This is a tracking issue to collect all the elements that are under consideration for a potential overhaul to the official RethinkDB image:

• Packaging from scratch (Packaging from scratch #43) or Alpine (Alpine version of container #32) instead of Debian
• Packaging for ARM (Arm Version #41)
• Running as a non-root user (Do not run as root #39)
• Removing the VOLUME directive (Can we get a version of this which doesn't mount a volume? #14)
• Housekeeping

Now, some of these have a larger potential for breaking changes than others (particularly removing the VOLUME directive), so I'm not necessarily going to do all of them immediately; however, many of these (like changing the user and packaging from scratch) shouldn't affect the daemon's normal operation, so I may tie them to the next minor bump for the official library. (Ideally, all this would be held off as potentially breaking changes that would only accompany a major version bump, but I don't see RethinkDB 3.0 happening any time in the foreseeable future.)

[nicodmf]

Can i help for the alpine container ?

[tianon]

Is this still something that's planned? Is RethinkDB something that's still maintained? Would it make sense to look at RebirthDB? (https://github.com/RebirthDB/rebirthdb)

[srh]

@nicodmf Yes, talk to the RebirthDB people.

@tianon All the activity is with RebirthDB (except for a small amount of stuff I do to fix stuff, and the miscellaneous PR's you see, and that'll end up in RebirthDB anyway) but it looks like they're going to become the people in charge of RethinkDB real soon. You'll notice that the main rebirthers are now owners of the RethinkDB org: https://github.com/orgs/rethinkdb/people

[stuartpb]

I like @daveisfera's work in #40 (comment) and currently that's probably the closest thing to what a new image would look like, but that is a hairy RUN line - I'd rather see something that uses more of upstream's build process, which (IIRC) uses Nix and was developed by @AtnNn.

In any case, what we really need is a new pull request implementing a build on Alpine using either of these two approaches (if we can't get this going from scratch, which is what I'd really rather have) - I can't merge a thread comment.

[daveisfera]

RUN commands to build a package usually are big and ugly. Multi-stage builds help clean that up, but they're currently not supported for official images.

The Dockerfile that I made uses the same build process as the Alpine package and is very close to the version used for the Debian builds (i.e. just modifications to work with Alpine), so I'm not sure what the changes you're requesting would be.

[stuartpb]

Is there a way we could pull the build command by reference from an upstream source, like how Arch Linux has the ABS? In other words, if this is how the Alpine package is built, can we perform the build and install it as our own package?

[dave-nm]

I'm not following. A Dockerfile is basically a build system like ABS. If you go dig into the details of an ABS build, they're just as big and ugly. They have better ways to break it up and structure things so it's easier to look at and manage, but once multi-stage builds are supported, then the Dockerfile can be cleaned up in the same sort of way.

[stuartpb]

I guess what I'm saying is, if this does the some thing as https://git.alpinelinux.org/aports/tree/community/rethinkdb/APKBUILD, why not just build from that aports tree source directly (which, from the looks of it, would also resolve #39 via https://git.alpinelinux.org/aports/tree/community/rethinkdb/rethinkdb.pre-install)? Why repeat ourselves?

[daveisfera]

Oh, do you mean to run the APKBUILD file from Alpine inside of the Dockerfile? If that's what you're asking, then I'm not sure what that would take but I'm guessing it's possible. The down side is that we'd be tied to the version that they have installed and if we're going to couple it to their packaging like that, then why not just install the already built binaries?

[stuartpb]

if we're going to couple it to their packaging like that, then why not just install the already built binaries?

I guess my original thinking behind this was that this was the only way to really pin a package release, since Alpine's repos only provide the latest version. The point of the Docker library seems to be about making the build process as deterministic as possible, though I'll admit that this seems like something of a farce when the first line of the Dockerfile bases the entire build on a mutable tag.

[stuartpb]

Argh... ultimately, what this comes down to is that RethinkDB did their own packaging for Debian/Ubuntu/CentOS, and they packaged the full matrix of releases to package versions, and if we're relying on distro packaging (which only caters to the latest version), the ability to maintain images for previous versions using only upstream-provided infrastructure falls apart.

I don't want the Dockerfiles repository to become a backports project, especially when RethinkDB upstream already maintains a backporting CI system for other distributions.

I feel like the real solution would be to get Alpine added to https://github.com/rethinkdb/rethinkdb-nix, but that would require Alpine to get added to Nix's whole VM building system first...

[stuartpb]

Agh, whatever, we don't list anything but the latest version in the library anyway (and I vaguely recall the exchange that led to this).

I could argue that it's a bad idea to let old images' dependencies stagnate, but there's also an argument to be made for not making untested changes to obscure backports... whatever.

Yeah, at this point I'm fine just using Alpine's package. When Alpine drops support for older versions, that's fine, because so does our building of them. Anybody who wants to keep an older image building will just have to invent their own CI system to support their specific blend of old and new, I guess!

[stuartpb]

Or, really, reflecting on #45 (comment), I think what'd be more appropriate than Ubuntu or Alpine going forward would be for the RethinkDB image to be based on a package for Nix: rethinkdb/rethinkdb-nix#2

[daveisfera]

Having a set of packages for each of the OSes is really nice (like what Postgres provides), but it's a ton of work. Also, even they don't maintain packages for Alpine and they just build the software in the Alpine base image, like I've done. Basically, they have two versions:

1. debian base that installs the prebuilt package ( https://github.com/docker-library/postgres/blob/85aadc08c347cd20f199902c4b8b4f736341c3b8/9.6/Dockerfile )
2. alpine base that builds dynamically ( https://github.com/docker-library/postgres/blob/85aadc08c347cd20f199902c4b8b4f736341c3b8/9.6/alpine/Dockerfile )

I would vote that that be the same strategy that Rethink uses for the flexibility and control that it provides, because honestly being tied to whatever Alpine has limits the Docker images in a way that I believe will likely make them less useful than they should be.