Node Development with Docker
I don’t use Docker for my development environments as a matter of course, but I do sometimes find that it is useful. I use it infrequently enough for development that I sometimes forget some of these tricks (specifically the one about the NPM cache, which I’ve now had to figure out twice). So, to save my future self some time, I thought I’d write these down.
If you’re not interested in the explanations, you can just jump down to the sample config.
Why Docker?
I tend to reach for Docker when a project involves multiple services that need to communicate with one another over a network. For example, you may have some backend services that are written as Amazon Web Services (AWS) Lambda Functions and a front-end that’s a single page app sending HTTP requests to those Lambda Functions. Or maybe you’ve got a Django application that needs to communicate with a relational database like PostgreSQL or MySQL.
In cases like these, I find it’s much easier to configure a few Docker containers with Docker Compose than it is to set these technologies up individually.
Docker Compose
I like to use Docker Compose for development environments for a few reasons.
-
Starting the development environment is usually as simple as running
docker-compose up
- Docker Compose handles all of the networking between containers, so it’s easy to configure connections between them
-
If I’m not deploying the application as a container, then I can
usually configure an official base image in
docker-compose.yml
directly instead of having to create aDockerfile
Binding the src/
Directory
Whether you’re using something like
Vite for hot module replacement
(HMR) or nodemon to
kick your Express server, most web development tools these days have
some capacity to watch your source code for changes and automatically
update the running application. I don’t want to give up this handy
aspect of modern tools just because I’m using Docker, so I usually
create a
bind mount
for my src/
directory into the
WORKDIR
of the Docker container.
Binding your host directory over the working directory of the container means any changes to the files in there will be immediately available in the container without having to rebuild it. So if you’re running some live-reloading server in the container, it will notice the changes whenever you save and reload the application.
In your docker-compose.yml
file using an official Node
image, it would look something like this:
frontend:
image: "node:14.18.1-alpine"
volumes:
- "./src/:/home/node/app/"
Setting the User ID in the Container
I’ve often found it to be the case that I need to run commands in the
container that generate files in my project directory. A common one
with Node would be npm install
, but if you’re using an
Object-Relational Mapping (ORM) to talk to a database, it
may provide tools for generating database migrations based on the
changes you make to your models (I run into this one a lot with Python
web applications).
Unfortunately, the user in the container may be different than your
host user, so if you run those commands in the container and it puts
files in your src/
directory, those files won’t be owned
by your user. You’ll have to sudo chown
them every time
you run one of these commands in the container. This is tedious, so
you’re left with two alternatives:
- Set up a development environment (Python, Node, etc.) on your host machine to run these commands as your user instead of the container’s user
- Change the user in the container to match your user
I prefer the second option. Duplicating the environment that already exists in the container seems fragile and undermines the value of Docker in the first place. Plus, I like to think about how much I’m asking of anyone who might want to contribute to a project. If I’ve already asked a hypothetical contributor to set up Docker on their computer, I don’t want them to then have to set up the correct version of Node (or Python), possibly involving the use a version manager like nvm, and install dependencies into both environments and keep those environments in sync.
Fortunately, there’s an easy way to override the user ID and group ID
of the current user in a container so that whatever files are created
by that user in the container will have the same ownership as if they
had been created by your user on the host. Simply set the
user
property of the service in your
docker-compose.yml
file to be the user ID and group ID of
your user, which you can easily find by running id -u
and
id -g
in a terminal. It looks something like this:
frontend:
# ...
user: "1000:1000"
A More Portable user
Setting
The only problem with the above configuration is that you can’t
guarantee that other contributors to your project all have a user ID
of 1000 and a default group ID of 1000 (or whatever your user and
group IDs are). You don’t want them editing
docker-compose.yml
for their user, because they might
accidentally commit it and then you end up in a commit war with your
collaborators over the user settings for the development environment.
Instead, I like to set these values as environment variables.
Conveniently, Docker Compose will check for a .env
file
in the same directory in which
docker-compose.yml
resides. So you can just put those IDs
in variables in a .env
and set them in your configuration
from the variables.
echo "UID=$(id -u)" >> .env
echo "GID=$(id -g)" >> .env
And then your docker-compose.yaml
refers to those
variables.
frontend:
# ...
user: ${UID}:${GID}
Make sure you add .env
to your
.gitignore
.
Et voilà, any files created by the user in the container will
always have the appropriate ownership on the host filesystem for all
contributors to the project. I find the .env
approach
quite convenient because most of the languages I work with have some
package for reading environment variables out of a
.env
file as well, so I often want a
.env
file for development environment configuration
anyway.
Running Commands in the Container
I like to override the
entrypoint
for the official Node image to be npm
instead of their
docker-entrypoint.sh
script. This makes executing commands in the container a little easier
for me, since I tend to use NPM scripts for all my tasks. So I’ll add
this to my
docker-compose.yml
:
frontend:
# ...
entrypoint: ["npm"]
command: ["run", "dev"]
By default, the container will run npm run dev
, but I can
easily run other commands such as npm install
by just
passing the subcommand (install
in this case) directly to
the image when I run it. So installing dependencies is simply:
docker-compose run --rm frontend install
npm
as the container entrypoint
Alternatively, leave the entrypoint
alone and just set up
the command to be whatever you like:
frontend:
# ...
command: ["npm", "run", "dev"]
And installing dependencies looks like:
docker-compose run --rm frontend npm install
npm
when
you haven’t configured the entrypoint
Moving the NPM Cache
This is the big “gotcha” with using Docker for a Node dev environment and is the reason I decided to write all of this nonsense down. It took me what seemed like ages to figure out a solution to this problem.
Because of how the official Node Docker image is configured, NPM tries
to put its cache of packages in /.npm/
. Unless you’re
running as root
, you will get an error from NPM whenever
you try to install packages. One way around this is to always run
npm install
as root
(docker-compose run --rm -u root frontend install
), but then you have to sudo chown
your
node_modules/
directory after every install or update.
I think a better solution is to move where NPM puts its cache to
somewhere you have permission to write. Fortunately, NPM allows you to
set environment variables for any config parameters
just by prefixing them with NPM_CONFIG_
. So we don’t have
to run npm config cache
to change the cache directory, we
can just set NPM_CONFIG_CACHE
to the desired location in
our docker-compose.yaml
.
frontend:
# ...
environment:
- NPM_CONFIG_CACHE=/home/node/app/.npm-cache-docker/
I like to give it a name that makes it very clear that this is a directory created by the Docker container, just to be safe.
You may have noticed that we’re putting the NPM cache directory in the
directory that we’ve mounted our src/
directory over,
which means that the the NPM cache will end up on the host filesystem
in src/.npm-cache-docker
, so you’ll want to remember to
add src/.npm-cache-docker
to your
.gitignore
.
Other than having to add the NPM cache to your
.gitignore
, the only other disadvantage that I can think
of with this approach is that you can’t take advantage of a
system-wide cache for NPM. So if you have multiple projects that have
some of the same dependencies (likely), they’ll be fetching and
caching them per-project instead of sharing them.
In practice, this has never been a problem for me, but if you have a better solution to this problem, I’d love to hear about it, so send me a webmention.
Wrapping Up
And that pretty much does it.
With all of this set up in Docker Compose, my
README
usually contains instructions that look something
like this to get a new development environment started:
echo "UID=$(id -u)" >> .env
echo "GID=$(id -g)" >> .env
docker-compose run --rm frontend install
docker-compose up
version: 3.8
frontend:
image: "node:14.18.1-alpine"
volumes:
- "./src/:/home/node/app/"
environment:
- NPM_CONFIG_CACHE=/home/node/app/.npm-cache-docker/
ports:
- "8080:8080"
user: ${UID}:${GID}
entrypoint: ["npm"]
command: ["run", "dev"]
# Other services ...
UID=1000
GID=1000
.env
src/.npm-cache-docker