docker-compose up seems to have worked… no errors, and Docker Desktop shows 3 containers: rabbitmq-1, db-1, and app-1. RabbitMQ and Postgres are running, but the app has crashed because it can’t connect to the database.
Looking more closely, it seems like the port directive in docker-compose.yml is being ignored. I’ve deleted the images and rebuilt this several times in the course of testing it, and each time the ports used come up with something more random (?) like 53665. If I set my database client to use that port, I can connect.
Can someone clarify why this port is changing? And I seem to remember that the list of services defined inside of the docker-compose.yml corresponded to hostnames? Is that accurate? Like, instead of localhost, you might reference a hostname of db – but that doesn’t seem to work (even if I use the numeric extension).
Yes, you can reference containers by their service names, no numeric extension should be added, the name should be exactly as it is defined in docker-compose.yml.
port option is for just exposing the container port to your host, the option port: "5432" will expose port 5432 from the container on your host on 5432.
I would recommend checking that your host 5432 port is free, otherwise map to a different port:
ports:
- "5432:5433"
Also ensure that you are not connecting to some already exsting volumes with some strange Postgres configurations, a clean postgres/postgis container will always serve on port 5432 by default.
The last and the first thing I should have mentioned is to check that the database container has started as expected, it can happen that it will not start if it’s missing some mandatory configuration options (like username), you should have logs that it started accepting clients on port 5432.
That’s the thing: the port seems to be ignored. Each time I bring up the stack, postgres is binding to a new port. There is not a conflict that I can see, i.e.
lsof -i -P | grep -i "listen"
shows nothing bound to port 5432.
When I re-run docker-compose up, the images come up including the postgres image, but it binds to a seemingly random port, e.g. 59716.
The hostname does not seem to apply… I can connect to postgres using my db client using localhost as the hostname (and whatever random port it binds to). Using db or db-1 gets an error, e.g. getaddrinfo ENOTFOUND db. Maybe that hostname is only available to the containers themselves and not to the host OS?
I’m thinking next steps for me is to try ditching docker-compose and try spinning up the needed services as simple docker conainters (without docker-compose). But I am stymied.
You sure your setting your ports correctly? BTW the setting should be ports not port in your compose file (as implied by your first post). Also make sure you’ve got the external and internal ports configured if you want to access it via localhost. For example:
ports:
- "5432:5432"
If you’re still having issues, then let’s see that compose file.
You are only specifying the container port (i.e. the port that the container is listening on). When that is done then the host port (the port that your machine will be listening on to be able to connect you to the DB) will be chosen randomly.
As @arcanemachine points out, you need to specify both ports if you want full control. Example:
ports:
- "6000:5432"
This will make PostgreSQL accessible from your dev machine on port 6000, while the DB container will listen on 5432. Docker will make a tunnel so when you connect to localhost:6000 you will be actually connecting to db:5432. Which means that you can do this from your dev machine / host:
Thanks. This is coming back to me. It makes sense. Seems like we might need a PR for updating that example repo? Maybe it only works if you start the app from inside the docker container
I’ve just ran the docker-compose and everything works as expected from the standpoint of database. It’s a very strange setup, but it works as expected without any issues.
What OS are you using? I’ve ran the example on ubuntu.
The only thing to keep in mind is that ecto migrations are not ran automatically, so you have to execute this command before starting the compose:
I just tried re-cloning this repo and I’m still seeing the same problems… docker-compose up brings up 2/3 of the containers – there are a bunch of db connection errors:
Just installed docker desktop on my mac and everything works. There is most probably an issue with your installation.
I’ve ran the following commands (note the usage of docker compose, the old command docker-compose is deprecated from long ago):
# Just because I am too lazy to setup the rabbitmq queue manually,
# you should get errors that tables don't exist from app,
# close after it runs for a few seconds
docker compose up
docker compose run app mix setup
docker compose up
I’ve used such docker-compose setups a lot of times in production and there is nothing wrong with it, most probably you have some installation issues on your machine.
Can you try connecting to the host IP+port with psql, just to make sure that you actually can reach the DB? You can also run docker ps just to make sure as well.
Interesting. Ok, I get a lot of errors, but after running those, it works.
@dimitarvp I can connect using psql or similar, but I have to check with Docker to see which port the database is running on (because it changes each time I start it).
This makes zero sense, I would recommend to check your setup, this never should happen and it’s not expected behavior, the exposed port is dictated by the configuration and it is never random.
It is random if he didn’t change the ports part of the configuration, the version he linked to was only specifying container port which means that the host port is chosen at random every time.
OK, but should you not specify the host port as well? It’s a trivial change. Doing this dance with the random port is not productive and I don’t understand why you keep sticking to it.
I think this all points to the Broadway example repo benefiting from a pull request that makes this all a bit more straight-forward. As has been suggested here, I think this would make a lot more sense if the ports were fixed.
They likely don’t plan on connecting to the containerized services from the host at all, hence their configuration.
I personally disagree with that choice of theirs but they have a point as well – they have produced something that works and they don’t see why you would need to poke inside it.
In your case however you can change the configuration very easily so I am not sure why do we keep debating the same thing in several different ways.
Again, very likely it’s because their docker-compose.yaml file is used for CI/CD and deployment, not for local development, that’s my hypothesis. If it was used for local dev then surely it would have the host:container port pairs.
