Cannot connect to Cassandra instance in Docker, getting ConnectionRefusedError

Question

I'm running a cassandra container with the the help of docker. For the below set of configuration I'm trying to run the script.sh script which should run after cassandra is up and running. However, the below script keeps logging ConnectionRefusedError(111) error:

dd_cassandra  | Connection error: ('Unable to connect to any servers', {'127.0.0.1:9042': ConnectionRefusedError(111, "Tried connecting to [('127.0.0.1', 9042)]. Last error: Connection refused")})
dd_cassandra  | Waiting for cassandra-node-1...

I used the solution described in this SO Post to check further execute the script only if cassandra is ready but it runs into an infinite wait.

docker-compose.yml

version: '3'

services:
  dd_cassandra:
    container_name: dd_cassandra
    build:
      context: ./cassandra
      dockerfile: Dockerfile
      args:
        BASE_IMAGE: cassandra:latest
    ports:
      - "9042:9042"

cassandra/Dockerfile

ARG BASE_IMAGE
FROM ${BASE_IMAGE}

COPY script.sh /
RUN chmod +x /script.sh
EXPOSE 9042
CMD ["/script.sh"]

cassandra/script.sh

#!/bin/bash

while ! cqlsh -e 'describe cluster' ; do
  echo "Waiting for cassandra-node-1...";
  sleep 10
done

# some other piece of code that adds runs `cqlsh` command 
# in the cassandra database

Your script runs instead of the CMD from the base cassandra image, so nothing is actually starting Cassandra. Usually you'd include something like this as a fragment in an entrypoint script in a client container, not the server proper. — David Maze
– David Maze, Commented Jan 16, 2024 at 11:10
@DavidMaze The base image for dd_cassandra is cassandra:latest. The script should run on top of that image, right? — asn
– asn, Commented Jan 16, 2024 at 11:11
The container only runs that script; yes, using the image, but ignoring the CMD that's present in the base image's Dockerfile. — David Maze
– David Maze, Commented Jan 16, 2024 at 11:49
@DavidMaze Your script runs instead of the CMD from the base cassandra image - do you mean my script execution has overridded some process which would have started the cassandra in that container? — asn
– asn, Commented Jan 16, 2024 at 11:53
@DavidMaze Also, what should be the image inside the client container, if I create one? Also, if I create the client container and run this script as a CMD/ENTRYPOINT inside that client container. Won't the script actually run inside the client instead of the server? — asn
– asn, Commented Jan 16, 2024 at 11:56

zori · Accepted Answer · 2024-01-16 16:23:52Z

Quick fix

What you are doing is not exactly correct how containers should be used, but to just fix it you need to add USER cassandra to Dockerfile

ARG BASE_IMAGE
FROM ${BASE_IMAGE}

COPY script.sh /
RUN chmod +x /script.sh
USER cassandra
EXPOSE 9042
CMD ["/script.sh"]

Second change is to add cassandra to script.sh before while (it's needed because when you added your script cassandra service wasn't starting because you have overrided CMD click here and scroll to 'CMD ["cassandra" "-f"]')

#!/bin/bash

cassandra
while ! cqlsh -e 'describe cluster' ; do
  echo "Waiting for cassandra-node-1...";
  sleep 2
done

This works as I tested.

More appropriate approach

I suggest you doing it a little bit differently. Looking into How to check that a Cassandra node is ready? someone started cassandra container without creating new Dockerfile and he just wanted to access it from localhost, probably it was something like this using docker-compose.yaml

version: '3'

services:
  dd_cassandra:
    container_name: dd_cassandra
    image: cassandra:latest
    ports:
    - "9042:9042"

Then from localhost(not any container) you can access it using cqlsh localhost 9042 -e 'describe cluster' if you don't have cqlsh installed you can spin-up another container using docker run -it --rm --network host cassandra:latest bash and inside type that command

To achieve what you want you can add something like this

version: '3'

services:
  dd_cassandra:
    container_name: dd_cassandra
    image: cassandra:latest
    ports:
      - "9042:9042"

  cassandra_script:
    build:
      context: ./cassandra
      dockerfile: Dockerfile
      args:
        BASE_IMAGE: cassandra:latest

And one more change to script.sh we need add host dd_cassandra(name of the service/container) instead of using localhost

#!/bin/bash

while ! cqlsh dd_cassandra -e 'describe cluster' ; do
  echo "Waiting for cassandra-node-1...";
  sleep 10
done

After running docker-compose up and waiting around an half minute I got:

cassandra-cassandra_script-1  | 
cassandra-cassandra_script-1  | 
cassandra-cassandra_script-1  | Cluster: Test Cluster
cassandra-cassandra_script-1  | Partitioner: Murmur3Partitioner
cassandra-cassandra_script-1  | Snitch: DynamicEndpointSnitch
cassandra-cassandra_script-1  | 
cassandra-cassandra_script-1 exited with code

First container dd_cassandra is just cassandara service, second container cassandra_script has your script.sh in CMD so it runs your script, the limitation here is that Dockerfile can contain only one CMD so if it's used more then once only last occurrence will be invoked.

Erick Ramirez · Accepted Answer · 2024-05-13 04:02:40Z

The Cassandra service doesn't start because you have overridden the default Cassandra startup process when you specified the CMD instruction in your Dockerfile (as noted by @David Maze and @zori).

In your script it looks like you're waiting for Cassandra to be operational. Cassandra is considered "ready" when the JVM process starts listening for client connections on the CQL port (default is 9042). The way to handle this is to perform container healthchecks. For example:

  healthcheck:
    test: ["CMD-SHELL", "netstat -ltn | grep -q 9042"]

For the rest of your script, it is better to run them in a separate client container that will only start when the Cassandra container is operational using the depends_on directive. For example:

services:
  cassandra:
    ...
    healthcheck:
      test: ["CMD-SHELL", "netstat -ltn | grep -q 9042"]
      ...

  cassandra-schema:
    ...
    depends_on:
      cassandra:
        condition: service_healthy

For more info, see the official Docker Docs. Cheers!

Collectives™ on Stack Overflow

Cannot connect to Cassandra instance in Docker, getting ConnectionRefusedError

2 Answers 2

Quick fix

More appropriate approach

Comments

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Quick fix

More appropriate approach

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related