Thanks @ivanhercaz !

I think there are a few steps to take:

• Try curl https://nerveshub.DOMAIN.EXT/api/health just as a simple connectivity check
• The NERVES_HUB_CERT (ca.pem) is your server certificate that it is present to the device on connect (like any other HTTPS server/website you visit). On device, that needs to go in :cacerts option as a list of DER format certs (or if you have the file, you can just use cacertfile: "ca.pem")
• You need to create a signer CA certificate used for creating device certificates. Admittedly, the documentation around this is dated and sparse. The easiest steps would be to use the task in GitHub - nerves-hub/nerves_key: NervesKey info and support for Elixir · GitHub
  • mix nerves_key.signer create some_signer_name
• Then, create a device certificate
  • mix nerves_hub.device cert create SERIAL --signer-cert /path/to/some_signer_name.cert --signer-key /path/to/some_signer_name.key
  • This device SERIAL needs to be created in NH with the newly generated certificate. Its probably easiest to do this in the UI and manually upload the certificate to it
• Register your some_signer_cert signer CA with NH either in the web UI or with mix nerves_hub.ca_certificates register /path/to/some_signer_cert.crt
• mix nerves_hub.ca_certificates will first need mix nerves_hub.user auth
  • Ensure your NervesHubUserAPI config has the ca.pem for the server in the options as well

# cacerts needs to be DER formatted. Probably easiest to use
# `cacerts =Enum.map(X509.from_pem("ca.pem"), &X509.Certificate.to_der/1)` and put that
# result in the cacerts key below
config :nerves_hub_user_api, ssl: [cacerts: cacerts]

So the walkthrough I followed is the next one:

1. Format the server to start all the process from zero.
2. ./bin/remote setup to setup the server.
3. Formateo el servidor, empiezo con él desde cero.
4. Create the database, user and assign password.
5. Generate the ca.key with openssl genrsa -out ca.key 2048.
6. Generate the ca.pem with openssl req -x509 -new -nodes -key ca.key -sha256 -days 1825 -out ca.pem -subj '/OU=NervesHub/'.
7. Generate nerveshub.DOMAIN.EXT-key.pem with openssl genrsa -out nerveshub.DOMAIN.EXT-key.pem
8. Generate nerveshub.DOMAIN.EXT.csr with openssl req -new -key nerveshub.DOMAIN.EXT-key.pem -out nerveshub.DOMAIN.EXT.csr -subj '/CN=nerveshub.DOMAIN.EXT/'
9. Create nerveshub.DOMAIN.EXT.ext with:

authorityKeyIdentifier=keyid,issuer
basicConstraints=CA:FALSE
keyUsage = digitalSignature, nonRepudiation, keynEcipherment, dataEncipherment
subjectAltName = @alt_names

[alt_names]
DNS.1 = nerveshub.DOMAIN.EXT

10. Generate nerveshub.DOMAIN.EXT.pem: openssl x509 -req -in nerveshub.DOMAIN.EXT.csr -CA ca.pem -CAkey ca.key -CAcreateserial -out nerveshub.DOMAIN.EXT.pem -days 825 -sha256 -extfile nerveshub.DOMAIN.EXT.ext
11. Move nerveshub.DOMAIN.EXT.pem y nerveshub.DOMAIN.EXT-key.pem to /etc/ssl.
12. Modify the Caddyfile..
13. Fill the environment variables in /etc/nerves_env.

DATABASE_URL=postgresql://user:password@localhost/nerveshub

LOCAL_IPV4=127.0.0.1
ERL_COOKIE=

LIVE_VIEW_SIGNING_SALT=generated with phx.gen.secrets...
SECRET_KEY_BASE=generated with phx.gen.secrets...

HOST=nerveshub.DOMAIN.EXT
PORT=4000
URL_SCHEME=https
URL_PORT=443

DEVICE_HOST=nerveshub.DOMAIN.EXT
DEVICE_PORT=4001

API_HOST=nerveshub.DOMAIN.EXT
API_PORT=4002
API_URL_PORT=443

FROM_EMAIL=ivan@DOMAIN.EXT
NERVES_HUB_APP=all

FIRMWARE_UPLOAD_BACKEND=local
FIRMWARE_UPLOAD_PATH=

14. Run ./bin/remote deploy and fail, it is because the systemd service file was wrong (Feedback issues with the self hosting · Issue #952 · nerves-hub/nerves_hub_web · GitHub), so I fix it with an empty ExecStart= before the existing ExecStart=.
15. Run ./bin/remote deploy and works!
16. Check the NervesHub log and it shows an error because it needs the ca.pem file in /etc/ssl, so I copy the ca.pem to /etc/ssl.
17. NervesHub web is already accesible and an API call to /api/health returns {"status": "UP"}.
18. Run ./bin/remote iex and create an account.
19. Download from the server the ca.pem into ~/.nerves-hub.
20. Set the next environment variables:
    export NERVES_HUB_HOST=nerveshub.DOMAIN.EXT
    export NERVES_HUB_PORT=4001
    export NERVES_HUB_TOKEN=$token_of_my_nerveshub_account$
    export NERVES_HUB_CERT=cat /home/ivanhercaz/.nerves-hub/ca.pem
    export NERVES_HUB_ORG=organization
21. Generate the signer certificate with mix nerves_key.signer create ivanhc, so now I have ivanhc.key and ivanhc.cert.
22. Go to NervesHub web and create a product, then create a device and upload ivanhc.cert, copy the serial and generate the device certificate with mix nerves_hub.device cert create SERIAL --signer-cert ivanhc.cert --signer-key ivanhc.key:
    NervesHub server: nerveshub.DOMAIN.EXT:4001
    NervesHub organization: asi
    Creating certificate for SERIAL
    Signer cert path: ivanhc.cert
    Signer key path: ivanhc.key
    Finished
23. Convert ca.pem to der with openssl x509 -outform der -in ca.pem -out ca.der.
24. Set the configuration in config/config.exs:

cacerts =
  "/home/ivanhercaz/.nerves-hub/ca.der"
  |> File.read!()

config :nerves_hub_user_api,
  ssl: [
    cacerts: [cacerts]
  ]

config :nerves_hub_link,
  device_api_host: "nerveshub.DOMAIN.EXT",
  device_api_port: 4001,
  device_api_sni: "nerveshub.DOMAIN.EXT",
  ssl: [
    certfile:
      "/home/ivanhercaz/.nerves-hub/SERIAL-cert.pem",
    keyfile:
      "/home/ivanhercaz/.nerves-hub/SERIAL-key.pem",
    cacerts: [cacerts]
  ]

25. And finally… any command I use to communicate with NervesHub fails, for example, if I run mix nerves_hub.device list it returns:

22:58:44.081 [notice] TLS :client: In state :wait_cert at ssl_handshake.erl:2113 gener
ated CLIENT ALERT: Fatal - Handshake Failure
 - {:bad_cert, :hostname_check_failed}
Unhandled error: {:error, {:tls_alert, {:handshake_failure, 'TLS client: In state wait
_cert at ssl_handshake.erl:2113 generated CLIENT ALERT: Fatal - Handshake Failure\n {b
ad_cert,hostname_check_failed}'}}}

About the config/config.exs and ca.pem. My ca.pem only has one certificate inside, I mean only one pair of -----BEGIN CERTIFICATE-----CERT-----END CERTIFICATE-----. Thus I use the File.read!() and then insert into a list in :cacerts.

But based on what @jjcarstens suggested and another examples I have seen, like Poser repository, I understand the normal situation is that ca.pem has more than one. Am I doing something wrong with the ca.pem file in any of the steps I described?