[error] Mongo.MongoDBConnection (#PID<0.866.0>) disconnected: ** (Mongo.Error) cluster0-shard-00-01.8vwd1.mongodb.net:27017 tcp recv: unknown POSIX error - :closed

Questions / Help
, ,
1

Using {:mongodb_driver, “~> 0.9.1”},

Supervisor child spec

%{
        id: Mongo2,
        start:
          {Mongo, :start_link,
           [
             [
               name: :mongo2,
               url: "Remote MONGO DB URL its not local ",
               timeout: 100_000,
               pool_timeout: 100_000,
             ]
           ]}
      }

I am using IMAP to get mails and store the Envelope into Mongo
Even though the application is ideal or mongo is writing suddenly it gives below error

[error] Mongo.MongoDBConnection (#PID<0.866.0>) disconnected: ** (Mongo.Error) cluster0-shard-00-01.8vwd1.mongodb.net:27017 tcp recv: unknown POSIX error - :closed

Any help appreciated

@zookzook

2

I created a database using the Atlas service:

{:ok, conn} = Mongo.start_link(url: "mongodb+srv://<secrect>p@cluster0.169dm4y.mongodb.net/myFirstDatabase")

> Mongo.insert_one(conn, "dogs", %{name: "Greta"})
[info] CMD insert "dogs" [documents: [[_id: #BSON.ObjectId<63135a954ae06e5ccaa3e990>, name: "Greta"]]] db=109.9ms (module=Mongo function=do_log/5 line=1800 )

{:ok, %Mongo.InsertOneResult{acknowledged: true, inserted_id: #BSON.ObjectId<63135a954ae06e5ccaa3e990>}}
> Mongo.insert_one(conn, "dogs", %{name: "Tom"})
[info] CMD insert "dogs" [documents: [[_id: #BSON.ObjectId<63135a9a4ae06e5cca171679>, name: "Tom"]]] db=32.7ms (module=Mongo function=do_log/5 line=1800 )

{:ok, %Mongo.InsertOneResult{acknowledged: true, inserted_id: #BSON.ObjectId<63135a9a4ae06e5cca171679>}}
> Mongo.find(conn, "dogs", %{}) |> Enum.to_list()
[info] CMD find "dogs" [] db=23.2ms (module=Mongo function=do_log/5 line=1800 )

[
  %{"_id" => #BSON.ObjectId<63135a954ae06e5ccaa3e990>, "name" => "Greta"},
  %{"_id" => #BSON.ObjectId<63135a9a4ae06e5cca171679>, "name" => "Tom"}
]

Everything works as expected. Maybe your IP address is not whitelisted.

1 Like
3

Mongo
Mongo1687×318 15.8 KB

its set to any, Mongo is getting connected but after some time it shows the error

Using Ubuntu Ubuntu 20.04.4 LTS 64 bit VM on Windows 10 PC
Steps to recreate the above issue

Create a new Pheonix project
add it to mongo db as child in supervisor with URL as shown in 1st message

Run the application and it will result in error after few seconds

4

Could you give me more information? How long does it take to see the error after connection was established?

5

It happens everyday but not sure why its not happening now
Below error is showing up now

** (FunctionClauseError) no function clause matching in Access.get/3
    (elixir 1.13.4) lib/access.ex:283: Access.get({:error, %DBConnection.ConnectionError{message: "connection not available and request was dropped from queue after 922ms. This means requests are coming in and your connection pool cannot serve them fast enough. You can address this by:\n\n  1. Ensuring your database is available and that you can connect to it\n  2. Tracking down slow queries and making sure they are running fast enough\n  3. Increasing the pool_size (although this increases resource consumption)\n  4. Allowing requests to wait longer by increasing :queue_target and :queue_interval\n\nSee DBConnection.start_link/2 for more information\n", reason: :queue_timeout, severity: :error}}, "user_email", nil)
    (syncengine 0.1.0) lib/syncengine/sync/mail_tag_proc.ex:178: Syncengine.Sync.Mailtagproc.format_envelope_map/2
    (syncengine 0.1.0) lib/syncengine/sync/mail_tag_proc.ex:160: anonymous fn/2 in Syncengine.Sync.Mailtagproc.update_mails_in_mongo/2
    (elixir 1.13.4) lib/stream.ex:572: anonymous fn/4 in Stream.map/2
    (elixir 1.13.4) lib/enum.ex:4475: Enumerable.List.reduce/3
    (elixir 1.13.4) lib/stream.ex:1719: Enumerable.Stream.do_each/4
    (elixir 1.13.4) lib/stream.ex:649: Stream.run/1
    (syncengine 0.1.0) lib/syncengine/sync/mail_tag_proc.ex:32: Syncengine.Sync.Mailtagproc.handle_call/3
    (stdlib 4.0.1) gen_server.erl:1146: :gen_server.try_handle_call/4
    (stdlib 4.0.1) gen_server.erl:1175: :gen_server.handle_msg/6
    (stdlib 4.0.1) proc_lib.erl:240: :proc_lib.init_p_do_apply/3
Last message (from #PID<0.1086.1>): {:update_mail_in_mongo, [{38001, %{body_structure: %Mailroom.IMAP.BodyStructure.Part{description: nil, disposition: nil, encoded_size: nil, encoding: nil, file_name: nil, id: nil, multipart: true, params: %{}, parts: [%Mailroom.

I am writing 40000 emails (envelopes only) into Mongo as1000 emails per request using stream

  def update_mails_in_mongo(messages, user_details) do
    messages
    |> Stream.map(fn {uid, mail} ->
      case mail.envelope do
        :error ->
          %{}

        _ ->
          Mongo.BulkOps.get_insert_one(format_envelope_map({uid, mail}, user_details))
      end
    end)
    |> Mongo.OrderedBulk.write(:mongo, "Mails", 10_000)
    |> Stream.run()

Mongo setup in supervisor as below

 %{
        id: Mongo,
        start:
          {Mongo, :start_link,
           [
             [
               name: :mongo,
               url:
                 "mongodb+srv://<secret>@cluster0.8vwd1.mongodb.net/Station?retryWrites=true&w=majority&keepAlive=true&poolSize=30&autoReconnect=true&socketTimeoutMS=360000&connectTimeoutMS=360000",
               timeout: 100_000,
               pool_timeout: 100_000,
               pool_size: 10
             ]
           ]}
      },
6

Ok, but that is a complete other problem. First the timeout comes from the connection pool. You can increase the pool size. Second: you are writing a bulk of 10_000 instead of 1_000. Maybe reducing the number of inserts could help. But I don’t think, it is an issue with the driver.

7 

     Reason: 'Option {verify, verify_peer} and cacertfile/cacerts is missing'

[warning] Description: 'Authenticity is not established by certificate path validation'
     Reason: 'Option {verify, verify_peer} and cacertfile/cacerts is missing'

[error] Mongo.MongoDBConnection (#PID<0.724.0>) disconnected: ** (Mongo.Error) cluster0-shard-00-01.8vwd1.mongodb.net:27017 tcp recv: unknown POSIX error - :closed
[error] Mongo.MongoDBConnection (#PID<0.703.0>) disconnected: ** (Mongo.Error) cluster0-shard-00-01.8vwd1.mongodb.net:27017 tcp recv: unknown POSIX error - :closed
[error] Mongo.MongoDBConnection (#PID<0.699.0>) disconnected: ** (Mongo.Error) cluster0-shard-00-01.8vwd1.mongodb.net:27017 tcp recv: unknown POSIX error - :closed
[warning] Description: 'Authenticity is not established by certificate path validation'
     Reason: 'Option {verify, verify_peer} and cacertfile/cacerts is missing'

[warning] Description: 'Authenticity is not established by certificate path validation'
     Reason: 'Option {verify, verify_peer} and cacertfile/cacerts is missing'

[warning] Description: 'Authenticity is not established by certificate path validation'
     Reason: 'Option {verify, verify_peer} and cacertfile/cacerts is missing'

[error] Mongo.MongoDBConnection (#PID<0.777.0>) disconnected: ** (Mongo.Error) cluster0-shard-00-00.8vwd1.mongodb.net:27017 tcp recv: unknown POSIX error - :closed
[error] Mongo.MongoDBConnection (#PID<0.774.0>) disconnected: ** (Mongo.Error) cluster0-shard-00-00.8vwd1.mongodb.net:27017 tcp recv: unknown POSIX error - :closed
[error] Mongo.MongoDBConnection (#PID<0.780.0>) disconnected: ** (Mongo.Error) cluster0-shard-00-00.8vwd1.mongodb.net:27017 tcp recv: unknown POSIX error - :closed
[warning] Description: 'Authenticity is not established by certificate path validation'
     Reason: 'Option {verify, verify_peer} and cacertfile/cacerts is missing'

[warning] Description: 'Authenticity is not established by certificate path validation'
     Reason: 'Option {verify, verify_peer} and cacertfile/cacerts is missing'

Error is showing up again
its connecting to my MongoDB collection operations are successful but after few mins its starts throwing this error
Will share a video

https://drive.google.com/file/d/1SKWIoYC0xaH7E4U4YZ15mTGJZ6GxUjlh/view?usp=sharing

As soon as i hit the API it starts to show the error

8

Thanks for the video. The error occurs when you start writing. I think, you need to increase the timeout, when writing to the database:

try this

Mongo.OrderedBulk.write(:mongo, "Mails", 1_000, timeout: 60 * 1_000 * 5)

for 5 minutes.

It is a good idea to turn on the logging:

config :mongodb_driver, log: true
9

Sure, Will test it and let you know, how it goes

10

after running the server with above change i see below errors

[error] GenServer #PID<0.5907.0> terminating
** (CaseClauseError) no case clause matching: {:ok, 25978972, {:op_msg, 2, [{:section, 0, {:payload, %{"$clusterTime" => %{"clusterTime" => #BSON.Timestamp<1662372686:8>, "signature" => %{"hash" => #BSON.Binary<1460ab0218194f03375c5bca5869fddb50bc412e>, "keyId" => 7091006257124868102}}, "connectionId" => 34966, "electionId" => #BSON.ObjectId<7fffffff0000000000000165>, "hosts" => ["cluster0-shard-00-00.8vwd1.mongodb.net:27017", "cluster0-shard-00-01.8vwd1.mongodb.net:27017", "cluster0-shard-00-02.8vwd1.mongodb.net:27017"], "ismaster" => false, "lastWrite" => %{"lastWriteDate" => ~U[2022-09-05 10:11:26.000Z], "majorityOpTime" => %{"t" => 357, "ts" => #BSON.Timestamp<1662372686:8>}, "majorityWriteDate" => ~U[2022-09-05 10:11:26.000Z], "opTime" => %{"t" => 357, "ts" => #BSON.Timestamp<1662372686:8>}}, "localTime" => ~U[2022-09-05 10:11:26.738Z], "logicalSessionTimeoutMinutes" => 30, "maxBsonObjectSize" => 16777216, "maxMessageSizeBytes" => 48000000, "maxWireVersion" => 13, "maxWriteBatchSize" => 100000, "me" => "cluster0-shard-00-02.8vwd1.mongodb.net:27017", "minWireVersion" => 0, "ok" => 1.0, "operationTime" => #BSON.Timestamp<1662372686:8>, "primary" => "cluster0-shard-00-02.8vwd1.mongodb.net:27017", "readOnly" => false, "secondary" => false, "setName" => "atlas-l5hgzq-shard-0", "setVersion" => 4, "tags" => %{"nodeType" => "ELECTABLE", "provider" => "AWS", "region" => "AP_SOUTH_1", "workloadType" => "OPERATIONAL"}, "topologyVersion" => %{"counter" => 7, "processId" => #BSON.ObjectId<63145642008789ae01d3d376>}}, nil}}]}, <<196, 3, 0, 0, 255, 108, 140, 1, 166, 107, 140, 1, 221, 7, 0, 0, 2, 0, 0, 0, 0, 175, 3, 0, 0, 3, 116, 111, 112, 111, 108, 111, 103, 121, 86, 101, 114, 115, 105, 111, 110, 0, 45, 0, 0, 0, ...>>}
    (mongodb_driver 0.9.1) lib/mongo_db_connection/utils.ex:120: Mongo.MongoDBConnection.Utils.recv_data/3
    (mongodb_driver 0.9.1) lib/mongo_db_connection/utils.ex:29: Mongo.MongoDBConnection.Utils.get_response/2
    (stdlib 3.17) timer.erl:166: :timer.tc/1
    (mongodb_driver 0.9.1) lib/mongo/mongo_db_connection.ex:274: Mongo.MongoDBConnection.execute_action/4
    (mongodb_driver 0.9.1) lib/mongo/mongo_db_connection.ex:252: Mongo.MongoDBConnection.handle_execute/4
    (db_connection 2.4.2) lib/db_connection/holder.ex:354: DBConnection.Holder.holder_apply/4
    (db_connection 2.4.2) lib/db_connection.ex:1364: DBConnection.run_execute/5
    (db_connection 2.4.2) lib/db_connection.ex:1459: DBConnection.run/6
    (db_connection 2.4.2) lib/db_connection.ex:652: DBConnection.execute/4
    (mongodb_driver 0.9.1) lib/mongo.ex:1527: Mongo.exec_more_to_come/2
    (stdlib 3.17) timer.erl:166: :timer.tc/1
    (mongodb_driver 0.9.1) lib/mongo/streaming_hello_monitor.ex:122: Mongo.StreamingHelloMonitor.get_server_description/1
    (mongodb_driver 0.9.1) lib/mongo/streaming_hello_monitor.ex:95: Mongo.StreamingHelloMonitor.update_server_description/1
    (mongodb_driver 0.9.1) lib/mongo/streaming_hello_monitor.ex:88: Mongo.StreamingHelloMonitor.handle_info/2
    (stdlib 3.17) gen_server.erl:695: :gen_server.try_dispatch/4
    (stdlib 3.17) gen_server.erl:771: :gen_server.handle_msg/6
    (stdlib 3.17) proc_lib.erl:226: :proc_lib.init_p_do_apply/3
Last message: []
State: {:ready, #Reference<0.2940303167.2156527617.196852>, %{delay: 0, idle: #Reference<0.2940303167.2157182977.160365>, idle_interval: 5000, interval: 1000, next: -576460671703, poll: #Reference<0.2940303167.2157182977.161208>, slow: false, target: 50}, {-576460673310017890, 0}}
[warning] Description: 'Authenticity is not established by certificate path validation'
     Reason: 'Option {verify, verify_peer} and cacertfile/cacerts is missing'

[warning] Description: 'Authenticity is not established by certificate path validation'
     Reason: 'Option {verify, verify_peer} and cacertfile/cacerts is missing'


[warning] Description: 'Authenticity is not established by certificate path validation'
     Reason: 'Option {verify, verify_peer} and cacertfile/cacerts is missing'

[warning] Description: 'Authenticity is not established by certificate path validation'
     Reason: 'Option {verify, verify_peer} and cacertfile/cacerts is missing'

[error] Mongo.MongoDBConnection (#PID<0.13428.0>) failed to connect: ** (Mongo.Error) cluster0-shard-00-02.8vwd1.mongodb.net:27017 tcp recv: unknown POSIX error - :closed
[warning] Description: 'Authenticity is not established by certificate path validation'
     Reason: 'Option {verify, verify_peer} and cacertfile/cacerts is missing'

[error] Mongo.MongoDBConnection (#PID<0.13428.0>) failed to connect: ** (Mongo.Error) cluster0-shard-00-02.8vwd1.mongodb.net:27017 tcp recv: unknown POSIX error - :closed
[warning] Description: 'Authenticity is not established by certificate path validation'
     Reason: 'Option {verify, verify_peer} and cacertfile/cacerts is missing'

[warning] Description: 'Authenticity is not established by certificate path validation'
     Reason: 'Option {verify, verify_peer} and cacertfile/cacerts is missing'

[warning] SSL closed
[error] Mongo.MongoDBConnection (#PID<0.13428.0>) failed to connect: ** (Mongo.Error) cluster0-shard-00-02.8vwd1.mongodb.net:27017 tcp recv: unknown POSIX error - :closed
[warning] Description: 'Authenticity is not established by certificate path validation'
     Reason: 'Option {verify, verify_peer} and cacertfile/cacerts is missing'

[error] Mongo.MongoDBConnection (#PID<0.13428.0>) failed to connect: ** (Mongo.Error) cluster0-shard-00-02.8vwd1.mongodb.net:27017 tcp recv: unknown POSIX error - :closed
[warning] Description: 'Authenticity is not established by certificate path validation'
     Reason: 'Option {verify, verify_peer} and cacertfile/cacerts is missing'

[error] Mongo.MongoDBConnection (#PID<0.13428.0>) failed to connect: ** (Mongo.Error) cluster0-shard-00-02.8vwd1.mongodb.net:27017 tcp recv: unknown POSIX error - :closed

What could be the issue here

11

Is the issue reproducible?

12

I think, the problem is solved. It’s a pity that no more feedback comes.

13

Hi @zookzook apologies for the delay was on leave for a week, Yeah the problem is solved.

Thank you very much. our production server is running fine since then.

1 Like
14

Thank you for your feedback.