gigalixir timeout on long requests

Phoenix Forum Questions / Help
, ,
1

so I am using openai api which has a very long timeout of 2 minutes it works fine in my dev enviorment but in prod I see it breaking and I am not sure why

this is the relevent logic in my code

defp request(body, _opts) do
	  IO.puts("requesting")
	  
	  response = 
	    Finch.build(:post, "https://api.openai.com/v1/chat/completions", headers(), body)
	    |> Finch.request(AiAssistant.Finch, timeout: 120_000) #long wait time are expected source https://community.openai.com/t/slow-response-time-with-gpt-4/107104
	  
	  IO.inspect(response, label: "Response")
2

Can you show us what you mean? What error do you get?

3 
2023-10-10T19:10:06.111744+00:00 ai-secretary-alpha[b'ai-secretary-alpha-58cccd74cc-rkstv']: web.1  | requesting
2023-10-10T19:10:06.111743+00:00 ai-secretary-alpha[b'ai-secretary-alpha-58cccd74cc-rkstv']: web.1  | }
2023-10-10T19:10:21.174598+00:00 ai-secretary-alpha[b'ai-secretary-alpha-58cccd74cc-rkstv']: web.1  | Error: {"Other Error", %Mint.TransportError{reason: :timeout}}

now from the time stamps u can see it waited no where near 2 minutes but I set the timeout to be 2 minutes so lke wtf?

i told the gigalixir team aparently they are looking into it.

4

Hi @nevakrien a couple things:

  1. Please use code formatting blocks ``` for code and errors, it helps with readability. If you aren’t sure about how that works I have edited your post to use them so just click “edit” and see how it looks.

  2. Punctuation also helps with communication.

If you look at the docs on the function you are calling Finch — Finch v0.16.0 :timeout isn’t an option, rather there :pool_timeout and :receive_timeout. You probably want :receive_timeout.

1 Like
5

yes that ended up being the issue.
weirdly enough playing with just timeout on my local environment seems to have worked so it took me forever to find it.

idk why it appeared that way but in my local environment I could get requests that wait for a minute + which is way higher than the deafualt.

If i had to guess its a different Finch version in both which made this act out