I’m trying to track down a weird behavior from req_athena when running queries that take longer than 10 seconds. This is all running in Livebook.

What happens is the req_athena query/2 call runs for 10 seconds, then the DataFrame that’s returned gets rendered by kino_explorer as “Output data no longer available, please reevaluate this cell”. If you wait a few more seconds (depending on the query), the data appears and everything is fine. Everyone’s first reaction is to reevaluate the cell as directed. That obviously doesn’t help, so I’m trying to figure out if I can either make the query call wait long enough to avoid the message or change the message.

I tried calling Explorer.DataFrame.collect/1 on the results. This helps, but I think it helps mostly by delaying a little longer. The “Output data no longer available” error still shows momentarily.

Just in case we’re doing it wrong, here’s how we’re calling req_athena in the cell:

query = "<long SQL query>"
query_hash = q |> :erlang.md5() |> Base.encode16()

opts = [format: :explorer, output_location: "#{bucket}/livebook-#{query_hash}", ...]
response = ReqAthena.new(opts) |> ReqAthena.query!(q)
response.body 

Any ideas or suggestions would be most appreciated.