sehHeiden

sehHeiden

Datetime in DataFrames

While being at it.

Not clear about the Date(time) in Explorer anyway. I wrote it as utc_datetime in the SQLite with Ecto. Than in the DB the date is a text (why). Explorer does read it as text and does not infer it from the format (which is okay, because the db should have the correct type in the first place.

The strings has the format of 2023-08-29T15:38:00 I convert the Series of strings to a list of DateTime with

dt = S.to_list(df["date"])
|> Enum.map(&elem(DateTime.from_iso8601(&1 <>"+00:00"),1))

a) Why couldn’t I not write with DateTime.from_iso8601(&1, :basic, "ETC/UTC")
b) Why cann’t I convert the list to a Series? Error: unsupported datatype: ~U[2023-08-29 15:38:00Z]
c) Why cannot I write it into the DataFrame with put? Sane error as in b)

Minimum example for a)

test_dates = S.from_list(["2023-08-29T15:38:00", "2023-08-29T14:17:28", "2023-08-29T12:53:17",])
dt0 = Enum.at(S.to_list(test_dates), 0)
DateTime.from_iso8601(dt0, :basic, "ETC/UTC")

Throws the error:

FunctionClauseError) no function clause matching in Calendar.ISO.parse_utc_datetime/2    
    
    The following arguments were given to Calendar.ISO.parse_utc_datetime/2:    
        # 1
        "2023-08-29T15:38:00"
    
        # 2
        "ETC/UTC"
    
    Attempted function clauses (showing 1 out of 1):
    
        def parse_utc_datetime(string, format) when is_binary(string) and format === :basic or format === :extended

Which is strange because I added the format as :basic …

Edit: In the end I only want to be able to filter the DataFrame by a minimum date and/or a maximum date.

Marked As Solved

billylanchantin

billylanchantin

Explorer only accepts NaiveDateTime. The following should work:

datetime_series =
  ["2023-08-29T15:38:00", "2023-08-29T14:17:28", "2023-08-29T12:53:17"]
  |> Enum.map(&NaiveDateTime.from_iso8601!/1)
  |> S.from_list()

# #Explorer.Series<
#   Polars[3]
#   datetime[μs] [2023-08-29 15:38:00.000000, 2023-08-29 14:17:28.000000,
#    2023-08-29 12:53:17.000000]
# >

Then you should be able to compute the min/max no problem:

S.max(datetime_series)
# ~N[2023-08-29 15:38:00.000000]

S.min(datetime_series)
# ~N[2023-08-29 12:53:17.000000]

Also Liked

al2o3cr

al2o3cr

SQLite doesn’t support a specific “date” type - only null / integer / real / text / blob

That’s not the arguments that from_iso8601 is documented to accept; why would you expect that to work? The second argument should be a calendar module and the third should be either :basic or :extended.

To your original question, the Series.strptime function seems like exactly what you’re looking for - takes a series of string values and parses them into a datetime series.

Where Next?

Popular in Questions Top

Harrisonl
We have an ECS cluster with 4 services, where each task joins a single cluster, via discovery ECS discovery service. Currently when I de...
New
marius95
Hello everyone, I try to use an Javascript Event Handler in my root.html.leex file. Therefore I created a function in the app.js file: ...
New
aadeshere1
I have a another noob question about loop. Since elixir is immutable, while loop is not directly possible. total = 10 while total != 0 ...
New
greenz1
I have a phoenix application from which a user can download multiple(5-6) files of size 1MB. I couldn’t find anything related to sending ...
New
LegitStack
I’m trying to make a websocket server in Phoenix or raw Elixir. I heard about gun, I think I could use cowboy, but since I’m not that sma...
New
stefanchrobot
What’s the safe way to decode a JSON string into a struct? I want to avoid calling String.to_atom. Jason.decode can give me a map with st...
New
itssasanka
Hi all, Trying to get some more clarity over utc_datetime and naive_datetime for Ecto: The documentation above suggests that while ...
New
Qqwy
Original source of discussion: This topic on the Pragmatic Programmers’ Functional Web Development with Elixir, OTP, and Phoenix forum. ...
New
vonH
When I run the Plug and I recompile I wind up having to use Ctrl C to quit iex and start again. Witht the help of rlwrap I can use the cu...
New
hariharasudhan94
Lets say I have map like this fetching from my database %{"_id" =&gt; #BSON.ObjectId&lt;58eb1a7a9ad169198c3dXXXX&gt;, "email" =&gt; ...
New

Other popular topics Top

AstonJ
Posting this to see if we can make things easier for people to get into Neovim. If you use Neovim and have a favourite distro please let ...
New
Patoshizzle
After calling mix ecto.create I get this error: 17:00:32.162 [error] GenServer #PID&lt;0.412.0&gt; terminating ** (Postgrex.Error) FATAL...
New
gausby
I asked this very same question on twitter and got some interesting feedback, but I thought it would be a good question to ask here as we...
1207 39523 209
New
grych
Hi folks, Few months ago I have announced the proof-of-concept of the library to manipulate the browsers DOM objects directly from Elixi...
639 52774 488
New
RisingFromAshes
I’ve read in another post that it may be possible with a router helper - but I couldn’t find an appropriate one, and tbh, I’m still just ...
New
sergio_101
I am VERY much an elixir newbie. I have taken one elixir course and one phoenix course on Udemy. During that course, I saw the instructor...
New
Brian
What is the proper way to load a module from a file in to IEX? In the python world, doing something like this pretty standard: from ....
New
JorisKok
I have a server on AWS, and was running a load test using artillery. When looking at the Phoenix dashboard I see the Ports going to 100% ...
New
minhajuddin
I have seen a lot of code which picks the first element from a list using Enum.at(0) instead of List.first. Is there a reason why people ...
New
sergio
Kind of like when jquery came out, it was super necessary. Existing drag and drop libraries have a bunch of baggage to support old browse...
New

We're in Beta

About us Mission Statement