Missing functions in Explorer DataFrames/Series

sehHeiden

Compared to Pythons pandas I am missing some functions for DataFrames and Series from Explorer, or I have to apply a workaround, which makes everything less expressive.

DataFrames have a drop_nil function. Why don’t have Series have it? Well, I can convert it to list, drop them from the list and then convert back…

By the documentation the series have a contains function, that use a regex, could not get with working with a compiled regex string. I did a mapping instead:

filter = df["content"]
|> S.to_list()
|> Enum.map(&Regex.match?(tags,&1))
|> S.from_list()

This returns a series of boolean. But I can not use it filter a DataFrame or a Series? At least I did not find it in the reference and also no way around. It neighter a filter, filter_with nor a slice. Slice needs indices. How do I get the indices, where the value is :true?

The last question is, about Date(time) columns. How can I convert a Date to a calendar week?

Update:

post_filter = df["content"]
|> S.downcase()
|> S.to_list()
|> Enum.map(&Regex.match?(tags,&1))
|> Enum.with_index()
|> Enum.filter(&(elem(&1, 0) == true))
|> Enum.map(&(elem(&1, 1)))

Can be used as input for the slicing…

13 comments

/nx #explorer

1 1783 13

2023-10-06 18:21:02 UTC

First Post!

Show Best Posts
Show All (oldest first)
Show All (newest first)

billylanchantin

I believe you want Explorer.Series.mask/2.

content = S.from_list(["aa", "ab", "bc"])
# #Explorer.Series<
#   Polars[3]
#   string ["aa", "ab", "bc"]
# >
mask = S.transform(content, &Regex.match?(~r/a.+/, &1))
# #Explorer.Series<
#   Polars[3]
#   boolean [true, true, false]
# >
S.mask(content, mask)
# #Explorer.Series<
#   Polars[2]
#   string ["aa", "ab"]
# >

Post #1

Most Liked

billylanchantin

Hi @sehHeiden,

The function is now on main (José reviews PRs quick!):

Add `day_of_year` and `week_of_year` by billylanchantin · Pull Request #717 · elixir-explorer/explorer · GitHub

So if you get the latest ref, you can now do:

[~D[2023-01-01], ~D[2023-01-08], ~D[2023-02-23]]
|> S.from_list()
|> S.week_of_year()
# #Explorer.Series<
#   Polars[3]
#   integer [52, 1, 8]
# >

Since you’re new to Elixir, you can get the latest ref by adding the following to your project’s mix.exs:

  # list of deps
  [
    # ...
    {:explorer,
     git: "https://github.com/elixir-explorer/explorer.git",
     # This is the current ref of main.
     # It may change, but as long as the ref is after this one,
     # you'll have access to the function.
     ref: "aef274989ab490b0a392ccd19ec24b286a8cda1c",
     override: true},
    # ...
  ]

There are other ways of doing this too, but that should get you started.

Post #8

josevalim

Creator of Elixir

We will also gladly accept PRs that add missing functions. We know that we are missing many of them.

Post #3

sehHeiden

Sorry for answering late. The messages from the elixir-forum always end up in my spam filter. About the PR… a) a I very new to Elixir not so bold to mess up some core libraries . Also barely have to time.