How do I replace specific values in an Explorer DataFrame?

Elixir Framework Forums Nx Forum
,
1

I loaded this dataset which contains a couple nil values which I want to replace. For example, the game “Hypnosis Card” misses the “developers” value.

How do I go about filling in these values? Ideally I want to do this based on the game’s name instead of assuming the rows are in some kind of order.

2

There are a couple options depending on what you want to accomplish specifically.

require Explorer.DataFrame, as: DF

df = DF.new(
  name: ["WWE 2K24", "Hypnosis Card"],
  developers: ["Visual Concepts", nil]
)

If you want to fill missing values with a static replacement, you can use fill_missing:

iex(13)> DF.mutate(df, developers: fill_missing(developers, "blah"))
# #Explorer.DataFrame<
#   Polars[2 x 2]
#   name string ["WWE 2K24", "Hypnosis Card"]
#   developers string ["Visual Concepts", "blah"]
# >

If you want to replace nil with the value in a different column, there’s select:

DF.mutate(df, developers: select(is_nil(developers), name, developers))
# #Explorer.DataFrame<
#   Polars[2 x 2]
#   name string ["WWE 2K24", "Hypnosis Card"]
#   developers string ["Visual Concepts", "Hypnosis Card"]
# >

select is also compatible with a transformation on names:

DF.mutate(df, developers: select(is_nil(developers), upcase(name), developers))
# #Explorer.DataFrame<
#   Polars[2 x 2]
#   name string ["WWE 2K24", "Hypnosis Card"]
#   developers string ["Visual Concepts", "HYPNOSIS CARD"]
# >

And if you want a more complicated relationship between multiple columns, you can use conditionals.

If you have a more specific goal I can say more.

1 Like
3

select is what I was looking for, thanks!

rows = [
  %{id: 1, firstname: "John", lastname: "Doe"},
  %{id: 2, firstname: "Jane", lastname: nil},
  %{id: 3, firstname: "Alice", lastname: nil}
]

df = DF.new(rows)

DF.mutate(df, lastname: select(firstname == "Jane", "Example", lastname))

#Explorer.DataFrame<
  Polars[3 x 3]
  firstname string ["John", "Jane", "Alice"]
  id s64 [1, 2, 3]
  lastname string ["Doe", "Example", nil]
>
1 Like