Removing empty values in a nested map

I have a nested map as such:

%{product: %{category: "45", code: nil}, date: "2021-04-01"}

I am trying to remove all items that do have an empty value.
I have tried it as follows:

def removeEmpty(%{} = map) do
    Enum.into(
      Enum.reject(map, fn {_k, v} ->
        if is_map(v) do
          removeEmpty(v)
        end

        is_nil(v)
      end),
      %{}
    )
  end

Alas my unit test fails because the result still has the empty value in it:

Assertion with == failed
     code:  assert result == %{product: %{category: "45"}, date: "2021-04-01"}
     left:  %{date: "2021-04-01", product: %{category: "45", code: nil}}
     right: %{date: "2021-04-01", product: %{category: "45"}}
     stacktrace:
       test/map_helper_test.exs:23: (test)

It works for single level maps, but I need help on making this work for multi-level maps.