snofang

snofang

Ecto.CastError Mixed Keys Issue

In a typical business development task, having a function in a context module which accepts attributes of map type and passes them to Ecto for a validated Changeset is a common case.

  def create_foo(%{} = attrs) do
    attrs
    |> Foo.create_changeset()
    |> Repo.insert()
  end

Those functions may be called from everywhere; Normally, it is quite common to express the keys by atom while calls from Phoenix Controllers and LiveViews have keys in string. So these business functions should support both types of keys and thanks to the cast method, they do by default.

But, what if by some requirement it is needed to change the passed in attributes before passing them to Ecto for changeset or validations? By which data type the attribute keys should be addressed? string or atom? Suppose the following function:

  def create_bar_foo(%{} = attrs) do
    attrs
    |> Map.put(bar_case: :default_or_computed_value)
    |> Foo.create_changeset()
    |> Repo.insert()
  end

If it called from a test with normal atom keys, every thing would be fine while if it called from a LiveView with string keys, there will be a raise:

(Ecto.CastError) expected params to be a map with atoms or string keys, got a map with mixed keys ...

So the keys should be normalized before processing and this article suggest some solution. Also this post addresses it somehow.

There can be lots of other solutions to this problem, for example one can simply cast passed attributes before manipulation them and so on. But I think this can be included in Ecto.Changeset.cast method because:

  • Ecto.Changeset.cast is already supporting both atom and binary keys and also touching the case by doing conversion from string keys to atom ones. Also it sounds feasible to support mix of both types.
  • There is no harm to existing codes as actually a new capability is being added.
  • It encourages having some business implemented in context before using Ecto directly or having lots of ###_changeset functions per specific business case.

Most Liked

tfwright

tfwright

I think a more specific example use case might me useful here because I do think it is quite rare, but otherwise I strongly agree with other commenters that modifying params is an antipattern. In fact I would go as far as to say even if it was supported I wouldn’t make use of it. We as developers are naturally lazy and that is generally speaking a good thing as it motivates us to avoid producing spaghetti code. But here the extra code actually simplifies the design because it represents the innate complexity of the data insofar as it implicitly must deal with multiple input sources (as José says if the param in question is system data there is no reason to cast it). When troubleshooting/debugging or even just grokking part of a program it is very useful to be able to clearly and easily trace the path of a piece of data, and if necessary, modify it in isolation. Like many things it might seem a simple matter to recognize the “mixing” these things tend to multiply overtime and in aggregate produce more fragile, less maintainable code.

A computed/calculated value wouldn’t necessarily call for an extra changeset, but as shown in examples above it has its own logic path which starts from the changeset, not the changeset params.

sodapopcan

sodapopcan

I respectfully disagree with this whole proposal.

You semi-handwaved this solution but for me this is the right answer and one of those things I’m pretty religious about in my own code. We should be converting—ie casting—our data into a known shape before doing anything with it, and this is exactly what Changeset.cast provides us. This is especially important in a dynamic language.

For me, string keys mean “untrusted”. Using this definition, general application code should rarely ever have a need to set a string key (of course there are always exceptions).

Multiple changesets are generally encouraged by the framework. “Citation needed,” yes sorry I don’t have doc or discussion links atm, the best I can give right now is to look at the User schema generated by phx_gen_auth. I actually do prefer to keep a single changeset myself when possible (it’s not a hard rule) but it results in me writing functions like maybe_assign_slug/1 which are probably more complex than they need to be if I’d just use multiple changesets.

If you really want to do these things in the context, there is no harm in manipulating changesets in the context—changesets are kind of wild as they are having a strong presence in the web layer as well as the business layer.

Of course, you could also do stuff like this in the context:

def create_article(attrs) do
  %Article{}
  |> Article.changeset(attrs)
  |> MyApp.ChangesetHelpers.assign_slug_from(:title) # module name for illustrative purposes :D
  |> Repo.insert()
end

Finally, I think any promotion of the idea that mixing map key types is ok is a bad idea, again, especially in a dynamic language.

josevalim

josevalim

Creator of Elixir

It all depends on what is the source of the data.

If the source of the data is your own application, validations are pointless, because it makes no sense to tell the user that “bar_case is invalid” when they have no power over setting :bar_case. My reply was written from that perspective (I will clarify this in my previous message to avoid future confusion).

However, if you are manipulating other external params to compute values (and now reading back on your original thread, you said that this is indeed the case), then I agree with your concerns. In such cases, I would try to cast the initial params, then compute additional changes, and then validate the additional changes:

data
|> cast(params, ~w(foo baz))
|> validate_foo_and_baz()
|> compute_bar_case_as_change()
|> validate_bar()

But even this requires care. If you add a validation error to bar, and it is a computed value from foo and baz, you need to make sure the error points to the correct place.

Last Post!

snofang

snofang

Yup! It sounds like my view came from considering more than needed dependency for each layer (probably from other ecosystems), to such an extent that there wasn’t any differentiation between internal and external data. And now it is getting clear to me how it sounds overly defensive.

Thank you all

But I still believe that the proposal itself and it’s workaround solution (the following) is true.

p.s. by all_validations I don’t mean necessarily “all validations in one place”, but “all necessary validations”.

Where Next?

Popular in Proposals: Ideas Top

alaadahmed
Hi folks, I tried Phoenix 1.7 and it is awesome, but I have small suggestion, which in my opinion will organize files in a better way. ...
New
rmoorman
Current situation Currently, the structure of the HTML returned by phoenix is determined by the layouts (components/layouts/[root,app].ht...
New
PJUllrich
Hey folks, I have the unique problem that I need to ignore all “change” and “input” events for one specific input element in a LiveView F...
New
jakeprem
Goal: To make JS.patch and JS.navigate more interoperable with JS.push. Scenario: Imagine making a reusable Phoenix component and you wa...
New
GenericJam
I’ve been messing around with image generation models recently and thought this could be wrapped in a library or people could just use th...
New
woylie
We are seeing a lot of warning logs like this: navigate event to "https://someurl" failed because you are redirecting across live_sessio...
New
eagle-head
Hi everyone, I’ve been researching Content Security Policy Level 3 support in Phoenix and wanted to share my findings and a proposal for...
New

Other popular topics Top

Darmani72
If I have a post route which an argument: post /my_post_route/:my_param1, MyController.my_post_handler How would get the post params ...
New
joeerl
Hello again - after a longish gap I’ve decided I really must dig into Elixir and see what’s been happening here - so I have a few questio...
New
sen
Hi All, I set a environment variables in dev.exs , like below code. when i start server, how can i set the ${enable} value? thanks. d...
New
sergio_101
I am VERY much an elixir newbie. I have taken one elixir course and one phoenix course on Udemy. During that course, I saw the instructor...
New
siddhant3030
Hi, I have to write a raw query for one of my project. But till now I have used ecto queries and don’t have much experience writing raw ...
New
sergio
Kind of like when jquery came out, it was super necessary. Existing drag and drop libraries have a bunch of baggage to support old browse...
New

We're in Beta

About us Mission Statement