dogweather

dogweather

Elixir style vs. "Parse don't validate" (from Haskell)

EDIT: Parse, don't validate (2019) | Hacker News

I finished this working code which takes in an HTML page and outputs legal citations it finds. I like the code, and I think it’s pretty much Elixir style. E.g., it uses the shape of the data when it can. But I realized that it has an ordering dependency in the logic. In bigger projects, this can lead to bugs:

  @spec find_citations(binary()) :: list()
  @doc """
  Find citations in a string of HTML.
  """
  def find_citations(html) do
    {:ok, document} = Floki.parse_document(html)

    leginfo_urls =
      document
      |> Floki.attribute("a", "href")
      |> List.flatten()
      |> Enum.map(&URI.parse/1)
      |> Enum.filter(&leginfo_url?/1)

    leginfo_urls
    |> Enum.map(&leginfo_url_to_cite/1)
    |> Enum.sort()
    |> Enum.uniq()
  end


  defp leginfo_url?(%{host: "leginfo.legislature.ca.gov"}), do: true
  defp leginfo_url?(_), do: false

  defp leginfo_url_to_cite(%{query: query}) do
    query
    |> URI.decode_query()
    |> make_cite()
  end

  defp make_cite(%{"lawCode" => code, "sectionNum" => section}) do
    "CA #{@code_abbrevs[@cal_codes[code]]} Section #{section}"
    |> String.replace_suffix(".", "")
  end

I.e., my leginfo_url?() predicate validates the data and just returns a boolean. And so, the code has to be written correctly so that it’s called before leginfo_url_to_cite().

I think that the “parse don’t validate” idea is meant to remedy this. Instead of returning a boolean, one would return a type that can only be obtained by a valid parse. This way, instead of the programmer remembering to check for implicit dependencies, we enable the compiler to do it for us.

Does anybody here use that approach with Elixir? I suppose that with the above code, that’d mean creating a struct with statically defined keys :law_code and :section_num. And then the function heads would be written to only accept the named struct.

For this small code—which also has complete test coverage—I’m not sure if it’s worth the work. But in larger codebases, maybe it’d make sense. ?

Most Liked

jhogberg

jhogberg

Erlang Core Team

While types are very useful, it’s still a good idea to do this in untyped (“dynamically typed”) languages like Elixir and Erlang. Transforming external data to a known structure that your application understands and refusing to operate on anything but that eliminates lots of possible issues on its own, and is frankly less work in the long run than validating and operating on more-or-less raw data. Don’t let the absence of types stop you from implementing a good idea. :slightly_smiling_face:

You don’t have to go so far as to declare a whole new struct every time, either, as you get many of the benefits just from consistently combining functions like leginfo_url? and leginfo_url_to_cite into a function that returns {:error, reason} | {:ok, {:ad_hoc_tag, value}}. For small things that can be good enough.

11
Post #5
ken-kost

ken-kost

  def find_citations(html) do
    html
    |> Floki.parse_document!(html)
    |> Floki.attribute("a", "href")
    |> List.flatten()
    |> Enum.map(&URI.parse/1)
    |> Enum.reduce([], &parse_valid/2)
    |> Enum.sort()
    |> Enum.uniq()
  end

  def parse_valid(uri, valids) do
    with %{host: "leginfo.legislature.ca.gov", query: query} <- uri,
         %{"lawCode" => code, "sectionNum" => section} <- URI.decode_query(query) do
      [String.replace_suffix("CA #{@code_abbrevs[@cal_codes[code]]} Section #{section}", ".", "") | valids]
    else
      _ -> valids
    end
  end

Is this something you had in mind (not tested)? In the above code nothing happens in else clause of with, the accumulator is just passed, but you could add here different pattern matches and maybe also parse the data and add it to the accumulator.

Also, since there’s a lot of enum calls consider using stream. :cowboy_hat_face:

Maybe I don’t understand what is parse don’t validate from haskell. :confused:

Adzz

Adzz

There are lots of data validation / casting libraries in Elixir that help with this sort of thing in different use cases. I wrote one myself GitHub - Adzz/data_schema: Declarative schemas for data transformations. · GitHub

but as mentioned it doesn’t mean everything has to be a non primitive type, it just means you should do type casting at the edge of the system and be confident that if you get further in that that type casting happened.

Where Next?

Popular in Discussions Top

matthias_toepp
I’d love to hear what people think about Wisp, the new Gleam web framework started by Gleam’s primary creator Louis Pilfold. Gleam, alon...
New
PragTob
Hello everyone, I know we had quite some threads (read through lots of them) about background job processing but it remains a hotly deba...
New
Donovan
Hello everyone, I’m so glad to have discovered this awesome community. Thanks for creating it! This is my second post, and apologies for...
New
MarioFlach
Hello, I want to share a project I’ve been working on for a while: https://github.com/almightycouch/gitgud Background Some time ago I ...
New
axelson
Decided against including more info in the title, but the gist is that Plataformatec sponsored projects will continue with the assets bei...
New
Fl4m3Ph03n1x
Background This question comes mainly from my ignorance. Today is Black Friday, one of my favorite days of the year to buy books. One boo...
New
sashaafm
Piggy backing a bit on @dvcrn topic BEAM optimization for functions with static return type?, I’ve been trying to understand in a deeper ...
New
fireproofsocks
This is more of a general question, but I’m wondering how other people in the community think about the pattern matching in function sign...
New
hazardfn
I suppose this question is effectively hackney vs. ibrowse but we are at a point in our project where we have to make a choice between th...
New
opsb
We’re considering our architecture from a viewpoint of scaling our traffic heavily over the next 6 months. Our current deployment is runn...
New

Other popular topics Top

sen
Hi All, I set a environment variables in dev.exs , like below code. when i start server, how can i set the ${enable} value? thanks. d...
New
sorentwo
Hello! tl;dr Announcing Oban, an Ecto based job processing library with a focus on reliability and historical observability. After spen...
985 42920 311
New
AstonJ
Posting this to see if we can make things easier for people to get into Neovim. If you use Neovim and have a favourite distro please let ...
New
greenz1
I have a phoenix application from which a user can download multiple(5-6) files of size 1MB. I couldn’t find anything related to sending ...
New
JorisKok
I have a server on AWS, and was running a load test using artillery. When looking at the Phoenix dashboard I see the Ports going to 100% ...
New
vonH
When I run the Plug and I recompile I wind up having to use Ctrl C to quit iex and start again. Witht the help of rlwrap I can use the cu...
New
jason.o
In the code below, if the create action is not set to accept “extra_key” as an input, it errors out with a message shown above. Is there ...
New
PeterCarter
There are pre-rolled solutions for other frameworks that do work. However, Phoenix does not seem to have these. Have people had good expe...
New
AstonJ
Seen any cool LiveView demos, sample apps or examples? Please post them here! :003:
New
hariharasudhan94
Lets say I have map like this fetching from my database %{"_id" =&gt; #BSON.ObjectId&lt;58eb1a7a9ad169198c3dXXXX&gt;, "email" =&gt; ...
New

We're in Beta

About us Mission Statement