dogweather

dogweather

EDIT: Parse, don't validate (2019) | Hacker News

I finished this working code which takes in an HTML page and outputs legal citations it finds. I like the code, and I think it’s pretty much Elixir style. E.g., it uses the shape of the data when it can. But I realized that it has an ordering dependency in the logic. In bigger projects, this can lead to bugs:

  @spec find_citations(binary()) :: list()
  @doc """
  Find citations in a string of HTML.
  """
  def find_citations(html) do
    {:ok, document} = Floki.parse_document(html)

    leginfo_urls =
      document
      |> Floki.attribute("a", "href")
      |> List.flatten()
      |> Enum.map(&URI.parse/1)
      |> Enum.filter(&leginfo_url?/1)

    leginfo_urls
    |> Enum.map(&leginfo_url_to_cite/1)
    |> Enum.sort()
    |> Enum.uniq()
  end


  defp leginfo_url?(%{host: "leginfo.legislature.ca.gov"}), do: true
  defp leginfo_url?(_), do: false

  defp leginfo_url_to_cite(%{query: query}) do
    query
    |> URI.decode_query()
    |> make_cite()
  end

  defp make_cite(%{"lawCode" => code, "sectionNum" => section}) do
    "CA #{@code_abbrevs[@cal_codes[code]]} Section #{section}"
    |> String.replace_suffix(".", "")
  end

I.e., my leginfo_url?() predicate validates the data and just returns a boolean. And so, the code has to be written correctly so that it’s called before leginfo_url_to_cite().

I think that the “parse don’t validate” idea is meant to remedy this. Instead of returning a boolean, one would return a type that can only be obtained by a valid parse. This way, instead of the programmer remembering to check for implicit dependencies, we enable the compiler to do it for us.

Does anybody here use that approach with Elixir? I suppose that with the above code, that’d mean creating a struct with statically defined keys :law_code and :section_num. And then the function heads would be written to only accept the named struct.

For this small code—which also has complete test coverage—I’m not sure if it’s worth the work. But in larger codebases, maybe it’d make sense. ?

Showing Posts 1 to 5

ken-kost

ken-kost

  def find_citations(html) do
    html
    |> Floki.parse_document!(html)
    |> Floki.attribute("a", "href")
    |> List.flatten()
    |> Enum.map(&URI.parse/1)
    |> Enum.reduce([], &parse_valid/2)
    |> Enum.sort()
    |> Enum.uniq()
  end

  def parse_valid(uri, valids) do
    with %{host: "leginfo.legislature.ca.gov", query: query} <- uri,
         %{"lawCode" => code, "sectionNum" => section} <- URI.decode_query(query) do
      [String.replace_suffix("CA #{@code_abbrevs[@cal_codes[code]]} Section #{section}", ".", "") | valids]
    else
      _ -> valids
    end
  end

Is this something you had in mind (not tested)? In the above code nothing happens in else clause of with, the accumulator is just passed, but you could add here different pattern matches and maybe also parse the data and add it to the accumulator.

Also, since there’s a lot of enum calls consider using stream. :cowboy_hat_face:

Maybe I don’t understand what is parse don’t validate from haskell. :confused:

krasenyp

krasenyp

As @ken-kost said, you might want to use stream. Instead of List.flatten/1 you can use Stream.transform/3. Your final call can be Enum.sort/1 which will turn the stream into a list.

jhogberg

jhogberg

Erlang Core Team

While types are very useful, it’s still a good idea to do this in untyped (“dynamically typed”) languages like Elixir and Erlang. Transforming external data to a known structure that your application understands and refusing to operate on anything but that eliminates lots of possible issues on its own, and is frankly less work in the long run than validating and operating on more-or-less raw data. Don’t let the absence of types stop you from implementing a good idea. :slightly_smiling_face:

You don’t have to go so far as to declare a whole new struct every time, either, as you get many of the benefits just from consistently combining functions like leginfo_url? and leginfo_url_to_cite into a function that returns {:error, reason} | {:ok, {:ad_hoc_tag, value}}. For small things that can be good enough.

11
Post #4
Adzz

Adzz

There are lots of data validation / casting libraries in Elixir that help with this sort of thing in different use cases. I wrote one myself GitHub - Adzz/data_schema: Declarative schemas for data transformations. · GitHub

but as mentioned it doesn’t mean everything has to be a non primitive type, it just means you should do type casting at the edge of the system and be confident that if you get further in that that type casting happened.

— All posts loaded —

Where Next? Top

Trending in Discussions Top

AstonJ
As the title says, please share what you’ve been up to with Elixir. Whether that’s been learning it, looking into it, making stuff with i...
2977 94592 917
New
cblavier
Hey there, It’s been more than a year since we started using LiveView as our main UI library and building a whole library of UI componen...
New
mudasobwa
I am happy to introduce the very α version of the new programming language compiled to BEAM. Welcome Cure. It has literally three kille...
New
heathen
Quite interesting article Google brought me. Didn’t find any mentions about it here. What do you think in general? Would you use togethe...
New
mhanberg
Hi everyone! The first release candidate for the Expert language server project is now available! We’ve published a press release detai...
New
axelson
Hi there! :wave: @frigidcode and I (but mostly him) have been running an Elixir Book club, we’re almost done with Designing Elixir Syste...
New
budgie
A little off-topic, but I feel like people here have a good head on their shoulders. I used to be quite good at making software. Was luc...
New

Other Trending Topics Top

GenericJam
Edit: 2026 May 15 - This post is archived. Mob is alive!! Main docs: mob v0.7.11 — Documentation A bit of explanation for the slightly c...
New
JesseHerrick
Hey, I’m Jesse and I’m the main contributor behind Dexter, a full-featured, lightning-fast Elixir LSP optimized for large codebases. It s...
New
garrison
Hobbes is a low-level distributed database for the Elixir programming language. Hobbes provides a simple, safe, and scalable storage lay...
New
marciok
Hi there! We created Gust: A task orchestrator inspired by Airflow. For those who have never heard about Aiflow, it’s a Python-based wor...
New
georgeguimaraes
Just published claude-code-elixir, a plugin marketplace for Claude Code with Elixir support. These are the plugins I’ve been using for my...
New
Dmk
Xamal is a deployment tool for Elixir apps that deploys native releases to bare metal servers over SSH. It’s a port of GitHub - basecamp/...
New

We're in Beta

About us Mission Statement

Options

Thread Display Mode




Thread Preview

Skip Thread Previews