Improving my code using idiomatic elixir and pipelines

feviskus

Hello there,
I am currently working on my first elixir-project which has a bit more business-logic and is not only CRUD.
I want to apply OTP/concurrency in a useful way, and a book-search aggregator seems to be a good use-case.
Currently I have a prototype implementation for a book-store which I scrape to extract the results (btw, Floki is surprisingly fast! At least it feels faster than python).
The pipe looks like this:

 @behaviour Book4Less.Searches.SearchBookBehaviour

  def search_books(query = %Query{} \\ %Query{title: "Elixir"}) do
    query
    |> build_url() # returns a string
    |> execute_request() # returns a HTTPoison.request
    |> validate_response!() # returns a html-string
    |> extract_books() # returns a list of %Book{}
  end

Questions

These functions are temporally-coupled, e.g. build_url/1 and execute_request/1 - because I need to set the url before I send a HTTP-request…
How to express this coupling best? Just making execute_request/1 call build_url/1?
So this pipeline deals with string, HTTP, HTML and the domain-object Book - all levels of abstractions & different topics. Would u unify the return values to e.g. a %Search{} struct?

4 comments

#code-review #pipeline #decoupling

0 1756 4

2018-09-16 23:46:51 UTC

Showing Posts 1 to 4

Show Best Posts
Show All (oldest first)
Show All (newest first)

kip

ex_cldr Core Team

I think this would be nicely expressed in a with clause since you are subject to errors at each stage of the pipeline and therefore would want to exit the pipeline if an error is detected. So as an example:

def search_books(query = %Query{} \\ %Query{title: "Elixir"}) do
  with {:ok, url} <- build_url(query),
       {:ok, response} <- execute_request(url),
       {:ok, response} <- validate_response(response) do
    extract_books(response)
  end
end

Noting that if at any stage the patterns do not match, the with clause will exit with the non-matching return.

With this approach I would likely delegate validate_response/1 to be called within execute_request/1 since with is best used to describe the happy path. So this time with explicit error management:

def search_books(query = %Query{} \\ %Query{title: "Elixir"}) do
  with {:ok, url} <- build_url(query),
       {:ok, response} <- execute_request(url) do
    extract_books(response)
  else
    {:error, reason} -> raise "whoops, we got error #{inspect reason}"
    other -> raise "unexpected error return #{inspect other}"
  end
end

Then for execute_request/1:

def execute_request(url) do
  with {:ok, response} -> http_call(url),
       {:ok, valid_response} -> validate_response(response) do
    {:ok, valid_response}
  end
end

Post #1

feviskus

Thanks to your answer I gained some knowledge about the concept of happy pathand error-handling!
One thing:
IMHO the code using the with statement is not really as expressive/pretty as a normal pipeline. Do you know some approaches which handle that?

Post #2

kip

ex_cldr Core Team

At some point your expressive pipeline will have to deal with errors since you have functions with side affects (like external requests) so I’m not sure its the right measure of success.

However perhaps the OK library by @Crowdhailer would reflect your tastes more?

Post #3