when i pull the html data from the url, I want to use check_data_exist to check to see if an identical result has already been persisted in the the database, and if so, stop the pipe, and be done.
This wouldn’t be an error situation, just no more processing needed.
Is there a way to just jump out of the pipe on some return of the function?
You might use with statement to manage error condition. it is Elixir’s monadic way to check for error.
Something like this…
with html_data <- get_html_data(url),
data_exist <- check_data_exist(html_data),
data_processed <- process_html_data(data_exist),
data_list <- parse_data_list(data_processed) do
# do something useful with data_list
else
_ -> # Oh, there was an error
end
The solution will be context dependent. Based on your problem description I would probably simply break the pipeline into two separate parts
data =
url
|> get_html_data()
|> check_data_exist()
case data do
{:ok, fetched_data} ->
fetched_data
{:error, :not_fetched} ->
data
|> process_html_data()
|> parse_data_list()
end
But it’ll depend on your specific scenario (which is why the video and associated blog posts are worth viewing).
OH! i like this… i should note that is not really an error case. The idea is, if i fetch the data, and it matches data that i have already fetched, then i don’t need to process the data… so i might return something like:
