Hi,
Please i’m having an issue with being able to rebuilding the value of a variable. I don’t know if my approach is correct. Please kindly help
My code:
processed = 0
Enum.each params, fn attachment ->
processed = processed + 1
end
# this is never called
if processed > 0 do
IO.inspect processed
end
Please how do I achieve this or what approach do i use?
In elixir everything is immutable and the altered processed is scoped to the anonymous function. It will be set to 1 for each iteration and forgotten thereafter.
What you are searching for is a Enum.reduce/3 in this case or just Enum.length/1 or even Enum.empty?/1.
Since you will need it later on anyway, let me explain the Enum.reduce/3 a bit
Enum.reduce(enumerable, acc, fun)
This is the function head. So it takes the enumerable which you want to reduce into a single value. It takes an acc which is the initial value used when reducing. And last but not least, it takes a function of arity 2. The first argument to the funtion is the current element of the enumerable and the other one is the current accumulator. The return value of the function is used as the new accumulator for the next element.
In your example above you are trying to increment the value of a variable by 1 for each element, don’t caring for the value of the element itself:
Enum.reduce(params, fn _, acc ->
acc + 1
end)
The processed variable is out of scope. You are incrementing a
processed variable inside the fn attachment -> ... end code
You could use something like Enum.reduce instead.
processed = Enum.reduce(params, 0, fn (attachment, processed) -> processed
+ 1 end)
Thanks @NobbZ for your response.
Possibly i didn’t make my sample code very clear enough. There are processes to be run within Enum.each on each element and once it’s successful i want it to save count of success.
For instance i have 3 elements passed and two was successfully processed, then I want know the number of success during the iteration
See below my code:
processed = 0
Enum.each params, fn attachment ->
if attachment.data do
processed = . processed + 1
end
end
if processed > 0 do
IO.inspect processed
end
By processes, do you mean spawned processes, which shall increment some variable in even another process? This is impossible in the BEAM for a reason.
I’m in a hurry now, but on a quick guess, it seems as if you want to have a parallel map which returns a list of booleans, after that you want to know the count of true (or false).
No not spawn. I meant processing data passed.
This is the full code of what i have inside Enum.each
Enum.each Map.get(attachment_params, "attachment"), fn attachment ->
media_type =
case attachment.content_type do
"image/jpeg" -> "photos"
"image/png" -> "photos"
"image/gif" -> "photos"
"image/jpg" -> "photos"
"video/mp4" -> "videos"
_ -> nil
end
# Get the file size of the attached file
%{size: size} = File.stat! attachment.path
if media_type != nil && size < 25000000 do
extension = Path.extname(attachment.filename)
unqiue_identifier = :rand.uniform()
media_source_filename = "#{review.slug}-#{review.id}-#{unqiue_identifier}#{extension}"
upload_path = "uploads/reviews/source/#{media_type}/#{media_source_filename}"
# Thumbnail
media_thumbnail_filename = "#{review.slug}-#{review.id}-#{unqiue_identifier}.jpg"
thumbnail_path = "uploads/reviews/thumbnail/#{media_thumbnail_filename}"
File.cp(attachment.path, upload_path)
# Generate Thumbnail
if media_type == "videos" do
Ceecare.Uploader.Thumbnail.generate_thumbnail_from_video(upload_path, thumbnail_path, 640, 400)
else
Ceecare.Uploader.Thumbnail.generate_thumbnail_from_image(upload_path, thumbnail_path, 640, 400)
end
#Save the Data inside Media
media_changeset = Media.changeset(%Ceecare.Media{
review_id: review_id,
filename: media_source_filename,
thumbnail: media_thumbnail_filename,
media_type: String.upcase(media_type),
})
case Repo.insert(media_changeset) do
{:ok, media} ->
processed = processed + 1
{:error, media_changeset}
IO.inspect media_changeset.errors
end
end
end
I have tried this as below:
processed = 0
processed = Enum.reduce Map.get(attachment_params, "attachment"), fn (attachment, processed) ->
processed + 1
end
but i’m getting bad argument in arithmetic expression
Because you are not passing in the acc. Therefore the first item from the enumerable is used.
Also after you got this just move your current processing into a reduce, and return an updated acc or not updated acc depending on the fund funs outcome
Thanks NobbZ.
Please just two weeks old in my journey with elixir and I can’t clearly figure out how to go with the information in your reply.
Please can you do a skeleton code just to grasp your intention.
Thanks.
I’ll do when in office and through the good morning rituals.
OK, I flew through your code, and it seems as if you treat a successfull insert into the DB as a success. Therefore your reduction could look like this:
attachement_params
|> Map.get("attachement")
|> Enum.reduce(0, fn (attachement, processed) ->
media_type = # ...
if media_type do
# prepare changeset
case Repo.insert(media_changeset) do
{:ok, _} -> processed + 1
{:error, media_changeset} ->
Logger.debug(inspect media.changeset.errors
processed
end
else
processed # It is important to also return it when there was no valid mediatype!
end
end
I have also stripped away some of your logic and replaced it by stub-comment to keep the example short. I think you should get it anyway.
Another approach to this stuff in general were to use some Streams to transform all the and get you a result. As a very rough draft, it can look like this then:
attachement_params
|> Map.get("attachement")
|> Stream.map(fn attachement ->
Map.put(attachement, :media_type, case attachement.content_type do
"image/" <> _ -> :image
"video/" <> _ -> :video
_ -> nil
end)
end)
|> Stream.filter(&valid_media_type?/1)
|> Stream.map(&inject_filesize/1)
|> Stream.filter(&small_enough?/1)
|> Stream.map(...)
|> Stream.map(...)
... # all the stuff you did before, but one by one, always updating/injecting into the original map until you have a list of `Media.changeset`
|> Stream.map(&Repo.insert/1)
|> Enum.reduce(0, fn
{:ok, _}, acc -> acc + 1
{:error, cs}, acc ->
# Log the error
acc
end)
This is a very rough draft. Showing in the very first mapper the idea of keeping the map as is and just adding some field to it, just that you have a grasp of the idea. For the later steps in the pipes you need to implement them yourself.
Important is then the last Enum.reduce! It is necessary to actually run and process your stream and actually get a result from it.
Thanks immense for your time.
I like the approach with stream and I will read through the doc to understand more how it works.
I got an error while trying the first skeleton code
protocol Enumerable not implemented for nil
The minimal code
def upload_attachment(conn, %{"review_id" => review_id, "media" => attachment_params}) do
attachment_params
|> Map.get("attachement")
|> Enum.reduce(0, fn (attachement, processed) ->
processed + 1
end)
end
That probably means, thate there is no key "attachement" in your map. Perhaps a misspleing one some of the ends?
Thank you so much. I didn’t catch that. It was a misspelt.
I’m sincerely grateful for the support all the way.
May God grant you more wisdom and strength in Jesus name, Amen.
I appreciate you!
Thanks for posting the question and for the response. I found it directly applicable to the problem of updating a ProgressBar inside of a long running process loop. I was struggling with Enum.each but found this thread and your tip on Enum.reduce did the trick!
I still have a scoping question, though. In the sample code below it looks like variables total_work and format are visible within the reducer function and I guess that’s fine since I don’t need to change these variables in the loop, just access them. On the other hand, if I had declared processed above as well I would still be able to see it inside the iteration function (via Enum.each, perhaps), but I wouldn’t be able to change or rebind it. So ultimately, this is not a scope or visibility issue but a mutability concern instead. Do I understand this correctly?
Also, in the code below, would the variables total_work and format be considered “closures”? (That term always confuses me and I’d like to confirm). Thanks for the help!
def post_transactions() do
work_items = from s in StagedTransaction, where: [post_flag: true]
total_work = Repo.aggregate(work_items, :count, :id)
format = progress_bar_format()
Repo.all(work_items)
|> Enum.reduce(0, fn work_item, processed ->
if ok_to_post(work_item) do
post_transaction(work_item)
ProgressBar.render(processed + 1, total_work, format)
end
processed + 1
end)
end
That is correct. The value of those variables is accessible, but not mutable.
Not quite. The function you pass to Enum.reduce (fn work_item, processed ->) is a “closure” in that it “closes over” the variables in scope and carries it around with the function. This is more obvious if we break up how this is written:
def post_transactions() do
work_items = from s in StagedTransaction, where: [post_flag: true]
total_work = Repo.aggregate(work_items, :count, :id)
format = progress_bar_format()
reduce_function = fn work_item, processed ->
if ok_to_post(work_item) do
post_transaction(work_item)
ProgressBar.render(processed + 1, total_work, format)
end
processed + 1
end
do_work(work_items, reduce_function)
end
defp do_work(work_items, reduce_function) do
work_items
|> Repo.all()
|> Enum.reduce(0, reduce_function)
end
See how we can do |> Enum.reduce(0, reduce_function) and this will still work, even though the variables total_work are no longer “inside” the function do_work. The reduce_function captured those values when it was defined and so it can still access them.
Sorry to be dense but to crystalize the terminology for me, does this mean that the reduce_function is a “closure” and that the variables total_work and format referenced within it are considered “enclosed variables” (or some other term)? If so, then I suppose a “closure” must always be a function… yes?
Also, if we had only referenced total_work but not format in the reduce_function, would both still have been carried along implicitly? I suppose it doesn’t matter whether both are or not, since if one wasn’t referenced in the closure it would just be extra baggage that causes no harm, but I"m trying to understand the way this work. Thanks again for your example. I hope I’m on the way to finally getting this.
© Copyright Elixir Forum | Terms | Privacy & Cookies
