elsatch

elsatch

File management sample codes?

Hi everyone!

My name is César and I’m learning how to use Elixir. My goal is to create some kind of web crawler to locate historical data. Anyway, right now, I am into a more mundane task.

I’m trying to create some programs in Elixir than help me sort my files. For example, I’d like to sort my files by extension or order my photos by modification date. I thought that it would be an easy task, but there are not so many examples of how to traverse directories and move files. I’ve checked on Exercism and several online books, but I am always taken back to the reference manual.

I’ve seen the Files.rename/2 information, but given my current Elixir level, I’m not sure about how to combine it with safeguards so the results are not disastrous. (For example, I might have several source files called IMG_0001.jpg and I don’t want to have them silently overwritten).

Do you know any source that might have this kind of file management examples? Any cookbook that covers this? Most resources flow like Install |> Strings and binaries |> Flow control |> OTP BeamVM, skipping the file chapter :slight_smile:

Thanks for your support!

Most Liked

Aetherus

Aetherus

defmodule sort_files_by_extension do
  def sort_files do
    File.ls!("./test")
    |> Enum.map(fn filename -> 
         extname = Path.extname(filename)  #=> ".jpg"
         basename = Path.basename(filename, extname)  #=> "IMG_XXXX"
         {basename, extname, filename}
       end)
    |> Enum.map(&make_extension_dir/1)
    |> Enum.map(&dedup/1)
    |> Enum.each(&move_extension_directory/1)

  defp make_extension_dir({_, "." <> extension, _} = arg) do
    File.mkdir_p!("./#{extension}")
    arg  # just return the argument for other `Enum.map`
  end

  defp dedup(arg, suffix \\ 0)

  defp dedup({basename, "." <> extension, filename} = arg, 0) do
    if File.exists?("#{extension}/#{filename}") do
      dedup(arg, 1)
    else
      arg
    end  
  end

  defp dedup({basename, "." <> extension = extname, filename} = arg, suffix) do
    if File.exists?("#{extension}/#{basename}_#{suffix}.#{extension}") do
      dedup(arg, suffix + 1)
    else
      {"#{basename}_#{suffix}", extname, filename}
    end 
  end

  defp move_to_extension_directory({basename, "." <> extension, filename}) do
    File.rename!("./test/#{filename}", "./#{extension}/#{basename}.#{extension}")
  end
end

A few suggestions:

  1. Enum.each is for pure side effects (e.g. pure file system operations). It’s not chainable. Use Enum.map instead.
  2. You can wrap all the information in a tuple, and pattern match on it in the function parameters.
  3. You can trust the file system to do the right thing (e.g. File.mkdir_p!)
  4. Hail recursion!
dimitarvp

dimitarvp

This might not be the answer you are looking for – but I’d reach for sqlite3. It also can be up to 35% faster than raw file access.

Trouble is, current state of the art of sqlite3 in Elixir does not support Ecto 3 (latest version of the de facto DB persistence library); only version 2 which is now quite old. I am working on bringing Ecto 3 to sqlite3 but it definitely is not going to be ready tomorrow.

Using sqlite3 will rid you of the potential file overwriting problem as well since there the ID is… you know, the database ID, not the filename itself. What’s more, an embedded DB like sqlite3 gives you the ability to sort and filter out of the box.


If that doesn’t sound tempting to you then I’d be happy to help you exactly with the File / IO API. However, you should have in mind that functions like stat can have differing behaviours between different OS-es. Which is all the more reason to opt for a database.

Any particular scenario you would like help with?

If you would like something that basically manages uploads to your app then Waffle might be exactly what you are looking for (it can put all stored files into Amazon’s S3 or in your local filesystem).

OvermindDL1

OvermindDL1

I’d probably do this:

  def sort_files(path) do
    files_by_dir =
      File.ls!(path)
      |> Enum.reject(&File.dir?/1)
      |> Enum.group_by(&Path.extname/1)

    Enum.each(files_by_dir, fn {"." <> ext, files} ->
      extpath = Path.join(path, ext)
      File.mkdir_p(extpath)

      Enum.each(files, fn file ->
        filepath = Path.join(extpath, file)
        # Keep only a max of 9, could easily make this unbounded though, but eh useful feature to add
        Enum.each(9..2, &File.rename("#{filepath}_#{&1-1}", "#{filepath}_#{&1}"))
        File.rename(filepath, "#{filepath}_1")
        File.rename!(Path.join(path, file), filepath)
      end)
    end)
  end

Could use some error reporting, but eh.

Last Post!

elsatch

elsatch

Thanks for your responses! I will review them during the weekend to learn about the different approaches. I really appreciate your suggestions and tips to improve my skills :slight_smile:

P.D As I was looking for other interesting examples, I stumbled upon this Bret Trepstra Ruby script to sort based on tags. I was so surprised to understand the syntax!!

Where Next?

Popular in Questions Top

vegabook
I’m brand new to Phoenix and I have stripped one of the demo applications to the bone. I just want to get an svg up on the screen. Here i...
New
vertexbuffer
Hello, can anybody help here..? I have a list of players and I what to delete an element, but every for loop the list is reverting to ori...
New
New
stefanchrobot
What’s the safe way to decode a JSON string into a struct? I want to avoid calling String.to_atom. Jason.decode can give me a map with st...
New
aalberti333
As the title describes, I’m trying to run Enum.map() over a list of key/value pairs, where the value is a map. My data looks like this: ...
New
vrod
I am using the Starship cross-shell prompt – it seems pretty nice, but I get some errors: [WARN] - (starship::utils): Executing command ...
New
fayddelight
I tried installing elixir 1.11.2 erlang 23.3.4 via asdf in my zsh shell. Enabled the versions locally and globally. When I list them ...
New

Other popular topics Top

Darmani72
If I have a post route which an argument: post /my_post_route/:my_param1, MyController.my_post_handler How would get the post params ...
New
gshaw
What is the idiomatic way of matching for not nil in Elixir? E.g., First way: defp halt_if_not_signed_in(conn, signed_in_account) when...
New
shijith.k
I am trying to start a new phoenix project with elixir 1.9, but mix phx.new does not work. It says that ** (Mix) The task "phx.new" could...
New
msaraiva
Surface is an experimental library built on top of Phoenix LiveView and its new LiveComponent API that aims to provide a more declarative...
564 44139 214
New
SoCreat
i’m a new one to elixir which editor can i use vs code? or atom? Thanks! :smiley:
New
jason.o
In the code below, if the create action is not set to accept “extra_key” as an input, it errors out with a message shown above. Is there ...
New

We're in Beta

About us Mission Statement