elsatch
File management sample codes?
Hi everyone!
My name is César and I’m learning how to use Elixir. My goal is to create some kind of web crawler to locate historical data. Anyway, right now, I am into a more mundane task.
I’m trying to create some programs in Elixir than help me sort my files. For example, I’d like to sort my files by extension or order my photos by modification date. I thought that it would be an easy task, but there are not so many examples of how to traverse directories and move files. I’ve checked on Exercism and several online books, but I am always taken back to the reference manual.
I’ve seen the Files.rename/2 information, but given my current Elixir level, I’m not sure about how to combine it with safeguards so the results are not disastrous. (For example, I might have several source files called IMG_0001.jpg and I don’t want to have them silently overwritten).
Do you know any source that might have this kind of file management examples? Any cookbook that covers this? Most resources flow like Install |> Strings and binaries |> Flow control |> OTP BeamVM, skipping the file chapter ![]()
Thanks for your support!
Most Liked
Aetherus
defmodule sort_files_by_extension do
def sort_files do
File.ls!("./test")
|> Enum.map(fn filename ->
extname = Path.extname(filename) #=> ".jpg"
basename = Path.basename(filename, extname) #=> "IMG_XXXX"
{basename, extname, filename}
end)
|> Enum.map(&make_extension_dir/1)
|> Enum.map(&dedup/1)
|> Enum.each(&move_extension_directory/1)
defp make_extension_dir({_, "." <> extension, _} = arg) do
File.mkdir_p!("./#{extension}")
arg # just return the argument for other `Enum.map`
end
defp dedup(arg, suffix \\ 0)
defp dedup({basename, "." <> extension, filename} = arg, 0) do
if File.exists?("#{extension}/#{filename}") do
dedup(arg, 1)
else
arg
end
end
defp dedup({basename, "." <> extension = extname, filename} = arg, suffix) do
if File.exists?("#{extension}/#{basename}_#{suffix}.#{extension}") do
dedup(arg, suffix + 1)
else
{"#{basename}_#{suffix}", extname, filename}
end
end
defp move_to_extension_directory({basename, "." <> extension, filename}) do
File.rename!("./test/#{filename}", "./#{extension}/#{basename}.#{extension}")
end
end
A few suggestions:
Enum.eachis for pure side effects (e.g. pure file system operations). It’s not chainable. UseEnum.mapinstead.- You can wrap all the information in a tuple, and pattern match on it in the function parameters.
- You can trust the file system to do the right thing (e.g.
File.mkdir_p!) - Hail recursion!
dimitarvp
This might not be the answer you are looking for – but I’d reach for sqlite3. It also can be up to 35% faster than raw file access.
Trouble is, current state of the art of sqlite3 in Elixir does not support Ecto 3 (latest version of the de facto DB persistence library); only version 2 which is now quite old. I am working on bringing Ecto 3 to sqlite3 but it definitely is not going to be ready tomorrow.
Using sqlite3 will rid you of the potential file overwriting problem as well since there the ID is… you know, the database ID, not the filename itself. What’s more, an embedded DB like sqlite3 gives you the ability to sort and filter out of the box.
If that doesn’t sound tempting to you then I’d be happy to help you exactly with the File / IO API. However, you should have in mind that functions like stat can have differing behaviours between different OS-es. Which is all the more reason to opt for a database.
Any particular scenario you would like help with?
If you would like something that basically manages uploads to your app then Waffle might be exactly what you are looking for (it can put all stored files into Amazon’s S3 or in your local filesystem).
OvermindDL1
I’d probably do this:
def sort_files(path) do
files_by_dir =
File.ls!(path)
|> Enum.reject(&File.dir?/1)
|> Enum.group_by(&Path.extname/1)
Enum.each(files_by_dir, fn {"." <> ext, files} ->
extpath = Path.join(path, ext)
File.mkdir_p(extpath)
Enum.each(files, fn file ->
filepath = Path.join(extpath, file)
# Keep only a max of 9, could easily make this unbounded though, but eh useful feature to add
Enum.each(9..2, &File.rename("#{filepath}_#{&1-1}", "#{filepath}_#{&1}"))
File.rename(filepath, "#{filepath}_1")
File.rename!(Path.join(path, file), filepath)
end)
end)
end
Could use some error reporting, but eh.
Last Post!
elsatch
Thanks for your responses! I will review them during the weekend to learn about the different approaches. I really appreciate your suggestions and tips to improve my skills ![]()
P.D As I was looking for other interesting examples, I stumbled upon this Bret Trepstra Ruby script to sort based on tags. I was so surprised to understand the syntax!!
Popular in Questions
Other popular topics
Categories:
Sub Categories:
Forums
Popular Tags
- #ecto
- #liveview
- #troubleshooting
- #learning-elixir
- #deployment
- #library
- #erlang
- #testing
- #genserver
- #mix
- #absinthe
- #remote-other
- #otp
- #plug
- #how-to-question
- #macros
- #postgres
- #channels
- #elixirconf
- #exunit
- #discussion
- #code-sync
- #javascript
- #podcasts
- #onsite
- #dialyzer
- #docker
- #authentication
- #umbrella
- #full-time-contract
- #podcasts-by-brainlid
- #ecto-query
- #elixir-ls
- #phoenix_html
- #iex
- #blog-post
- #graphql
- #genstage
- #ai
- #websockets
- #supervisor
- #elixirconf-us
- #advent-of-code
- #distillery
- #processes
- #forms
- #api
- #metaprogramming
- #security
- #hex










