Any interest in a library that wraps rsync?

Your Libraries & OS Mentoring Seeking Feedback
, , , ,
1

I’ve written a crude module for retrieving files from a remote server using rsync, since I didn’t see any existing tools already. This could be useful anywhere one needs to efficiently move files across filesystem or network boundaries, and could even be used for better control and monitoring of deployments. Does it sound like something I should package into a library?

The inner logic takes advantage of the --info=progress2 reporting mode for rsync, reading status lines to monitor progress:

port = Port.open(
  {:spawn_executable, System.find_executable("rsync")},
  [args: ["-a", "--info=progress2"] ++ remote_urls ++ [local_path],
    :binary, :exit_status, :hide, :use_stdio, :stderr_to_stdout]
)

receive do
  {^port, {:data, data}} ->
    ....

  {^port, {:exit_status, status}} ->
    ....
end
6 Likes
2

I am not a reliable source but IMO that would be more interesting as a blog post showing how did you wrap rsync.

4 Likes
3

A blog post is a good idea, and I ended up publishing as a library so there’ll be something to write about:

2 Likes
4

Thanks. Had a quick look. IMO you should include a complete example with the error messages that your GenServer can also emit (not only the happy path). I’d be interested in just copy-pasting such an entire block of code in a project of mine and then tuning the separate case branches for my goals. Better peace of mind.

5

I was also curious about error cases, so I added some tests that show reasonable behavior when rsync emits random junk (forward to Logger at :warning level) and when the process crashes (forward error to linked parent).
Deleted: random claims that the tests are a full example.

Feel free to share the case statements you add for your own application goals.

6

I struggle with basic OTP principles. Is the normal way to handle a start_link failure to run it under a Supervisor? Anyone feel like sharing a snippet showing how to run a brittle command like this and catch the error?

In my own app I’m letting Oban handle the retry logic and in tests I’ve trapped the exit but these are side-stepping typical usage I suspect.