I am trying to stream a file from an SFTP server using OTP’s :ssh_sftp.pread/4 which allows you to read a chunk of a file from a given offset. While the first call to pread/4 succeeds, all subsequent calls fail with an {:error, :no_such_file}. This happens even if the second pread/4 call is for exactly the same chunk of the file as the first call. This is only happening with one particular SFTP server (which I do not own), I haven’t been able to reproduce it using the in-build macOS SFTP server. Has anyone seen a similar issue, or does anyone know how I could go about investigating the issue further?
Here’s the failing code. The first call to pread/4 succeeds because the file exists and we are reading within the bounds of the file size. I would have expected the second call to pread/4 to also succeed.
file_name = "/path/to/remote/file.zip"
:ok = :ssh.start()
{:ok, channel_pid, connection_ref} =
:ssh_sftp.start_channel('the-remote-sftp-server', 22, user: 'my-username', password: 'my-password')
{:ok, file_info} = :ssh_sftp.read_file_info(channel_pid, to_charlist(file_name))
# Just to proved I'm not reading beyond the end of the file - reports file size of 69207847
IO.inspect(elem(file_info, 1), label: "remote file size")
{:ok, handle} = :ssh_sftp.open(channel_pid, to_charlist(file_name), [:read, :raw, :binary])
# Succeeds
{:ok, data} = :ssh_sftp.pread(channel_pid, handle, 0, 1000)
# Fails with "(MatchError) no match of right hand side value: {:error, :no_such_file}"
{:ok, data} = :ssh_sftp.pread(channel_pid, handle, 0, 1000)
Some context: I am implementing a protocol for the unzip package which allows you to stream and unzip files no matter where they are stored as long as you implement functions for size and pread. One of the example actually describes how to do this with SFTP, but that code fails for me with this one particular SFTP server. It works for me with my local SFTP server, even when I use the actual ZIP file from the remote server.
Strangely, I can get this working by calling :ssh_sftp.open/3 to get a new handle before each call to pread/4, but that slows down streaming significantly.
Am I wrong to assume I should be able to use pread/4 like this? Could this be an OTP problem, or has anyone seen an SFTP configuration that could cause something like this?
