Tcp file transfers - feedback


Hi all,

I’ve been reading through “Programming Erlang” and got interested in one of the exercises for writing Yet Another File Server. So I’m doing that in a very naive, only basic processes primitives (except gen_tcp) way and all is fine, it’s for learning. But I would like to know how usually one deals with the file transfer info.

Right now I have something like this:

maybe_copy_file(Client, File) ->
    case not filelib:is_dir(File) and filelib:is_file(File) of
        true -> copy_file(Client, File);
        _ ->
            io:format("invalid file path: ~p~n", [File]),

copy_file(#client{socket= Socket} = Client, File) ->
    Filename = filename:basename(File),
    {ok, #file_info{size= Size}} = file:read_file_info(File),
    Bt = term_to_binary({Filename, Size}),
    gen_tcp:send(Socket, "sendfile:" ++ Bt),
    file:sendfile(File, Socket),

Where I read the file size, then term_to_binary that info, and send, first a message with sendfile:binary_term_info

Then on the #file_server.erl I do, after receiving that first message:

file_receive(#server_client{socket= Socket, fs= #dir_struct{cwd= Cwd}} = Client, Info) ->
    {Filename, Size} = binary_to_term(Info),
    Path = filename:absname_join(Cwd, Filename),
    {ok, FileHandle} = file:open(Path, [write, binary]),
    receive_file(Socket, FileHandle, Size),

receive_file(_, FileHandle, 0) ->
    io:format("Finished Receiving File"),
receive_file(Socket, FileHandle, Size) ->
    {N_Size, ToRead} = case Size - 500 of
                           N when N >= 0 -> {N, 500};
                           N -> {0, 500 + N}
    case gen_tcp:recv(Socket, ToRead) of
        {ok, Data} ->
            io:format("~ndata: ~p~n", [Data]),
            case file:write(FileHandle, Data) of
                ok -> receive_file(Socket, FileHandle, N_Size);
                {error, Error} ->
                    io:format("Error Writing ~n~p~n", [Error]),
        {error, Reason} ->
            io:format("Received finished with: ~p~n", [Reason]),

So with the file size I sent earlier I just read chunks from the socket until I’ve read the whole amount. This works fine, but that part looks ugly. I’m wondering how does one usually encode this information (size & filename for instance) in a regular tcp conversation? Is there any protocol that is commonly used? Or does everyone just write their own protocol and handling semantics? Also, what plays a role in deciding the size of the “chunks” to be read from the socket?

I’m keeping the socket connection open on both ends even after the transfer finishes, otherwise I could just not worry about the size and once closed it would mean the transfer was done. I had started with gen_tcp:recv(Socket, 0), but then I can not discern when the file is finished?

Any info/samples would be welcomed as well (about either protocols themselves, or done in elixir or erlang).