Encoding multipart/form-data HTTP body

Chat / Discussions
, , , , ,
1

Does anyone here know of an Elixir library which would help me assemble bodies of HTTP requests with “multipart/form-data” Content-type?

This would be the reverse of https://github.com/elixir-plug/plug/blob/v1.8.2/lib/plug/parsers/multipart.ex#L39 i.e. parsing “multipart/form-data” requests by Plug.

Ideally I could either assemble it and get it as String, or in streaming form (for large file uploads).

Hackney does that and I can try translating that to Elixir, but would be ideal if I didn’t have to for obvious reasons :D.

Does anyone know of a piece of Elixir code that does that, even if it’s part of bigger library?

2 Likes
2

HTTPoison wraps hackney so you can avoid translating and supports multipart. Haven’t used it for this myself.

3

Thanks. A bit of a background I should have posted: I want to avoid using HTTPoison / hackney and use either mint directly or with a wrapper called mojito. Unfortunately these don’t have part that assembles “multipart/form-data” requests, so I either have to find one from some other library, or write one myself.

4

Ah, I see. Yeah, since they’re pretty new and Mint itself is not necessarily designed for “regular” use cases (although mojito is), you’re likely to have to write something yourself if mojito doesn’t support it.

Out of curiosity, why does it have to be mint?

5

Hackney has got issues that I stumbled upon recently (among others before), but I don’t think I like how it depends on application configuration either, things get quite difficult once you have to deal with SSL / self-signed certificates or client SSL authentication. And I had fair amount of issues on production with using it too, although not recent. There’s a thread on this forum from 2017 where I argued we need a better built-in HTTP client library in Elixir itself.

Mint comes as close as it gets to what I wanted the standard HTTP client to be, it’s idiomatic Elixir, it’s low level enough that you can build idiomatic clients on top of it (like mojito) and from what I saw really like it’s design. It’s a good reading if one wants to learn how to deal with TCP/SSL connections in Elixir too. Very educational :slight_smile:

I think I prefer to put in some effort working on getting it up to desired shape myself rather than plan for another battle agains HTTPoison/hackney.

1 Like
6

Got it, that makes sense :slight_smile: I’ve had some obscure problems with HTTPoison (hackney) too, although nothing I couldn’t just ignore…

1 Like
7

tesla can do MultiPart on all of its backends as far as I know… You are free too choose between hackeny, httpc and ibrowse IIRC…

1 Like
8

@NobbZ ah, great catch! It’s there indeed: https://github.com/teamon/tesla/blob/master/lib/tesla/multipart.ex

I’ll have to think on how to extract this possibly to shared lib but for now I can just include tesla as a proof of concept.

9

Ah, so you do not actually want to send the request, you just want to have the encoded body?

Then I missunderstood your intial question, but I’m happy that I still was able to help :smiley:

10

You did but that’s helpful anyway, I was counting on such example code but didn’t think about Tesla.

11

does Finch support multipart data request?

12

There is an open PR to support it.

13

when they will merge? is there time span on that?

I am going to switch from HTTPoison to Finch, and the whole application based on sending multipart requests.

14

I think they will but need some minor tweaks in terms of naming things I thunk. I will try to do it tonight and let you know. Nothing from API should change so I guess you can try from my branch/pr, in fact, another pair of eyes testing this would be very much welcome.

niedz., 13 gru 2020, 14:01 użytkownik Ijunaidfarooq via Elixir Programming Language Forum <noreply@elixirforum.com> napisał:

15

Sure thank you.

17

It seems like they can solve your issue. Why do you think using them wouldn’t fit into your project?

18

I could use hackney directly, yes, and I have been doing it but it’s a low level API. As far as I know, and I inspected HTTPoison code pretty intensely to find a way to do it, it can’t stream request body. It can stream the response from the server, but it doesn’t allow me to stream request. I think hackney or ibrowse (or a Tesla wrapper for either) used directly could do it, however.

19

And I think you may have asked here about documentation and you are right, once my PR is merged I will work on some documentation to how to use it. I suppose you figured out the way to use http basic auth but otherwise hit me up, I am doing the same.

20

I did this way

credentials = "#{username}:#{password}" |> Base.encode64()
EvercamFinch.request(:get, url, [Authorization: "Basic #{credentials}"])

and module is

defmodule EvercamFinch do

  def request(method, url, headers \\ [], body \\ nil, opts \\ []) do
    transformed_headers = tranform_headers(headers)
    request = Finch.build(method, url, transformed_headers, body)

    case Finch.request(request, __MODULE__, opts) do
      {:ok, %Finch.Response{} = response} ->
        {:ok, response}

      {:error, reason} ->
        {:error, reason}
    end
  end

  defp tranform_headers([]), do: []
  defp tranform_headers(headers) do
    headers
    |> Enum.map(fn({k, v}) ->
      {Atom.to_string(k), v}
    end)
  end
end

are you doing any different?

1 Like
21

Hi, I am little confused about the PR you pointed me out, on Finch.

my use case is, I am trying to send a multipart request using Finch.

in past with HTTPoison. it was like this

HTTPoison.post(
  url,
  {:multipart, [{file_path, image, []}]},
  [],
  hackney: [pool: :seaweedfs_upload_pool, recv_timeout: 15_000]
)

URL = http://localhost:8888/everc-dupzo/snapshots/oldest-1572566369.jpg
file_path = /snapshots/oldest-1572566369.jpg
and the image is a binary data, which you can get while doing File.read!.

I have done every other replacement of HTTPoison but this has become a road blocker for me.

I have also tried to do simple post request using the data but it’s not working either.

Can you guide me how I Can send an Image binary data as a multipart request?

NOTE: in above HTTPoison request, the image is not saved on disk at all.