So I need users to upload a few hundred to a few thousand tiny image files, obviously doing that picture-by-picture would be extraordinarily painful, so using the multiple option of the standard file input element to allow selecting the entire directories contents.
Now, when doing <input type="file" id="images" name="images"> I get a %Plug.Upload{} structure just fine, however if I do <input type="file" id="images" name="images" multiple> then I get a binary that is the name of one of the listed files, so changing that to <input type="file" id="images" name="images[]" multiple> gets the params giving me a list of binaries of the file names, however these are not %Plug.Upload{} structures, so I don’t know anything beyond the filenames, like say the contents of the files.
I’ve found this past thread here that was useful enough to get me to add the [] part to the name, but don’t actually help me getting to the contents (and if you check the attached github issue they never seemed to handle it either):
Furthermore, using inotifywait on the entirety of the /tmp directory shows there is no file writing happening at all for a multiple file upload, yet single file uploads without the multiple attribute save and are deleted just fine.
The docs of Plug.Upload are not useful on this issue either.
The total size of the files are well well below the max file size (only a couple hundred kb while testing and just trying to get it working).
Considering this seems to be a pretty basic use-case I have to be screwing up pretty big somewhere, thus any assistance would be very welcome! ^.^;
A quick test with php and the same <input type="file" id="images" name="images[]" multiple> bit worked fine, so… something on the plug/phoenix side that I’m doing wrong?
Only for standard HTTP upload. Is not WebSockets standard better for that? You can send multiple files at a time in their own small requests without sending HTTP headers, extra (every request) validation etc. It’s also better to for future changes like showing upload progress, chucking etc. Personally I don’t like also ideas of sending multiple files in one request. Application should work as long as it can. In your case if you would have a network problem in the middle of process then you would need to send again all of hundred or even few thousand files. Even if they are tiny it’s not good for user experience and it’s also not scalable solution. Remember that there are still people which do not have fast fiber internet.
I believe that @josevalim should know best this problem as he is most active contributor to plug. I think that you should create a GitHub issue for this problem in elixir-plug/plug. I did not looked at source code of plug, but if I would guess then I think that plug is trying to handle multiple upload as same as normal upload which means that it hits an edge case of file upload implementation in plug where developers does not expect to parse request content of multiple files. Also I believe that Issue when uploading a file with multipart/form-data with Chrome but trying to upload a directory instead of a fileplug issue is really similar to your case (not properly handled request content standard). Again I recommend to create new issue with providing all data as in linked issue. Also if you would look on that issue then you would see that different browsers could send requests content in different format, so make sure to test them and give us information about it as in linked issue.
Yeah, just found that @josevalim commented about it 2 years ago:
It doesn’t seen the multiple option is working for file_input. So you need to do rather something like: file_input f, :files, name: "files[]" or send a PR so we support multiple.
This means that you need to have multiple file_input with name which ends with [] (because it shows Phoenix that we want to save value of multiple inputs into one list). [] suffix should not change parsing parameters for only one field. Just instead of value you would have [value] and nothing more, so it’s useless for you.
If I remember it correctly then there needs to be some specification regarding content parsing - probably from W3C, but I’m not sure about it. I saw really long specification for HTML / HTML5 document parsing and as it’s related to it (as a part of <input/> specification) it should be described well or at least linked to 3rd-party specification page. It may be also described or linked on browser developer pages (like MDN) in contents of Gecko or WebKit specifications. My last idea are issue trackers with tickets for adding support for it, so there should be a commit(s) which are encoding form to valid request body, so by knowing how it works somebody could write a parser.
In short somebody would need to find up-to-date specification page and update plug body parser related code.
I need to be able to fall back to no javascript and still have it fully work, requirements of my job. ^.^;
All local network, if there are network issues then there are bigger problems and this suddenly doesn’t matter. Uploading of the entire set is only about 40 megs as it is and this will happen rapidly on some days and then be pretty dead for weeks at a time.
Doesn’t matter one tiny iota here, gigabit network. ^.^
I don’t want to create a github issue for what seems to be a pure usability issue. This is the first time I’ve needed to upload files to a phoenix server so it’s outside of my realm of knowledge. It works fine in php and with python servers though and those were pretty trivial to get working (iterate a file list).
I would highly doubt this, it is part of the spec and they seem to follow the spec fairly well for most other things.
Hmm, I wonder how this would respond to me trying to upload a directory… testing…
Edge and Firefox both don’t allow me to upload a directory, only multiple explicit files.
I’ve tested in Chrome, Firefox, and Edge and all had no issues with the PHP test script for them.
Yep, PHP has the same thing, you need to have [] at the end of the name for PHP’s built-in handlers to handle it.
Well both PHP (built-in) and Python (flask) had no issues with them? In PHP you just iterate the file list (it’s an array) and in flask you just call request.file.getlist("files[]"), it was all super simple (unlike phoenix/plug so far but I still assert it has to be me doing something wrong, at the very least in needs a doc fix though…).
Ah hah! I am the idiot! To forever encode my missing bit into history and maybe help others later, don’t forget the multipart: true option to the form call! I noticed it within seconds of opening the file when I got in today. Yesterday was a loooong day and I missed the obvious. ^.^
More importantly - post the entire form fragment when discussing problems with input elements. On MDNenctype="multipart/form-data" kind of sticks out like a sore thumb - but I figured that you had that covered given you got the PHP version working.
Aside: Does anybody have any information on the origin of the “[]” name suffix convention for multiple files? It seems that the HTML specification doesn’t require it but it seems to be more of a server-side convention (PHP specifically). Or am I wrong about this?
About “[]” suffix, The specs are fairly simple, here.
It is a server-side convention. HTTP specs see a[]=1&a[]=2 and a=1&b=2 and a[a]=1&a[b]=2 all as some key-value pairs. It’s the server-side who decides to represent first one as array, second one as simple values and third one as hash-maps or objects.
You could easily write a server-side framework to parse a_1=1&a_2=2 into array, but since almost all languages use [] notion for arrays, it seemed natural to the first developers.
Nice functionality you got there OverMindDL1, Please help me on how i can structure my controller i’m new to elixir and i need your help on how the controller and the schema should look like should look like, the one which handles this post request. And please mention if there is a way of achieving this without using a library. Personally i’m using Arc for my image upload but its only able to send one image at at time. I don’t know where i’m going wrong.
