fireproofsocks

fireproofsocks

S3 operations returning HTTP 505 error

Given a particularly devious S3 object key, e.g.

"users/f8db13a0-e2b8-45cd-a902-5eede074eb46/~*\\|:\"<> +`!@#$%^&()-=_[]{};',.?đź§Ş//fakedata~*\\|:\"<> +`!@#$%^&()-=_[]{};',.?đź§Ş.txt"

I’m running into HTTP 505 errors from the ExAws client, e.g.

    bucket
    |> ExAws.S3.head_object(object_key)
    |> ExAws.request()

I can see the objects there if I run ExAws.S3.list_objects/2 or ExAws.S3.list_objects_v2/2

A python script uploaded the file (probably using official boto), but I’m failing to retrieve it.

Frustratingly, the AWS docs about 5xx errors do not mention 505 errors anywhere. There are few other mentions of this error (e.g. this one).

Because I’m able to interact with other “normally named” objects in the same bucket, the culprit is likely the weird object key. Has anyone else encountered this? Is this perhaps a bug in ExAws.S3?

Most Liked

evadne

evadne

The proper fix/workaround is to store objects with GUID keys in S3 only (actual file name stored in database) and then use content-disposition to emit correct file names later.

As @LostKobrakai discussed, you can consider sanitising/encoding the characters or changing them to hex representation, however there is no standardised way to pass string literal within JSON & there may be significant work required to handle these files.

I think only place in ex_aws to change would be to do this encoding/decoding transparently (as breaking change) in ExAws.Operation.S3.add_bucket_to_path/2 or maybe ratchet up a step and use XML for outgoing requests (so you get proper string literals that can represent the object names you need), however as it would create such huge support burden for the maintainers + is already discouraged by AWS + would encourage further bad application design, I don’t think it will be considered AT ALL

So my best suggestion is to change design as previously mentioned to obviate the need of this rigmarole. It is also the right thing to do, if you could sanitise the names with some kind of URI encoded string and avoid a whole class of security bugs… such as user putting a file name that effectively results in a path traversal exploit. The bigger the attack surface the more bigger your risk.

Where Next?

Popular in Questions Top

marius95
Hello everyone, I try to use an Javascript Event Handler in my root.html.leex file. Therefore I created a function in the app.js file: ...
New
qwerescape
Is there a way to get the call stack or stack trace at any point in the code? Not from exceptions, but an expression that returns how the...
New
fireproofsocks
I’m working on defining a simple Ecto schema for a table (in PostGres), but I don’t see where I can define a column as NOT NULL. Conside...
New
baxterw3b
Hi guys, i’m new in the Elixir world, and i have to say, that i love it! i’m having some problem to understand anonymous functions with ...
New
aalberti333
As the title describes, I’m trying to run Enum.map() over a list of key/value pairs, where the value is a map. My data looks like this: ...
New
sergio_101
I am VERY much an elixir newbie. I have taken one elixir course and one phoenix course on Udemy. During that course, I saw the instructor...
New
Brian
What is the proper way to load a module from a file in to IEX? In the python world, doing something like this pretty standard: from ....
New
joaquinalcerro
Hi there, I am working with Ecto-Postgresql and I need to call all of the records from a specific table but the table has 40,000 records...
New
hariharasudhan94
Lets say I have map like this fetching from my database %{"_id" =&gt; #BSON.ObjectId&lt;58eb1a7a9ad169198c3dXXXX&gt;, "email" =&gt; ...
New
jononomo
For some reason my phoenix channels are working for me in my local dev environment, but as soon as I deploy via Docker, I get a 403 error...
New

Other popular topics Top

sen
Hi All, I set a environment variables in dev.exs , like below code. when i start server, how can i set the ${enable} value? thanks. d...
New
JeremM34
Hello, how can I check the Phoenix version ? Thanks !
New
jononomo
I am trying to figure out how Mix knows whether the environment is test, dev, or prod – where is this set? Thanks.
New
jerry
Good day to you all. I have been struggling to get a query involving like and ilike to work. Can anyone assist me on this, please? pro...
New
jay1
Why is it that the mnesia database isn’t the most preferred database for use in Elixir/Phoenix?
New
vegabook
I’m brand new to Phoenix and I have stripped one of the demo applications to the bone. I just want to get an svg up on the screen. Here i...
New
saif
Hello everyone, Long time lurker first time poster here. I’ve recently begun working on Elixir full-time again! :raised_hands: It’s been...
New
komlanvi
Hi everyone, I was playing with phoenix liveView but I run into an issue. I have a form and want to validate each input text when the te...
New
openscript
Hello! Sorry for this astonishing simple question, but I’m really stuck. I try to set up the intellij-elixir plugin, but I don’t know ho...
New
lanycrost
Hi everyone! I need implement if…else if…else condition from my elixir code, and anymore of this control flow structures not work proper...
New

We're in Beta

About us Mission Statement