Backblaze and Phoenix LiveView Uploads

maz

Has anyone uploaded to backblaze with Phoenix LiveView Upload? I think I am very close with the upload(it seems to upload the entire file but at the last moment I see this error):

Cross-Origin Request Blocked: The Same Origin Policy disallows reading the remote resource at https://s3.us-east-005.backblazeb2.com/the-bucket-name. (Reason: CORS header ‘Access-Control-Allow-Origin’ missing). Status code: 501.

I’m pretty sure my CORS bucket policy is extremely liberal and I do not see why I’m getting the allow origin issue:

these are the CORS rules on the bucket:

    "corsRules": [
        {
            "allowedHeaders": [
                "*"
            ],
            "allowedOperations": [
                "s3_head",
                "b2_download_file_by_id",
                "b2_upload_part",
                "b2_upload_file",
                "s3_put",
                "b2_download_file_by_name",
                "s3_post",
                "s3_get"
            ],
            "allowedOrigins": [
                "*"
            ],
            "corsRuleName": "downloadFromAnyOriginWithUpload",
            "exposeHeaders": [
                "x-bz-content-sha1"
            ],
            "maxAgeSeconds": 3600
        }
    ]

and here is how I presign:

  def presign_upload(entry, socket) do
    uploads = socket.assigns.uploads
    bucket = System.get_env("BACKBLAZE_S3_BUCKET_ID")
    key = Ecto.UUID.generate() <> Path.extname(entry.client_name)
    config = %{
      region: System.get_env("BACKBLAZE_S3_REGION"),
      access_key_id: System.get_env("BACKBLAZE_S3_APPLICATION_KEY_ID"),
      secret_access_key: System.get_env("BACKBLAZE_S3_APPLICATION_KEY")
    }

    {:ok, fields} =
      sign_form_upload(config, bucket,
        key: key,
        content_type: entry.client_type,
        max_file_size: uploads[entry.upload_config].max_file_size,
        expires_in: :timer.hours(1)
      )

    dbg(fields)
    dbg(key)

    meta = %{
      uploader: "S3",
      key: key,
      url: "https://s3.us-east-005.backblazeb2.com/#{System.get_env("BACKBLAZE_S3_BUCKET_ID")}",

      fields: fields
    }

    dbg(meta)

    {:ok, meta, socket}
  end

The presign data looks like this:

[(word_app 1.5.1) lib/word_app/file_uploads/s3_backblaze.ex:124: WordApp.FileUploads.S3Backblaze.presign_upload/2]
key #=> "ebd306d1-ca78-4bf6-95a6-44e0bb1808ac.jpg"

[(word_app 1.5.1) lib/word_app/file_uploads/s3_backblaze.ex:137: WordApp.FileUploads.S3Backblaze.presign_upload/2]
meta #=> %{
  fields: %{
    "acl" => "public-read",
    "content-type" => "image/jpeg",
    "key" => "ebd306d1-ca78-4bf6-95a6-44e0bb1808ac.jpg",
    "policy" => "ewogICJleHBpcmF0aW9uIjogIjIwMjMtMDctMjBUMDU6NTY6MjYuNTk4Njg1WiIsCiAgImNvbmRpdGlvbnMiOiBbCiAgICB7ImJ1Y2tldCI6ICAiZ29zaGVuLW1lZGlhLW1haW4ifSwKICAgIFsiZXEiLCAiJGtleSIsICJlYmQzMDZkMS1jYTc4LTRiZjYtOTVhNi00NGUwYmIxODA4YWMuanBnIl0sCiAgICB7ImFjbCI6ICJwdWJsaWMtcmVhZCJ9LAogICAgWyJlcSIsICIkQ29udGVudC1UeXBlIiwgImltYWdlL2pwZWciXSwKICAgIFsiY29udGVudC1sZW5ndGgtcmFuZ2UiLCAwLCA4MDAwMDAwXSwKICAgIHsieC1hbXotc2VydmVyLXNpZGUtZW5jcnlwdGlvbiI6ICJBRVMyNTYifSwKICAgIHsieC1hbXotY3JlZGVudGlhbCI6ICIwMDU1MDgxYzYzOWQxYmYwMDAwMDAwMDAxLzIwMjMwNzIwL3VzLWVhc3QtMDA1L3MzL2F3czRfcmVxdWVzdCJ9LAogICAgeyJ4LWFtei1hbGdvcml0aG0iOiAiQVdTNC1ITUFDLVNIQTI1NiJ9LAogICAgeyJ4LWFtei1kYXRlIjogIjIwMjMwNzIwVDA1NTYyNloifQogIF0KfQo=",
    "x-amz-algorithm" => "AWS4-HMAC-SHA256",
    "x-amz-credential" => "0055081c639d1bf0000000001/20230720/us-east-005/s3/aws4_request",
    "x-amz-date" => "20230720T055626Z",
    "x-amz-server-side-encryption" => "AES256",
    "x-amz-signature" => "1d16e41aef14551b9efed9aab874fa18e5e326962dcf93d6e4cdbc8cd5952182"
  },
  key: "ebd306d1-ca78-4bf6-95a6-44e0bb1808ac.jpg",
  url: "https://s3.us-east-005.backblazeb2.com/the-bucket-name",
  uploader: "S3"
}

21 comments

/phoenix #liveview-upload

0 2551 21

2024-01-15 12:06:18 UTC

Showing Posts 1 to 10

Show Best Posts
Show All (oldest first)
Show All (newest first)

thomas.fortes

Can you upload to B2 using something else like postman for example?

CORS is client related, so as long as the generated presigned url is correct the only code that matter is the javascript one for the S3 uploader, if you can upload from postman and not from liveview we can take a look and see what is happening.

Now from experience, if you’re using the examples from the docs for external uploads, try replacing the post with a put, most S3 compatible services that I used don’t handle presigned POST uploads very well unless you’re doing a multipart upload, which is a tad more complex than a simple POST upload.

Post #1

TwistingTwists

I worked with Cloudflare R2.

changing POST to PUT worked for liveview uploads.

Post #2

maz

Changing to PUT resulted in a 403 instead of a 501.

xhr.open("PUT", url, true)

Post #3

TwistingTwists

Are these settings correct at your end?

Post #4

TwistingTwists

Tried PUT just now. Getting 403 with most liberal bucket CORS.

Post #5

maz

I’ve looked for that dialog but I can’t find it. Where is it?

I found “CORS Rules” only:

Post #6

metadaddy

Hi there - I don’t have any experience with Phoenix LiveView in particular, but, as Chief Technical Evangelist at Backblaze, I have worked with Backblaze B2 quite a bit! I’ll work through some of the points in this thread in the hope that it moves you a bit closer to getting presigned URLs working.

@thomas.fortes is correct in his advice to use PUT rather than POST - B2 does not support POST for the S3 PutObject operation.

@maz It’s revealing that the error code changed with the switch from POST to PUT. When you tried to use POST, you received a 501, “not implemented”. If you were to look at the payload in the response, it would be

<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<Error>
    <Code>NotImplemented</Code>
    <Message>This API call is not supported.</Message>
</Error>

With POST, you are receiving 403, “forbidden”, which indicates that something is wrong in either the presigned URL, or the key you are using to generate it. Debugging tip - the response payload will give you more detail on what the problem actually is.

CORS is a bit of a red herring here. The browser complains about the missing Access-Control-Allow-Origin header, but it’s never going to see one on a 501 or 403 error response. Start worrying about CORS when you see 20x responses.

Moving along to the code… The bad news is that sign_form_upload (code) does not actually presign a URL. It submits a POST request with the signature and other parameters sent as form fields (AWS doc), which, as mentioned earlier, is not supported by B2. Unfortunately, changing POST to PUT won’t make it work, as the payload will still be a form submission. B2 responds with 403 since it can’t see the signature it’s expecting as either an HTTP header or a query parameter.

In a presigned URL, the signature and other params are query parameters on the URL (AWS doc). Here’s a real presigned, but expired, URL for uploading the file HelloWorld.txt to my metadaddy-private B2 bucket, with line breaks added so you can see the query parameters clearly:

https://s3.us-west-004.backblazeb2.com/metadaddy-private/HelloWorld.txt?
X-Amz-Algorithm=AWS4-HMAC-SHA256&
X-Amz-Credential=00415f935cf4dcb0000000046%2F20230720%2Fus-west-004%2Fs3%2Faws4_request&
X-Amz-Date=20230720T230347Z&
X-Amz-Expires=60&
X-Amz-SignedHeaders=host&
X-Amz-Signature=5b9d9762a8aec0eedc54abacabdd22c8cb9c120e3811f331e2821dc6b2217915

Another debugging tip - if you can capture the presigned URL, you can test it at the command line with curl, like this:

% curl -i -X PUT --data-binary @HelloWorld.txt 'https://s3.us-west-004.backblazeb2.com/metadaddy-private/...'
HTTP/1.1 200 
x-amz-request-id: 05d9e319b2960ffb
x-amz-id-2: aMZc1tmblObEzMzXXY0pm/zRVZBtjHWIJ
ETag: "59ca0efa9f5633cb0371bbc0355478d8"
x-amz-version-id: 4_z0145cfc9e3f5ec0f74ed0c1b_f409ee060e487b08b_d20230720_m234423_c004_v0402011_t0013_u01689896663895
Cache-Control: max-age=0, no-cache, no-store
Content-Length: 0
Date: Thu, 20 Jul 2023 23:44:23 GMT

The good news… In researching this, I came upon a comment on the sign_form_upload gist pointing to a dependency-free implementation of presigned URLs. I don’t have the ability to test this, but it looks like it does the right things. Even better, it looks like the author of that code, @denvaar, has an account here and might be able to help, too.

Hope this helps - good luck getting it working!

Post #7