How to provide x-amz-content-sha256 to ex_aws_s3 presigned_url

Questions / Help
,
1

hello, I would like to receive pre-computed sha256 hash of file upload from client and generate presigned url for client to upload the file to s3.

here is the code to generate the presigned_url

size = 123
mime_type = "image/jpg"
storage_class = "STANDARD_IA"
key = "some/path/0723faba-72b5-45bf-9373-ad37dbcf6a80.jpg"
hash = "output:{openssl dgst -sha256 /path/to/test/file.jpg}"

:s3
|> ExAws.Config.new()
|> ExAws.S3.presigned_url(
    :put,
    bucket_name(),
    key,
    [
      expires_in: @upload_expiry,
      query_params: [
        {"x-amz-acl", "private"},
        {"x-amz-storage-class", storage_class},
        {"x-amz-server-side-encryption", "AES256"}
      ],
      headers: [
        {"content-length", size},
        {"content-type", mime_type},
        {"x-amz-content-sha256", hash}
      ]
    ]
  )

I took the url generated from the above code and upload give me this error

<?xml version="1.0" encoding="UTF-8"?>
<Error>
	<Code>SignatureDoesNotMatch</Code>
	<Message>The request signature we calculated does not match the signature you provided. Check your key and signing method.</Message>
	<AWSAccessKeyId>AKXXXXXXXXXXXXXXX</AWSAccessKeyId>
	<StringToSign>AWS4-HMAC-SHA256
20220118T095403Z
20220118/ap-southeast-1/s3/aws4_request
abc123</StringToSign>
	<SignatureProvided>abc456</SignatureProvided>
	<StringToSignBytes>41 57 XX XX</StringToSignBytes>
	<CanonicalRequest>PUT
some/path/0723faba-72b5-45bf-9373-ad37dbcf6a80.jpg
X-Amz-Algorithm=AWS4-HMAC-SHA256&amp;X-Amz-Credential=AKXXXXXXXXXXXXXXX%2F20220118%2Fap-southeast-1%2Fs3%2Faws4_request&amp;X-Amz-Date=20220118T095403Z&amp;X-Amz-Expires=16&amp;X-Amz-SignedHeaders=content-length%3Bcontent-type%3Bhost%3Bx-amz-content-sha256&amp;x-amz-acl=private&amp;x-amz-server-side-encryption=AES256&amp;x-amz-storage-class=STANDARD_IA
content-length:123
content-type:image/jpg
host:s3.ap-southeast-1.amazonaws.com
x-amz-content-sha256:

content-length;content-type;host;x-amz-content-sha256
UNSIGNED-PAYLOAD</CanonicalRequest>
	<CanonicalRequestBytes>50 55 XX XX</CanonicalRequestBytes>
	<RequestId>9DXXXXXXXXXX</RequestId>
	<HostId>DZ7XXXXXX</HostId>
</Error>

When I remove the x-amz-content-sha256 from headers, the request upload the file as expected, and I could test around providing wrong content-type or content-length so the request would fail(as expected) meaning the presigned_url does work when providing headers value.

2

Are you sure the content of the header in the request is exactly the same you create the signature for? No extra whitespace characters or different encoding?

Looking at the canonical request it seems like the value is just two newlines…

3

hello, sorry for late reply, I am sure the content of the sha256 is there but let me double check again and back to you later

4

hello, I have confirmed that the x-amz-content-sha256 value are there, so I modify ex_aws library locally in ExAws.Auth module to add IO.puts and IO.inspect and do mix deps.compile ex_aws then run my server and generate the presigned url to test the upload, I saw the value being output along with other headers.

def presigned_url(
        http_method,
        url,
        service,
        datetime,
        config,
        expires,
        query_params \\ [],
        body \\ nil,
        headers \\ []
      ) do
    with {:ok, config} <- validate_config(config) do
      service = service_name(service)
      signed_headers = presigned_url_headers(url, headers)

      uri = URI.parse(url)
      uri_query = query_from_parsed_uri(uri)

      IO.puts "+++++++"
      IO.inspect headers
      IO.puts "+++ signed +++"
      IO.inspect signed_headers

      org_query_params =
        Enum.reduce(query_params, uri_query, fn {k, v}, acc -> [{to_string(k), v} | acc] end)

      amz_query_params =
        build_amz_query_params(service, datetime, config, expires, signed_headers)

      query_to_sign = (org_query_params ++ amz_query_params) |> canonical_query_params()

      amz_query_string = canonical_query_params(amz_query_params)

      IO.puts "+++ query to sign +++"
      IO.inspect query_to_sign

      query_for_url =
        if Enum.any?(org_query_params) do
          canonical_query_params(org_query_params) <> "&" <> amz_query_string
        else
          amz_query_string
        end

      path = url |> Url.get_path(service) |> Url.uri_encode()

      signature =
        signature(
          http_method,
          url,
          query_to_sign,
          signed_headers,
          body,
          service,
          datetime,
          config
        )

      {:ok,
       "#{uri.scheme}://#{uri.authority}#{path}?#{query_for_url}&X-Amz-Signature=#{signature}"}
    end
  end

output

... more output on top omitted ...

+++++++
[
  {"content-length", 35719},
  {"content-type", "image/jpg"},
  {"x-amz-content-sha256",
   "f501d1054318366d02808221e4f7b94fd8aaacd2ad08e8699371323f7f9d125c"}
]
+++ signed +++
[
  {"content-length", 35719},
  {"content-type", "image/jpg"},
  {"host", "s3.ap-southeast-1.amazonaws.com"},
  {"x-amz-content-sha256",
   "f501d1054318366d02808221e4f7b94fd8aaacd2ad08e8699371323f7f9d125c"}
]
+++ query to sign +++
"X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIA2XGNGWKQLAGMRKRM%2F20220126%2Fap-southeast-1%2Fs3%2Faws4_request&X-Amz-Date=20220126T171506Z&X-Amz-Expires=16&X-Amz-SignedHeaders=content-length%3Bcontent-type%3Bhost%3Bx-amz-content-sha256&x-amz-acl=private&x-amz-server-side-encryption=AES256&x-amz-storage-class=STANDARD_IA"

... more output below omitted ...
5

I was talking about the request you’re making to the resulting URL (the upload). Make sure that the actual request also contains the header with the exact same value (no extra whitespaces etc.)

6

hello, thanks I was indeed forgot to pass x-amz-content-sha256 when using the upload url to do the PUT request, but the error is still the same after I copy over the sha256 and add the header when i do the PUT request. I have tried with content-md5 instead and calculate the hash with openssl dgst -md5 -binary /path/to/file.jpg | base64 and it work, maybe the way I encode the sha256 wrong? for sha256 I simply just take sha256 of it in hex output without base64 using this command openssl dgst -sha256 /path/to/file.jpg and copy only the hash part