Port of Java to Elixir for HMAC signature generation causes invalid security header error in Apigee API

earthtrip

Hi all

I’m banging my head against the wall on this.. I have to port a bit of Java code over to Elixir/erlang and can’t seem to get this working.. I mean I can get my code to output base64 encoded strings but the server is rejecting them as invalid.

This is the function in question:

    private static String generateMacSignature(final String payload, final String resourceURI, final String host,
					       final String port, final String macId, final String key, final String httpMethod) {
    	String signature = null;
    	try {
	    String timestamp = Long.toString(System.currentTimeMillis()).trim();
	    String nonce = UUID.randomUUID().toString().trim();
	    String bodyHash = encode(payload, key);
	    // Create MAC input string
	    String macInput = timestamp + "\n" + nonce + "\n" + httpMethod + "\n" + resourceURI + "\n" + host + "\n"
		+ port + "\n" + bodyHash + "\n";
	    String encodedMacInput = encode(macInput, key);
	    String macRequestAuthFmt = "MAC id={0},ts={1},nonce={2},bodyhash={4},mac={3}";
	    String[] authHeaderInputs = new String[]{"\"" + macId + "\"", "\"" + timestamp + "\"", "\"" + nonce + "\"",
						     "\"" + encodedMacInput + "\"", "\"" + bodyHash + "\""};
	    signature = MessageFormat.format(macRequestAuthFmt, authHeaderInputs);
    	} catch (Exception e) {
	    System.err.println("Exception while generating MAC Signature: " + e.getMessage());
    	}
    	return signature;
    }

    private static String encode(String data, String key) throws Exception {
    	String encodedData = null;
    	try {
	    // get an HmacSHA256 signing- key from the raw key bytes
	    SecretKeySpec signingKey = new SecretKeySpec(key.getBytes(), "HmacSHA256");
	    // get an HmacSHA256Mac instance and initialize with the signing key
	    Mac mac = Mac.getInstance("HmacSHA256");
	    mac.init(signingKey);
	    // compute the hmac on input data bytes
	    byte[] rawHmac = mac.doFinal(data.getBytes());
	    // base64-encode the hmac
	    encodedData = new String(Base64.encodeBase64(rawHmac)).replace("\r\n", StringUtils.EMPTY);
    	} catch (Exception e) {
	    throw e;
    	}
    	return encodedData;
    }

I’ve basically tried a few different variants of this based on some posts I’ve seen here and SO as well as asking GPT4 and haven’t been able to get the server integration to work.

Here’s my basic function but I’m not completely sure how those Java libraries work or what they’re doing that’s different.

  defp encode(data, secret) do
    :crypto.mac(:hmac, :sha256, secret, data) |> Base.encode64
  end

Thanks for any assistance!

16 comments

#crypto

0 670 16

2024-05-15 17:27:55 UTC

Showing Posts 1 to 10

Show Best Posts
Show All Posts (oldest first)
Show All Posts (newest first)

earthtrip

FWIW - I also tried using this in my encode method to no avail

Plug.Crypto.MessageVerifier.sign(data, secret, :sha256)  |> Base.encode64()

Post #1

katafrakt

Just from the top of my head, try Base.encode64(string, padding: false)

Post #2

earthtrip

Thanks @katafrakt . I forgot to mention I tried that as well but no luck. I also tried Base.url_encode64 and Base.url_encode64(padding: false)

Post #3

al2o3cr

This looks like the Apache Commons base64 encoder, which doesn’t put newlines in the output unless requested with an additional argument isChunked so it’s peculiar that it’s doing extra work to remove them.

+1 for checking padding.

Beyond that, it would help a lot to see some “good” and “bad” outputs for comparison.

Post #5

earthtrip

Thanks @al2o3cr . I didn’t notice it was using commons - but you’re right. Let me investigate that a bit more.

Post #9

earthtrip

I just updated the original question with the signature generation code (which now that apache commons was mentioned you can see how it’s adding newlines in the macRequestAuthFmt string. I tried removing the newlines to see if that would work but I still received an error.

Post #10

earthtrip

@al2o3cr here’s a sample that’s valid and I was able to get a response from the API server and the elixir one that’s invalid and returns Invalid Security Header error.

MAC id="SECRET",ts="1715726915441",nonce="3ac98550-dde0-4a13-b5d6-501b422bb6ea", bodyhash="iaAUxVAlo6tYuNomux+O5JdzENHXTljcZeTJ9bL8yCw=", mac="EJlFvXdr3AJ2DV+nsUFotiY4IjeWK4QkxqVd+Rojs+M="

and an invalid one from elixir

MAC id="SECRET",ts="1715727001889",nonce="00775b0b-b353-4848-a923-24f8defae0b3", bodyhash="3XbUmkO3KwtcWd/U9y9Ab3IPfdA5rwSXicxHiaiB3RI=", mac="oEGHhmuJD8aJNHkvHGn0RE6rTBicS756AT6EoNTYDpU="

Post #11

al2o3cr

To clarify, I meant “good” and “bad” outputs for the same request. Those two don’t have the same bodyhash value…

What we can spot from the “good” sample:

the output is expected in the standard base64 alphabet (with +, versus the “url-safe” one that substitutes -)
the output should include base64 padding (the trailing =s)

Together those mean that Elixir’s Base.encode64 will work with no additional options.

HOWEVER

The base64 flavor is the least of the problems. HMAC is explicitly designed to change a lot (and unpredictably) for even a single-bit difference in its input, so any mis-formatting before the HMAC will produce wrong values that don’t provide any clues to the problem.

This isn’t a “try it until it works” situation; the results of the failures aren’t going to give any feedback about why things are wrong. Can you post a link to the documentation for the specific API you’re trying to call?

As for troubleshooting the whole thing, you’ll want to capture the inputs & outputs of the pieces of generateMacSignature for comparison with the reimplementation:

what is the value of payload? Your Elixir code should be able to produce an identical bodyhash value for the same payload.
what is the exact value (newlines included!) of macInput? Again, your Elixir code needs to produce the same bytes given the same inputs

Post #12

Hermanverschooten

I would start by printing all intermediate values in both the original and the elixir code, arriving with what is passed to the base64 encoding, checking for differences along the way. eg what does the SecretKeySpec do in regards with just passing the key to :crypto.mac/4, etc.

Post #13

earthtrip

Thanks everyone for the continued assistance. I moved to using a javascript sample that I can run in postman much easier and play around with. I have that working against the API. In my tests I did find that Erlang has the secret/payload swapped compared to how it needs to work using the CryptoJS library so I’m investigating that.

I’m still getting Invalid Security header but will start going through line by line and outputting and comparing.

Post #14

Load more posts (6 remaining)