In a bit of a continuation of my last post Bitstring with codepoints of non-ASCII characters, I’m trying to understand why the following issue with Poison happens.

I am uploading a PDF to AWS S3 with the PutObject action from the S3 API through the ExAws.S3.put_object/4 function. I have used this many times before and hadn’t stumbled upon this problem. The complications here arise from the fact that the PDF is received from an email from AWS SES that is first uploaded raw (the whole email) to AWS S3 separately. In it, the PDF attachment is base64 encoded. After decoding it and uploading it as a binary to AWS S3 I receive the following error:

** (FunctionClauseError) no function clause matching in Poison.Encoder.BitString.chunk_size/3
        (poison) lib/poison/encoder.ex:139: Poison.Encoder.BitString.chunk_size(<<255, 255, 255, 255, 255, 255, 255, 255, 255, 147, 97, 80, 94, 84, 131, 5, 40, 48, 108, 15, 6, 230, 195, 5, 108, 23, 42, 195, 87, 247, 13, 72, 192, 100, 255, 106, 8, 31, 251, 80, 64, 255, 216, 80, 131, 255, 13, 66, 15, 253, ...>>, nil, 0)
        (poison) lib/poison/encoder.ex:134: Poison.Encoder.BitString.escape/2
        (poison) lib/poison/encoder.ex:95: Poison.Encoder.BitString.escape/2
        (poison) lib/poison/encoder.ex:136: Poison.Encoder.BitString.escape/2
        (poison) lib/poison/encoder.ex:95: Poison.Encoder.BitString.escape/2
        (poison) lib/poison/encoder.ex:136: Poison.Encoder.BitString.escape/2
        (poison) lib/poison/encoder.ex:95: Poison.Encoder.BitString.escape/2
        (poison) lib/poison/encoder.ex:136: Poison.Encoder.BitString.escape/2

I should mention that I’ve tried with two different PDFs and the bitstring in the exception is, I suppose naturally, different in those cases, but everything else behaves in the same manner.

I have tried to pinpoint exactly where it happens and log messages after each function call and mostly this was happening in the logs after the ExAws.request!/2 calls, but I there were also cases when it was logged even before ExAws.S3.put_object/4, but I suppose this might be related to some Logger details. In general, I think it happens after the ExAws.S3.request!/2 call.

BUT, nevertheless, everything works as expected - the PDF is uploaded to AWS S3 as a binary and is a valid PDF, and everything after this point also works fine. What is even stranger is that our Phoenix application is configured to use Jason and ExAws is also configured to use Jason, but I still get the Poison exception. Which leads me to think that it might be from another library, but this means that I have even less of a clue as to what is going on, as ExAws is the only library that is used in this functionality, although there might be a bug with it, I guess.

Still, I always get this Poison exception. I assume it has something to do with the encoding of the email. I believe that the encoder expects utf-8 encoded data, but I’m not sure why the Poison encoder is called here in the first place and why this exception is raised even though the binary is a valid PDF and I have not had any problems using the same functionality to upload other PDFs in the past.