dimitarvp
So I have this callback endpoint where I accept a delayed status update for jobs that I tasked a remote server with. I have made a table that only has ID, data field (jsonb type) and inserted_at and just store the response with zero processing. Here’s what error I get upon doing Repo.insert:
** (Postgrex.Error) ERROR 22P05 (untranslatable_character) unsupported Unicode escape sequence
\u0000 cannot be converted to text.
Reason is that the inspect-ed POST form parameters look similar to this (redacted for privacy):
%{
"stuff" => "246",
"time" => "1728063410",
"inqueue" => "2",
"someid" => "26",
"outqueue" => "6",
"binarystuff" => <<113, 0>>
}
I believe the last value trips it up. I wrote a test to reproduce it but that’s still just using Elixir code to POST the payload.
Two questions:
- How do you even
curlsuch a payload to an HTTP endpoint? Namely with one form parameter containing a byte array. - How would you serialize a byte array inside a
jsonbfield?
Where Next?
Trending in Questions
Other Trending Topics
Latest Phoenix Threads
Latest on Elixir Forum
Sponsor Spotlight
Supporting innovation across the BEAM ecosystem.
Categories:
Sub Categories:
Forums
Popular Tags
- #ecto
- #liveview
- #troubleshooting
- #learning-elixir
- #library
- #deployment
- #erlang
- #testing
- #genserver
- #mix
- #absinthe
- #remote-other
- #otp
- #plug
- #how-to-question
- #macros
- #postgres
- #elixirconf
- #channels
- #exunit
- #discussion
- #code-sync
- #podcasts
- #javascript
- #onsite
- #dialyzer
- #docker
- #authentication
- #umbrella
- #full-time-contract
- #podcasts-by-brainlid
- #ecto-query
- #ai
- #elixirconf-us
- #blog-post
- #elixir-ls
- #phoenix_html
- #iex
- #graphql
- #genstage
- #websockets
- #supervisor
- #advent-of-code
- #distillery
- #processes
- #api
- #forms
- #elixirconf-eu
- #metaprogramming
- #hex
Our Sponsors
Supporting innovation across the BEAM ecosystem.
We build reliable cloud platforms for business-critical systems.
Catch errors, track performance, monitor hosts and more.
Error tracking, logs, uptime, and Just Enough APM. Built for developers.
Elixir, Erlang & BEAM conferences from Code Sync - join us in 2026!
Master modern tools, write cleaner code, and tackle real-world challenges.
Producing high quality Elixir screencasts since 2017.
The deployment platform built for Elixir: PaaS ease, VPS control.
The team behind Membrane, Popcorn, LiveDebugger. Available for hire.
Courses that'll move you from confusion to "Aha, now I get it!"
Showing Posts 1 to 4- Show Best Posts
- Show All (oldest first)
- Show All (newest first)
sbuttgereit
I haven’t futzed with the Elixir side of this, but on the PostgreSQL side I’d either use:
encode(PostgreSQL: Documentation: 17: 9.5. Binary String Functions and Operators) or (in a pinch) something along the lines ofSET bytea_output = 'escape';in the transaction.Not sure that helps at all in this case, but that’s the gist.
al2o3cr
FWIW, there appears to be a hard limitation on PG’s end regarding
\u0000values in strings insidejsonb:Usual practice for this situation would be to base64 the value somewhere in the dataflow; doing it at the origin might be easiest since then you’re always
POSTing “normal” characters.dimitarvp
Well, there are two options:
<<1, 2>>to itsinspect-ed form i.e.""<<1, 2>>". That introduces a specially-treated value you have to look for and I hate it because it’s easy to forget to do so.jsonbtojsonbecause that one allows\u0000’.@al2o3cr Thanks, I found the same, also this: https://stackoverflow.com/questions/68303357/how-to-insert-json-containing-escape-codes-into-a-jsonb-column-in-postgresql-usi
So I opted for solution #2 and it all worked immediately.
Oh, and as for
curl:Got super ambitious because some of my skills proved a bit dusty but quickly refreshed them.
dimitarvp
Maybe, but I have no clue how to specially treat base64… I can put a prefix and treat the value specially but this is just error-prone, especially if you are scrambling to cancel and then retry an Oban job in a production k8s pod.
It’s a project from another person in the company and I am reasonably sure he will not change a thing to pander to my fragile little HTTP callbacks.
But yeah, just switching from
jsonbtojsonworked.