I’m communicating between Python and Elixir. Python reads my (Linux-standard, ssh-keygen with no options) ~/.ssh/id_rsa.pub, pem-encodes it, and sends it to Elixir via encrypted websocket. This is a (fake) version of it
pry(7)> key
"-----BEGIN PUBLIC KEY-----\nMIIBojANBgkqhkiG9w0BEH6ycVAgMBAAE=\n-----END PUBLIC KEY-----\n"
(of course it’s much longer)
Now how do I use this plaintext pem public key, to encrypt a string say, “hello python”, so that I can send the encrypted version back via the same websocket, to be decoded python side using the local corresponding private key ~/.ssh/id_rsa.
Basically I’m looking for some way to get an industry standard cross-language public-private key interaction between Elixir and Python. There’s a lot on the web about pem vs der but I’m a bit confused by it.
(for guide I’m basically rolling my own python ↔ elixir authentication mechanism. If something that does this already exists, then happy to hear about it)
I completely get what you mean @vegabook. I too struggled longer than I’d like to admit before I got my assymetric RSA encryption working. Please see below for a module I wrote, that hopefully explains things well enough.
defmodule Ls.Crypto.Assymetric do
@moduledoc """
Functionality releated to assymetric encryption, e.g. public key encryption.
Create the keys used here in the following way:
# Public key.
openssl genrsa -out private.pem 2048
# Private key.
openssl rsa -in private.pem -pubout -out public.pem
"""
@encrypt_decrypt_opts [rsa_padding: :rsa_pkcs1_oaep_padding]
def encrypt_rsa(text, key_path) do
key = load_key(key_path, :RSAPublicKey)
:public_key.encrypt_public(text, key, @encrypt_decrypt_opts)
end
def decrypt_rsa(cipher_text, key_path) do
key = load_key(key_path, :RSAPrivateKey)
:public_key.decrypt_private(cipher_text, key, @encrypt_decrypt_opts)
end
defp load_key(key_path, expected_key_type) do
{:ok, raw_key} = File.read(key_path)
[enc_key] = :public_key.pem_decode(raw_key)
pem_entry = :public_key.pem_entry_decode(enc_key)
# Check that we loaded a key that's of the expected type (private vs public).
# It's an easy mistake to load the wrong key, so this check can
# save a lot of debugging time.
if elem(pem_entry, 0) != expected_key_type do
raise("Expected a key of type #{expected_key_type}, got #{elem(pem_entry, 0)}")
end
pem_entry
end
end
Caveats
A .pem file is a container format, so it can contain many different things. This is why we need to do the :public_key.pem_decode/1 and :public_key.pem_entry_decode/1. My code assumes the PEM file contains just a single key.
It works on my machine and for my particular use case. It’s definitely not the most general, universally applicable way of doing things .
This works for strings up to about the size of the key, then doesn’t. You need a box. The most ubiquitous box JWE (encryption) which is supported by jose. JWK does your PEM decoding and key management. It’s not easy to figure all out, but nothing with crypto is.
Yeah as much as I had “fun” working on the self-rolled solution, there are just too many permutations on multiple dimensions (data structures, string encodings, cryptography algorithms, key formats, etc) that it’s becoming a cartesian explosion of trial and error. I got the python to elixir side working in both stacks, but the return side (back in python) has me flummoxed. JOSE appears to be exactly what I need - a set standards with actual implementations in a bunch of languages including of course, elixir, and python. Though of course, as is typical, python has too many implementations :-/
With all due thanks to @ErikNaslund I’m going to go with this as a solution.
I had the exact “fun” you’re talking about. Crypto is really interesting, deep and complex. Learning more about how any part of it works is well worth the effort (or rabbit hole). Good luck!
That sounds like a great idea @vegabook! Using something standardized / well thought out that someone else spent a lot of time thinking about is almost always the way to go. It’s way too easy to miss out on some subtleties otherwise, especially when it comes to security and cryptography.
Like @danj wisely pointed out - using asymmetric encryption limits your data to approx. the size of the key. If you want to send more data you need to mix in symmetric encryption (e.g. AES) as well and encrypt the AES key using RSA. Suddenly you need to make sure you don’t make yourself vulnerable to silly things like the two-time-pad attack etc etc. TL;DR - it gets hairy very quickly! I don’t even know this stuff particularly well, just well enough to be dangerous .
I was re-reading your original question a bit more properly just now…
If what you really want it a secure and authenticated websocket link between Python and Elixir…I’m thinking about it like this.
A websocket connection always starts out with a normal HTTP connection that then gets UPGRADEd to a websocket connection.
HTTPS has pretty solid security built in. Could you perhaps simply use server + client certificates, and make sure you use HTTPS / WSS connections. This way the HTTPS/Websocket libraries you use should take care of all the heavy lifting when it comes to securing the connection, and authenticating the user.
I don’t really know your use case that well, so this idea might make no sense at all
If you’re not completely married to the idea of websockets, but still need bidirectional communications, then there’s always the option of going with something like Postgres listen/notify (pubsub), Redis Pubsub or something like ZeroMQ.
If you want something more “ready made” and opinionated it might also make sense to look at gRPC or Thrift. They’re creates specifically for cross-language function calls, and they can come in very handy at times.
If you don’t even need the bidirectional bit then good ol HTTPS with client & server certificates probably makes a lot of sense.
Yes I’ll probably lean into the Phoenix stack’s authentication methods. Yep once I’m authed wss will sort everything out in terms of privacy.
Unfortunately Bloomberg does not have API access from Linux or Mac unless purchasing one of their multi-tens-of-thousands dollars per month “Bpipe” services, so this library will serve the dual purposes of talking Elixir, but also talking Linux.
Once the communication is setup, then there’s a lot of chatter between the two because of the large numbers of streaming data going around, so websockets work best. Thanks for the hints though. Have used ZeroMQ before (local network) and it was very good. Actually the UDP broadcast stuff was super useful in ZeroMQ for local service discovery. But in this case I want the option of traversing over the internet so ZeroMQ is not on the menu unless I were to do autossh reverse port forwarding which would introduce yet another point of failure.
Ah you’re interactiving with Bloomberg over the internet - gotcha!
Cool that you used ZeroMQ before. I only suggested it because most people I know never heard of it, and much less used it .
But you’re 100% right - it won’t be a good solution in this case. Seems like you have a pretty good plan on how to do it. Good luck, hope it works out great!
.
