cgrimmett

cgrimmett

Creating a BEP3 compatible percent encoder (URI encoder)

I would like to announce torrents to my opentracker using an Elixir program. I have a test qbittorrent and opentracker running and I’ve identified how qbittorrent sends it’s announces to opentracker.

METHOD: GET
URL
http://localhost:8000/announce?info_hash=%ac%c3%b2%e43%d7%c7GZ%bbYA%b5h%1c%b7%a1%ea%26%e2&peer_id=-qB5020-r3FSX0qNU6Oo&port=4993&uploaded=0&downloaded=0&left=0&corrupt=0&key=F7F879A3&event=started&numwant=200&compact=1&no_peer_id=1&supportcrypto=1&redundant=0
HEADERS
Accept-Encoding:
gzip
Connection:
close
Host:
localhost:8000
User-Agent:
qBittorrent/5.0.2

I would like to build this same GET request into my own code. My confusion comes from the percent encoded info_hash. Neither URI.encode/2 nor URI.encode_www_form/1 encode the URI the same way that qbittorrent does it.

Let me show you what I’ve tried so far.


# We start with the `Info hash v1` of the torrent, as copied from qBittorrent.
info_hash_v1 = "acc3b2e433d7c7475abb5941b5681cb7a1ea26e2"

# Next we decode the hexadecimal representation into binary.
binary = Base.decode16!(info_hash_v1, case: :lower)

I’m not sure what comes next. Digging through Elixir URI source code, I can see the method of percent encoding that creates a binary in almost the right format.

https://github.com/elixir-lang/elixir/blob/78f63d08313677a680868685701ae79a2459dcc1/lib/elixir/lib/uri.ex#L428C7-L428C13

The problem here is that the hex/1 function only outputs 16 possible values, A-F, 0-9. There is no lowercase! The way qbittorrent is doing their percent encoding, they have lowercase too-- a-f, A-F, 0-9.

I read qbittorrent source code to see how they’re percent encoding, but I couldn’t find the code that does that. I found some info_hash references though. I can barely read C++, but I think the url encoding might be abstracted away in a request library.

I also looked through transmission-qt code. That code is greek to me, but I was able to find their percent encoder implementation.

https://github.com/transmission/transmission/blob/87bcf1a1d5bc60070315335a85fb8526843b28ac/libtransmission/web-utils.h#L100

Seems similar to what I read in Elixir URI source code related to unreserved and unescaped characters. RFC 3986 - Uniform Resource Identifier (URI): Generic Syntax

I bounced some ideas off of ChatGPT, borrowed code from Elixir URI, and I got a bep3_encode/1 function put together.


  @doc """
  Encodes `string` as BEP3's weird URL encoded string.

  ## Example

      iex> bep3_encode("a88fda5954e89178c372716a6a78b8180ed4dad3")
      "%A8%8F%DAYT%E8%91x%C3rqjjx%B8%18%0E%D4%DA%D3"

  """
  @spec bep3_encode(binary) :: binary
  def bep3_encode(string) when is_binary(string) do
    string = Base.decode16!(string, case: :lower)
    URI.encode(string, &URI.char_unreserved?/1)
  end

Given an Info hash v1 input of acc3b2e433d7c7475abb5941b5681cb7a1ea26e2, the expected output should be as follows.

%ac%c3%b2%e43%d7%c7GZ%bbYA%b5h%1c%b7%a1%ea%26%e2

However, I haven’t figured out how I preserve the case on the various letters. I assume I need to have exactly the same case-sensitive output as qBittorrent because an ASCII A is not the same as a.

It seems like the way qbittorrent does it, and I apologize for not having the language to communicate this.. hex values that can be displayed as ASCII are output as ASCII. Otherwise, the hex representation is displayed.

I wrote this out to make a visual comparison of expected input and output values, because this is the only way I could think of understanding what is happening during the encoding.

 a8   8f   da   59   54   e8   91   78   c3   72   71   6a   6a   78   b8   18   0e   d4   da   d3
%A8  %8F  %DA   Y    T   %E8  %91   x   %C3   r    q    j    j    x   %B8  %18  %0E  %D4  %DA  %D3

On the second line (expected output) See the ASCII Y? That matches up with Hex 54. The three values before that were all not able to be displayed as ASCII, so the hex value was used instead.

Later on, we can see lowercase x, r, q, j, j, x. From what I can tell, Elixir’s built-in URI.encode/1 can’t do this, because like I mentioned earlier, that can only output A-F,0-9. No lowercase!

By the way, https://www.asciitohex.com/ has been very helpful.

Anyway, I am very confused. I’m going to rest now and pick this up in the morning.

Marked As Solved

cgrimmett

cgrimmett

Eureka!

defmodule App.BittorrentUrlEncoder do
  @moduledoc """
  URL encoding for Bittorrent Info hash v1. Designed to be compatible with qBittorrent's percent encoding.
  """

  import Bitwise

  @doc """
  Encodes `string` as a Bittorrent-flavored percent-encoded string.

  ## Example

      iex> encode("a88fda5954e89178c372716a6a78b8180ed4dad3")
      "%a8%8f%daYT%e8%91x%c3rqjjx%b8%18%0e%d4%da%d3"

  """
  @spec encode(binary()) :: binary()
  def encode(hex_string) when is_binary(hex_string) do
    hex_string
    |> Base.decode16!(case: :lower) # Decode from hex to raw bytes
    |> encode_bytes()
  end

  defp encode_bytes(<<>>), do: ""

  defp encode_bytes(<<byte, rest::binary>>) do
    percent_encode(byte) <> encode_bytes(rest)
  end

  defp percent_encode(byte) when byte in ?0..?9 or byte in ?a..?z or byte in ?A..?Z or byte in ~c"~_-.!" do
    <<byte>>
  end

  defp percent_encode(byte) do
    "%" <> <<hex(bsr(byte, 4)), hex(band(byte, 15))>>
  end

  defp hex(n) when n <= 9, do: n + ?0
  defp hex(n), do: n + ?a - 10
end

The secret sauce was changing ?A to ?a in the hex/1 function

-defp hex(n), do: n + ?A - 10
+defp hex(n), do: n + ?a - 10

Also Liked

al2o3cr

al2o3cr

This is not true, as trying it will demonstrate:

iex(1)> s = <<0xA8, 0x59, 0x72>>
<<168, 89, 114>>

iex(2)> URI.encode(s)
"%A8Yr"

Where Next?

Trending in Questions Top

lanycrost
Hi everyone! I need implement if…else if…else condition from my elixir code, and anymore of this control flow structures not work proper...
New
senggen
Erlang/OTP 25 [erts-13.2.2] [source] [64-bit] [smp:8:8] [ds:8:8:10] [async-threads:1] 15:22:35.803 [error] gen_event {lager_file_backend...
New
hariharasudhan94
Lets say I have map like this fetching from my database %{"_id" =&gt; #BSON.ObjectId&lt;58eb1a7a9ad169198c3dXXXX&gt;, "email" =&gt; ...
New
tj0
I’ve been following the steps here for the upgrade from 1.6 to 1.7 and it has gone relatively smoothly all the way till the phoenix_view ...
New
cgraham
Hi! What is currently the best library/method for parsing text and tabular data out of PDF files in Elixir or Erlang?
New
stefanchrobot
Hi, I need a way to handle data migrations in my application. I found an article by @wojtekmach about manual migrations: Automatic and ma...
New
stjefim
Hello! Suppose you are building workflow (order / task / payment) processing system with the following requirements: Each workflow con...
New

Other Trending Topics Top

GenericJam
Edit: 2026 May 15 - This post is archived. Mob is alive!! Main docs: mob v0.7.11 — Documentation A bit of explanation for the slightly c...
New
JesseHerrick
Hey, I’m Jesse and I’m the main contributor behind Dexter, a full-featured, lightning-fast Elixir LSP optimized for large codebases. It s...
New
kip
Localize is the next generation localisation library for Elixir. Think of it as ex_cldr version 3.0. The first version will be released ...
New
webofbits
Squid Mesh is an open source workflow automation runtime for Elixir applications. It is aimed at Phoenix and OTP apps that want to defin...
New
jimsynz
Beam Bots (or just BB for short) is a framework for building fault-tolerant robotics applications in Elixir using familiar OTP patterns. ...
New
kip
In 2021 I started a new library called Tempo with the objective of modelling time as a set of intervals - not as instants. In 2022 I gave...
New

We're in Beta

About us Mission Statement