Help SWAR optimize URI.encode_www_form/1

Hello there!

Oh I believe it was me who some time ago took the first SWAR optimisation into OTP :json module. So disclaimer, I think it was me who opened this pandora box

  @swar_wordsize :erlang.system_info(:wordsize)

This is hardcoded at compile time, so compiling on a 64bit and then running the BEAM files on a 32bit machine, or the other way around, would get the whole optimisation wrong. The weirdest edge case, but still worth keeping in mind. I believe I just hardcoded 7b ints in both OTP and Elixir, so actually maybe I even made 32bit machines a bit slower

  defp encode_unreserved(string, :percent)
       when @swar_wordsize != 8 or byte_size(string) < @swar_threshold do

I’d make that 8 also an attribute with a name, something like @64bit or whatever, just to avoid magic numbers.

  def encode_www_form(string)
      when is_binary(string) and
             (@swar_wordsize != 8 or byte_size(string) < @swar_threshold) do
    for <<byte <- string>>, into: "" do
      case percent(byte, &char_unreserved?/1) do
        "%20" -> "+"
        percent -> percent
      end
    end
  end

Now something cooler, the original code. I checked the generated assembly, and actually, the biggest difference is in the loop through the binary. When the code is looping byte by byte, it is doing that snippet above, which compiles to an Erlang’s bitstring comprehension, with a function call, a new allocated binary (the output of percent, and a case-do comparison, that the new code is not doing, and that is actually the largest performance saving and something you could write on the @swar_wordsize != 8 or byte_size(string) < @swar_threshold clause as well and it would also improve massively. Because you didn’t try to do a for-loop with bigger binary slots but instead you pattern-matched those bigger binary slots into clauses, here

  defp encode_unreserved(<<word::56, rest::binary>>, acc, mode)
       when unreserved_word?(word) do
    encode_unreserved(rest, <<acc::binary, word::56>>, mode)
  end

the function becomes first of all tail-recursive without intermediate function calls nor intermediate garbage.

When it comes to the SWAR code itself, well, all those indexes are the most confusing thing on earth so can’t say I’ve carefully checked right now, but might do if you put this into a PR! encode_unreserved gets ugly with so many quite repetitive clauses, but I guess that’s the trade-off of an obsessive optimisation. Maybe the clauses browsing the bad byte can be autogenerated? Claude suggested me this:

# The word guard failed, so one of bytes 2..7 is disallowed. Emit through the
# first one, so the next stride starts past it instead of re-checking
# overlapping words.
for n <- 2..7 do
  leading = Macro.generate_arguments(n - 1, __MODULE__)

  defp encode_unreserved(<<unquote_splicing(leading), byte, rest::binary>>, acc, mode)
       when byte_size(rest) >= unquote(7 - n) and not unreserved_char?(byte) do
    encode_unreserved(
      rest,
      <<acc::binary, unquote_splicing(leading), encode_unreserved_byte(byte, mode)::binary>>,
      mode
    )
  end
end

Thank you for your feedback, I see if I can make this better based on this.