fireproofsocks

fireproofsocks

Where did the name "binaries" come from? And how does this relate to Base2

I’ve been diving deep into charlists, encodings, code points, etc… thank you to the community who have humored me in the forums and in Slack.

I have a question about Elixir’s use of the term “binary”. It seems like the term was bogarted… if I showed you a series of 1’s and 0’s, you’d think “Ah, that’s binary.” As in, that data is being represented in a binary format.

But nope, in Elixir, a binary is for all intents and purposes synonymous with a string in other languages. But in Elixir, we don’t take “binary” to mean a series of 1’s and 0’s, but rather, a binary means “a sequence of bytes” (i.e. yes, a sequence of 1’s and 0’s, but specifically, groups of 8 of them).

This has the weird side-effect of making us refer to ACTUAL series of 1’s and 0’s as “Base2” encoding, which isn’t supported in the core, but is available in a little-known package: base2 | Hex

Can someone enlighten me on this?

Thanks!

Marked As Solved

rvirding

rvirding

Creator of Erlang

I think we called it a binary just to separate from all other Erlang terms. It is just a collection, or array, of raw bytes where we add meaning to it. If the number of bits is not divisible by 8 then it is a bitstring. Again we are the ones who add meaning to it not Erlang/Elixir/BEAM.

Remember that in Erlang we mostly represent strings as lists of integers. Again we put meaning to the bytes, whether they are ascii/latin1/unicode codepoints. We NEVER make lists of utf-8/16/32 encoded characters. What’s the point? There was a good talk on CodeBeamSF on this by Marc Sugiyama Unicode, Charsets, Strings, and Binaries. I don’t know when it will be on line.

Also Liked

al2o3cr

al2o3cr

These two are not the same - String.split with an empty string splits on graphemes, which are a further unit bigger than codepoints.

The rules for clustering codepoints into graphemes are defined by the Unicode standards, and the code for handling them is generated from canonical text files.

iex(4)> String.codepoints("🇺🇸")
["🇺", "🇸"]
iex(5)> String.split("🇺🇸", "", trim: true)
["🇺🇸"]
iex(6)> "🇺🇸" <><<0>>
<<240, 159, 135, 186, 240, 159, 135, 184, 0>>

The single displayed character :united_states: is a grapheme, composed of two codepoints U+1F1FA and U+1F1F8, represented by 8 bytes.

Another way that codepoints and graphemes can diverge is combining characters; for instance, U+0308 is “Combining Diaresis” which will add ¨ to the preceding character. Example:

iex(9)> s = "ca\u0308t"
"cät"
iex(10)> String.codepoints(s)
["c", "a", "̈", "t"]
iex(11)> String.split(s, "", trim: true)
["c", "ä", "t"]

(note that the combining character prints very oddly when isolated inside ")

NobbZ

NobbZ

  • bitstrings are everything what can be represented using <<>>
  • binaries are bitstringths which length is a multiple of 8
  • Strings are binaries which bytes represent valid utf-8 encoded codepoints.

First 2 terms are from Erlang

josevalim

josevalim

Creator of Elixir

FWIW, there is nowhere in the docs with that line. If they were, they would indeed be incorrect.

Other than that, you are right, codepoints are integers. String.codepoints returns codepoints as UTF-8 encoded binaries, i.e. codepoints as strings. The precise definition of this would be “code unit” but we wanted to avoid introducing yet another term. I have updated the docs to make it clear that String.codepoints returns an encoded representation, not integers. Thanks.

Where Next?

Popular in Questions Top

sen
Hi All, I set a environment variables in dev.exs , like below code. when i start server, how can i set the ${enable} value? thanks. d...
New
Harrisonl
We have an ECS cluster with 4 services, where each task joins a single cluster, via discovery ECS discovery service. Currently when I de...
New
Tee
can someone please explain to me how Enum.reduce works with maps
New
qwerescape
Is there a way to get the call stack or stack trace at any point in the code? Not from exceptions, but an expression that returns how the...
New
mgjohns61585
Could someone help me? I’m making my first elixir program, number guessing game. I can’t figure out how to convert the user’s guess from ...
New
ycv005
I have followed this StackOverflow post to install the specific version of Erlang. And When I am running mix ecto.setup then getting fol...
New
freewebwithme
Using vs code and installed ElixirLS: support and debugger. And I got an error popped up on start up says Failed to run ‘elixir’ comma...
New
srinivasu
How to handle excepions in elixir? Suppose i have A, B, C ,D, E modules. and each module has get() function. A.get() method will call t...
New
yawaramin
In the Dialyzer docs ( dialyzer — OTP 29.0.2 (dialyzer 6.0.1) ), there is a way to turn off a specific warning for a function: -dialyzer...
New
vonH
In asking this question I am more interested about the expressiveness of the language itself and less concerned about the availability of...
New

Other popular topics Top

vertexbuffer
Hello, can anybody help here..? I have a list of players and I what to delete an element, but every for loop the list is reverting to ori...
New
hariharasudhan94
lets say i have a sample like a = 20; b = 10; if (a &gt; b) do {:ok, "a"} end if (a &lt; b) do {:ok, b} end if (a == b) do {:ok, "equa...
New
marius95
Hello everyone, I try to use an Javascript Event Handler in my root.html.leex file. Therefore I created a function in the app.js file: ...
New
belgoros
I’m not a pro in using Regex and can’t figure out why the following behaviour happens, especially if we take into account the difference ...
New
AngeloChecked
What learn first? Rust or Elixir Hi Elixir community! I’m here because i want learn a new language. I’m a junior developer and mainly i ...
New
AstonJ
Please see the new poll here: Which code editor or IDE do you use? (Poll) (2022 Edition) It’s been a while since we first asked this, I...
208 31265 143
New
vonH
When I run the Plug and I recompile I wind up having to use Ctrl C to quit iex and start again. Witht the help of rlwrap I can use the cu...
New
grych
Hi folks, Few months ago I have announced the proof-of-concept of the library to manipulate the browsers DOM objects directly from Elixi...
639 52673 488
New
ashish173
I am using Ecto timestamps with postgres, I can see the timestamps() use the :naive_dateime but for my use case I wanted to store the ti...
New
lanycrost
Hi everyone! I need implement if…else if…else condition from my elixir code, and anymore of this control flow structures not work proper...
New

Latest on Elixir Forum

Elixir Forum

We're in Beta

About us Mission Statement