Erlang bit syntax help for myxql geo support

jeroenbourgois

For a project we work on we need geo support in myxql. Instead of waiting and complaining why it is not there, we decided to step in and try to add it. Having little to no understanding of bit syntax, this has already been quite a journey for me. I started out with the Mariaex code as a basis, got into the MySQL and PostGIS documenation, and with a lot of trial and error, now already able to parse a Point and a Polygon. The big goal for our own project is to get MultiPolygon support.

I am trying to figure out this line, that concerns itself with parsing the rings inside a polygon, more specifically the value of the ‘points’ variable, how I should interpret the ‘binary-size’ and ‘unit’ keywords. The actual value is the amount of points inside a polygon. It is needed to extract the correct amount of bits from the binary. I am trying to decode a MultiPolygon and used this as a reference:

<<num_points::32-little, points::binary-size(num_points)-unit(128), rest::bits>>

I found some IBM documentation as well to explain the 32 bit ‘num_points’:

wkbMultiPolygon {
  byte              byteOrder;
  uint32            wkbType;     // 6=wkbMultiPolygon
  uint32            num_wkbPolygons;
  WKBPolygon        wkbPolygons[num_wkbPolygons];
};

In order to decode the multipolygon, I guess I also have to read out the ‘num polygons’ from the binary. Getting hold of the amount of polygons works, but the next step is thus unclear, knowing how far to read.

Other documenation about the actual internal bit format is hard to find so it seems. Any help is appreciated!

PS: if you are interested and have knack for bit syntax stuff, feel free to join the fork! We already have one volunteer which is great!

https://github.com/jackjoe/myxql/

4 comments

#erlang #myxql

0 1002 4

2019-11-21 20:10:56 UTC

Showing Posts 1 to 4

Show Best Posts
Show All (oldest first)
Show All (newest first)

OvermindDL1

binary means that points will be a binary, size(...) means that the binary will have the size of num_points worth of bytes (8-bits) (where num_points was parsed previously as a 32-bit integer of little-endian order), and the unit(...) will redefine the unit size of size(...) to instead of being 8-bits will instead be 128-bits, so points will be a binary of num_points amount of points where each point is 128 bits in size (maybe some other internal structure?). The rest is just whatever’s left over unparsed as a bitstring.

For note, all documented at: Kernel.SpecialForms — Elixir v1.20.2

Post #1

wojtekmach

Hex Core Team

Thank you for working on this feature!

I’m not clear if you need learning resources for binary pattern matching, representing GIS data in the mysql protocol, or both?

If you already have a way to decode a single polygon then maybe try using it num_points times on the given binary.

Post #2

jeroenbourgois

Thanks, this already helps! The 128 bits will be the size of one point, which is {x,y} where each element is a 64-bit float. Thanks to point to the docs too.

I will ping the topic once we get further.

Post #3

jeroenbourgois

@OvermindDL1 thanks to work from our side and @wojtekmach we managed to land a version of myxql with geo support. Just wanted to let you and the topic know