ibGib - a different approach to code and data

Like I’ve said many times, ibGib is tough to nail down…but I’m starting a new video series called Clear A Path, wherein I start to talk about ibGib, distributed architecture and related concepts, aaaaaaand…the Bible.

In this series, I plan on at least touching on the following topics (topic names likely to evolve):

  • Non-Axiomatic Logic, the Word, & Loving Your Enemy
  • Jesus, Duck Typing, & Clearing Up Some Misconceptions About Christ
  • Jesus, His Sheep, Self-Similarity and Neural Networks
  • Gödelian Numbers, Tree-Shaking, Blockchain Scalability & Counting Hairs

If you or anyone you know is a YouTube graphic design person, and you are interested in a deep, old Logical Foundation, let me know! I could really use your help with some neat animations or something for these topics!

Clear A Path: Part 1 - The Bible, Isaiah, and…Cache Invalidation?

In this first introductory video, I read a passage from Isaiah 40, wherein there is one of many references to cache invalidation in the Bible. I also touch on just a couple very basic operations in the current ibGib web app.

NOTES:

  • Thanks @AstonJ for previously recommending YouTube videos on how to use ibGib.
  • I’m looking for anyone interested in understanding our existence in a living, self-similar universe.
  • You can view this wiki’s history to see previous incarnations of this post. It has been, and continues to be, a long road.
5 Likes

Well all rightey then…

I’ve been working through the architecture for Elixir, I went ahead and refactored it from “thingGib” with a Thing-oriented paradigm back to my ibGib paradigm. This is super abstract and fundamentally newish and a little more difficult to grok, but it still is the same approach essentially: expressing transforms to create things. Only ibGib is a slightly more generalized case. “Expressing transforms” is more like “interacting ibgib, the primary interaction being applying transforms to begin with”, and “things” are more like “ibGib is a recursive concept structure with some explicit syntactical reminders that it is an infinitely recursive function”. Same difference.

If you are into quantum physics and the like, and you are looking for an actual intuitive understanding behind the logic of it and not just moving ahead on the mechanics of the various math approaches, check out my updated higher-level readme and my new ibGib wiki that I’m starting up.

If you wonder why you might want to do that, check out the before diving in…why give a s**t about this library? section.

If you want to see how I’m approaching (re)developing the actual elixir code to implement these concepts, check out my ib_gib app’s readme.

4 Likes

I’m curious about this and it seems an interesting pattern but, and this feels odd to say, I cannot seem to grok what its purpose is or why/how it would be used? Most of the examples do not follow how I think is the main cause, so might I ask for some comparison examples perhaps? Such as:

  1. Solve a problem using traditional Elixir.
  2. Then solve that problem using ibgib.
  3. Then document what additional possibilities, checks, restraints, etc. that the ibgib form gives in that specific case.

I am quite curious in such a description set added. :slight_smile:

2 Likes

@OvermindDL1 Not grokking the purpose and how/why ibGib will be used is perfectly “non-odd” hah. Rather, that certainly is reasonable to say, and it speaks to the abstractness of the concept.

In short, its purpose is unification, something of a rethink on existing patterns/tools on a very broad scale. But since that won’t help much, we can think of the near term. The most immediate use case is “replacing” the back-end, much like Event Sourcing + DDD is challenging Relational Databases more recently because of the cheapness of data => explosion of “Big Data” and data mining. But there are various key fundamental differences, the primary one probably being that ibGib’s domain language is an abstract meta-language addressing combining/mutating/splitting nodes. Also, it does not have “past events” like Event Sourcing, rather it has something more akin to “expressing DNA”, a mechanism of which replaying past events (called “hydrating” in ES) is only one special case.

So in short, for starters, think of a back-end “evolution” of Event Sourcing (though I was working on this before ever hearing the term 15 years ago).

If you want a more concrete history, I can point you to my previous Xamarin.Android implementations of three Android apps:

  1. languageGib for learning multiple foreign languages by studying songs (and any other ordered texts)
  2. coffeeGib for coffee geeks and snobs to track coffee history: coffees, coffee gadgets like aeropress/chemexes/etc., and individual roasts.
  3. ibGib/mobileGib, which contained both languageGib and coffeeGib.

So you can think of these as the “problems”. But you could say that it was the separate domain languages that was the “primary” problem. I wanted a simpler back-end creation mechanism so I would haven’t to keep implementing data source stuff that I didn’t care about. I wanted business object creation/validation to be able to be shared across all of my implementation “domain languages” (e.g. languageGib had Song, Author, Language, etc; coffeeGib had Coffee, Roast, Composition).

So, in order to write these various data models (in Elixir, these would be structs I imagine), I had to write the models themselves, migrate the database, add validations, try to share validation logic (I had written my own plumbing architecture all the way), implement CRUD, and then test all of these things, then test them across the wire to a cloud database that I had to take care of, blah blah blah, etc. etc. I just need one back end for my particular use cases, and I abstracted it into a nodal form that actually keeps the history of this process. It “worked” in my POC javascript implementation, but the expression part is just plain clunky in javascript. I got tired of trying to compose complex parallel execution at a high level, hence erlang and then elixir.

It’s such a broad problem though, the “simplest case” use case is a “note” taking app. This is “simply” creating a “text” thing and then a text implementation. The “Hello World” is actually almost the entire thing! But even just Hello World is difficult because of the abstraction. But the abstraction buys us the ability to use each individual “datum” instance of a “text” as a template for others. So if user A creates some template, then user B should be able to “fork” that template and tweak it. The browsing process is shared, so you can look online for others’ “templates”, which can actually be just their data, i.e. they don’t have to do anything special to mark it as a “template”. (Edit: Think of this like navigating to someone’s github repo, then saying “Hey, I like they’re data model X” and then being able to fork just that data model and not the entire repo).

This then goes beyond “notes”, which would be text “instances”.

So with coffeeGib, I had a Coffee model. I chose the properties and architecture that I wanted, me as the programmer. I chose Name, Roast Date, etc. Well this kind of templating could be created one time online. So imagine some other company/user likes my coffee template but doesn’t want the roast date or the brand name or whatever. They would fork it, split off what they don’t want, and then they could create instance data of that template.

I could go on and on all day of what it could do…for example, because of this architecture, you would know when refactoring exactly which data would be affected by the “refactor”. When you make a “change” on a data object on the server, your UI could know exactly which properties need to be updated. It’s specifically designed for micro-service structure, i.e. “Big Data”…On and on…anyway, I’ve written too much!

Thanks for your interest, and let me know if there is anything else. You can also check out the wiki for more abstract stuff. Also, I am unsure if talking about this more is the goal of this forum, and if you want to continue this you could also create a GitHub issue and I could go on and on there. Maybe one of the admins could help me out on that point?

2 Likes

Well, I’ve got a rudimentary POC app going. What a great language! I didn’t have to write the word “async” once, nor did I have to write any callbacks. :wink:

I’ve created a wiki page which walks you thru some ibgib basics - which at this early stage, is just about all it can do! The walkthru does have pics if you’re feeling lazy and don’t want to clone it. I’ve also cleaned up the readme to be a bit more useful for cloning and getting it up and running.

I’d love for y’all to check it out, and I’m certainly down for anyone wanting to contribute! :smiley:

Speaking of which, because of his curiosity in this thread, I’ve already discussed some aspects with @OvermindDL1, who recommended I post our discussion up to this point. So I’ve appended the brief back and forth here:

@OvermindDL1:

I’m doing a quick look over web-gib right quick as I am curious how it works with phoenix.

So far I note, main entrance point is unchanged, as is repo and endpoint, though I see a new plug: https://github.com/ibgib/ibgib/blob/master/ib_gib_umb/apps/web_gib/lib/web_gib/plugs/ib_gib_defaults.ex#L22
Unsure what it does, but will be looking for ‘root’ in assigns in other code.

Web, router, and gettext files seem all normal. This new constants file has a lot of magic constants (<<226,142,135>> for example). Any chance on adding documentation on all of those, why they are what they are and what they are for?

PageController has a new import and a couple of module constants
defined, unsure what those are for. Found the included module at https://github.com/ibgib/ibgib/blob/master/ib_gib_umb/apps/ib_gib/lib/ib_gib/helper.ex
and looking at it, and I see what the functions do but not necessarily
why or what input they usually take. I might recommend a lot more doctests (see what I do in my apps for example at permission_ex/lib/permission_ex.ex at master · OvermindDL1/permission_ex · GitHub). Either way, the helper module does not seem to do anything with those module constants so no clue what they are for.

Continuing to IbGibController I see the helper and module constants
again, and a lot of connection assigning happening. Still almost no
inline docs or doctests, so at this point I cloned and ran the project,
it showed a few pages until I typed something in the textbox and I hit
enter (fork-button) and it gave back a Internal server error. I then decided to run mix test
to see if anything was odd there, and it failed due to needing a
database and not finding one (odd, I’ve not seen any Repo calls around
here)?

So that is my first experience with it, out of time for now, will look more later.

@ibgib (hey that’s me):

There is a hash comment just above each of those unicode binaries that contains which unicode character the code point refers to. It is not a full documentation because most of web_gib is still in flux. More on this in my summary…

You can ignore PageController. I will add more documentation on the Helper module for you. Last night, already went through all of your repos and I did see those doctests you referred to. I do a lot of commenting, and in the past I’ve done commenting to that extent before. But it came to naught, and now I have to be much more selective about how far I go down that particular rabbit hole. But as I said, I will add the docs to the Helper module, and many more where you request it.

I looked through the IbGibController and yes, I agree with you, not nearly enough commenting. The primary reason for this, though, is because I’m simply not ninja at web development and don’t really know what the heck I’m doing in the controller. If you look at the ib_gib app itself, you will find many more comments, though still not remotely complete…but a lot more than the files you did look at.

I am currently working on adjust the readme to be more useful for someone to get it going. Yes, what I consider the “data” layer to be is actually in the ib_gib app, and there are a couple steps to get the repo going. I will put it in the readme and let you know. (Long and short of it is, I use a Postgres docker container which I’ve configured the ib_gib app to talk to in the dev.exs and test.exs files.)

As I mentioned near the beginning of this message, the web_gib app is in flux. My primary focus has been, and will continue to be, the ib_gib engine aspect of it. I’ve only now reached the point where UX will help drive where I need to focus on the engine. But let’s get you up and running with the working POC web app is-is, and if you can see the potential, you will need to understand some of the novel aspects of the ib_gib side of things. I would keep it as technical as possible, without digressing too much into “philosophy”.

So let me fix up the readme a bit, and I will message you when I’m done. Then, we can discuss your thoughts on it some more. I shall also go ahead and document the Helper module. :wink:

2 Likes

Just wanted to post some neat, but still abstract, UI pics of the interface I’m working on using d3.js v4.

This is actually a query result, showing the current sessions’ queries (it’s a “meta” query!).

The yellow circle is the query_result itself. The gray ones are the ancestors (similar to class “inheritance”, but also analogous to a forked repo’s upstream repo). The pink ones are the dna, which is akin to event sourcing “events” that it took to “create” the ibgib. The purple ones are the query data that was run to create this query_result.

Any green circles are the “root” ibgib. If you look closely, the little green one is there both as data and as a query result. This is because it’s special :laughing:

Here is what it looks like when we navigate to the query ibgib that generated this query:

Anyway, I just thought it would be fun to post these pics, because it’s taking a LONG time to do UI stuff! But it’ll be worth it! :smiley:

3 Likes

Heh, pics are always awesome! Really helps to visualize the code too with the way you are starting down.

3 Likes

Absolutely! It’s such an abstract concept, but I’m determined to make the UX “simple” with growth for complexity. Certainly easier said than done :laughing:

3 Likes

I decided to reply to this again, as I continually have trouble explaining ibGib to others. This is not their fault but mine. But I still try! :wink:

Starting a few months ago, I became aware of something called IPFS (website, question on elixir forum). Despite obvious implementation differences (they are looking to work all the way down at the transport/protocol level), their approach shares many characterstics with ibGib.

And just now, I’ve watched a great talk by Joe Armstrong, and at one point he talks about content addressable storage, at which point he shows two slides with very concrete aspects: 1) “Content-addressable web” - URL addressing scheme, 2) “API” - Content Repo interface.

I thought showing these might help explain ibGib a little, partly in the vague hopes of attracting anyone with a UX mindset (I suck at the front end), but partly just to show something concrete on the somewhat abstract subject.

“Content-addressable web” - URL addressing scheme

JA Video (at time 36:14)
http://my.site.land/get/sha1/afg123def23,
http://some.other.site/get/md5/adf1234

ibGib (current implementation, not encoded for example purposes):
http://ibgib.com/ibgib/ib^gib,
http://ibgib.com/ibgib/some ib here^AFG123DEF23

The ibGib ib is a string, and the gib is usually a SHA-256 hash. These are like first and last names. Lots of people share first names, just like the ib can be non-unique, and this can be specified/controlled by the end user. The gib last name is always generated by the engine and is usually calculated as a SHA-256 hash of the ib, data, and rel8ns. The data is a key/value map used for storing “internal” data to a node, e.g. %{"value" => "Hello World"}. The rel8ns is a map in the form of "relationship" => ["array^of", "other^ibgib"], e.g. %{"ancestor" => ["ib^gib"], "dna" => [ib^gib]}.

“API” (Content Repo API)

JA Video (at time 37:34):
put(Data) => void (no return)
get(SHA) => Data (or no return)

IbGib (In IbGib.Data):
def save(info)
def load(ib, gib)

Mr. Armstrong’s focus is mainly centered on the access of the content at this point, and so his API is basically the content repo API. I think his particular “phrasing” here is a little misleading. Either he is including the hash inside of the data he is passing to put, in which case he needs no return but the “Data” is actually Data + Metadata. Or he actually wants to return the hash on the put, and that way when you put the data, you get an address that is a hash. Then you’re able to get at that data again by passing in that same hash.

In ibGib’s save/1 function, the info corresponds to his Data construct. I have implemented this as a map that contains the ib, gib, data, and rel8ns as described above. The load/2 is the get equivalent, passing in the ib and gib which form a unique address to a resource.

Discussion

In the above ibGib URL addressing scheme, I list only the ibgib domain. But I have specifically designed it to be entirely distributed and a primary goal is to allow others to get copies of whatever information is pertinent to them, much like cells in a body exchanging information (and like a compiler removing unused code). This distribution is planned in the form of initially a single server, then nodes and clusters, then multiple sites, then eventually non-server based. So ultimately, just using the system would provide all of the benefits of fault tolerance, availability, etc. The BEAM’s implementation along these lines is one of the primary reasons I chose Elixir!

Also, notice the SHA aspect of both JA’s URL scheme and the API. Yes he lists md5 as an alternative, but this is really still an implementation detail. The ib mechanism provides this at an abstract level, not presupposing what its own use is going to be. Stepping back, all of this stuff is a huge paradigm shift and who knows what else it could end up being! So for people looking to encode information in the ib, they can do so because of its abstract flexibility. For people who don’t have this focus, they don’t have to. Plus it allows me (or whoever is doing coding) to use this as metadata as well, although this is only one route for metadata (there are plenty others). But still ibGib’s design is compacted, IMO quite elegantly (I’m biased :wink:), to the ib, gib, data, and rel8ns.

Summary

So according to the video (also this one about the mess we’re in), Mr. Armstrong is looking for us to solve these kinds of problems, and ibGib is my approach to this.

The near-term goal is to have this kind of structure with notes, pictures, etc. Think git (forking, merging, and more) for any type of content, as well as enabling quite purposefully big data and ML. The far-term goals…well, the applications of the architecture are boundless…I could go on and on about them.

It’s the upcoming IoT!..but even more than just that! :scream: :smile: I chose Elixir because I think it is better suited than Go for many reasons, as I’ve expounded elsewhere on this forum. I hope that it comes to fruition and I’m right! :laughing:

Anyway, this is wayy too long. Thanks for reading! :wave:

3 Likes

ibGib v0.1.0 is up on the interwebs! :globe_with_meridians: :tada:

Right now, the ibGib web app can act as an innovative way of keeping track of notes. It works both on desktop browsers, as well as mobile devices. You can check it out at www.ibgib.com.

Current Features

  • :vulcan: Fork ibGib
    • Acts like a “create new”, but includes a timeline and identity much like version control, e.g. fork in git
  • :id: Authenticate with multiple email addresses
  • :link: :link: Add links
  • :left_speech_bubble: Add notes/comments
  • :frame_photo: Upload pictures (must be authenticated with an email address)

Walkthrus and Wiki

It’s only 0.1.0…

There are a ton of issues that I have prepared for going forward. But even still, it was surprisingly easy to use for the WaffleGib walkthru. You can check out the actual end product in the ibGib web app here, but you’ll first have to read the privacy (or lack thereof) caution first. I don’t have it setup just yet to automatically redirect after this interstitial step, but I’m workin’ on it! :construction: :wink: Once you’re there, you can click on individual any node and navigate to it, using the :recycle: refresh button to go to the latest version, etc. Think of it as a visual entity git-timeline explorer :earth_americas:

Distillery + Docker + AWS Deployment Journal

I also wanted to mention that I kept a journal of my entire deployment process which you can find in my 0.1.0 deploy github issue. I’ve been reading up for quite awhile on deployment techniques, but this issue chronicles my entire journey over the last week or two when deploying. There is a TL;DR Summary as well.

P.S. @AstonJ Would it be possible to turn my first post into a wiki where I could place this stuff? I had originally started it as looking for people interested in joining, but now I’d definitely rather put this kind of information there if that’s all right!

4 Likes

Done :023:

3 Likes

Wow that was quick! Thank you! I need a break at the moment though… :laughing: :sweat_smile:

4 Likes

ibGib v0.1.1 Live!

Check out the totally revamped first post in this thread for more information. :smile:

Thanks for making it a wiki @AstonJ :thumbsup:

Also, you can the bug fixes and other issues for v0.1.1 on GitHub.

3 Likes

Looking nice!

2 Likes

Much thanks @OvermindDL1 ! We’re slowly getting there, and fortunately I’m getting a lot of help here and always appreciate it. :sweat_smile:

:up: next is a move towards more JSON api stuff and an Alexa app. (I’ve made two already: Bible Seeds and Such and GoSpursTrivia). I think the future is AI + ML, and combining this with Elixir being so darn strong :muscle: on the back end with parallel processing and scaling, that voice-controlled devices like the Echo :speaking_head: - in addition to Nerves/Embedded devices in general - is :key: .

I’ve tried to ensure that ibGib’s architecture dovetails nicely with voice control, and a “list” type app is my “simple” voice app implementation.

3 Likes

ibGib v0.2.1 is out! :smile:

I’ve edited this thread’s wiki post to reflect the changes, which is basically an excerpt from my README.

There are two youtube videos to check out now:


Login:


Basics:

3 Likes

Awesome and next-level work. I’ll be busy researching it for a while. :icon_rolleyes:

1 Like

Thank you!! :grinning:

I’ve just released a completely new version that really shows the potential for the architecture. Here are some screenshots for the new look (of the front end…the back end is almost exactly the same. Very stable)

https://www.ibgib.com/as-chat/ibGib%20Tutorials^1E371C4463D0EA08AEB4A65AC7A5C0805526B605587B869553E34DA5AAFFE4FF


https://www.ibgib.com/as-chat/comment^E35129F3F75834BAEFA4C05255750D20DB8EE8B65B02096E0969883C72B4D69D


https://www.ibgib.com/as-file/pic^83CBA46A243E38860CD81A16FAB714B3F6832FF8565A320E3D411F08956C40D5


Unfortunately I hit my second burnout finishing this release, so things are going a little slowly now and I need to step away. I’d be happy to answer any questions you have in your research. The concepts behind it are more important than the front end web app I’ve come up with. :sweat_smile: (with of course huge help from all of the open source community, not least of which the elixir and erlang family!)

3 Likes

Also, of course, if you or anyone else would like to contribute, there are many directions to take and so any skill set would be useful. Here are just a couple of the possibilities:

  1. Distributed architecture
  2. Additional “app” plugins and views
  3. CLI
  4. API (currently works over channel awesomeness - thanks Phoenix team!)
  5. Additional and alternative storage implementations.
  6. Class/entity building on top - think git for data structures combined with OWL-like class structures/conventions

These are largely due to the self-similar nature of the data construct: every ibGib is its own self contained “database” / projection. This makes it ideal for IoT, distribution, etc. !

3 Likes

I started researching this lately and read your posts but was bummed to discover the GitHub repo is no longer live. :confused:

Are you still working on this?

2 Likes