I’ve continued my journey and committed to the project every now and then.

So here’s a little update .

Distributed Setup

The current version is running on Fly . I’ve got a small cluster (two nodes: FRA, LAX) setup with :libcluster so adding new nodes should work automatically.

Each Fly instance has it own storage attached for storing Git repositories. When a user creates a new repository it is assigned to the local’s node storage and all the Git objects will be stored there:

I live in Austria, so my Git repositories are stored on the closest instance running in Frankfurt.

When accessing one of my repos from the US, the instance in Los Angeles will route all Git commands to the right node.

In order to get things working without too much latency, I had to refactor a big chunk of code to batch Git commands together and keep the number of roundtrips between instances low. In the end I’m quite happy with the results

Try for yourself:

• redrabbit/elixir is running in FRA.
• redrabbit/phoenix is running in LAX.

Repository Pools

I’ve implemented some kind of distributed routing pool on top of Erlang’s :global.

Here’s a screenshot of the supervision tree:

image1200×363 22.6 KB

When a node start’s up, GitGud.RepoSupervisor does a few things:

• tags the local storage (if not already tagged) and registers the resulting id across the cluster.
• starts a GitGud.RepoStorage worker for handling filesystem operations.
• starts a GitGud.RepoPool supervisor for handling Git commands.

The GitGud.Repo schema has a :volume field which points to the storage VOLUME where it data is stored. When creating a repositories it is assigned to the local storage:

field :volume, :string, autogenerate: {GitGud.RepoStorage, :volume, []}

With this in mind, let see how we can run Git commands on a specific repository:

repo = GitGud.RepoQuery.user_repo("redrabbit", "git-limo")
{:ok, agent} = GitRekt.GitRepo.get_agent(repo)
{:ok, head} = GitRekt.GitAgent.head(agent, head)
{:ok, commit} = GitRekt.GitAgent.peel(agent, head)
{:ok, commit_msg} = GitRekt.GitAgent.commit_message(agent, commit)
IO.puts commit_msg

The above example prints the HEAD commit message for redrabbit/git-limo.

The interesting part here is GitRekt.GitRepo.get_agent/1 which is implemented by GitGud.Repo:

  defimpl GitRekt.GitRepo, for: GitGud.Repo do
    def get_agent(repo), do: GitGud.RepoPool.checkout(repo)
  end

As you can see, it rely on GitGud.RepoPool for retrieving a Git agent from the pool on the right node. Let’s dive into it .

Internally, the pool can be seen as a DynamicSupervisor of DynamicSupervisors. GitGud.RepoPool.checkout/1 being the entry-point for fetching agents. It also provides a few nice things:

• auto scale – grows/shrinks the number of agent processes based on demand.
• global cache – agents in a pool share a global ETS table.
• round robin – agents are distributed using round-robin.
• node aware – a pool will always start on the right node based on the repo’s VOLUME.

Git Agents

The GitRekt.GitAgent module is the backbone for running Git commands. While the public API is quite easy to grasp, it hides a lot of complexity.

An agent is basically a wrapper around GitRekt.Git. Here’s a very basic usage example:

{:ok, agent} = GitRekt.GitAgent.start_link("path/to/workdir")
{:ok, tags} = GitRekt.GitAgent.tags(agent)
for tag <- tags do
  IO.puts "Tag #{tag.name} -> #{Base.decode16(tag.oid)}"
end

In the above example, agent is a dedicated process for running Git commands.

Note that it is also allowed to run Git commands in the current process as well:

{:ok, agent} = GitRekt.Git.repository_open("path/to/workdir")
{:ok, branches} = GitRekt.GitAgent.branches(agent)
for branch <- branches do
  IO.puts "Branch #{branch.name} -> #{Base.decode16(branch.oid)}"
end

In the above example, agent is a NIF-resource representing a libgit2 repository.

Transactions

GitRekt.GitAgent provides support for transactions aka. batching a bunch of Git operations in one call. This is very important when running Git commands on a separate node:

# agent is a PID running on an other node
{:ok, head} = GitRekt.GitAgent.head(agent, head) #1
{:ok, commit} = GitRekt.GitAgent.peel(agent, head) #2
{:ok, commit_msg} = GitRekt.GitAgent.commit_message(agent, commit) #3
IO.puts commit_msg

Running the above code would make three separate GenServer.call/2 resulting in quite some latency. We can fix this by batching the commands in a transaction:

# agent is a PID running on an other node
{:ok, commit_msg} =
  GitRekt.GitAgent.transaction(agent, fn agent ->
    with {:ok, head} <- GitRekt.GitAgent.head(agent, head),
         {:ok, commit} = GitRekt.GitAgent.peel(agent, head) do
      GitRekt.GitAgent.commit_message(agent, commit)
  end)

In the above example, the three commands are execute in a single call on the dedicated agent process. Reducing the overall latency…

Caching

An additional feature of GitRekt.GitAgent is caching. When running transaction/3 we can pass a cache key as the 2nd argument:

def commit_info(agent, commit) do
  GitAgent.transaction(agent, {:commit_info, commit.oid}, fn agent ->
    with {:ok, author} <- GitAgent.commit_author(agent, commit),
        {:ok, committer} <- GitAgent.commit_committer(agent, commit),
        {:ok, message} <- GitAgent.commit_message(agent, commit),
        {:ok, parents} <- GitAgent.commit_parents(agent, commit),
        {:ok, timestamp} <- GitAgent.commit_timestamp(agent, commit),
        {:ok, gpg_sig} <- GitAgent.commit_gpg_signature(agent, commit) do
      {:ok, %{
        oid: commit.oid,
        author: author,
        committer: committer,
        message: message,
        parents: Enum.to_list(parents),
        timestamp: timestamp,
        gpg_sig: gpg_sig
      }}
    end
  end)
end

You may have noticed the {:commit_info, commit.oid} tuple given to transaction/3. This tells the agent that the transaction should be cached using this key.

Calling commit_info/2 two times in a row would result in the following log output:

[debug] [Git Agent] transaction(:commit_info, "b662d32") executed in 361 µs
[debug] [Git Agent] > commit_author(<GitCommit:b662d32>) executed in 6 µs
[debug] [Git Agent] > commit_committer(<GitCommit:b662d32>) executed in 5 µs
[debug] [Git Agent] > commit_message(<GitCommit:b662d32>) executed in 1 µs
[debug] [Git Agent] > commit_parents(<GitCommit:b662d32>) executed in 4 µs
[debug] [Git Agent] > commit_timestamp(<GitCommit:b662d32>) executed in 11 µs
[debug] [Git Agent] > commit_gpg_signature(<GitCommit:b662d32>) executed in 6 µs
[debug] [Git Agent] transaction(:commit_info, "b662d32") executed in ⚡ 3 µs

We can observe that the first call executes the different commands one by one and cache the result while the second one fetches the result directly from the cache without having to actually run the transaction.

There’s a lot more to tell about GitRekt.GitAgent’s internals (streaming support, mechanism to prevent the garbage collector for deleting NIF-resources, etc.). If you’re interested I can write a small post about it.

LiveView

On the frontend, I’ve managed to introduce Phoenix LiveView and replace all my React/Relay components with a LiveView counterpart. For example, the GitGud.Web.TreeBrowserLive is used to navigate across a Git repository tree. Here’s a list of all views/components:

• GitGud.Web.BlobHeaderLive
• GitGud.Web.BranchSelectLive
• GitGud.Web.CommentFormLive
• GitGud.Web.CommentLive
• GitGud.Web.CommitDiffLive
• GitGud.Web.CommitLineReviewLive
• GitGud.Web.GlobalSearchLive
• GitGud.Web.IssueEventLive
• GitGud.Web.IssueFormLive
• GitGud.Web.IssueLabelSelectLive
• GitGud.Web.IssueLive
• GitGud.Web.MaintainerSearchFormLive
• GitGud.Web.TreeBrowserLive

Fast Git Backend Server

I also refactored the Git backend aka. GitRekt.WireProtocol which was slow and consumed a lot of resources.

When pushing a repository, the incoming PACK file is now directly streamed to the disk. This increases raw performances about 700% and greatly reduced the amount of RAM and CPU used for the operation.

The performance boost allows to fetch/push across nodes in a cluster setup. When you push via SSH you will send the PACK to the nearest node which is then streamed to the right node in the cluster.

I’ve been working a lot on this project lately. Just released version 0.2.7.

Also I decided to publish the project on a dedicated server for testing purposes.

So here it is: https://git.limo

Or here if you want to browse some repositories:

This is pretty limited currently because you are not allowed to create new repositories. I need to provide monitoring tools and other things before letting anybody host their repositories.

Here’s a brief summary of the project’s current state:

Basic CRUD operations

• User registration/authentication
  • with login/email - password credentials.
  • with OAuth2.0 for GitHub and Gitlam.
• Email management:
  • Each registered user need at least one verified email address in order to create new repositories and maintain other’s repositories.
  • Each verified email address is used to associate Git commits. This is also the case for GPG signed commits.
  • The primary email address is also used to retrieve a user’s Gravatar.
• SSH and GPG key management:
  • We support Git over SSH and HTTPS. By adding SSH public keys, a user can authenticate without the need of a password.
  • GPG public keys are used to verify that a Git commit has been signed by a given user. In order to show the verification icon:
    1. The committer’s email must match a verified email address
    2. The committer’s email must also match the GPG key email address
• Repository management:
  • Users can create repositories if they have at least one verified email address.
  • Each repository can have multiple maintainers with read/write/admin rights.

Git Repositories

We support both HTTPS and SSH transport protocols. While HTTPS support Basic Authentication via credentials. SSH supports authentication via public key and password.

Access to repositories depend on the user’s authorisations:

• Anybody can view/clone a public repo.
• Only owner and maintainers can view/clone a private repo.
• Only owner and maintainers with at least :write access can write/push to a repo.
• Only owner and maintainers with :admin access can edit settings of a repo.

Browsing repositories is similar to GitHub/GitLab. You can browse trees, view blobs (with syntax-highlighting), walk the commit history, walk the commit history for a given tree/blob, display diffs, etc.

Accessing a Git repository content is done via the GitRekt.GitAgent abstraction. Basically the agent provides an API to manipulate Git objects and refs.

The nice part here is that the access to a repository can be done from multiple processes simultaneously (by wrapping the NIFs resources in a GenServer and serialising function calls).

This will be important when implementing support for clustering aka. distributing repositories on multiple nodes.

The GitRekt.GitAgent provides two different modes:

• :inproc - calling GitRekt.Git functions (NIFs) directly.
• :shared - serialise function calls through GenServer.

The GitRekt.GitRepo protocol helps to implement custom logic (clustering) for both modes. Currently, only GitGud.Repo implements this protocol.

GraphQL API

The available GraphQL API is quite fully featured. It also provide subscriptions via Phoenix Websockets.

Issues and reviews

There is no issue system at the moment. I hope to get started with issues soon.

I also want to provide a review mechanism, basically the possibility to review changes across commits. Currently, any registered user can comment on any commit. Comments are stored globally (shown at the end of a diff) or on a per-line basis.

Wiki

I’ve played a little bit with a Git based Wiki implementation. But this will come in a later future.

Also, the project is sponsored by AppSignal . You can check the appsignal branch if you’re interested:

image1200×1064 48.1 KB

You can also check-out the fly branch if you want to deploy on your own Fly instances.