That’s what I’m doing right now. I use exclusively libgit2 to interact with the underlying git objects.
I currently have two interchangeable storage backends available :
:filesystem(default) –.gitdirectory:postgres(experimental) – usinggit_referencesandgit_objectstables.
The problem is that traversing the commit history is quiet slow by design. There is no fast way of querying the history of a given pathspec (tree, blob) or even counting commit ancestors.
I’m not quite sure how you would implement caching effectively. I don’t want to cache on demand, the first hit will always be slow…
Currently, I’m experimenting with storing meta data for commit and tree objects and the performance boost I got so far is impressive.
In future I would like to extend my :postgres backend to write Git objects in separated tables (references, commits, tags, blobs, trees). Being able to perform more complex queries (authors, committers, query avatars, GPG signatures, ACLs, etc.) directly on the DB would be a 
