polypush135

polypush135

Advice on analytics for web traffic

I’ve been building my own personal project https://morphic.pro and I’m at a place where I want to start collecting page hits and other metrics about where the request came from and so forth. So its time to add some analytics to my personal site.

I’ve used Google Analytics many times in the past, and overall it’s a wonderful tool. That said I feel we are living in a time where I trust Google far less than I used to. I’m not going to go full tinfoil hat right now my, but my intuition says I should avoid google.

I’ve also used Mixpanel and again a wonderful tool, but for the same reasons I don’t feel compelled to share my data with Google is the same I feel about sharing my data with anyone.

So that leads me to constantly think, why do I want to outsource this solution? I mean elixir/phoenix is more than capable of recording these requests in a way I can build a nice dashboard for.

And since I think it would be fun and a good showcase of my skill sets I keep going back to this idea.

So before I just jump in and start writing code I wanted to see what other people’s experiences have been when dealing with analytics in their own projects. Did you try your own homegrown solution only to find you went back to google anyways? Did you hit any performance walls that just didn’t make it feasible to do your own thing?

Anyhow, please share your thoughts.

Most Liked

ukutaht

ukutaht

Thanks for mentioning Plausible @LostKobrakai. I’ve been working on it for about a year now.

I’ve thought a lot about offering a self-hosted solution. To be clear, the license allows you to self-host and I’m not stopping anyone. When I say that plausible does not offer a self-hosted solution, it means that I have not taken up the responsibility for making it easy to install and upgrade. If you want to grab the code and run it, by all means go ahead.

I’m wary of offering a self-hosted solution because:
a) The product is still early stage and it would take away from my time focusing on new features
b) It’s likely that the infrastructure will evolve to a point where self-hosting is just not realistic

For instance, I’m currently moving from Postgres to Clickhouse as the database so that I can accomodate sites with >10m pageviews per month. In the future, it might be necessary to add Kafka in the data pipeline to better handle big traffic spikes etc.

There’s a tension between making the product easy to self-host vs making it cost-efficient for large customers and a free tier. For example, Matomo uses a standard MySQL database and it’s very easy to self-host. On the other hand, their hosted service is crazy expensive. With 10m monthly pageviews you’re looking at €849 per month, and that only includes 6 months data retention.

If they built a more streamlined data pipeline, it would make their product much harder to self-host but it would bring the cost of the hosted service down.

Hope that clarifies things a bit. I’m not ruling out a self-hosted version in the future, potentially with different database and ingestion adapters to simplify the infra requirements for self-hosting. However, at the moment I’m focusing on the core business and especially scaling up to larger customers.

LostKobrakai

LostKobrakai

Twitter just made me aware of an elixir based alternative: https://plausible.io/ and it’s (not yet) licensed to not allow self hosting.

LostKobrakai

LostKobrakai

I like https://usefathom.com/ either as a service or even self hosted.

Where Next?

Popular in Discussions Top

matthias_toepp
I’d love to hear what people think about Wisp, the new Gleam web framework started by Gleam’s primary creator Louis Pilfold. Gleam, alon...
New
PragTob
Hello everyone, I know we had quite some threads (read through lots of them) about background job processing but it remains a hotly deba...
New
mikl
I wanted to capitalize a string, and tried using String.capitalize(). That generally works well, until you try to capitalize a word like...
New
Fl4m3Ph03n1x
Background A few days ago I was listening to The future of Elixir from Elixir Talks, with Dave Thomas (@pragdave ) and Brian Mitchell. I...
New
Qqwy
Looking at the stacks that existing large companies have used, WhatsApp internally uses Mnesia to store the messages, while Discord uses ...
New
Fl4m3Ph03n1x
Background This question comes mainly from my ignorance. Today is Black Friday, one of my favorite days of the year to buy books. One boo...
New
lucaong
Hello Elixir and Nerves community, I have been working for a while on an open-source embedded key-value database for Elixir, that I call...
230 13924 124
New
Qqwy
I would like to spark a discussion about the static access operator: .. For whom does not know: it is used in Elixir to access fields of...
New
chulkilee
Here are the list of HTTP client libraries/wrappers, and some thoughts on HTTP client in general. I’d like to hear from others how they w...
New
joeerl
I’m playing with Elixir - It’s fun. I think @rvirding does give Elixir courses these days. Re: files and database - when I given Erlang ...
New

Other popular topics Top

New
mcarvalho
What is the difference between System.get_env and Application.get_env? For example, what are best practices to use one versus another.
New
skosch
To my knowledge, put_in, Map.update etc. all have the one limitation of not automatically creating intermediate keys when needed (for exa...
New
chrismccord
As promised, the first release candidate of Phoenix 1.3.0 is out! This release focuses on code generators with improved project structure...
New
Patoshizzle
After calling mix ecto.create I get this error: 17:00:32.162 [error] GenServer #PID<0.412.0> terminating ** (Postgrex.Error) FATAL...
New
johnnyicon
Hi all, I’ve just started learning Elixir and Phoenix Framework, so please pardon my n00bness at this stage. I’m trying to use Postgres...
New
Fl4m3Ph03n1x
About me? ( if you have nothing better to do than reading about some random guy in the internet :stuck_out_tongue: ) Hello all, this is ...
New
vegabook
I’m brand new to Phoenix and I have stripped one of the demo applications to the bone. I just want to get an svg up on the screen. Here i...
New
grych
Hi folks, Few months ago I have announced the proof-of-concept of the library to manipulate the browsers DOM objects directly from Elixi...
639 52341 488
New
RisingFromAshes
I’ve read in another post that it may be possible with a router helper - but I couldn’t find an appropriate one, and tbh, I’m still just ...
New

We're in Beta

About us Mission Statement