D3 can render to canvas and/or svg.
What is your reasoning for putting a GenServer in front of the db for the chart data? Is the data expensive to calculate and/or rarely updated?
cmo
pelopo
Oh, that’s great. I was not aware of it. Thanks
Re GenServer, I don’t know why. My AI tool suggested it and said it’s best practice. Probably it’s because I fed it the data from PhoenixAnalytics library.
I’m very new to Phoenix and Elixir and don’t even know what GenServers are for, but when I questioned AI About it, it said it could be used as a common state management system as well as cashing solution. Also, it could be a central point where connections to the DB unmaintained.
https://github.com/lalabuy948/PhoenixAnalytics/blob/master/lib/phoenix_analytics/repo.ex
0
dhony
I was thinking about the same thing and then I’ve created this repo to consolidate my learnings.
There’s nothing yet. I’m only gathering the information and organizing the ideas on the readme file. The purpuse is trying different graphics libs.
https://github.com/dhonysilva/maaite
Moreover, since the Oban Web OOS release, I’m thinking about open it and understand what graphic lib they utilize to present the analytics, but I couldn’t manage it install it yet.
2
garrison
Libraries like Ecto will manage the DB connection pool for you, so you don’t need to worry about that. If you’re new I wouldn’t bother with GenServers for now, you probably won’t need them for this project.
If you want to implement caching, Cachex is a great library. But avoid caching until you’re sure you really need it.
For a DB, stick with SQLite or Postgres until you run into performance problems. DuckDB and Clickhouse are much faster for analytical queries, but also more annoying to use, and if you don’t have much data it won’t matter.
1
pelopo
Thanks @garrison. This is exactly my plan to go forward. I’ve done some hight level tests and since I’m mostly using SELECT and WHERE, the Duckdb, does not seem to be much faster at all. I do have a lot of data, but for purposes of displaying data, it’s seem like SQLite is enough for now.
Also I found that Duckdb does not seem to have any tool like ‘sqlite3_rsync’ or ‘streamline’ to update the production DB instance in a running state does not exist for Duckdb.
For now, I’m sticking with SQLite
1
garrison
I don’t know exactly what your data looks like, but I guarantee if you try again with some analytical queries (like sum or count with a simple predicate) and a billion rows you will find DuckDB is a couple thousand times faster.
Like I said, if you don’t have much data (you probably don’t!) it’s not worth it.
0
pelopo
I meant Duckdb is not much faster if I only need to display data for a chart, not to run any analytical SQL transform with it. I do it in Duckdb withing a python data pipeline and Duckdb flies … I wish Ecto added it…
0
garrison
For analytics data you usually do what are called “analytical queries”, stuff like
select count(*) from pageviews where page='/home' group by day;
Then you would dump those rows into a chart.
Column stores like DuckDB or Clickhouse can perform this particular type of query much faster than row stores like SQLite or Postgres, but at the cost of a lot of other functionality.
If you have even millions of rows you are unlikely to notice much difference. If you have billions it will matter.
1
cpursley
We’ve put together a little package that brings together several ideas into a pluggable Elixir package: GitHub - agoodway/goodanalytics: Visitor intelligence, link tracking, source attribution, and behavioral analytics for Phoenix · GitHub
1