One of my web apps reached a point that it is now time for me to build a search feature for it. On the one hand user’s need to be able to find relevant content using a search bar, on the your hand the content that is loaded on the initial page load should be relevant to the user that is logged in. Therefore, I am eager to learn about how to query my database ‘by relevance’.
I am pretty much starting from scratch. I will need to learn about full text searches, indexing, user profiling, signals, Inverse Document Frequency, etc. (these are some of the topics I read about somewhat so far). Also, I also hope to become knowledgable about best practices, limitations and optimizations.
Any advice regarding learning resources? Or how to approach this learning journey?
One of the books I liked on this topic is AI-Powered Search by Manning. It employs Solr for the provided examples.
BTW, I remember a discussion I had with @dimitarvp wherein he mentioned Meilisearch as a lightweight alternative to ElasticSearch and Solr.
Myself, I have used in practice ElasticSearch. It provides the functionality I need, it is easy to interact with Elixir via the snap library. Of course, optimising the cluster and dealing with the required resources might be an issue.
My first thought was to rely on Postgres without using any other database tools/search engines like Solr or ElasticSearch. I have used Ecto/Postgres so far for all my database needs for this web app.
Is that a bad or naive idea? Do the pros of using ElasticSearch or Solr outweigh the cons?
I guess it depends on your use-case. I have not used Postgres for full text search myself, yet. If the queries are simple and the content not too large, I guess it could work.
My needs were rather towards quite complex queries such as the one I mentioned in another topic.
My use case is similar to a library that keeps all it’s books online. The content is mostly text with some pictures.
Users can set interests, can consume the library content and can follow other users. Planning to use all three in my queries, along the literal search phrases user’s might use when explicitly searching the content using a search bar.
But that’s my perspective now. With more knowledge about searching I might see reason to make the queries significantly more complex. Hard to tell.
Ty for the links by the way. Working my way through them!
Hey @coen.bakker
In the past I implemented a fuzzy search with postgres using the pg_trim extension with good results, but I don’t think it can work reasonably good and fast when dealing with full books, maybe you can give it a try.
If it is necessary I can decide to use ElasticSearch or Solr, by the way. Having reasonable search speeds is important. But find it difficult to estimate how much extra complexity adding one of the two will add, beyond just using Postgres.
There is typesense too, which is billed as an algolia alternative or a lightweight alternative to elastic. IIRC it does want to keep a lot of stuff in memory.
Yes🙃, it did caught my attention in a positive way. Today I got familiar with Full Text Search and Fuzzy search in Postgres. Liked it. But will look into Meilisearch now also. The self hosted option, that is.
