Curated HEX packages

For example, if I just type ORM keyword I will separate in two list the packages. I would put the “recommended by the community” on top and whatever else in the bottom.

As you can see, I never said: hide packages, but just at least is something that help the community. I prefer people contributing to some package than creating their own, but again Owners and Contributors have to be really open to feedbacks and changes.

Maybe it’s too late now but Ember does something like this. Every package has a score:

https://emberobserver.com/addons/ember-awesome-macros Click on “Explain this” under the score.

Package curation is tricky and imprecise though, especially in the early days of a language. Also, open source is not a job. I may write the best and most maintained package ever, but tomorrow I can stop working on it forever. What then?

I don’t believe in having a lot of packages that literally fix the same problem. I prefer to work together.
I would like to continue writing but I think you get the main idea.

Too late for that. Search for “cache” in Hex :stuck_out_tongue: I’m a pessimist so I think Hex will be a mess like every package manager is, but we’ll get on by using the same 10-20 dependencies in most of our apps so whatever’s in Hex won’t matter much. Hex’s problem is to serve and host packages, not decide which is “best”, which is hard to do and will lead to politics/drama anyway. Just let Hex be messy. I take the time to review possible dependencies and their repos to see if they’re maintained well, but I don’t think the community can do that as a whole.

Check out: https://npms.io/about We can at least have a search like that for Hex. It all comes down to search: I don’t care if there are 1000 bad cache implementations if search returns the best one.

Again, doesn’t have to be Hex because Hex is used fro package management but the idea is the same one. I would love to see in hex website because people truth more.

@ericmj I don’t want to accept that “Hex” will finish in the same. This is why I am calling this problem now when we are not that big. Whatever takes I am wiling to help but please don’t let that happen.

The problem is that there’s no solution.

Hex’s only job is to server and store packages. No one in the Hex team wants to review packages, especially as Elixir gains traction and we get hundreds+ a week. They also don’t want a high barrier of entry for people to publish stuff either. Remember, Hex is an open source project done for fun, not anyone’s day job (correct me if I’m wrong).

The community is made up of tons of people. People are impossible to get together to agree on many things, especially tools and software. It happens very rarely and only for very big things like Phoenix or Express in Node (even then, many want Koa or Hapi).

Maintainers come and go and projects die after a few years. It’s an extra burden to keep track of which projects go unmaintained. Some projects don’t need maintenance at all after 1.0.0 because they’re simple.

Also, it’s not that bad. I get stuff done in Node all the time and rarely have a problem finding good libraries. I use npms which I linked above—their algorithm takes into account stuff you’ve mentioned and returns great results. Someone could build the same for Hex. After a few projects you just know which packages and people are good.

Phoenix, HTTPoison, Poison, Plug, Poolboy, Ecto (just naming a few) are all set in stone for years to come and the community has already decided on them, but I can go and publish 20 mediocre JSON parsers on Hex right now if I want. But you’ll never find them or use them with a good search system because you’ll get Poison instead. That’s my only solution: Have a good search algorithm and keep up with the community. Or have your own site that lists curated packages based on some algorithm you come up with.

You can’t stop people from publishing trash unless you hire someone to review packages 24/7, and then you’ll be known as the language that’s not friendly to beginners/people experimenting with packages and new ways to do stuff. I’ve even seen people accidentally publishing their Phoenix web app on Hex! Some programmers hate using dependencies, some like reinventing the wheel. People will always be silly :slight_smile:

Anyway, that’s my rantish answer. Let me just say I do agree with you and I’d love to have a perfect package repository with only the best stuff on it, but it’s just not going to happen.

7 Likes

As long as hex lets me install packages or view their documentation in a timely manner, that’s all I care about. And honestly I be pretty against any kind of moderation/curation on hex itself, as it could just lead to whole other set of problems.

Now having a separate service that curates the packages is most welcome, currently I use awesome-elixir quite often. Although solving the problem of project curation certainly isn’t easy. There’s a few different problems that come to mind.

  1. How do you determine what is the better package? What might be the best package for one user’s use case, might not be the best for another. Look at two popular GraphQL implementations graphql-elixir and absinthe, both are great choices (and from what I’ve previously read, there’s no intention of them to be merged, they’ll both continue to be developed separately), but they take very different approaches to their API. Certain users will prefer the workflow of one of them over the other, while other users will think the opposite.

  2. Curated lists (unless actively maintained and use filtering criteria that considers this) often don’t allow for new ideas to flourish. While you could say, those ideas should be contributed to an already established package. That’s always easier said than done, the already established project might not be at the liberty to make/adopt those changes. Sometimes it really is easier to just establish it as a new project. Now imagine someone develops an alternative to Ecto that’s better than Ecto in every possible way (I know, I know, how could that be possible :dizzy_face:). Given that Ecto is already established as the goto library in that space, it’s most likely that new project will never get exposure on a curated list (unless this has been carefully considered for).

  3. Effective categorisation is a lot more tricky than it sounds. As someone gave the example of searching for “cache”. What would be the best cache package? What even is a cache? A cache related package could be applied to many different areas, solving many different problems. If the categorisation/search isn’t considerate of this, it could lead to suggested packages that don’t even solve what the person is looking for.

Now I don’t know the answer to… Well, any of those :sweat: But they are definitely issues I think need to be considered. And they are at least some of the reasons why I wouldn’t want hex to try tackle it, at least a separate service can afford to make those mistakes while discovering what’s the best way to go about solving this problem.

3 Likes

I agree with most of the people here that Hex is not the right place for this idea. But I think the difference between Hex and other package managers is that it is already backed by the Elixir community, largely and well-represented on this forum, previously on the mailing list, and also now on Slack.

Building on this awesome community, I have found out that we have sticky “wiki” posts for editors and IDEs: Spacemacs, Emacs, Atom, and others. If you are researching any particular packages with a specific use case, could you not create a specific post for that use case and explain the packages you’ve found? I know I was recently involved in a discussion on “happy pipes”/railway-oriented programming libraries. Each of the different libraries had strengths, weaknesses, flavors that could conceivably be grown into/turned into a wiki?

If you create said post, and it is well researched, then the sticky post could show the various packages that exist for the use case, profiling any package’s strengths. Then over time, it grows and evolves, depending on the need for the use case. You could include any information you find relevant there, and it would be driven by moderation from the community, just as this forum is in general.

But I’m not a mod and I don’t know if this is really feasible. :smile:

1 Like

We definitely encourage Wikis here on the forum - anyone at Trust Level 1 or higher can edit them and the wiki owner always gets notified of any changes. Have a look here for more info:

https://elixirforum.com/t/wikis-suggestions-thread/352

2 Likes

@ibgib: Yes, it frequently happens, both on the Slack chat group and on this forum, that people have questions like ‘I am facing problem X, what is the best package to help me solve it?’

The nice thing about that, is that multiple people can give a very specialised opinion, based on the state of the packages at that point in time, and based on the given problem’s exact circumstances. I think this is really valuable, and one of the things I love the most about this community :).

2 Likes

There is a solution to the problem. I glad that somebody made it

1 Like

So, here’s my issue with this kind of thing. I did a bit of searching for libraries related to 2 areas I know a lot about: AWS and GraphQL.

Hex.pm

Search term: results

AWS: ExAws at 188k+ downloads comes in first, followed by AWSAuth and aws at 3k downloads.
Graphql: Absinthe comes in first at 24.6k downloads, the graphql package comes in 3rd at 14k downloads.

Both of these give the correct answer. ExAws is wildly more feature complete and tested than the aws library. The holds true for Absinthe and GraphQL, and the latter project hasn’t had a commit in 3 months.

elixir.libhunt.com

Search term: results

AWS: AWSAuth comes in first for… no particular reason as far as I can tell? All the numbers on the page are lower than they are for other libraries. Next is aws, finally followed by ex_aws
Graphql: graphql package is up top, followed by their parser, followed by Absinthe.

How is this supposed to be better? Someone using this site to try to use AWS or start with GraphQL would be directed towards two projects neither if which constitutes the best the Elixir community has to offer in that area.

3 Likes

Searching “web” ranks sugar higher than pheonix. Grouping things by category is of no value if the results in a category send people in the wrong direction.

4 Likes

Mentioned this earlier, but I wonder if something like https://npms.io/about would be useful for Hex. I’d wanna build it for fun :smiley:

1 Like

LOL, this site is a joke. It’s just the Awesome Elixir repo README (not curated at all) which is parsed into a web page.

1 Like

Why you don’t take the idea and build it. Everyone agree on the fact that is an issue in our industry and at least that app is trying to fix it.

I encourage you to create something or just use the platform, giving feedback, using the ranking system, I don’t know but definitely something more than just laugh about it.

this is the reason why I don’t believe on 100% freedom. This is why I prefer having seniors with a lot of experiences driving the community decisions.

I don’t believe on the freedom when I know ignorant people can screw a decision. I am ignorant enough for no be able rank correctly the packages because I don’t know that much about ErLang or Elixir.

I prefer to create a role (like seniors inside the community) that you have to gain the truth and respect for be able to be in that role.

:pensive: I just want to see this issue to be fix. I bother me that we knowledge that is an issue and we don’t try to fix it.

My point wasn’t just that the site in question was doing a poor job. My point was also that Hex.pm is actually doing a very good job. If you search for a topic you get a relevant list of items, the description, and the downloads count which helps you figure out how much the community relies on the packages.

2 Likes

Yes, and that’s why I don’t want to use something external and actually extend Hex.pm.

Like at least have a badge that said: Recommended by the community; or something similar.

For non curated lists that particular feature can cause problems down the road. Meteor had a similar aggregate for all packages with metrics (including a fancy graph widget.)

The problem arises when you as a new user see a package has 99e99 downloads only to find out that it’s been abandoned for two years. Well, hey it used to be popular!

1 Like

I guess in the end it’s a mixture of existing solutions that yields best results but requires some amount of research:

  • total downloads
  • recent downloads (love that hex provides those)
  • README - aka what is this package good at, API etc.
  • is there a Changelog?
  • when was the last release?
  • how long has the package been around
  • github stars
  • open issues
  • open PRs
  • recent commit activity
  • blog posts
  • what do my friends use…

In the end no one can take the tough decision of what library to use of your hands and responsibility.

I think 3rd party websites can help with this, so while libhunt.com fails to sort the search correctly I think it’s a good step. I liked ruby-toolbox a lot but it is broken for some time now…

That is always the problem, someone has to maintain lists and whatever. A particular problem I have with a “committee” is who decides what gets in and who’s on that committee. For instance in Ruby land the most interesting stuff happening imo is rom-rb, hanami and trailblazer. For all I know, dhh probably doesn’t like them very much and I also heard that they’re not that well known/highly regarded in the U.S. I also think it’d be great if they’d take their talents to other languages (like… Elixir) but that might just be me :slight_smile:

I agree that rather than creating new stuff working together to improve existing libraries is preferable, it’s not always possible though. Sometimes project philosophies just diverge too much or a better solution to a problem requires a wholly different approach and it’s easier to implement something new than make a backwards incompatible release of library X (like lots of people said Angular2 should have had a new framework name).

Sometimes breaks are needed and a new thing needs to come up. Like shrine in ruby for instance. Also after evaluating all other benchmarking libraries I could find in Elixir I decided to make my own (a dormant PR and low test coverage among others later though). It’s hard to gain traction for new things though…

2 Likes

I think what hex could do better is providing more information, and let the user do their own decisions.