Performance guide and a design pattern, learning path

Question:
Can someone recommend me some good resources on learning performance for phoenix elixir applications and a design pattern I should use to develop my apps?

Thanks in advance for any suggestions on this.

2 Likes

I would say that you can start by reading the hexdocs and have a general knowledge of what is possible to do with it.

Then depending of what you want to do with the application, might have several possibilities of optimizations / performance. I would recommend also if you are doing something with HTML/JavaScript to take a look into LiveView.

Here are some topics about specific things, but not necessary related to Phoenix.


3 Likes

OK, thank you for your answer and guide on this matter.

What i looking for when I asked this question is this:

Minimal calls to the database

Fast performance 2=2 also (2-1)*2 is also 2 but takes more work so I am looking to understand how I should structure my apps and what functions to use to gain performance. A example would be that Enum.map is slower then a recursive head tail and acc.

How to cache correctly and only what is needed.

So if anybody else would like to add there personal opinions or experiences I would be grateful.

Thanks in advance.

I’m not sure there is very much value in generic advice to give for your questions. There’s https://github.com/devonestes/fast-elixir, which is useful to have seen, but I don’t think it’s a good guideline for general programming. It promotes testing performance changes I guess, which is a good thing.

In general I’d like to point to the often quoted Joe Armstrong:

Make it work, then make it beautiful, then if you really, really have to, make it fast. 90 percent of the time, if you make it beautiful, it will already be fast. So really, just make it beautiful!

So unless you really need to – use Enum.map over a custom, optimised recursive function. It’ll be more expressive, easier to read and maintain.

Sticking to the quoted advice has not only the benefit of the generic meaning, but also that once you hit the “make it fast” part you’ll already know your problem quite well. So you can go around and ask people about “how to make X faster”, which is usually way more productive.

This is even less answerable in a generic way. Maybe cache as few things as possible and as much as needed? What to cache highly depends on what data you handle (how big, shape, …), from where you retrieve it and how fresh it needs to be for your users to be useful. Big, hardly changing data, which is fine to get stale -> great for caching; Constantly changing data -> not so great. Retrieval of some piece of data is too slow? Check how long the data can be stale.

I hope you see that I’m hardly talking about programming at this point. This is about constraints to the business/users/sources of the service your app provides. It takes talking to those to find out what you can or cannot do.

Edit:

I guess one generic thing to know would be how different datatypes work and how they interact with Algorithmic complexity. It’ll give you topics/vocabulary to talk about with other people. Things like: Appending is expensive for linked lists.

9 Likes

The link towards the GitHub repo is very useful to me.

Also you other thought on this matter are very helpful for me to understand better my requirements and possibilities.

Thanks for writing you idea and opinions

Also is there something similar to POODR from ruby on rails in elixir phoenix?

I am asking because i know alot of memebrs of this forum come form ruby on rails so maybe some of you know of good book or resource similar to the one above.

I’m not familiar with Ruby neither POODR, but in terms of books I can recommend Elixir in Action by @sasajuric.

Also have a general overview how GenServers, Ets and NIFs work. Maybe GenServers and Ets will more useful for you to know than NIFs, but knowing that they exists might be handy.

5 Likes

Thanks for the recommendations.

Also found this for anyone that wants to go this path of learning design patterns:

https://elixirforum.com/tags/design-patterns

There are a couple of threads on this that really caught my eye.

I found this video while looking at functional design patterns, I like it a lot…

6 Likes

Thanks i will watch it and come back with my impressions, if that is alright.

1 Like

I will update this as I am watching because it’s a lot of info to cover:

This is my personal opinion on the video provided by @kokolegorille with the title Functional Design Patterns

Also feel free to offer feedback on this as i am editing thanks.

So I like very much how the gentleman Scott Wlaschin uses a Summary for his presentation and explains what to expect from his talk and what not. Then he uses familiar concepts to present what you can learn and how much deep, he will cover the topics that were in the summary.

Summary of the talk:

  • Core principles of Functional Programming design

    • Functions types and composition
  • Functions as parameters

    • Functions as interfaces
    • Partial application & dependency injection
    • Continuations, chaining & the pyramid of doom
  • Monads

    • Error handling, Async
  • Maps

    • Dealing with wrapped data
  • Monoids

    • Aggregating data and operations

So let’s start with the first part of the summary:

Core principles of Functional Programming design

  • Functions types and composition

The introduction to functions and the possibility of using them as input and an output was made very easy by using the analogy of fruits and also by providing an real world example later on.

Also the use of diagrams to represent the functions and how the data flows helped me understand better the principles of function composition.

Thus the following quote is born:

Composition can only be achieved using a single parameter function

The real world example was how an web application can be built from multiple functions that were composited together to form the final application.

I think a good analogy to composition would be building a snow man from snow. You can make different parts and then glue them together. Too form the whole snow man.

Types can also be put together or composited in a single big function and the analogy of the salad made this very easy to understand the function of AND. The OR function was also easy to grasp when the analogy of the snack was used to show the difference between the two.

The use case of OR and AND was present in a real world example related to card processing scenario which made the whole theory go into practice really nice.

Then we go to THE DESIGN BY TOTALITY

Using the types to enforce some rules really helped me to see the usefulness of guards from elixir here and also the need of constraints on data to handle each possible scenario so that my application doesn’t return unexpected results.

The second part:

  • Functions as parameters
    • Functions as interfaces
    • Partial application & dependency injection
    • Continuations, chaining & the pyramid of doom

Creating function using generic and parameter-ize everything for a dry code really impressed me and offered a good example of what decoupling can do.As a result the function become much more flexible and the DRY principle took charge.

The functions as interfaces part really change the way I was thinking about sharing the same functions with other functions. Basically if we have multiple functions that need the same data types then an interface is a good way to keep the code DRY. Probably that the equivalent in elixir for this part would be the behaviors specifications.

Then we go to THE STRATEGY PATTERN

That adds and extra parameter to function that leads to another function that contains the strategy.

Then we go to THE DECORATOR PATTERN

That makes sure that the first function and the last function keep there expect types. An example offered in this talk was like this

int is_even bool

So between the int there is a function is_even and it expects a result in a type of the bool true or false. But it’s can’t return a result of a different type like string or int.

And no matter how many compositions o function i make the input should be an int and the result should also be returned as a bool.

I think this part is similar to guard in elixir where only certain types can be returned.

So every function is a single parameter function in functional programming

Then the author provides some examples to prove that what he stated above is true and the following qoute is born:

Every multiple parameter function can be converted into a single parameter function

Then we go to THE PARTIAL APPLICATION PATTERN & DEPENDENCY INJECTION

Chunks of the function are taken out and converted once again into a SINGLE PARAMETER FUNCTION

Then to demonstrate this pattern in a real example using the retrieval of a customer from the database.

Also in this example appears the PERSISTENCE IGNORANCE, which can be translated as I don’t care from which source my data comes. In our example we don’t care if the Customer comes from server or a cache.
As far as i know elixir doesn’t have something similar for dependency injection. The closest Thing i could find was this post How to use dependency injection pattern in Elixir?.

Then we go to the sow on the road with THE HOLLYWOOD PRINCIPLE’*’: continuation PATTERN

Also known as DON’T CALL US WILL CALL YOU

Then we visit the pyramids of doom, doesn’t sound like dungeon of the game, weird right? But it is not game, it’s the multiple null check hell or nested null. In elixir we use a monad every time when we define a module or a function and that is the… you guessed it the DO.

Also monads can be defined as chaining continuation.

Then this hit me in the video:

How do we combine mismatch functions?

We glue them together using bind

Bind all the things

So transforming a one input and 2 outputs in match of 2 inputs and 2 outputs will allow us to convert the pyramid of doom into single parameter function. Well that is good but what do we do in elixir, we don;t have bind could we use cond?

Then the author goes in demonstrating this principle using an error validation on certain database transactions.

OK so now we go on a train ride because this type of solving this particular problem is called by the author Railway Oriented Programming

So what can I say as an ending. Definitely a nice talk in an easy and fun way to look at FP principles and some patterns that could be used in different situations.

A must see video for an easy introduction to FP(Functional Programming) Programming

Thanks @kokolegorille for showing to me this video.

Also If i made some mistakes let me know and i will correct them.

Time for me to fly, I hope that my insight would be enjoyed by others.

2 Likes

Also if someone can point out the elixir equivalents in this talk i would be very grateful.

1 Like

The author of the talk is also the author of…

After watching this talk, I bought the book, even if it has only F# code

I tried to translate types in this post Type and spec - Dialyzer not detecting error

But it’s hard to compare type_spec with F# type system :slight_smile:

3 Likes

Thanks for the link explaining the types of F# and also the book link

yes typed language is typed language … :slight_smile:
specially like ML languages

1 Like

We don’t :slight_smile:

the Elixir way is to use with do… end

As mentionned, F# belongs to ML family, while Erlang is FP only to facilitate CSP.

To quote Joe Armstrong in a previous post, Forget about monads, at least in Erlang/Elixir.

The most significant thing I have learned is to understand the concurrent flow of data through your system – a common mistake is to throw GenThingies at every piece of data; Elixir is not an OOP environment.

So, for example, data flows into your system from HTTP request, or from some database call. At this point, your DB layer or HTTP server will probably have a function invocation per request.

Ensure that each request can be processed through as many functions as possible – not processes as possible. Each move of data between processes, and possibly GenServers, is another opportunity for copying data, moving things into mailboxes (a singleton queue), and slows you down.

Make sure you know what each function is doing, for example ConCache is excellent, but you need to be aware that it is using ETS tables behind the scenes. ETS requires copying data in & out of the heap. Other modules will have GenServers with a functional interface - usually for good reason, but again, another opportunity to block things in mailboxes/queues if that GenServer is handling multiple user requests itself.

If all your data flows smoothly through functions, minimising process changes, then this allows the BEAM to maximise its performance without needing to shuffle data around in memory.

Make it work, then make it beautiful, then if you really, really have to, make it fast. 90% of the time, if you make it beautiful, it will already be fast. So really, just make it beautiful!
—Joe Armstrong

Wiser words than that are harder to find around here.

1 Like

Thanks for providing your experience and suggestions to this discussion