@imetallica, @karmajunkie, thanks for your replies. I guess I need to do more exercise on Elixir/Phoenix, including PubSub
s. thanks again!
Can you give an example how that would work with phoneix pubsub? I have searched the forum but couldnât find one.
Ideally, I wish I could put phoenix pubsub in itâs own app apps/pubsub
and point apps/web
to it.
config :web, Test.Web.Endpoint,
url: [host: "localhost"],
secret_key_base: "...",
render_errors: [view: Test.Web.ErrorView, accepts: ~w(html json)],
pubsub: [app: :pubsub] # where :pubsub is an app in umbrella
The problem Iâm having with pubsub being in apps/web
is that I canât be sure that Test.Web.Endpoint
is loaded when I make a call to it like Test.Web.Endpoint.subscribe
from some other app like apps/user
.
And if I am to use Phoenix.PubSub.subscribe
from inside apps/user
, then that (I think) requires me to add phoenix_pubsub
as a dependency to apps/user
.
Now that I think about it, it might be better to leave phoenix pubsub to deal with websockets and the like, and either write a global distributed registry/pubsub myself or use something like swarn [0]. Not sure though.
I am also not sure how some things are implemented in phoenix pubsub (Iâve been planning to read it, but Iâm afraid I donât know elixir well enough to understand it fully), for example, does it batch messages when passing between nodes or not? Because it seems it can be somewhat of a bottleneck [1] if every message is sent with send
.
[0] https://github.com/bitwalker/swarm
[1] http://www.ostinelli.net/boost-message-passing-between-erlang-nodes/
Something on the following should work:
def MyApp.MyPubSubListener do
use GenServer
alias Phoenix.PubSub
def publish(what), do: PubSub.broadcast SomeName, "channel_to_send_receive_messages", what
# Callbacks
def init() do
PubSub.subscribe SomeName, self, "channel_to_send_receive_messages"
{:ok, %{}} # The initial state of this GenServer.
end
def handle_info(message, state) do
# ... do something with your message/state ...
{:no_reply, state}
end
end
That should do the trick. Do not forget to start it in a supervision tree of sorts. And remember to create one for each channel/topic you want to subscribe.
So what happen when you want to change the database user schema? Since individual user schemas are all over the contexts and you donât have a single point of truth, will that become a headache when you introduce changes to these denormalized schemas?
Posted by @michalmuskala earlier:
http://michal.muskala.eu/2017/05/16/putting-contexts-in-context.html
I believe that it is important to delineate the difference between a normalized data model, a denormalized data model, a document-oriented data model and a segregated data model.
Normalized: This is the âold schoolâ approach to creating relational database structures, where fields are stored in separate tables and relations between them (such as JOINS) are used to gather the data combination(s) your application might desire.
Denormalized: Opposed to a normalized relational database, certain pieces of data are replicated in multiple tables, making sure that no inter-table queries are necessary, resulting in faster read speeds, but slower write speeds and a more complicated structure for the developer to maintain (making sure the copies of the data all stay up-to-date).
Document oriented: Every âuserâ might have different fields in the database, and care must be taken to handle missing fields. This is the storage approach that some NoSQL databases take.
Segregated: Here, there is no single point of truth, but this also means that there is no âthe database user schemaâ. The different contexts store different aspects of your user, which are referenced by the same symbolic identifier (which might be a UUID, for instance). The outside world does not care about the persistence layer that the different contexts use to store/retrieve their information from. We therefore might be talking about multiple different kinds of databases alltogether!
Of course, when your contexts are separated in a logical fashion (note, YMMV; exactly how to structure your contexts is very application-specific), there will be one context handling general user info, one context handling e.g. the set of products the user manages, one context handling the forum that is part of your app, etc. This means that only the âuser contextâ needs to know about most changes in your user, with the other contexts only knowing about the userâs UUID, and maybe from time to time asking the âuser contextâ if the specific user is authorized to perform a certain action.
So, to answer your question: If your separation of contexts is logical, then this will not become a headache.
will that become a headache when you introduce changes to these denormalized schemas?
Now I look back, I may or may not have expressed myself correctly. My question could be rephrased to âwill that become a headache when you introduce changes to these fragmented schemas?â
But your reply kind of also answered this question already. Thanks!
I still have some confusions about contextual schema. Guess itâs quite off topic, so I post a separate question at Contextual schema VS out-of-context schema (in Phoenix 1.3)
@chrismccord Thank you and the rest of the contributors for all your work. Elixir / Phoenix is what got me a taste of FP, and Iâve been trying to learn that style of programming ever since.
Now, on to my actual contribution to this thread. I got very lucky when I started learning Phoenix to have happened on Lanceâs Phoenix is Not Your Application talk before I even knew what Phoenix was. (Not really but you get what I am saying). I keep harping on Lanceâs talk as well as Wojtekâs talk on umbrellas. Because I built my app this way, I can update to Phoenix 1000 without breaking a sweat.
So, personally, I think I would always go the umbrella route. (And I gotta check this --umbrella
flag you mentioned). Nonetheless, I commend the Phoenix folks for taking on a lot of work, likely a lot of flak, and nudging end-users to move closer to the umbrella ideal.
@karmajunkie Thanks for the example you gave. Can you provide a simple sample in GitHub that less experienced programmers like us in elixir can look at?
For me, the truth is using phx is more difficult than Phoenix.
I have not been able to grasp things like handling relationship across context.
I live in my own world here in Nigeria and I have only found one person other than me using elixir. Samples can make things easier for me
Of importance is how to handle relationships between models.
How do you handle 1:M relationships?
What if a model in a context say A requires certain data available in another model in a different context B?
How do you ensure data integrity, such that changes made to a model A in context A is propagated to model A in context B?
Please your prompt response is highly appreciated
What if a model in a context say A requires certain data available in another model in a different context B?
Thereâs no correct answer to that question. Itâs just the perfect place to ask yourself, why you need to cross those context boundries.
E.g. if one context is the help desk and another one is the sales department then itâs quite strange that one does need to depend on a struct of the other. In that case Iâd add a new schema to the help desk requesting just the data the help desk does require from the db (multiple schemas of the same table are no problem in ecto).
If one context is the checkout context and the other your auth context holding the current user and itâs shopping cart, than itâs a relation, which feels quite natural. I personally wouldnât split that up into multiple context based schemas.
It has to be weighed if itâs better to add more code, but have the ability to evolve both contexts independently (probably the better choice for bigger projects / bigger teams) or to keep things lightweight and use some relations beyond context boundries if they make sense.
How do you ensure data integrity, such that changes made to a model A in context A is propagated to model A in context B?
With ecto itâs first and foremost the db. If you query your tables youâll get the updated data of them. As mentioned above you can have multiple schemas handle the same tables. Everything beyond that is in your own hands, like it was before contexts as well (e.g. two simultaneous update requests might overwrite the first one).
What if a model in a context say A requires certain data available in another model in a different context B?
Your first mistake here is in using the database as an integration layer between different contextsâall youâre doing is coupling together two things which are explicitly meant to be decoupled. If youâre going to use contexts as a service boundary, then they should be independent. This means that context B doesnât borrow the data of context Aâcontext A broadcasts to the world what it thinks the world ought to know, and context B maintains its own store of data based on what context A has to say. If these contexts are actually decoupled there probably isnât much that B is interested in from A.
How do you ensure data integrity, such that changes made to a model A in context A is propagated to model A in context B?
Thatâs the job of context B, listening to messages broadcast from context A.
If youâre dead-set on coupling the data in the way you describe, donât create artificial boundaries where youâre not enforcing a systemic boundary. Just use context Aâs schemas in context B. You can have a separate schema that refers to the same underlying database table from a different context; I have proffered this option in the past, but to be honest I donât think its a great ideaâjust a way of fitting a round peg in a square hole.
For me, the truth is using phx is more difficult than Phoenix.
I have not been able to grasp things like handling relationship across context.
Putting together a good example application that applies this style of architecture and design is not really trivial, and trivial examples just create more questions when you invariably get away from the happy path.
What I would recommend for you is to ignore the notion of contexts for now. Elixir doesnât actually care about them, and Phoenix really doesnât either outside of the generators. Learn all the other aspects of Elixir, OTP, and Phoenix firstâat some point, the context stuff will just come naturally. Its also good to do some reading around analysis patternsâthere are many ways of approaching architecture, and DDD isnât the best one in every situation.
What if a model in a context say A requires certain data available in another model in a different context B?
Why getter and setter methods are evil (Allen Holub, 2003)
Donât ask for the information you need to do the work; ask the object that has the information to do the work for you.
Replace âobjectâ in the above statement with âcontextâ and itâs still accurate. These Days Holub phrases this as âAsk for help, not informationâ.
A little bit earlier Andy Hunt and Dave Thomas talk about the âTell donât Askâ principle in:
The Art of Enbugging (pdf, IEEE Software January/February 2003) and
Tell, Donât Ask
Martin Fowler summarizes his take in TellDontAsk
âTell Donât Askâ is a consequence of Responsibility-Driven Design.
Practicing Ruby: Responsibility-centric vs. data-centric design (2012)
- Objects tend to be highly cohesive around their behavior, because roles are defined by behavior, not data.
In âcontext-speakâ:
Contexts tend to be highly cohesive around their capabilities.
Essentially, if your thinking is even somewhat rooted in the CRUD world your are going to have a hard time isolating contexts.
Essentially, if your thinking is even somewhat rooted in the CRUD world your
are going to have a hard time isolating contexts.
Spot on! Iâve noticed this design âproblemâ ever since the first MVC web
frameworks popped up. They are all quite centered around the database
table (the model) and generators and documentation show how to have 1
âContextâ/âApiâ/âBoundaryâ per table and then use them directly from
your code.
This goes through in everything and even form generators are done from
the model. Following this forces you into a design that is not good for
you and hard to get out of. It is also hard to change how you reason
about it.
Generally determining contexts is hard unless you know your domain well,
but you should not base it on your database tables, rather think what
your software is trying to solve and what service it provides.
In terms erlang everything is based around modules (which is true for
elixir as well). In erlang the âbest practiceâ is something like this:
A module should be coherent and loosely coupled. A module should have a
small, well defined and documented API.
An OTP application is a set of modules. The OTP application should have
1 module which is exposed to the âoutsideâ. It should have a small,
well defined and documented API. No implementation details should leak
outside of the application. That is, if you return a data-structure that
data-structure should be opaque and only modifiable by using provided
functions.
Modules within an OTP application can depend on each other, they are an
implementation detail.
I see contexts as something similar. A context should have a small, well
defined API and no implementation details shouild leak outside of the
context.
If you have a small, API you should easily be able to swap out the
implementation of a module/context/OTP application without affecting any
other code.
Because it seems it can be somewhat of a bottleneck [1] if every message is sent with
send
.[1] http://www.ostinelli.net/boost-message-passing-between-erlang-nodes/
This is an old article (2009) that relates to a very specific use case. AFAIK this does not even apply anymore.
AFAIK this does not even apply anymore.
So the messages are batched by default when sent to other nodes? I thought not, since there are some projects like GitHub - discord/manifold: Fast batch message passing between nodes for Erlang/Elixir. which explore approaches to do that similar to what youâve described in your article âŠ