How to determine contexts with Phoenix 1.3

Well, depends on how you want to approach it.

The first approach I would see is to have tables namespaced by context with no relations cross-contexts schemas. So you would have two different tables for users - one in the services context and another in the account context. The tricky part is to coordinate changes between then, which you could do using transactions - again, passing through all contexts that have the user schema for example if you want to change the user. The other way is to create a small GenServer module, where you would do a simple PubSub registration/listening and, everytime one of your users tables are changed it should broadcast a message with relevant information that will be captured by the other PubSub GenServers, so it can be propagated into their own versions of users.

The first approach albeit simple, when you need to scale into microservices your monolith will force you towards the second approach and you’ll have lots of pain because you coupled your cross-contexts relations in the same app. The second approach is more decoupled and, as I stated before, can be moved towards its own service, separated database whatsoever without you caring much of what’s going on.

It’s either a mixture of personal taste and what you’ll need. Do you think you’ll need to scale to millions of users? If yes, then, I would go second approach. If not, well, the first one is cheap and does the job as well.

1 Like

I think your approach is pretty well tied to the Rails way of thinking here. What you’re trending towards is tightly coupled contexts, which is worse than just building a monolith.

For example, why are you storing them in the database? A user being in a room is inherently session-based—this is where you should maybe reach for a GenServer instead of writing to the database. If you want to store a list of rooms the user should be automatically logged into when returning, maybe consider a context for user preferences. Then a layer higher up can orchestrate between the preferences and the room subscriptions.

Secondly, why does the room need to know anything about the user, like its account information? Would simply telling it which IDs or nicks that are currently in the room suffice for it to do its job?

Try to take a step back and ask why it seems hard when you run into questions like this. Generally I find that trying to puzzle where a particular notion goes is a signal to me that I’ve gotten a boundary wrong.

3 Likes

I really like the second approach. I personally I am struggling to understand the GenServer stuff. I don’t know if you can write a sample on how PubSub can be achieved.

I read about bounded context yesterday and I was wondering the best way to implement it in Phoenix.

Thanks for your response

@smithaitufe I suggest you create a small elixir application and play with Supervisors, GenServers, GenStages, etc… :slight_smile:

@imetallica Thanks. I have started studying those topics now. GenServer is making sense now.

I don’t know if you can write a sample on how PubSub can be achieved.

I’m not sure if that’s what you need, but have you checked Registry module in elixir? It has a note about how to use it as a local pub/sub system in the docs [0].

You can put it in a separate app under an umbrella and let processes from other apps subscribe to certain topics in it.

apps/pubsub/lib/pubsub.ex

defmodule Test.PubSub do

  def start_link do
    Registry.start_link(:duplicate, __MODULE__)
  end

  def subscribe(topic) do
    Registry.register(__MODULE__, topic, [])
  end

  def unsubscribe(topic) do
    Registry.unregister(__MODULE__, topic)
  end

  def publish(topic, message) do
    Registry.dispatch(__MODULE__, topic, fn entries ->
      for {pid, _value} <- entries, do: send(pid, message)
    end)
  end
end

apps/pubsub/lib/pubsub/application.ex

defmodule Test.PubSub.Application do
  @moduledoc false

  use Application

  def start(_type, _args) do
    import Supervisor.Spec, warn: false

    children = [
      supervisor(Test.PubSub, [])
    ]

    opts = [strategy: :one_for_one, name: Test.PubSub.Supervisor]
    Supervisor.start_link(children, opts)
  end
end

To subscribe for updates in user app in a phoenix channel you would do something like

defmodule Test.Web.UserChannel do
  use Test.Web, :channel

  def join("user:" <> name = topic, _params, socket) do
    {:ok, _owner_pid} = Test.PubSub.subscribe(topic) # subscribe to updates in "user:#{name}"
    {:ok, socket}
  end

  # and handle these updates
  def handle_info(:sacked, socket) do
    broadcast! socket, "sacked", %{}
    {:noreply, socket}
  end
end

Somewhere in the user app you would publish the message :sacked in case of a ban

...
Test.PubSub.publish("user:idiot", :sacked)
...

These are basically the examples from the Registry docs, and there are some more about other possible uses for it.

Note however that this pub/sub implementation is local. And if you are using phoenix, you might want to pick Phoenix.PubSub [1] instead which is not local, so that for the example above (with channels) you would probably use phoenix’s pub/sub library.

[0] https://hexdocs.pm/elixir/master/Registry.html#module-using-as-a-pubsub
[1] https://hexdocs.pm/phoenix_pubsub/Phoenix.PubSub.html

Using Phoenix’s PubSub is better for this because it can broadcast messages between nodes, either using Erlang’s pg module, Redis or RabbitMQ. I think there’s a Kafka as well, but I’m not sure.

2 Likes

@imetallica, @karmajunkie, thanks for your replies. I guess I need to do more exercise on Elixir/Phoenix, including PubSubs. thanks again!

Can you give an example how that would work with phoneix pubsub? I have searched the forum but couldn’t find one.

Ideally, I wish I could put phoenix pubsub in it’s own app apps/pubsub and point apps/web to it.

config :web, Test.Web.Endpoint,
  url: [host: "localhost"],
  secret_key_base: "...",
  render_errors: [view: Test.Web.ErrorView, accepts: ~w(html json)],
  pubsub: [app: :pubsub] # where :pubsub is an app in umbrella

The problem I’m having with pubsub being in apps/web is that I can’t be sure that Test.Web.Endpoint is loaded when I make a call to it like Test.Web.Endpoint.subscribe from some other app like apps/user.

And if I am to use Phoenix.PubSub.subscribe from inside apps/user, then that (I think) requires me to add phoenix_pubsub as a dependency to apps/user.

Now that I think about it, it might be better to leave phoenix pubsub to deal with websockets and the like, and either write a global distributed registry/pubsub myself or use something like swarn [0]. Not sure though.

I am also not sure how some things are implemented in phoenix pubsub (I’ve been planning to read it, but I’m afraid I don’t know elixir well enough to understand it fully), for example, does it batch messages when passing between nodes or not? Because it seems it can be somewhat of a bottleneck [1] if every message is sent with send.

[0] https://github.com/bitwalker/swarm
[1] http://www.ostinelli.net/boost-message-passing-between-erlang-nodes/

Something on the following should work:

def MyApp.MyPubSubListener do
  use GenServer
  alias Phoenix.PubSub

  def publish(what), do: PubSub.broadcast SomeName, "channel_to_send_receive_messages", what

  # Callbacks

  def init() do
    PubSub.subscribe SomeName, self, "channel_to_send_receive_messages"
    {:ok, %{}} # The initial state of this GenServer.
  end

  def handle_info(message, state) do
    #  ... do something with your message/state ...

    {:no_reply, state}
  end
end

That should do the trick. Do not forget to start it in a supervision tree of sorts. And remember to create one for each channel/topic you want to subscribe.

So what happen when you want to change the database user schema? Since individual user schemas are all over the contexts and you don’t have a single point of truth, will that become a headache when you introduce changes to these denormalized schemas?

Posted by @michalmuskala earlier:

http://michal.muskala.eu/2017/05/16/putting-contexts-in-context.html

I believe that it is important to delineate the difference between a normalized data model, a denormalized data model, a document-oriented data model and a segregated data model.

Normalized: This is the ‘old school’ approach to creating relational database structures, where fields are stored in separate tables and relations between them (such as JOINS) are used to gather the data combination(s) your application might desire.

Denormalized: Opposed to a normalized relational database, certain pieces of data are replicated in multiple tables, making sure that no inter-table queries are necessary, resulting in faster read speeds, but slower write speeds and a more complicated structure for the developer to maintain (making sure the copies of the data all stay up-to-date).

Document oriented: Every ‘user’ might have different fields in the database, and care must be taken to handle missing fields. This is the storage approach that some NoSQL databases take.

Segregated: Here, there is no single point of truth, but this also means that there is no ‘the database user schema’. The different contexts store different aspects of your user, which are referenced by the same symbolic identifier (which might be a UUID, for instance). The outside world does not care about the persistence layer that the different contexts use to store/retrieve their information from. We therefore might be talking about multiple different kinds of databases alltogether!
Of course, when your contexts are separated in a logical fashion (note, YMMV; exactly how to structure your contexts is very application-specific), there will be one context handling general user info, one context handling e.g. the set of products the user manages, one context handling the forum that is part of your app, etc. This means that only the ‘user context’ needs to know about most changes in your user, with the other contexts only knowing about the user’s UUID, and maybe from time to time asking the ‘user context’ if the specific user is authorized to perform a certain action.

So, to answer your question: If your separation of contexts is logical, then this will not become a headache.

Now I look back, I may or may not have expressed myself correctly. My question could be rephrased to “will that become a headache when you introduce changes to these fragmented schemas?”

But your reply kind of also answered this question already. Thanks!

I still have some confusions about contextual schema. Guess it’s quite off topic, so I post a separate question at Contextual schema VS out-of-context schema (in Phoenix 1.3)

1 Like

@chrismccord Thank you and the rest of the contributors for all your work. Elixir / Phoenix is what got me a taste of FP, and I’ve been trying to learn that style of programming ever since.

Now, on to my actual contribution to this thread. I got very lucky when I started learning Phoenix to have happened on Lance’s Phoenix is Not Your Application talk before I even knew what Phoenix was. (Not really but you get what I am saying). I keep harping on Lance’s talk as well as Wojtek’s talk on umbrellas. Because I built my app this way, I can update to Phoenix 1000 without breaking a sweat.

So, personally, I think I would always go the umbrella route. (And I gotta check this --umbrella flag you mentioned). Nonetheless, I commend the Phoenix folks for taking on a lot of work, likely a lot of flak, and nudging end-users to move closer to the umbrella ideal.

1 Like

@karmajunkie Thanks for the example you gave. Can you provide a simple sample in GitHub that less experienced programmers like us in elixir can look at?

For me, the truth is using phx is more difficult than Phoenix.

I have not been able to grasp things like handling relationship across context.

I live in my own world here in Nigeria and I have only found one person other than me using elixir. Samples can make things easier for me

Of importance is how to handle relationships between models.
How do you handle 1:M relationships?
What if a model in a context say A requires certain data available in another model in a different context B?
How do you ensure data integrity, such that changes made to a model A in context A is propagated to model A in context B?

Please your prompt response is highly appreciated

What if a model in a context say A requires certain data available in another model in a different context B?

There’s no correct answer to that question. It’s just the perfect place to ask yourself, why you need to cross those context boundries.

E.g. if one context is the help desk and another one is the sales department then it’s quite strange that one does need to depend on a struct of the other. In that case I’d add a new schema to the help desk requesting just the data the help desk does require from the db (multiple schemas of the same table are no problem in ecto).

If one context is the checkout context and the other your auth context holding the current user and it’s shopping cart, than it’s a relation, which feels quite natural. I personally wouldn’t split that up into multiple context based schemas.

It has to be weighed if it’s better to add more code, but have the ability to evolve both contexts independently (probably the better choice for bigger projects / bigger teams) or to keep things lightweight and use some relations beyond context boundries if they make sense.

How do you ensure data integrity, such that changes made to a model A in context A is propagated to model A in context B?

With ecto it’s first and foremost the db. If you query your tables you’ll get the updated data of them. As mentioned above you can have multiple schemas handle the same tables. Everything beyond that is in your own hands, like it was before contexts as well (e.g. two simultaneous update requests might overwrite the first one).

1 Like

Your first mistake here is in using the database as an integration layer between different contexts—all you’re doing is coupling together two things which are explicitly meant to be decoupled. If you’re going to use contexts as a service boundary, then they should be independent. This means that context B doesn’t borrow the data of context A—context A broadcasts to the world what it thinks the world ought to know, and context B maintains its own store of data based on what context A has to say. If these contexts are actually decoupled there probably isn’t much that B is interested in from A.

That’s the job of context B, listening to messages broadcast from context A.

If you’re dead-set on coupling the data in the way you describe, don’t create artificial boundaries where you’re not enforcing a systemic boundary. Just use context A’s schemas in context B. You can have a separate schema that refers to the same underlying database table from a different context; I have proffered this option in the past, but to be honest I don’t think its a great idea—just a way of fitting a round peg in a square hole.

Putting together a good example application that applies this style of architecture and design is not really trivial, and trivial examples just create more questions when you invariably get away from the happy path.

What I would recommend for you is to ignore the notion of contexts for now. Elixir doesn’t actually care about them, and Phoenix really doesn’t either outside of the generators. Learn all the other aspects of Elixir, OTP, and Phoenix first—at some point, the context stuff will just come naturally. Its also good to do some reading around analysis patterns—there are many ways of approaching architecture, and DDD isn’t the best one in every situation.