sean

sean

PartitionDispatcher Consumers not receiving events?

Playing around with getting a basic GenStage with PartitionDispatcher working.

Unfortunately my Consumers are never receiving any events.

GitHub repo with my code: GitHub - hauntedhost/gen_stage_partition_dispatcher: Experimenting with GenStage PartitionDispatcher. · GitHub

And here is a trimmed view of the relevant code:

  # PRODUCER

  def init(initial_state) do
    dispatcher_options = [
      partitions: partitions_list,
      hash: &partition_by_first_letter/1
    ]
    {:producer, initial_state, dispatcher: {PartitionDispatcher, dispatcher_options}}
  end

  def handle_demand(demand, state) when demand > 0 do
    words = Enum.map(1..demand, fn(_) -> Faker.Lorem.word end)
    {:noreply, words, state}
  end

  def partitions_list do
    ~w(one two three four)
  end

  def partition_by_first_letter(event) do
    partition = case first_char_downcase(event) do
      c when c in ~w(a b c d e f g h i) -> "one"
      c when c in ~w(j k l m n o p q r) -> "two"
      c when c in ~w(s t u v w x y z)   -> "three"
      _                                 -> "four"
    end
    {event, partition}
  end

  # CONSUMER

  def init(initial_state) do
    {:consumer, initial_state}
  end

  def handle_events(words, _from_producer, state) do
    IO.puts("=== received words ===")
    IO.inspect(words)

    {:noreply, [], state}
  end

  # START

  {:ok, producer} = Producer.start_link
  Producer.partitions_list
  |> Enum.each(fn(partition) ->
    {:ok, consumer} = Consumer.start_link
    GenStage.sync_subscribe(consumer,
      to: producer,
      partition: partition,
      min_demand: 5,
      max_demand: 10
    )
  end)

Any tips appreciated. :slight_smile:

Thanks,
Sean

Marked As Solved

josevalim

josevalim

Creator of Elixir

Done, this bug has been fixed in master. You can use strings as your partitions now although I would recommend using atoms as they are faster for comparisons.

Where Next?

Popular in Questions Top

Emily
I have VueJS GUIs with the project generated using Webpack. I have Elixir modules that will need to be used by the VueJS GUIs. I forese...
New
albydarned
Hello all! I am typing this post from my new MacBook Pro with the M1 chip. I’m loving it so far, and will probably use it as my daily dr...
New
shahryarjb
Hello, I have map which I want to convert it to string like this: the map: %{last_name: "tavakkoli", name: "shahryar"} the string I ne...
New
johnnyicon
Hi all, I’ve just started learning Elixir and Phoenix Framework, so please pardon my n00bness at this stage. I’m trying to use Postgres...
New
pmjoe
I have a relationship of love and hate with Elixir. Lots of things are just absolutely right, but there are some things that are kind of ...
New
JulienCorb
I am trying to implement my new.html.eex file to create new posts on my website. new.html.eex: <h1>Create Post</h1> <%= ...
New
jason.o
In the code below, if the create action is not set to accept “extra_key” as an input, it errors out with a message shown above. Is there ...
New
dblack
I’ve got an issue with an app and I’ve no idea of how to troubleshoot it. I’m hoping someone here might have seen something similar. I p...
New
chensan
I have a User schema with a :from_id field set to type :string: defmodule TweetBot.Repo.Migrations.CreateUsers do use Ecto.Migration ...
New
earth10
Hi, I’m just starting to build a side-project with Elixir and Phoenix and doing some basic test with Elixir alone. What strikes me is th...
New

Other popular topics Top

siddhant3030
Hi, I have to write a raw query for one of my project. But till now I have used ecto queries and don’t have much experience writing raw ...
New
sen
Hi All, I set a environment variables in dev.exs , like below code. when i start server, how can i set the ${enable} value? thanks. d...
New
stefanchrobot
What’s the safe way to decode a JSON string into a struct? I want to avoid calling String.to_atom. Jason.decode can give me a map with st...
New
AngeloChecked
What learn first? Rust or Elixir Hi Elixir community! I’m here because i want learn a new language. I’m a junior developer and mainly i ...
New
AstonJ
Please see the new poll here: Which code editor or IDE do you use? (Poll) (2022 Edition) It’s been a while since we first asked this, I...
208 31307 143
New
vonH
When I run the Plug and I recompile I wind up having to use Ctrl C to quit iex and start again. Witht the help of rlwrap I can use the cu...
New
grych
Hi folks, Few months ago I have announced the proof-of-concept of the library to manipulate the browsers DOM objects directly from Elixi...
639 52774 488
New
nobody
Hi! In PHP: $_SERVER[‘SERVER_ADDR’] - in Elixir? Searched the docs for ip address and the web, no good results. Thanks!
New
openscript
Hello! Sorry for this astonishing simple question, but I’m really stuck. I try to set up the intellij-elixir plugin, but I don’t know ho...
New
vegabook
I’m brand new to Phoenix and I have stripped one of the demo applications to the bone. I just want to get an svg up on the screen. Here i...
New

We're in Beta

About us Mission Statement