Properly Testing Telegram integration within app

I am in the process of building an app that allows users to register through Telegram. I am moving from the Rails community to the Elixir community and have been trying to TDD this section of the application.

My application is using Nadia as the wrapper for the Telegram API. I have webhooks set up right now so when a user messages the bot it sends a POST request to my /webhooks route.

The webhooks route pattern matches for the data it needs and sends it to a CommandHandler.process function that then calls the appropriate function based on which command.

Users will be able to register for accounts solely through the Telegram platform and that is what I am trying to test right now. What is the appropriate way to test a user that sends a command that requires multi-responses from the bot (i.e. registering; asking for email, phone…)

That would have been easier to test if Nadia had some kind of middleware which would return the command struct like %Nadia.Command{name: "send_message", opts: [...]} but it doesn’t, it sends the message immediately. You might write a wrapper around Nadia with this functionality. That’s what I did for the commands I used (send_message, send_document). Then testing becomes a bit easier, you send the data you received in the webhook, and match the returned command struct with the one that was expected.

Or you can mock telegram endpoint like in the Nadia’s tests

I also needed the wrapper to spread out broadcasted messages so that telegram wouldn’t send me 429 Too Many Requests. So for me it wasn’t just for testing.

2 Likes

Did you ever have to handle chained messages within Telegram? This is another challenge I can’t seem to find much on how to tackle just yet. I would like to write an acceptance test that goes through the user registration interaction if possible.

Writing a wrapper is a good idea and I think will work with what I am doing. I’ll take a look at the stubbed tests too because that was what I was originally looking for.

This has really been pushing my technical ability because I’ve mostly build boring web applications.

There is something called force_reply, maybe it would work? But I’ve never used it.

It was something like this in Nadia (not sure)

Nadia.send_message(chat_id, "heya, enter your email, please", force_reply: true)

and the user had to enter some text, I guess? Otherwise the message wouldn’t disappear or something like that.

You might want to make a state machine (:gen_fsm or :gen_statem) which would go over the needed steps for each chat (register → enter email → confirm email and so on). And have a process keeping a map of chats that have entered this state machine maybe together with their current states. Then for each request to the webhook you would check if the chat in the request is in this map and act accordingly.

Here’s a funny intro to making state machines in Erlang http://learnyousomeerlang.com/finite-state-machines.

2 Likes

You are the best. I am pretty sure you’ve answered every question I’ve asked on this forum.

The most OTP thing that I have done has been working with GenServers. What would be the difference using a state machine vs spinning up a GenServer that holds all the chat_ids that are in the registration process and just checking each incoming chat_id against the state of the GenServer to see if it needs more handling? I’m just trying to wrap my head around the OTP/erlang world.

I’m going to start reading about state machines now.

1 Like

I think state machines in Erlang are basically genservers with some extra functionality, so not much difference.

What would be the difference using a state machine vs spinning up a GenServer that holds all the chat_ids that are in the registration process and just checking each incoming chat_id against the state of the GenServer to see if it needs more handling?

I think that would totally work.

Or (a bit different from what you suggested here and from what I suggested earlier) you could make a genserver that would hold just a mapset with chat_ids of those who entered the registration, and then have Registry lookup the registration process (by chat id) which would be a state machine (also a genserver, basically). That would be more inline with all this Erlang/OTP/“a process for everything” lifestyle.

That would include extra complexity though, you would need a supervisor to supervise these registration processes, and you would also need to have them registered with Registry to be able to look them up later.

1 Like

I’ve never worked with Registry before, but everything else seems to make sense. I am looking at having anywhere from 2-3000 concurrent users if this is accepted as the solutions, so I like the idea of making it more fault tolerant with supervisors.

I’m going to read more into Registry

Right now I decided that instead of using a force_reply, a user will enter the registration stage (a new GenSever starts with an empty map).

The user then can use commands to update the info:
/email "joe@example.com"

and this will update the state with their email added: %{email: "joe@example.com"}
and so on for other fields.

Once the user is ready to register there is a create_telegram_user command within the GenServer that takes the state and inserts it into a TelegramUser changeset and inserting it into the db.

My next step is creating the Registry and the GenServer to hold a MapSet of chat_ids.

My question now pertains to the supervision tree. Doesn’t the supervisor start the child? So if I defined a Supervisor and used a :one_for_one strategy and added the RegistrationServer as a child, how do I start up a new GenServer for every user that engages and have it linked to the Supervisor?

how do I start up a new GenServer for every user that engages and have it linked to the Supervisor

For that you need a new supervisor with :simple_one_for_one strategy (in elixir 1.7 it will be replaces by DynamicSupervisor which better describes what it does), which would be in the children list of your :one_for_one supervisor which also starts the registry and RegistrationServer.

defmodule My.Application do
  @moduledoc false
  use Application

  def start(_type, _args) do
    # List all child processes to be supervised
    children = [
      {Registry, keys: :unique, name: My.Registry},
      My.Registration.Supervisor
    ]

    # See https://hexdocs.pm/elixir/Supervisor.html
    # for other strategies and supported options
    opts = [strategy: :one_for_one, name: My.Supervisor]
    Supervisor.start_link(children, opts)
  end
end

and

defmodule My.Registration.Supervisor do
  @moduledoc "Supervises registration processes"
  use Supervisor

  @spec start_link(any) :: Supervisor.on_start
  def start_link(_opts) do # don't care about opts
    Supervisor.start_link(__MODULE__, [], name: __MODULE__)
  end

  @doc "Starts a registration process for a user"
  @spec start_registration(My.User.telegram_id) :: Supervisor.on_start_child
  def start_registration(telegram_id) do
    Supervisor.start_child(__MODULE__, [telegram_id])
  end

  @doc "Stops a registration process"
  @spec stop_registration(My.User.telegram_id) :: true
  def stop_registration(telegram_id) when is_integer(telegram_id) do # what if you pass a string, I sometimes did and couldn't understand why it wasn't working
    case Registry.lookup(My.Registry, {:registration, telegram_id}) do
      [] -> true
      [{pid, _}] -> Process.exit(pid, :shutdown) # or something like this see https://hexdocs.pm/elixir/Process.html#exit/2 for more ways to stop a process
    end
  end

  @spec init(list) :: {:ok, {:supervisor.sup_flags, [:supervisor.child_spec]}} | :ignore
  def init([]) do
    children = [
      worker(My.Registration, [], restart: :transient) # restrarted only on fail (not on success)
    ]

    opts = [
      strategy: :simple_one_for_one
    ]

    supervise(children, opts)
  end
end
defmodule My.Registration do
  @moduledoc "Keeps the state of a registration process for a user"

  use GenServer

  defstruct [
    telegram_id: nil,
    state: %{} # or something else
  ]

  @doc """

    * `:telegram_id` - telegram ID, equals to `:telegram_id` in corresponding `%My.User{}`

    * `:state` - current state of the registration process

  """
  @type t :: %__MODULE__{
    telegram_id: My.User.telegram_id, # non_neg_integer
    state: map
  }

  @spec start_link(My.User.telegram_id) :: GenServer.on_start
  def start_link(telegram_id) do
    GenServer.start_link(__MODULE__, telegram_id, name: via(telegram_id))
  end

  @spec via(My.User.telegram_id) :: {:via, module, {module, {:registration, My.User.telegram_id}}}
  def via(telegram_id) when is_integer(telegram_id) do
    {:via, Registry, {My.Registry, {:registration, telegram_id}}}
  end

  @spec init(My.User.telegram_id) :: {:ok, t}
  def init(telegram_id) do
    {:ok, %__MODULE__{telegram_id: telegram_id}}
  end

  # ... other functions for handling registration
end
3 Likes

This looks like exactly what I need! I seriously can’t thank you enough. This is my first semi-complex elixir app and I’ve been learning on my own so I am not even sure how close I am to doing things the elixir way.

I don’t know how I missed the simple_one_for_one before, but it makes a lot more sense. Once I get it integrated I’ll post an update how it works.

Success!

I think I have, for the most part, integrated your advice into my app. I have a registry that maps telegram_id’s to the RegistrationServer process that is responsible for building out a TelegramUser. I was able to set up a Supervisor with a :simple_to_one strategy and a method that starts new registration processes.

This has really opened my eyes to OTP and how cool it is. I can’t tell you how grateful I am that you’ve taken this time. The next step I have is creating a GenServer that holds the currently registering telegram_ids.

2 Likes

I’m glad I could help.

For learning more about OTP I would suggest reading Learn You Some Erlang by Fred Hébert. Yeah it’s about erlang and not elixir but they are practically the same.

2 Likes