Dsm - Declarative finite state machines and pipelines

Your Libraries & OS Mentoring Libraries
,
1

Hello,

I’m excited to introduce dsm, a zero-dependency abstraction library that allows you to define Finite State Machines and pipelines in a declarative manner.

Why?

I was building a networking app that dealt with enveloped messages. Upon receiving a message the app has to unwrap the message, read the type of the message, decode the message and then process accordingly. There were about 15 different message types and I ended up writing 15 case statements. While the code looked good, I yearned for a better way to describe the transformations and the processing sequence. dsm was built for this.

How is this different from other FSM libraries?

All FSMs and associated FSM libraries expect an input event which tiggers a state transition.

I’ve observed that many libraries expect this event to be defined while defining the state machine. My issue with this was that I couldn’t define an event at compile time, instead I wanted to define functions that decide whether the input event is valid and a state transition can be triggered.

An example dsm definition to read and parse a .wav file would look like the following snippet

defmodule Example.Demo do
  import Example.WaveFormat
  import Example.Validations
  use Dsm

  dsm do
    [
      name: "wav-file-reader-state-machine",
      initial_state: :ready,
      all_states: [:ready]
    ]
  end

  dsm_state :ready do
    [
      transformations: [&read_file_to_bytes/1],
      msg_handler_pipeline: [
        {&valid_read_output?/1, &read_riff_chunk/1},
        {&valid_intermediate_output?/1, &read_fmt_subChunk/1},
        {&valid_intermediate_output?/1, &read_data_subChunk/1}
      ],
      error_handlers: [],
      telemetry: &__MODULE__.telemetry/2
    ]
  end

  def telemetry({state, m, f, a}, data) do
    IO.inspect({"Telemetry Data for state :#{state} -- #{m}.#{f}/#{a}", data})
  end
end

I would very much appreciate it if you could use dsm and provide any feedback you may have on this.

The components and structure are documented in detail in my Github and on Hexdocs.

Thanks!

Hex url: dsm | Hex

Github:

1 Like
2

As an author of finitomata, I’d be interested what does “I couldn’t define an event at compile time” exactly mean. Your example could be easily rewritten in finitomata as (non-tested)

 defmodule Example.Demo do

  @fsm """
    ready --> |read_output| reading_riff_chunk,failed
    reading_riff_chunk --> |read_fmt_subchunk!| reading_fmt_subchunk,failed
    reading_fmt_subchunk --> |read_data_subchunk!| success,failed
  """

  use Finitomata, fsm: @fsm, auto_terminate: false

  @impl Finitomata
  def on_transition(:ready, {:read_output, file}, _, state) do
    case read_file_to_bytes(file) do
      {:ok, bytes} -> {:ok, :reading_riff_chunk, Map.put(state, :bytes, bytes)}
      {:error, reason} -> {:ok, :failed, Map.put(state, :error, {:read_output!, reason}}
    end
  end

  # Other transitions handling is very same
end

That way you ① do have a real FSM, that is in the determined state, ② fantastic testing framework with Finitomata.ExUnit, ③ transparent telemetry attach and whatnot, ④ OTP distribution for free.

What part of the above does not cover your needs?

If you insist on having a single state :ready which is wrong IMHO, you might rewrite the FSM declaration to have a single state and on_transition/4 having more case clauses.

3

Hey, thanks for checking out dsm.

I acknowledge that Finitomata is a neat library, especially the telemetria integration. However, the reason for writing this library was to avoid the large number of case statements (about 15) I had to write. Using Finitomata defeats this purpose. I wanted a more declarative / schema driven alternative.

“I couldn’t define an event at compile time”

I wanted the validation logic to be a function that acts on the input and validates it as opposed to defining a struct that could be pattern matched against as part of the validation logic. It is resemblant in using Finitomata as well, the validation logic is part of the case statement case read_file_to_bytes(file) and the return value determines the next transition. This is exactly what I wanted, but on a larger scale.

Consider the scenario where the input event has to be passed through multiple functions before concluding whether it was a success or failure. The case statements turn into a with statement. My use case required using matching functions on the return values within the with statements, my code turned into something like this

with output1 <- pass_1(input),
        true <- is_output_correct?(output_1),
        output_2 <- pass_2(output_1),
        true <- is_output_correct?(output_2) do
 ....
else 
 {:incorrect_output_1, error} -> # do something
 {:incorrect_output_2, error} -> # do something
end

I was able to refactor the code but no matter how many times I polished it I ended up with a large with, else block. This is exactly what I wanted to avoid since I had many passes :slightly_smiling_face:.

If you insist on having a single state :ready which is wrong IMHO, you might rewrite the FSM declaration to have a single state and on_transition/4 having more case clauses.

This represents the dual purpose of this library, to be able to write transformations and pipelines in a declarative manner, while writing less case clauses.

4

Well, you want to have a single state managing multiple actual states. I rarely recommend reference material, but Introduction to the Theory of Computation by Michael Sipser is well worth a look to better understand what FSM is and what it’s not (including but not limited to what guarantees a correct FSM provides beyond “state management.”)

Well, you’ve said you went through FSM libs and I was under impression you’ve read the very top page of finitomata’s docs, which lists callbacks and all you basically need is to have a single with/1 statement without else and several on_failure/3 handlers (on_transition/4 natively supports returning {:error, _} tuples.)

Please don’t get me wrong, I am not advocating for finitomata, I am willingly try to understand what is lacks, and still there is no clean understanding (it might be just me.)

After a while you’ll want to test your pipelines in with/1 and I can barely see how it would be ever possible without making pass_N/1 functions public and demand them to be implementing some behaviour. I’ve been there and I was forced to develop a framework with test generators to not only simplify testing for the end user, but simply make it possible.

The last, but not the least, would be running in a distributed cluster with dozens of thousands FSMs. From what I was able to grasp from the sources, that’s not something you are prepared to.

I mean, the library is definitely worth it if it suits your needs, I was never to argue if there was no “finite state machine” bait in the title. What you are building is not an FSM, AFACT.

5

I appreciate the honesty. As it appears, I do seem to lack a concrete understanding of what FSMs are not. I will read through the material. I’m not a CS student, I’m trying to figure things out on my own, but it’s not an excuse. Based on our discussion, I feel “declarative pipelines” would be a better fit. About testing, I am yet to figure it out. Let me see what I can do to make it better.

1 Like