Anyone care about Temporal?

Chat / Discussions
1

Alright,

So I’m not anything to do with Temporal, except a customer.. I am also not competent enough to build Elixir SDK Support In The Roadmap? - #17 by Rudi - Community Support - Temporal Community Forum

But I am someone who loves Elixir and would like this to not be released under a commercial license.

I also thing that this person should be rewarded for their efforts, it ain’t easy to build this.. so I’m trying to tip the hat to Temporal to incentives this person to make this SDK more open source friendly.

I can guess that many don’t know what temporal is, or why they should care about it for Elixir.. BUT….

On the surface it’s not an easy sell as this language gives us all the foundations for distributed computing. All I can say is that it gets harder the more you go beyond the basics and Temporal solves a lot of the “beyond the basics”, without feeling any harder than the basics.

I’m not a shill, but I am a fan of Temporal so I’m happy to answer any questions.

My goal is completely transparent…I would love for both this guy to be paid and for this SDK to be opened up more.

Chris

1 Like
2

I heard of it many times over my career but never found it doing anything that Oban on multiple nodes can’t do as well – and we’re not even talking Oban Pro, just the free Oban.

I have rarely worked for a long time on projects that require such a scale – but I remain skeptical of the paid offering.

Admittedly though, I never sat to compare it with Oban, feature by feature, side by side. Have you? What does Temporal offer that’s making it indispensable?

3

Hey @dimitarvp I’ve been in non elixir world for some time so I’m probably way off.

Last time I looked at Oban, it was the best of the many worker queue implementation in elixir.

To create durable workflows required building your own external state. Im very, very very happy to be wrong. I know Oban Pro had some workflow equivalent but it seemed more expensive and clunky that temporal at the time.

Does it have the equivalent of signals and queries and support mutil day workflow?

Honestly my main thing is that I don’t get to work with Elixir on a daily basis, and anything that can add Elixir as a possibility is good for me.

Can you say specifically what Oban did I might be able to do a half ass job of explaining the differences :slight_smile:

4

I don’t think Oban is more expensive. Oban pro doesn’t charge on the amount of jobs you execute, Temporal does.

You don’t own the infrastructure where Temporal execute, that’s why included in their pricing is storage.

Oban you pay to have access to the code, it gonna run on your own infrastructure, and it can run as many jobs as you want, the price won’t change. I’ve worked in places where what we run daily on oban would consume all the jobs available on temporal basic plan(the only one that is “cheaper” than oban)

So, to reproduce exactly what temporal does you would need more than just Oban or Oban Pro, but there is a big trade-off being done there. Temporal abstraction “encapsulate” and save state of the world to execute, but if that state becomes stale it can byte you, Oban Workflows from Pro is not as powerfull as Temporal but with a good experience on how to properly build the job to give you the guarantees that you need to a paritcular use case you wont need something as powerfull as Temporal.

Temporal is very appealing to runtimes that doesn’t have the safety, resiliency and isolation that the beam does. temporal deal with more then that, but the biggest appeal is exactly what we already get with using the beam itself, the extra stuff is only additional and can be engineered depending on your real needs.

3 Likes
5

I’m in SE Asia, so it’s late here and I’m tired. I’ll respond tomorrow…

Tonight I’ll just say that I never intended an Oban vs Temporal debate. I actually have more questions than answers about that…

That said, from my little knowledge, I already know that there’s are more than a few “apples to oranges” comparison here, which I’ll respond to tomorrow.

Either way my point is that the standard “BEAM can do everything” conversation doesn’t help Eixir get more adoption… and I want more adoption!.

IMO this is a good product and we should support a BEAM based SDKs for it.

Like I said I’ll give a good shot at responding tomorrow, but not for comparison against Oban, just to get more traction for something I think it’s worth highlighting.

6

Nobody is saying that. But as @cevado pointed out, Temporal’s pricing can be viewed as predatory – I sure don’t want to get billed on tiers of amount of work processed.

That I think we all agree with.

This is a dev forum. But maybe “getting traction” will find sympathetic ears. I don’t speak for the whole community, obviously.

To me Temporal is a relic from other times. The world has changed. That’s my surface-level evaluation of it.

1 Like
7

Damn, man! I was on the verge of sleep and saw you typing so thought I’d wait for the response.. and man was it worth it. I can’t resist but to respond :slight_smile: that post was too hilarious!

Relic!!! Hahaha, let me sleep man you’re cracking me up too much!

You know how old the BEAM is? Even Elixir is older I think.

As for preditory, what? Their Cloud offering is cheap, literally everything in our org costs more than this.

Tbh we were self hosting the open source version for ages but it’s cheaper and easier for us to use the SaaS than maintain ourself.

8

I am simply being honest about what I would buy and would not buy. As a dev I don’t want tiered pricing. To me that’s predatory. Feel free to disagree but you’ll not change my mind. :person_shrugging:

1 Like
9

I will not try to change your mind.

I’d love to use the single priced, unlimited scale infra provider your using because it sounds awesome. Where do I sign up?

10

My question: so how do you use Temporal today, roll your own Elixir wrapper or use another fully supported language? If it is the second case, do you think there is any practical limitation of writing a port server in eg Python to interface with Temporal, other than being inelegant?

11

The point is not “beam can do everything”, is that the appealing points of temporal, are not so appealing to someone already using beam.

I agree, I’m not against having the sdk available. I was just trying to explain why I think temporal is not so appealing to people using elixir.

1 Like
12

Hey @derek-zhou

Today I would use the SDK I was promoting in the OP i.e. GitHub - andrzej-mag/temporal_sdk: Temporal Erlang and Elixir SDK · GitHub

Previously Ive used the typescript, python and java SDKs for this. When I was working elixir I had many disjointed Oban jobs which I converted to Temporal workflows and this was really nice but meant any Elixir functionality I needed to expose a phoinex api endpoint or similar.

Today in Elixir I would want to use this SDK, which looks pretty great to me.

13

Hey @cevado, I was actually using elixir and Oban specifically when I started using Temporal.

I looked at Oban Pro again, it’s nice but I still see it serving a different process.

My understanding is (correct if wrong)

  • Oban - solid worker queues
  • Oban Pro - orchestration platform
  • Temporal - Durable execution and orchestration platform

The durable part has great benefits if needed but also costs if throughput is a key driver. It’s not better, just different.

At my time I needed long running workflows that could span days, had manual steps in the middle and needed to survive failures and restarts for all that time.

Temporal essentially a event system under the hood and all worker scheduling and state events are is persisted at every step. So if the whole system goes down it can just replay all events and catch-up. It also supports signals which are external events that can be sent to the workflow so the support for external manual steps like approval were trivial to add.

Generally I’ve no bad word to say against Oban, and it’s only got better since I looked at it. But I still thing it serves very different purposes and while you can build some of the things to Oban to make it somewhat act like Temporal it’s not as trivial as it first seems when you get into the weeds.

For example as noted, keeping all this state and replaying several days later is problematic if you need to upgrade these workflows. Temporal has support for workflow versioning that would definitely not be trivial to replicate.

On costs, when I said we are comparing “apples to oranges” - the costs from Temporal are only their cloud offering. To run on your own infra (like you would with Oban Pro) it’s free and fully open source.

What you may optionally want to pay for cloud is their infra and managed service. Don’t need to I use it if your projects don’t have much demand and are happy managing your own infra.

For me though the main driver was that Temporal was simple, just add some annotations to functions and register them and you get full durable and reliable execution, took all the hard thought out of distributed orchestration.

My point here was merely that I think Temporal is a good product and fits well in the Elixir ecosystem and worth a look.

If you don’t need the guarantees Temporal offers for your use case that’s cool, but I know many projects that do and I would rather use tools that are well tested on this than build my own attempt on top of technologies with different goals. Even on “relic” software :joy:

14

With the recorded option, and particularly cascading workflows, Pro is also durable. That’s not limited to Workflows either, any job’s output can be recorded and referenced later.

That’s precisely how Workflows operate. Jobs are transparently distributed across your running nodes, the return value is recorded for retrieval later, and downstream deps can read the preserved values. It’s even easier with cascades because the accumulated context is passed to each function in the workflow:

  def insert(source, date) do
    Workflow.new()
    |> Workflow.put_context(%{source: source, date: date})
    |> Workflow.add_cascade(:extract, &extract/1)
    |> Workflow.add_cascade(:transform, &transform/1, deps: :extract)
    |> Workflow.add_cascade(:load, &load/1, deps: :transform)
    |> Workflow.add_cascade(:notify, &notify/1, deps: [:transform, :load])
    |> Oban.insert_all()
  end

  def extract(%{source: source, date: date}) do
    if source =~ ~r/[a-z0-9]+/ do
      %{records: [1, 2, 3], extracted_at: DateTime.utc_now()}
    else
      {:cancel, "unprocessable source"}
    end
  end

  ...

That’s also possible for Oban Workflows, but functionality to make it simpler and more powerful is landing in Pro v1.7.

1 Like
15

Temporal Erlang and Elixir SDK author here.
@chrisjowen, thank you for your kind words and for explaining Temporal concepts to this Community.

For the sake of completeness, here are the Temporal Erlang and Elixir SDK links:

SDK is under active development. Building a native Temporal SDK is a substantial effort due to Temporal’s extensive feature set.

For those interested in learning more about the Temporal, here are some helpful resources:

2 Likes
16

(OK, so I really should learn to just never comment at the end of a long day. Half-doing comments and being not well-informed should just be fully off-limits. Self-remark. I do it less and less with time but zero is the ideal number here.)

First, @chrisjowen sorry, I was hasty. I mistook Temporal for something else completely. Temporal does indeed support self-hosting and the pricing is based solely on the hosted offering. While I still don’t agree with tiered pricing and would not choose it for my projects, I get that people need to make money and to not be on the negative by taking customers with tens of millions of jobs for $5 a month. Obviously.

Secondly, RE: the “relic” thing, that was mostly me commenting on the annotations. But I have to be super transparent here: I started changing my mind on this with time. We all got stuff to do and I no longer over-focus on technical details that bothered me before. Nowadays I am like: “Is the code readable and does it get the job done?”. While having annotations is teetering on the verge of “magic” – something many of us here in the Elixir ecosystem do not like – it’s also a real genuine convenience.

Thirdly, again the “relic” thing – to me that also relates to the choice of programming language. Just by choosing Elixir (or Erlang) you get much more guarantees that you must emulate in almost any other PL. To me that’s hugely valuable and must not be glossed over. That being said, everyone is free to use whatever PL they wish, for their own reasons. Though, integrating through Ports or other similar means kind of goes into implementation churn – that’s why I’d still choose Oban + Pro even if it has less features than Temporal in the strict sense.

So, fair is fair. I was wrong on the features / hosting model – admitting fault. Being able to self-host should generally nullify most criticism towards a piece of software’s pricing models.

4 Likes
17

Thanks for posting the links. How is the integration with Erlang done exactly? Is it with a Port, or something else? What is the integration glue?

18

Temporal Server uses gRPC for communication with SDKs, see Temporal gRPC API.
Temporal Erlang and Elixir SDK is a native implementation and therefore uses gRPC to communicate directly with the Temporal Server.
gpb package handles protobuf definitions.
gun is used as a default HTTP/2 client, see temporal_sdk_grpc_adapter_gun. Other HTTP/2 client packages can be used by implementing temporal_sdk_grpc_adapter behavior.

1 Like
19

Having used temporal the last few months (free only), and having used oban pro at past positions for decently mid scale amounts of jobs…

Temporal blows it out of the water. It isn’t even close. Not only does it work well, support tons of features, is really stable, at its bottom end it is just postgres and a few pods in my k8s cluster. It has a good CLI and great logs and a great dashboard. You can version workflow activities so that running workflows continue through deploys, you can rewind problem executions to an earlier point in time, getting updates on progress through to the ui, the list just goes on and on.

In my opinion oban never felt very elixir like to me. I found developers dipping into oban’s database to give basic functionality like polling for workflow progress and so forth. So if elixir had good bindings in elixir I’d use that every time.

Just my opinion.

1 Like
20

PgFlow is an option if you are using Postgres:

comparison here: docs/COMPARISON.md

1 Like