Recommendation for configuring long term tasks with many interations

Questions / Help
,
1

Hi, I am struggling with a problem how to create configuration files for this kind of application:

LabWorker

  • libs for communicating with other hardware over the multiple protocols
  • tasks for running defined steps manually (debug, fast obtained result)
  • main worker takes the keyword list from configuration file, process one by one item and shows results
  • in the future web interface for steps configuration and observing results
  • configuration temporarily hard coded in config/#{env}.exs

Configuration looks like that:

config :tasklist,
steps: [
{:dev1, getData: %{:channel => 11, :name => “x”}},
{:dev2 :getData},
{:dev3, setParam: %{:steps => 10, :min => 0, :max => 10}}, // here is a problem
{:dev1, :getPower},
{:dev5, :getData}
]

config :dev1,
conn: :rest,
endpoint: …

config :dev2,
conn: :scpi,
ip: …

This config style starting to be very messy while I need some complex task list. Examples:

Steps:
dev1, get power for channel 1
dev2, get data
dev3, set param power to x
dev1, get power for channel 1
dev2, get data

– so far so good… I need ordered keywords list to preserve steps order

but what I wanted this 100x bigger and only one device has variable input, something like this:

while power < 10 do
dev1, get power for channel 1
dev2, get data
dev3, set param #{power} to x
power += 0.1
end

I need to configure min and max value together with step size at least and this is just in case that I have one configurable device…

It seems to me that would be faster to write some config file generator for that instead of trying to parse too complex config file, but I am afraid of final size of such keyword list…

2

Why are you concern about keyword list size ?

You are also heading into the realms of a more complicated DSL. The quick DSL is to use XML or JSON to describe what you are trying to do.

3

Aren’t keyword list slower when they are big ? but its true that I don’t have an idea what the big means…

yes, its starting to be more complicated… But is might not be, maybe it is just so complicated in my head…

When I look at that from web front end point of view, fronted will be creating this configurations somewhere in memory. It may drop off configuration files at all which is good but the rest of my question is still here, how to configure something of this complexity the easiest way ?

It’s horrible example but anyway, I remember vagrantfiles for configuring Vagrant boxes, with this code style:

Vagrant.configure(“2”) do |config|
config.vm.box = “dummy”
config.ssh.username = “ubuntu”
end

but here you can do something like that:

def machine name, config
config,vm.define “#{name}”
other options…
end

machine “comp1”, config do |comp1|
option1
option2
end

the config has some limited ruby programming functionality

Maybe the web interface should create pieces of code that could be run directly for test to be performed? What is the best practice in such cases?

4

Sketch out 3+ different example configurations and that should help you determine where the common patterns are located. This will aid in deciding if you should use keywords lists, or pseudo code like config, or something else.

1 Like
5

No. In fact they can even be faster than maps as long as they are accessed iteratively. :slight_smile:

Now looking up a value in a kwlist like opt[:something] is indeed slower than maps after about 32 or so entries, give-or-take (because maps are also just keyword-style lists internally up to that size, though implemented in C so it is probably slightly faster for any given size). But iterative access on lists is likely faster regardless (need to bench to verify…).

1 Like