Is there here document and here string support in Elixir?

Questions / Help
1

I have tried to write mix task or even generate escript binary file, but it looks like that none of those ways supports some useful shell input methods. I tried to inspect arguments simply with IO.inspect(args)

here-document (<< operator)

A here document is a special-purpose code block. It uses a form of I/O redirection to feed a command list to an interactive program or a command, such as ftp, cat, or the ex text editor.

Source: https://www.tldp.org/LDP/abs/html/here-docs.html

What I have tried:

# Escript example:
$ ./example --test <<EOL
line 1
line 2
line 3
EOL
["--test"]

# Mix task example:
mix example.demo --test <<EOL
line 1
line 2
line 3
EOL
["--test"]

here-string (<<< operator)

A here string can be considered as a stripped-down form of a here document .
It consists of nothing more than COMMAND <<< $WORD ,
where $WORD is expanded and fed to the stdin of COMMAND .

Source: Here Strings

What I have tried:

# Escript example:
./example --test <<< 5*4
["--test"]

# Mix task example:
mix example.demo --test <<< 5*4
["--test"]

Maybe I need to compile Erlang with some extra flags or maybe it’s just not supported at all?

2

Heredocs/-strings are written to stdin of your application, but from the output you show, it seems as if you only inspect parsed options.

foo <<< bar is just syntactic sugar for echo bar | foo, while << is (roughly) equivalent to echo "l1\nl2\nl3\n" | foo.

2 Likes
3

To new project I have only added:

a) escript option in project function:

defmodule Example.MixProject do

  # …

  def project do
    [
      # …
      escript: [main_module: Example],
      # …
    ]
  end

  # …
end

b) Replaced lib/example.ex to shortest form:

defmodule Example, do: def main(args), do: IO.inspect(args)

c) and finally added mix task with also minimal code:

defmodule Mix.Tasks.Example.Demo do
  use Mix.Task

  def run(args), do: IO.inspect(args)
end

Not sure what exactly you mean by parsed options. As you can see I’m not using OptionParser - I’m just trying to debug everything I got in args.

4

Okay, you are not using OptionParser but that doesn’t matter, you are inspecting the wrong thing as I said already.

You need to read stdin.

5

Here-docs are handled by your shell. As NobbZ explained, the content of a here-doc is passed on the standard input stream to the program. Try this example:

$ elixir -e 'IO.puts IO.read(:stdio, :all)' <<EOL
> hey
> there
> EOL
hey
there
1 Like
6

Ah, now I can see … However there is one problem with this solution.

./example --test "single line"
./example --test << EOF
multi
line
EOF

Of course we can check it like this:

last_arg = List.last(args)
all_args = if option_type(last_arg) == :string do
  args ++ [IO.read(:stdio, :all)]
else
  args
end

defp option_type("--test"), do: :string
defp option_type(_option), do: nil

but what to do if user by mistake will not pass << operator?

$ elixir -e 'IO.puts IO.read(:stdio, :all)'
abc
def
EOF
EOL

In my opinion expected behavior (for app - not for IO.read/2 :exclamation:) here would be raise describing problem, but I’m not sure how I could make a check for using << operator … I do not even see an option for setting timeout in documentation (at least not for IO.read/2 which means it will collect input until user would terminate app, right?

Note: Of course I could use \n, but it’s extra requirement for end-user - not really problem for me.

7

No program will ever see << except for the shell, which then does some magic to turn it into the started programs stdin.

This is by design.

If --test option expects further data from stdin document as such in your programs manual, if it does not, then do not try to do stdin magic.

4 Likes
8

My code snippet was meant to demonstrate how a here-document’s content is passed on the standard input to the program. I wasn’t proposing a solution because I don’t know the requirements of what you’re trying to achieve.

If you want your program to take user input which may span more than one line, you may want to read the input one line at a time and stop upon encountering an empty line. Alternatively, using IO.read(:stdio, :all) is also fine, in my opinion. It reads everything up to the end of input which can be signaled in several different ways.

When a program is invoked in a shell, users should know that they can signal the end of input by pressing Ctrl-D on an empty line. This is just a way to let the shell know there won’t be any more input from the user, so that it can close the input pipe connected to the running program at which point the running program will encounter the EOF/end-of-input if it’s reading from stdin.

The same outcome can be achieved by using shell pipes:

$ echo "This is my input." | elixir -e 'IO.puts IO.read(:stdio, :all)'
This is my input.

or by using shell’s stream redirection to read from a file:

$ echo "This is my input." >foo

$ elixir -e 'IO.puts IO.read(:stdio, :all)' <foo
This is my input.

If you’re still looking for help, then I’d like to ask you to rephrase your question because the way it’s currently formulated doesn’t explain what the actual problem you’re trying to solve is.

1 Like
9

I was just looking if there is any way to pass multi line argument value instead of using \n or \ character at end of each line which are not handy and requires to pass/edit data manually. I have found such here-* syntactic sugar in BASH

./example --test << EOF
multi line content goes here …
EOF

Notice that here we have only prefix --test << EOF and suffix EOF for such value i.e. value is not changed and it could be just copied. Single line have only prefix (--test), but it’s much easier that other solutions. Simple example of use case here is multi line JSON parsing, so end-user could use such option by simply copying JSON data - without any need to replace new line with \n or adding \ at end of each data line (just imagine huge prettified JSON).

10

Have you tried this:

$ ./example "multi
line"
2 Likes
11

I wonder how I could forget about so simple way. :smiley:
It’s always darkest before dawn. :077:

12

Ahh, really sorry - just reminded something. With "…" notation you can’t copy and paste strings with not escaped " character. I was interested in << EOL … EOL syntactic sugar, because you can set prefix and suffix on yourself.

13

Then use single quotes.

The user has to choose the appropriate quotes anyway. If they don’t use bash but eg fish the shell might behave different from your documentation/examples. Therefore it’s best to only document the meaning of the named/positional argument in an executable.

2 Likes
14

I see. So the question is not about Elixir at all, it’s about passing multiple lines as a single positional argument to a program when running it in a shell.

The common practice is to use quotes like NobbZ has suggested. You could use shell substitution but that would look rather weird:

$ elixir -e "IO.inspect System.argv" -- --test $(cat <<EOL
bar
baz)
EOL
)
["--test", "bar", "baz)"]

# Must use quotes around the shell substitution
$ elixir -e "IO.inspect System.argv" -- --test "$(cat <<EOL
"bar"
baz)
EOL
)"
["--test", "\"bar\"\nbaz)"]
2 Likes
15

It’s definitely best answer! With it everything could be passed and automatically escaped without any worry. End-user could decide when to stop adding text pretty easy and there is no need to change code.

I just wonder why it’s not possible to do it directly (not inside $() - command substitution) like cat could.

16

cat does read from stdin if no filename is provided as positional argument. It also reads from stdin if - is given as a filename.

2 Likes
17

Yeah, but it also does not freezes when there is nothing on stdin, right? I do not see any equivalent in Elixir documentation (maybe I have missed something). I though about doing it in with Task timeout, but then I was not sure what timeout should I use (assuming that you don’t know how much data is in stdin), so everything goes to point where I would like to check

last_arg = List.last(args)
all_args = case do
  option_type(last_arg) != :string -> args
  is_stdin_empty() -> raise "Missing #{last_arg} option value!"
  true -> args ++ [read_stdin()]
end

defp option_type("--test"), do: :string
defp option_type(_option), do: :value
18

It will wait for input until it reads EOF. Try cat as a single command without anything else. It will not exit, but wait for you to provide input on stdin.

1 Like
19

oops, my bad - cat here is like IO.read(:line) in loop

So there is no way to check if stdin is empty for any language, right?

20

You will never know if stdin was empty or not before you received an end of file, and after that you can check for emptyness by simply counting bytes. That works in any language.

2 Likes