ityonemo
Every year I help my friend with his taxes, he has an itinerary of places that he’s been to for work and I scrape the google maps API to get driving distances. This year I did it in elixir. The major challenge is figuring out how to load external libraries (CSV, JSON) into an .exs script (no framework). Mix isn’t like python-pip (and that’s a really good thing TM) but every once in a while you just want it to do what you want it to. I can’t say that what I did is necessary “best practice”, but after a lot of hemming and hawwing, I figured out a reasonable solution. I’m leaving it here to hopefully help/inspire other people that need to do something quick 'n dirty.:
# fail early if no filename is provided.
[filename] = System.argv()
File.exists?(filename) || raise "bad filename."
file_root = Path.basename(filename, ".csv")
#
# before running, install jason and csv.
#
# mix archive.install hex jason 1.1.2
# mix archive.install hex csv 2.3.1
# mix archive.install hex parallel_stream 1.0.6
#
# load up all of the libraries.
~w(jason 1.1.2 csv 2.3.1 parallel_stream 1.0.6)
|> Enum.chunk_every(2)
|> Enum.map(fn [lib, ver] -> "~/.mix/archives/#{lib}-#{ver}/#{lib}-#{ver}/ebin" end)
|> Enum.map(&Path.expand/1)
|> Enum.map(&Code.prepend_path/1)
# make sure the libraries are there.
Code.ensure_compiled(Jason)
Code.ensure_compiled(CSV)
# marshall the address into a dict.
addresses = "addresses.csv"
|> File.stream!
|> CSV.decode!
|> Enum.map(&List.to_tuple/1)
|> Enum.into(%{})
# read the paths.
driving_lines = filename
|> File.stream!
|> CSV.decode!
# check that they are all kosher
driving_lines
|> Stream.with_index
|> Enum.each(fn
{_, 0} -> :ok
{[_, start_name, dest_name | _], line} ->
:erlang.is_map_key(start_name, addresses) || raise "bad line #{line + 1}, #{start_name}"
:erlang.is_map_key(dest_name, addresses) || raise "bad line #{line + 1}, #{dest_name}"
_ -> :ok
end)
# this time I will remember to deactivate this API key
api_key = "XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX"
# generate the curl query Mostly just using google's API.
curl_query = fn start_addr, dest_addr ->
start_http = Regex.replace(~r/\s/, start_addr, "+")
dest_http = Regex.replace(~r/\s/, dest_addr, "+")
"https://maps.googleapis.com/maps/api/distancematrix/json?units=imperial&origins=#{start_http}&destinations=#{dest_http}&key=#{api_key}"
end
{rows, _} = driving_lines
|> Stream.reject(fn [_, a, b | _] -> a == "" || b == "" end) # get rid of empty lines.
|> Stream.with_index
|> Enum.map_reduce(%{}, fn
# ignore the first line, which is just for fun titles.
{line, 0}, acc -> {line, acc}
# if our start and destination are cached, then use that instead.
{[r, start_name, dest_name | _], _}, acc when
:erlang.is_map_key({start_name, dest_name}, acc) ->
start_addr = addresses[start_name]
dest_addr = addresses[dest_name]
{[r, start_addr, dest_addr, acc[{start_name, dest_name}]], acc}
{[r, start_name, dest_name | _], _}, acc ->
# dereference the stard and destination addresses from our dictionary.
start_addr = addresses[start_name]
dest_addr = addresses[dest_name]
# run the curl command
{res, _} = System.cmd("curl", ["--stderr", "/dev/null", curl_query.(start_addr, dest_addr)])
# destructure google's insane JSON here.
v = case Jason.decode!(res) do
%{"rows" => [%{"elements" => [%{"distance" => %{"text" => distance}}]}]} ->
# of course the string has "mi" attached to it. Strip it out.
{v, _} = Float.parse(distance); v
_ -> "?"
end
# put the line back together, this time with full addresses and distances for the IRS.
# cache the result into our accumulator map.
{[r, start_addr, dest_addr, v], Map.put(acc, {start_name, dest_name}, v)}
# lines which don't conform to our dogma get released
{line, _}, acc -> {line, acc}
end)
# encode back into CSV.
rows
|> CSV.encode
|> Stream.into(File.stream!(file_root <> "-finished.csv"))
|> Stream.run
Trending in Discussions
As the title says, please share what you’ve been up to with Elixir. Whether that’s been learning it, looking into it, making stuff with i...
New
I want to open this thread for you all to discuss and help those who really like Ash but are still hesitant to use it in a real project. ...
New
I am happy to introduce the very α version of the new programming language compiled to BEAM.
Welcome Cure.
It has literally three kille...
New
I’m posting this in response to Jose’s recent tweet (Cr. link) :
People are sleeping on Elixir for a coding harness:
Hot-code swappi...
New
Hello,
I wrote Stop My Hand, a Scattergories-like web application using Phoenix/LiveView as my learning project for Elixir (after readin...
New
It would be helpful to have a list of companies worldwide that hire engineers without prior experience in Elixir. Often, it can be quite ...
New
Anyone running long-lived stateful processes on BEAM? We’re building an AI agent runtime and would love to compare notes.
We’re a small ...
New
Other Trending Topics
Hobbes is a low-level distributed database for the Elixir programming language.
Hobbes provides a simple, safe, and scalable storage lay...
New
Hi there! We created Gust: A task orchestrator inspired by Airflow.
For those who have never heard about Aiflow, it’s a Python-based wor...
New
Beam Bots (or just BB for short) is a framework for building fault-tolerant robotics applications in Elixir using familiar OTP patterns. ...
New
Xamal is a deployment tool for Elixir apps that deploys native releases to bare metal servers over SSH. It’s a port of GitHub - basecamp/...
New
Hello everyone. After busy few months I am happy to announce v0.1.0 of Emerge & Solve.
They are GUI (Emerge) and State management (S...
New
There are three potential reasons for members of this forum to have a look at https://vutuv.de
You are tired or annoyed of LinkedIn.
Yo...
New
Categories:
Sub Categories:
Forums
Popular Tags
- #ecto
- #liveview
- #troubleshooting
- #learning-elixir
- #library
- #deployment
- #erlang
- #testing
- #genserver
- #mix
- #absinthe
- #remote-other
- #otp
- #plug
- #how-to-question
- #macros
- #postgres
- #elixirconf
- #channels
- #exunit
- #discussion
- #code-sync
- #podcasts
- #javascript
- #onsite
- #dialyzer
- #docker
- #authentication
- #umbrella
- #full-time-contract
- #podcasts-by-brainlid
- #ecto-query
- #elixirconf-us
- #blog-post
- #ai
- #elixir-ls
- #phoenix_html
- #iex
- #graphql
- #genstage
- #websockets
- #supervisor
- #advent-of-code
- #distillery
- #processes
- #api
- #forms
- #hex
- #security
- #metaprogramming










Showing Posts 1 to 6- Show Best Posts
- Show All (oldest first)
- Show All (newest first)
LostKobrakai
You can put .exs files in a mix project and run them using
mix run script.exs. That’ll start the script with all dependencies of the mix project present.ityonemo
who’s got time to make a mix project =D
dimitarvp
Is this a joke?
mix new your_projectis all it takes.AndyL
Also you can put a shebang in the file,
chmod a+rx <yourfile>and just run it standalone like a bash script (without the dependencies).NobbZ
Please do not do that! That can cause a lot of nasty to debug errors when you happen to develop an application that has direct or transient dependencies to one of this packages!
The proper way is to create a mix project, specify dependencies and compile it into an escript!
I prefer this way:
ityonemo
Yeah escript is a better option.