makeitrein
Hey all, just started picking up Elixir last week and am writing a scraper as a learning project.
Baby step #1 is extracting the number from a URL on the target web page… here’s what I’ve written:
# url is in "https://xxyyzz.com/xxyyzz.383254/" format... goal is to extract 383254
def get_id_from_url(url), do: Regex.run(~r"\d+\/", url) |> Enum.at(0) |> Integer.parse |> elem(0)
This seems a bit clunky of a function to me for a simple integer extraction… is there a better way of going about this?
Trending in Questions
Hey guys,
I’ve got a huge CSV ( around 10 GB ) that needs to be processed hourly
Do you guys have any suggestions what is the best prac...
New
Hello!
Could someone please give me a help/sample code, how to delete a file from s3 using waffle/waffle_ecto from Phoenix app.
I creat...
New
I have what I’ve heard referred to as a “lookup table” in my database. This is a way of assigning codes to common values. One common lo...
New
Hello,
I’m developing a online persistent chat system (what’s app) like using elixir/dynamodb/aws for a mobile app(flutter).
The diffic...
New
What approach to take when sending live updates to “random” users Hi! I have a question, I have a little chat app, and when I create a DM...
New
Anyone here using Honeybadger?
My Honeybadger account is being overwhelmed with noise from some bots. Seeing a lot of
Bandit.HTTPError...
New
I’m seeing that a list inside a Kino.DataTable will be interpreted as a charlist, even if the Kino.configure() is set to charlists: :as_l...
New
Other Trending Topics
Hobbes is a low-level distributed database for the Elixir programming language.
Hobbes provides a simple, safe, and scalable storage lay...
New
ExRatatui lets you cook up rich terminal UIs in Elixir, powered by Rust’s ratatui via Rustler NIFs. Build interactive terminal applicatio...
New
Hello everyone. After busy few months I am happy to announce v0.1.0 of Emerge & Solve.
They are GUI (Emerge) and State management (S...
New
Corex is an accessible, unstyled UI component library for Phoenix that integrates Zag.js state machines using Vanilla JavaScript and Live...
New
There are three potential reasons for members of this forum to have a look at https://vutuv.de
You are tired or annoyed of LinkedIn.
Yo...
New
Aludel - LLM Evaluation Workbench
Aludel is an embeddable Phoenix LiveView dashboard for evaluating and comparing LLM prompts across mult...
New
Categories:
Sub Categories:
Forums
Popular Tags
- #ecto
- #liveview
- #troubleshooting
- #learning-elixir
- #deployment
- #library
- #erlang
- #testing
- #genserver
- #mix
- #absinthe
- #remote-other
- #otp
- #plug
- #how-to-question
- #macros
- #postgres
- #elixirconf
- #channels
- #exunit
- #discussion
- #code-sync
- #javascript
- #podcasts
- #onsite
- #dialyzer
- #docker
- #authentication
- #umbrella
- #full-time-contract
- #podcasts-by-brainlid
- #ecto-query
- #blog-post
- #elixir-ls
- #ai
- #elixirconf-us
- #phoenix_html
- #iex
- #graphql
- #genstage
- #websockets
- #supervisor
- #advent-of-code
- #distillery
- #processes
- #api
- #forms
- #hex
- #security
- #metaprogramming










Showing Posts 1 to 10- Show Best Posts
- Show All (oldest first)
- Show All (newest first)
hassan
How about
dimitarvp
Test it in
iex:You can abuse Elixir’s allowed syntax of pattern matching on a string suffix (you cannot pattern-match strings in the middle of the bigger string though, have that in mind). Not sure if I am not taking your example too literally but if I understood you correctly, that’s how I would approach the problem.
NobbZ
You can use
hdinstead ofEnum.at. that saves you some characters.Also you can use
String.to_integerinstead of piping throughInteger.parseandelem.But that’s the way to go.
makeitrein
Many thx for suggestions! Realized my URL example is a bit lacking… the xxyyzz is representative of any numbers and letters of unknown length, so just stripping non-digits won’t work, nor will the pattern matching (I think, awol from computer).
I guess the challenge is to find a series of numbers at the end of a string that starts with a period and ends with a slash… String.to_integer looks like it might be what I need, good enough for government work.
dimitarvp
Nope! We have enough borked gov’t systems. Let’s do better.
If you give us a few examples and/or explain the whole URL schemata then I can help you better.
makeitrein
Hah, fair game ^^
https://foster.com/death-pancake.1468/ === 1468
https://hkd33.net/mr-rogers101.690153/ === 690153
https://space-force911.gov/sauce-master.13257777/=== 13257777
Here’s a selection of random URLs… all begin with https, have a base domain, followed by the username of the person who submitted the domain followed by a period followed by the id of the post followed by a trailing slash…
Only the id of the post is relevant so we can ignore the base domain, username, period, and ending slash…
dimitarvp
So all URLs always end in a number plus a forward slash? No exceptions?
makeitrein
Correctamundo
dimitarvp
(EDIT 1: Account for invalid values.)
(EDIT 2: Trim empty strings when splitting.)
(EDIT 3: Included explanations.)
Test it:
Breaking it down:
~w(. /)equals[".", "/"](soString.splitis called with multiple separators).parts: 1000is used to prevent denial-of-service attacks, in case somebody manages to smuggle huge strings to your code.trim: trueremoves empty strings from the result. CheckString.splitdocs."https://foster.com/death-pancake.1468/" |> String.split(~w(. /), parts: 1000, trim: true)yields this:…so we are calling
List.laston it to give us the desirable piece of data.parse_idhas to also handle invalid data:String.splitreturns[],List.lastwould returnnil.String.splitreturns["single_invalid_url"],List.lastwould return"single_invalid_url".Both cases would make our internal function
parse_idto return:error. (Integer.parsewill return:errorif you supply it a string that does NOT start with an integer.)The
fetch_idinternal function uses function heads instead ofiforcaseto extract successful integer parsing and return it, or react to an:errorreturn value and just pass it down the line to your consumer code.One caveat: notice that
fetch_idmatches on{number, ""} when is_integer(number)which means the function will be called only if a full integer string is passed, namely “123” or “456” will succeed but “123xyz” will not. If you expect URLs like “https://whatever.man/1234abcd”, this code won’t work.makeitrein
Dang, that’s some good looking code! Copy and pasting in 3, 2, 1…