dogweather

dogweather

I have a function repair_file_in_place(path) that makes HTTP connections to check links for 404 errors.

Here’s the Python code to run repair_file_in_place() concurrently on all the markdown files in a directory:

async def repair_files_in_directory(path: str):
    """
    Repair all markdown files recursively in a directory.
    """
    async with aiohttp.ClientSession() as session:
        async with asyncio.TaskGroup() as group:
            for root, _, files in os.walk(path):
                for file in files:
                    if file.endswith(".md"):
                        group.create_task(repair_file_in_place(os.path.join(root, file), session))
  • How would the code be structured in Elixir / OTP?
  • Is there some kind of semaphore-like ability to limit the concurrency? My use case is not blowing the ulimit on open files.

Showing Posts 1 to 6

zachallaun

zachallaun

Probably something like:

path
|> Path.join("**/*.md")
|> Path.wildcard()
|> Task.async_stream(&repair_file_in_place/1)
|> Stream.run()

Concurrency can be limited using options passed to async_stream. Docs: Task — Elixir v1.20.2

(Note: untested and possibly not optimal, but this is the first that came to mind!)

arcyfelix

arcyfelix

You are almost there, you forgot the if part:

paths
|> Enum.filter(&String.ends_with?(&1, ".md"))
...
zachallaun

zachallaun

This is handled by the wildcard, which expands the glob "your/path/**/*.md" into a list of all the .md files that live anywhere in your/path. :smiling_face:

dogweather

dogweather OP

It’s amazing that this actually works, and is so simple. Especially compared with the code I showed.

dimitarvp

dimitarvp

People dig Elixir for a reason, my dude. :smiley:

That kind of expressive and readable code hooked me to Elixir almost 8 years ago.

arcyfelix

arcyfelix

Yeah, my bad. Good job.

— All posts loaded —

Where Next? Top

Trending in Questions Top

RSP87
I’m working on a project that simulates the bumbl example in the programming phoenix book. It acts almost like an email client. We have a...
New
kpanic
Hi everyone, I am toying with the idea of building a “match maker” for giving personal help to people that wants to start coding. I sta...
New
nseaSeb
Hello, I know there is an approach for handling lists that allows for optimized traversal, but I can’t recall the specific method (somet...
New
brecabral
Documentation While reading the Scoped Routes section, I noticed that the documentation currently refers to a problem without explainin...
New
velrest
So my question is quite simple and i have found no conclusive answer on forum, google or AI. Should we use :erlang.float for Integer to ...
New
asweet-confluent
I recently noticed that Elixir’s Logger defaults its primary log level to :debug when no :logger, :level application configuration is pre...
New
ryanwinchester
apply_graft/2 doesn’t rewrite an add_many sub-workflow’s deps on an add step. Grafted jobs cancel with “upstream job was deleted” Version...
New

Other Trending Topics Top

JesseHerrick
Hey, I’m Jesse and I’m the main contributor behind Dexter, a full-featured, lightning-fast Elixir LSP optimized for large codebases. It s...
New
marciok
Hi there! We created Gust: A task orchestrator inspired by Airflow. For those who have never heard about Aiflow, it’s a Python-based wor...
New
mhanberg
Hi everyone! The first release candidate for the Expert language server project is now available! We’ve published a press release detai...
New
jimsynz
Beam Bots (or just BB for short) is a framework for building fault-tolerant robotics applications in Elixir using familiar OTP patterns. ...
New
Dmk
Xamal is a deployment tool for Elixir apps that deploys native releases to bare metal servers over SSH. It’s a port of GitHub - basecamp/...
New
webofbits
With AI doing more of the implementation work, I’ve been wondering how much coding I should deliberately keep doing myself. My main conc...
#ai
New

We're in Beta

About us Mission Statement

Options

Thread Display Mode




Thread Preview

Skip Thread Previews