adamwight
Seeking early adopters and reviewers of the mediawiki_client library (docs), which intends to wrap all of the major access methods for Wikipedia content.
The supported APIs are functionally complete and the library includes tests. My current goal is to get feedback on the exposed interfaces, and stabilize on an idiomatic Elixir style to begin the 1.x releases.
Examples
Watch all Wikimedia sites and print a link to the next 6 articles as they are created:
Wiki.EventStreams.start_link(streams: "page-create")
Wiki.EventStreams.stream()
|> Stream.take(6)
|> Enum.each(fn event -> IO.puts(event["meta"]["uri"]) end)
# https://www.wikidata.org/wiki/Q111963009
# https://en.wikipedia.org/wiki/Rhinella_xerophylla
# https://fr.wiktionary.org/wiki/stachst_aus
# https://commons.wikimedia.org/wiki/File:St_Swithin,_Quenington_-_Wall_monument_-_geograph.org.uk_-_3514076.jpg
# https://commons.wikimedia.org/wiki/File:Nebria_brevicollis_(Carabidae)_-_(imago),_Henshuisterveld,_the_Netherlands.jpg
# https://commons.wikimedia.org/wiki/Category:Protected_areas_of_Russia/5630230
Get current site statistics for German Wikipedia:
|> Wiki.Site.get!("dewiki")
|> Wiki.Action.new()
|> Wiki.Action.get!(
action: :query,
meta: :siteinfo,
siprop: :statistics
)
# %Wiki.Action.Session{
# ...
# result: %{
# "batchcomplete" => true,
# "query" => %{
# "statistics" => %{
# "activeusers" => 19687,
# "admins" => 188,
# "articles" => 2583285,
# "edits" => 211219883,
# "images" => 130199,
# "jobs" => 0,
# "pages" => 7163473,
# "queued-massmessages" => 0,
# "users" => 3715461
# }
# }
# },
# ...
# }
Fetch all information about Douglas Adams contained on Wikidata:
```elixir
Wiki.Site.get!("wikidatawiki")
|> Wiki.Action.new()
|> Wiki.Action.get!(
action: :wbgetentities,
ids: "Q42"
)
# result: %{
# "entities" => %{
# "Q42" => %{
# "aliases" => %{
# "uk" => [
# %{"language" => "uk", "value" => "Дуглас Ноел Адамс"},
# %{"language" => "uk", "value" => "Адамс Дуглас"}
# ],
# "pt" => [
# %{"language" => "pt", "value" => "Douglas Noël Adams"},
# %{"language" => "pt", "value" => "Douglas Noel Adams"}
# ],
Request ORES scoring for an edit to predict whether it was spam or not:
Wiki.Ores.new("enwiki")
|> Wiki.Ores.request!(
models: ["damaging"],
revids: 456789
)
# %{
# "enwiki" => %{
# "models" => %{
# "damaging" => %{"version" => "0.5.1"}
# },
# "scores" => %{
# "456789" => %{
# "damaging" => %{
# "score" => %{
# "prediction" => false,
# "probability" => %{
# "false" => 0.9784615344695441,
# "true" => 0.021538465530455946
# }
# }
# }
# }
# }
# }
# }
"""
Trending in Announcing
Flop is an Elixir library that applies filtering, ordering and pagination parameters to your Ecto queries.
offset-based pagination with...
New
I needed to reuse React components from my Chrome extension in my Phoenix/LiveView backend. I noticed that for Svelte/Vue, there are live...
New
I released Doggo, a collection of unstyled Phoenix components.
https://github.com/woylie/doggo
Features
Unstyled Phoenix components....
New
Edit: 2026 May 15 - This post is archived.
Mob is alive!!
Main docs: mob v0.7.11 — Documentation
A bit of explanation for the slightly c...
New
Hey, I’m Jesse and I’m the main contributor behind Dexter, a full-featured, lightning-fast Elixir LSP optimized for large codebases. It s...
New
Hi everyone,
I’ve been working on this protobuf library for 3 years. We use it in the company I work for, EasyMile, to communicate with ...
New
Hi there! We created Gust: A task orchestrator inspired by Airflow.
For those who have never heard about Aiflow, it’s a Python-based wor...
New
Other Trending Topics
I am happy to introduce the very α version of the new programming language compiled to BEAM.
Welcome Cure.
It has literally three kille...
New
Hi everyone!
The first release candidate for the Expert language server project is now available!
We’ve published a press release detai...
New
With AI doing more of the implementation work, I’ve been wondering how much coding I should deliberately keep doing myself.
My main conc...
New
This showed up on my feed.. anyone heard of it? Just hype?
Ox Alpha is a reasoning model designed for coding, sustained ag...
New
Hey folks,
I just published a post about Hologram’s funding and where the project goes next - the short version:
Curiosum as Main Spons...
New
A little off-topic, but I feel like people here have a good head on their shoulders.
I used to be quite good at making software. Was luc...
New
Categories:
Sub Categories:
Forums
Popular Tags
- #ecto
- #liveview
- #troubleshooting
- #learning-elixir
- #library
- #deployment
- #erlang
- #testing
- #genserver
- #mix
- #absinthe
- #remote-other
- #otp
- #plug
- #how-to-question
- #macros
- #postgres
- #elixirconf
- #channels
- #exunit
- #discussion
- #code-sync
- #podcasts
- #javascript
- #onsite
- #dialyzer
- #docker
- #authentication
- #umbrella
- #full-time-contract
- #podcasts-by-brainlid
- #ecto-query
- #ai
- #elixirconf-us
- #blog-post
- #elixir-ls
- #phoenix_html
- #iex
- #graphql
- #genstage
- #websockets
- #supervisor
- #advent-of-code
- #distillery
- #processes
- #api
- #forms
- #elixirconf-eu
- #metaprogramming
- #hex











Showing Posts 1 to 3- Show Best Posts
- Show All (oldest first)
- Show All (newest first)
adamwight
I’ve spun out a streaming bzip2 codec as part of the work to support Elixir processing of Wikipedia dumps.
Example usage:
I would have liked to give a fully streamed example, but
:httpcrequires more than a dab of glue to do so, and I can’t find any other currently working library to stream HTTP.adamwight
Since it’s fun, I also decided to upgrade the Server-sent Events listener to use a new Broadway integration, replacing the homespun GenStage relay. The library is already functional but very rough—feedback encouraged! Adam Wight / Off-Broadway Server-sent Events · GitLab
adamwight
Now with Livebook examples.
Also, at day job my team is writing a complementary library to parse wiki article HTML dumps, which works already but will be finalized in a few weeks, then we run it on all available dumps.
We’re looking at article footnotes for our particular use case, but it can be generalized to run whatever other parsing modules.