hfiguera
Hi everyone,
I’ve released Obscura 0.1.1, an Elixir library for detecting and anonymizing personally identifiable information before it reaches logs, external APIs, analytics systems, or LLM providers.
The dependency-light core recognizes structured identifiers such as email addresses, phone numbers, credit cards, SSNs, IBANs, IP addresses, URLs, and domains. Optional local Nx/Bumblebee profiles add person, location, and organization recognition without requiring a hosted recognition service.
A basic redaction looks like this:
{:ok, result} =
Obscura.redact(
"Contact jane@example.com or call 202-555-0188",
profile: :fast,
entities: [:email, :phone]
)
result.text
#=> "Contact [EMAIL] or call [PHONE]"
Obscura also supports nested Elixir data:
input = %{
customer: %{email: "jane@example.com"},
password: "secret"
}
{:ok, result} =
Obscura.redact(input,
entities: [:email],
field_policies: %{password: :drop}
)
result.data
#=> %{customer: %{email: "[EMAIL]"}}
The available operators are replacement, redaction, masking, hashing, pseudonymization, and application-defined callbacks. Pseudonymization can use an ETS-backed vault when an application needs to restore selected values after an LLM or external-service round trip.
The three stable profiles serve different use cases:
:fastuses dependency-light deterministic and parser-backed recognition.:balancedadds one local model for general person, location, and organization recognition.:accurateuses a two-model cascade for the highest measured general accuracy.
Model-backed profiles require explicit preparation and reusable runtime resources. Normal analysis never downloads a model implicitly.
One important asset note: :balanced and :accurate currently use an OntoNotes-trained TNER checkpoint. Obscura does not bundle or license that checkpoint, and LDC confirmed that commercial use requires LDC for-profit membership. The dependency-light :fast profile is unaffected. The complete review is documented in the model-asset licensing guide.
Links:
- Hex: obscura | Hex
- Documentation: obscura v0.1.1 — Documentation
- Example workbench: GitHub - hfiguera/obscura_examples: Phoenix LiveView workbench demonstrating Obscura's public PII detection and anonymization APIs · GitHub
- Practical guide: Protecting PII in Elixir Before It Reaches Logs, APIs, and LLMs · Obscura
- Model licensing: Third-Party Dependency And Model Asset Review — obscura v0.1.1
This is the first public release, and I’d particularly appreciate feedback about the API, entity coverage, structured-data behavior, model preparation, deployment, performance, and use cases that the current profiles do not handle well.
Where Next?
Trending in Announcing
Other Trending Topics
Latest on Elixir Forum
Sponsor Spotlight
Producing high quality Elixir screencasts since 2017.
Categories:
Sub Categories:
Forums
Popular Tags
- #ecto
- #liveview
- #troubleshooting
- #learning-elixir
- #library
- #deployment
- #erlang
- #testing
- #genserver
- #mix
- #absinthe
- #remote-other
- #otp
- #plug
- #how-to-question
- #macros
- #postgres
- #elixirconf
- #channels
- #exunit
- #discussion
- #code-sync
- #podcasts
- #javascript
- #onsite
- #dialyzer
- #docker
- #authentication
- #umbrella
- #full-time-contract
- #podcasts-by-brainlid
- #ecto-query
- #elixirconf-us
- #ai
- #blog-post
- #elixir-ls
- #phoenix_html
- #iex
- #graphql
- #genstage
- #websockets
- #supervisor
- #advent-of-code
- #distillery
- #processes
- #api
- #forms
- #metaprogramming
- #hex
- #security
Our Sponsors
Build Elixir applications with speed and confidence.
Supporting innovation across the BEAM ecosystem.
Catch errors, track performance, monitor hosts and more.
Error tracking, logs, uptime, and Just Enough APM. Built for developers.
Elixir, Erlang & BEAM conferences from Code Sync - join us in 2026!
Producing high quality Elixir screencasts since 2017.
**Building the future of software with AI, trust, and human connection.**
The deployment platform built for Elixir: PaaS ease, VPS control.
The team behind Membrane, Popcorn, LiveDebugger. Available for hire.
Courses that'll move you from confusion to "Aha, now I get it!"
Showing Posts 1 to 10- Show Best Posts
- Show All (oldest first)
- Show All (newest first)
hfiguera
I wrote a follow-up engineering case study about validating Obscura’s model-backed profiles on Linux with an NVIDIA Tesla T4 and EXLA.
The article covers the complete path from a CPU environment to real CUDA inference, including an NVSHMEM ABI failure that initially prevented EXLA from loading.
Rather than treating CUDA discovery as sufficient proof, I verified four independent signals:
nvidia-smiusing GPU memory.I then prepared the
:balancedand:accurateprofiles and ran real Obscura inference. This is a compatibility report on one Tesla T4, not a production throughput benchmark.thiagomajesk
Thanks for sharing @hfiguera, this is extremely useful!
alvinkatojr
Woah, this is neat! Thanks for sharing!
hfiguera
I published a third Obscura engineering article: A Model Card Is Not a License: What We Learned Shipping Local NER in Obscura.
While reviewing the
:balancedand:accurateprofiles, I found that the MIT license of the base RoBERTa model did not answer whether the fine-tuned OntoNotes checkpoint was authorized for commercial use.I traced the model’s provenance and contacted LDC directly. They confirmed that commercial use of this specific checkpoint requires an LDC for-profit membership. That conclusion is checkpoint-specific and does not apply to TNER models generally.
The article documents the investigation and the resulting Obscura changes, including machine-readable licensing metadata and notices before model downloads:
fairwaydm
Very Cool - Excited about checking this out!
hfiguera
I published a new engineering article about privacy-safe request logging in Phoenix.
The problem is subtle:
Obscura.Phoenix.Plugcan create a redacted assign while leavingconn.paramsunchanged for controllers, but Phoenix’s default telemetry logger still reads the original parameters.The article explains the opt-in
Obscura.Phoenix.Loggerintegration, why the default Phoenix logger must be disabled, how route templates and sanitized parameters are logged, and how failures are handled without falling back to raw request data.The integration is available in Obscura 0.1.3. Feedback from people operating Phoenix applications would be especially useful.
hfiguera
I published a new article about keeping customer identity inside an Elixir application while an agent and remote model work with stable pseudonyms.
The difficult part is not only protecting the initial prompt. Raw identifiers can also escape through tool arguments, tool results, streamed responses, logs, and telemetry.
The reference application uses Phoenix LiveView, Jido, ReqLLM, OpenAI, and Obscura. Trusted tools restore only the value needed for a local lookup, protect their results before returning them, and leave final restoration to the trusted LiveView.
The complete example is public and can run in deterministic mode without an OpenAI key. I would be interested in feedback from anyone building agents that call application tools.
MikaelFangel
I played with a similar idea for a while and landed on the same name. My question is what does this add over https://presidio.dataprivacystack.org (Python framework)?
hfiguera
That is a fair question. Presidio is the closest comparison, and it is currently broader and more mature. It has a larger recognizer ecosystem, multiple NLP engines and languages, image redaction, structured data support, and several deployment options. If I were working in Python, or comfortable operating Presidio as a separate service, I would seriously consider using it.
The main value Obscura adds is native integration with Elixir and the BEAM. Its fast path runs inside the application without a Python service or HTTP hop. It works directly with text, nested Elixir values, structs, Logger metadata, Phoenix request and realtime logging, LLM messages, streaming responses, and session vaults for reversible pseudonymization.
That means the privacy boundary can live where Elixir data actually crosses into logs, Phoenix telemetry, external APIs, or model providers, while remaining part of the application supervision and lifecycle.
Obscura is not a complete Presidio replacement. It is still an early
0.1.xlibrary, while Presidio covers more entities, languages, media, and deployment scenarios. I have also compared Obscura against Presidio using pinned datasets. The results are mixed: Obscura performs better in some measured profiles and datasets, while Presidio remains better for some entities and recall characteristics. I do not claim universal superiority. The complete comparison is available in this report.So the short answer is: Presidio is the stronger general privacy framework today. Obscura is useful when an Elixir application needs privacy controls inside the BEAM and at Elixir specific application boundaries, without introducing a separate Python service.
hfiguera
Following up on the Presidio discussion: I’ve released Obscura 0.2.0, which adds a stable
:efficientCPU profile.It combines Obscura’s deterministic recognizers with spaCy’s
en_core_web_lg3.8.0 NER—the same model used in our pinned Presidio comparison. Obscura applies its own entity mapping, overlap resolution, and boundary policy, so sharing the model does not make the complete detectors equivalent.NER runs in a standalone Rust executable through an Erlang Port, with supervised workers owned by the Elixir application. Python is used during asset provisioning, but inference requires neither Python nor network access. Linux x86-64 and ARM64 are supported server targets, alongside Apple Silicon macOS for development.
The evaluation shows a tradeoff. On 5,000 heldout synthetic English documents, compared with
:fast, the profile produced:It also achieved higher exact-span F1 than the pinned Presidio configuration, yet left more annotated values completely uncovered. That distinction matters: better F1 does not automatically mean stronger privacy protection.
I wrote up the accuracy and coverage results, native runtime design, sustained CPU workloads, and installation workflow here:
I’d particularly welcome feedback from people operating native inference in Elixir applications, and synthetic examples of missed names, incomplete address redaction, or unnecessary replacement.