hfiguera

hfiguera

Hi everyone,

I’ve released Obscura 0.1.1, an Elixir library for detecting and anonymizing personally identifiable information before it reaches logs, external APIs, analytics systems, or LLM providers.

The dependency-light core recognizes structured identifiers such as email addresses, phone numbers, credit cards, SSNs, IBANs, IP addresses, URLs, and domains. Optional local Nx/Bumblebee profiles add person, location, and organization recognition without requiring a hosted recognition service.

A basic redaction looks like this:

{:ok, result} =
  Obscura.redact(
    "Contact jane@example.com or call 202-555-0188",
    profile: :fast,
    entities: [:email, :phone]
  )

result.text
#=> "Contact [EMAIL] or call [PHONE]"

Obscura also supports nested Elixir data:

input = %{
  customer: %{email: "jane@example.com"},
  password: "secret"
}

{:ok, result} =
  Obscura.redact(input,
    entities: [:email],
    field_policies: %{password: :drop}
  )

result.data
#=> %{customer: %{email: "[EMAIL]"}}

The available operators are replacement, redaction, masking, hashing, pseudonymization, and application-defined callbacks. Pseudonymization can use an ETS-backed vault when an application needs to restore selected values after an LLM or external-service round trip.

The three stable profiles serve different use cases:

  • :fast uses dependency-light deterministic and parser-backed recognition.
  • :balanced adds one local model for general person, location, and organization recognition.
  • :accurate uses a two-model cascade for the highest measured general accuracy.

Model-backed profiles require explicit preparation and reusable runtime resources. Normal analysis never downloads a model implicitly.

One important asset note: :balanced and :accurate currently use an OntoNotes-trained TNER checkpoint. Obscura does not bundle or license that checkpoint, and LDC confirmed that commercial use requires LDC for-profit membership. The dependency-light :fast profile is unaffected. The complete review is documented in the model-asset licensing guide.

Links:

This is the first public release, and I’d particularly appreciate feedback about the API, entity coverage, structured-data behavior, model preparation, deployment, performance, and use cases that the current profiles do not handle well.

Showing Posts 1 to 6

hfiguera

hfiguera OP

I wrote a follow-up engineering case study about validating Obscura’s model-backed profiles on Linux with an NVIDIA Tesla T4 and EXLA.

The article covers the complete path from a CPU environment to real CUDA inference, including an NVSHMEM ABI failure that initially prevented EXLA from loading.

Rather than treating CUDA discovery as sufficient proof, I verified four independent signals:

  • EXLA discovered the CUDA client.
  • A compiled Nx operation returned an EXLA-backed tensor.
  • XLA identified the Tesla T4 and cuDNN runtime.
  • The BEAM process appeared in nvidia-smi using GPU memory.

I then prepared the :balanced and :accurate profiles and ran real Obscura inference. This is a compatibility report on one Tesla T4, not a production throughput benchmark.

thiagomajesk

thiagomajesk

Thanks for sharing @hfiguera, this is extremely useful! :rocket:

alvinkatojr

alvinkatojr

Woah, this is neat! Thanks for sharing!

hfiguera

hfiguera OP

I published a third Obscura engineering article: A Model Card Is Not a License: What We Learned Shipping Local NER in Obscura.

While reviewing the :balanced and :accurate profiles, I found that the MIT license of the base RoBERTa model did not answer whether the fine-tuned OntoNotes checkpoint was authorized for commercial use.

I traced the model’s provenance and contacted LDC directly. They confirmed that commercial use of this specific checkpoint requires an LDC for-profit membership. That conclusion is checkpoint-specific and does not apply to TNER models generally.

The article documents the investigation and the resulting Obscura changes, including machine-readable licensing metadata and notices before model downloads:

fairwaydm

fairwaydm

Very Cool - Excited about checking this out!

hfiguera

hfiguera OP

I published a new engineering article about privacy-safe request logging in Phoenix.

The problem is subtle: Obscura.Phoenix.Plug can create a redacted assign while leaving conn.params unchanged for controllers, but Phoenix’s default telemetry logger still reads the original parameters.

The article explains the opt-in Obscura.Phoenix.Logger integration, why the default Phoenix logger must be disabled, how route templates and sanitized parameters are logged, and how failures are handled without falling back to raw request data.

The integration is available in Obscura 0.1.3. Feedback from people operating Phoenix applications would be especially useful.

— All posts loaded —

Where Next? Top

Trending in Announcing Top

bluzky
You may know https://ui.shadcn.com/, a UI component library for React. I really love it’s design style and components. I’ve built some co...
387 15136 120
New
woylie
Flop is an Elixir library that applies filtering, ordering and pagination parameters to your Ecto queries. offset-based pagination with...
New
restlessronin
The repo is at GitHub - cyberchitta/openai_ex: Community maintained Elixir library for OpenAI API · GitHub. Docs are at OpenaiEx User Gu...
152 11030 135
New
JesseHerrick
Hey, I’m Jesse and I’m the main contributor behind Dexter, a full-featured, lightning-fast Elixir LSP optimized for large codebases. It s...
New
shahryarjb
The Chelekom project is a library of Phoenix and LiveView components generated via Mix tasks to fit developer needs seamlessly. One of i...
New
woylie
Phoenix components for pagination, sortable tables and filter forms with Flop and (optionally) Ecto. pagination cursor pagination sorta...
New
kip
Please say hi to a new lib, Astro that aims to deliver easy-to-consume astronomy calculations of practical use. For now it only calculat...
New

Other Trending Topics Top

akoutmos
@hugobarauna and I (Alex Koutmos) have been hard at work on writing a book on Nerves that takes you from simply blinking LEDs to building...
New
spammy
I’m looking to build a personal workflow to quickly deploy web applications written in elixir/phoenix, for local consumption (ie not on t...
New
mudasobwa
I am seeing a lot of aplications of Argumentum ad Vericundiam in software discussions. They do link some piece of writing and point us to...
New
alexslade
Fly’s CEO posted this recently - Turn And Face The Strange · The Fly Blog It says that Fly is going all-in on sprites, which is a worry ...
New
bartblast
Hey folks, I just published a post about Hologram’s funding and where the project goes next - the short version: Curiosum as Main Spons...
New
Herve37
We’re evaluating API mocking tools for OpenAPI-based projects and would love to hear what other teams are using. We’re particularly inte...
New

We're in Beta

About us Mission Statement

Options

Thread Display Mode




Thread Preview

Skip Thread Previews