ejstembler

ejstembler

I want to seed my Users table using a CSV file. I have it reading the file okay, but I don’t know how to typecast what CVS decodes (map?) into my Ecto Model.

File.stream!(Path.expand("priv/repo/users.tsv"))
  |> Stream.drop(1) # Skip header row
  |> CSV.decode(separator: ?\t, headers: [:id, :username, :first_name, :middle_name, :last_name, :email, :position, :last_login_at, :last_request_at, :last_request_url, :active, :inserted_at, :updated_at])
  |> Enum.each(&Repo.insert!/1)

Also, my CSV file was exported from SQL Server, and the null columns have NULL as text. Is there a way to replace with nil in a pipeline. Only certain columns could have NULL, not all of them.

Lastly, I’ve already seen the blog post from Wendy; it seems there’s been breaking changes since it was written.

Showing Posts 1 to 1

ejstembler

ejstembler OP

Well, I got this to work after a bit:

alias MyProject.Repo
alias MyProject.User

defmodule MyProject.Seeds do

  def row_to_user(row) when is_map(row) do
    %User{
      id: normalize_id(row.id),
      username: normalize_username(row.username),
      first_name: normalize_null(row.first_name),
      middle_name: normalize_null(row.middle_name),
      last_name: normalize_null(row.last_name),
      email: normalize_null(row.email),
      position: normalize_null(row.position),
      last_logon_at: normalize_date(row.last_logon_at),
      last_request_at: normalize_date(row.last_request_at),
      last_request_url: normalize_null(row.last_request_url),
      active: normalize_active(row.active),
      inserted_at: normalize_date(row.inserted_at),
      updated_at: normalize_date(row.updated_at)
    }
  end

  def normalize_id(value) when is_binary(value) do
    {result, _} = Integer.parse(value)
    result
  end

  def normalize_username(value) when is_binary(value) do
    String.downcase(value)
  end

  def normalize_active(value) when is_binary(value) do
    case value do
      "0" -> false
      "1" -> true
    end
  end

  def normalize_date(value) when is_binary(value) do
    case value do
      "NULL" -> nil
      "" -> nil
      _ ->
        parts = String.split(value, " ")
        Ecto.DateTime.cast!("#{Enum.at(parts, 0)}T#{Enum.at(parts, 1)}")
    end
  end

  def normalize_null(value) when is_binary(value) do
    case value do
      "NULL" -> nil
      "" -> nil
      _ -> value
    end
  end

end

# Users
File.stream!(Path.expand("priv/repo/users.tsv"))
  |> Stream.drop(1) # Skip header row
  |> CSV.decode(separator: ?\t, headers: [:id, :username, :first_name, :middle_name, :last_name, :email, :position, :last_logon_at, :last_request_at, :last_request_url, :active, :inserted_at, :updated_at])
  |> Enum.map(fn (row) -> MyProject.Seeds.row_to_user(row) end)
  |> Enum.each(&Repo.insert!/1)
— All posts loaded —

Where Next? Top

Trending in Questions Top

RSP87
I’m working on a project that simulates the bumbl example in the programming phoenix book. It acts almost like an email client. We have a...
New
kpanic
Hi everyone, I am toying with the idea of building a “match maker” for giving personal help to people that wants to start coding. I sta...
New
nseaSeb
Hello, I know there is an approach for handling lists that allows for optimized traversal, but I can’t recall the specific method (somet...
New
brecabral
Documentation While reading the Scoped Routes section, I noticed that the documentation currently refers to a problem without explainin...
New
velrest
So my question is quite simple and i have found no conclusive answer on forum, google or AI. Should we use :erlang.float for Integer to ...
New
asweet-confluent
I recently noticed that Elixir’s Logger defaults its primary log level to :debug when no :logger, :level application configuration is pre...
New
apz
I’m new to elixir and just tried to install the elixirLS extension for VScode(ium) and it is throwing some errors that I would like help ...
New

Other Trending Topics Top

JesseHerrick
Hey, I’m Jesse and I’m the main contributor behind Dexter, a full-featured, lightning-fast Elixir LSP optimized for large codebases. It s...
New
mudasobwa
I am happy to introduce the very α version of the new programming language compiled to BEAM. Welcome Cure. It has literally three kille...
New
marciok
Hi there! We created Gust: A task orchestrator inspired by Airflow. For those who have never heard about Aiflow, it’s a Python-based wor...
New
mhanberg
Hi everyone! The first release candidate for the Expert language server project is now available! We’ve published a press release detai...
New
jimsynz
Beam Bots (or just BB for short) is a framework for building fault-tolerant robotics applications in Elixir using familiar OTP patterns. ...
New
Dmk
Xamal is a deployment tool for Elixir apps that deploys native releases to bare metal servers over SSH. It’s a port of GitHub - basecamp/...
New

We're in Beta

About us Mission Statement

Options

Thread Display Mode




Thread Preview

Skip Thread Previews