I have a special need to set the id of newly created entities to some integer relating to UNIX timestamp (not autoincrement based). Any idea how to do this? i am using Phoenix 1.3. Thank you.
Are you asking about ecto or elixir’s structs?
With structs you can add something like a new function to return them somewhat prepopulated
defmodule Entity do
defstruct [:name, :timestamp]
def new(name) do
%__MODULE__{
name: name,
timestamp: NaiveDateTime.utc_now()
}
end
end
And you can use @primary_key attribute to configure the schema’s primary key.
@primary_key {:your_id, :id, autogenerate: false}
I have tried your code but it fails. defstruct .. gives an error defstruct has already been called for Entre.Warehouse.Item, defstruct can only be called once per module and if I omit that line and keep the def new(name) do block I get Entre.Warehouse.Item.__struct__/1 is undefined, cannot expand struct Entre.Warehouse.Item.
I am using Phoenix 1.3. I am not sure if the errors I get are related to the newly introduced contexts.
Sorry, I’ve probably confused you …
The example with defstruct that I’ve provided seems to not be what you want. I just wondered if you were asking about ecto or elixir structs.
Judging by the error you get, you were asking about ecto.
To get custom ids you should use @primary_key.
1 Like
@idi527: Sorry, I accidentally hit reply in your post. This reply was meant for the original post.
If your question is about how to do custom primary keys with Ecto then here is a guide in phoenix blog
1 Like
I am going to use the id field itself but instead of having auto-generated serial values I want to set Unix timestamps as ids. I am using Ecto. I understand that the place to set the value is inside the model definition file but still cannot figure out the way to do that. The Postgres solution can do the job, but I am interested in knowing how to do that in Phoenix. Any help is highly appreciated. Thank you.
In the blog post that I linked it says how to edit the migration
defmodule Hello.Repo.Migrations.CreatePlayer do
use Ecto.Migration
def change do
create table(:players, primary_key: false) do
add :name, :string, primary_key: true
add :position, :string
add :number, :integer
timestamps
end
end
end
If I am not mistaken there is nothing stopping you from naming your primary key “id”. I haven’t tried naming a custom primary key “id”. You have to experiment a bit with this :).
Also if you look in the add/3 documentation you will see it can take the option :default
the column’s default value. can be a string, number or a fragment generated by fragment/1
So you can use that to apply the Postgres function.
1 Like
I am just looking at the whole thing from a RoR background. There I could just do self.id = n and that’s all 
Here it looks like the more professional way to do this is inside the schema definition and possibly by PostgreSQL itself. My little question in the meantime, fragment/1 uses a PostgreSQL function/procedure or can it consume a value returned from an Elixir function?
Fragment runs SQL code/functions as prepared statements. You can use it if you want to use a custom Postgresql function to handle your ID generation inside the database. Here is an example about using fragments in Ecto: http://learningelixir.joekain.com/fragments-in-ecto/
If you wan’t to handle ID generation with an Elixir function you don’t need a fragment. You can use put_change/3 in your changeset in your schema when you create it.
Another option is, if you want to make your code cleaner, you can use a custom ecto type as your field and define an autogenerate/0 function. This way you will still be generating your code with Elixir but you will be able to use that type of field in different structs. Here is an example of using uuids for primary key https://blog.fourk.io/uuids-as-primary-keys-in-phoenix-with-ecto-and-elixir-1dd79e1ecc2e. This example looks a bit like what you are trying to achieve. Also here is the code for the UUID type. Notice how it defines an autogenerate/0 function that just calls generate/0.
Disclaimer: I haven’t actually had the chance to use a custom field.
1 Like
Thank you for the valuable information. I have tried to use the put_change/3 inside def changeset(%Item{} = item, attrs) do. It works sort of. It sets the value at the time the form is rendered, not when the entity is saved to the database. In my situation, the value is time related so it should reflect the time when an entity is saved.
I guess, first and 3rd solutions you described can help me achieve the desired result. I have considered setting the value inside the create function. I am not sure if that’s a proper way to do it. I am not sure even how that can be done.
UPDATE
I have just realized that setting the id in the controller is not possible, it must be done either in DB, by custom data-type as suggested by @voger or by some other model way.
Please keep in mind that you are talking with an noob that now learns the ropes so what I say may be wrong. I haven’t tried most of the options myself.
That being said I will try to create a simple project using the link tomconry posted and will post back. Maybe we can both learn something from this :D.
Hello. Here is the example repo. https://github.com/voger/custom_id_test
It took me a while to make the pgsql function work and I didn’t have much time to add controllers or example of custom id using changesets. This example demonstrates how to do it using the database with a sequence as in the link from tomconry. Take a look at the migration.
To use it after you clone it in your computer and do the migrations run iex -S mix and try
iex(1)> alias CustomIdTest.Context
CustomIdTest.Context
iex(2)> Context.create_entry(%{entry: "Reandom text"})
[debug] QUERY OK db=28.5ms queue=0.2ms
INSERT INTO "entry_fragment" ("entry") VALUES ($1) ["Reandom text"]
{:ok,
%CustomIdTest.EntryFragment{__meta__: #Ecto.Schema.Metadata<:loaded, "entry_fragment">,
entry: "Reandom text", id: nil}}
iex(3)> Context.get_entries
[debug] QUERY OK source="entry_fragment" db=4.6ms decode=18.1ms queue=0.2ms
SELECT e0."id", e0."entry" FROM "entry_fragment" AS e0 []
[%CustomIdTest.EntryFragment{__meta__: #Ecto.Schema.Metadata<:loaded, "entry_fragment">,
entry: "random", id: 1616258530136822785},
%CustomIdTest.EntryFragment{__meta__: #Ecto.Schema.Metadata<:loaded, "entry_fragment">,
entry: "random", id: 1616258765361779714},
%CustomIdTest.EntryFragment{__meta__: #Ecto.Schema.Metadata<:loaded, "entry_fragment">,
entry: "Reandom text", id: 1616264079603667971}]
Update:
@acrolink
I added another example on how to set the id from the changeset. This time I used a generator to speed up and you can see the results in a browser. I probably messed up naming and spelling because it was done in a hurry. I hope it doesn’t mind. 
To use it go to http://localhost:4000/things_with_changeset
Especially interesting I believe are [the migration] (https://github.com/voger/custom_id_test/blob/master/priv/repo/migrations/20171002055049_create_things_with_changeset.exs ) and the schema
I modified the controller with this line alias CustomIdTest.CustomIdTest, as: CIT and renamed the relevant function calls. This is irrelevant to the problem at hand. I only did this because I messed up the naming. You don’t have to do this in your controllers.
It is not a well thought example but I hope it helps.
2 Likes
Thank you very very much for the efforts put into this. I am new to Phoenix and pretty impressed by the speed it offers. Yet, I see that the documentation can be sometimes not clear and there is currently small amount of contributed libraries (elixir gems or whatever they are called) and rather few HOWTO articles. Your code for sure makes a difference.
When it comes to doing it by PostgreSQL, I think your solution should work fine for most uses but think if entities will be created in bulk (e.g. 100 at a time), there is a possibility that the auto generate procedure would produce identical ids.
The changeset method is fine. I have already tried it. However, it sets the value already when the form is loaded. In a real system (with bulk insertions are made by custom controller calls or by Angualar UI with a JSON API backend) that should not be a problem I guess (the id should be generated at the time of insertion only, not before).
So, you think if @primary_key {:id, :id, autogenerate: false} is set then the id can be set inside the params object or does it get only set by the changeset function upon its initialization?
1 Like
This function originally comes from this instagram article:
https://engineering.instagram.com/sharding-ids-at-instagram-1cf5a71e5a5c
we can generate 1024 IDs, per shard, per millisecond
I have just got time only now to give your code the attention and implementation it deserves Thank you very much, more details later.
I am glad you found my examples helpful
When it comes to doing it by PostgreSQL, I think your solution should work fine for most uses but think if entities will be created in bulk (e.g. 100 at a time), there is a possibility that the auto generate procedure would produce identical ids.
These are not solutions. The SQL example is just that. An example how you can apply a SQL function in your migration. The example I used is just a copy paste from the link so I can have some function to showcase. I didn’t even took the time to understand what it does :D. You probably want to use your own implementation.
The changeset method is fine. I have already tried it. However, it sets the value already when the form is loaded. In a real system (with bulk insertions are made by custom controller calls or by Angualar UI with a JSON API backend) that should not be a problem I guess (the id should be generated at the time of insertion only, not before).
Well, indeed it generates an ID when the form is created but it replaces it with a new ID when the form is submitted. The current behavior is not optimal so you could either create two changesets (one for new and one for create)
@doc false
def changeset(%ThingWihChangeset{} = thing_wih_changeset, attrs) do
thing_wih_changeset
|> cast(attrs, [:entry])
|> validate_required([:entry])
end
def create_changeset(%ThingWihChangeset{} = thing_wih_changeset, attrs) do
thing_wih_changeset
|> changeset(attrs)
|> put_change(:id, get_current_unix_time())
end
and then in lib/custom_id_test/custom_id_test/thing_wih_changeset.ex the create function becomes
def create_thing_wih_changeset(attrs \\ %{}) do
%ThingWihChangeset{}
|> ThingWihChangeset.create_changeset(attrs)
|> Repo.insert()
end
or alternatively you can remove completely the put_change from the changeset and use it in your create function
def create_thing_wih_changeset(attrs \\ %{}) do
%ThingWihChangeset{}
|> ThingWihChangeset.changeset(attrs)
|> Ecto.Changeset.put_change(:id, get_current_unix_time())
|> Repo.insert()
end
defp get_current_unix_time do
DateTime.utc_now() |> DateTime.to_unix()
end
Also for the :autogenerate option I went after the documentation. Ecto.Schema — Ecto v3.11.1 where it says that if set to true the database is responsible for setting it.
1 Like
It is fine since what is important is that the time-stamp that gets stored in the database reflects time at save. Using put_change inside the create method is what I was originally looking for (thanks for setting me on that direction, in Phoenix 1.3 this function is found inside the context definition, I failed to find it before).
When defining the custom id field in the schema, do you know how to set the column type as BIGINT (postgres)? In the migration itself? And how to define it as such in the schema file? Thank you.
Hello. I don’t have access to elixir right now so I can’t test what I say is accurate. Anyway here is my attempt
If you need autoincrementing bigint you may not need to do anything. [According to the docs]
(https://hexdocs.pm/ecto/Ecto.Migration.html#content)
:migration_primary_key - Ecto uses the :id column with type :bigserial but you can configure it via:
config :app, App.Repo, migration_primary_key: [id: :uuid, type: :binary_id]
So it seems Ecto already uses an autoincrementing bigint as an :id. Not the full range. Only the positive values
If for some reason you don’t want autoincrementing values then you do this in your schema
def change do
create table(:books, primary_key: false) do
add :id, :bigint, primary_key: true
add :title, :string
timestamps()
end
but now you have to handle the key yourself.
Again, I can’t test the accuracy of the above.
1 Like
