Trying to understand Ash Field Policy Behaviour

[torepettersen]

I am trying to wrap my head around field_policies, but they don’t quite make sense to me.

From the docs: “If any field policies exist then all fields must be authorized by a field policy.”

So then let’s start with an example:

attributes do
  uuid_primary_key :id

  attribute :subject, :string, public?: true
  attribute :admin_note, :string, public?: true
  attribute :message, :string

  timestamps()
end

relationships do
  belongs_to :user, User
end

policies do
  policy action_type(:read) do
    authorize_if actor_attribute_equals(:is_admin, true)
    authorize_if relates_to_actor_via(:user)
  end
end

field_policies do
  field_policy :admin_note do
    authorize_if actor_attribute_equals(:is_admin, true)
  end

  field_policy :subject do
    authorize_if actor_attribute_equals(:is_admin, true)
    authorize_if relates_to_actor_via(:user)
  end
end

If I now query the ticket:

iex> Ash.read!(Helpdesk.Support.Ticket, actor: user)
[
  #Helpdesk.Support.Ticket<
    user: #Ash.NotLoaded<:relationship, field: :user>,
    __meta__: #Ecto.Schema.Metadata<:loaded, "tickets">,
    id: "4ce1002e-2c45-4dba-badc-d37216398475",
    subject: "Something is broke",
    admin_note: #Ash.ForbiddenField<field: :admin_note, type: :attribute, ...>,
    message: "We are investigating",
    inserted_at: ~U[2024-07-03 05:16:08.245708Z],
    updated_at: ~U[2024-07-03 05:16:08.245708Z],
    user_id: "98e8e6b1-486a-412f-9421-ee2b447ad75d",
    aggregates: %{},
    calculations: %{},
    ...
  >
]

As expected, the admin_note is hidden, since the user is not an admin. But why can I now see all the other fields, like message? message is private, so why can I still see that field? It seems like the only way to make message hidden would be to make it public, which does not quite make sense to me.

I could of course make every field public, but that feels a bit counterintuitive since all public fields are accessible through the :* shortcut. If I then only have default actions, I would be able to access and change the admin_note, although I should not be able to change that field.

actions do
  defaults [:read, :destroy, create: :*, update: :*]
end

iex> Ash.update!(ticket, %{admin_note: "I should not be allowed to do this"}, actor: user)
#Helpdesk.Support.Ticket<
  user: #Ash.NotLoaded<:relationship, field: :user>,
  __meta__: #Ecto.Schema.Metadata<:loaded, "tickets">,
  id: "4ce1002e-2c45-4dba-badc-d37216398475",
  subject: "new message",
  admin_note: #Ash.ForbiddenField<field: :admin_note, type: :attribute, ...>,
  message: "We are investigating",
  inserted_at: ~U[2024-07-03 05:16:08.245708Z],
  updated_at: ~U[2024-07-03 06:24:09.825098Z],
  user_id: "98e8e6b1-486a-412f-9421-ee2b447ad75d",
  aggregates: %{},
  calculations: %{},
  ...
>
iex> [ticket] = Ash.read!(Helpdesk.Support.Ticket, actor: admin)
[
  #Helpdesk.Support.Ticket<
    user: #Ash.NotLoaded<:relationship, field: :user>,
    __meta__: #Ecto.Schema.Metadata<:loaded, "tickets">,
    id: "4ce1002e-2c45-4dba-badc-d37216398475",
    subject: "new message",
    admin_note: "I should not be allowed to do this",
    message: "We are investigating",
    inserted_at: ~U[2024-07-03 05:16:08.245708Z],
    updated_at: ~U[2024-07-03 06:26:50.409949Z],
    user_id: "98e8e6b1-486a-412f-9421-ee2b447ad75d",
    aggregates: %{},
    calculations: %{},
    ...
  >
]

So then I have a few questions:

• Why do I need to make a field public to be able to hide it? Wouldn’t it have made sense to hide all fields if you add a field policy to one field? Just like the docs says.
• Why do I need to make a field public for it to have an effect on the field policy? Sounds a bit countrer intutive to make all fields public, to hide them.
• Why can only public fields be part of the field policies?

field_policies:
 Invalid field reference(s) in field policy: [:message]

Only non primary-key, public attributes, calculations and aggregates are supported.

• Lastly, it is hopefully a bug that I am allowed to write to a field that I should not be allowed to see?

[zachdaniel]

@torepettersen

So then I have a few questions:

• Why do I need to make a field public to be able to hide it? Wouldn’t it have made sense to hide all fields if you add a field policy to one field? Just like the docs says.
• Why do I need to make a field public for it to have an effect on the field policy? Sounds a bit countrer intutive to make all fields public, to hide them.
• Why can only public fields be part of the field policies?

I think these are all essentially the same question? Perhaps this will clarify:

1. Only public attributes currently support having field policies because fields that are private are considered “internal” to the resource and shouldn’t be displayed to users, etc. For example AshJsonApi and AshGraphql do not include private fields in interfaces. We can potentially add something (opt-in) that will replace all private fields with %Ash.ForbiddenField{}, because they are private there would never be a case where a user should be able to see them. With that said, typically private fields are still used by parts of your system, just not displayed to users.

2. field policies are only for reading. You prevent writes to attributes using regular policies.

What kind of thing are you building? If you’re building APIs private fields are hidden automatically by the API extensions.

If you’re building a UI with LiveView, I assume you’re primarily looking for things like protections around accidentally displaying private attributes? If so, the proposed “hide all private fields” could be implemented like this:

field_policies do
  hide_private? true
  
  field_policy ...
end

[torepettersen]

Thank you for the detailed explanation

I am building a UI with Phoenix LiveView and have never tried AshJsonApi and AshGraphql, so maybe that is where my confusion around public attributes comes from. If field policies only apply to public attributes, does that mean that field policies was initially intended for use in AshJsonApi and AshGraphql, and that it is less practical if I want to use it with LiveView?

I don’t think what I want is to hide private fields if I apply field policies. Then I would not even be able to use those fields, even for internal use. I think what I would have rather wanted, is for a way to include all fields (even private once) into field policies.

And if visibility of all fields would be covered by fields polices, that would also make the use of public attributes redundant, since the field polices would be deciding what should be visible to a user or not. I personally would prefer fields policies deciding what should be visible to a user or not, since that seems more explicit and give you more fine-grained control than a boolean. It also makes it a bit confusing having both the public field and field polices share the control of what should be visible or not. But that might be hard to change and don’t know if you agree.

field policies are only for reading. You prevent writes to attributes using regular policies.

Got to admit this one feels like a foot gun. My initial assumption was that field policies would cover both read and write. In the example case with a ticket system, both the admin and a regular user would normally be allowed to do writes to a ticket. But the regular user would normally be a bit more restricted and can only add follow up messages, while it should not be allowed to make changes to admin only information like an internal admin note. So would it be possible to make them also cover write operations?

[zachdaniel]

@torepettersen

I don’t think what I want is to hide private fields if I apply field policies. Then I would not even be able to use those fields, even for internal use. I think what I would have rather wanted, is for a way to include all fields (even private once) into field policies.

What policies would you write for private fields? If a field policy fails, the value is hidden, so you would not have a way to use them in a LiveView for example. If the field policy would ever pass(meaning a user should be shown the value), then the field should be marked public to indicate that.

Unfortunately, we can’t control what is done inside of a liveview with private fields, and hiding them is generally not desired for the reason you mentioned (using for internal purposes). But in api interfaces we have more control and so private fields are never shown and public fields are shown when their field policy passes.

It might be a footgun that field policies only cover reads, but it’s stated in the documentation and even cursory testing would highlight that fact.

Field policies are not necessary for writing, and that is best expressed as regular policies because any forbidden write is forbidden in its entirety while field policies pass/fail allowing a “partial read”. This is the reason for them being separate and being grouped by the relevant fields.

[torepettersen]

What policies would you write for private fields? If a field policy fails, the value is hidden, so you would not have a way to use them in a LiveView for example. If the field policy would ever pass(meaning a user should be shown the value), then the field should be marked public to indicate that.

I don’t necessarily want to write a policy for private fields. But with the current behavior, I can still use private fields in the LiveView’s since Ash has no way of distinguishing if the field is used internally or in a LiveView. I could however let Ash know if it is a internal or regular user that is calling.

So maybe I can show it better with an example.

attributes do
  attribute :admin_only_field, :string, public?: true
  attribute :not_so_secret_field, :string, public?: true
  attribute :super_secret_internal_field, :string
end

field_policies do
  field_policy :admin_only_field do
    authorize_if actor_attribute_equals(:is_admin, true)
  end

  field_policy :not_so_secret_field do
    authorize_if actor_attribute_equals(:is_admin, true)
    authorize_if relates_to_actor_via(:user)
  end
end

With the current behavior I would still be able to use super_secret_internal_field in my LiveView, while the admin_only_field would be hidden if I am not an admin.

iex> Ash.read!(SomeResource, user: regular_user)
%{
  admin_only_field: #Ash.ForbiddenField<>,
  not_so_secret_field: "Nothing secret",
  super_secret_internal_field: "No one should see this",
}

Imagine that there are no difference between public and private fields and the field policy would force you to list all fields. Then the code could look like this:

attributes do
  attribute :admin_only_field, :string
  attribute :not_so_secret_field, :string
  attribute :super_secret_internal_field, :string
end

field_policies do
  field_policy :admin_only_field do
    authorize_if actor_attribute_equals(:is_admin, true)
  end

  field_policy :not_so_secret_field do
    authorize_if actor_attribute_equals(:is_admin, true)
    authorize_if relates_to_actor_via(:user)
  end
  
  field_policy :super_secret_internal_field do
    authorize_if is_internal()
  end
end

Then super_secret_internal_field would be hidden to both regular users and admins:

iex> Ash.read!(SomeResource, user: regular_user)
%{
  admin_only_field: #Ash.ForbiddenField<>,
  not_so_secret_field: "Nothing secret",
  super_secret_internal_field: #Ash.ForbiddenField<>,
}

If I then want to use the field internally, I can let Ash know:

iex> Ash.read!(SomeResource, user: :internal)
%{
  admin_only_field: "Some admin note",
  not_so_secret_field: "Nothing secret",
  super_secret_internal_field: "No one should see this",
}

[zachdaniel]

Ah, so I had discounted that as an option. requiring a separate read because that basically means that the pattern would necessitate multiple queries. You’d be opting into worse performance with that pattern, and would still have both records with the fields hidden and records without in the same liveview, so the mistakes that can be made are just moved, not actually eliminated.

There is a field called original_value that we use in certain circumstances to track the hidden value (this is required to write-back embedded resources when the user can’t see all the fields). We could populate that when hide_private? is used, so that would have the effect of making one additional step required to use a private field in your liveview (prompting the developer to reconsider), and would also prevent the need for making multiple queries to prevent that class of mistakes.

The reason hide_private? makes sense to me is:

1. It’s backwards compatible with existing usage.
2. There should never be a private attribute with a field policy that can pass. That means it’s public (to at least someone).

Thoughts?

EDIT: so like if you just did record.private_field you’d see %Ash.ForbiddenField{}, but if it’s an “internal usage” you could say record.private_field.original_value (only for private fields)

[torepettersen]

Thanks for explaining Then I think the hide_private? would make more sense to me. Would also be cool if it would be possible to set it as a global parameter.

And doubt I would frequently need the extra step for truly internal stuff. In those cases I would probably rather just make myself an internal user and set up propper field policy rules for the internal user as well.

[zachdaniel]

Can you open a feature request in ash for this? We can look into it, likely not a near term priority though. (PRs welcome of course!)

[torepettersen]

Thank you very much Yes, I added an issue on Github. Will see if I manage to create a PR, but got to admit, it seems a bit complex.