I fill the virtual fields recursively after fetching data, e.g.
User
|> # some Ecto query with dynamic preloads
|> # Repo call
|> fill_virtual_fields()
And the fill_virtual_fields/1 function will fill the virtual fields for the entity and all the nested entities; this allows the Context not having to care about which field is virtual or not, and how to fill it. fill_virtual_fields/1 will call an implemented behaviour on all entities that have a need to fill virtual fields.