Looking for guidance on creating a new admin panel for Phoenix

[sobolevn]

Not so time ago I have spent a lot of time thinking about and prototyping new admin panel for phoenix. No one loves to write admin panels by himself.

But I have a strong feeling that I need to discuss my ideas with community members before reinventing the wheel.

I am also not quite sure that I can handle a project like that by myself, it requires a lot of skills: choosing good code structure, creating good API design. I will highly appreciate any help with it.

I am also looking for guidance from more experienced phoenix developers.

Existing technologies

I have tried ex_admin and found it great. I have even contributed to the project. By there are some issues with it and something does not feel right. I can name several things:

1. It uses dsl, which is pure magic. It is stated to be inspired by ‘ActiveAdmin’ (I have never actually seen Ruby on Rails in my life). But elixir is not like ruby. It is hard to reimplement some features.
2. It uses it’s own tools for almost anything: generating html, routing, etc. It does not follow the same rules and principles which phoenix declares.
3. Source code is very hard to read and modify. It almost impossible to find a bug there. That is caused by custom rich dsl and a lot of macro magic.
4. Frontend is tightly coupled with the backed, in contrast to phoenix-js which is a separate module

Anyway, I am grateful for the great tool I am using on 2 of my projects today.

Ideas

My core idea is: new tool should reuse existing phoenix-way of doing things, but include some magic to get rid of (or reduce) the boilerplate code.

Routes

I really like routes from phoenix, so I would like to reuse them. Here’s a brief example of what seems to me as a reasonable API:

defmodule TestProject.Router do
  use TestProject.Web, :router
  use PhoenixAdmin.Router
  alias TestProject.Accounts.User

  # Pipelines:

  pipeline :browser do
    plug :accepts, ["html"]
    plug :fetch_session
    plug :fetch_flash
  end

  pipeline :require_auth do
    plug Guardian.Plug.EnsureAuthenticated,
      handler: TestProject.AuthController
  end

  # Scopes:

  scope "/admin" do
    pipe_through :browser  # or `:api` if you need only `json`
    pipe_through :require_auth  # you can reuse ANY already existing plugs 

    scope "/user", PhoenixAdmin.CRUD(User) do
      # Registering model for our basic CRUD controller,
      # note, that we are using the same DSL as Phoenix itself.
      resource "/users", UserController, only: [:new, :create, :update, :destroy]
      # Maybe we should specify context and context functions to work with the objects via context?
    end
  end
end

This seems to be a standard way of doing things in phoenix. Where’s the magic? Actually, there’s no magic here. Just pass a regular controller module. It might be even a single controller for both admin and application code. But what to do if that functionality differs?

At first, I thought that passing models and auto-generating controllers and forms from them might be a good idea. Right now I am not so sure.

phoenix uses generators to create new controllers. Maybe we should do so as well? It offers some very good points:

1. Have all your code in front of you. Need to modify something? No problem
2. You don’t have to learn new dsl
3. It is easy to have single code base for your application and admin panel
4. Reuse everything!

How generations should work? What are the reasonable defaults?

Forms, views and templates

I like how phoenix renders templates and json. It uses pretty much the same API.
So, let’s reuse it!

We also need to provide some kind of default template: AdminLTE or similar.
And it feels like providing some helper functions via View to render styled fields (or other useful parts) is pretty useful.

And also maybe something like “dynamic forms” described here.

It should be possible to include, modify, and extend already existing admin templates and template parts.

Configuration

Admin panels have a lot of configuration. Like (warning, django terms ahead): list_fields, search_fields, and so on.

I see it like this:

defmodule AdminTest.Web.UserController do
  use AdminTest.Web, :controller
  
  defimpl PhoenixAdmin.Config do
     # Here's a configuration for this specific controller:
     def list_items(val), do: ...
     def search_items(val), do: ...
  end
end

Conclusion

That’s about it.
What do you think of it? Are there any other ideas?
Would you use such a thing? Or not? Why?

Here’s a link to the repo: https://github.com/elixir-lang-moscow/phoenix_admin
Here’s a link to the package (there’s currently nothing there): https://hex.pm/packages/phoenix_admin

If you would like to contribute to the development, it is more than welcomed. We can continue a discussion on github.

Thanks!

[hrishikeshchoudhari]

@sobolevn

2. It uses it’s own tools for almost anything: generating html, routing, etc. It does not follow the same rules and principles which phoenix declares.

I agree with #1 and #2 – I tried ExAdmin a few months ago, and couldn’t integrate it well with the rest of my Phoenix app. It felt like a whole “framework-y” way of doing things…

[StefanHoutzager]

I also use ex_admin, it doesn’t feel right to me also. I heard of a company that switched to https://github.com/infinitered/torch , maybe that’s an option for you?

[tmbb]

You might want to collaborate on the talon admin framework, which is being developed by @smpallen99, the author of ExAdmin. It tries to address your concerns that ExAdmin doesn’t follow the “phoenix way”.

[smpallen99]

Hey guys. I feel ur pain on ex_admin’s magic and DSL hell. It was the first phoenix library I wrote while building my first phoenix app. I’m taking all my learnings from ex_admin and applying them to the new [Talon](https://github.com/talonframework/talon] project.

Our goal with Talon is to use “the Phoenix way” as much as possible, while providing an out of box solution that automatically renders your CURD interface with intelligent defaults. We’ve removed the DSL and replaced it with overridable API calls.

One further goal is that we are expanding the focus beyond an admin interface, and creating something we believe can be used for both the front end and backend admin interface.

We will your feedback and/or contributions to Talon. Drop by and open/comment on an existing issue. Borrow some ideas if you wish. We welcome your contributions if you choose to do so.

[sobolevn]

Thanks, @smpallen99 !
Looking forward contributing to your project.

[tmbb]

I’ve been playing with a different approach (not even close to release), inspired by torch. I’ts mostly vapourware at this point.

This post assumes familiarity with VueJS templates and single file components.

My approach is based on the following “axioms”:

1. Drab is awesome but (1) doesn’t encapsulate complexity very well and (2) the latency is too high for animations and stuff like that
2. Javascript is not that bad
3. If you’re using Javascript at all, better go all the way and use a Javascript framework (in my case, VueJS)
4. The ability to customize the UI is more important than flawless out of the box experience

If you disagree with the above axioms you won’t like it.

The idea is to move a lot of functionality to the client. Instead of having generators that generate HTML, I’ll have generators that generate VueJS singl-file components built from custom VusJS components I’ve build or reused from other libraries.

VueJS components are great because they can encapsulate a lot of complexity (validation, conversion between the UI state and the model state, etc) into a single line of code. Also, VueJS single-file components can use template languages other than HTML. I’m currently using Pug, which is basically a well defined and standardized DSL to write HTML using indentation instead of matched begin and end tags. Example:

form(class="form-horizontal")
  div(class="box box-info")
    div(class="box-header with-border")
      label(class="box-title") ABC properties
    div(class="box-body")
      TextInput(v-model="model.a" label="A Property")
      TextInput(v-model="model.b" label="B Property")
      TextInput(v-model="model.c" label="C Property")
  div(class="box box-info")
    div(class="box-header with-border")
      label(class="box-title") XYZ Properties
    div(class="box-body")
      TextInput(v-model="model.x" label="X Property")
      TextInput(v-model="model.y" label="Y Property")
      TextInput(v-model="model.z" label="Z Property")

Which renders as:

This is an example of insets: Separating fields into logical groups. How will the generators generate such a thing? The easy answer is that they won’t. They’ll generate something like this:

div(class="box box-info")
  div(class="box-body")
    form(class="form-horizontal")
      TextInput(v-model="model.a" label="A Property")
      TextInput(v-model="model.b" label="B Property")
      TextInput(v-model="model.c" label="C Property")
      TextInput(v-model="model.x" label="X Property")
      TextInput(v-model="model.y" label="Y Property")
      TextInput(v-model="model.z" label="Z Property")

which renders as something like this:

It’s up to the user to customize it so that it looks like the one above. Why won’t the generator generate the version above? Because it’s really complex to specify that in the command line, and it’s better to have a simple skeleton you can customize later. PUG is already a good DSL; you can program in Pug directly. The goal is to allow for most customizations by writing Pug and not requiring the users to change the Javascript.

The good thing about using VueJS components is that the components manage the state intependently of the UI. You can customize the UI however you want, and the Backend will receive the same JSON. Want to split the fields into insets? Great. Want to add freeform text in the middle of the form? It still works.

All of the above is easy to do with HTML instead of VueJS components, but VueJS components really shine if you need client-side validation or nested forms:

Note the toolbar above the nested forms. The buttons allow you to create a new element above or below the current one, move the element above or below or even delete the element. These changes can be reflected in real time, or only after the form is saved. The state will map naturally to JSON which you can send to the backend, and which ecto handles almost out of the box. Most of the time, you can actually write VueJS components as if you were writing pure HTML anyway.

Because VueJS components are drawn automatically based on the state in real time, you can listen to phoenix channels to update the state, and the UI will update accordingly when the object being edited is updates by another client. You just have to edit your contexts to generate the events (I plan on writing new context generators so that this is done automatically). With a little more work, components can be written that allow collaborative text editing by two or more people at the same time (I’ve written such components, but I haven’t turned them into VueJS components yet).

I’ve been playing with the client more than with the backend (I expect ecto to do most of the heavy lifting for me xD), but it seems like a viable approach.

This achitecture plays on Phoenix’s strengths to create a very dynamic Admin interface that reflects the DB state in real time. All of these ideas can be used for UI elements independently of the Admin interface, of course.

So, what are the disadvantages of this approach?

1. Initial render times might (will) be slower than pure HTML. Performance after the initial render might be better if I can avoid page reloads.
2. You’ll have to write some Javascript. The generators will try to write all Javascript for you, but it won’t be always possible.
3. Most of the time, the extra complexity VueJS brings to the table will not be worth it. After all, HTML is already pretty great for forms, and Phoenix comes with good tools to handle forms automatically… On the other hand, it will handle complex cases cleanly (remember Phoenix can’t generate a nested association in the form to which you can add or delete elements)
4. You’ll have to use webpack instead of brunch. Support for VueJS single file components is very bad in brunch.
5. You’ll have an Elixir program that generates a VueJS single-file component, which is compiled by node into javascript. These might be too many abstraction layers for some users.

[OvermindDL1]

@tmbb

Drab is awesome but (1) doesn’t encapsulate complexity very well and (2) the latency is too high for animations and stuff like that

What about complexity? Or are you waiting for it to support a commander per named element?

Latency depends on the connection, you of course would want to bounce javascript over in many cases or set a CSS style to do animations and such. You do not want to animate via javascript regardless of your framework, those are battery killers, only animate via CSS (and certain CSS at that).

@tmbb

Javascript is not that bad

Drab does make it awesomely easy to send javascript over. ^.^

@tmbb

If you’re using Javascript at all, better go all the way and use a Javascript framework (in my case, VueJS)

If others are intended to use it then it really needs to fit into 2 requirements (at least for my uses):

1. It better not require some weird front-end compilation stupidity like webpack.
2. It better work sans javascript too with a fallback (like all my drab pages work without javascript, just reloads the page a lot).

This is what I really really hate about react and such, no fallback. My projects #1 use unpoly, with a bit of drab around. I require javascript driven things for stuff that are only clientside, which a rendered GUI is not.

@tmbb

The ability to customize the UI is more important than flawless out of the box experience

Heh, that is where javascript frameworks are even worse than server-side rendering. ^.^;

@tmbb

Which renders as:

How does it render in elinks?

/me probably uses elinks near as much as chrome

Not read the rest of it yet, busy at work… ^.^;

[tmbb]

@OvermindDL1

How does it render in elinks?

Lol, it doesn’t

@OvermindDL1

You do not want to animate via javascript regardless of your framework

I meant dynamic, UI, not real animation. I hate animation. Animate via javascript = use javascript to set a CSS class. I don’t even know how to animate anything using Javascript xD

[tmbb]

@OvermindDL1

It better work sans javascript too with a fallback (like all my drab pages work without javascript, just reloads the page a lot).

You’re clearly not the target user for this library then…

@OvermindDL1

What about complexity? Or are you waiting for it to support a commander per named element?

What I mean by complexity is that VueJS components make it trivial to write a component that turns a list of clicked checkboxes into a list of JSON objects, with effortless client-side validation: Say you need between 3 and 7 selected checkboxes. It’s easy to write a component that disables the unselected checkboxes once 7 are selected, and disables the selected ones when 3 are selected. Is this easy to do with Drab? On a first reading (and admittedly little exerimentation) this seems hard, but I might be wrong… On the other hand, while the UI might be more complex, the architecture is simpler in Drab (you probably need something like 2 files, while my VueJS approach needs 3-4 files).

I mean, Drab is the coolest thing ever (the only thing that I’ve seen that comes close is the Wt framework for C++, but I don’t program in C++). It would be awesome to have a Drab powered Admin interface, I just don’t know if I can make it work, while I know I can implement all of the features above (badly) with VueJS xD

[tmbb]

On further thought, by (ab)using the __using__ macro we can usually make an elixir module as “reusable” as we wish… It can probably define some clever functions inside __using__ that turn Drab events into a new state. I usually thing about turning events into state as a form of integration: the events are the derivative of the state in relation to time, and the event handlers perform the integration to get the state. This is how I think about UI code in general. If I can “integrate” events into the new state with very few lines of code, then Drab works. If I can’t then it doesn’t.

Since Elixir can perform spooky magic with the __using__ macro (or even with normal macros, of course), this should not be too hard…

Maybe I’ll look into it.

[OvermindDL1]

@tmbb

Since Elixir can perform spooky magic with the using macro (or even with normal macros, of course), this should not be too hard…

Not just __using__ but any macro yeah. ^.^

[tmbb]

Macros with var! are the spookiest of them all. Might be useful.

[gregvaughn]

@tmbb

Macros with var! are the spookiest of them all. Might be useful.

You clearly have a strong sense of hygiene

[tmbb]

I do have a strong sense of hygiene, but sometimes, in the wee hours of the morning, and no one is really watching, I do like to get dirty with var! and her dirty friends… It is so wrong, but it feels so right… Ah, the thrill!

[tmbb]

Inmight keep working very slowly on my VueJS idea… VueJS does require webpack (yuck!) and has no fallback for people without Javascript, but (1) VueJS components offer amazing composability and (2) server-to client push doesn’t seem to work without Javascript, so I think avoiding it might be a bit pointless if I want to integrate that feature.

Another thing I’ve been playing with is a non-js admin frontend… Why can’t the admin frontend be a Qt application written with PyQt, ehich communicates with Phoenix through channels? xD I’ve got some utilities that make PyQt apps with lots of forms a breeze to develop. I guess distributing updates is a pain with Desktop software, so this won’t get pst the planning stage.

[halostatue]

You can get VueJS working with brunch; vue-brunch has recently transferred ownership to someone who wants to maintain it. (Shipping a vue-based app with brunch.)

[tmbb]

but does it support importing npm modules inside .vue components?

[sulphur]

and did you tried admin-on-rest ? I started tu use it and it seems fine for the moment being(if of course you like react )

marmelab/admin-on-rest

[halostatue]

Yes. You do need to make sure your brunch-config.js and your package.json are properly configured, but we have several NPM modules we import.