I suspect the biggest hurdle to universally employing webpack as the asset pipeline is for projects relying heavily on EEx - which at the moment still seems to be a common dependency for Phoenix projects (as far as I’m aware there isn’t anything like an eex-template-loader).
@tmbb: the speed is great, and honestly I’m just building out a webpack branch on a project that I’m working on and have yet to test the speed in comparison. Indeed the docs aren’t great, I haven’t really experienced any flat-out failures though.
Also, maybe it’s just the very plugin-oriented architecture that brunch takes, and just the number of packages available compared to webpack, but I find myself often needing to roll my own plugins, or lean on micro-plugins with no tests available elsewhere in the community to acheive more advanced build tasks.
I’m running a modernizr js builder in my build cycle, and even though I was able to use the preload() hook, I needed the before-brunch plugin to run an npm command out of the box. It seems like it shouldn’t have been so difficult to achieve what seemed like a trivial addition to the overall build cycle.
There were a couple of such instances where I just looked for webpack plugins in edge cases where I happened to be connecting odd tools in the js or css worlds together, and there were in almost all instances multiple plugins available for the same purpose.
If the goal of phoenix is to offer a more complete, robust, easily-maintainable end-to-end platform out of the box, I don’t see how it will be able to stick with brunch long-term, unless the mentality is that the front end is supposed to be a more extensible, hands-on experience for the developer.
Either way, it seems that the two paths are inherently going to be brunch or webpack, and I don’t see a compelling reason to offer default generators for buliding an app with one and not the other.
Additionally, though I think npm is overall a much better build tool than yarn, I could imagine people wanting to use yarn out of the box. Because there isn’t too much documentation on using yarn with webpack with phoenix (indeed there’s only one example app on the web of an end-to-end web app with both Phoenix 1.3 and Webpack 3).
Maybe it’s a seperate project, but it may be worth considering generators for just one or two more of these best-of-breed build tools to make things easier on newcomers, and show them that there are multiple options for front-end management in Phoenix.
@peerreynders: maybe we’re talking about different things here, but to my knowledge regardless of how the front-end build is being handled, EEx rendering would still happen from the Phoenix.Html hex package and phoenix_html npm packages respectively. Does that sound right?
1 Like
What does EEx have to do with Webpack, Brunch or any other frontend solution? EEx templates are just functions that generate strings right?
1 Like
Maybe I simply haven’t gone down the Webpack rabbit hole deep enough. But to put my current understanding in simple terms:
-
Webpack’s use-case is geared completely towards JavaScript rendered pages - either in the browser or through JavaScript server side rendering. It wants to bundle everything, your JS, CSS, images, etc - and to get it’s dependency graph it looks at all of it.
-
EEx is Elixir server side templating as it was done in the pre-Node.js days and Brunch largely supports the needs of that style of (JQuery-era) web development. You manually decide what goes where and you decide how simple or fancy your setup is.
As far as I can tell EEx (Elixir) and Webpack (JavaScript) don’t easily mix as they represent totally different styles of web development. Now it may be possible to use EEx templates with Webpack but that would have to be entirely on Webpack’s terms and I doubt that there would be much value that EEx templating features could bring to the table.
People using Webpack with Phoenix typically are using React or Vue.js and therefore see no need for EEx features. But there still seems to be a significant audience for using Phoenix and server side templating with EEx - and I have yet to discover how Webpack could be coerced to support that style of web development - I suspect that for that scenario using Brunch is simply easier.
Ultimately Webpack may be too specialized (opinionated?) to become a universal default.
Hey @peerreynders: I guess I’m still kind of confused about your use case. It seems to me that EEx is just as mutually exclusive to Webpack as it is to Brunch. You could just as easily use Webpack just to bundle your client-side assets as you could use brunch to do that. To my understanding, In neither case would you need to obstruct Phoenix’s ability to render eex or haml, etc. in order to also serve assets out of webpack.
To clarify, to my understanding the fact that webpack is able to render server-side javascript does not necessarily suggest that it has to render all server-side assets. Does that make sense? This may be an incorrect presumption, but I’ve already begun integrating Webpack into an existing Phoenix application in lieu of Brunch and don’t see any signs that Webpack will require me to obstruct or alter my server-side template rendering.
For the record I’m using a combination of html rendered through HAML, EEx (where I’ve actually found some bugs in the phoenix_haml and/or calliope parsers), and a substantial set of front-end react templates with some animation support from both jquery and vanilla javascript.
Either way, I’m going to be working on this more early next week so I’ll report back with my findings on getting all of this running through webpack, vs its current setup, which is a brunch build.
You ‘could’ take EEX’s template output and run it through ElixirScript or so to get javascript. Would not be as efficient as in Elixir (at all), but eh…
1 Like
That’s probably the case. Without explicit setup webpack won’t even touch anything html and for vuejs I know for sure it can quite happily work in conjunction with server side rendered markup. Even things like webpacks hot reloading are in the end just: load your javascript file from the webpack started server instead of the phoenix one. That’s easy to do with eex. While I fully understand the decision to use brunch in phoenix webpack could handle it’s tasks just as well.
The difficulty I see with running webpack as default is rather that I’ve yet to see a webpack setup which does work out of the box in such a general/baseline fashion as brunch (the closest I know would be laravel-mix). With brunch you just add a file to the right folder and it will be bundled. That’s not highly controlled, but damn simple. For webpack there are so many features and ways to do things, that it’s probably quite hard to generalise things without creating a ton of additional work – just for the assets pipeline.
Sounds like a JavaScript-based templating engine with an appropriate xxx-template-loader or equivalent that enables Webpack to rifle through it at compile-time for analysis.
While I fully understand the decision to use brunch in phoenix webpack could handle it’s tasks just as well.
To me Webpack feels like a much tighter straight-jacket than Brunch - but one would be a fool not to use it for React or Vue.js …
With brunch you just add a file to the right folder and it will be bundled. That’s not highly controlled, but damn simple.
… which is pretty much in line with the style of web development that was pervasive when (non-JavaScript) server-side templating (rendering) was the norm.
I find that Webpack has a very particular way it wants you to organize your assets and whenever you try to deviate from it’s norm it becomes a near “fight to death” with the configuration. Also unless the asset is referenced somewhere in the loaded (mostly JavaScript and CSS) files it rifles through - as far as it’s concerned it doesn’t (need to) exist. And all of a sudden matters that were trivial before (e.g. simply make sure that all of the images needed are in fact in the directory) require elaborate JavaScript-based work arounds.
It feels like all of the drawbacks of poor static typing without an equivalent return; the markup could still reference something not being referenced in the JavaScript (… so don’t let the markup reference anything).
Ultimately one can be left with the impression that as far as Webpack is concerned, it should be the one creating the markup (HtmlWebpackPlugin).
Webpack does not need to touch the EEx bits of your server rendered app. You can use it simply to compile ES6 for example, or the way I use it, to compile .vue files. You just specify you js main file and import your other js stuff there, webpack will compile it and throw it on the target folder, say priv/assets/app.js and from there you simply add a script tag on you EEx template.
1 Like
It’s also about how well the technologies mesh together (as far as I’m concerned, not very well).
Last May I performed a little experiment, as a learning experience: generate a stock 1.2 Phoenix project - and simply replace Brunch with Webpack 2 with the original stock behaviour and look restored - which resulted in lots of WTF moments.
- Default filenames:
By default the filename of the resulting file is the MD5 hash of the file’s contents with the original extension of the required resource.
…
You can configure a custom filename template for your file using the query parametername. For instance, to copy a file from yourcontextdirectory into the output directory retaining the full directory structure, you might use?name=[path][name].[ext].
Keeping the original filename requires a custom action 
- so to keep those
<%= static_path(@conn, "/res/resource.rs") %>
references working in the EEx templates, you need to take active control of the file names in the Webpack configuration. On the flip side if you want to take advantage of the hashed file names there will need to be another step in the toolchain that updates those file references in the EEx templates.
-
Previously this was enough in
js/app.js:
.import “phoenix_html”;
Now I need all this
import appStyles from '../css/app.css';
import phoenixStyles from '../css/phoenix.css';
import phoenixImg from '../images/phoenix.png';
import faviconIco from '../favicon.ico';
import robotTxt from '../robots.txt';
import "phoenix_html";
for the sole purpose of communicating to Webpack that these files are “part of the package” - some of them are already sitting there, ready to go in the source directory and are being referenced in the markup where they are relevant - and just because I may have used the correct spelling in the JavaScript doesn’t mean I did the same in the markup.
- Webpack complains that there are fonts missing
So apparently css/app.css is a Bootstrap v3.3.5 file which references the glyphicons-halflings-regular font. But the stock installation doesn’t even provide it - making it necessary to grab it somewhere else and retro-fit it in order to shut Webpack up.
Now the stock brunch-config.js looks something like this:
exports.config = {
sourceMaps: false,
production: true,
modules: {
definition: false,
// The wrapper for browsers in a way that:
//
// 1. Phoenix.Socket, Phoenix.Channel and so on are available
// 2. the exports variable does not leak
// 3. the Socket, Channel variables and so on do not leak
wrapper: function(path, code){
return "(function(exports){\n" + code + "\n})(typeof(exports) === \"undefined\" ? window.Phoenix = window.Phoenix || {} : exports);\n";
}
},
files: {
javascripts: {
joinTo: 'phoenix.js'
},
},
// Phoenix paths configuration
paths: {
// Which directories to watch
watched: ["web/static", "test/static"],
// Where to compile files to
public: "priv/static"
},
// Configure your plugins
plugins: {
babel: {
// Do not use ES6 compiler in vendor code
ignore: [/^(web\/static\/vendor)/]
}
}
};
Nothing too terribly terrifying if something has to be changed. Now to emulate the effect of the Brunch configuration with Webpack 2, I ended up with this unwieldy looking thing:
const path = require("path");
const ExtractTextPlugin = require("extract-text-webpack-plugin");
const extractCSS = new ExtractTextPlugin({ filename: 'css/[name].css', allChunks: true });
module.exports = {
context: path.resolve(__dirname, "./web/static"),
entry: {
app: "./js/app.js"
},
output: {
filename: "js/[name].js",
path: path.resolve(__dirname, "./priv/static")
},
resolve: {
modules: ["node_modules"],
extensions: [".js", ".json", ".css"],
alias: {
"phoenix": path.resolve(__dirname, "./deps/phoenix/priv/static/phoenix.js"),
"phoenix_html": path.resolve(__dirname, "./deps/phoenix_html/priv/static/phoenix_html.js")
}
},
module: {
rules: [
{ // for css/app.css
test: /\.css$/,
use: extractCSS.extract({ fallback: "style-loader", use: "css-loader" })
},
{ // for image/phoenix.png
test: /\.(jpe?g|png|gif|svg|ico|txt)$/i,
use: 'file-loader?name=[path][name].[ext]'
},
{ // Needed for bootstrap 3 fonts - retrofitted into web/static/font for phoenix.css
test: /\.(woff|woff2|ttf|svg|eot)/,
loader: 'url-loader',
options: {
limit: 10000
}
}
]
},
plugins: [ extractCSS ]
}
(I think at this point I gave up trying to get the font file names in non-hashed forms into static/font).
Finally if the development setup requires that the Webpack dev server is installed and up -and-running we really haven’t reached a “Brunch equivalent state” yet where the pages are being served by the Phoenix Server rather than the Webpack dev server.
I found it amusing that while the vue-cli “webpack-simple” template uses the Webpack dev server, the fully fledged “webpack” template does not - it instead elects to build an express-based server around the Webpack dev middleware - I can only assume that somebody decided that the dev server wasn’t providing adequate presentation fidelity.
There are lots of good reasons to adopt Webpack if you are using React or Vue.js - you just have to commit to actually learn and adopt the Webpack way. But I have yet to see a convincing argument that it is worth tolerating the burden of maintaining a Webpack configuration for an EEx based project which just uses a tried-and-true “sprinkling” of JavaScript.
FYI: Apparently HMR and RHL are in fact “not the same thing”.
1 Like
Some days ago I was doing a simple example app for an experimental project of mine, using webpack 2 for the SPA and phoenix (1.2, but things should be the same for 1.3). And found it quite simple, at least for my needs, here’s some things you might be interested in:
The web SPA defines an npm start command on its package.json file, which gets started as a phoenix watcher and I used an elixir ReverseProxy to forward asset requests to webpack-dev-server, so you could have some eex views rendered by phoenix and let webpack serve the js-bundles.
On a production build, I’d create a mix task that just calls webpack for production bundles and places them into the priv directory to be served by phoenix. Hope the links above can be of any help.
1 Like
Hey @peerreynders: thanks for the taking the time to elaborate on your end-to-end experience here. It sounds ot me like it really comes down to your usecase and what you’re trying to get out of Phoenix.
You seem to keep coming back to the value of Webpack when all you’re doing is rendering EEx templates through Phoenix on your front-end. I suppose that’s a completely valid claim if you don’t have any/much dynamic front-end and are really just rendering html to spit back out into a rendered response to the client.
There are lots of good reasons to adopt Webpack if you are using React or Vue.js - you just have to commit to actually learn and adopt the Webpack way
However that boundary, as you say, would be if you’re using React, or Vue, or just have a lot of JS happening on the front-end and need more tools available to help configure, build, and test the front-end as thoroughly as you want, and in a way that suits your tastes, using the tools you want to use. Brunch could obviously support pretty much anything you want it to if you extend it properly with plugins, but I guess there’s some pain threshold where it might feel like webpack is just intended to do more of the heavy lifting than brunch ever was.
But I have yet to see a convincing argument that it is worth tolerating the burden of maintaining a Webpack configuration for an EEx based project which just uses a tried-and-true “sprinkling” of JavaScript.
I’ve read the Thoughtbot article that you posted here, and it brings up good points if you’re a large distributed team, and you have a varied amount of experience from front-end and back-end developers working in collaboration. However I’d argue that the dynamism on the front-end provided by React outweighs the burden of having to extend your overall javascript suite and maintain as much front-end as back-end. That’s going to come down to a personal preference, but if you’re handy with javascript and react you can really be very productive in developing front-ends, and more than that even you can be allowed to think about your front-end implementation in more holistic ways, rather than sort of patching-on vanilla js or jQuery to add some effects post-hoc. I guess this is a bit of a digression, but it seems to hit some of the points in your post.
Either way, I’m trying not to make too many judgment calls here, just kind of thinking all of this out as we go back and forth. As somebody mid-webpack-implementation I don’t have working experience using webpack, but I’m familiar with the limitations I’ve experienced managing my front-end architecture in brunch. Maybe part of that was just learning curve with some of the more advanced uses of Brunch to setup my frontend. Ultimately it may not be easier at all to implement Webpack as you suggested (those config files are definitely substantial), but I wonder if with that additional configuration comes additional flexibility that may be handy for more advanced front-end setups.
Thanks again for your thoughtful replies here and for sharing your experiences with both build tools in question.
@vic - thanks a bunch for your input here. I’m sure these will prove to useful resources as I build out my webpack branch (haven’t scrapped Brunch yet, just exploring at the moment). Your thoughts here are greatly appreciated. I’ll check back in if I have any other comments or questions.
1 Like
That’s entirely it - and the stock front end that Phoenix spins up (short of specifying --no-html) is EEx (server side template) based - this is also the front end that the Phoenix Guides and the Book assume you have set up.
Now I may have expressed myself poorly in the earlier post. It was never my intention to imply “Webpack doesn’t work (well) with Phoenix” - quite the contrary; if you are using React or Vue.js I’d say, ditch Brunch ASAP and run, don’t walk towards Webpack because for better of for worse those communities have embraced it (just don’t expect it to be a cakewalk) otherwise you are just going to feel like a salmon swimming upstream trying to make do with something else like Brunch.
My response was to the point of Webpack potentially replacing Brunch as part of the Phoenix distribution some time in the future i.e.:
should probably be a candidate for being integrated by default in Phoenix 1.4+.
I simply don’t see that happening - I don’t know if you are familiar with this topic:
My takeaway from that topic:
- The Phoenix team wants to minimize the dependencies that the distribution has to the (ever changing) JavaScript ecosystem. There is just enough JavaScript to support channels and that is it. This is simply a pragmatic decision to minimize the opportunity cost of keeping up with the Joneses in JavaScript land - the Phoenix team wants to focus on Elixir and Phoenix and not get distracted by supporting everybody’s favourite JavaScript frontend and toolchain.
- Brunch was chosen because it was good enough to provide a relatively “low maintenance” solution for the stock EEx frontend which wouldn’t get in the way of the Phoenix newcomer trying to learn “just Phoenix” - i.e. get distracted with too many JavaScript-y (toolchain) things. It probably requires fewer interventions than npm scripts (personally I consider Webpack potentially “high maintenance”). This should not be construed as an endorsement by Phoenix as Brunch being the greatest asset build tool ever - far from it, as Phoenix goes through great lengths not to integrate tightly with Brunch (hence the
--no-brunchoption).
That is why I don’t see Webpack replacing Brunch in the Phoenix distribution.
So ultimately it is the developer’s own responsibility to chose and integrate the necessary/optimal asset build tools for their own particular use case. Phoenix’s responsibility is largely confined to not getting in the way.
Somehow that doesn’t stop the wishes/demands for the generators to set up projects with everybody’s favourite JS framework and build tools. Providing that level of support is ultimately up to the greater Phoenix community (not the Phoenix distribution - good documentation helps though) - and it’s often rendered in the form of template projects or “how to” blog post.
The template projects are great if you can just run with what is supplied - personally I prefer those nitty-gritty blog posts because they tend to give me a better starting point to build up the necessary knowledge to customize and troubleshoot my configuration.
2 Likes
So I can confirm that I’ve successfully swapped out Brunch with Webpack 3 on a Phoenix 1.3 project and everything is building fine. The build is actually about twice as fast as my build with Brunch, with the added benefit of being able to see the build time of each individual asset and build step, as well as the detailed size of all of my assets.
The config is definitely a bit more heavy-handed than the brunch config, but I actually like that it’s more expressive, and that it’s more obvious how to manage the rules for building various assets and which ‘loaders’ as they’re called in webpack are responsible for handling the various asset types.
I’m not sure whether I’ll jump ship and go with webpack just yet, but I might.
On another pass I might take @OvermindDL1’s advice and rewrite my entire build process to run entirely through npm. It was noted on another thread that this might not necessarily be cross-platform compatible, but I’m running Ubuntu 16.04 end-to-end so for me personally there wouldn’t be any environmental gotchas.
I may post all of my findings here though more likely I’ll write a blog post about it, as there’s a fair bit to write out, and I’d prefer to transcribe the whole process rather than just bits and pieces. That said, the webpacker demo app that I mentioned earlier is a very good template for getting Webpack 3 working with Phoenix 1.3, but there are a couple of errors specifically with react-hot-loader 1.x.x, and some clean-up that could be done with the plugin includes in the config file itself.
1 Like
@peerreynders: in short, I agree with you. After setting up Webpack in my project, it’s an 800-pound gorilla. Brunch makes sense as a default with Phoenix because of the philosophical alignment that you mentioned. Being unable to anticipate exactly what anybody wants to use Phoenix for doesn’t mean you should try to accommodate all possible solutions.
That said, I guess the gap that we’re experiencing, that I experienced here, is a thorough overview of these dynamics. I suppose this thread will (hopefully) serve to bridge it, but we likely need more writing, discussion, and documentation around various approaches for building front-ends on top of Phoenix for newbies. I’ll try to do my part by doing more write-ups about my experiences, but this is still a bit of a hole in the onboarding process for Phoenix.
1 Like
Update #12: I’ve merged my Webpack setup into master, no turning back now.
It is cross platform just fine if you use helpers like npm-run-cmd and others too, they are designed to make npm be the build system on any platform. ^.^
1 Like
I had some time this past week to play with Vue / Webpack / Phoenix / Bootstrap / PostCSS.
This is pretty rough but it’s a start. Still making my way through the Webpack documentation.
3 Likes
