How to setup a Phoenix 1.3 app with Gulp/ npm scripts

So it seems the brunch project is on life support, so I’m considering using gulp (which seems like a nice alternative) to build my phoenix apps moving forward (see thread Potentially removing brunch from the Phoenix new template generator).

So how do you setup a phoenix 1.3 app with gulp having the same features as brunch?

Why not Webpack?

3 Likes

Gulp seems simpler to configure

  • Brunch and Webpack are module bundlers. As such they have an opinion about the build process but by and large they are configured rather than scripted. In Webpack’s case things get a bit blurry because configurations are often assembled in a scripted manner.
  • Gulp is a task runner and as such sees itself as taking the place of npm scripts. So to bundle modules it still has to use a bundler plugin such as Browserify which often can be done simply in npm itself. Gulp has more advanced facilities than npm scripts for coordinating build tasks - but from what I’ve seen it isn’t really configured as much as it is being scripted by writing Gulp tasks. One issue is that often in cases where Gulp seems simpler it really isn’t necessary (Why I Left Gulp and Grunt for npm Scripts (2016-Jan-17)).

Modern JavaScript for Dinosaurs gives an overview of using npm scripts and Webpack (see also How JavaScript bundlers work).

Aside: Here is an example where simple shell scripts are used to build a simple react application with browserify, babel-cli, watch, and uglify, cssshrink, jest-cli, babel-jest, react-addons-test-utils, eslint, eslint-plugin-react, eslint-plugin-babel, and flow-bin.

Now I don’t know whether that is the case here but I think often people try to choose a build tool before they fully understand the build process. I’m not saying that npm scripts are the be all and end all but they are likely the best way to get acquainted with the build process and as such can be pushed quite a ways. Once the build process is better understood and one’s needs are much clearer it should be easier to pick the right tool, be it webpack (SurviveJS - webpack), parcel, rollup.js or whatever.

5 Likes

I never used gulp so I don’t really know, but I recently changed brunch to webpack and it was sort of straightforward following some tutorials (scss & assets the most annoying stuff - I can share my webpack.config if you want).

Thanks @peerreynders, @mnussbaumer.

I kinda took on @peerreynders advice and gave a try on npm scripts. I created a new phoenix project with brunch and then removed the brunch-config.js file and brunch dependencies (seems to be the most used approach to change the build tool).

Everything seems to be working, except for the js part. The built app.js file (on priv/static) on a “normal” phoenix project has a bunch of code coming from require.js, module-definitions.js, phoenix.js, phoenix_html.js, socket.js. Right now my built app.js has only the “import “phoenix_html”” statement (which is not recognized) and the socket.js code (Here is the repo of the phoenix app).

How should I deal with the javascript files, so that i can have all the default functionality that comes with phoenix?

@antalvarenda You aren’t processing your javascript through babel(?) or so, probably forgot that step? :slight_smile:

EDIT: Yep, you are only running it through a minimizer but aren’t processing the ES6 to ES5 via whatever it was called. :slight_smile:

I’m more on the side of learning rather than forgetting things :sweat_smile:
Changed the build pipeline to run babel instead of uglify (i don’t want it minified on dev mode anyway).
Same result (though not minified anymore) :confused:.
app.js

1 Like

Try this

$ npm i -D babel-cli babel-env-preset

add build_test/assets/.babelrc

{
  "presets": [
    ["env", {
      "targets": {
        "browsers": ["last 2 Chrome versions"]
      }
    }]
  ]
}

add these to scripts of the package.json:

"babel": "babel js/ -d build/",
"browserify": "browserify ./build/socket.js ./build/app.js ../deps/phoenix_html/priv/static/phoenix_html.js ../deps/phoenix/priv/static/phoenix.js -o ../priv/static/js/app.js",
"build:js": "npm run lint && npm run babel && npm run browserify",

For source maps you have to work a little bit harder:

// assets/build.js
const path = require('path');
const fs = require('fs');
const babelify = require('babelify');
const exorcist = require('exorcist');
const browserify = require('browserify');

const babelrc = {
  "presets": [
    ["env", {
      "targets": {
        "browsers": ["last 2 Chrome versions"]
      }
    }]
  ]
};

const config = {
  ignore: ['deps/**','node_modules/**']
};

const resolve = segment => path.resolve(__dirname, segment);

const target = fs.createWriteStream(resolve('../priv/static/js/app.js'));
const mapPath = resolve('../priv/static/js/app.js.map');

browserify({debug: true}) // turn on source maps
  .add(resolve('./js/app.js'))
  .add(resolve('./js/socket.js'))
  .add(resolve('../deps/phoenix_html/priv/static/phoenix_html.js'))
  .add(resolve('../deps/phoenix/priv/static/phoenix.js'))
  .transform(babelify.configure(config), babelrc)
  .bundle()
  .pipe(exorcist(mapPath))
  .pipe(target);

$ node build.js runs browserify, babelify and exorcist (to move the inline source map into a separate file).

1 Like

@peerreynders but at this point, when you introduce browserify it’s basically similar to using scripts&webpack right?

(this is great stuff though, thanks for sharing)

My impression was that one of the primary reasons for using Brunch in Phoenix was to have support for (ES) modules right out of the box as it allows for cleaner frontend dependency management. With that in mind you are going to need a bundler of some description.

It’s my personal preference to use ECMAScript rather than CommonJS/Node.js style JavaScript which bundlers manage so that makes Babel a requirement.

When sticking to a “npm script based” approach it’s important to start out just using the CLI features, with a file based approach.

There is usually a point where some features aren’t directly accessible via the CLI and you have to use the JS API to get the desired results - it’s probably at this point that some people start reaching for grunt/gulp. However I’ve found that the JS API documentation for many npm packages (plug-ins) is often lacking. For example with babelify it became necessary to dive into the index.js to find out how it used babel-core to determine that the ignore configuration actually supported “A glob, regex, or mixed array of both” rather than just a single regex.

This is probably the point where it becomes a bit easier to accept dealing with the Webpack even through it may seem incredibly complex. Browserify’s API is rather restrictive. Other packages like browserify-directory can help but at this point it starts to feel like you are creating some sort of Frankenstein. Webpack has a well developed ecosystem, so there are plenty of loaders and there is usually little reason to also reach for grunt/gulp (supporting npm scripts are usually enough). Again with Webpack it is important that you start small (SurviveJS). But now having experienced all the bits and pieces that go into a JS module toolchain it should be a bit easier to understand what Webpack is trying to accomplish.

The transition to Webpack isn’t mandatory. Having a basic understanding of the aspects of the JS module tool chain it should be much easier to transition to any other tool like Rollup.js or parcel. I think the biggest hurdle for beginners with Brunch was that they had no concept of what it was doing (because they had never used JS modules before) so they had no basis from which to understand the configuration process.

Trying to use something as restricted as Browserify exposes a lot of the moving pieces which helps to build your conceptual model of the build process. That is why I’m a bit sceptical with “zero config” claims like the one’s I hear for parcel. These things are usually “zero config” until they’re not. But because so far you’ve done “zero config” you have no concept or idea where to start when it comes time to do “some config”.

2 Likes

Wow great stuff, thanks so much. Yeah I was killing myself to find a way to output the sourcemap file, should have come here sooner! I’ll try it out.

1 Like

Let us know if the source maps work (don’t have Postgres installed right now, so couldn’t fire up your project). It may be simpler to remove exorcist and leave the source map in line.

Yeah inline works fine…But at some point it should be in its own file, right?

Separating them is useful for production. When they are separate the browser should be able to detect the source map but there is aways room for hiccups.

The last line of app.js is

//# sourceMappingURL=app.js.map

sourceMappingURL and sourceURL syntax changed

Yes - probably learning the “lower level” of calling directly the build tools from the CLI (& respective configs) would have been (and probably for me still warrants trying to learn it) a better option that jumping immediately from brunch to webpack - I will check this further thanks.

(having said that, webpack & its loaders do so much heavy lifting I don’t even feel like going to the gym anymore - sourcemaps, separate bundles, css/scss dependencies/requirements from JS, fonts, image compression, dev/prod separation in config… I’m hoping that the next iteration will also output cups of coffee on demand)

1 Like

Yep this seems to work fine! Thanks.
One thing - here:

const config = {
  ignore: ['deps/**','node_modules/**']
};

Shouldn’t it be '../deps/**' ?

There is a problem - but that’s not the solution. In your build.js throw in

const minimatch = require('minimatch');

other packages (i.e. babel-core) already use it so it should already be there; then throw in

console.log(
  minimatch(
    resolve('./node_modules/lodash/lodash.js'),
    'node_modules/**'
  )
);

console.log(
  minimatch(
    resolve('./node_modules/lodash/lodash.js'),
    '**/node_modules/**'
  )
);

console.log(
  minimatch(
    resolve('../deps/phoenix_html/priv/static/phoenix_html.js'),
    '../deps/**'
  )
);

console.log(
  minimatch(
    resolve('./js/socket.js'),
    '**/deps/**'
    )
);

console.log(
  minimatch(
    resolve('../deps/phoenix_html/priv/static/phoenix_html.js'),
    '**/deps/**'
  )
);

and see what happens when you $ node build.js

So it should have been ['**/deps/**','**/node_modules/**']

see also Globtester

Update This still works:

    browserify({debug: true}) // turn on source maps
      .add(resolve('./js/app.js'))
//      .add(resolve('./js/socket.js'))
//      .add(resolve('../deps/phoenix_html/priv/static/phoenix_html.js'))
//      .add(resolve('../deps/phoenix/priv/static/phoenix.js'))
     .transform(babelify.configure(config), babelrc)
     .bundle()
  .pipe(exorcist(mapPath))
  .pipe(target);

So it seems that Browserify is smart enough to follow the breadcrumbs through the various package.json files to find phoenix_html.js and phoenix.js. Now when you comment out socket.js the bundle does get smaller but as soon as you remove the comment in the original assets/js/app.js for

import socket from "./socket";

the bundle is back to its original size. That means that Browserify automatically loads imports for you (assuming they use the correct path in the first place). So just specifying only the entry file (assets/js/app.js) should be enough.

1 Like

false
true
false
false
true
:+1:

Why that config, though?

The ignore list excludes files from transpilation and Babel passes them on as is. npm packages for the browser are already supposed to be transpiled and if you look at phoenix.js and phoenix_html.js you’ll notice that they are already transpiled. So there is no need to process them - and excluding them reduces that transpilation load.

Yesterday Babel was complaining about something in the Phoenix JS but that seems to have gone away now.

Note that Chrome may mark the imports as errors in the source map view because browsers are only now starting to support <script type="module"> and are breaking things along the way. For example ECMAScript strongly prefers that the .js extension is omitted while it has to be specified in the browser.

1 Like