That’s going to make matters extremely difficult - there really is no escaping JavaScript when developing web applications.
You can improve matters with Functional-Light JavaScript and JavaScript Allongé, the “Six” Edition after you get the basics down first (see also ECMAScript book).
As such there is “no real hurry” with Elm - it’s probably good enough to get a hold of the Elm in Action MEAP and simply work through each chapter as it is made available and reinforce what you have learned by referencing that material on elm-lang.org.
Focus some time on Elixir before you get into Phoenix (which in my opinion is somewhat macro heavy which can be a bit disorienting for neophytes).
