Using ElixirLSP with Vagrant

I have been putting off using ElixirLSP for a while and figure it it is time to jump on the bandwagon. However I have not been able to find one thing. I am running my code inside a vagrant VM and I am not sure how to setup emacs to do that. Has anyone else done thing and how do you set it up?