How to configure inets (httpc) default profile at startup to connect via a proxy?

Questions / Help
, ,
1

Has anyone configured inets default profile (to use a proxy) on startup?

I am using the new relic elixir agent dependency and :inets is started by that (as part of extra applications in that project). My primary aim is to have the :httpc client used by the new relic agent, connect via a proxy (that exists in my company’s production environment) when connecting to the new relic servers.

I tried following the example in the inets user guide which said to add this to the configuration: [{inets, [{services, [{httpc, PropertyList}]}]}] I translated that to

config :inets, services: [ httpc: [ proxy: {{'http://proxy.com', 1234}, []} ] ]

But that doesnt work.

iex>:httpc.get_options(:all) 
{:ok, [ proxy: {:undefined, []}, https_proxy: {:undefined, []}, pipeline_timeout: 0, max_pipeline_length: 2,...

Please do correct me if I’m approaching this the wrong way, folks.

2

I see the same result as you. I did have success with:

iex> Application.start(:inets)
:ok
iex> :httpc.set_options([{:proxy, {{'http://proxy.com', 1234},[]}}])
:ok
iex> :httpc.get_options(:all)
{:ok,
 [
   proxy: {{~c"http://proxy.com", 1234}, []},
   https_proxy: {:undefined, []},
   pipeline_timeout: 0,
   max_pipeline_length: 2,
   max_keep_alive_length: 5,
   keep_alive_timeout: 120000,
   max_sessions: 2,
   cookies: :disabled,
   verbose: false,
   ipfamily: :inet,
   ip: :default,
   port: :default,
   socket_opts: [],
   unix_socket: :undefined
 ]}

So maybe add this to your supervision tree rather than config.exs (even if you drive the content from config.exs or env vars) and perhaps that will get you going?

3

@kip im unable to figure out where to enter the module with this startup code.

:inets is part of the new relic agent’s supervision tree. So there is the supervision tree in the phoenix app. :inets is started under extra applications by the new relic agent. Where should I enter the module adds the proxy config?

How do I figure out the startup order? Is it the pheonix app followed by the dependency apps?

4

I think you should be able to put it in your application.ex file. Something like the foilowing (which is just editing the generated application.ex for an application called ppp):

  def start(_type, _args) do
    children = [
      # Start the Telemetry supervisor
      PppWeb.Telemetry,
      # Start the Ecto repository
      Ppp.Repo,
      # Start the PubSub system
      {Phoenix.PubSub, name: Ppp.PubSub},
      # Start Finch
      {Finch, name: Ppp.Finch},
      # Start the Endpoint (http/https)
      PppWeb.Endpoint
      # Start a worker by calling: Ppp.Worker.start_link(arg)
      # {Ppp.Worker, arg}
    ]

    # Add these lines
    Application.start(:inets)
    :ok = :httpc.set_options([{:proxy, {{'http://proxy.com', 1234},[]}}])

    # See https://hexdocs.pm/elixir/Supervisor.html
    # for other strategies and supported options
    opts = [strategy: :one_for_one, name: Ppp.Supervisor]
    Supervisor.start_link(children, opts)
  end

I’m sure this is not the most sophisticated solution but it might be a path forward.

2 Likes
5

@kip Thank you for this idea. I tried it out. There is still a problem here that I’m unable to solve. I suspect that when the new relic agent tries to connect at startup, the proxy config is still not applied at that time. because the error that I get is:

{"application":"pixy","env":"prod","level":"ERROR","message":"new_relic_agent - preconnect: (failed_request)
 {:failed_connect, [{:to_address, {'collector.newrelic.com', 443}}, {:inet, [:inet], :timeout}]}","mfa":"{New
Relic.Logger, :handle_cast, 2}","service":"pixy","timestamp":"2023-09-04T08:11:08.166Z"}

But when I log into the remote shell and try to connect using :httpc.request("collector.newrelic.com") I get a different error

{:error,
 {:failed_connect,
  [
    {:to_address, {'http://company-proxy.com', 3128}},
    {:inet, [:inet], :nxdomain}
  ]}}

So at this point I can be sure that the request is atleast going through the proxy, while the earlier log isn’t very clear. I also confirmed this when I ran :httpc.get_options(:all). Now my core problem here is probably unrelated, but looking at the difference in the error messages, I want to investigate the order in which these config is applied.

The folder structure of my app is like this

pixy
|_ deps
|   |_new_relic_elixir_agent
|       |_ mix.exs (with extra applications: :inets)
|       |_ lib > new_relic > application.ex
|_ config > releases.exs
|_ lib > pixy > application.ex

I’m finding it a bit hard to understand how can the new relic agent app even start even though it need not be added to the supervision tree in pixy > application.ex or the extra applications in pixy’s mix.exs

Can you explain what is happening here, or maybe point me to some resources that explain this order of startup between dependencies and the core app ?

6

A couple of thoughts (not tested):

  1. You can tell OTP not to start new_relic_agent automatically by configuring runtime: false. For example:
defp deps do
  [
    {:new_relic_agent, "~> 1.0", runtime: false}
  ]
end

Then you can start it up whenever you want. For example, using my previous code:

Application.start(:inets)
:ok = :httpc.set_options([{:proxy, {{'http://proxy.com', 1234},[]}}])
Application.start(:new_relic_agent)

I also see that you can configure :httpc_request_options so its possible that you could simply do the following (untested): Seems :proxy is not a valid http_option.

2 Likes