Questions & Help>Questions

HTTPoison request with authentication

#httpoison#elixir-httpoison
undeadko

undeadko

Hello everyone,

I am trying to access a web page with some pictures in it, so that I can download the pictures and then do some fun things with them. Very much like this post here with the smoothies https://medium.com/@Julien_Corb/web-scraping-with-elixir-using-httpoison-and-floki-26ebaa03b076 .

The difference is, the access is gated behind a login screen → username and password. I am fairly new to all of this, so my rookie mistake was to just send a request to the link, which naturally returns to me a %HTTPoison.Response{} structure. The body of the response being the logging screen.

For some reason the documentation of Httpoison is lacking… or rather not helpful, as it is just “This is an existing function. It takes parameters.” There is no example for what I am trying to do, which I thought would be a very common thing. Should I go to Hackney to find proper documentation?

So my question is, Can I do this all in one go? - Send a request with the appropriate logging parameters and directly receiving the body / the images straight away. How? How do I know, where do I look for further help?

Much appreciated if somebody could help with this. Thank you in advance!

1 comment
#httpoison#elixir-httpoison
249 1

Showing Posts 1 to 1

D4no0

D4no0

First of all you should inspect the login page and find the actual POST request that is done to the server when you input the username and password and press submit, you should be able to identify it in browser debug window → network tab, that is the request you want to use to login with a http client like HTTPoison.

The second part is a little bit more tricky, you need to save the session cookie that you get after successful login and use it in subsequent requests to protected routes, you can take some inspiration from here: Help with HTTPoison POST - #7 by Aetherus

PS: That might not be the best example as its filled with macro magic, what you are interested in are the following lines:

# Options sent to hackney for setting the request cookies
options = [hackney: [cookie: cookies]]]

# Example usage
headers = []
HTTPoison.get("my_url", headers, options)

# Extracting cookies from the response
defp cookies(%HTTPoison.Response{} = resp) do
    resp.headers
    |> Enum.filter(fn
      {"Set-Cookie", _} -> true
      _ -> false
    end)
    |> Enum.map(fn{_, cookie} -> cookie end)
  end
3
Post #1
— All posts loaded —

Where Next? Top

View thread on forum (has 1 reply!)
httpoison
elixir-httpoison
Home Questions & Help>Questions
3 244 1
Last post
REPLY VIEW THREAD

Trending in Questions Top

stjefim
Questions & Help>Questions

Durable Workflow Execution (temporal alternative for elixir)
Hello! Suppose you are building workflow (order / task / payment) processing system with the following requirements: Each workflow con...
/oban#workflow#durable-execution
22 4198 10
New
jonnycharles
Questions & Help>Questions

Looking for a Prawn-Like PDF Generation Library in Elixir
I’m in search of an Elixir library that offers PDF generation capabilities similar to Ruby’s Prawn. While there have been discussions abo...
#learning-elixir
117 2980 30
New
spammy
Questions & Help>Questions

Simple/light way to self host microapps and services?
I’m looking to build a personal workflow to quickly deploy web applications written in elixir/phoenix, for local consumption (ie not on t...
#deployment
35 1251 21
New
Blokh
Questions & Help>Questions

Huge CSV processing
Hey guys, I’ve got a huge CSV ( around 10 GB ) that needs to be processed hourly Do you guys have any suggestions what is the best prac...
/phoenix
8 1145 5
New
dli
Questions & Help>Questions

How to integrate Hologram with existing LiveView app?
Before I dive in myself, did anyone successfully sprinkle Hologram into their existing LiveView app? Looking for hints regarding: Addi...
/hologram#liveview
11 799 6
New
roeland
Questions & Help>Questions

Elixir-google-api deprecated, reason and alternatives?
Kia ora, We have been using elixir-google-api to connect to Google Drive. However, with the updates to Tesla due to CVEs this is now bro...
#googleapi
9 573 3
New
bottlenecked
Questions & Help>Questions

How to do proper post-release steps
Hi all, I wanted to ask how the community is dealing with post-release steps. Today we have Ecto migrations, which make sure that the db...
/phoenix#deployment
13 581 9
New

Other Trending Topics Top

JesseHerrick
News>Announcing

Dexter - A fast, full-featured Elixir LSP optimized for large codebases
Hey, I’m Jesse and I’m the main contributor behind Dexter, a full-featured, lightning-fast Elixir LSP optimized for large codebases. It s...
#library#lsp#dexter
177 5417 56
New
jimsynz
News>Announcing

Beam Bots - Resilient Robotics on the BEAM
Beam Bots (or just BB for short) is a framework for building fault-tolerant robotics applications in Elixir using familiar OTP patterns. ...
/beam-bots#library#robotics#bb
119 2398 27
New
mcass19
News>Announcing

ExRatatui - Elixir bindings for the Rust ratatui terminal UI library
ExRatatui lets you cook up rich terminal UIs in Elixir, powered by Rust’s ratatui via Rustler NIFs. Build interactive terminal applicatio...
#library#rust#terminal#tui
58 2534 15
New
netoum
News>Announcing

Corex - Accessible and unstyled UI Phoenix components
Corex is an accessible, unstyled UI component library for Phoenix that integrates Zag.js state machines using Vanilla JavaScript and Live...
/phoenix#library#ui#liveview#hooks#accessibility#component#tableau
61 2202 19
New
Damirados
News>Announcing

Emerge & Solve - a GUI framework for Elixir
Hello everyone. After busy few months I am happy to announce v0.1.0 of Emerge & Solve. They are GUI (Emerge) and State management (S...
/nerves#library#frameworks#gui#native#state
59 2012 22
New
ausimian
News>Announcing

Emily - A new MLX-based backend for Nx
Emily is an Elixir library that runs Nx computations on Apple’s MLX. Install it as the default Nx backend and Nx, defn, Axon, Nx.Serving,...
#library#libraries#emily
63 2036 19
New
Back to top

Questions & Help>Questions

Questions Questions ❯

Latest on Elixir Forum

Elixir Forum ❯

Sponsor Spotlight

Pragmatic Bookshelf

Practical resources that improve the lives of professional developers.

Our Sponsors

Dashbit

Build Elixir applications with speed and confidence.

EEF

Supporting innovation across the BEAM ecosystem.

Coders51

We build reliable cloud platforms for business-critical systems.

AppSignal

Catch errors, track performance, monitor hosts and more.

Honeybadger

Error tracking, logs, uptime, and Just Enough APM. Built for developers.

Pragmatic Bookshelf

Practical resources that improve the lives of professional developers.

Manning

Develop your skills with books, videos, and courses.

ElixirCasts

Producing high quality Elixir screencasts since 2017.

Software Mansion

The team behind Membrane, Popcorn, LiveDebugger. Available for hire.

Pragmatic Studio

Courses that'll move you from confusion to "Aha, now I get it!"

View all our sponsors ❯

Categories:

Sub Categories:

Forums

Popular Tags

Our Sponsors

Dashbit

Build Elixir applications with speed and confidence.

EEF

Supporting innovation across the BEAM ecosystem.

Coders51

We build reliable cloud platforms for business-critical systems.

AppSignal

Catch errors, track performance, monitor hosts and more.

Honeybadger

Error tracking, logs, uptime, and Just Enough APM. Built for developers.

Pragmatic Bookshelf

Practical resources that improve the lives of professional developers.

Manning

Develop your skills with books, videos, and courses.

ElixirCasts

Producing high quality Elixir screencasts since 2017.

Software Mansion

The team behind Membrane, Popcorn, LiveDebugger. Available for hire.

Pragmatic Studio

Courses that'll move you from confusion to "Aha, now I get it!"

View all our sponsors ❯

We're in Beta

About us Mission Statement

Options

Thread Display Mode




Thread Preview

Skip Thread Previews