brainlid

brainlid

Blog Post: Can Phoenix Safely use the Zip Module?

Elixir has a built-in Zip library that comes with OTP. This post explores how to use the zip module and asks the important question: “Is this safe to use with user provided zips?” We explore two different types of zip-based attacks and see what we learn from it.

Most Liked

akash-akya

akash-akya

Nice post! zip format is full of dark corners like this.

Another fun fact. Because of the way zip spec is, which is unlike other formats, zip file index is at the end. You can just append new index to delete/rename files! Without actually touching file data. Basically, you can just take the existing index, update it, and append at the end of the file, without actually modifying any of the file data. And according to spec, this is a valid file. And there are many tools that abuse this fact for various “features”. Also, that’s how most of the “recover deleted files” feature works.

IMO one way to reduce the issue surface is to avoid touching the file system altogether if you don’t need to. You can just keep them on memory if they are small, or you can stream them if you can’t fit them on memory, or extract only the file you need.

If you are automating something which requires reading from zip, then likely you already know the filenames, or structure, so you can just fetch the ones you need.

There are mainly two approach to stream files from a zip:

Since both approach use streams, and don’t write/extract anything to the file system, both are not susceptible to path traversal attack

  • read the zip from the beginning to the end. Example: zstream
    Since you can’t really reverse stream (or move to an arbitrary position), it is not susceptible to zip bomb. The code does not even need to be aware of zip bomb!

    Cons:

    • because this approach relay on local file header which is optional according to spec, you won’t be able to parse all the zip files (especially the ones which are written in a streaming way)
    • since the approach does not strictly follow the spec, you might see some of the ghost files, or duplicate files or, the files which does not show when you open zip in a GUI etc.
    • can not selectively extract a single file efficiently
  • use seek & read API - Example: unzip (full disclosure, I am the author)
    approach might be susceptible to zip-bomb, if the library is not explicitly handling it (In case of unzip I am handling it). Since this approach follows the spec, it should work with all types of zip files.

Where Next?

Popular in Blog Posts Top

lawik
One of the Erlang ecosystem’s spiciest nerd snipes are hot code updates. Because it can do it. In ways that almost no other runtime can.
New
brainlid
Building a LiveView powered chat app is easier than ever when using Streams! Sophie DeBenedetto shows us how in this article. She createa...
New
AstonJ
Elixir Blog Posts How to use this section You can post links to your blog posts either in one of the Official Blog Posts threads, or, vi...
New
JKWA
I didn’t have room for this in my book, Advanced Functional Programming with Elixir , but I still thought it was worth sharing on my blog.
New
ellispritchard
Somebody was asking about Erlang OS signal handling on Slack; I’ve posted up my learnings: https://medium.com/@ellispritchard/graceful-s...
New
brainlid
Elixir has a built-in Zip library that comes with OTP. This post explores how to use the zip module and asks the important question: “Is ...
New
lawik
We all have varying degrees of exposure to Big Tech. Some of it seems fine, stable and can be relied on. Some of it feels like shifting s...
New
danschultzer
How to set up the Content Security Policy header with Phoenix LiveView and support inline styles and scripts with CSP nonces.
New
RudManusachi
Hi there! Recently I was playing around with extracting and updating data in the DB and for fun challenged myself to try to implement a ...
New
AstonJ
Update: How to use the blogs section You can post in one of the Official Blog Posts threads (like this one), or, via Devtalk and a new t...
New

Other popular topics Top

danschultzer
None of the current solutions worked well for me, so I went ahead and built a user management system from scratch. This project took far...
548 29377 241
New
sorentwo
Hello! tl;dr Announcing Oban, an Ecto based job processing library with a focus on reliability and historical observability. After spen...
985 42920 311
New
TunkShif
This post is an instruction guide to help you setup your Neovim for Elixir development from scratch. It includes general information on h...
274 41539 114
New
minhajuddin
I have seen a lot of code which picks the first element from a list using Enum.at(0) instead of List.first. Is there a reason why people ...
New
Fl4m3Ph03n1x
About me? ( if you have nothing better to do than reading about some random guy in the internet :stuck_out_tongue: ) Hello all, this is ...
New
AngeloChecked
What learn first? Rust or Elixir Hi Elixir community! I’m here because i want learn a new language. I’m a junior developer and mainly i ...
New
grych
Hi folks, Few months ago I have announced the proof-of-concept of the library to manipulate the browsers DOM objects directly from Elixi...
639 52341 488
New
Qqwy
Original source of discussion: This topic on the Pragmatic Programmers’ Functional Web Development with Elixir, OTP, and Phoenix forum. ...
New
rms.mrcs
Hi, I need to transform a list of numbers into a map where the keys are the indexes and the values are the original values of the list. ...
New
WestKeys
Currently suffering from paralysis by [HTTP client] analysis. This is rather unusual in Elixirland as there tends to be consensus on the ...
New

We're in Beta

About us Mission Statement