Hello, @wmnnd is Belt working without pain with umbrella’s structure in Phoenix 1.3 rc1?
odyright
wmnnd
Hey @odyright, thank you for your question!
Belt does not in any way rely on Phoenix or the project structure created by Phoenix’s generators; it is its own independent OTP application.
This means you can simply declare Belt a dependency and add it to your extra_applications array in one (or multiple) of your own umbrella applications.
The Getting Started guide explains how the installation works in more detail.
2
wmnnd
##New version 0.1.6
For this announcement I have joined the two small releases 0.1.5 and 0.1.6 that bring a couple of small bug fixes and improvements.
Changelog
- Only define SFTP and S3 providers if dependencies are available
- Improve compatibility with non-AWS S3 providers by normalizing headers
- Correctly assemble non-presigned S3 URLs
- Fix incompatible API in Belt.Ecto
- Ignore
:enoenterrors in Filesystem Provider implementation ofBelt.delete
0
wmnnd
#New version 0.1.8
Since DBLSQD is finally up and running, there is at least one instance of Belt out in the wild now 
More features will come as DBLSQD and Belt evolve. For now, these are the changes of the most recent release:
Changelog
- Improvement: When storing files on SFTP, hashes are now calculated locally. Retrieving hashes when none were requested is now a noop.
- Improvement: Better compatibility with SFTP servers with small maximum package sizes. Thanks to the obscure and undocumented (yet public) functions
:ssh_sftp.send_window/2and:ssh_sftp.recv_window/2, package size is now determined dynamically. - Fixed bug causing infinite loops when listing files on some SFTP servers
- Fixed connecting to SFTP servers with
verify_hostset tofalse
##roadmap
- Make jobs more flexible
- Explore the possibility of using ETS directly or ConCache as a backend for jobs
Feedback and suggestions are welcome as always!
4
wmnnd
@abitdodgy Thank you for your question. Direct S3 uploads are currently not supported because they would not allow for certain features such as calculating a file’s hashes and file size.
Could you explain what your use-case would be for Belt with direct uploads some more?
0
NobbZ
A relayed transfer costs bandwith and therefore money.
- When I have to receive an upload to my personal server first, I have to pay for that transfer.
- Then I have to push the payload over to S3, which is another transfer I have to pay
- Last but not least, I have to pay whatever amazon charges me for using their service.
I’d like to elimate the first 2 costs from this list.
(I’m not a user of Belt, nor of S3, but thats the first thing that comes to my mind when thinking even briefly about it)
I do assume though, that I can ask S3 for at least the filesize after the fact. If it is a proper service it will also have some way to retrieve a hasvalue for uploads to easily verify them before pushing the same file again.
0
abitdodgy
@wmnnd Precisely at what @NobbZ said, but I want to add a couple of more notes. Beyond cost, the issue becomes more obvious when using a PaaS like Heroku, where Dyno sizes are small and can’t handle large sizes. By passing the app removes the need for heavy processing power and memory consumption. In Ruby, I used Refile, which is incredibly easy and pleasant to work with… except when it comes to medium and large files, where out of memory exceptions happen all the time, sometimes even for small files, like 1mb.
Granted, Elixir’s architecture makes a big difference, but I never tested this. My assumption is that PaaS services aren’t suited for dealing with files.
1
wmnnd
@abitdodgy If you are using a recent version of Plug/Phoenix, you shouldn’t get out-of-memory problems anymore since the bug concerning this was fixed last month. The only limiting factor for uploads then is the disk space you have available for temporary usage which I realize could end up being a problem.
Are you already using Belt? If so, what would be a reasonable API for adding direct upload links that can work across different providers?
2