Best Approach for Building a Sideapp in Elixir: Generating Images and PDFs

Hey everyone,

I’ve been tasked with creating a sideapp from our main Rails monolith at my company. I’m considering using Elixir for this project, but I’m unsure about the best approach to take. The sideapp’s main purpose is to generate images, PDFs, and other file formats. I’m seeking guidance on how to structure the entire cycle effectively and whether this approach would be considered a ‘microservice’.

My current plan is to utilize Exq to tap into the Sidekiq Redis server and process any tasks that come in through there. Albeit, not sure if that’s a good approach either. I’m uncertain about where to handle the database updates: on the Rails side or within the Elixir side. Let’s say someone wants to generate a PDF. I enqueue the process in Rails, Elixir picks it up, generates the file, and uploads it. But what should happen next? Should I create another job to pass the file back to Rails using Shrine for saving, or should I use and handle all the validation and persistence within Elixir?

We’re using shrine on the rails side with it’s specific format hence the hesitation to perform the persistence on the Elixir side.

Thank you in advance

2 Likes

Suggest to evaluate the following popular paradigms

  1. Shared DB task queue with Postgres

  2. Message queue (AWS SNS or custom message broker)

  3. Webhook request & response

  4. Synchronous HTTP/HTTPS request & response if performance is good

Do NOT touch the Redis instance used by Sidekiq, it is quite bad practice

5 Likes

could you be a little more specific. What wiill be the content?

It’ll be invoice building with a bunch of merge fields :slight_smile:

OK I did exactly that already (not in Elixir). Can’t say too much about the architecture, but I can sure help you produce high quality (layout-wise) invoices - if you are not already settled with this part.

1 Like

Now I’m curious…

Hey Sebb, can you tell me more? I haven’t settled in on anything. It’s all more of a proof of concept but I am definitely down to hear more!

There are many ways to create a PDF. For an invoice HTML to PDF is the easiest (you do not want or need docbook, latex, XSL-FO, …).

For HTML to PDF there are in general two ways to it:

  1. not using CSS3 paged media module. You just create a heex template for the invoice and run it through something like https://wkhtmltopdf.org or GitHub - bitcrowd/chromic_pdf: Convenient HTML to PDF/A rendering library for Elixir based on Chrome & Ghostscript (this is easy but you can’t do some things like count pages, but could very well be enough for simple invoices)
  2. using CSS3 paged media module. This gives CSS the power to produce high quality content for print. The “lessons” section here print-css.rocks - PrintCSS CSS Paged Media tutorial and information gives a good overview of the features, for example
  • the ability to count pages, know the amount of pages in a document
  • remove content from the body and put it in the page margins (running content, like current chapter for a book)
  • orphans/widows (do not have one single line of a para alone on a page)
  • break-control (no not break directly after a heading)
  • create TOC, create doc-entries with leaders (chaptername .......... page 7)
  • repeat table-headers when a table is continued on a new page (or add a subtotal at the page-break)

I would use (2) if you expect your invoices to have multiple pages* or if you plan to create more complex documents for print. Beware, that you may have to use a paid tool, see the tools section of print-css-rocks. But for an easy document like an invoice, some free tools should be able to do it. (browsers do not support CSS3 paged media)

*) maybe there are some tricks to get the page-count without CSS3-paged-media pages()-counter, so you could use (1) for that, but I’m not familiar with this tooling.

8 Likes

I’ve used https://weasyprint.org/ successfully in the past, and it’s free.

8 Likes

I went with embedded latex compiler on a project, however those were some fancy 200 page reports. Generally I find latex much more superior compared to html to pdf, however it is quite painful to set up for the first time and does take quite some time and resources to compile the documents.

I’m not a latex expert, but when it comes to more advanced designs, what can beat CSS?

See these samples

can latex do that?
What can latex do I can’t do with CSS3 paged media (there are some things, but minor).

But most important: what is easier than heex+tailwind*->PDF?

*) tailwind will most likely not work with free tools, Prince is fighting, but it works. @typography plugin has to be set to legacy mode, but then its really nice for chunks of static text as you can just do: markdown -> earmark -> prose

@Morzaram I just see that there is a nice example for an invoice here: https://www.pdfreactor.com/product/samples/invoice/invoice.pdf

2 Likes

There’s also typst, a new typesetting system that is open source as well and looks very promising. You could use its cli or create a NIF using rustler.

4 Likes

that looks really cool, but more a replacement for latex as it seems.

Using CSS you can achieve the same results and also every other layout you could think about. For example you are fairly restricted what’s in the header and footer and it seems like you can’t put anything in the x-margins.

This is not a problem for the use-case, but if your use case differs you are better off with CSS. But: you wont have such nice tooling.

Very nice project!

I just wrapped up a project with an old client designing a proposal template. Template is html/css and Is used to generate a project specific proposal. Their CRM merges in the fields using liquid, then sends to DocRaptor for rendering.

It used to be arduous to get the design right because browser != pdf render styling.

So last week I built a little build system.

Html + tailwind. Build script processes tailwind, attach the tailwind output into a style tag, then hydrates with liquid, resulting in a few files:

  • single html file with style tag that I then hydrate with my own object that mocks what the CRM would do, I send this to docRaptor using test api key. I use Skim pdf app which has file-change reloading, so 5secs after I hit build i Can see the changes in my pdf editor.
  • I can also open that html file to do some browser debugging with dev tools
  • outputs the production-ready template with the {{variables}} in place

Anyway, used to take me 10 steps and a few minutes to manually test the layout, trial and error, repeat. Now I can see my changes in 5 secs. Plus, I was able to go from 2000+ lines of irrational css classes and 1000 lines of html to about 700 lines total with tailwind utility classes.

Figured some of you in this thread might find that interesting!

I definitely had fun building it, had been an idea of mine for years. Plus cut time to develop by prob 80%, while increasing my dev experience by an order of magnitude.

P.s then they wanted to create a diff variation where just one page was slightly different. Couldn’t dynamically do that with liquid because the right variable wasn’t available in the CRM, so I just took the static but variable parts and made em into variables, then I hydrate those into two separate files with unique objects for each. They get their two versions while I only need to make changes in a single file to tweak.

Plus, of course the liquid engine allows for for loops and if statements inside like Heex, which is nifty.

3 Likes

sounds interesting, I’m also working on a PDF-publishing tool, maybe I’ll open source this if someone would be interested in contributing. Also using docraptor but heex instead of liquid.

Can you show your code? (I can’t yet, because it still has our production templates in it)