Phoenix Product Codex: Develop and deploy a REST API for Product Data Management with Elixir and Phoenix (self-published/Leanpub)

It’s done well so far, with great feedback, despite only being available through Leanpub for now, and only an estimated 28% complete. We’ll see! :grin:

Hello there!

It’s been a while since the last release of Phoenix Product Codex, and for good reason: I just published a new release with two new chapters spanning 72 pages.

Chapter 6 is about modeling the families table, but doesn’t stop there. We also make it possible to accept a :category_name/:category_code and/or :brand_name in the JSON body of a POST to the /api/families route, besides only :id values of the associated category and brand records. Moreover, we implement association fields across all modeled schemas, so that we can flexibly get a family’s supplier, a supplier’s families, a category’s suppliers, etc.–eventually, also by adding more nested routes to all top-level routes.

Chapter 7 is a break from database modeling, and deals with implementing the improvements of the previous chapter across all types of records. We implement a “flexible identifier” feature for GETting records by their unique identifiers other than their :id, so that you can now do (for example). GET /api/categories/Rotary%20Valves (by the category :name) or GET /api/categories/161 (by the category :code). Most importantly in this chapter, we evaluate the associated fields that we should Repo.preload/2 by default for each type of entity, and even more importantly, we make JSON responses consistent across all entity types by centralizing the *JSON modules’ data/1 function within a new ResponseData module and using the defdelegate/2 macro from the *JSON modules to the new ResponseData.data/1 function. Therein, we use recursion and Ecto’s schema reflection capabilities to automatically populate the response map with non-nil association values. It works out nicely.

Many thanks to @belgoros for the first testimonial and his kind words:

An amazing and completely unusual approach to presenting the material of the book, as if you are looking for a solution to the problem with the author and go through all stages of searching for a solution with him. This is exactly what is happening in practice and, in my opinion, this is a huge advantage of this book compared to other books or courses.

With the summer already underway, things are getting relatively quieter and more predictable in my other businesses, so expect to see more frequent updates during July and August.

With the next release, the book will break the barrier of 200 pages. Though the number of pages is a misleading indicator of progress (due to the amounts of code snippets), I am marking the book as 28% complete in terms of scope (not pages). We still have a lot of ground to cover, but the data bedrock of the Products Elixir application are almost complete, and due to be completed in Chapter 8 (variants and kits/bundles in the Catalog context) and Chapter 9 (descriptions in a new Content module, and tags and other metadata for sales-item status, availability, discontinuation, etc. in the Taxonomy module).

Though it’s tentative for now, I foresee that we’ll deal with business logic for issuing unique and lexicographically (i.e. also more typo-resistant) item codes for variants in Chapter 10. After that chapter we’ll be ready to import real-world data from the ISATEK product portfolio into our database (and also implement an exporting feature into CSV/ODS/XLS/XLSX), and we can then move on to implementing features like caching, pagination/filtering/sorting/search and HATEOAS, API rate limiting, webhooks for event notifications, API documentation with OpenAPI, audit logging, and monitoring with the Phoenix LiveDashboard, before we finally learn how to deploy our app to production.

A big thank you to the 49 early adopters who have purchased Phoenix Product Codex at such an early state. Enjoy the two new chapters!

Cheers from Athens,
Isaak
isaak@overbring.com

2 Likes

Awesome. Thanks :slight_smile:

1 Like

Hi Isaak, I bought this book recently. However i found out that Viex doesn’t fetch any tin status anymore (I assume API changed or only offer paid service). What do you recommend to follow along your book.

1 Like

Hi @noargs, it works just fine here, albeit with a Greek TIN:

iex(1)> Viex.lookup("EL802586446")
...%Viex.Response{
  valid: true,
  company: "OVERBRING ΜΟΝΟΠΡΟΣΩΠΗ ΙΔΙΩΤΙΚΗ ΚΕΦΑΛΑΙΟΥΧΙΚΗ ΕΤΑΙΡΕΙΑ||OVERBRING LABS",
  address: "ΛΕΩΦΟΡΟΣ ΑΝΔΡΕΑ ΣΥΓΓΡΟΥ 110       11741 - ΑΘΗΝΑ"
}

Note that EU VIES doesn’t work correctly for some countries, most notably Germany, due to national data privacy laws. However, this is already addressed in the book w.r.t. testing.

Each country (should) have its own national commercial registry anyway that offers an API with richer (and better) data. For example, Greece has the GSIS RgWsPublic2 SOAP web service (for which I wrote VatchexGreece) and the ΓΕΜΗ Open Data API that one day I’ll write an Elixir client for.

In case you are interested in data for businesses in countries with crappy VIES info, there are paid services around: Google Search. I found this one that has a free tier with 100 requests that should be plenty for running through the book: Start using the API | vatlayer API. You’ll need to make GET requests with Req, and the shape of the returned data will be different, but it should be trivial to use. No affiliation or endorsement on my end, and I have never used it.

In any case, you could also mock Viex until you find a solution, or simply avoid it altogether and enter data manually. EU VIES it is not central to the book’s quest, even though it is a useful part of the app, when its data are available for a country.

Hope this helps!

Just to add it to the list - the creator of req also built a tiny wrapper GitHub - wojtekmach/vies: VIES VAT number check library using Req.

2 Likes

Very nice! And so much simpler to use the REST API that returns JSON instead of using the SOAP API that returns XML! Vies on-the-Web - European Commission

Thanks for the detailed explanation. I did try your tin but it came out false along with others. However i was only interested to learn building Api with elixir. So i think i would carry on without going much into tin. :grinning_face:

Interesting! Maybe it’s geo-blocked?

If thats the case they shouldn’t have blocked it if someone access from Europe. We have VAT here

Hello Phoenix and PDM fans,

After an Elixir-packed summer to “sharpen the functional-programming saw” for the upcoming chapters of Phoenix Product Codex, I just released Chapter 8 on Leanpub!

In this one, we almost complete the modeling of the database with the new table for product variants (variants) and its Variant schema module with validation functions. All that’s left for completing the database ERD is a consequence of what we come up with in this chapter.

We explore three distinct approaches to modeling product variants and bundles (kits):

  1. a simple, battle-tested method,
  2. a wastefully redundant one, and
  3. one (that we pick) that ends up making use of our relational database.

An analysis of real-world scenarios and practical considerations informs our decision, such as “breaking bulk”, equipment breakage/shrinkage, minimum order quantities, and the need to track both component variants and bundles without losing hierarchical or pricing context.

With this approach, we make the need for the separate items table of the original ERD concept redundant, as we’ll be dynamically generating item codes from category and variant codes.

So, when we return to modeling bundles/kits in Chapter 10 after the upcoming Chapter 9’s side-quest, we’ll be using a kit_compositions table to define bundle recipes relationally. This will enable nested bundles (bundles including bundles that might themselves include bundles, etc.) while ensuring data integrity through custom Ecto changeset validations to prevent self-references and circular dependencies.

Beyond that, we extend the flexible-identifier feature to variants, and also come up with a way to avoid redundant information in the returned JSON with nested associations.

Chapter 9 (not released today yet) is a side-quest; we need to find a way to generate unique, typo-resistant variant codes within product categories that doesn’t succumb to combinatorial explosion–especially if we want to extend this to typo-resistance across all item codes and not just within a product category. We’ll be seeing how a brute-force approach of “rolling the dice” is slow, unreliable and doesn’t scale, and so we’ll be developing a solution with a Markov Chain.

Already in Chapter 8 we evaluate the performance of two different approaches of testing uniqueness with MapSet.intersect/2 and MapSet.member?/2 and see how these scale very differently.

Chapter 9 is progressing nicely, so expect to see an update within this week.

Happy coding,
Isaak

There’s another read who mentioned similar issues with the EU VIES service. Given that I cannot reproduce them (it still works fine here), I’ll modify this part of the book.