Using undocumented %Postgrex.Error{} values

Questions / Help
1

Hi–

I’ve run into a Postgrex documentation question related to if my error parsing is legit enough to rely upon. To be clear, this is a “because I can, should I?” question… from a functional perspective everything currently works as desired.

The Scenario:

In the application I’m developing, I use a fair number of PostgreSQL functions, mostly in the form of INSTEAD OF trigger functions. These are chiefly aimed at “must never be violated/must always happen” data integrity rules which cannot be satisfied using simple database constraints. Really, the kind of functions and triggers aren’t very relevant to the question, but serve as context.

Unsurprisingly, When the PL/pgSQL functions find some non-conformance/error condition/problem, they will RAISE EXCEPTION halting any in-progress transaction and typically causing a Postgrex.Error exception to be raised in the application. All of this is good and desirable. I’ve written the PL/pgSQL exceptions to return my own SQL Error Codes (simplified example):

IF old.syst_defined AND new.internal_name != old.internal_name THEN
    RAISE EXCEPTION
        USING
            MESSAGE = 'The requested data update included changes to fields disallowed ' ||
                      'by the business rules of the API View.',
            ERRCODE = 'PM003'
END IF;

Where the PM003 value is the SQL Error Code for this exception.

In the Elixir application, if this exception gets thrown, I get something along the lines of (again, simplified): %Postgrex.Error{postgres: %{pg_code: "PM003"}}.

Great! I can work with that in the application by rescuing (or similar) the exception from Postgrex and return an error in a non-fatal way, assuming that the database exception shouldn’t be treated as fatal. But…

The Question:

When I look at the Postgrex documentation for Postgrex.Error.t(), or the exception generally (Postgrex.Error — Postgrex v0.19.3), I only see:

@type t() :: %Postgrex.Error{
  __exception__: true,
  connection_id: term(),
  message: term(),
  postgres: term(),
  query: term()
}

In which the postgres field is simply documented as term(). If I understand Elixir conventions correctly, “undocumented” in a library like this typically means internal implementation detail which shouldn’t be relied upon by client applications. Is this really the case here? I know of some instances of struct members being undocumented and really being considered private, but in this case I’m not sure that’s intended to be true in this case. I’m kinda hoping that deconstructing the database exception to this degree is just rare enough in Elixir applications that nobody thought it worthwhile to document rather than the absence of documentation being a signal that this should be considered an internal implementation detail.

So either I’m trying to get some sort of comfort that interrogating the returned postgres map is OK or that there are enough others doing similar parsing that changes there will be treated like a change to any other public function or struct for practical reasons (I expect the latter to be unlikely, to be fair).

Thanks!
Steve

1 Like
2

It’s hard to answer exactly not being the maintainer of the library, but it is probably fine.

I think with any library there is the risk that something changes in a way that semantic versioning doesn’t capture. If you rely on specific behaviour I would have tests that capture that and run them on each postgres upgrade. If they did fail raise an issue.

Looking at the source code the type definition is a standard boilerplate-y:

@type t :: %Postgrex.Error{}

You often see that for structs I don’t think it necessarily means the fields are to be considered private.

Similarly you can see that postgres field in use in the tests of the lib:

So something is relying on the :postgres key being in the Exception.

1 Like
3

FWIW, here’s where ecto_sql pattern-matches on values from that key of Postgrex.Error:

2 Likes
4

Thanks for the answers!

That’s actually documented in the Postgrex.Error.t() documentation I posted earlier. The postgres key is documented to exist, but it’s type is only referred to as term(). The thing that worries me is that I am using specific keys from the map that gets assigned to postgres, specifically pg_code.

I do agree that there is little reason for this to change and that it’s probably unlikely to change (I’m not rushing out to change my implementation, I think a small expense on my “risk budget”). But as a rule of thumb I try not to rely on internal implementation details which this looks like; pg_code as a name is a Postgrex convention and not directly from the database. PostgreSQL refers to this value as the SQLSTATE and allows it to be set, such as in exceptions, by setting a pseudo-variable called ERRCODE in the RAISE statement. In neither case does the PostgreSQL software refer to this as pg_code.

I’m actually worried less on this count. PostgreSQL has been returning SQLSTATE since at least version 7.4 (2003) and has allowed setting this value to custom codes via RAISE/ERRCODE since version 8.4 (2009) and my personal experience with it goes back to version 8.0. As functionality it’s been pretty stable for quite some time, but there is some risk in changes to the actual code assignments. While the codes that PostgreSQL uses haven’t really changed in some time now, there is no range of these codes officially designated as “user defined SQLSTATE values”. So while there are many, many unused values and ranges of values there’s nothing stopping my chosen range from being allocated by the database or some extension developers at some point. But… again… very low risk. And as you point out, I do have tests which exercise this, though they are more about my code in both the database and Elixir being created correctly than it is explicitly trying to test Postgrex/PostgreSQL behaviors.

Close… but not quite :slight_smile: . The code lines you’ve quoted match on the code key of the same map that I want to match on, but because I’m using custom SQLSTATE values, the code value will be nil so I have to stick with pg_code which is just the raw string. The code key value is actually a conversion of standard PostgreSQL SQLSTATE values mapped to atom name equivalents; this mapping happens at compile time using a text file with the standard error codes and their standard names. So my custom codes won’t be there.

However, the lines you selected were just a few lines away from another clause which does match on pg_code…

Admittedly, my concerns in this question are quite minor and more about wanting thoroughness and good practice. And maybe even to make a statement that getting the raw database SQLSTATE can have value even if it’s not the norm or typical.

Thanks again for the responses!

1 Like
5

but it’s type is only referred to as term()

Yes what I’m saying is it’s likely term() not because someone specifically added it as term to indicate the details can change, but more just that the exception was typed as @type t() :: %Postgrex.Error{} which is common for structs and leads to what you see.

I’d maybe feel different if it was written as:

@type t() :: %Postgrex.Error{
  postgres: term(),
  ...
}

I’m actually worried less on this count.

This was a typo on my part I meant postgrex, not postgres. But yes if you app relies on the behaviour likely the behaviour you want is being tested already

6

It seems to take the fields defined in the PG protocol here PostgreSQL: Documentation: 17: 53.8. Error and Notice Message Fields, where the RAISE operator forwards the ERRCODE as the SQLSTATE to the C section of the ErrorResponse message type. Generally it should be stable to match on this given its well defined by the protocol. Maybe try not to reuse preexisting error codes defined here PostgreSQL: Documentation: 17: Appendix A. PostgreSQL Error Codes else Postgrex could hook into those.