It depends a little bit on your use case.

If there is only a single public key you might need, i.e. it is a global value for the whole application, then you could use Application.put_env/Application.get_env.

If however you want to handle multiple values, like one per user in an application that might serve multiple users at the same time, then you probably want to use a caching library instead.
There are some great caching libraries which you could use on HexPM, the most well-known one probably being cachex, though there are many.

If you want to build something yourself, you might start with an Agent to store some common state in. (If speed ever becomes a problem, you could move on to :ets instead). But a caching library probably suffices.