beno

beno

I am trying to get AWS authentication (for S3) working for a company AWS environment that relies heavily on IAM roles. This has to be one of the least transparent/most confusing things I have encountered for a long while, and I can’t find a fully configured working example.

Questions:

  • Am I correct in thinking the AccessDenied I currently get with standard KEY/SECRET is due to the different role I have when creating S3 buckets?
  • How do I retrieve my role_arn and how does this fit in to the ex_aws config?
  • What is my profile_name and where does this come from?
  • Is there a working example somewhere I can take a look at?

Hopefully someone has this working

Showing Posts 1 to 6

hauleth

hauleth

For me it worked OOtB, so it seems that IAM role assigned to that machine do not allow for accessing that S3 bucket.

beno

beno OP

Care to share how you set up your configuration? Where does the role_arn go?

hauleth

hauleth

Nowhere, role is assigned to the machine, not to application. So in the end anything that is running on that particular machine can access given S3 bucket.

benwilson512

benwilson512

Author of Craft GraphQL APIs in Elixir with Absinthe

I’m sorry to hear that you’re having trouble. ExAws is completely compatible with and thus to some extent assumes a familiarity with the general AWS authorization mechanics. As a general rule, if you can get the configuration working with the aws cli it will also work fine with ExAws.

Concretely, in order to have key based authentication you need an AWS user. This is not the same thing as a role. However you can have a user assume a role when taking on actions provided that the role has a trust association with the user that allows the user to assume the role:

[default]
output = json
region = us-east-1
role_arn = arn:aws:iam::1234567890:role/YourRoleHere
beno

beno OP

Thanks for chiming in, I have made some progress and got it working to an extent. Reasons for confusion:

  • how STS.assume_role works, I believe a name like fetch_role_credentials would be more apt, since as far as I can tell it is fully up to developers to use/cache/refresh/propagate these temporary credentials. assume_role suggests some stateful (shudder) change that applies to subsequent requests. I stumbled on this and found it very helpful, maybe link to it from the README?
  • differences between the release and master for ex_aws_sts, there is a awscli_auth_adapter: ExAws.STS.AuthCache.AssumeRoleCredentialsProvider in master and READMIE that is not in the 2.0 release.
  • incompatibility with tools like stavro/arc, which rely solely on the fixed, hard coded credentials from aws_ex config, so I don’t think IAM roles can be used with those tools (I have created an issue for arc)
  • diabolically opaque error messages from Amazon

Again, thanks for your input, any further help is equally appreciated.

nubunto

nubunto

I’m having issues with exactly that as well.

I have the following dependencies on my mix.exs:

      {:ex_aws, "~> 2.0"},
      {:ex_aws_sqs, "~> 2.0"},
      {:ex_aws_sts, "~> 2.0"},
      {:configparser_ex, "~> 2.0"}

I have configuration like this (per the ex_aws README):

config :ex_aws,
  debug_requests: true,
  region: "sa-east-1",
  access_key_id: [{:system, "AWS_ACCESS_KEY_ID"}, {:awscli, "foo-dev", 30}, :instance_role],
  secret_access_key: [{:system, "AWS_SECRET_ACCESS_KEY"}, {:awscli, "foo-dev", 30}, :instance_role],
  awscli_auth_adapter: ExAws.STS.AuthCache.AssumeRoleCredentialsAdapter,
  json_codec: Jason

In my ~/.aws/config file I have the foo-dev as a valid profile.

But when trying to, say, receive messages via SQS:

iex(2)> ExAws.SQS.receive_message("MY_QUEUE") |> ExAws.request
[debug] Request URL: "https://sqs.sa-east-1.amazonaws.com/MY_QUEUE"
[debug] Request HEADERS: [{"Authorization", "**REDACTED**"}, {"host", "sqs.sa-east-1.amazonaws.com"}, {"x-amz-date", "20190531T211829Z"}, {"content-type", "application/x-www-form-urlencoded"}]
[debug] Request BODY: "Action=ReceiveMessage"
[warn] ExAws: HTTP ERROR: {:closed, ""}

Turns out that the queue URL is actually wrong (in the AWS dashboard the URL is https://sqs.sa-east-1.amazonaws.com/ROLE_ACCOUNT_ID/MY_QUEUE)

But still getting the {:error, :closed} from hackney which I suspect should be a different issue.

Any help would be appreciated

— All posts loaded —

Where Next? Top

Trending in Questions Top

water
I’m aware of 2 such projects, are there others? git: GitHub - team-alembic/realworld: A fullstack Phoenix LiveView application with back...
New
katta
I having some trouble figuring out if I have set myself too strict of standards for my production server. Currently I can handle 75% of r...
New
peck
Hi everyone. My team and I have been working on a fairly modest app based around video streaming and chat, but we’ve landed a customer t...
New
bradley
I really like the adapter patterns that ecto, nebulex, waffle, etc. use and would love find something similar for a key management servic...
New
unaware8150
Hello folks! So at work, we are seeing some situations where we have to define some “fixed” strings that are used across the codebase in...
New
Alvinkariuki
How Can I Optimise Compile Time Dependencies I have been building an elixir application for about 2 years now. Many modules and files ha...
New
leoschweizer
Hi, we run different LLM models across a cluster, with each node loading a configurable subset. Model-specific queues let nodes consume ...
New

Other Trending Topics Top

budgie
A little off-topic, but I feel like people here have a good head on their shoulders. I used to be quite good at making software. Was luc...
New
KristerV
Hey. Is there anyone here who creates agents in their apps? Not talking about using agents, but creating them. I’m finding it pretty diff...
New
mudasobwa
I fully migrated to my own harness from Anthropic/Gemini and I think it’s time to share it. Welcome DSH, the DeepSeek Harness, fully writ...
New
jimsynz
Beam Bots (or just BB for short) is a framework for building fault-tolerant robotics applications in Elixir using familiar OTP patterns. ...
New
juhalehtonen
There has been a thread to discuss the Stack Overflow Developer Survey on this forum every year since 2018, so here’s yet another one for...
New
budgie
I love Elixir. It’s one of 2 programming languages I’ve ever fallen in love with. But I don’t use it anymore. Serverless was the promis...
New

We're in Beta

About us Mission Statement

Options

Thread Display Mode




Thread Preview

Skip Thread Previews