Friday, February 22, 2019

AWS Account Alias as part of security

Good AWS practices require that AWS Organizations is used to separate the testing from production. It means that there is multiple AWS accounts to manage. That can be real nightmare where you can accidentally do bad things for the production.

To reduce the risks for accidents I've started to use account aliases to describe which account I'm currently managing. But how to name the accounts? There should be some standard way to name the account. They should clearly tell if the account is dev, testing or production account.

My organization is Bad Boys of Quality. Shortly it's BBoQ. That's good prefix for my accounts. Then dev, test and prod are good labels to describe the state of the environment. The final part should be the descriptive word for the account.

First - and most important account - for organization is master account. It's unique. I've decided that it's production. I've named it to bboq-prod-master. At the shared credentials I've got prod-master-admin profile to administer this account.

At the shared credentials I'm following the same naming convention as account aliases. This way I can easily change my account with awsume.

To set the account alias I would use AWS CLI. I don’t want to mess up the role history of my browser. Before doing anything else I’ve set the role to my shared credentials. I usually use ~/.aws/config for this. So now we are creating alias for bboq-dev-website. First I have to define the shared credential.


1
2
3
4
[profile dev-website-admin]
role_arn = arn:aws:iam::1234566543321:role/Admin
source_profile = otherprofile
mfa_serial = arn:aws:iam::1234566543324:mfa/myaccountt


To set the account alias with AWS CLI the line is:
aws iam create-account-alias --account-alias bboq-dev-website --profile dev-website-admin

One cool side is that when you assume the role at AWS web console, you can use the account alias. So instead of cryptic Admin @ 123456654321 you are Admin @ bboq-dev-website. This reduces the amount on confusions. It again underlines what kind of account is I'm managing. Destruction at dev account shouldn't be bad, but destruction at production is.

So why the account alias is part of security and safety? The more clearly you see things, more easily you notice if something is wrong, and less mistakes you make. I can tell you that before I started to use account alises at naming, I got almost totally lost what I was doing. Juggling with account ids is nearly impossible when you have than 2 accounts. And we have… almost 10 already and part of the teams doesn’t have them yet.

So there is six accounts: development, test and production website and three support accounts. Accounts are named bboq-dev-website, bboq-test-website and bboq-prod-website. Then there’s several support accounts like bboq-prod-accounts (all IAM users are here), bboq-prod-master, bboq-prod-compliance (all logs are going to this one).

Next post will present some ideas how to enhance readability and safety with Terraform and account aliases.

Monday, October 29, 2018

Secure CI deployment pipeline at AWS pt 1: Theory

Jenkins is security nightmare when you want to use it at deployment to production. If the Jenkins node does the deployment, the node has to have all access rights which are needed for deployment. So it might be related to infrastructure, IAM policies, access to AWS secret manager etc. Anyone who has modification access to the Jenkins job, has also full access to all of these. This might be even GDPR related risk, so building the secure deployment needs some thinking.

I'm expecting that every company has multiple accounts at their AWS Organization. The CI pipeline and other DevOps related systems are at the own AWS account. The production and test environments have own accounts. If we want Jenkins node to do the deployment, we'd have to give quite big access rights to production environment for it. That violates the least privilege approach.

So let's start to think about building blogs. AWS Lambda is nice way to run small functions, which can be triggered quite easily. So maybe that can solve part of the deployment process. AWS SNS is good way to start the Lambda. Now if the Jenkins node sends SNS message, which starts the Lambda which starts the deployment process the security is much better. But problem is that Lambda can't execute external applications like kubectl, terraform or some other fancy deployment tool. ECS is really nice for that. If it's executed at Fargate cluster the costs are minimal. But that needs also the feedback to Jenkins. SQS is perfect for that.

The basic architecture is like the diagram below. It's missing e.g. logging related components.



So what we have to program?

  • The script which is executed by CI pipeline. It first sends the triggering message to SNS. Then it starts to listen the feedback queue. Only required cross account permissions are SNS sending and SQS reading.
  • Lambda script which starts the deployment container. Only rights which it needs are starting the ECS container.
  • Deployment container requires the script which submits all standard output and error from deployment process to the SQS. This container should have role which has all required permissions to do the deployment. 
So now we've got the components which are required for the secure cross account deployment. Jenkins can be replaced with almost any CI/CD tool. 

I'll continue this series as soon as I get some programming done.