1. Introduction
Hello! We are a writer team from Definer Inc.
In this issue, you are wondering how to deploy a CI/CD pipeline cross account using AWS CodePipeline and CodeCommit.
Let's take a look at the actual screens and resources to explain in detail.
2. Purpose/Use Cases
This article will utilize the CodePipeline technology for the purpose of building CI/CD across AWS accounts.
It summarizes information and practices that can be helpful when you want to deploy a cross-account CI/CD pipeline using AWS CodePipeline and CodeCommit in an IT setting.
3. What is cross-account deployment?
Cross-account deployment refers to the process of deploying resources and services from one AWS (Amazon Web Services) account to another AWS account. It involves establishing a connection or trust relationship between multiple AWS accounts and enabling the sharing of resources and services across those accounts.
Cross-account deployment is often used in scenarios where organizations have multiple AWS accounts for different purposes, such as development, testing, and production. It allows for separation of resources and greater control over access and permissions.
Here are some specific use cases for cross-account deployment in AWS:
- Multi-Environment Deployment: Organizations often have separate AWS accounts for different environments like development, testing, and production.
- Software as a Service (SaaS) Applications: Companies offering SaaS applications may need to deploy their application components to customer AWS accounts. Cross-account deployment enables seamless deployment of the SaaS application components, allowing the application to run in the customer's account while maintaining the necessary segregation and security.
- Shared Services: In larger organizations, central IT teams might establish shared services accounts to provide common services like authentication, logging, monitoring, or databases to multiple business units or application teams. Cross-account deployment enables these shared services to be accessed by different AWS accounts securely, reducing duplication and promoting consistency.
- Federated Identity and Single Sign-On (SSO): Cross-account deployment is often utilized with federated identity and SSO solutions. By establishing trust relationships between AWS accounts, users authenticated in one account can assume roles in another account without the need for separate credentials.
- Compliance and Regulatory Requirements: Some industries have specific compliance or regulatory requirements that necessitate separation of data and infrastructure. Cross-account deployment helps meet these requirements by isolating sensitive workloads or data into separate accounts while still allowing controlled access and resource sharing.
- Third-Party Integration: Organizations may engage third-party vendors or partners that require access to specific AWS resources. Cross-account deployment allows for granting limited access to external entities without sharing the full administrative control of an account.
4. What is CodePipeline?
AWS CodePipeline is a fully managed continuous delivery service provided by Amazon Web Services (AWS). It helps automate the release process of software applications and allows you to build, test, and deploy code changes rapidly and reliably.
At its core, CodePipeline facilitates the creation and orchestration of a continuous delivery pipeline, which is a series of steps or stages that code changes go through before reaching production. These stages typically include source code management, building, testing, and deploying the application.
Key components and features of AWS CodePipeline include:
- Pipelines: A pipeline in CodePipeline represents the workflow of a software release process.
- Integration with Other AWS Services: CodePipeline integrates seamlessly with a wide range of AWS services, including AWS CodeCommit (source code repository), AWS CodeBuild (continuous integration service), AWS CodeDeploy (application deployment service), and AWS CloudFormation (infrastructure as code service).
- Flexibility and Extensibility: CodePipeline provides flexibility and extensibility through its integration with third-party tools and services.
- Visualization and Monitoring: CodePipeline provides a visual representation of your pipeline, allowing you to easily track the progress of code changes through different stages.
- Security and Access Control: CodePipeline integrates with AWS Identity and Access Management (IAM), allowing you to manage permissions and access control for different stages and actions within the pipeline.
- Automation and Scalability: With CodePipeline, you can automate the entire release process, reducing manual effort and increasing efficiency.
5. What is CodeCommit?
AWS CodeCommit is a fully managed source code repository service provided by Amazon Web Services (AWS). It offers a secure and scalable platform for storing and managing your private Git repositories.
CodeCommit provides a reliable and highly available Git repository hosting service with features that facilitate collaboration among software development teams. Some key aspects of AWS CodeCommit are:
- Git-Compatible: CodeCommit supports the popular Git version control system, allowing you to use Git commands and workflows that you are already familiar with. It seamlessly integrates with Git clients, tools, and IDEs, making it easy to work with your repositories.
- Security and Access Control: CodeCommit ensures the security of your code by encrypting data in transit and at rest. It integrates with AWS Identity and Access Management (IAM), allowing you to manage fine-grained access control and permissions at both the repository and branch levels.
- Scalability and Performance: CodeCommit is designed to handle repositories of any size and scale.
- Integration with AWS Services: CodeCommit integrates seamlessly with other AWS services, enabling you to build end-to-end CI/CD pipelines. It integrates with AWS CodePipeline for continuous integration and delivery, AWS CodeBuild for building and testing your code, and AWS CodeDeploy for automating application deployments.
- Collaboration and Code Reviews: CodeCommit allows multiple developers to collaborate on the same codebase by enabling branch and pull request workflows. You can create branches to work on different features or bug fixes, and then merge changes back into the main branch using pull requests. Code reviews can be conducted within the service, facilitating code quality control and team collaboration.
- Auditing and Monitoring: CodeCommit provides detailed logs and audit trails, allowing you to monitor repository activity, track changes, and review commit history.
6. Setup 1. KMS and S3
We will implement a mechanism that triggers a push to <Account A>CodeCommit and deploys with <Account B>CodePipeline.
In this article, we will focus on the cross-account authorization area.
As a prerequisite, we assume that S3 and CodeCommit already exist to store the configuration files for the ECS service and pipeline to be deployed.
Please refer to "Implementing ECS auto-deployment (CD) using Codepipeline" for details on building this area.
(1) <Account B> S3 bucket policy settings
The following policies have been applied to S3.
The provided AWS policy is granting specific permissions for accessing an Amazon S3 bucket (
test-codepipeline-account-b) in Account B from an AWS Identity and Access Management (IAM) user or role in Account A.
{
"Version": "2012-10-17",
"Id": "S3Policy",
"Statement": [
{
"Sid": "CrossAccountS3GetPutPolicy",
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam::${Account A}:root"
},
"Action": [
"s3:Get*",
"s3:Put*"
],
"Resource": "arn:aws:s3:::test-codepipeline-account-b/*"
},
{
"Sid": "CrossAccountS3ListPolicy",
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam::${Account A}:root"
},
"Action": "s3:ListBucket",
"Resource": "arn:aws:s3:::test-codepipeline-account-b"
}
]
} (2) <Account B> KMS policy settings
KMS was created and the following key policies were applied.
"Enable IAM User Permissions": Allows the root user of Account B to perform any KMS action (
kms:*) on any resource (Resource: "*") in the account."Allow access for Key Administrators": Allows the IAM user
${username} in Account B to perform specific KMS actions on any resource."Allow use of the key": Allows the IAM user
${username} in Account B, as well as the root user of Account A and the AWSCodePipelineServiceRole in Account B, to perform specific KMS actions on any resource."Allow attachment of persistent resources": Allows the root user of Account A and the AWSCodePipelineServiceRole in Account B to perform specific KMS actions related to creating, listing, and revoking grants on KMS resources, under the condition that the grant is for an AWS resource.
"Allow attachment of persistent resources" (another statement): Allows the IAM user
${username} in Account B to perform specific KMS actions related to creating, listing, and revoking grants on KMS resources, under the condition that the grant is for an AWS resource. {
"Version": "2012-10-17",
"Id": "key-consolepolicy-3",
"Statement": [
{
"Sid": "Enable IAM User Permissions",
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam::${My account（B）}:root"
},
"Action": "kms:*",
"Resource": "*"
},
{
"Sid": "Allow access for Key Administrators",
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam::${My account（B）}:user/${username}"
},
"Action": [
"kms:Create*",
"kms:Describe*",
"kms:Enable*",
"kms:List*",
"kms:Put*",
"kms:Update*",
"kms:Revoke*",
"kms:Disable*",
"kms:Get*",
"kms:Delete*",
"kms:TagResource",
"kms:UntagResource",
"kms:ScheduleKeyDeletion",
"kms:CancelKeyDeletion"
],
"Resource": "*"
},
{
"Sid": "Allow use of the key",
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam::${My account（B）}:user/${username}"
},
"Action": [
"kms:Encrypt",
"kms:Decrypt",
"kms:ReEncrypt*",
"kms:GenerateDataKey*",
"kms:DescribeKey"
],
"Resource": "*"
},
{
"Sid": "Allow use of the key",
"Effect": "Allow",
"Principal": {
"AWS": [
"arn:aws:iam::${Account A}:root",
"arn:aws:iam::${My account（B）}:role/service-role/AWSCodePipelineServiceRole-ap-northeast-1-test-cicd"
]
},
"Action": [
"kms:Encrypt",
"kms:Decrypt",
"kms:ReEncrypt*",
"kms:GenerateDataKey*",
"kms:DescribeKey"
],
"Resource": "*"
},
{
"Sid": "Allow attachment of persistent resources",
"Effect": "Allow",
"Principal": {
"AWS": [
"arn:aws:iam::${Account A}:root",
"arn:aws:iam::${My account（B）}:role/service-role/AWSCodePipelineServiceRole-ap-northeast-1-test-cicd"
]
},
"Action": [
"kms:CreateGrant",
"kms:ListGrants",
"kms:RevokeGrant"
],
"Resource": "*",
"Condition": {
"Bool": {
"kms:GrantIsForAWSResource": "true"
}
}
},
{
"Sid": "Allow attachment of persistent resources",
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam::${My account（）}:user/${username}"
},
"Action": [
"kms:CreateGrant",
"kms:ListGrants",
"kms:RevokeGrant"
],
"Resource": "*",
"Condition": {
"Bool": {
"kms:GrantIsForAWSResource": "true"
}
}
}
]
} 7. Setup 2. IAM Roles
Next, we will configure IAM.
(1) <Account B>Configure IAM roles for CodePipeline.
The following policies were used.
"iam:PassRole": Allows the passing of IAM roles to specific AWS services (
cloudformation.amazonaws.com, elasticbeanstalk.amazonaws.com, ec2.amazonaws.com, and ecs-tasks.amazonaws.com).
CodeCommit: Grants various permissions related to CodeCommit, such as canceling uploads, getting branch and commit information, and managing repositories.
"AssumeRolePolicy": Allows assuming any role within the AWS account with the ARN "arn:aws:iam::201920823605:role/*".
CodeDeploy: Grants permissions for creating and managing deployments, application revisions, and deployment configurations.
CodeStar Connections: Allows the use of CodeStar Connections.
Elastic Beanstalk, EC2, Elastic Load Balancing, Auto Scaling, CloudWatch, S3, SNS, CloudFormation, RDS, SQS, ECS: Grants a wide range of permissions for various services, including resource management and monitoring.
AWS Lambda: Allows invoking and listing Lambda functions.
OpsWorks: Grants permissions for creating and managing deployments, apps, commands, instances, and stacks.
CloudFormation: Grants permissions for managing stacks, change sets, and stack policies.
CodeBuild: Allows retrieving information about builds and starting build processes.
Device Farm: Grants permissions for managing projects, device pools, runs, and uploads.
Service Catalog: Allows managing provisioning artifacts and products.
Amazon Elastic Container Registry (ECR): Grants permissions for describing images.
Step Functions: Allows describing executions, state machines, and starting executions.
AppConfig: Grants permissions for starting and stopping deployments and retrieving deployment information. {
"Statement": [
{
"Action": [
"iam:PassRole"
],
"Resource": "*",
"Effect": "Allow",
"Condition": {
"StringEqualsIfExists": {
"iam:PassedToService": [
"cloudformation.amazonaws.com",
"elasticbeanstalk.amazonaws.com",
"ec2.amazonaws.com",
"ecs-tasks.amazonaws.com"
]
}
}
},
{
"Action": [
"codecommit:CancelUploadArchive",
"codecommit:GetBranch",
"codecommit:GetCommit",
"codecommit:GetRepository",
"codecommit:GetUploadArchiveStatus",
"codecommit:UploadArchive"
],
"Resource": "*",
"Effect": "Allow"
},
{
"Sid": "AssumeRolePolicy",
"Effect": "Allow",
"Action": "sts:AssumeRole",
"Resource": [
"arn:aws:iam::201920823605:role/*"
]
},
{
"Action": [
"codedeploy:CreateDeployment",
"codedeploy:GetApplication",
"codedeploy:GetApplicationRevision",
"codedeploy:GetDeployment",
"codedeploy:GetDeploymentConfig",
"codedeploy:RegisterApplicationRevision"
],
"Resource": "*",
"Effect": "Allow"
},
{
"Action": [
"codestar-connections:UseConnection"
],
"Resource": "*",
"Effect": "Allow"
},
{
"Action": [
"elasticbeanstalk:*",
"ec2:*",
"elasticloadbalancing:*",
"autoscaling:*",
"cloudwatch:*",
"s3:*",
"sns:*",
"cloudformation:*",
"rds:*",
"sqs:*",
"ecs:*"
],
"Resource": "*",
"Effect": "Allow"
},
{
"Action": [
"lambda:InvokeFunction",
"lambda:ListFunctions"
],
"Resource": "*",
"Effect": "Allow"
},
{
"Action": [
"opsworks:CreateDeployment",
"opsworks:DescribeApps",
"opsworks:DescribeCommands",
"opsworks:DescribeDeployments",
"opsworks:DescribeInstances",
"opsworks:DescribeStacks",
"opsworks:UpdateApp",
"opsworks:UpdateStack"
],
"Resource": "*",
"Effect": "Allow"
},
{
"Action": [
"cloudformation:CreateStack",
"cloudformation:DeleteStack",
"cloudformation:DescribeStacks",
"cloudformation:UpdateStack",
"cloudformation:CreateChangeSet",
"cloudformation:DeleteChangeSet",
"cloudformation:DescribeChangeSet",
"cloudformation:ExecuteChangeSet",
"cloudformation:SetStackPolicy",
"cloudformation:ValidateTemplate"
],
"Resource": "*",
"Effect": "Allow"
},
{
"Action": [
"codebuild:BatchGetBuilds",
"codebuild:StartBuild",
"codebuild:BatchGetBuildBatches",
"codebuild:StartBuildBatch"
],
"Resource": "*",
"Effect": "Allow"
},
{
"Effect": "Allow",
"Action": [
"devicefarm:ListProjects",
"devicefarm:ListDevicePools",
"devicefarm:GetRun",
"devicefarm:GetUpload",
"devicefarm:CreateUpload",
"devicefarm:ScheduleRun"
],
"Resource": "*"
},
{
"Effect": "Allow",
"Action": [
"servicecatalog:ListProvisioningArtifacts",
"servicecatalog:CreateProvisioningArtifact",
"servicecatalog:DescribeProvisioningArtifact",
"servicecatalog:DeleteProvisioningArtifact",
"servicecatalog:UpdateProduct"
],
"Resource": "*"
},
{
"Effect": "Allow",
"Action": [
"cloudformation:ValidateTemplate"
],
"Resource": "*"
},
{
"Effect": "Allow",
"Action": [
"ecr:DescribeImages"
],
"Resource": "*"
},
{
"Effect": "Allow",
"Action": [
"states:DescribeExecution",
"states:DescribeStateMachine",
"states:StartExecution"
],
"Resource": "*"
},
{
"Effect": "Allow",
"Action": [
"appconfig:StartDeployment",
"appconfig:StopDeployment",
"appconfig:GetDeployment"
],
"Resource": "*"
}
],
"Version": "2012-10-17"
} (2) <Account A>Create IAM role for CodeCommit
The following policies have been set.
"UploadArtifactPolicy": Grants permissions related to uploading artifacts to an S3 bucket (
test-codepipeline-account-b). The actions allowed are s3:PutObject and s3:PutObjectAcl, which enable putting objects and modifying object ACLs. The permissions apply to all objects within the specified bucket."KMSAccessPolicy": Grants permissions related to AWS Key Management Service (KMS) operations. The actions allowed are
kms:DescribeKey, kms:GenerateDataKey*, kms:Encrypt, kms:ReEncrypt*, and kms:Decrypt. The permissions apply to a specific KMS key identified by its ARN (arn:aws:kms:ap-northeast-1:${Account B}:key/${KMSのID})."CodeCommitAccessPolicy": Grants permissions related to CodeCommit operations. The actions allowed are
codecommit:GetBranch, codecommit:GetCommit, codecommit:UploadArchive, codecommit:GetUploadArchiveStatus, and codecommit:CancelUploadArchive. The permissions apply to all CodeCommit repositories. {
"Version": "2012-10-17",
"Statement": [
{
"Sid": "UploadArtifactPolicy",
"Effect": "Allow",
"Action": [
"s3:PutObject",
"s3:PutObjectAcl"
],
"Resource": [
"arn:aws:s3:::test-codepipeline-account-b/*"
]
},
{
"Sid": "KMSAccessPolicy",
"Effect": "Allow",
"Action": [
"kms:DescribeKey",
"kms:GenerateDataKey*",
"kms:Encrypt",
"kms:ReEncrypt*",
"kms:Decrypt"
],
"Resource": [
"arn:aws:kms:ap-northeast-1:${Account B}:key/${KMSのID}"
]
},
{
"Sid": "CodeCommitAccessPolicy",
"Effect": "Allow",
"Action": [
"codecommit:GetBranch",
"codecommit:GetCommit",
"codecommit:UploadArchive",
"codecommit:GetUploadArchiveStatus",
"codecommit:CancelUploadArchive"
],
"Resource": [
"*"
]
}
]
} 8. Pipeline Construction
Next, we will build the pipeline.
(1)<Account B>CodePipeline construction
Cross-accounts cannot seem to be set up from the console, so we will create them via the CLI.
Prepare an "input.son" file as input.
The below file:
{
"pipeline": {
"name": "test-cicd-cross-account-b",
"roleArn": "arn:aws:iam::${Account B}:role/service-role/AWSCodePipelineServiceRole-ap-northeast-1-test-cicd",
"artifactStore": {
"type": "S3",
"location": "test-codepipeline-account-b",
"encryptionKey": {
"id": "arn:aws:kms:ap-northeast-1:${Account B}:alias/test-codepipeline",
"type": "KMS"
}
},
"stages": [
{
"name": "Source",
"actions": [
{
"name": "Source",
"actionTypeId": {
"category": "Source",
"owner": "AWS",
"provider": "CodeCommit",
"version": "1"
},
"runOrder": 1,
"configuration": {
"RepositoryName": "test-account-a",
"BranchName": "main"
},
"outputArtifacts": [
{
"name": "SourceArtifact"
}
],
"inputArtifacts": [],
"roleArn": "arn:aws:iam::${Account A}:role/codecommit-account-a"
},
{
"name": "s3",
"actionTypeId": {
"category": "Source",
"owner": "AWS",
"provider": "S3",
"version": "1"
},
"runOrder": 1,
"configuration": {
"PollForSourceChanges": "false",
"S3Bucket": "test-codepipeline-account-b",
"S3ObjectKey": "imagedefinitions.json.zip"
},
"outputArtifacts": [
{
"name": "s3output"
}
],
"inputArtifacts": [],
"region": "ap-northeast-1"
}
]
},
{
"name": "Deploy",
"actions": [
{
"name": "Deploy",
"actionTypeId": {
"category": "Deploy",
"owner": "AWS",
"provider": "ECS",
"version": "1"
},
"runOrder": 1,
"configuration": {
"ClusterName": "default",
"ServiceName": "test-nginx"
},
"outputArtifacts": [],
"inputArtifacts": [
{
"name": "s3output"
}
],
"region": "ap-northeast-1",
"namespace": "DeployVariables"
}
]
}
],
"version": 2
}
}
Let's break down the configuration:
- Pipeline Name: The name of the pipeline is set to "test-cicd-cross-account-b".
- Role ARN: Specifies the IAM role ARN (
- Artifact Store: Defines the storage location for pipeline artifacts. In this case, it uses an S3 bucket named "test-codepipeline-account-b". The configuration includes an encryption key defined by its KMS ARN (
- Stages: The pipeline consists of two stages: "Source" and "Deploy".
- Source Stage: Contains two actions:
+ "Source" action retrieves code from an AWS CodeCommit repository named "test-account-a" on the "main" branch.
+ "s3" action retrieves an artifact from an S3 bucket named "test-codepipeline-account-b" with the object key "imagedefinitions.json.zip". This action does not poll for changes.
- Deploy Stage: Contains a single "Deploy" action:
+ "Deploy" action deploys the artifacts to an AWS ECS cluster named "default" and the ECS service named "test-nginx". The input artifacts for this action are the output artifacts from the "s3" action in the Source stage.
- Version: Specifies the version of the pipeline configuration. In this case, it is set to 2.
After building the pipeline, we actually push it to Account B's CodeCommit to confirm that Account A's pipeline works!
- Pipeline Name: The name of the pipeline is set to "test-cicd-cross-account-b".
- Role ARN: Specifies the IAM role ARN (
arn:aws:iam::${Account B}:role/service-role/AWSCodePipelineServiceRole-ap-northeast-1-test-cicd) used by the pipeline for execution and access permissions. This role allows CodePipeline to perform necessary actions on resources.- Artifact Store: Defines the storage location for pipeline artifacts. In this case, it uses an S3 bucket named "test-codepipeline-account-b". The configuration includes an encryption key defined by its KMS ARN (
arn:aws:kms:ap-northeast-1:${Account B}:alias/test-codepipeline).- Stages: The pipeline consists of two stages: "Source" and "Deploy".
- Source Stage: Contains two actions:
+ "Source" action retrieves code from an AWS CodeCommit repository named "test-account-a" on the "main" branch.
+ "s3" action retrieves an artifact from an S3 bucket named "test-codepipeline-account-b" with the object key "imagedefinitions.json.zip". This action does not poll for changes.
- Deploy Stage: Contains a single "Deploy" action:
+ "Deploy" action deploys the artifacts to an AWS ECS cluster named "default" and the ECS service named "test-nginx". The input artifacts for this action are the output artifacts from the "s3" action in the Source stage.
- Version: Specifies the version of the pipeline configuration. In this case, it is set to 2.
After building the pipeline, we actually push it to Account B's CodeCommit to confirm that Account A's pipeline works!
## CodePipelineCreate command
aws codepipeline update-pipeline --cli-input-json file://input.json 9. Cited/Referenced Articles
10. About the proprietary solution "PrismScaler"
・PrismScaler is a web service that enables the construction of multi-cloud infrastructures such as AWS, Azure, and GCP in just three steps, without requiring development and operation.
・PrismScaler is a web service that enables multi-cloud infrastructure construction such as AWS, Azure, GCP, etc. in just 3 steps without development and operation.
・The solution is designed for a wide range of usage scenarios such as cloud infrastructure construction/cloud migration, cloud maintenance and operation, and cost optimization, and can easily realize more than several hundred high-quality general-purpose cloud infrastructures by appropriately combining IaaS and PaaS.
11. Contact us
This article provides useful introductory information free of charge. For consultation and inquiries, please contact "Definer Inc".
12. Regarding Definer
・Definer Inc. provides one-stop solutions from upstream to downstream of IT.
・We are committed to providing integrated support for advanced IT technologies such as AI and cloud IT infrastructure, from consulting to requirement definition/design development/implementation, and maintenance and operation.
・We are committed to providing integrated support for advanced IT technologies such as AI and cloud IT infrastructure, from consulting to requirement definition, design development, implementation, maintenance, and operation.
・PrismScaler is a high-quality, rapid, "auto-configuration," "auto-monitoring," "problem detection," and "configuration visualization" for multi-cloud/IT infrastructure such as AWS, Azure, and GCP.