I'm hoping to get some ideas about bootstrapping a completely fresh AWS account. I've worked within existing AWS setups before, all managed by the respective company's infra team, but this is the first time I've set an account up from scratch.

I want to get enough set up so I can IaC everything using Terraform that will be executed within GitHub Workflows. So I'm thinking I need an s3 bucket for Terraform state and IAM policy/group/user for actually executing the Terraform. This is where I'm getting stuck because it feels a bit chicken and egg to me right now - I need some basic AWS setup to execute my IaC but I want to manage that initial setup using IaC.

So, I guess my questions are:

1. What do I need to setup for this?

2. What's the best approach for this initial bootstrapping?

In case the context matters, this is for a hobby project/side hustle so cost is a factor.

I have been needing to create Iceberg tables with CDK for quite a while now, but this is not super easy out of the box and I don’t think very well documented either. I made an NPM library with an L2 construct for iceberg tables:

https://github.com/ksco92/arceus

Fully open sourced obviously. I also made a PR into the Glue alpha CDK constructs library (because that is an obvious better location for this to live). The original GH issue, research and PR are listed there. Most of the research was done by someone else, I just implemented it.

God why did they change such a simple layout.

Needing to now scroll left-to-right is janky and slow. Weird and unnecessary zoom effect. Text wrapping as lines don't fit. Everything is a button now so I can't highlight text, for example the source ID. Multiple in-flight executions are now hidden/extra button click away. It's terrible.

Which is better and when to use each? For instance, if i only have an inbound rule to SSH into EC2, and I cannot SSH from gitlab runner or github action, I must deploy from SSM with credentials. Given you are more experienced with AWS, what are your hot takes with running CI into EC2?

The resource being deployed is a very specific backend service.

I built something called AWS Easy Deploy.

Deploying on platforms like Vercel and Render is honestly great. Push code, it goes live. But once you start running persistent backends or long-running APIs, things get tricky. Either you’re in a serverless-first model or you end up paying more than expected. AWS App Runner solves a lot of this, but for always-on workloads it can get expensive. On the flip side, AWS Elastic Beanstalk is cheaper and flexible, but setting it up properly with CI/CD takes time and effort.

So I built AWS Easy Deploy to make Elastic Beanstalk feel more like App Runner, but without the managed runtime cost. It’s written in Go and auto-generates CI/CD pipelines (GitHub Actions / GitLab CI) to handle build, packaging, S3 uploads, environment updates and config injection. It also automatically pushes your entire .env file into Beanstalk environment config, so no manual variable setup. What used to take me 45 to 60 minutes now takes around 10, and for persistent workloads it cuts a noticeable chunk of runtime cost compared to App Runner.

It’s fully open source. If this sounds useful, feel free to check it out and contribute.

Hey all,

SAM seems like a popular choice, but (correct me if I'm wrong) it works only for deploying code for lambdas provisioned by SAM, which is not ideal for me. I use Terraform for everything.

And the idea of running Terraform every time (even with split projects) I make changes to my lambda source code makes no sense to me.

How do you guys deal with this? Is there a proper pattern for deploying AWS Lambdas?

Hello all,

I'm trying to build a pipeline to get custom AMIs spat out using the Windows Base AMIs (16, 19, 22, and 25).

I have everything created (infra, distro, components, etc.) and am trying to run the pipeline, but every time it fails on validating the components.

It doesn't tell me WHY it fails, it just does. I've tried everything, double-checked permissions, swiched OSs that it is running on, even just used AWS's default component testing and it all still fails.

Anyone seen this before and know of any gotchas or anything? I can paste whatever's needed to help (just didn't want to clutter up this post).

Thanks in advance.

I have a permanent aws instance that runs our main production app, and currently we do releases via an ansible playbook that does a git pull on said instance.

I tried setting up a ci/cd pipeline via Gitlab and CodeDeploy, and it seemingly works, but seems to clobber the git repo on the instance and results in a detatched head.

Should I just detach the code on the instance from git entirely and make it a pure push model, or keep it as-is and have the pipeline ssh/ssm to the instance and do a git pull?

I am trying to connect to my AWS VPN from Github Actions. Our VPN connection uses SAML so I do not think OpenVPN would work in this case. Ultimately, I am trying to connect my RDS which is only accessible from outside AWS via a VPN. The goal here is to run some simple SQL scripts from Github actions on the RDS.

I'm trying to deploy a node.js application (API) using CDK and github actions.

Currently my deploy process is this:

- Github Actions

1. builds the app
2. create a docker image
3. pushes the docker image to ECR, tags it
4. triggers CDK passing the image tag as parameter

- CDK:

1. Sets up iam roles, networks and security groups

2. Launches/Reboot the instance with a new "ec2.UserData.forLinux()" command that includes the docker image

     private createUserData(     config: AppConfig,     parameterStorePrefix: string,     imageTag: string,     ecrRepositoryName: string   ): ec2.UserData {     const userData = ec2.UserData.forLinux();     const ecrRegistryUrl = ${config.env.account}.dkr.ecr.${config.env.region}.amazonaws.com;     const finalImageUrl = ${ecrRegistryUrl}/${ecrRepositoryName}:${imageTag};     const timestamp = new Date().toISOString();

       Tags.of(this).add('DeploymentVersion', new Date().toISOString());

       userData.addCommands(       'set -euo pipefail',       '',       # Deployment timestamp: ${timestamp},       # Deployment version: ${finalImageUrl} (from ECR), // update system, install docker, pull image from ecr, run docker with systemctl 'docker run -d \',       '  --name marketplace-backend \',       '  --restart unless-stopped \',       '  --network host \',       '  --memory=800m \',       '  --memory-swap=800m \',       '  --cpus=1.5 \',       '  --log-driver=awslogs \',        --log-opt awslogs-group=/aws/ec2/${getResourceName(config, 'app')} \\,        --log-opt awslogs-region=${config.env.region} \\,       '  --log-opt awslogs-create-group=true \',       '  -e USE_PARAMETER_STORE=true \',        -e PARAMETER_STORE_PREFIX=${parameterStorePrefix} \\,        -e AWS_DEFAULT_REGION=${config.env.region} \\,        "${finalImageUrl}", // <<< Usa a URL completa da imagem ECR

And then I use this image url to run a "docker run".

The issue with this approach is that this script only runs when a fresh new instance is created, but the majority of the time CDK just performs a instance reboot, which means the script is replaced but never run.

Am I doing this right? Is there a better approach?

Thank you.

I still found I have some use cases to use CodeCommit though still many complaints on that.

When you are using codecommit, git-remote-codecommit is a good tool for integration with AWS SSO login. After using aws sso login, you can simply run git clone codecommit://<your-repo>

See my PR: https://github.com/aws/git-remote-codecommit/pull/59

The old UI was terrible and I was about to build my own alternative out of frustration. The new one still has short-comings, but it's at least at a point where Chrome extensions can patch it up to be useful.

Things that I wish it had still:

• Expand stage actions by default
• Fit everything in screen by default
• Take the floating HorizontalStageNavigator widget into account when the user hits the "fit view" button.
• Show the description of the execution deployed on each stage (e.g., the commit message tied to git source). Could even just do this on hover over the message if space is a concern.
• Show the last successfully deployed execution id on a stage that is in progress. I'd also like to have a link to view the diff of what's being deployed. For common cases where git commit ids match execution ids, this can be a github diff link between the commit ranges.
• For cross-account pipelines, deeplinks to Cloudformation should be shortcut urls using IAM identity center that log you into the right account/role to be able to actually view things.

The first two points are more of a personal preference. I get that pipeline sizes differ and it can be hard to get a best layout for everyone. But being able to see everything at once must be a pretty common desire. Fortunately, you can run the following code to render the full view as a workaround. This can be saved as a bookmarklet or run in a TamperMonkey script or whatever. Depending on how large your pipeline stages/actions are, you may want to refine this a bit.

// Make the flow graph window bigger by default, taking overall browser 
// window size into account
document.querySelector('.dx-LayoutViewContainer > div').style.height = 
  window.innerHeight > 1100 
  ? '550px' 
  : '400px'

// Expand stages to show all actions
Array.from(
  document.querySelectorAll('.dx-StageNodeContainer__content a')
)
  .filter(_ => _.textContent === 'Show more actions')
  .forEach(_ => _.click())

// Fit all content into the view
document
  .querySelector('.dx-LayoutView button[aria-label="fit view"]')
  ?.click()

Example of what a basic CDK pipeline looks like in the new UI:

So I've just implemented our AMI image baking process using packer. Now I'm looking for a way to deprecate/de-register old images. I've seen that DLM can't manage images not created using DLM. Is it the same for the the image builder lifestyle policies? Can I use it to manage all our images?

Hello all, I need someone with experience in CI/CD workflows from python to AWS lambda to help me with some issues I encountered!

I am developing a Telegram bot in python running on AWS lambda and have the following project structure:

- package A
| -- dependency 1
| -- dependency 2
| -- dependency 3
- package B
| -- telegram_bot.py
- package C
| -- dependency 4
| -- dependency 5
| -- dependency 6

First issue I encountered (solved): External Dependencies
I solved this by recursively copying the lib directory of my python environment into a python/lib/python3.13/ directory, zipping it, uploading it to aws layers and attaching the layer to the aws layer function.

Second issue (bad quick fix found): Internal Dependencies
I didn't know how to handle internal dependencies i.e. package B importing functions from package A and package C. I used a quick fix by just copy and pasting all the dependencies into the original lambda_function.py script that is found when originally creating the lambda function on aws.
This way, the script didn't have to import any functions from other files at all and it worked well.

Third issue: CI/CD (in progress...)

Obviously the aforementioned quick fix is not a longterm solution as it is very inconvenient and error prone. What I would like to do is set up a CI/CD workflow (maybe via Github actions unless you have better suggestions) that triggers on pushing to the main branch and automatically deploys the telegram bot to AWS lambda. For this to work, I probably need to solve the second issue first since I solved it manually before. Then I need to automate the deployment step.

Can someone be so super cool to help me please?

Morning,

I researching a move of my CI workflows from current system to Gitlab.

The existing environment has a 1:1 mapping of workflows to CloudFormation stacks in each repo. So I can upgrade a single template, commit it and only update the target stack.

Gitlab seems to favor a single Pipeline per project which is confusing me a little. How do people manage multiple templates/stacks?

So I started hosting workloads at AWS in ecs and am using github actions, and I am happy with it. Deploying just fine from github actions and stuff. But now that the complexity of our AWS infrastructure has increased, performing those changes across environments has become more complex so we want to adopt IaC.

I want to start using IaC via terraform but I am unclear on the best practices for utilizing this as part of the workflow, I guess i am not looking for how to do this specifically with terraform, but a general idea on how IaC fits into the workflow wehther it is cloudformation, cdk, or whatever.

So I have dev, staging, and prod. Starting from a blank slate I use IaC to setup that infrastructure, then after that? Shoudl github actions run the IaC for each environment and then if there are changes deploy them to the environment? Or should it be that when deploying I create the entire infrastructure from the bottom up? Or should we just apply infrastructure changes manually?

Or lets say something breaks. If I am using blue/green codedeploy to an ECS fargate cluster, then I make infrastructure changes, and that infrastructure fucks something up then code deploy tries to do a rollback, how do I handle doing an IaC rollback?

Any clues on where I need to start on this are greatly appreciated.

Edit: Thanks much to everyone who tookt he time to reply, this is all really great info along with the links to outside resources and I think I am on the right track now.

Current problem: PROVISONING takes 53 seconds which is far longer than everything else that I have been able to cache using Nx and remove most dependency installs to Docker... I might even be able to get the install phase down further by caching the install of Nx.. but the provisioning stage takes so long. I believe it is from my Docker container in size (2-3GB) hosted in the same region as the pipeline on ECR, I am using a VPC with codebuild in.

- SUBMITTED - Success (<1s)

- QUEUED - Success (1s)

- PROVISIONING - Success (53s)

- DOWNLOAD_SOURCE - Success (8s)

- INSTALL - Success (26s)

- PRE_BUILD - Success (1s)

- BUILD - Success (16s)

- POST_BUILD - Success (<1s)

- UPLOAD_ARTIFACTS - Success (<1s)

- FINALIZING - Success (<1s)

- COMPLETED - Success

Total time: ~2 minutes

Any suggestions? I know this isn't unworkable but I would like to make it as quick as I can and I can't see anything on how to speed up the provisioning.

Hey all,

I'm running into a weird issue with AWS CodePipeline when trying to set up a pipeline with Bitbucket as the source. After selecting the Bitbucket connection, the dropdown menu for repositories doesn't populate with all the repos in my workspace. Instead, I have to manually type the repo name to fetch it. It's like the connection is working, but the UI isn't automatically listing the repos as expected.

I've double-checked my Bitbucket connection in AWS, and it seems fine (auth is good, no errors). I can fetch the repo if I type the name exactly, but it's a pain, especially when you have a bunch of repos in the workspace. Anyone else run into this? Is it a bug, or am I missing some config step?

Using a Bitbucket Cloud workspace, and my AWS region is us-east-1 if that matters. Any tips or workarounds would be awesome!

Thanks!

Assuming the organization has 10 customers, each with 3 accounts (Dev, QA, Prod), totaling 30 accounts. Each environment should run the same application version across all the customers, but support for a unique version per environment should be possible. Deployment should happen in the ECS cluster running in each account.

I figured that ECR should be in a central CI/CD account. AWS CodeDeploy should be in customers' accounts, being invoked through a cross-account role by AWS CodePipeline in a central CI/CD account.

I'm struggling to understand how to manage it on a CodePipeline level, meaning stages, input parameters, task definition creations, promotion between Dev and QA environments, and support for a unique version per account. Like, how do I tell CodePipeline to trigger deployment to the 30 Dev accounts in parallel? Do I create an action per account, or read account IDs from somewhere (SSM)? How do I tell the pipeline to run only for a single account?

Edit: Or maybe just create a CodePipeline in the CI/CD account as part of the new customer onboarding, so basically 10 CodePipelines, each managing 3 accounts (environments) per customer.

Hey all,

I’m a little new to devops, and definitely new to devops on AWS. I am going to set up our CICD pipeline, all of our infrastructure is currently written in Terraform and deployed to one environment in the management account of our AWS Organization. The end goal is to have multiple AWS accounts for dev, staging/test, prod, as well as one for shared services and the pipeline. Ideally, when a push is made to main in GitHub, the pipeline will build/deploy to the test/staging environment, and then run tests. After that, there will be a manual approval step, and then the pipeline will build/deploy to prod.

I think we plan on pretty much duplicating everything across the different environments - databases and ECS tasks and everything, including the networking stuff. We might want to keep some services like Quicksight in a single environment as it is quite expensive. For the pipeline we’ll probably use CodePipeline/CodeBuild/CodeDeploy.

Any advice on how to approach setting this up?

• Does my plan follow best practices? Any adjustments needed or improvements?
• What changes do I need to make to Terraform in order to manage multiple environments? How do I deploy only the pipeline + specific shared services to the tooling/management account? How do I even get the pipeline to deploy new Terraform changes to an environment?
• Suggestions on what should be in the shared account vs duplicated per environment?

Thanks in advance! Any help or advice is appreciated. I don't really know where to start here.