kmoberg.io

Back online

2026-08-19T00:00:00.000Z

This blog used to run on Ghost, in a container, in a DMZ, behind a Cloudflare Tunnel — right up until the tunnel quietly died and took the whole thing with it.

It now runs as a static site: markdown files in a git repo, built with Eleventy, deployed automatically on push. Nothing to patch, nothing to tunnel, nothing to collapse at 2 AM.

Old posts are on their way back as they get migrated over.

AI/KI er ikke en kilde

2025-07-28T00:00:00.000Z

Jeg gjentar - AI ER IKKE EN PRIMÆRKILDE.

Jeg ble gjort oppmerksom på en ganske skummel utvikling i dag. Profesjonelle kommentatorer i riksmedier bruker AI/KI som primærkilder. Dette er ett problem som har blitt mer og mer tydlig bare de siste månedene, og det er ikke meningen å henge ut spesifikke journalister, men Aftenposten i dag er det ett perfekt eksempel. (https://lnkd.in/diVywUFb)

I denne artikkelen var det ett avsnitt som fikk meg til å bråstoppe når jeg leste det.

«De var såkalte edubbaer, sumerske skoler hvor man lærte å lese, skrive og utføre matematikk», ifølge den alltid så hjelpsomme roboten ChatGPT.

La oss ta ett øyeblikk å tenke litt ekstra på hva som står her. Her baserer forfatteren seg på ChatGPT som en primærkilde, noe ChatGPT, Gemeni, Claude, Grok, eller de ørten andre KI modellene som eksisterer der ute, absolutt ikke er!

Kunstig intelligens lyver. Hele tiden.

Som utvikler, er jeg flittig bruker av AI. Jeg bruker ChatGPT til å skrive-om kode, eller komme med både outlines til boilerplate kode og forslag til løsninger på problemer. Dette er AI veldig bra til, og har mye informasjon å bygge seg på, men allikevel opplever jeg, hver. Eneste. Gang. Jeg bruker AI til kode, at den lyver til meg - den generer kode som ikke stemmer, og er HELT overbevist om at det den har skrevet, er riktig.

Det samme gjelder når du bruker AI til fakta-opplysninger, og jeg har to ufattelig gode eksempler: Google har (dessverre) begynt å fylle Google-søk med resultater fra Gemeni (Googles egen AI løsning) i toppen av søk. Dette er ment for å gjøre livet ditt lettere, men i realiteten er det fyllt med objektive fakta feil. Som eksempel:

Hvis du i dag (28. Juli 2025) søker etter: "Skolestart grunnskole 2025" så er første resultatet link til Stavanger Kommune som har en flott liste. Allikevel, gir Gemeni en oppsummering i toppen: "Skolestart i Stavanger grunnskole for skoleåret 2025-2026 er mandag 18. august, ifølge Stavanger Kommune", med en flott link til Stavanger Kommunes side, som kilde. Problemet er bare det, at skolestart i følge kilden som Gemeni oppgir, er Torsdag 14. August.

Ett annet prima eksempel er dette: Hvis du søker etter "trenger barn pass til danmark med foreldrene" får du dette til svar: "Ja, barn trenger pass eller nasjonalt ID-kort for å reise til Danmark. Selv om det er passfrihet i Norden." Gemeni legger ut i fem avsnitt om hvorfor barn trenger pass, selv sammen med foreldre. Her linkes det til regjeringen.no, og hvis du klikker på linken, er det første som kommer opp: "NEI, barn under 18 trenger ikke pass sammen med foreldre".

Hvis AI tar feil om disse to, ganske enkle tingene å sjekke opp, hva tror du skjer når det er mer komplekse spørsmål du ønsker å få svar på?

Med alt dette sagt - jeg bruker ChatGPT daglig selv til å løse spesifikke problemer. Jeg tenker vi må bare være forsiktige, og huske - de er ikke en kilde!

Denne artikkelen ble opprinnelig postet på LinkedIn den 28. Juli 2025, og er arkivert her for fremtidig historikk.

Storage for Photographers

2025-06-17T00:00:00.000Z

Let's be honest - storing photos has become a chore. Our cameras has larger file sizes for images, we take more images, and the idea of losing everything, sucks.

This is why you need a backup strategy - and you need to stick to it, and maintain it. Trust me - I know. Over my 20 years as a digital photographer, I've lost both the originals and high-res versions of some of my favorite images, including multiple once-in-a-lifetime situation photos that I'll never be able to get back, so I've sworn never to let that happen again, and to help others with the same.

Backup Strategy

You need to make one

Most people swear by the 3-2-1 backup strategy, and so do I. The key is that you want three different copies of your images, on at least two different mediums, where at least one copy is off-site. With storage being relatively cheap these days, this goal is pretty achievable, and I'll explain how!

My storage strategy

Today - I'm happy with how my storage strategy works. It's fast, accessible, and doesn't rely on too many outsiders.

Simple diagram of the strategy

In my workflow, these days I work in CaptureOne. This means I have a catalog file, and the image files and can chose to separate them if I want, which I do. So I store my catalog files on my external SSD (since the catalog files can be pretty big), and I store the originals on my NAS. The catalog files are also backed up to the NAS.

The most important files to worry about are the images, stored on the NAS. The NAS runs on a Synology DS923+, which has 4 hard-drives, in SHR, meaning one drive can die without dataloss. This in itself is not backup. This is high-availability or high-reliability, NOT backup.

Part of the Studio Moberg IT stack. It does not need to be this complicated for you!

To back the files up, I once a day, backup the entire image volume of my NAS to AWS Glacier using the Synology Glacier Backup app. Glacier in AWS is relatively cheap per TB, and functions as both my off-site backup, but also as a last-resort disaster recovery, since recovery of data from Glacier can be pricey.

Therefore, I also have an external 16TB WB MyBook Duo hard-drive that I sync photos to. This is done using a Mac application called Get Backup Pro ($29.99) which runs once a day to keep the NAS and drive in sync. It's simple, and reliable, and doesn't require manual intervention.

As a side-note, 16TB isn't that much when we're talking about both photo and video storage, so I have a second 16TB drive archiving only video projects plus multiple 4TB SSDs that I use as working-drives when editing projects.

All of this costs me roughly $10 a month.

So you may ask: "What do you do when you are not at the office with access to the NAS?" Good question! The answer is VPN. I have a Wireguard VPN configured on my Unifi router, that is configured on all my devices to auto-connect as soon as I'm not on my office network. That means I always have access, and performance is really reliable, even on poor/slow internet connections. I've had no issues editing on a 20mbit Wifi in a cabin in the woods, even with relatively-large image source files.

No problems editing from the cabin in the woods over VPN!

The only downside is uploading over poor connections which can take some time, but once the images are uploaded, it's no problem. Another solution when there is a poor connection during a shoot is to use a CaptureOne session, then importing that into the main catalog later, or just to move the source files once on a stable connection.

Deliverables to Clients

This is not strictly backup related, but relates to storage. Personally, I offer clients two sources. We use Pixieset for their client galleries. It's cheap, their galleries are amazing, and stupid easy to setup and allow for client downloads. We've been using them for a while and they are great! Love them.

Additionally, I have a custom workflow configured where we upload files to Amazon S3, and give them a custom link for download.

Our custom solution for allowing sharing of ZIP files or download links to images.

It's dead simple to upload and it generates a short-url using our domain using the free software Kutt.it which we also self host, meaning there are no expenses. Files uploaded to S3 are deleted after 30 days per our agreement with the client.

Final Thoughts

Backup doesn't have to be complicated. Most of the time it can be automated, really easily. My strategy might be slightly more complicated than others, but you can achieve the same goal with just a set of external hard-drives and Backblaze.

My main point is this: you need three copies of source files. One should always be off-site. You'll thank yourself later, trust me on this. Losing historic images suck!

K8s is overkill for your workload

2023-12-01T00:00:00.000Z

I hate to be the bearer of bad news, but since you’re reading this, I guess that, most likely, you’re running Kubernetes in production. Sorry to break it to you: chances are - you’re using the wrong tool for the job.

In the IT industry, few other products in recent years have become so well known or become such an industry buzzword as Kubernetes. In a relatively short time, Kubernetes has become how organizations, from small startups to internet-sized enterprises, run their container workloads to organize, connect, and scale applications on demand.

What is Kubernetes?

If you haven’t heard of Kubernetes before, let me summarize. Kubernetes, known as k8s, is a widely recognized and extensively used platform in the IT industry. It allows organizations to manage and scale their container workloads by providing comprehensive tools and features. Kubernetes enables seamless organization, connectivity, and scalability of applications, making it a popular choice for businesses of all sizes, from small startups to large enterprises.

K8s has its roots in 2015 as a Google project and was built upon Google’s cluster management system known as Borg. Google worked with the Linux Foundation to kickstart the Cloud Native Foundation (CNCF), a non-profit organization that nourishes and grows open-source software projects. It offered Kubernetes to the CNCF as its initial seed project.

Why Kubernetes has become so popular

Kubernetes has gained immense popularity recently thanks to its ability to manage and scale container workloads effectively, and its comprehensive set of tools and features allows organizations to organize, connect seamlessly, and scale applications. K8s has become the preferred choice for businesses of all sizes. Kubernetes provides a resilient and highly available infrastructure, automates tasks, ensures efficient resource utilization, and simplifies application deployment and management. Its flexibility, portability, and compatibility with various cloud providers have further contributed to its widespread adoption.

Kubernetes’ excellent cloud compatibility is also one of its most substantial advantages. All three major public cloud vendors and many private cloud vendors have implemented Kubernetes in some form, creating abstraction layers to reduce the complexity for the end users.

But this is also the crux of K8s - complexity

DevSecOps and Kubernetes

Taking one step back for a second, there is another factor we need to keep in mind. Alongside the rise of Kubernetes, a movement has risen in the IT world: bringing developers (Dev), Security (Sec), and Operations (Ops) together. Although DevSecOps is a topic for a post on its own, I argue that without its rise of DevSecOps, Kubernetes would not have been as popular as it has become in that short time.

Allowing developers to write (relatively) simple application declarations and then have a (somewhat) easy way to throw them up in a cluster and have the application up and running quickly is a massive plus for many businesses! If you have a company pushing for SRE and app teams to work together to build something great, this can be a potent combination.

However - this perfect world is often not the reality we live in…

Kubernetes might not be the right tool for the job

Imagine you’re going to build a house. If you want to do the job well, you will need several tools. A power drill will undoubtedly help you a long way, but you will get to a point where you probably will need other tools too: you will probably need a hammer, a nail gun, a bubble level, a saw - you name it, and this is also the case with Kubernetes.

Kubernetes is great. But if you’re only running a few containers or microservices here and there, using Kubernetes for that work is like bringing a battleship to a water balloon fight - it’s overkill. Even if you’re running larger workloads, 10 or 20 containers, you could still have a better chance using other readily available container runtime environments or something completely different, such as serverless functions.

The issue we keep encountering across hundreds of businesses is that Kubernetes is not just Kubernetes. If you’re going to run a Kubernetes cluster effectively, you need so much more. You most likely need a way to deploy to K8s in some CI/CD pipeline, then have something keep track of your running pods and versioning, such as Argo. It would be best if you had a way to keep track of secrets, logs, authentication, some service mesh, and so much more. Using Kubernetes in production is an iceberg. It's a vast iceberg; like real icebergs, you only typically see the top 10%.

Fig 1: An example Kubernetes Iceberg

And here is the crux of the problem - most businesses are not set up to handle this sort of iceberg! We don’t have enough engineers with the infrastructure experience to deal with the intricacies that come with such a complex issue that is K8s, and what we keep seeing time and time again happen to businesses is developers being forced to do infrastructure work and hating it.

This isn’t always the case. Many developers enjoy working with K8s, and doing infrastructure work is often a new and exciting challenge for them, but we see burnout rates explode for those who don’t enjoy it and are being “forced” to do that sort of work.

Developer satisfaction is not the only thing to consider here. What we see time and time again is a lack of resources on the operations/platform or security sides, and you struggle to maintain the cluster. Kubernetes has a reasonably frequent release schedule and regularly releases breaking changes, and this is just talking about the cluster itself. It becomes much more complicated when you start looking at maintaining the nodes and their operating systems, plus all the Kubernetes dependencies.

And the fact is - chances are, you’re not developing an internet-scale application. You’re bringing a battleship to a water balloon fight.

Managed Kubernetes

One way to relieve some of this pain is to run managed Kubernetes in some form, typically from a public cloud provider. This is undoubtedly an excellent way to reduce significantly the workload your team needs to do to keep up with the platform. Google Kubernetes Engine (GKE), Amazon Elastic Kubernetes Service (EKS), and Azure Kubernetes Service (AKS) are all excellent ways to avoid having to maintain the K8s nodes themselves. However, you’re still not eliminating the issue of Kubernetes dependencies, such as tools and services that typically are the leading resource thieves.

The alternatives

So, what should you consider instead? First, let’s clarify: we don’t want to return to Virtual Machines. We want to move forward to newer and better things. So what are they?

If you still want to keep running containers with minimal changes to your workflow, consider running in a completely managed container environment such as Amazon Elastic Container Service (ECS) or, if you have a small app or limited services, Azure Container Apps. ECS is an excellent production and internet-scale equivalent to K8s/EKS, but without the need to maintain a cluster - everything is done by using managed Services in AWS. Here, you still use the same containers you have been using and have a very minimal declaration and configuration setup, then let Amazon do the rest for you.

However, if you want even less maintenance and are prepared for an application rewrite, cloud functions alongside an event-driven architecture can do wonders for your teams with less complexity and less work for everyone involved. For some, it might be slightly more expensive per compute hour, but based on experience, you’ll quickly make up that difference in engineering hours, which are typically significantly more expensive.

Fig 2: Example of an event-driven architecture (Source: Amazon Event-Driven Architecture)

The future?

Managed services are usually the way to go for a lot of teams. Most organizations simply don’t have the resources to build a well-designed and well-built production environment without cutting back on developer productivity. By combining the usage of well-maintained managed services and considering using a different tool for the job, you can significantly increase developer satisfaction and productivity - both of which are often in short supply. Adopting these managed services is also often not something that takes a lot of time or is hard to do, and it can be the quick win many teams are looking for.

As stated before - if you’re looking for long-term changes or starting from scratch - an event-based infrastructure will set you up for the future and allow you to scale quickly and easily.

But what do you think? Should everyone adopt Kubernetes, or do you know of good alternatives businesses should consider?

My Favorite AWS Tools and Resources that I Use Every Day

2022-06-09T00:00:00.000Z

Working as a developer and cloud platform engineer working with AWS, I use a ton of tools every day. A lot of these are OS agnostic, but there are some that are not — I’m a Mac user: sorry.

Terminal: iTerm2 + Fig + Starship = ❤️

My main tool is my terminal. I use it all day, every day and it is probably my most important tool, which is also why it has gone through a ton of modifications over the years and has now gotten to a point where I’m mostly happy with it.

As for which shell I’m using, I’m still using ZSH with Oh-my-zsh installed, but with mostly default settings and a theme applied.

My Terminal Setup

iTerm2

iTerm is the base of the terminal. It has a ton more options than the built in terminal, although I rarely use most of them these days.

Cost: Free! from https://iterm2.com

Starship

Starship is where the customization begins! Starship a customizable prompt that gives me a ton of information at a glance. The prompt is very much context aware, and will change depending on what folder you are in, displaying only relevant information to you.

Starship.rs in action

In my case I display things like current AWS profile in use, git status, and battery information if I’m running low.

The starship config file

Cost: Free from https://starship.rs/

Fig

This is my latest tool in my arsenal and it’s fantastic!

Fig is a terminal addition that adds IDE-style autocomplete to your existing terminal. I know you don’t need this functionality, but boy is it nice to have in a lot of situations when you can’t exactly remember a command or what the heck you named your file or folder.

Fig also allows you to create custom shortcuts and commands the same way aliases let you execute commands more efficiently. It’s hard to stay enough good things about Fig and it’s well worth a try!

Demo of fig

Cost: Free from https://fig.io

Jetbrains

I’ve used pretty much all the code- and text editors as well as IDEs that are on the market. I was a die-hard VScode user since one of the earliest betas, but after a friend showed me the power if the Jetbrains suite, it’s hard to go back. Jetbrains offers a myriad of various IDEs for different languages, but they all share the same basic concepts and functionality. It is extremely customizable thanks to a well stocked plugin library from 3rd party developers as well as Jetbrains them selves and as as result, a ton of their software have gotten a permanent place in my dock as I use it on the daily.

The dock on my Mac always have a number of Jetbrains apps.

Specifically my most used applications include PyCharm, GoLand, WebStorm and DataGrip. As I’m also still responsible for some PHP apps, PHPStorm is also still used, but not as much.

If you haven’t tried out DataGrip yet, it got a significant revamp a while back and is honestly one of the best database management tools I’ve used.

Cost: €649 per year, ex. VAT. (For Students: FREE!) from https://www.jetbrains.com/

Github Copilot

This is going to cause some controversy I’m sure. Other developers I know claim Copilot is terrible and makes it look like a junior developer wrote the code and is terrible insecure. I disagree. If you think this way, you’re not using Copilot right and you’re just accepting whatever Copilot suggests without no critical thought. That’s not how it’s designed, and not how you should be using it.

Copilot can help you significantly speed up your workflow by suggesting the code you are going to type. In the example below, I’m creating a Terraform security group for AWS. I specify the name and ingress, and Copilot suggests the next steps. However, as you can see, it suggests the wrong ports, and I’m not blindly accepting it. You need to be weary of what it suggests and correct what needs to be corrected. However, it even so, it saves me significant time when building these code blocks.

Github Copilot Demo

There are some significant privacy concerns about using copilot, as copilot transmit a good amount of telemetry when it is used, and its suggestions are based on other peoples open-source code submitted to GitHub, so if you are developing top-secret government secrets, I would read the privacy policy before blindly jumping into Copilot. However, if you’re already putting your code in Github SaaS and not hosting your own server, and especially if you’re using public repos, the telemetry Copilot uses should be no huge privacy concern.

Cost: Free (requires invite) from https://copilot.github.com/

AWS CLI (and aws-shell)

AWS has a great CLI, but what’s even cooler is “aws-shell”! From AWS labs, you can run an interactive shell that connects to your AWS account and from there manipulate it as you wish. It provides auto completion not only for commands, but also for resource names and is incredibly powerful!

AWS Shell Demo

I’ll be the first to admit that I typically just use the normal CLI, but as soon as I have to deal with instance names and more, the shell is a fantastic tool to be able to use!

In case you don’t know — thanks to AWS “API First” philosophy, any resource in AWS should be available as an API and in the CLI, so anything you do in the console should have an API available to you.

Cost: Free from https://github.com/awslabs/aws-shell

Infrastructure as Code (IaC) — Terraform and Pulumi

If you’re working with cloud and haven’t looked at infrastructure as code yet — you need to do so. Today. If you’re still configuring your infrastructure by clicking in the console, or running a single CLI command here and there to maintain your infrastructure, it’s time to move on — trust me.

Infrastructure as Code, like the name suggest, allow you to define your infrastructure in code. This means it can be checked into version control, and that it is easily reproducible. It makes it incredibly easy to organize and make sure that knowledge isn’t lost when someone leaves the company. The idea is that the infrastructure is *immutable* and should not change after it has been deployed without a redeploy or replacement.

There are multiple tools available for this task such as CloudFormation, Chef, Puppet and more, but I’d like to recommend two sets of tools: Terraform and Pulumi.

Terraform is probably the most well-known IaC tool on the market in 2022. It has a very robust set of tools, providers and developer support available making it incredibly powerful and easy to get started with, building your infrastructure. Terraform is made by Hashicorp, a company known for providing robust and well-tested infrastructure tools. You write your code in HCL (Hashicorp Configuration Language) a custom language that is easy to write and easy to read, but there is also a JSON version available if you absolutely prefer JSON for some reason.

In my opinion, Terraform is *perfect* for infrastructure or someone in operations starting on their IaC journey, or if you have a team of infrastructure people working on this. It is extremely powerful and extendable, and provides the tools a platform team needs.

Terraform is a fantastic companion to AWS and especially if you’re testing new things and labbing a project as it is incredibly easy to create a new resource and then destroy everything afterwards.

Example terraform

However, if you’re a developer with limited or no infrastructure experience, let me suggest an alternative to you: Pulumi.

Unlike Terraform, Pulumi doesn’t have use it’s own markup language, but instead extends the programming language you’re already working in, such as Python, Go, Typescript, C# or Java! Allowing you to define your infrastructure in a language you’re already familiar with.

Pulumi has a number of other extremely powerful tooling as well, allowing you to more easily deal with expressions, and logic, not directly possibly in Terraform.

For a lot of developers, it is much easier to get started with Pulumi over Terraform, so it’s worth looking into. However, as Pulumi is a newer tool, it is not as well-known as Terraform, but it **uses terraform providers** meaning it has the same support for infrastructure as Terraform.

Pulumi using Python

Cost: FREE! from https://www.pulumi.com/ and https://www.terraform.io
(Terraform has some paid cloud functionality for working with enterprise functionality)

Alfred

Ok — on to some very Mac specific apps I use, that are incredible. First of all: Alfred.

I first used Alfred on my very first MacBook, in 2006 — back when Spotlight on the Mac in OS X 10.4 was awful and took multiple minutes to launch an application. However, once 10.6 and 10.7 rolled around, Spotlight got to a point where I really didn’t feel I needed to install Alfred anymore, since I was just using it as a fancy app launcher anyway.

That was a mistake.

I started using Alfred again this year, after not having used it for 10 years, and boy has it been great! Yes — Alfred can function as a fancy app launcher, but that’s not why you want it. You want Alfred for all the small functionality here and there you can do. Like moving a file from one folder to another with two presses, or searching through Apple Music instantly from your keyboard and without ever leaving your current application.

Alfred in use

Alfred is incredibly powerful in ways you could probably never imagine. It’s FREE except for some functionality, so it’s well worth you trying it out!

Cost: FREE (some paid features) from https://www.alfredapp.com/

Snippetslab

This is a small useful utility that I have yet to utilize to it’s full potential, but it’s a very lightweight and useful application for storing snippets of code for re usability.

Snippetslab

It has built in integration with Github Gists, allowing you to instantly publish and share code with others, as well as syntax highlighting for a ton of languages.

However, what makes Snippetslab most useful to me, is the incredible Alfred integration! With a simple CMD + Space, then type in snip <search> followed by ENTER to instantly insert a snippet.

There are tons of other snippet managers around, however what makes this one great, is the fantastic Alfred integration.

Cost: $9.99 from https://www.renfei.org/snippets-lab/ or on SetApp

Dropshare

There are again plenty of uploading tools available, but Dropshare has been my favorite for a while. It’s a simple menubar application that allows you to easily upload screenshots, documents, videos, or what you want to a myriad of destinations, including AWS S3!

It’s also highly customizable, including giving you the ability to automatically shorten URLs to a custom domain.

Cost: $24.99 from https://dropshare.app or on SetApp

Conclusion

These are just a few of my every day applications that I use. I have a plan on expanding this series with a few more posts with more of the really useful Mac applications that I’ve found over the years.

AWS Can Be Confusing: Where should I run my crap?

2022-05-15T00:00:00.000Z

AWS has over 180 different services with a myriad of different names and it can be incredibly hard to figure out where you should run your applications. In this post, my goal is to make it a bit easier for you to get started with AWS, and know what services you should consider researching moving forward.

Basic Services

We'll start with a quick introduction of some of AWS most popular and well known services, just to get a brief understanding on what services we are going to be talking about later:

Amazon EC2 - The backbone and arguably one of AWS most famous services. EC2 is AWS Virtual Machines. You select how much CPU, RAM and disk space you need, then what operating system to run. You can pick from a myriad of Linux distributions, Windows Server and Desktop and even macOS (special rates and restrictions apply). You’re billed by the second you have the virtual machine running.
Simple Storage Service (S3) - Amazons unlimited object storage, making it easy to store files for access by other AWS services or serving them to the internet (although you might want to put a cache such as Amazon CloudFront in front if you chose this last service).
Amazon Elastic Cloud Service (ECS) - Have you containerized your application and need somewhere to run them, but don’t want to deal with the complexity of Kubernetes? ECS is for you. Amazons ECS service is a fully managed container orchestration service making it easier for you by abstracting away most of the complexity that comes with container orchestration.
Amazon Elastic Kubernetes Service (EKS) - A managed container service to run and scale Kubernetes applications in the cloud or on-premises with EKS anywhere!
AWS App Runner - Just want to run a stupid simple container, and not have to worry about it? Try out App Runner. It connects directly to a code repository, or to a container registry then builds the application on the fly for you. Whenever you push changes, App Runner reacts, rebuilds and republish the application.
Amazon Relational Database Service (RDS) - Databases are a mess to maintain on your own, so why do it yourself when Amazon can take care of it for you? Pick a database engine, MySQL, PostgreSQL, Oracle or even SQL Server, select how much CPU, memory and how you want to store the data, and Amazon takes care of the rest for you.
AWS Lambda - Serverless computing! Lambda is a serverless, event-driven compute service that lets you run code for virtually any type of application or backend service without provisioning or managing servers. At all. You can trigger Lambda functions from most AWS Services, or your own code.
Amazon CloudWatch - Collect, access, and correlate data on a single platform from across all your AWS resources, applications, and services.

Flow Diagram

For a quick and dirty suggestion to where you can run your application, here is a quick flow diagram:

What are you trying to run?

With the services primer in place, the question becomes "What are you actually trying to run?" and "how do you want to run it"?

If you’re migrating a work load directly from an on-premise virtual machine, without the ability to containerize or refactor the application, Amazon EC2 is probably the right choice for you. You pick the number of CPU cores and how much memory you need, and you can use Amazon Application Migration Service (AWS MGN) to automate the migration for you!
Got a brand new application you’re building, with limited dependencies? Consider going a brand new direction, where you don’t have to worry about servers, or infrastructure. Serverless compute, using services such as AWS Lambda and Amazon API Gateway lets you build applications and run them only when you need them, and only pay for when the functions are executed - and the cost is dirt cheap. Fractions of what it would cost to run code in any other service. I recommend checking out how a company such as “A Cloud Guru” runs their entire video training platform on AWS Lambda and Serverless. This architecture requires you to rethink how you’re building your application, but it’s not very complicated, and you can save a significant amount of energy and money.

Containers

When you have containers to run, you have multiple choices depending on how much control you want over the the orchestration of the containers themselves.
- If you just have a simple container, with minimal dependencies, AWS App Runner is a great way to run your container.
- Alternatively if you do not expect to get much traffic to the application, Amazon Lightsail is a dirt cheap way to run containers too, just don’t expect too much integration with other AWS Services.
- Want container orchestration, with automatic scaling out and in, using load balanced solutions, but don’t want to maintain it? Amazon Elastic Container Service (Amazon ECS) on Fargate is perfect! Running on Fargate you have no servers to maintain or have to worry about provisioning capacity, AWS takes care of it all for you, including scaling, load balancing, and maintaining the control plane. It’s simple to configure, and integrates very well with other tools such as CodePipeline, CodeBuild and CodeCommit.
- Kubernetes is an amazing technology allowing for incredible scaling and expandability thanks to its open source nature, but it can be complicated to set up, and especially to maintain in the long run. Amazon EKS works to solve a lot of the complicated parts of running a Kubernetes cluster by delivering a managed control plane, across multiple data centers for high availability and taking care of availability and scaling, as well as automatically detecting and replacing unhealthy control plane nodes. EKS gives you the flexibility of Kubernetes, without necessarily requiring an entire infrastructure team to manage your cluster. It also makes it much easier to integrate and authenticate with other AWS services.

Databases

In most cases, you not only need a compute service, you also need somewhere to store your data. Amazon provides multiple database services depending on your needs:
- After having spent a number of years working with databases in AWS, I cannot emphasize enough how much I recommend Amazon Aurora if you’re running MySQL or PostgreSQL in production. Amazon Aurora takes the concept of a managed database to the next level, and automates replication across multiple availability zones, or regions, distributed storage, and has incredible performance. No longer do you have to worry about scaling out, or increasing database storage - Aurora takes care of it all. For production, we’d go for Aurora in 10/10 cases.
- For smaller projects, development, or if you need OracleDB, MSSQL or even some custom databases, Amazon RDS is a managed database service, abstracting away the need to maintain the database server itself, and making it incredibly easy to maintain a database. You still need to provision capacity and monitor it, but it is still an amazing service in order to avoid having to maintain a database server.
- The last half a decade has also seen the meteoric rise of NoSQL databases. AWS offers a fantastic managed, Key-Value NoSQL database service known as DynamoDB. It features single-digit millisecond performance with nearly unlimited throughput and storage. It has encrypted data at rest, backup and restore and automatic multi-region replication. The storage is cheap, and is well worth a shot for projects that work well in NoSQL databases.
- Another good alternative for NoSQL workloads is Amazon DocumentDB with MongoDB compatibility. Built upon Amazon Aurora, and supporting MongoDB 3.6 and 4.0 APIs, DocumentDB allow you to manage your database in JSON with incredible performance and AWS tooling support.

On top of all the traditional database types, AWS also has a number of other databases such as Amazon ElastiCache with support for Redis and Memcached, graph databases with Amazon Neptune, data warehouses with AWS Redshift and more. All of these are well documented and often require some special use case, so I’m not spending much time on them.

Thoughts about Vendor Lock-in

A lot of these services mentioned in this post are Amazon proprietary software, and a question we get a lot when working with cloud, is customers being worried about being locked into a vendor and not being able to move to a different cloud vendor at a later point if needed.

This is very much a valid concern that the business needs to consider when building their cloud infrastructure, and I’ll provide a few observations from my years working with all three major cloud vendors.

As long as you’re working with a cloud vendor, you will always have a lot of features that you are required to use, that are cloud proprietary. Examples of this include IAM and account provisioning, VPC design, database design (unless you run your own database as a VM - and if you do that: why are you in the cloud, and not just in a datacenter…) (also note, this does not include data!) and a lot more.
Making sure you’re declaring all of your cloud infrastructure in a cloud agnostic IaC language will mitigate a lot of these issues.
Your data is almost never locked in, but how you manage that data might be to some degree or another. However: no matter what cloud you use, and what service you use in that cloud, migrating that data OUT of any cloud vendor, will be insanely expensive if you have more than a few TB of data. We’re talking tens if not hundreds of thousands of dollars in data egress cost depending on how much data you got. Most cloud vendors, including AWS, have migration tools you can use, to very easily migrate data into their service, meaning your data is safe. There are some caveats: AWS Lambda isn’t directly transferrable to Azure Cloud Functions without a rather large refactor. Moving from Google Kubernetes Engine to Amazon EKS will require you to refactor how your Kubernetes cluster integrates with other services, but migrating from an Azure managed MySQL database to a Google managed MySQL database, is not complicated - at all.
Very, VERY few companies ever migrate to another cloud vendor. I’ve seen companies build their infrastructure, avoiding the use of any cloud vendor native solution, but instead managing it themselves, to be cloud agnostic, yet never ever actually investigate migrating their infrastructure. Instead, they are paying many, many times more for their infrastructure than they need to. Managed infrastructure is often cheaper, and is absolutely cheaper when you start counting the amount of hours of maintenance they require.
Having a plan for what you need to do if you wish to migrate, is extremely important. It’s a plan B, incase AWS goes under, or Azure decides they no longer want you on their platform. Having spent a few hours thinking through, and writing down a sketch of what you need to do, in case you need to migrate, is extremely helpful. Additionally, writing down the reasons why you picked your cloud provider in the first place, can also help ease the doubts you might have later on, and reduce the stress of “what if we want to switch” years down the road.

For a vast majority of companies, a migration will be a costly job no matter how you design your infrastructure, including just using non-propriatorey solutions, so for most, it is much easier just to use the services you need to, in the first place and deal with it, if it ever becomes a problem. Trust me, your engineers will be much happier.

Conclusion

Amazon Web Services consist of a vast web of interconnected services, and with over 200 different services with confusing sounding names, it can be a very complicated and confusing area to get into. Starting with the basic services mentioned in this post, can be a great starter for a lot of users that are just getting into cloud computing. As with most AWS services, all the services listed here integrate very well with the rest of AWS and allow you to keep expanding on and further build your infrastructure.

Additionally, if you want to try out most of these services (EKS not included), AWS has a very generous free-tier allowing you to try out most services for free, without paying anything.

If you have recently started using AWS and want to know where to go from here, I highly recommend looking into the study materials for some of the AWS certifications. You don’t need to take the certifications if you don’t want to, but the materials for certifications such as the AWS Certified Developer - Associate give you a great introduction to a vast number of services in a short amount of time.

And if you're ever in need of help picking a service, or have questions about how you should run your crap - feel free to reach out to me!

How we migrated an entire AWS Organization to a new one, with no-one noticing!

2022-05-09T00:00:00.000Z

Background

Usually when working with AWS, you typically either work with an existing AWS organization that already has guardrails and services in place, or you’re starting from scratch and building a new one. This last one however, is fairly uncommon and being in a situation where you need to establish these things yourself is something even consultants like myself rarely do. However, something that is even more infrequent, is migrating all the accounts from one organization or MSP to another. Amazon has excellent documentation on how to do some of these steps, but they are all typically from individual actions such as settings up a new organization, and not about moving like this.

So how does the process work, and how would you go about migrating from one MSP to another?

In this post, I will discuss how we migrated a production organization from one MSP to another, and try to keep in mind all the minute annoyances that are important to keep in mind when performing such a process.

Why are we doing this migration?

First, a bit of background on why we’re doing this entire process in the first place.

When the organization I consult for, first started to dip their toes into AWS and cloud production workloads, the parent organization already had an agreement with a MSP that had a good existing security framework and procedures in place. So with limited AWS competency in the organization, it was natural for the company to utilize the help available from the MSP and the guardrails they had in place. However, as the production workloads grew and competency and maturity in the organization grew along with the increased production workloads, the existing framework and procedures to get certain things done felt clunky and slow, resulting in less productive- and happy developers.

After doing more research, and figuring out the parent organization had reseller agreements with other partners that allowed for more options for us as a company, the decision was made — we were to migrate the entire organization to a new provider. This — as it turns out — is not a simple process…

Prep Work

In theory, AWS organizations are independent of the accounts, and it should be a simple process to move from one MSP to another. However, in reality, there are a lot of steps you need to complete before, during and after the migration. A couple of key things to remember:

In our case, the MSP had the responsibility for the accounts, and set the root email and MFA. This meant that because of how AWS accounts work, we needed to create unique email accounts (or aliases) for each account we owned. Not a huge problem, but if you’re using Office 365, each account can only have a very limited number of aliases meaning, we ended up with a huge number of mail accounts that someone needs to manage and keep track of.
MFA needs to be removed, and then re-applied. For a single account? No big deal. For 50+ accounts, this becomes real tiring, real quick. You also need to establish a system for keeping track of your MFA (or virtual MFA) devices, and which device belongs to which account — then, you need to secure these devices so no-one that should not have access to them, has access.
You actually need to submit a PDF document to AWS where you need to apply to do the migration. This takes time. You first need to contact AWS support, that sends you the PDF that needs to be signed by a valid signatory from both the outgoing MSP and the new owner, then sent back to AWS. Once submitted, it takes 2–4 weeks for them to process it. This is a step that a lot of MSPs are not aware of, and is vital you don’t forget…
In most cases, if the outgoing MSP had AWS Security Hub, Config, etc. security policies in place, they will most likely remove them before the move. This means you need to ensure you have a plan in place for how to deal with security in the new organization. This is a major task!
There are billing considerations to be taken as well, an AWS Solutions Advisor will give out specific details here, but it is recommended to do the migration on the last day, or the first day of the month.

With plans laid out for all these hurdles, we set a date for a couple of days where we would do the migration agreed upon by the outgoing and incoming MSPs, and AWS.

Migrating

The migration itself, turns out is a pretty smooth process, as long as you have done the prep work, and the outgoing MSP know what they are doing.

On each account, SCPs were removed, MFA was removed, the old organization was left, then invited to the new one. Once complete, a new MFA device was added to the root user to complete the initial migration.

On average, it took somewhere between 5–20 minutes per account we migrated, depending on how many policies were applied to the account and was an incredibly smooth process. The biggest annoyance was reapplying MFA, which is not fun when dealing with this many accounts.

In the end, we spent about a day and a half migrating all accounts, with zero downtime for any applications, and no users noticing that we even did anything, except that guardrails that had been in place were gone.

Post Migration

With the main migration complete, the next step is to establish and rebuild new SCPs, AWS Control Tower, and Guardrails to meet the standards of the new organization.

Although this is a pretty smooth and straight forward process, we discovered one major hurdle that was pretty annoying. AWS Config uses “Delivery channels” and “Configuration Recorders” to do it’s thing, but didn’t automatically delete these when Config was removed. You need to manually remove them. No big deal, right…? Except that they are applied to each region AWS Config is enabled in, and can only be removed via the AWS CLI or SDK… “No big deal”, you say, AWS Config can just overwrite the existing ones? Nope. They need to be removed. One by one. In each region. Note, you only need to remove the default ones, if you have your own custom, they should be fine.

This can somewhat be automated, but you need to swap credentials between each account, so it isn’t amazing.

Checking if channels and recorders exist for the account

Deleting potential objects — note, you need to do this for EACH region:

Once each recorder and delivery channel are deleted, you can enroll the account in AWS Config and Control Tower again!

Conclusion

Migrating an entire AWS Organization from one MSP to another takes a lot of time. It isn’t necessarily that much physical work, but there is a lot of waiting for each step in the process to complete, and there are a lot of them.

However, if you do the prep work prior to the migration starting, creating a plan and ensuring everything is in place prior to the move, it turns out it can be a pretty smooth process!

I highly recommend reaching out to an AWS Solutions Architect prior to such a migration so they can oversee the process and ensure that it goes smoothly, because there are a lot of steps that can be screwed up, and could lead to potential problems. Fortunately, in most cases it should be pretty simple!

How you can try out new technology in the cloud - for FREE!

2021-09-06T00:00:00.000Z

Background

In IT, we do a lot of “lab” work. Trying new things, testing out a theory or an application that you have developed, or simply learning a new technology, such as Linux or Windows server.

This labbing can, of course, be done in a multitude of ways. On your computer, on an old server or even a Raspberry Pi! All of these methods work very well, however, they all have some downsides: your computer might run slowly because you’re running a virtual machine on top of your ordinary OS. Old servers consume a lot of power (and noise), and a Raspberry Pi might not support the software you want to run, or might not be powerful enough for your task. On top of all of that, a home server or Raspberry Pi might not be available to you from outside of home without configuring VPNs which adds extra complexity.

There is however, a simple way to allow you to run your lab, from anywhere, for free* - in the cloud, using Amazon Web Services. AWS is the worlds largest public cloud provider, and according to [Forbes (2019)] it accounts for nearly half of the worlds public cloud revenue with almost every Forbes top 100 companies using AWS for some of their digital needs.

Ok, I hear you say, how does this benefit you and your lab? Well - Amazon offers one of the most generous free-tier offers for its new users out of all the large public cloud providers - a full year of free services! And if you know the limitations of the free tier, you can run clusters of virtual machines at a time if you want to try out auto-scaling instances, a platitude of databases, or want to crunch a dataset or two a month. Or, maybe you just want to host a website? You can do that too, for free, for a full year!

** = AWS offers a specific “free tier” of products for 12 months for new accounts with set limitations to usage per month. This does not include all products, but is limited to specific ones. We strongly recommend you configure a billing alert to warn you if any cost accumulate on your account. Refer to this post #FIXME to see how you can configure this sort of alert.

The AWS Free Tier

Amazon Web Services (AWS) has one of the most generous free tiers of the major public cloud providers. It offers a mix of over 100 always free and 12 months free products that you can use to your hearts content to lab out your new applications and it is important that you understand how the free tier works, if you want to avoid being charged for your AWS usage.

Product Naming

If you have never used AWS before, there are some common names you need to be aware of before we get started. And yes - AWS naming of products can be... Hard to understand. If you already have experience with AWS, you can skip this section. (Note: If you already have experience with AWS, you can skip this section.)

Amazon EC2 is the AWS Virtual Machine service. Ever created a virtual machine in VMware, Hyper-V or Azure? This is the same thing. An “EC2 Instance” refers to a virtual machine. Oh, and since it’s called EC2, you might think there was an EC1 at some point? Nope. EC2 stands for Elastic Compute Cloud = EC2.

Amazon S3 is for object storage. What does object storage mean? In S3, you can store images (I use it to store and serve all my screenshots out of), documents, config files, text files, you name it. If it is an object, you can store it here. However, keep in mind, it is NOT a file system! So what happened to S1, and S2 you might say? Again, never existed. 🤦‍♂️ S3 = Simple Storage Service.

AWS Lambda a serverless compute service. Ever wanted to write a function or an API without worrying how to run it? Write a lambda function! The code can be triggered multiple ways, and will run, without you ever configuring a server!

Amazon RDS, AWS managed Relational database service! Not only does it save you from having to become a Windows or Linux server guru just to get a simple database running, it might also even be cheaper with out the overhead of having to have an operating system as well. It also out of the box supports scaling, high availability, and can be much more secure than you having to become a leet hacker yourself.

DynamoDB SQL not your thing? DynamoDB is for you then! A NoSQL database with MongoDB support! If you ask MongoDB, they’ll say that their cloud offering is much better than DynamoDB, and you can’t compare the two, but from experience... DynamoDB is pretty good for 9 out of 10 use cases!

VPC, Virtual Private Cloud. A VPC allows you to segment your resources into multiple networks, completely segmented from each other. Think of this, kinda like a VLAN, except it’s not a VLAN. Confusing, I know, but you don’t need to worry too much about it for now.

And that’s only barely touching the surface of AWS products. As of writing, in September 2021, there are over 200 AWS services, with more being added almost monthly. In 2020, a fantastic [YouTube Video] was made of every AWS product. The week after it was made, another 10 services were launched.

Free Tier Offers

For the most up to date, and complete list of current free tier offers, make sure to check out Amazons own page on the different products. Keep in mind that although this article does list a few, very long running and most common free tier offers, they may change at Amazons discretion, so make sure to double check [https://aws.amazon.com/free/] for the up to date info.

Amazon EC2: t2/t3.micro - 750 hours/month for 12 months.
AWS offers you 750 hours per month, for 12 months. You can chose to spend these 750 hours however you like, if you want to run 10 instances for 75 hours, or 75 instances for 10 hours, you can do so, or you can run 1 instance for an entire month, 24/7. This offer is limited to the t3.micro instance, which is equivalent to 2 vCPUs and 1GB RAM. Before you think that this is not enough: A single t3.micro instance, can easily serve a hundred thousand monthly web users, if configured well and using managed RDS.

Amazon RDS: t3.micro - 750 hours/month for 12 months.
Same deal as with EC2, you can run a single DB server 24/7 for a month or 75 instances for 10 hours. Combined with an EC2 web server, this combination is very powerful.

AWS Lambda: 1 million free requests/month, always free!
Pretty self explanatory! 1 million requests a month for free! Always!

Amazon S3 5 GB storage for 12 months.
You get 5 GB of object storage for free, however, one metric to keep in mind here is that you do get a limited number of GET and PUT requests: **20,000 GET **requests, 2,000 PUT requests.

Again, there are over 100 products available on the free tier, so make sure to check out Amazons page to see details on each subject.

Before We Get Started

With an understanding of how the free tier works, it’s now time to get started with some labbing!

There are two ways to follow along with the labs in this article:

Follow along in the AWS Console, and click the appropriate buttons to complete the configuration.
Use the AWS CLI for your operating system. Everything you can do in the AWS Console, you can do with the AWS CLI from your computer. Learning the AWS CLI can be very useful and allows for scripting of commands and automatically building and tearing down configurations once you are done.

There is nothing wrong with either alternatives, and they are both perfectly legitimate ways of interacting with AWS. In fact, most people work with a combination of both tools depending on what task they are doing. However: We do recommend that you configure your AWS CLI and try it out, even if you want to primarily use the web console, as there are some things, like CloudFormation that require that you use the CLI.

Terraform

Finally, there is one more way to do all configuration, which I will touch on in a separate article, as it is another huge project in and of it self - Terraform. Terraform is an Infrastructure as Code (IAC) tool, a declarative programming language that allows you to tell the code what you want it to look like, then it will do the work for you. You can tell it to start a specific instance, with X storage and Y RAM etc. and it will do it for you. It will also allow you to destroy everything, with a single command line command. THIS is what makes labbing in the cloud extremely easy. Watch out for part two of this article series for that!

Let’s get labbing!

In the first drafts of this blog, I had a step-by-step guide included directly in the post, but it very quickly becomes very overwhelming in a post, so as a result, I'm going to link to the same workshop over on GitHub: https://github.com/Bergen-Cloud-User-Group/01-Introduction-To-Cloud-Computing/tree/master/Workshop/01 - Stage 1

I recommend you try it out over on GitHub, as it is a great and fun exercise to try out! Good ouck!