<?xml version="1.0" encoding="UTF-8"?><rss version="2.0"
	xmlns:content="http://purl.org/rss/1.0/modules/content/"
	xmlns:wfw="http://wellformedweb.org/CommentAPI/"
	xmlns:dc="http://purl.org/dc/elements/1.1/"
	xmlns:atom="http://www.w3.org/2005/Atom"
	xmlns:sy="http://purl.org/rss/1.0/modules/syndication/"
	xmlns:slash="http://purl.org/rss/1.0/modules/slash/"
	>

<channel>
	<title>OIDC &#8211; Blog of Kliment Andreev &#8211; A place so I won&#039;t forget things</title>
	<atom:link href="https://blog.andreev.it/tag/oidc/feed/" rel="self" type="application/rss+xml" />
	<link>https://blog.andreev.it</link>
	<description></description>
	<lastBuildDate>Sun, 21 Jan 2024 13:41:42 +0000</lastBuildDate>
	<language>en-US</language>
	<sy:updatePeriod>
	hourly	</sy:updatePeriod>
	<sy:updateFrequency>
	1	</sy:updateFrequency>
	
	<item>
		<title>AWS: Install EKS cluster and Ingress Controller using Terraform and GitLab using OIDC and web identity</title>
		<link>https://blog.andreev.it/2023/11/aws-install-eks-cluster-and-ingress-controller-using-terraform-and-gitlab-using-oidc-and-web-identity/</link>
					<comments>https://blog.andreev.it/2023/11/aws-install-eks-cluster-and-ingress-controller-using-terraform-and-gitlab-using-oidc-and-web-identity/#respond</comments>
		
		<dc:creator><![CDATA[Kliment Andreev]]></dc:creator>
		<pubDate>Tue, 21 Nov 2023 18:39:16 +0000</pubDate>
				<category><![CDATA[AWS]]></category>
		<category><![CDATA[Cloud]]></category>
		<category><![CDATA[Containers]]></category>
		<category><![CDATA[DevOps]]></category>
		<category><![CDATA[Kubernetes]]></category>
		<category><![CDATA[EKS]]></category>
		<category><![CDATA[GitLab]]></category>
		<category><![CDATA[ingress]]></category>
		<category><![CDATA[OIDC]]></category>
		<guid isPermaLink="false">https://blog.andreev.it/?p=9581</guid>

					<description><![CDATA[In this post, I&#8217;ll explain how to create an EKS cluster in AWS and&#8230;]]></description>
										<content:encoded><![CDATA[<div id="bsf_rt_marker"></div><p>In this post, I&#8217;ll explain how to create an EKS cluster in AWS and add an Ingress Controller. We&#8217;ll use GitLab as a repo and execute the CI/CD pipeline from there.<br />
For this you&#8217;ll need:</p>
<ul>
<li>GitLab and AWS account</li>
<li>Terraform, AWS CLI, helm installed</li>
</ul>
<h1>AWS</h1>
<p>From the IAM console, create a new Identity Provider in AWS and use <strong>https://gitlab.com</strong> for both <em>Provider URL</em> and <em>Audience</em>. For self hosted GitLab use <em>gitlab.domain.com</em> or whatever is your GitLab URL.<br />
Click on <strong>Get thumbprint</strong> before clicking on <strong>Add provider</strong>.<br />
<a href="https://blog.andreev.it/wp-content/uploads/2023/11/P167-02.png"><img fetchpriority="high" decoding="async" src="https://blog.andreev.it/wp-content/uploads/2023/11/P167-02-1020x1024.png" alt="" width="1020" height="1024" class="aligncenter size-large wp-image-9585" srcset="https://blog.andreev.it/wp-content/uploads/2023/11/P167-02-1020x1024.png 1020w, https://blog.andreev.it/wp-content/uploads/2023/11/P167-02-300x300.png 300w, https://blog.andreev.it/wp-content/uploads/2023/11/P167-02-150x150.png 150w, https://blog.andreev.it/wp-content/uploads/2023/11/P167-02-768x771.png 768w, https://blog.andreev.it/wp-content/uploads/2023/11/P167-02-1170x1175.png 1170w, https://blog.andreev.it/wp-content/uploads/2023/11/P167-02-585x587.png 585w, https://blog.andreev.it/wp-content/uploads/2023/11/P167-02.png 1288w" sizes="(max-width: 1020px) 100vw, 1020px" /></a><br />
In the IAM console, create a role. I&#8217;ll name the role <strong>rolGitLab</strong>, name it whatever you like. When you create the policy, choose <strong>Custom trust policy</strong>.<br />
<a href="https://blog.andreev.it/wp-content/uploads/2023/11/P167-04.png"><img decoding="async" src="https://blog.andreev.it/wp-content/uploads/2023/11/P167-04-1024x647.png" alt="" width="1024" height="647" class="aligncenter size-large wp-image-9587" srcset="https://blog.andreev.it/wp-content/uploads/2023/11/P167-04-1024x647.png 1024w, https://blog.andreev.it/wp-content/uploads/2023/11/P167-04-300x190.png 300w, https://blog.andreev.it/wp-content/uploads/2023/11/P167-04-768x485.png 768w, https://blog.andreev.it/wp-content/uploads/2023/11/P167-04-1536x971.png 1536w, https://blog.andreev.it/wp-content/uploads/2023/11/P167-04-1170x739.png 1170w, https://blog.andreev.it/wp-content/uploads/2023/11/P167-04-585x370.png 585w, https://blog.andreev.it/wp-content/uploads/2023/11/P167-04.png 1725w" sizes="(max-width: 1024px) 100vw, 1024px" /></a><br />
Now, here is what you have to decide. If you want to manage the cluster from GitLab only, you can ommit lines 16-23 where I allow the IAM user called gitlab to assume this role. But if you want some IAM user to manage the cluster using aws cli or kubectl, add the ARN of that IAM user in line 20. Add <em>AdministratorAccess </em>to that gitlab user but you can fine-tune those policies later. If you want to manage the cluster from GitLab, you have to register an agent and provide a config file. You can read how to do that <a href="https://docs.gitlab.com/ee/user/clusters/agent/" rel="noopener" target="_blank">here</a>.<br />
Replace the default custom trust policy with this JSON.</p>
<pre class="brush: xml; highlight: [7,12,20]; title: ; notranslate">
{
	&quot;Version&quot;: &quot;2012-10-17&quot;,
	&quot;Statement&quot;: &#x5B;
		{
			&quot;Effect&quot;: &quot;Allow&quot;,
			&quot;Principal&quot;: {
				&quot;Federated&quot;: &quot;arn:aws:iam::123456789012:oidc-provider/gitlab.com&quot;
			},
			&quot;Action&quot;: &quot;sts:AssumeRoleWithWebIdentity&quot;,
			&quot;Condition&quot;: {
				&quot;StringLike&quot;: {
					&quot;gitlab.com:sub&quot;: &quot;project_path:*/eks-gitlab:ref_type:branch:ref:main&quot;
				}
			}
		},
		{
			&quot;Sid&quot;: &quot;&quot;,
			&quot;Effect&quot;: &quot;Allow&quot;,
			&quot;Principal&quot;: {
				&quot;AWS&quot;: &quot;arn:aws:iam::123456789012:user/gitlab&quot;
			},
			&quot;Action&quot;: &quot;sts:AssumeRole&quot;
		}
	]
}
</pre>
<p>In line 7 and 20, change the <em>123456789012 </em>with your account number and in line 7 change <em>gitlab.com</em> with however you named the Identity Provider (IdP). In line 12 replace <em>eks-gitlab </em>with how you plan to name your GitLab project. In line 20 change the <em>gitlab </em>user to however you named your IAM user to access the EKS cluster. Click <strong>Next </strong>and when asked for a permission policy, choose <strong>AdministratorAccess</strong>. Best practice is to limit access to only what&#8217;s needed, but EKS has so many policies, it will take a lot to list them all.<br />
In case you created the gitlab IAM user, attach the AdministratorPolicy (fine tune later) and this inline policy. Change the ARN in line 7. Name this inline policy something like <em>polAssumeGitLabRole</em>. </p>
<pre class="brush: bash; highlight: [7]; title: ; notranslate">
{
    &quot;Version&quot;: &quot;2012-10-17&quot;,
    &quot;Statement&quot;: &#x5B;
        {
            &quot;Effect&quot;: &quot;Allow&quot;,
            &quot;Action&quot;: &quot;sts:AssumeRole&quot;,
            &quot;Resource&quot;: &quot;arn:aws:iam::123456789012:role/rolGitLab&quot;
        }
    ]
}
</pre>
<h1>GitLab</h1>
<p>Create a new blank project in Gitlab with the default settings. In my case the project name is called <strong>eks-gitlab</strong>.<br />
<a href="https://blog.andreev.it/wp-content/uploads/2023/11/P167-01.png"><img decoding="async" src="https://blog.andreev.it/wp-content/uploads/2023/11/P167-01-1024x737.png" alt="" width="1024" height="737" class="aligncenter size-large wp-image-9615" srcset="https://blog.andreev.it/wp-content/uploads/2023/11/P167-01-1024x737.png 1024w, https://blog.andreev.it/wp-content/uploads/2023/11/P167-01-300x216.png 300w, https://blog.andreev.it/wp-content/uploads/2023/11/P167-01-768x553.png 768w, https://blog.andreev.it/wp-content/uploads/2023/11/P167-01-1170x843.png 1170w, https://blog.andreev.it/wp-content/uploads/2023/11/P167-01-585x421.png 585w, https://blog.andreev.it/wp-content/uploads/2023/11/P167-01.png 1497w" sizes="(max-width: 1024px) 100vw, 1024px" /></a><br />
Go to <strong>Settings </strong>on the left, then <strong>CI/CD</strong> and then <strong>Variables</strong>. Add these two variables. </p>
<ul>
<li>AWS_PROFILE and a value of <strong>oidc</strong>. Yes, the value is <strong>oidc</strong>. Just type these 4 characters for the value.</li>
<li>ROLE_ARN and a value of the role ARN from the role you just created for <strong>rolGitLab</strong></li>
</ul>
<p>Create a <strong>.gitlab-ci.yml</strong> file with the following content. Change the domain for <em>gitlab.com</em> at line 7 if you use self-hosted GitLab.</p>
<pre class="brush: yaml; highlight: [7]; title: ; notranslate">
assume role:
  image: 
    name: amazon/aws-cli:latest
    entrypoint: &#x5B;&quot;&quot;]
  id_tokens:
    GITLAB_OIDC_TOKEN:
      aud: https://gitlab.com
  before_script:
    - mkdir -p ~/.aws
    - echo &quot;${GITLAB_OIDC_TOKEN}&quot; &gt; /tmp/web_identity_token
    - echo -e &quot;&#x5B;profile oidc]\nrole_arn=${ROLE_ARN}\nweb_identity_token_file=/tmp/web_identity_token&quot; &gt; ~/.aws/config
  script:
      - aws sts get-caller-identity
      - aws s3 ls
</pre>
<p>By the time you commit changes, the pipeline will start and if everything is OK, you&#8217;ll see your S3 buckets in the logs of the job if you go to <strong>Build | Jobs</strong>.<br />
<a href="https://blog.andreev.it/wp-content/uploads/2023/11/P167-05.png"><img loading="lazy" decoding="async" src="https://blog.andreev.it/wp-content/uploads/2023/11/P167-05-878x1024.png" alt="" width="878" height="1024" class="aligncenter size-large wp-image-9617" srcset="https://blog.andreev.it/wp-content/uploads/2023/11/P167-05-878x1024.png 878w, https://blog.andreev.it/wp-content/uploads/2023/11/P167-05-257x300.png 257w, https://blog.andreev.it/wp-content/uploads/2023/11/P167-05-768x895.png 768w, https://blog.andreev.it/wp-content/uploads/2023/11/P167-05-585x682.png 585w, https://blog.andreev.it/wp-content/uploads/2023/11/P167-05.png 1012w" sizes="(max-width: 878px) 100vw, 878px" /></a><br />
One more thing related to Terraform state as we&#8217;ll keep the state in GitLab is that we have to create an <strong>Access Token</strong>. If you have a self-hosted or paid GitLab account, create an <em>Access Token</em> from the <em>Project | Settings | Access Tokens</em>.<br />
If you have a free GitLab account, you have to create a <strong>Personal Access Token</strong> and you do that if you click on your avatar in the upper left corner, then select <strong>Edit Profile</strong> and then click <strong>Access Tokens</strong> on the left. Click <strong>Add new token</strong>, name it <strong>GITLAB_PAT</strong> and select API only. Click <strong>Create personal access token</strong>. Once the token has been created click on the eye icon where it says <em>Your new personal access token</em> and copy &#038; paste somewhere safe.<br />
<a href="https://blog.andreev.it/wp-content/uploads/2023/11/P167-03.png"><img loading="lazy" decoding="async" src="https://blog.andreev.it/wp-content/uploads/2023/11/P167-03-852x1024.png" alt="" width="852" height="1024" class="aligncenter size-large wp-image-9620" srcset="https://blog.andreev.it/wp-content/uploads/2023/11/P167-03-852x1024.png 852w, https://blog.andreev.it/wp-content/uploads/2023/11/P167-03-250x300.png 250w, https://blog.andreev.it/wp-content/uploads/2023/11/P167-03-768x923.png 768w, https://blog.andreev.it/wp-content/uploads/2023/11/P167-03-585x703.png 585w, https://blog.andreev.it/wp-content/uploads/2023/11/P167-03.png 964w" sizes="(max-width: 852px) 100vw, 852px" /></a><br />
Go to the project settings and add these 2 variables.</p>
<ul>
<li>TF_USERNAME which is your username, click on the avatar, look at the @ under your name, your username is without the @ sign</li>
<li>TF_PASSWORD which is the value of your personal access token you just created</li>
</ul>
<h1>Local machine</h1>
<p>Now that we know the pipeline is working properly, let&#8217;s clone the project on our local machine.<br />
If you plan to manage the cluster from your local machine, now it&#8217;s time to configure the <strong>gitlab </strong>IAM user on your local machine using access keys.<br />
You need to have a valid <a href="https://docs.gitlab.com/ee/user/ssh.html" rel="noopener" target="_blank">key </a>in order to clone your private repo.<br />
Upload your .pub key to GitLab by clicking your avatar, then <strong>Edit Profile</strong>, click <strong>SSH Keys</strong> and then <strong>Add new key</strong>.<br />
Finally, check the connection with the following command. It doesn&#8217;t matter what your local Linux user is, just use git@gitlab.com as the username@hostname. Type as is. </p>
<pre class="brush: bash; highlight: [1]; title: ; notranslate">
ssh -T git@gitlab.com
Welcome to GitLab, @&#x5B;your_gitlab_username_here]
</pre>
<p>Once you clone the repo add the following files locally.<br />
<strong>FILE: .gitignore</strong></p>
<pre class="brush: bash; collapse: true; light: false; title: ; toolbar: true; notranslate">
# Local .terraform directories
**/.terraform/*

# .tfstate files
*.tfstate
*.tfstate.*

# Crash log files
crash.log
crash.*.log

# Exclude all .tfvars files, which are likely to contain sensitive data, such as
# password, private keys, and other secrets. These should not be part of version 
# control as they are data points which are potentially sensitive and subject 
# to change depending on the environment.
*.tfvars
*.tfvars.json

# Ignore override files as they are usually used to override resources locally and so
# are not checked in
override.tf
override.tf.json
*_override.tf
*_override.tf.json

# Include override files you do wish to add to version control using negated pattern
# !example_override.tf

# Include tfplan files to ignore the plan output of command: terraform plan -out=tfplan
# example: *tfplan*

# Ignore CLI configuration files
.terraformrc
terraform.rc
</pre>
<p><strong>FILE: LICENSE</strong></p>
<pre class="brush: plain; collapse: true; light: false; title: ; toolbar: true; notranslate">
This is free and unencumbered software released into the public domain.

Anyone is free to copy, modify, publish, use, compile, sell, or
distribute this software, either in source code form or as a compiled
binary, for any purpose, commercial or non-commercial, and by any
means.

In jurisdictions that recognize copyright laws, the author or authors
of this software dedicate any and all copyright interest in the
software to the public domain. We make this dedication for the benefit
of the public at large and to the detriment of our heirs and
successors. We intend this dedication to be an overt act of
relinquishment in perpetuity of all present and future rights to this
software under copyright law.

THE SOFTWARE IS PROVIDED &quot;AS IS&quot;, WITHOUT WARRANTY OF ANY KIND,
EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.
IN NO EVENT SHALL THE AUTHORS BE LIABLE FOR ANY CLAIM, DAMAGES OR
OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
OTHER DEALINGS IN THE SOFTWARE.

For more information, please refer to &lt;https://unlicense.org&gt;
</pre>
<p><strong>FILE: alb_ingress_iam_policy.json</strong></p>
<pre class="brush: xml; collapse: true; light: false; title: ; toolbar: true; notranslate">
{
    &quot;Version&quot;: &quot;2012-10-17&quot;,
    &quot;Statement&quot;: &#x5B;
        {
            &quot;Effect&quot;: &quot;Allow&quot;,
            &quot;Action&quot;: &#x5B;
                &quot;iam:CreateServiceLinkedRole&quot;
            ],
            &quot;Resource&quot;: &quot;*&quot;,
            &quot;Condition&quot;: {
                &quot;StringEquals&quot;: {
                    &quot;iam:AWSServiceName&quot;: &quot;elasticloadbalancing.amazonaws.com&quot;
                }
            }
        },
        {
            &quot;Effect&quot;: &quot;Allow&quot;,
            &quot;Action&quot;: &#x5B;
                &quot;ec2:DescribeAccountAttributes&quot;,
                &quot;ec2:DescribeAddresses&quot;,
                &quot;ec2:DescribeAvailabilityZones&quot;,
                &quot;ec2:DescribeInternetGateways&quot;,
                &quot;ec2:DescribeVpcs&quot;,
                &quot;ec2:DescribeVpcPeeringConnections&quot;,
                &quot;ec2:DescribeSubnets&quot;,
                &quot;ec2:DescribeSecurityGroups&quot;,
                &quot;ec2:DescribeInstances&quot;,
                &quot;ec2:DescribeNetworkInterfaces&quot;,
                &quot;ec2:DescribeTags&quot;,
                &quot;ec2:GetCoipPoolUsage&quot;,
                &quot;ec2:DescribeCoipPools&quot;,
                &quot;elasticloadbalancing:DescribeLoadBalancers&quot;,
                &quot;elasticloadbalancing:DescribeLoadBalancerAttributes&quot;,
                &quot;elasticloadbalancing:DescribeListeners&quot;,
                &quot;elasticloadbalancing:DescribeListenerCertificates&quot;,
                &quot;elasticloadbalancing:DescribeSSLPolicies&quot;,
                &quot;elasticloadbalancing:DescribeRules&quot;,
                &quot;elasticloadbalancing:DescribeTargetGroups&quot;,
                &quot;elasticloadbalancing:DescribeTargetGroupAttributes&quot;,
                &quot;elasticloadbalancing:DescribeTargetHealth&quot;,
                &quot;elasticloadbalancing:DescribeTags&quot;
            ],
            &quot;Resource&quot;: &quot;*&quot;
        },
        {
            &quot;Effect&quot;: &quot;Allow&quot;,
            &quot;Action&quot;: &#x5B;
                &quot;cognito-idp:DescribeUserPoolClient&quot;,
                &quot;acm:ListCertificates&quot;,
                &quot;acm:DescribeCertificate&quot;,
                &quot;iam:ListServerCertificates&quot;,
                &quot;iam:GetServerCertificate&quot;,
                &quot;waf-regional:GetWebACL&quot;,
                &quot;waf-regional:GetWebACLForResource&quot;,
                &quot;waf-regional:AssociateWebACL&quot;,
                &quot;waf-regional:DisassociateWebACL&quot;,
                &quot;wafv2:GetWebACL&quot;,
                &quot;wafv2:GetWebACLForResource&quot;,
                &quot;wafv2:AssociateWebACL&quot;,
                &quot;wafv2:DisassociateWebACL&quot;,
                &quot;shield:GetSubscriptionState&quot;,
                &quot;shield:DescribeProtection&quot;,
                &quot;shield:CreateProtection&quot;,
                &quot;shield:DeleteProtection&quot;
            ],
            &quot;Resource&quot;: &quot;*&quot;
        },
        {
            &quot;Effect&quot;: &quot;Allow&quot;,
            &quot;Action&quot;: &#x5B;
                &quot;ec2:AuthorizeSecurityGroupIngress&quot;,
                &quot;ec2:RevokeSecurityGroupIngress&quot;
            ],
            &quot;Resource&quot;: &quot;*&quot;
        },
        {
            &quot;Effect&quot;: &quot;Allow&quot;,
            &quot;Action&quot;: &#x5B;
                &quot;ec2:CreateSecurityGroup&quot;
            ],
            &quot;Resource&quot;: &quot;*&quot;
        },
        {
            &quot;Effect&quot;: &quot;Allow&quot;,
            &quot;Action&quot;: &#x5B;
                &quot;ec2:CreateTags&quot;
            ],
            &quot;Resource&quot;: &quot;arn:aws:ec2:*:*:security-group/*&quot;,
            &quot;Condition&quot;: {
                &quot;StringEquals&quot;: {
                    &quot;ec2:CreateAction&quot;: &quot;CreateSecurityGroup&quot;
                },
                &quot;Null&quot;: {
                    &quot;aws:RequestTag/elbv2.k8s.aws/cluster&quot;: &quot;false&quot;
                }
            }
        },
        {
            &quot;Effect&quot;: &quot;Allow&quot;,
            &quot;Action&quot;: &#x5B;
                &quot;ec2:CreateTags&quot;,
                &quot;ec2:DeleteTags&quot;
            ],
            &quot;Resource&quot;: &quot;arn:aws:ec2:*:*:security-group/*&quot;,
            &quot;Condition&quot;: {
                &quot;Null&quot;: {
                    &quot;aws:RequestTag/elbv2.k8s.aws/cluster&quot;: &quot;true&quot;,
                    &quot;aws:ResourceTag/elbv2.k8s.aws/cluster&quot;: &quot;false&quot;
                }
            }
        },
        {
            &quot;Effect&quot;: &quot;Allow&quot;,
            &quot;Action&quot;: &#x5B;
                &quot;ec2:AuthorizeSecurityGroupIngress&quot;,
                &quot;ec2:RevokeSecurityGroupIngress&quot;,
                &quot;ec2:DeleteSecurityGroup&quot;
            ],
            &quot;Resource&quot;: &quot;*&quot;,
            &quot;Condition&quot;: {
                &quot;Null&quot;: {
                    &quot;aws:ResourceTag/elbv2.k8s.aws/cluster&quot;: &quot;false&quot;
                }
            }
        },
        {
            &quot;Effect&quot;: &quot;Allow&quot;,
            &quot;Action&quot;: &#x5B;
                &quot;elasticloadbalancing:CreateLoadBalancer&quot;,
                &quot;elasticloadbalancing:CreateTargetGroup&quot;
            ],
            &quot;Resource&quot;: &quot;*&quot;,
            &quot;Condition&quot;: {
                &quot;Null&quot;: {
                    &quot;aws:RequestTag/elbv2.k8s.aws/cluster&quot;: &quot;false&quot;
                }
            }
        },
        {
            &quot;Effect&quot;: &quot;Allow&quot;,
            &quot;Action&quot;: &#x5B;
                &quot;elasticloadbalancing:CreateListener&quot;,
                &quot;elasticloadbalancing:DeleteListener&quot;,
                &quot;elasticloadbalancing:CreateRule&quot;,
                &quot;elasticloadbalancing:DeleteRule&quot;
            ],
            &quot;Resource&quot;: &quot;*&quot;
        },
        {
            &quot;Effect&quot;: &quot;Allow&quot;,
            &quot;Action&quot;: &#x5B;
                &quot;elasticloadbalancing:AddTags&quot;,
                &quot;elasticloadbalancing:RemoveTags&quot;
            ],
            &quot;Resource&quot;: &#x5B;
                &quot;arn:aws:elasticloadbalancing:*:*:targetgroup/*/*&quot;,
                &quot;arn:aws:elasticloadbalancing:*:*:loadbalancer/net/*/*&quot;,
                &quot;arn:aws:elasticloadbalancing:*:*:loadbalancer/app/*/*&quot;
            ],
            &quot;Condition&quot;: {
                &quot;Null&quot;: {
                    &quot;aws:RequestTag/elbv2.k8s.aws/cluster&quot;: &quot;true&quot;,
                    &quot;aws:ResourceTag/elbv2.k8s.aws/cluster&quot;: &quot;false&quot;
                }
            }
        },
        {
            &quot;Effect&quot;: &quot;Allow&quot;,
            &quot;Action&quot;: &#x5B;
                &quot;elasticloadbalancing:AddTags&quot;,
                &quot;elasticloadbalancing:RemoveTags&quot;
            ],
            &quot;Resource&quot;: &#x5B;
                &quot;arn:aws:elasticloadbalancing:*:*:listener/net/*/*/*&quot;,
                &quot;arn:aws:elasticloadbalancing:*:*:listener/app/*/*/*&quot;,
                &quot;arn:aws:elasticloadbalancing:*:*:listener-rule/net/*/*/*&quot;,
                &quot;arn:aws:elasticloadbalancing:*:*:listener-rule/app/*/*/*&quot;
            ]
        },
        {
            &quot;Effect&quot;: &quot;Allow&quot;,
            &quot;Action&quot;: &#x5B;
                &quot;elasticloadbalancing:ModifyLoadBalancerAttributes&quot;,
                &quot;elasticloadbalancing:SetIpAddressType&quot;,
                &quot;elasticloadbalancing:SetSecurityGroups&quot;,
                &quot;elasticloadbalancing:SetSubnets&quot;,
                &quot;elasticloadbalancing:DeleteLoadBalancer&quot;,
                &quot;elasticloadbalancing:ModifyTargetGroup&quot;,
                &quot;elasticloadbalancing:ModifyTargetGroupAttributes&quot;,
                &quot;elasticloadbalancing:DeleteTargetGroup&quot;
            ],
            &quot;Resource&quot;: &quot;*&quot;,
            &quot;Condition&quot;: {
                &quot;Null&quot;: {
                    &quot;aws:ResourceTag/elbv2.k8s.aws/cluster&quot;: &quot;false&quot;
                }
            }
        },
        {
            &quot;Effect&quot;: &quot;Allow&quot;,
            &quot;Action&quot;: &#x5B;
                &quot;elasticloadbalancing:AddTags&quot;
            ],
            &quot;Resource&quot;: &#x5B;
                &quot;arn:aws:elasticloadbalancing:*:*:targetgroup/*/*&quot;,
                &quot;arn:aws:elasticloadbalancing:*:*:loadbalancer/net/*/*&quot;,
                &quot;arn:aws:elasticloadbalancing:*:*:loadbalancer/app/*/*&quot;
            ],
            &quot;Condition&quot;: {
                &quot;StringEquals&quot;: {
                    &quot;elasticloadbalancing:CreateAction&quot;: &#x5B;
                        &quot;CreateTargetGroup&quot;,
                        &quot;CreateLoadBalancer&quot;
                    ]
                },
                &quot;Null&quot;: {
                    &quot;aws:RequestTag/elbv2.k8s.aws/cluster&quot;: &quot;false&quot;
                }
            }
        },
        {
            &quot;Effect&quot;: &quot;Allow&quot;,
            &quot;Action&quot;: &#x5B;
                &quot;elasticloadbalancing:RegisterTargets&quot;,
                &quot;elasticloadbalancing:DeregisterTargets&quot;
            ],
            &quot;Resource&quot;: &quot;arn:aws:elasticloadbalancing:*:*:targetgroup/*/*&quot;
        },
        {
            &quot;Effect&quot;: &quot;Allow&quot;,
            &quot;Action&quot;: &#x5B;
                &quot;elasticloadbalancing:SetWebAcl&quot;,
                &quot;elasticloadbalancing:ModifyListener&quot;,
                &quot;elasticloadbalancing:AddListenerCertificates&quot;,
                &quot;elasticloadbalancing:RemoveListenerCertificates&quot;,
                &quot;elasticloadbalancing:ModifyRule&quot;
            ],
            &quot;Resource&quot;: &quot;*&quot;
        }
    ]
}
</pre>
<p><strong>FILE: backend.tf</strong></p>
<pre class="brush: bash; title: ; notranslate">
terraform {
  backend &quot;http&quot; {
  }
}
</pre>
<p><strong>FILE: data.tf</strong></p>
<pre class="brush: bash; title: ; notranslate">
data &quot;aws_availability_zones&quot; &quot;available&quot; {
}

data &quot;aws_eks_cluster&quot; &quot;cluster&quot; {
  name = module.eks.cluster_name
  depends_on = &#x5B;module.eks.cluster_name]
}

data &quot;aws_eks_cluster_auth&quot; &quot;cluster&quot; {
  name = module.eks.cluster_name
  depends_on = &#x5B;module.eks.cluster_name]
}
</pre>
<p><strong>FILE: main.tf NOTE:</strong> Make sure you add lines 40-41 and specify the <strong>rolGitlab</strong> role created earlier. If you don&#8217;t do this when you destroy the cluster, the KMS key for EKS will stay there and there is no way you can delete it unless you call AWS support.</p>
<pre class="brush: bash; collapse: true; light: false; title: ; toolbar: true; notranslate">
# Create a VPC
module &quot;vpc&quot; {
  source  = &quot;terraform-aws-modules/vpc/aws&quot;
  version = &quot;5.0.0&quot;

  name = var.vpcName

  cidr = var.vpcCIDR
  azs  = slice(data.aws_availability_zones.available.names, 0, 3)

  private_subnets = var.privateSubnets
  public_subnets  = var.publicSubnets

  enable_nat_gateway   = var.enable_nat_gateway
  single_nat_gateway   = var.single_nat_gateway
  enable_dns_hostnames = var.enable_dns_hostnames

  public_subnet_tags = {
    &quot;kubernetes.io/cluster/${var.cluster_name}&quot; = &quot;shared&quot;
    &quot;kubernetes.io/role/elb&quot;                    = 1
  }

  private_subnet_tags = {
    &quot;kubernetes.io/cluster/${var.cluster_name}&quot; = &quot;shared&quot;
    &quot;kubernetes.io/role/internal-elb&quot;           = 1
  }
}

# Create an EKS cluster
module &quot;eks&quot; {
  source  = &quot;terraform-aws-modules/eks/aws&quot;
  version = &quot;19.15.3&quot;

  cluster_name    = var.cluster_name
  cluster_version = var.cluster_version

  vpc_id                         = module.vpc.vpc_id
  subnet_ids                     = module.vpc.private_subnets
  cluster_endpoint_public_access = var.cluster_endpoint_public_access
  kms_key_administrators         = &#x5B;&quot;arn:aws:iam::123456789012:role/rolGitLab&quot;]
  kms_key_enable_default_policy  = true

  eks_managed_node_group_defaults = {
    ami_type = var.ami_type
  }

  eks_managed_node_groups = {
    one = {
      name = var.nodeGroup

      instance_types = var.instanceTypes

      min_size     = var.minSize
      max_size     = var.maxSize
      desired_size = var.desiredSize
    }
  }
}

# Create an IAM policy for the ALB ingress
resource &quot;aws_iam_policy&quot; &quot;worker_policy&quot; {
  name        = &quot;worker-policy&quot;
  description = &quot;Worker policy for the ALB Ingress&quot;

  policy = file(var.iamALBIngressPolicyFileName)
}

resource &quot;aws_iam_role_policy_attachment&quot; &quot;additional&quot; {
  for_each = module.eks.eks_managed_node_groups

  policy_arn = aws_iam_policy.worker_policy.arn
  role       = each.value.iam_role_name
}

# Crete the Ingress using Helm chart from AWS
resource &quot;helm_release&quot; &quot;ingress&quot; {
  name       = &quot;ingress&quot;
  chart      = &quot;aws-load-balancer-controller&quot;
  repository = &quot;https://aws.github.io/eks-charts&quot;
  version    = &quot;1.6.1&quot;

  set {
    name  = &quot;autoDiscoverAwsRegion&quot;
    value = &quot;true&quot;
  }

  set {
    name  = &quot;autoDiscoverAwsVpcID&quot;
    value = &quot;true&quot;
  }

  set {
    name  = &quot;clusterName&quot;
    value = module.eks.cluster_name
  }
}
</pre>
<p><strong>FILE: outputs.tf</strong></p>
<pre class="brush: bash; title: ; notranslate">
output &quot;cluster_endpoint&quot; {
  description = &quot;Endpoint for EKS control plane&quot;
  value       = module.eks.cluster_endpoint
}

output &quot;cluster_security_group_id&quot; {
  description = &quot;Security group ids attached to the cluster control plane&quot;
  value       = module.eks.cluster_security_group_id
}

output &quot;region&quot; {
  description = &quot;AWS region&quot;
  value       = var.region
}

output &quot;cluster_name&quot; {
  description = &quot;Kubernetes Cluster Name&quot;
  value       = module.eks.cluster_name
}
</pre>
<p><strong>FILE: providers.tf</strong></p>
<pre class="brush: bash; title: ; notranslate">
provider &quot;aws&quot; {
  region = var.region

  shared_config_files      = &#x5B;&quot;~/.aws/config&quot;]  
  profile                  = &quot;oidc&quot;
}

provider &quot;helm&quot; {
  kubernetes {
    host                   = data.aws_eks_cluster.cluster.endpoint
    token                  = data.aws_eks_cluster_auth.cluster.token
    cluster_ca_certificate = base64decode(data.aws_eks_cluster.cluster.certificate_authority.0.data)
  }
}
</pre>
<p><strong>FILE: variables.tf</strong></p>
<pre class="brush: bash; collapse: true; light: false; title: ; toolbar: true; notranslate">
variable &quot;region&quot; {
  type    = string
  default = &quot;us-east-2&quot;
}

variable &quot;vpcName&quot; {
  type    = string
  default = &quot;vpcFromGitLab&quot;
}

variable &quot;vpcCIDR&quot; {
  type    = string
  default = &quot;10.0.0.0/16&quot;
}

variable &quot;privateSubnets&quot; {
  type    = list(string)
  default = &#x5B;&quot;10.0.1.0/24&quot;, &quot;10.0.2.0/24&quot;, &quot;10.0.3.0/24&quot;]
}

variable &quot;publicSubnets&quot; {
  type    = list(string)
  default = &#x5B;&quot;10.0.4.0/24&quot;, &quot;10.0.5.0/24&quot;, &quot;10.0.6.0/24&quot;]
}

variable &quot;enable_nat_gateway&quot; {
  type    = bool
  default = true
}

variable &quot;single_nat_gateway&quot; {
  type    = bool
  default = false
}

variable &quot;enable_dns_hostnames&quot; {
  type    = bool
  default = true
}

variable &quot;cluster_name&quot; {
  type    = string
  default = &quot;eksFromGitLab&quot;
}

variable &quot;cluster_version&quot; {
  type    = string
  default = &quot;1.27&quot;
}

variable &quot;cluster_endpoint_public_access&quot; {
  type    = bool
  default = true
}

variable &quot;ami_type&quot; {
  type    = string
  default = &quot;AL2_x86_64&quot;
}

variable &quot;nodeGroup&quot; {
  type    = string
  default = &quot;ngFromGitLab&quot;
}

variable &quot;instanceTypes&quot; {
  type    = list(string)
  default = &#x5B;&quot;t3.small&quot;]
}

variable &quot;minSize&quot; {
  type    = number
  default = 1
}

variable &quot;maxSize&quot; {
  type    = number
  default = 3
}

variable &quot;desiredSize&quot; {
  type    = number
  default = 3
}


variable &quot;iamALBIngressPolicyFileName&quot; {
  type    = string
  default = &quot;alb_ingress_iam_policy.json&quot;
}
</pre>
<p>&#8230;and finally your .gitlab-ci.yml file.<br />
<strong>FILE: .gitlab-ci.yml. NOTE: </strong>Remove line 38 if you don&#8217;t use it, don&#8217;t keep it with a comment or you&#8217;ll get an error in the pipeline.</p>
<pre class="brush: bash; highlight: [38]; title: ; notranslate">
stages:          
  - validate
  - plan
  - apply
  - destroy

image:
  name: hashicorp/terraform:1.6
  entrypoint:
    - &quot;/usr/bin/env&quot;
    - &quot;PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin&quot;

variables:
  TF_DIR: ${CI_PROJECT_DIR}
  TF_STATE_NAME: &quot;eks-tf-state&quot;
  TF_ADDRESS: &quot;https://gitlab.com/api/v4/projects/${CI_PROJECT_ID}/terraform/state/${TF_STATE_NAME}&quot;

default:
  id_tokens:
    GITLAB_OIDC_TOKEN:
      aud: https://gitlab.com

  before_script:
    - terraform --version
    - cd ${TF_DIR}
    - terraform init
      -reconfigure 
      -backend-config=&quot;address=${TF_ADDRESS}&quot; 
      -backend-config=&quot;lock_address=${TF_ADDRESS}/lock&quot; 
      -backend-config=&quot;unlock_address=${TF_ADDRESS}/lock&quot; 
      -backend-config=&quot;username=${TF_USERNAME}&quot; 
      -backend-config=&quot;password=${TF_PASSWORD}&quot; 
      -backend-config=&quot;lock_method=POST&quot; 
      -backend-config=&quot;unlock_method=DELETE&quot; 
      -backend-config=&quot;retry_wait_min=5&quot;
    - mkdir -p ~/.aws
    - echo &quot;${GITLAB_OIDC_TOKEN}&quot; &gt; /tmp/web_identity_token
    - # echo &quot;${GITLAB_OIDC_TOKEN}&quot; | base64 - If you want to see the token, reveal it this way and then use base64 --decode to get the real one
    - echo -e &quot;&#x5B;profile oidc]\nrole_arn=${ROLE_ARN}\nweb_identity_token_file=/tmp/web_identity_token&quot; &gt; ~/.aws/config

job-validate:
  stage: validate
  script:
    - terraform validate

job-plan:
  stage: plan
  script:
    - terraform plan -out &quot;tf_plan&quot;
  dependencies:
    - job-validate
  artifacts:
    paths:
      - tf_plan

job-apply:
  stage: apply
  script:
    - terraform apply &quot;tf_plan&quot;
  dependencies:
    - job-plan
  allow_failure: true

job-destroy:
  stage: destroy
  script:
    - terraform destroy  -auto-approve
  dependencies:
    #- job-plan
    #- job-apply
  when: manual
</pre>
<p>Push the code back to Gitlab (<em>git add, git commit, git push</em>) and once everything completes successfully, you&#8217;ll have a working EKS cluster after 10-15 mins.<br />
If you choose to manage the EKS from GitLab, you can skip to the next section. If you decide to manage the cluster outside GitLab and you already have another IAM user added to the role <strong>rolGitLab </strong>as described above, then do the following. Replace account # in line 1.</p>
<pre class="brush: bash; highlight: [1]; title: ; notranslate">
aws eks update-kubeconfig --region us-east-2 --name eksFromGitLab --role-arn arn:aws:iam::123456789012:role/rolGitLab
aws eks list-clusters
eksctl get cluster --name=eksFromGitLab --region us-east-2
kubectl get svc
</pre>
<h1>Manage EKS from GitLab</h1>
<p>Generally, you would want this to be in another project. Our first project was to build the cluster, but our second project builds on top of that and manages the cluster. In addition, the second project also uses the CI/CD pipeline file <strong>.gitlab-ci.yml</strong> so in order to avoid a single complex project &#8211; just create a new project. I&#8217;ll name my new project &#8211; <strong>project1</strong>. Clone the project on your local machine using <em>git clone</em>.<br />
Create an empty config file for the agent on your local machine. It has to be under this folder structure.</p>
<pre class="brush: bash; title: ; notranslate">
mkdir -p .gitlab/agents/eks-agent &amp;&amp; touch .gitlab/agents/eks-agent/config.yaml
</pre>
<p>In my case the name of the agent is <strong>eks-agent</strong>. Use whatever name you want but follow DNS naming <a href="https://docs.gitlab.com/ee/user/clusters/agent/install/index.html" rel="noopener" target="_blank">convention</a>.<br />
Push the local changes to GitLab.<br />
On the left side in GitLab, go to <strong>Operate | Kubernetes clusters</strong> and then click <strong>Connect a cluster</strong> from the upper right corner.<br />
Select the agent from the drop down and click <strong>Register</strong>.<br />
<a href="https://blog.andreev.it/wp-content/uploads/2023/11/P169-02.png"><img loading="lazy" decoding="async" src="https://blog.andreev.it/wp-content/uploads/2023/11/P169-02-1024x304.png" alt="" width="1024" height="304" class="aligncenter size-large wp-image-9767" srcset="https://blog.andreev.it/wp-content/uploads/2023/11/P169-02-1024x304.png 1024w, https://blog.andreev.it/wp-content/uploads/2023/11/P169-02-300x89.png 300w, https://blog.andreev.it/wp-content/uploads/2023/11/P169-02-768x228.png 768w, https://blog.andreev.it/wp-content/uploads/2023/11/P169-02-1170x347.png 1170w, https://blog.andreev.it/wp-content/uploads/2023/11/P169-02-585x174.png 585w, https://blog.andreev.it/wp-content/uploads/2023/11/P169-02.png 1522w" sizes="(max-width: 1024px) 100vw, 1024px" /></a><br />
You&#8217;ll get an access token and a bunch of helm commands to install the agent on the cluster. Copy and paste and run those commands.</p>
<pre class="brush: bash; title: ; notranslate">
helm repo add gitlab https://charts.gitlab.io
helm repo update
helm upgrade --install eks-agent gitlab/gitlab-agent \
    --namespace gitlab-agent-eks-agent \
    --create-namespace \
    --set image.tag=v16.9.0-rc2 \
    --set config.token=glagent-2r_uW63e1w_mkNzR_keU2hmaNhhcKauSpujg8OdLQG74wrF73g \
    --set config.kasAddress=wss://kas.gitlab.com
</pre>
<p>Refresh the GitLab screen after the helm chart has been installed and you&#8217;ll see your cluster connected.<br />
<a href="https://blog.andreev.it/wp-content/uploads/2023/11/P169-03.png"><img loading="lazy" decoding="async" src="https://blog.andreev.it/wp-content/uploads/2023/11/P169-03-1024x174.png" alt="" width="1024" height="174" class="aligncenter size-large wp-image-9769" srcset="https://blog.andreev.it/wp-content/uploads/2023/11/P169-03-1024x174.png 1024w, https://blog.andreev.it/wp-content/uploads/2023/11/P169-03-300x51.png 300w, https://blog.andreev.it/wp-content/uploads/2023/11/P169-03-768x131.png 768w, https://blog.andreev.it/wp-content/uploads/2023/11/P169-03-1536x261.png 1536w, https://blog.andreev.it/wp-content/uploads/2023/11/P169-03-2048x348.png 2048w, https://blog.andreev.it/wp-content/uploads/2023/11/P169-03-1920x326.png 1920w, https://blog.andreev.it/wp-content/uploads/2023/11/P169-03-1170x199.png 1170w, https://blog.andreev.it/wp-content/uploads/2023/11/P169-03-585x99.png 585w" sizes="(max-width: 1024px) 100vw, 1024px" /></a><br />
Now you need to <a href="https://docs.gitlab.com/ee/user/clusters/agent/ci_cd_workflow.html#authorize-the-agent" rel="noopener" target="_blank">authorize</a> the agent to access our project.<br />
In the blank <strong>config.yaml</strong> file on the local machine under <em>.gitlab/agents/eks-agent/</em> directory, add these lines. Change the path the project in line 3 (look at the URL, that&#8217;s your project name after gitlab.com) and change the manifest file if you want in line 6. You can also change the namespace in line 5 as well.</p>
<pre class="brush: yaml; highlight: [3,6]; title: ; notranslate">
gitops:
  manifest_projects:
  - id: andreevkliment/project1
    default_namespace: default
    paths:
      - glob: &#039;manifest.yaml&#039;
</pre>
<p>Then create the manifest file under the root of the project. Save it as <em>manifest.yaml</em> or however you referenced it in line 6 in the previous file.</p>
<pre class="brush: yaml; title: ; notranslate">
apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx-deployment
spec:
  selector:
    matchLabels:
      app: nginx
  replicas: 2
  template:
    metadata:
      labels:
        app: nginx
    spec:
      containers:
      - name: nginx
        image: nginx:1.14.2
        ports:
        - containerPort: 80
</pre>
<p>If you push this code back to GitLab, the agent will execute it and you&#8217;ll (with <em>kubect get pods</em>) see the nginx pods running on your cluster.<br />
Another way of managing the EKS cluster is using <strong>.gitlab-ci.yml</strong> file.<br />
Create <strong>.gitlab-ci.yml</strong> with these lines. Replace your username, project and the agent name in line 7.</p>
<pre class="brush: yaml; highlight: [7]; title: ; notranslate">
deploy:
  image:
    name: bitnami/kubectl:latest
    entrypoint: &#x5B;&#039;&#039;]
  script:
    - kubectl config get-contexts
    - kubectl config use-context andreevkliment/project1:eks-agent
    - kubectl get pods
</pre>
<p>If you push this back to GitLab, the pipeline will start and you&#8217;ll see the pods from the cluster in the logs. This example doesn&#8217;t execute <strong>manifest.yaml</strong>. You can have both ways of interacting with the cluster or you can use them separately.</p>
]]></content:encoded>
					
					<wfw:commentRss>https://blog.andreev.it/2023/11/aws-install-eks-cluster-and-ingress-controller-using-terraform-and-gitlab-using-oidc-and-web-identity/feed/</wfw:commentRss>
			<slash:comments>0</slash:comments>
		
		
			</item>
	</channel>
</rss>
