-
Notifications
You must be signed in to change notification settings - Fork 59
Create guide explaining how to connect OpenCost with AWS and Argo CD K8 resource #1397
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Closed
Closed
Changes from 10 commits
Commits
Show all changes
12 commits
Select commit
Hold shift + click to select a range
c5ba8b2
Implement guide for opencost and argocd
lordsarcastic 36d2339
Fixed failing builds
lordsarcastic 53a0452
Updated integrations index
lordsarcastic cb8c2ba
Made comment fixes on PR
lordsarcastic fb5de63
Merge remote-tracking branch 'origin/main' into port-7910
lordsarcastic bcbf8d9
Merge remote-tracking branch 'origin/main' into port-7910
lordsarcastic ba23408
Fix comments on opencost prs
lordsarcastic c034975
Add missing image
lordsarcastic c1c09c9
Merge remote-tracking branch 'origin/main' into port-7910
lordsarcastic 4d7b12e
Merge branch 'main' into port-7910
PeyGis e8dfac5
Merge remote-tracking branch 'origin/main' into port-7910
lordsarcastic 9ae2a74
Merge remote-tracking branch 'origin/main' into port-7910
lordsarcastic File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
4 changes: 4 additions & 0 deletions
4
docs/build-your-software-catalog/sync-data-to-catalog/argocd/guides/_category_.json
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,4 @@ | ||
{ | ||
"label": "Guides", | ||
"position": 1 | ||
} |
93 changes: 93 additions & 0 deletions
93
...to-catalog/argocd/guides/connect-opencost-resource-with-argocd-k8s-resources.md
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,93 @@ | ||
--- | ||
sidebar_position: 1 | ||
title: Connect Opencost Cloud Cost with ArgoCD/K8 resource in Port | ||
--- | ||
|
||
import PortTooltip from "/src/components/tooltip/tooltip.jsx" | ||
|
||
# Connect Opencost Cloud Cost with ArgoCD/K8 resource in Port | ||
|
||
This guide demonstrates how to link OpenCost cloud cost data with your ArgoCD K8 resources within Port, enabling you to gain insights into the cost breakdown of your infrastructure and applications. | ||
|
||
## Use Case Examples: | ||
- Cost Attribution: Determine which Kubernetes resource contribute the most to your overall cloud costs. | ||
- Budget Management: Track the costs of individual pods or clusters against your budget allocations. | ||
- Anomaly Detection: Identify unusual spikes in resource costs and investigate potential causes. | ||
- Optimization Opportunities: Pinpoint resources that are underutilized or overprovisioned, potentially leading to cost savings. | ||
|
||
::: | ||
|
||
|
||
## Prerequisites | ||
- This guide assumes you have a Port account and that you have finished the [onboarding process](/quickstart). | ||
- Ensure you have [Port's Opencost integration installed and configured in your environment](https://docs.getport.io/build-your-software-catalog/sync-data-to-catalog/cloud-cost/opencost). | ||
- Ensure you have [Port's ArgoCD integration installed and configured in your environment](https://docs.getport.io/build-your-software-catalog/sync-data-to-catalog/argocd/). | ||
|
||
|
||
## Create the ArgoCD K8 relation | ||
|
||
With Opencost and ArgoCD installed, you should see some blueprints created by both integrations. | ||
|
||
Now that Port is synced with our ArgoCD and Opencost resources, let's map the Opencost Resource Allocations to the ArgoCD K8 resource. | ||
|
||
First, we will need to create a [relation](/build-your-software-catalog/customize-integrations/configure-data-model/relate-blueprints/relate-blueprints.md) between our `argocdKubernetesResource` and the corresponding `openCostResourceAllocation`. | ||
|
||
1. Head back to the [Builder](https://app.getport.io/settings/data-model), choose the `Kubernetes Resource | ||
|
||
` <PortTooltip id="blueprint">blueprint</PortTooltip>, and click on `New relation`: | ||
|
||
<img src='/img/build-your-software-catalog/sync-data-to-catalog/argocd/newRelationMapping.png' width='60%' border='1px' /> | ||
|
||
<br/><br/> | ||
|
||
2. Fill out the form like this, then click `Create`: | ||
|
||
<img src='/img/build-your-software-catalog/sync-data-to-catalog/argocd/createRelation.png' width='60%' border='1px' /> | ||
|
||
<br/><br/> | ||
|
||
|
||
Now that the <PortTooltip id="blueprint">blueprints</PortTooltip> are related, we need to assign the relevant ArgoCD Kubernetes resource to each of our Opencost Resource Allocation. This can be done by adding some mapping logic. Go to your [data sources page](https://app.getport.io/settings/data-sources), and click on your ArgoCD integration | ||
|
||
Under the `resources` key, locate the Kubernetes block and replace it with the following YAML block to map ArgoCD Kubernetes Resource to the respective Opencost Allocation Resource. Then click `Save & Resync`: | ||
|
||
<details> | ||
<summary><b>Relation mapping (click to expand)</b></summary> | ||
|
||
```yaml showLineNumbers | ||
- kind: managed-resource | ||
selector: | ||
query: "true" | ||
port: | ||
entity: | ||
mappings: | ||
identifier: .__application.metadata.uid + "-" + .kind + "-" + .name | ||
title: .__application.metadata.name + "-" + .kind + "-" + .name | ||
blueprint: '"argocdKubernetesResource"' | ||
properties: | ||
kind: .kind | ||
namespace: .namespace | ||
version: .resourceVersion | ||
annotations: .liveState | fromjson | .metadata.annotations | ||
labels: .liveState | fromjson | .metadata.labels | ||
relations: | ||
application: .__application.metadata.uid | ||
image: 'if .kind == "Deployment" then .liveState | fromjson | .spec.template.spec.containers[0].image else null end' | ||
openCostResourceAllocation: .name | ||
``` | ||
|
||
</details> | ||
|
||
:::tip Mapping explanation | ||
The configuration mapping above maps `openCostResourceAllocation` using the `name` property of the Kubernetes resource since Opencost Resource Allocation have their identifiers as the name of the Kubernetes resource | ||
|
||
::: | ||
|
||
<img src='/img/build-your-software-catalog/sync-data-to-catalog/argocd/argoCDToOpencostRelation.png' border='1px' /> | ||
|
||
|
||
More relevant guides and examples: | ||
- [Port's Opencost integration](https://docs.getport.io/build-your-software-catalog/sync-data-to-catalog/cloud-cost/opencost) | ||
- [Port's ArgoCD integration](https://docs.getport.io/build-your-software-catalog/sync-data-to-catalog/argocd/) | ||
|
||
|
4 changes: 4 additions & 0 deletions
4
docs/build-your-software-catalog/sync-data-to-catalog/cloud-cost/opencost/_category_.json
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,4 @@ | ||
{ | ||
"label": "Opencost", | ||
"position": 2 | ||
} |
4 changes: 4 additions & 0 deletions
4
...ild-your-software-catalog/sync-data-to-catalog/cloud-cost/opencost/guides/_category_.json
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,4 @@ | ||
{ | ||
"label": "Guides", | ||
"position": 1 | ||
} |
103 changes: 103 additions & 0 deletions
103
...talog/cloud-cost/opencost/guides/connect-opencost-resource-with-aws-resource.md
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,103 @@ | ||
--- | ||
sidebar_position: 1 | ||
title: Connect Opencost Cloud Cost with AWS Resource in Port | ||
--- | ||
|
||
import PortTooltip from "/src/components/tooltip/tooltip.jsx" | ||
|
||
# Connect Opencost Clou- d Cost with AWS Resource in Port | ||
|
||
This guide will show you how to connect an Opencost cloudcost resource with an AWS resource in Port to analyze your cloud cost allocation. | ||
|
||
## Use Case Examples: | ||
|
||
- Cost Attribution: Determine which AWS resources (EC2 instances, RDS databases, etc.) contribute the most to your overall cloud costs. | ||
- Budget Management: Track the costs of individual resources or groups of resources against your budget allocations. | ||
- Anomaly Detection: Identify unusual spikes in resource costs and investigate potential causes. | ||
- Optimization Opportunities: Pinpoint resources that are underutilized or overprovisioned, potentially leading to cost savings. | ||
|
||
::: | ||
|
||
## Prerequisites | ||
- This guide assumes you have a Port account and that you have finished the [onboarding process](/quickstart). | ||
- Ensure you have [Port's Opencost integration installed and configured in your environment](https://docs.getport.io/build-your-software-catalog/sync-data-to-catalog/cloud-cost/opencost). | ||
- Ensure you have [Port's AWS integration installed and configured in your environment](https://docs.getport.io/build-your-software-catalog/sync-data-to-catalog/cloud-providers/aws/installations/installation). This guide does not support the [deprecated AWS Exporter integration](https://docs.getport.io/build-your-software-catalog/sync-data-to-catalog/cloud-providers/aws/aws-exporter/) | ||
|
||
|
||
## Create the Opencost Cloudcost relation | ||
|
||
With Opencost and AWS integrations installed, you should see some blueprints created by both integrations | ||
|
||
Now that Port is synced with our AWS and Opencost resources, let's map the Opencost Cloudcost to the AWS Cloud Resources. | ||
|
||
First, we will need to create a [relation](/build-your-software-catalog/customize-integrations/configure-data-model/relate-blueprints/relate-blueprints.md) between our `openCostCloudcost` and the corresponding `cloudResource`. | ||
|
||
1. Head back to the [Builder](https://app.getport.io/settings/data-model), choose the `Opencost Cloudcost` <PortTooltip id="blueprint">blueprint</PortTooltip>, and click on `New relation`: | ||
|
||
<img src='/img/build-your-software-catalog/sync-data-to-catalog/cloud-cost/opencost/guides/newRelationOpenCost.png' width='60%' border='1px' /> | ||
|
||
<br/><br/> | ||
|
||
2. Fill out the form like this, then click `Create`: | ||
|
||
<img src='/img/build-your-software-catalog/sync-data-to-catalog/cloud-cost/opencost/guides/createNewRelationOpencost.png' width='60%' border='1px' /> | ||
|
||
<br/><br/> | ||
|
||
|
||
Now that the <PortTooltip id="blueprint">blueprints</PortTooltip> are related, we need to assign the relevant Opencost Cloudcost to each of our AWS Cloud Resource. This can be done by adding some mapping logic. Go to your [data sources page](https://app.getport.io/settings/data-sources), and click on your Opencost integration: | ||
|
||
<img src='/img/build-your-software-catalog/sync-data-to-catalog/cloud-cost/opencost/guides/datasourcesOpencost.png' border='1px' /> | ||
|
||
<br/><br/> | ||
|
||
Under the `resources` key, locate the Cloudcost block and replace it with the following YAML block to map Opencost Cloudcost to the respective AWS Cloud Resource. Then click `Save & Resync`: | ||
|
||
<details> | ||
<summary><b>Relation mapping (click to expand)</b></summary> | ||
|
||
```yaml showLineNumbers | ||
- kind: cloudcost | ||
selector: | ||
query: .properties.providerID | ||
aggregate: providerID | ||
accumulate: week | ||
window: week | ||
port: | ||
entity: | ||
mappings: | ||
identifier: .properties.providerID | ||
title: .properties.providerID | ||
blueprint: '"openCostCloudcost"' | ||
properties: | ||
startDate: .window.start | ||
endDate: .window.end | ||
listCost: .listCost.cost | ||
netCost: .netCost.cost | ||
amortizedNetCost: .amortizedNetCost.cost | ||
invoicedCost: .invoicedCost.cost | ||
amortizedCost: .amortizedCost.cost | ||
relations: | ||
awsCloudResource: .properties.providerID | ||
``` | ||
|
||
</details> | ||
|
||
:::tip Mapping explanation | ||
The configuration mapping above ingests cloud cost data with selector properties designed to retrieve the AWS resource's ID | ||
|
||
- `query: .properties.providerID` ensures that only entities with a providerID are ingested | ||
- `aggregate: providerID` aggregates data by provider ID (AWS ARN) which ensures that entities are mapped to their respective AWS resource | ||
- `accumulate: week` defines the step size which accumulates data week by week | ||
- `window: week` grabs data for the past one week | ||
|
||
For the `awsCloudCost` relation, we extract the resource ID from the Provider ID of the cloudcost information. Therefore, each Cloudcost entity will be mapped to their respective AWS resource. | ||
By following these steps, you can seamlessly connect Opencost Cloudcost data with its respective AWS resources. | ||
|
||
::: | ||
|
||
<img src='/img/build-your-software-catalog/sync-data-to-catalog/cloud-cost/opencost/guides/awsToCloudcostRelation.png' border='1px' /> | ||
|
||
More relevant guides and examples: | ||
- [Port's Opencost integration](https://docs.getport.io/build-your-software-catalog/sync-data-to-catalog/cloud-cost/opencost) | ||
- [Port's AWS integration](https://docs.getport.io/build-your-software-catalog/sync-data-to-catalog/cloud-providers/aws/installations/installation) |
8 changes: 4 additions & 4 deletions
8
...nc-data-to-catalog/cloud-cost/opencost.md → ...o-catalog/cloud-cost/opencost/opencost.md
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Binary file added
BIN
+158 KB
...-your-software-catalog/sync-data-to-catalog/argocd/argoCDToOpencostRelation.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added
BIN
+95.9 KB
.../img/build-your-software-catalog/sync-data-to-catalog/argocd/createRelation.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added
BIN
+75.4 KB
.../build-your-software-catalog/sync-data-to-catalog/argocd/newRelationMapping.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added
BIN
+145 KB
...alog/sync-data-to-catalog/cloud-cost/opencost/guides/awsToCloudcostRelation.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added
BIN
+85.5 KB
...g/sync-data-to-catalog/cloud-cost/opencost/guides/createNewRelationOpencost.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added
BIN
+26.4 KB
...catalog/sync-data-to-catalog/cloud-cost/opencost/guides/datasourcesOpencost.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added
BIN
+88.7 KB
...catalog/sync-data-to-catalog/cloud-cost/opencost/guides/newRelationOpenCost.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.