一千萬個為什麽

搜索

如何清理孤立的AWS EC2快照?



我們最終得到了相當數量的AWS EC2快照,其中AMI已被刪除,但快照被遺忘。我希望以非手動的方式<�識別和刪除這些孤兒,以節省我們的資金和空間。

理想情況下,我正在考慮利用CLI bash腳本,但是我的AWS-fu很弱。我假設以前有人做過這個,但是我找不到實際上有效的腳本。

在最好的情況下,這也會檢查數量並清理這些數據,但這可能更適合第二個問題。

轉載註明原文: 如何清理孤立的AWS EC2快照?

一共有 4 個回答:

很大程度上受到博客文章和其他答案中已有鏈接的啟發,這裏是我解決問題的方法。

我確實使用了一些令人費解的 JMESpath 函數來獲取快照列表,而不需要 tr

Disclaimer: Use at your own risks, I did my best to avoid any problem and keep sane defaults, but I won't take any blame if it cause problem to you.

#!/bin/sh
# remove x if you don't want to see the commands
set -ex

# Some variable initialisation with sane defaults
DRUN='--dry-run'
DO_DELETE=${1:-'no'}
REGION=${2:-'eu-west-1'}
ACCOUNTID=${3:-'self'}

# Get two temporary files
SNAP_FILE=$(mktemp)
IMAGE_FILE=$(mktemp)

# Get the snapshot list and the volume list
aws --region "$REGION" ec2 describe-snapshots --owner-ids "$ACCOUNTID" --query 'join(`\n`,Snapshots[*].SnapshotId)' --output text > "$SNAP_FILE"
aws --region "$REGION" ec2 describe-images --owners "$ACCOUNTID" --filters Name=state,Values=available --query 'Images[*].BlockDeviceMappings[*].Ebs.SnapshotId' --output text > "$IMAGE_FILE"

# Check if the outputed command should be dry-run (default) or not
if [ "$DO_DELETE" = "IAMSURE" ]
then
 DRUN=''
fi

# count each snapshot id, decrease when a volume reference it, print delete command for those with no volumes
awk -v REGION="$REGION" -v DRUN="$DRUN" '
FNR==NR { snap[$1]++; next } # increment snapshots and get to next line in file immediately

{ snap[$1]-- } # we changed file, decrease the snap counter when a volume reference it

END {
 for (s in snap) { # loop over the snapshots
   if (snap[s] > 0) { # if we did not decrese under 1 that means there is no volume referencing this snapshot
    cmd="aws --region " REGION " " DRUN " ec2 delete-snapshot --snapshot-id " s
    print(cmd)
  }
 }
}
' "$SNAP_FILE" "$IMAGE_FILE"
# Clean up the temp files
rm "$SNAP_FILE" "$IMAGE_FILE"

我希望劇本本身足夠評論。

默認用法(no-params)將列出當前帳戶和區域eu-west-1的孤立快照的刪除命令,提取:

aws --region eu-west-1 --dry-run ec2 delete-snapshot --snapshot-id snap-81e5856a
aws --region eu-west-1 --dry-run ec2 delete-snapshot --snapshot-id snap-95c68c7e
aws --region eu-west-1 --dry-run ec2 delete-snapshot --snapshot-id snap-a3bf50bd

您可以在將它輸出到執行所有命令之前,將此輸出重定向到一個文件以供審閱。

如果您希望腳本執行命令而不是打印它們,請用 system(cmd)替換 print(cmd)

用法如下名為 snap_cleaner 的腳本:

用於美國西部地區的空運命令

./snap_cleaner no us-west-1

在eu-central-1中使用可用的命令

./snap_cleaner IAMSURE eu-central-1 

第三個參數可以用來訪問另一個帳戶(我更喜歡在之前將角色切換到另一個帳戶)。

用awk腳本剝離腳本版本作為oneliner:

#!/bin/sh
set -ex

# Some variable initialisation with sane defaults
DRUN='--dry-run'
DO_DELETE=${1:-'no'}
REGION=${2:-'eu-west-1'}
ACCOUNTID=${3:-'self'}

# Get two temporary files
SNAP_FILE=$(mktemp)
IMAGE_FILE=$(mktemp)

# Get the snapshot list and the volume list
aws --region "$REGION" ec2 describe-snapshots --owner-ids "$ACCOUNTID" --query 'join(`\n`,Snapshots[*].SnapshotId)' --output text > "$SNAP_FILE"
aws --region "$REGION" ec2 describe-images --owners "$ACCOUNTID" --filters Name=state,Values=available --query 'Images[*].BlockDeviceMappings[*].Ebs.SnapshotId' --output text > "$IMAGE_FILE"

# Check if the outputed command should be dry-run (default) or not
if [ "$DO_DELETE" = "IAMSURE" ]
then
 DRUN=''
fi

# count each snapshot id, decrease when a volume reference it, print delete command for those with no volumes
awk -v REGION="$REGION" -v DRUN="$DRUN" 'FNR==NR { snap[$1]++; next } { snap[$1]-- } END { for (s in snap) { if (snap[s] > 0) { cmd="aws --region " REGION " " DRUN " ec2 delete-snapshot --snapshot-id " s; print(cmd) } } }' "$SNAP_FILE" "$IMAGE_FILE"
# Clean up the temp files
rm "$SNAP_FILE" "$IMAGE_FILE"

我在GitHub上通過Rodrigue Koffi(bonclay7)使用了下面的腳本,它工作的很好。

https://github.com/bonclay7/aws-amicleaner

命令:

amicleaner --check-orphans

從文檔博文它會做更多的事情:

It actually does a bit more than that, at of today it allows:

  • Removing a list of images and associated snapshots
  • Mapping AMIs:
    • Using names
    • Using tags
  • Filtering AMIs:
    • used by running instances
    • from autoscaling groups (launch configurations) with a desired capacity set to 0
    • from launch configurations detached from autoscaling groups
  • Specifying how many AMIs you want to keep
  • Cleaning orphan snapshots
  • A bit of reporting

這裏有一個腳本可以幫助你找到孤立的快照

comm -23 <(echo $(ec2-describe-snapshots --region eu-west-1 | grep SNAPSHOT | awk '{print $2}' | sort | uniq) | tr ' ' '\n') <(echo $(ec2-describe-images --region eu-west-1 | grep BLOCKDEVICEMAPPING | awk '{print $3}' | sort | uniq) | tr ' ' '\n') | tr '\n' ' '

(來自此處</一>)

Also you can check this article from

serverfault

附:當然你可以改變這個地區以反映你的情況

P.P.S.這是更新代碼:

 comm -23 \
<(echo $(aws ec2 describe-snapshots --region eu-west-1 |awk '/SNAPSHOT/ {print $2}' | sort -u) | tr ' ' '\n') \
<(echo $(aws ec2 describe-images --region eu-west-1 |  awk '/BLOCKDEVICEMAPPING/ {print $3}' | sort -u) | tr ' ' '\n') | tr '\n' ' '

示例exaplanations代碼的作用是:

echo $(aws ec2 describe-snapshots --region eu-west-1 | awk '/SNAPSHOT/ {print $2}' | sort -u) | tr ' ' '\n')

發送到STDOUT快照列表。這種結構:

<(...)

創建虛擬臨時文件處理程序,以便從兩個“文件”中讀取 comm 命令並對它們進行比較

Here is a GitHub Gist code snippet of exactly what you are asking for by Daniil Yaroslavtsev.

它使用所有圖像及其快照的列表,並將這些ID與所有快照ID列表進行比較。無論遺留的是孤兒。代碼的工作原理與上面的答案相同,但格式更好,可讀性更好。

該代碼利用JSON查詢   - 查詢Snapshots [*]。SnapshotId 選項(您也可以使用jq命令行實用程序,它是相同的JSON查詢語言)。輸出文本格式為 - 輸出文本。以下是API參考的鏈接和幾個例子。它比grep/awk/sort/uniq/tr管道的長鏈稍微優雅。