AWS

Backlog GitのWebフックからServerless FrameworkでS3へ

| by 優

AWS

 

環境

  • Mac Book Pro
  • AWS
  • Backlog

AWS Cliのインストール

 

curl "https://s3.amazonaws.com/aws-cli/awscli-bundle.zip" -o "awscli-bundle.zip"
unzip awscli-bundle.zip
sudo ./awscli-bundle/install -i /usr/local/aws -b /usr/local/bin/aws

 

$ aws --version
aws-cli/1.16.253 Python/2.7.10 Darwin/17.7.0 botocore/1.12.243

 

$ aws configure --profile test-kanehiro

AWS Access Key ID [None]: <アクセスキーID入力>
AWS Secret Access Key [None]: <シークレットキー入力>
Default region name [None]: ap-northeast-1
Default output format [None]: json

 

※利用時

$ aws ~~~ --profile ${profile}

 

$ cat $HOME/.aws/config

[profile test-kanehiro]
output = json
region = ap-northeast-1

 

$ cat $HOME/.aws/credentials

[]
aws_access_key_id = xxxx
aws_secret_access_key = xxxxxxxx

 

 

pipのインストール

 

$ curl https://bootstrap.pypa.io/get-pip.py -o get-pip.py
$ sudo python get-pip.py

 

boto3のインストール

$ sudo pip install boto3 --upgrade --ignore-installed six

 

 

 

Serverless Frameworｋのインストール

 

$ brew install nodebrew

 

$ mkdir -p ~/.nodebrew/src
$ nodebrew install-binary latest

 

Node.jsがインストールされたか確認

$ nodebrew list
v12.11.1

current: none
develop $ nodebrew list
v12.11.1

current: none

 

$ nodebrew -v

nodebrew 1.0.1

Usage:
    nodebrew help                         Show this message
    nodebrew install <version>            Download and install <version> (from binary)
    nodebrew compile <version>            Download and install <version> (from source)
    nodebrew install-binary <version>     Alias of `install` (For backword compatibility)
    nodebrew uninstall <version>          Uninstall <version>
    nodebrew use <version>                Use <version>
    nodebrew list                         List installed versions
    nodebrew ls                           Alias for `list`
    nodebrew ls-remote                    List remote versions
    nodebrew ls-all                       List remote and installed versions
    nodebrew alias <key> <value>          Set alias
    nodebrew unalias <key>                Remove alias
    nodebrew clean <version> | all        Remove source file
    nodebrew selfupdate                   Update nodebrew
    nodebrew migrate-package <version>    Install global NPM packages contained in <version> to current version
    nodebrew exec <version> -- <command>  Execute <command> using specified <version>

Example:
    # install
    nodebrew install v8.9.4

    # use a specific version number
    nodebrew use v8.9.4

 

$ nodebrew use v12.11.1
use v12.11.1

 

$ ls $HOME/.nodebrew/current/bin
node		nodebrew	npm		npx

 

$ echo 'export PATH=$PATH:$HOME/.nodebrew/current/bin' >> ~/.bash_profile

 

$ cat ~/.bash_profile
 
if [ -f ~/.bashrc ]; then
  . ~/.bashrc
fi
 
export PATH="/usr/local/opt/mysql-client/bin:$PATH"
export COMPOSE_HTTP_TIMEOUT=200
export PATH=$PATH:$HOME/.nodebrew/current/bin ←●追記された！！

 

シェルへの再ログイン

$ exec $SHELL -l

 

$ npm -v
6.11.3

 

$ node -v
v12.11.1

 

 

Serverless Framework

 

$ npm install -g serverless

 

$ sls -v

Framework Core: 1.53.0
Plugin: 3.1.1
SDK: 2.1.1
Components Core: 1.1.1
Components CLI: 1.2.3

 

テスト

$ sls create -t aws-python -p slstest

Serverless: Generating boilerplate...
Serverless: Generating boilerplate in "/Users/kanehiroyuu/Documents/develop/slstest"
 _______                             __
|   _   .-----.----.--.--.-----.----|  .-----.-----.-----.
|   |___|  -__|   _|  |  |  -__|   _|  |  -__|__ --|__ --|
|____   |_____|__|  \___/|_____|__| |__|_____|_____|_____|
|   |   |             The Serverless Application Framework
|       |                           serverless.com, v1.53.0
 -------'

Serverless: Successfully generated boilerplate for template: "aws-python"

 

 

$ cd /slstest
$ npm install --save serverless-python-requirements

 

handler.js
関数定義するためのスケルトン

 

event.json
ServerlessのCLIでファンクションを実行する際に
入力値となるデータを定義するファイルです。
Lambdaファンクション内でevent変数に展開されます。

 

※GitHubに上がっている既存サービスをインポートする場合

$ serverless install -u [GITHUB URL OF SERVICE]

 

$ sls config credentials --provider aws --key <キー> --secret <シークレット> --profile test-kanehiro

 

$ vi $HOME/Documents/develop/slstest/serverless.yml

provider:
  name: aws
  runtime: python2.7
+ region: ap-northeast-1
+ profile: test-kanehiro

 

 

 

 

 

$ sls deploy --profile test-kanehiro

 

デプロイされたLambda関数をチェック

$ sls invoke -f hello
{
    "body": "{\"input\": {}, \"message\": \"Go Serverless v1.0! Your function executed successfully!\"}",
    "statusCode": 200
}

 

 

 

Tips

全体のデプロイ

$ sls deploy -v

 

関数単位のデプロイ

$ serverless deploy function -f <yourfunction>

 

プロジェクトの削除

$ sls remove

Serverless: Getting all objects in S3 bucket...
Serverless: Removing objects in S3 bucket...
Serverless: Removing Stack...
Serverless: Checking Stack removal progress...

Serverless: Stack removal finished...

 

 

Backlog Git Webhook + API Gateway + Lambda

 

Backlog Git WebhookのJSONレスポンスサンプル

{
  "before": "5aef35982fb2d34e9d9d4502f6ede1072793222d",
  "after": "c0b8abaa6df37ea682454c25f2d602dffb5de6ed",
  "ref": "refs/heads/master",
  "repository": {
    "url": "https://demo.backlog.jp/git/DORA/himitsu",
    "name": "himitsu",
    "description": "ひみつ道具を格納しているリポジトリ"
  },
  "revisions": [
    {
      "id": "8e82fe274af30adbb378785628db509da1c969d1",
      "url": "https://demo.backlog.jp/git/DORA/himitsu/commit/8e82fe274af30adbb378785628db509da1c969d1",
      "author": {
        "email": "nobi@example.com",
        "name": "nobi"
      },
      "message": "どこでも行けるドアを作成しました",
      "timestamp": "2013-04-01T14:57:17+09:00",
      "added": ["html/anywhere.html", "css/anywhere.css"]
    },
    {
      "id": "c0b8abaa6df37ea682454c25f2d602dffb5de6ed",
      "url": "https://demo.backlog.jp/git/DORA/himitsu/commit/c0b8abaa6df37ea682454c25f2d602dffb5de6ed",
      "author": {
        "email": "gouda@example.com",
        "name": "gouda"
      },
      "message": "DORA-1 竹でできているヘリコプターが壊れました #fixed",
      "timestamp": "2013-04-01T18:22:10+09:00",
      "removed": ["html/take-copter.html"]
    }
  ]
}

 

event.json

$ cat event.json

{
  "before": "5aef35982fb2d34e9d9d4502f6ede1072793222d",
  "after": "c0b8abaa6df37ea682454c25f2d602dffb5de6ed",
  "ref": "refs/heads/master",
  "repository": {
    "url": "https://yuu-k.backlog.com/git/TEST/TEST-Repo",
    "name": "TEST-Repo",
    "description": "ひみつ道具を格納しているリポジトリ"
  },
  "revisions": [
    {
      "id": "052e8855e9cbf2fa41dda08d3184e34631018740",
      "url": "https://yuu-k.backlog.com/git/TEST/TEST-Repo/commit/052e8855e9cbf2fa41dda08d3184e34631018740",
      "author": {
        "email": "nobi@example.com",
        "name": "nobi"
      },
      "message": "どこでも行けるドアを作成しました",
      "timestamp": "2013-04-01T14:57:17+09:00",
      "added": ["html/anywhere.html", "css/anywhere.css"]
    },
    {
      "id": "052e8855e9cbf2fa41dda08d3184e34631018740",
      "url": "https://yuu-k.backlog.com/git/TEST/TEST-Repo/commit/052e8855e9cbf2fa41dda08d3184e34631018740",
      "author": {
        "email": "gouda@example.com",
        "name": "gouda"
      },
      "message": "DORA-1 竹でできているヘリコプターが壊れました #fixed",
      "timestamp": "2013-04-01T18:22:10+09:00",
      "removed": ["html/take-copter.html"]
    }
  ]
}

 

 

handler.py

$ cat handler.py


# -*- coding: utf-8 -*-

import json
import urllib.parse
import os
import tempfile
import shutil
import boto3
from dulwich import porcelain
from datetime import datetime

BUCKET_NAME = os.environ['BUCKET_NAME']
ZIP_FILE_NAME = os.environ['ZIP_FILE_NAME']
USER = os.environ['USER']
PASS = os.environ['PASS']
REPOSITORY = os.environ['REPOSITORY']
BLANCH = os.environ['BLANCH']

def lambda_handler(event, context):


    repository = "TEST-Repo"
    url = "https://yuu-k.backlog.com/git/TEST/TEST-Repo"
    branch = "master"

    # gitパスの生成
    site = urllib.parse.urlparse(url)
    userStr = urllib.parse.quote(USER)
    passStr = urllib.parse.quote(PASS)
    uri = site.scheme +"://" + userStr + ":" + passStr +"@" + site.netloc + site.path + ".git"

    # 作業ディレクトリの生成
    tmpDir  = tempfile.mkdtemp()

    # clone/zip/upload
    porcelain.clone(uri, tmpDir)
    print("git clone success")

    zipFileName = tmpDir+ '/' + os.path.splitext(ZIP_FILE_NAME)[0]
    #uploadFileName = datetime.now().strftime('%Y-%m-%d-%H-%M-%S') + '_master.zip'
    uploadFileName = 'master.zip'
    shutil.make_archive(zipFileName, 'zip', tmpDir )
    print("zip success")

    s3 = boto3.client('s3')
    s3.upload_file(zipFileName + '.zip', BUCKET_NAME, uploadFileName)
    print("s3 upload success")

    # 後始末
    shutil.rmtree(tmpDir)

 

$ cat serverless.yml


service: slstest
provider:
  name: aws
  runtime: python3.6
  region: ap-northeast-1
  profile: test-kanehiro
  iamRoleStatements:
    - Effect: "Allow"
      Action:
        - "s3:PutObject"
      Resource: "arn:aws:s3:::yuu-test-repo/*"

plugins:
  - serverless-python-requirements

functions:
  hello:
    handler: handler.lambda_handler
    memorySize: 128
    timeout: 60
    events:
      - http:
          path: git/push
          method: post
          integration: lambda
          request:
            parameters:
              querystrings:
                payload: true
    environment:
        USER: hoge@example.net
        PASS: hogePassWord
        ZIP_FILE_NAME: TEST
        REPOSITORY: https://yuu-k.backlog.com/git/TEST/TEST-Repo
        BUCKET_NAME: yuu-test-repo
        BLANCH: master

 

 

デプロイする

$ sls deploy --profile test-kanehiro

 

  • BacklogのGitのWEBフックにAPIゲートウェアイのAPIのURLを指定
  • Backlogにpushすると、API Gatewayを通じてLambdaからS3にzip圧縮されてアップロードされる

 

さらにCodepipelineに繋げると、EC2までデプロイできる。

 

AWS S3 + CodePipeline + CodeBuild + CodeDeploy+ EC2

 

 

 

関連記事 - More from my site -

コメントを残す

メールアドレスが公開されることはありません。 * が付いている欄は必須項目です

日本語が含まれない投稿は無視されますのでご注意ください。（スパム対策）