GCP BigQueryの作成 CloudFunctions+BigQuery+CloudRun+ESPv2+独自ドメイン

GCP

関連

データセット 
 └ テーブル・ビュー

データセットはコンテナのようなもの

テーブルの確認

% bq ls -p
            projectId                     friendlyName
 ------------------------------- -------------------------------
  test-cloud-functions-20211208   test-cloud-functions-20211208

データセット作成

$ bq mk test_dataset_kanehiro

Dataset 'test-cloud-functions-20211208:test_dataset_kanehiro' successfully created.

テーブルの作成

login_histories.json

[
  {
    "name": "user_id", 
    "type": "INTEGER",
    "mode": "REQUIRED"
  },
  {
    "name": "anonymous_token",
    "type": "STRING",
    "mode": "REQUIRED"
  },
  {
    "name": "login_type",
    "type": "STRING",
    "mode": "REQUIRED"
  },
  {
    "name": "created_at",
    "type": "TIMESTAMP",
    "mode": "REQUIRED"
  }
]

変数定義

PROJECT_ID=test-cloud-functions-20211208
DATASET=test_dataset_kanehiro
TABLE_NAME=${DATASET}.login_histories

テーブル作成

created_atに日別でPartitioning
user_idでclustering_fields指定

$ bq mk \
  --project_id=${PROJECT_ID} \
  --schema  login_histories.json \
  --time_partitioning_type=DAY \
  --time_partitioning_field=created_at  \
  --clustering_fields=user_id \
  --use_legacy_sql=false \
  --table ${TABLE_NAME}

Table 'test-cloud-functions-20211208:test_dataset_kanehiro.login_histories' successfully created.

% bq show test_dataset_kanehiro.login_histories
 
 
Table test-cloud-functions-20211208:test_dataset_kanehiro.login_histories

   Last modified                    Schema                   Total Rows   Total Bytes     Expiration      Time Partitioning   Clustered Fields   Labels
 ----------------- ---------------------------------------- ------------ ------------- ----------------- ------------------- ------------------ --------
  08 Dec 17:07:11   |- user_id: integer (required)           0            0             06 Feb 17:07:10
                    |- annonymous_token: string (required)
                    |- login_type: string (required)