Launch the service on Google Cloud

This document describes steps required to host the Silent Network node on Google Cloud. Audience of this documents are DevOps, or software engineers, preferably with basic knowledge of Google Cloud and the Shell.

Install gcloud

We will use official Google Cloud CLI for most of the time. Install it from here

Create Project on Google Cloud

Follow official instructions here. You also need to connect billing account.

Enable Compute Engine API to allow start up the VM instances

Go to https://console.cloud.google.com/apis/library/compute.googleapis.com

Then click "Enable"

Enable secret manager api

https://console.cloud.google.com/apis/library/secretmanager.googleapis.com

Setup gcloud

Set the env variables that we will use over the course of this tutorial. The easiest way to do so, is to create a file with following content.

# Your project ID (not project number!)
export WORKLOAD_OPERATOR_PROJECT_ID=

# VM related parameters
export VM_ZONE=
export VM_NAME=operator
export VM_ADDRESS=

# Bucket where user's data shall be stored
export WORKLOAD_OPERATOR_OUTPUT_BUCKET_NAME=
export BUCKET_LOCATION=

# Do not change. Parameters for operator's container registry
export WORKLOAD_AUTHOR_PROJECT_ID=silent-network-426409
export REGISTRY_NAME=silent-network-registry
export CONTAINER_IMAGE=operator-container-image
export WORKLOAD_VERSION=v0.1.5

# Service account attached to the VM
export WORKLOAD_OPERATOR_SERVICE_ACCOUNT_NAME=operator-service-account

The empty variables are going to be set in following sections:

Project

Please fill up WORKLOAD_OPERATOR_PROJECT_ID with the ID of the project you've created. Immediately after set valid project for gloud CLI, so subsequent commands will operate on valid project.

Make sure gcloud is pointing to valid project by calling

gcloud config set project $WORKLOAD_OPERATOR_PROJECT_ID

VM Zone

Please select a VM_ZONE for the hosted VM in Europe. Go to Available regions and zones , filter location by Europe and machine series by N2D. Select one of available zones and set the VM_ZONE variable.

VM IP address

Set the static IP for the VM instance.

1. Pick region for the IP address. Needs to be the same as the one selected for the VM. The VM_ZONE variable contains zone, whereas for IP the region is needed. To get region from a zone, simply discard last two characters.

export IP_REGION=${VM_ZONE%??}

1. Create static IP in this region:

gcloud compute addresses create operator-static-ip --region=$IP_REGION

1. Get the IP address that was assigned from previous step

gcloud compute addresses describe operator-static-ip --region=$IP_REGION --format='get(address)'

1. Save the IP address in VM_ADDRESS variable

Bucket

Set the WORKLOAD_OPERATOR_OUTPUT_BUCKET_NAME . The name has to be globally unique, the simplest way to do so is to add random suffix:

echo workload-output-bucket-$(head /dev/urandom | tr -dc a-z0-9 | head -c10)

Select a location of a bucket from here. Choose one from Europe. Set the BUCKET_LOCATION.

Save the configuration

With all variables set, save the file as env.operator.configuration and in the console load the variables by:

> source env.operator.configuration

Create bucket

Create the bucket by calling:

gcloud storage buckets create gs://${WORKLOAD_OPERATOR_OUTPUT_BUCKET_NAME} --location=${BUCKET_LOCATION}

The read, write permissions to it are going to be set next.

Create Service Account

The Service Account will be attached to the VM, and used to access cloud resources. Create a service account by executing:

gcloud iam service-accounts create $WORKLOAD_OPERATOR_SERVICE_ACCOUNT_NAME

Next, set valid permissions for the account

# grant workload SA serviceAccountUser, so can create VM workload
gcloud iam service-accounts add-iam-policy-binding \
    $WORKLOAD_OPERATOR_SERVICE_ACCOUNT_NAME@$WORKLOAD_OPERATOR_PROJECT_ID.iam.gserviceaccount.com \
    --member=user:$(gcloud config get-value account) \
    --role=roles/iam.serviceAccountUser

# grant workload SA the confidentialcomputing.workloadUser role, so it can generate an attestation token
gcloud projects add-iam-policy-binding $WORKLOAD_OPERATOR_PROJECT_ID \
    --member=serviceAccount:$WORKLOAD_OPERATOR_SERVICE_ACCOUNT_NAME@$WORKLOAD_OPERATOR_PROJECT_ID.iam.gserviceaccount.com \
    --role=roles/confidentialcomputing.workloadUser

# grant workload SA the logging.logWriter role, so we can check progress of the workload
gcloud projects add-iam-policy-binding $WORKLOAD_OPERATOR_PROJECT_ID \
    --member=serviceAccount:$WORKLOAD_OPERATOR_SERVICE_ACCOUNT_NAME@$WORKLOAD_OPERATOR_PROJECT_ID.iam.gserviceaccount.com \
    --role=roles/logging.logWriter
    
# grant workload SA storage.objectUser role, so can read delete and update from the bucket
gcloud storage buckets add-iam-policy-binding gs://${WORKLOAD_OPERATOR_OUTPUT_BUCKET_NAME} \
    --member=serviceAccount:$WORKLOAD_OPERATOR_SERVICE_ACCOUNT_NAME@$WORKLOAD_OPERATOR_PROJECT_ID.iam.gserviceaccount.com \
    --role=roles/storage.objectUser
    
# grant the secretmanager.secretAccessor and viewer role, so can read secrets from secret manager
gcloud projects add-iam-policy-binding $WORKLOAD_OPERATOR_PROJECT_ID \
    --member=serviceAccount:$WORKLOAD_OPERATOR_SERVICE_ACCOUNT_NAME@$WORKLOAD_OPERATOR_PROJECT_ID.iam.gserviceaccount.com \
    --role=roles/secretmanager.secretAccessor

gcloud projects add-iam-policy-binding $WORKLOAD_OPERATOR_PROJECT_ID \
    --member=serviceAccount:$WORKLOAD_OPERATOR_SERVICE_ACCOUNT_NAME@$WORKLOAD_OPERATOR_PROJECT_ID.iam.gserviceaccount.com \
    --role=roles/secretmanager.viewer

Share the Service account details with us

Provide us output of this command

echo $WORKLOAD_OPERATOR_SERVICE_ACCOUNT_NAME@$WORKLOAD_OPERATOR_PROJECT_ID.iam.gserviceaccount.com

Our team will grant to that Service Account access the container registry.

Setup the firewall

Enable http and https connection to the VM instance.

gcloud compute firewall-rules create allow-http \
 --allow tcp:80 --target-tags http-server
 
 gcloud compute firewall-rules create allow-https \
 --allow tcp:443 --target-tags https-server

Put secrets to Secret Manager

Signing key for MPC protocol

Store the secret key in Google's Secret Manager, the Container will fetch it upon launch.

openssl rand 32 | gcloud secrets create SILENT_NETWORK_NODE_MSG_SIGN_KEY \
    --replication-policy="automatic" \
    --data-file=-

Launch the VM instance

Create an environment variables file (name it container-env-file) that is going to be passed to the running container.

Set following variables:

1. KEYSHARE_STORAGE should point to created bucket, with gs:// prefix. For example: gs://workload-output-bucket-xxxxxxxxxx

2. OPERATOR_ETH_ADDRESS the ETH address derived from public ECDSA key generated during registration to the AVS

KEYSHARE_STORAGE=
OPERATOR_ETH_ADDRESS=

# Do not change
AGGREGATOR_PUBLIC_KEY=fbf777920d5853faa49d50216030ab9fa1ef263d588392d79fff078c5f0a371a
RUST_LOG=debug
LISTEN=0.0.0.0:80
ETH_RPC_URL=https://ethereum-holesky-rpc.publicnode.com
ETH_WS_URL=wss://ethereum-holesky-rpc.publicnode.com
# Addresses from official SilentShard AVS deployment (proxies) on Holesky
OPERATOR_STATE_RETRIEVER=0xD1aC03296bF5131BC592F005bE94319Bb02c778d
REGISTRY_COORDINATOR=0x76a49007EfB04d114D047095A49EdAF9A5dD334b
# https://holesky.eigenlayer.xyz/avs/0x7e06dbbe4fc83a1bccd20af3079e08f57c19cf34
DKLS_SERVICE_MANAGER=0x7e06DbBe4fC83a1bCcd20AF3079e08F57C19cf34
NODE_API_ADDRESS=0.0.0.0:80

Save the file as container-env-file

Start the VM. The container shall be fetched from the private registry, and then executed. The container-env-file is used to set up variables.

gcloud compute instances create-with-container ${VM_NAME} \
    --maintenance-policy=TERMINATE \
    --zone=${VM_ZONE} \
    --tags=http-server,https-server \
    --container-image europe-docker.pkg.dev/$WORKLOAD_AUTHOR_PROJECT_ID/$REGISTRY_NAME/${CONTAINER_IMAGE}:${WORKLOAD_VERSION} \
    --service-account=$WORKLOAD_OPERATOR_SERVICE_ACCOUNT_NAME@$WORKLOAD_OPERATOR_PROJECT_ID.iam.gserviceaccount.com \
    --address=${VM_ADDRESS} \
    --tags=http-server,https-server \
    --scopes=https://www.googleapis.com/auth/cloud-platform \
    --machine-type=n2d-standard-16 \
    --metadata google-logging-enabled=true \
    --container-env=GC_PROJECT_ID=${WORKLOAD_OPERATOR_PROJECT_ID} \
    --container-env-file ./container-env-file 

Remember to set valid path to thecontainer-env-file option (last line)

Troubleshooting

Check if the VM and the container started up without any problems:

curl ${VM_ADDRESS}

The command should return text response:ok

Error: The zone 'XXX' does not have enough resources available to fulfill the request.

There might be unlikely event upon VM start to face the error ZONE_RESOURCE_POOL_EXHAUSTED_WITH_DETAILS. It means there are insufficient HW resources in the selected VM_ZONE Please choose different zone as described in VM Zone chapter.

There is no way to move the IP, across regions, so if new VM_ZONE is in different region, release old IP by calling:

gcloud compute addresses delete operator-static-ip

And then create VM IP address in new region and try again.

Serial console

You can connect to serial port of the VM to read the logs of the Host OS before the container launches. It's helpful in case if container does not starts up.

# Enable serial port
gcloud compute instances add-metadata ${VM_NAME} \
    --zone=${VM_ZONE} \
    --metadata serial-port-enable=TRUE

# Connect to serial port to see the logging of the Host OS
gcloud compute connect-to-serial-port --zone=${VM_ZONE} ${VM_NAME}

If everything went well and the container started up, you should be able to see following logs:

[   31.297213] konlet-startup[1705]: 2024/06/17 12:07:21 Starting a container with ID: 6c9f3fd7c8f89328d736ee7526d3f15648b94e5f6b897bc7de69bcbe693125ab
[   31.395243] konlet-startup[1705]: 2024/06/17 12:07:21 Saving welcome script to profile.d

To exit the console, hit Enter, and then type: ~.

Logs from the container

If the VM is running, the output from serial console suggests container started up correctly, but there is no response from curl, SSH to the VM:

gcloud compute ssh --project=${WORKLOAD_OPERATOR_PROJECT_ID} --zone=${VM_ZONE} ${VM_NAME}

Find name of running container:

docker ps

It should be in a form: xxx-operator-xxxx. Get the logs:

docker logs xxx-operator-xxxx

Feel free to contact us if you need any assistance, please provide these logs

Share the VM instance information with us

Share following data with our team in order to be able to connect you to the MPC network

• The IP address from VM_ADDRESS variable,

• The Operator ETH address

• The VM_ZONE.

• Public part of the MPC signing key stored in Secret Manager. Read it from the logs of the running VM instance:

gcloud logging read \                                                                                                                                                                   
"resource.type=gce_instance AND resource.labels.instance_id=$( gcloud compute instances describe --format "get(id)" ${VM_NAME} --zone=${VM_ZONE} )" \
    --limit 100 \
    --format json \
    --freshness 1d | grep "Master VK from GC Secret Manager"

Example output:

"message": "2024-07-09T15:30:03.686380Z  INFO el_party_svc::signer: Master VK from GC Secret Manager: \"21dba548b92ad65bfdc6328e98f5099c17702f75d1fd82c98cff81e5abc26e29\"\n",