Access GCR Docker images from other Google Cloud Projects using Terraform

If you have multiple Google Cloud Projects and want to pull Container Registry images from other projects, you will need to grant each project access by adding a role to the a few service accounts specific to each Project.

Container Registry has no permissions of it’s own, but actually relies on permissions to the Cloud Storage bucket that is created automatically for it when you first push an image.

Confusingly this requires the use of the Project ID as opposed to the Project Name.

Access Compute Engine VM images from other Google Cloud Projects using Terraform

If you have multiple Google Cloud Projects and want to access your GCE VM images from a single project, you will need to grant each project access by adding a role to the Google APIs service account specific to each Project.

Confusingly this requires the use of the Project ID as opposed to the Project Name.

Note that using google_project_iam_member in Terraform will simply add that role to the service account in each Project. Be careful if you update this to use google_project_iam_binding or google_project_iam_policy as they are authoritative and may revoke existing roles from the service account if used incorrectly!

go install: no install location for directory outside GOPATH on macOS

If you are trying to install packages using dep from gopkg.toml or update your gopkg.lock with new dependencies on macOS, you may come across this error…

go install: no install location for directory /Users/somebody/project outside GOPATH

Your environment may not have GOPATH or GOBIN set, in which case as a simple solution you can set them relative to your home directory.

(more advanced use-cases will require them to be set differently of course)

No cached mapping for this field. Refresh field list from the Index Patterns page

With newly indexed Elastic Stack Elasticsearch documents (from Elastic APM or Filebeat/Logstash for example) you may see a warning “No cached mapping for this field. Refresh field list from the Management > Index Patterns page.” in Kibana when inspecting an event.

To refresh the field list from Kibana you can go into the Management tab from the navigation on the left-hand side, then Index Patterns.

There should be a refresh icon in the top-right, which should notify you that “This action resets the popularity counter of each field.”

Repeat this for any indices you would like the fields automatically mapped for (in the above example we are automatically mapping the fields that Filebeat is populating in each event)

Return to the Discover tab and the warnings should be gone! Note this will only work if Elasticsearch is able to accurately determine a consistent data type for each field.

Port forward Elasticsearch Head to Kubernetes Statefulset

If you prefer Elasticsearch Head to oversee cluster-wide operations but miss having it installed on-cluster (now that it is not available ES v5 and above) plus run Elasticsearch as a Kubernetes Statefulset, then you can run Elasticsearch Head as a Chrome plugin and port forward to the Statefulset.

This assumes that you have installed Elasticsearch in a standard way such as via the Elastic Helm chart, and that you have a Statefulset with Pods named elasticsearch-n and a Service named elasticsearch pointing to the Statefulset.

If you then point Elasticsearch Head at localhost:9200 (which is the default) it will port forward to your Kubernetes cluster and the overview should populate.

Clone a private GitHub repository without an SSH key using a Personal Access Token

If you need to clone a private GitHub repository without an SSH key you can use a Personal Access Token. This can be useful in a build/deployment pipeline for example where you are using a programmatic GitHub user to clone into a temporary server or container. You can then pass the token and other information via environment variables.

Iterate over set for resource using count and index in Terraform

Even in Terraform 0.12 you cannot use iteration with resources, so you may still have to resort to using count to create multiple resources.

However you cannot iterate over a set (unlike a list) normally because it has no indices. Luckily Terraform will convert sets to lists without modifying the order of items, which allows you to use count and index as you would with a list.

CentOS/RHEL 6 Python PIP error, SyntaxError: invalid syntax {str(c.version) for c in all_candidates}

If you’re attempting to upgrade the version of PIP on CentOS or RHEL 6 from the default 7.1.0 to a newer version, as of April 14th 2018 when PIP 10.0.0 was released¬†you will run into issues since EL6 ships with Python 2.6 as standard. Since you cannot upgrade the default version of Python because it will break in-built tooling such as Yum, you’ll need to version lock PIP when you upgrade it to 9.0.3. This is the last stable version of PIP that is compatible with Python 2.6

Older posts