docker example: use gcsfuse in container

.DS_Store

.vagrant

service-account.json

data

 No newline at end of file

The MIT License (MIT)

Copyright (c) 2017 Xu Wang

Permission is hereby granted, free of charge, to any person obtaining a copy

of this software and associated documentation files (the "Software"), to deal

in the Software without restriction, including without limitation the rights

to use, copy, modify, merge, publish, distribute, sublicense, and/or sell

copies of the Software, and to permit persons to whom the Software is

furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all

copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR

IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,

FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE

AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER

LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,

OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE

SOFTWARE.

include envs

export

help: ## this info

	@# adapted from https://marmelab.com/blog/2016/02/29/auto-documented-makefile.html

	@echo '_________________'

	@echo '| Make targets: |'

	@echo '-----------------'

	@cat Makefile | grep -E '^[a-zA-Z_-]+:.*?## .*$$' | awk 'BEGIN {FS = ":.*?## "}; {printf "\033[36m%-30s\033[0m %s\n", $$1, $$2}'

build: ## build gcsfuse image

	docker-compose build producer

up: ## bring system up, see docker-compose.yml

	docker-compose up -d --force-recreate --remove-orphans

down: ## bring system dev-down, see docker-compose.yml

	docker-compose down --remove-orphans --volumes

restart: ## restart test system

	docker-compose restart

follow: ## docker-compose logs -f

	docker-compose logs -f

prune: ## prune local docker containers and images

	docker container prune -f

	docker image prune -f

	docker volume prune -f

auth: ## Activate gclound service account

	gcloud auth login

	#gcloud auth activate-service-account --key-file keys/service-account.json

bucket:  ## Create the bucket if not exists

	@if ! gsutil ls -p ${GCP_PROJECT_ID} gs://${BUCKET} &> /dev/null; \

	then \

		echo creating gs://${BUCKET} ... ; \

		gsutil mb -p ${GCP_PROJECT_ID} -c regional -l ${GCP_REGION} gs://${BUCKET}; \

		sleep 10; \

	fi

destroy-bucket: ## Destroy the bucket if exists

	@if gsutil ls -p ${GCP_PROJECT_ID} gs://${BUCKET} &> /dev/null; \

	then \

		echo destroy gs://${BUCKET} ... ; \

		gsutil rb -f gs://${BUCKET}; \

	fi

sync: ## Rsync the gcs bucket

		gsutil -m rsync -r -d ./bucket gs://${BUCKET}

sync-docker-time: ## sync docker vm time with hardware clock

	@docker run --rm --privileged alpine hwclock -s

.PHONY: build prune push pull help up down restart follow

.PHONY: bucket destroy-bucket sync-docker-time

 No newline at end of file

# Docker Example - Use [gcsfuse]() volume in containers

## Prerequisites

#### Get GCS credentials

This example will require GCS credentials. The easiest way

to set up your credentials for testing is to run the [gcloud tool][]:

```

$ gcloud auth login

```

Alternatively, you can set the GOOGLE_APPLICATION_CREDENTIALS environment variable to the path to a JSON key file downloaded from the Google Developers Console:

```

$ export GOOGLE_APPLICATION_CREDENTIALS=/path/to/key.json

```

#### Set the default GCP project:

Edit envs file, set the GCP_PROJECT_ID to your GCP project id.

```

$ source envs; gcloud config set project ${GCP_PROJECT_ID}

```

#### Create the bucket

Before invoking gcsfuse, you must have a GCS bucket that you want to mount. If

your bucket doesn't yet exist, create one:

```

$ make bucket

```

## Build the gcsfuse docker image

```

$ make build

```

## See it works

```

$ make up follow

...

consumer_1  | total 0

producer_1  | Using mount point: /mnt/gcsfuse

producer_1  | Opening GCS connection...

producer_1  | Opening bucket...

producer_1  | Mounting file system...

producer_1  | File system has been successfully mounted.

consumer_1  | total 0

consumer_1  | -rw-r--r--    1 root     root             0 May  1  2017 index.html

consumer_1  | -rw-r--r--    1 root     root             0 May  1  2017 test.html

...

```

Go to [GCS Console] upload/delete some files to see changes in local logs.

## Shut it down

```

$ make down

```

[GCS Console]: https://console.cloud.google.com/storage

[gcsfulse]: https://github.com/GoogleCloudPlatform/gcsfuse

[console]: https://console.developers.google.com

[gcloud tool]: https://cloud.google.com/sdk/gcloud/

[app-default-credentials]: https://developers.google.com/identity/protocols/application-default-credentials#howtheywork