Commit 0d11a1c5 authored by xuwang's avatar xuwang
Browse files

docs: slim README, fan out detail to docs/, drop stale lists



README was 1344 lines. Moved Common Workflows -> docs/WORKFLOWS.md (plus an
Agentic workflow reference section) and Environment Variables + Best Practices +
Troubleshooting -> docs/GUIDE.md. Dropped the per-module Makefile Modules prose
(505 lines) and the All-Makefiles/Scripts reference lists (97 lines) — both
duplicated `make help` and docs/OTICA_TECHNICAL_REFERENCE.md and went stale on
every module change — replacing them with pointers. Each section heading is kept
as a stub so the [[_TOC_]] outline is unchanged. README is now ~500 lines; all
moved links/anchors verified, make validate green.

Changelog: changed

Co-Authored-By: default avatarClaude Opus 4.8 <noreply@anthropic.com>
parent 4bc7e872
Loading
Loading
Loading
Loading
+21 −861

File changed.

Preview size limit exceeded, changes collapsed.

docs/GUIDE.md

0 → 100644
+177 −0
Original line number Diff line number Diff line
# OTICA Usage Guide

Configuration reference, recommended practices, and troubleshooting for working with OTICA.
See the [README](../README.md) for setup and the
[Technical Reference](OTICA_TECHNICAL_REFERENCE.md) for module/script internals.

## Environment Variables

### Global Variables

```makefile
OTICA_DIR=${HOME}/.otica                    # OTICA installation directory
OTICA_REPO=https://code.stanford.edu/...    # OTICA repository URL
OTICA_SCRIPTS_DIR=${OTICA_DIR}/scripts      # Scripts directory (on PATH)
```

### Azure Variables

```makefile
CAE_RG_NAME=<resource-group>
CAE_NAME=<environment-name>
ACA_NAME=<app-name>
ACR_NAME=<registry-name>
AZ_SUBSCRIPTION_ID=<subscription-id>
AZ_LOCATION=<location>
```

### GCP Variables

```makefile
GCP_PROJECT_ID=<project-id>
GCP_REGION=<region>
GCP_ENVIRONMENT=<environment-name>
GKE_CLUSTER_NAME=<cluster-name>
```

### Terraform Variables

```makefile
TF_VERSION=<version>              # required
TF_DIR=Terraform
TF_BUILD_DIR=.tf_build
TF_BACKEND_TYPE=gitlab
GITLAB_PROJECT_URL=<project-url>  # required for GitLab backend
```

### Vault Variables

```makefile
VAULT_ADDR=https://<your-vault-addr>
VAULT_AUTH_METHOD=token
VAULT_ENABLED=true
```

### Docker Variables

```makefile
DOCKER_REGISTRY=<registry-url>
DOCKER_IMAGE_NAME=<image-name>
DOCKER_IMAGE_TAG=<tag>
```

## Best Practices

### 1. Always Use Help

```bash
make help              # Show all targets
make help grep=docker  # Filter help for docker targets
```

### 2. Check Tools Before Running

```bash
make tools-check
```

### 3. Confirm Before Destructive Operations

OTICA includes `confirm.sh` which is called by destructive targets automatically. For custom targets:

```makefile
my-destroy-target: ## Destroy resources
    @confirm.sh "Are you sure you want to destroy resources?"
    # ... destructive commands
```

To bypass the prompt in CI or other non-interactive runs, set `NONINTERACTIVE`
to any non-empty value — `confirm.sh` then auto-confirms (exits 0) without
reading from the terminal:

```bash
make my-destroy-target NONINTERACTIVE=1
```

### 4. Keep TF_BUILD_DIR Out of Git

`tf-render` checks that `TF_BUILD_DIR` is gitignored before running. Add it to `.gitignore`:

```
.tf_build/
```

### 5. Pin OTICA, Upgrade Deliberately

Pin `OTICA_VERSION` to a release tag so framework changes never land unexpectedly.
Upgrade on your schedule after reviewing the changelog:

```bash
# review CHANGELOG.md, bump OTICA_VERSION in env.mk, then:
make update-otica
```

### 6. Document Custom Targets

Always add help comments to custom targets:

```makefile
my-custom-target: ## Description of what this does
    @echo "Custom operation"
```

### 7. Use Environment-Specific Variables

Organize variables by environment:

```
project/
├── common/env.mk    # Shared variables
├── dev/env.mk
├── stage/env.mk
└── prod/env.mk
```

## Troubleshooting

### OTICA Not Found

**Error**: `include: ~/.otica/makefiles/help.mk: No such file or directory`

**Solution**:
```bash
git clone https://code.stanford.edu/iac/cloud-framework.git ~/.otica
```

### Missing Environment Variables

**Error**: `missing env var(s): VAR_NAME`

**Solution**: Define the required variable in `env.mk`:
```makefile
VAR_NAME=value
```

### Outdated OTICA Version

**Symptoms**: Missing targets or features, or a "newer release available" notice
on `make help`.

**Solution**: review the [`CHANGELOG.md`](../CHANGELOG.md) for breaking changes,
bump `OTICA_VERSION` in `env.mk` to the desired release tag (or `main`), then:
```bash
make update-otica
```

**Wrong version checked out?** `~/.otica` is shared by every project and `make`
re-asserts each project's `OTICA_VERSION` on the next run, so a stale checkout
self-corrects. For full isolation, give each project its own `OTICA_DIR`.

### Permission Denied on Scripts

**Error**: `Permission denied` when running scripts

**Solution**:
```bash
chmod +x ~/.otica/scripts/*.sh
```

docs/WORKFLOWS.md

0 → 100644
+119 −0
Original line number Diff line number Diff line
# OTICA Common Workflows

Copy-pasteable recipes for the most common OTICA operations. Run `make help` for the full
target list, see the [README](../README.md) for setup, and the
[Technical Reference](OTICA_TECHNICAL_REFERENCE.md) for module internals.

### Standard Terraform Deploy

```bash
make tf-env       # install correct TF version via tfenv
make tf-render    # render .tf.tmpl templates → .tf_build/
make tf-plan      # init + validate + plan
make tf-apply     # apply
```

### OPA Policy Validation (Source-Dir Mode)

For local development with a checked-out policy repository:

```bash
# Gate: plan then scan — exits 1 if violations found
make tf-opa-plan

# Gate: plan + scan + apply in one step
make tf-opa-apply

# Full report (violations + passing checks, never exits 1)
make opa-scan-report

# Violations as JSON (for CI / Splunk)
make opa-scan-json
```

### OPA Policy Validation (Bundle Mode)

For CI and production using a signed GitHub release bundle:

```bash
# Download bundle (cached; re-fetch with OPA_BUNDLE_FORCE=1)
make opa-bundle-download

# Gate: verify signature + scan; exits 1 on violations
make opa-bundle-scan

# Gate: plan + bundle scan + apply
make tf-opa-bundle-apply
```

Required in `env.mk`:
```makefile
OPA_BUNDLE_REPO=my-org/opa-policy-hub
```

### Policy Quality Gates (opa-policy-hub Developers)

```bash
make opa-quality       # fmt-check + syntax + regal lint + unit tests
make opa-test          # unit tests with coverage >= OPA_COVERAGE_THRESHOLD
make opa-fmt           # format .rego files in-place
```

### Vault Operations

```bash
make vault-login
make vault-read SEC_PATH=secret/myproject/prod KEY=db_password
make vault-write SEC_PATH=secret/myproject/prod KEY=db_password VAL=newvalue
make vault-list SEC_PATH=secret/myproject/prod
make vault-logout
```

### Azure Container App Deployment

```bash
make az-login
make tf-plan
make tf-apply
make aca-allow-me     # whitelist your IP
make aca-ssh          # SSH into container for debugging
# Promote to production (from prod directory):
make aca-release
```

### GKE Cluster Maintenance (Cordon / Drain)

```bash
make gke-login
make gke-cordon      # interactively select nodes to cordon
make gke-drain       # drain all cordoned nodes
# ... perform maintenance ...
make gke-uncordon    # restore scheduling
```

### GitLab CI/CD Secret Management

```bash
# Define secrets in .gitlab-ci.sec, then:
make gl-add-sec      # push secrets to GitLab CI/CD variables
make gl-rm-sec       # remove secrets
```

### Docker Image Build and Push

```bash
make docker-build
make docker-push
git tag v1.2.3 && git push --tags
```


## Agentic workflow (developer/maintainer)

OTICA ships an optional set of cooperating Claude agents that drive the GitLab
**issue → fix → review → merge** lifecycle (reporter → resolver → reviewer → integrator),
coordinating only through GitLab scoped labels. It is a maintainer workflow, not part of a
consumer deploy. Full runbook — setup, deployment modes (background vs. unattended cron),
identity model, and every knob:
**[../agent-instructions/agentic-workflow.md](../agent-instructions/agentic-workflow.md)**
(quick summary in the [README](../README.md#agentic-workflow-developermaintainer)).