# Private services with VPC tunnels

> Publish a private service through the default Gateway, or call it from an Apoxy compute Service.

This guide connects a private HTTP service to Apoxy's VPC overlay, publishes it through the
default Gateway, and then calls the same service from an Apoxy compute Service.

<Callout label="Alpha" variant="warn">
VPC tunnels use `apoxy alpha tunnel`. Pin the CLI version in automation and test upgrades before
rolling them into production.
</Callout>

The direct request path is:

```text
HTTPS → default Gateway → HTTPRoute → VPCService → tunnel → private service
```

## Check prerequisites

- The Apoxy CLI is installed and authenticated.
- The managed Apoxy project is ready.
- You have a domain in Apoxy. See [Custom domains](/docs/guides/custom-domains.md) if you still need one.
- Python 3 is available for the demo backend.

Check the active project before creating anything:

```bash title="terminal"
apoxy auth --check
apoxy gateway get default
apoxy vpc network get default
apoxy vpc relay list
```

## Publish a private service

### Start the backend

Run a small HTTP server on the private machine:

```bash title="terminal"
mkdir -p private-api
printf '%s\n' 'Hello through an Apoxy VPC tunnel.' > private-api/index.html
python3 -m http.server 8080 --directory private-api
```

Leave it running.

### Connect the tunnel

In another terminal, connect the machine to the default VPC network:

```bash title="terminal"
apoxy alpha tunnel run \
  --name private-api \
  --vpc default \
  --admin-addr localhost:8081 \
  --socks-addr ""
```

The admin listener is disabled by default. This command enables it on port 8081, so the backend
can keep port 8080. Use `/livez` for process health, `/readyz` for connection readiness, and
`/metrics` for Prometheus metrics. VPC peers cannot reach the admin listener through the tunnel.

The agent registers the label `tunnel.apoxy.dev/name=private-api`, which the VPCService below
selects.

### Select the tunnel with a VPCService

```yaml title="vpc-service.yaml"
apiVersion: vpc.apoxy.dev/v1alpha1
kind: VPCService
metadata:
  name: private-api
spec:
  networkRef:
    name: default
  selector:
    matchLabels:
      tunnel.apoxy.dev/name: private-api
```

Apply it and confirm that it has at least one endpoint:

```bash title="terminal"
apoxy apply -f vpc-service.yaml
apoxy vpc service get private-api -o yaml
```

Continue when the `Ready` condition is `True` and `status.endpoints` is not empty.

You can also watch this from the [Apoxy console](https://dashboard.apoxy.dev): the **VPC** page
lists your networks, and each network's detail page shows its VPC services with their member
tunnels, the live tunnel connections, and the relays they terminate on. A service that shows
`NoEndpoints` has no connected tunnel matching its selector yet.

### Attach the hostname to the default Gateway

Replace `api.example.com` with your hostname. Omit `metadata.name`; Apoxy generates it from the
hostname and target type.

```yaml title="domain.yaml"
apiVersion: core.apoxy.dev/v1alpha3
kind: DomainRecord
spec:
  name: api.example.com
  tls: {}
  target:
    ref:
      group: gateway.apoxy.dev
      kind: Gateway
      name: default
```

For a hostname below an Apoxy-managed zone, also set `spec.zone` to that zone's root. For a
customer-owned hostname, publish the CNAME described in [Custom domains](/docs/guides/custom-domains.md).

Apply the DomainRecord before the HTTPRoute. Route admission rejects hostnames that are not yet
claimed by the project.

```bash title="terminal"
apoxy apply -f domain.yaml
```

### Route to the VPCService

Reference the VPCService directly from an HTTPRoute:

```yaml title="route.yaml"
apiVersion: gateway.apoxy.dev/v1
kind: HTTPRoute
metadata:
  name: private-api
spec:
  parentRefs:
  - name: default
    port: 443
  hostnames:
  - api.example.com
  rules:
  - backendRefs:
    - group: vpc.apoxy.dev
      kind: VPCService
      name: private-api
      port: 8080
```

```bash title="terminal"
apoxy apply -f route.yaml
```

The `port` is the port your private service listens on behind the agent. Traffic uses
HTTP/1.1 by default; set `spec.appProtocol` on the VPCService to `kubernetes.io/h2c` for
cleartext HTTP/2 or `grpc` for gRPC upstreams.

A Backend that wraps the service's VPC name works the same way and remains useful when you
need Backend-only features such as TLS to the upstream:

```yaml title="backend-route.yaml"
apiVersion: core.apoxy.dev/v1alpha2
kind: Backend
metadata:
  name: private-api
spec:
  endpoints:
  - fqdn: private-api.default.vpc.apoxy.net
---
# Reference it with group: core.apoxy.dev, kind: Backend in the backendRefs.
```

VPC-backed routes are not limited to the default Gateway. An HTTPRoute on a dedicated
Gateway (one bound to its own Proxy through `infrastructure.parametersRef`) reaches the
VPCService the same way; only the `parentRefs` entry changes.

When a VPCService selects agents in more than one location, requests automatically prefer
the lowest-latency healthy path and spill over to the others on failure.

## Verify the direct route

Check certificate status, then send a public request:

```bash title="terminal"
apoxy domain list \
  --field-selector spec.name=api.example.com \
  -o yaml
curl https://api.example.com/
```

The first certificate normally takes 30-60 seconds. A successful request returns
`Hello through an Apoxy VPC tunnel.`

## Call the VPCService from a compute Service

Compute Services use the same project-scoped VPC name with ordinary `fetch()`. There is no
VPCService binding to add to the Service manifest.

Create a worker in a new directory:

```js title="private-worker/index.js"
const upstream = "http://private-api.default.vpc.apoxy.net:8080/";

export default {
  async fetch() {
    const response = await fetch(upstream);
    return Response.json({
      upstream,
      status: response.status,
      body: await response.text(),
    });
  },
};
```

```yaml title="private-worker/service.yaml"
apiVersion: compute.apoxy.dev/v1alpha1
kind: Service
metadata:
  name: private-worker
```

Deploy it:

```bash title="terminal"
cd private-worker
apoxy auth --check
apoxy deploy . --yes
apoxy compute service get private-worker -o yaml
cd ..
```

The CLI requires `--yes` for a production project. Always inspect the active project immediately
before that command; omit the flag when you want the safety check to stop a mistaken deployment.

Create a second DomainRecord and route for the worker:

```yaml title="worker-domain.yaml"
apiVersion: core.apoxy.dev/v1alpha3
kind: DomainRecord
spec:
  name: worker.example.com
  tls: {}
  target:
    ref:
      group: gateway.apoxy.dev
      kind: Gateway
      name: default
```

```yaml title="worker-route.yaml"
apiVersion: gateway.apoxy.dev/v1
kind: HTTPRoute
metadata:
  name: private-worker
spec:
  parentRefs:
  - name: default
    port: 443
  hostnames:
  - worker.example.com
  rules:
  - backendRefs:
    - group: compute.apoxy.dev
      kind: Service
      name: private-worker
```

Apply the domain first, then the route:

```bash title="terminal"
apoxy apply -f worker-domain.yaml
apoxy apply -f worker-route.yaml
```

## Verify the compute route

```bash title="terminal"
apoxy domain list \
  --field-selector spec.name=worker.example.com \
  -o yaml
curl https://worker.example.com/
```

The response includes the VPC upstream, its `200` status, and the private backend body. Compute
egress uses the project's default EgressGateway unless the Service selects another one; see
[Outbound traffic](/docs/guides/deploying-compute-services.md#outbound-traffic-egress) for policy controls.

<Callout type="warn">
**Cloud Run is not yet supported.** Default Cloud Run egress drops the agent's 1,280-byte QUIC
Initial above its measured 1,252-byte UDP payload limit. Direct VPC egress reaches the relay, but
the current userspace data plane then fails underlay address-family selection. Kernel mode also
needs `NET_ADMIN` and `/dev/net/tun`, which Cloud Run does not expose. Use a VM or Kubernetes. For
AWS, follow [EC2 kernel-mode VPC tunnels](/docs/guides/ec2-vpc-tunnels.md). For Google Cloud, follow
[GCE userspace VPC tunnels](/docs/guides/gce-vpc-tunnels.md).
</Callout>

## Read tunnel metrics

The metrics API reports what the tunnels of a VPC network carried. It counts every packet, so
east-west traffic and non-HTTP protocols appear here and not in the `http.*` recipes.

Point the two variables at your project:

```bash title="terminal"
export APOXY_API_KEY="<api-key>"
export APOXY_API="https://<project-id>.api.apoxy.dev"
```

Read the last hour for the network, broken down per tunnel:

```bash title="terminal"
curl -s -H "X-Apoxy-API-Key: $APOXY_API_KEY" \
  "$APOXY_API/apis/vpc.apoxy.dev/v1alpha1/vpcnetworks/default/metrics?window=1h&include=tunnels"
```

You should see the network totals under `metrics`, with `network.bytes`, `network.packets`,
`network.drops`, `network.keepalives`, and `network.tunnels`, and a `tunnels` array carrying the
same recipes per connection. Add `include=services` for the gateway traffic that reached each VPC service.

List the connections to get a tunnel name:

```bash title="terminal"
apoxy vpc tunnel list
```

Then read one connection on its own:

```bash title="terminal"
curl -s -H "X-Apoxy-API-Key: $APOXY_API_KEY" \
  "$APOXY_API/apis/vpc.apoxy.dev/v1alpha1/tunnels/<tunnel-name>/metrics?window=1h"
```

That response adds `tunnel.rtt`, the round trip time between the agent and its relay, in seconds.

A tunnel's history belongs to one connection. When the agent reconnects it gets a new `Tunnel` with
a new name, and its metrics start over with it. Read the network with `include=tunnels` for a view
that spans reconnects. See the [Metrics API reference](/docs/reference/metrics.md) for the full parameter
list and the time-series form of the same recipes.

## Clean up

Stop the local tunnel and backend with Ctrl-C, then remove the resources you applied:

```bash title="terminal"
apoxy delete -f worker-route.yaml
apoxy delete -f worker-domain.yaml
apoxy delete -f private-worker/service.yaml
apoxy delete -f route.yaml
apoxy delete -f domain.yaml
apoxy delete -f vpc-service.yaml
```

---

**Navigation** (Guides)

- Previous: [Tunnels with Docker](/docs/guides/tunnels-with-docker.md)
- Next: [GCE userspace VPC tunnels](/docs/guides/gce-vpc-tunnels.md)
- All pages: [index](/docs/llms.txt)