refactor(keybase): migrate to UCAN v1.0.0-rc.1 envelope format

2026-01-08 16:37:36 -05:00
parent 1b2a57ca98
commit 4fbfdf1e4d
11 changed files with 1465 additions and 999 deletions
--- a/TODO.md
+++ b/TODO.md
@@ -11,7 +11,7 @@ Remaining tasks from [MIGRATION.md](./MIGRATION.md) for the Nebula Key Enclave.
 | Generated Code | Complete | `internal/keybase/*.go` |
 | Basic Plugin Functions | Complete | `generate`, `load`, `exec`, `query`, `ping` |
 | Encryption | Not Started | WebAuthn PRF key derivation needed |
-| **UCAN v1.0.0-rc.1** | **CRITICAL** | Go implementation uses deprecated JWT format |
+| **UCAN v1.0.0-rc.1** | **In Progress** | Core types, builders, and policies complete. Uses `go-ucan v1.1.0` |
 | MPC Key Shares | Not Started | Key share management missing |
 | Database Serialization | Incomplete | Export dumps comments only |

@@ -19,114 +19,116 @@ Remaining tasks from [MIGRATION.md](./MIGRATION.md) for the Nebula Key Enclave.

 ## 1. UCAN v1.0.0-rc.1 Migration (CRITICAL PRIORITY)

-> **Breaking Change**: Current Go implementation (`internal/crypto/ucan/`) uses deprecated JWT-based UCAN format. Must migrate to envelope format per v1.0.0-rc.1 spec.
+> **Status**: Core implementation complete using `github.com/ucan-wg/go-ucan v1.1.0`. Deprecated JWT-based files deleted. Remaining work is database integration and MPC signing.

-### Current State (DEPRECATED - Must Replace)
+### Completed Implementation

-The following files use the **old JWT-based format** and must be rewritten:
+The following files implement UCAN v1.0.0-rc.1 using the official go-ucan library:

-| File | Status | Issue |
-|------|--------|-------|
-| `jwt.go` | DEPRECATED | Uses `github.com/golang-jwt/jwt/v5`, old `can`+`with` format |
-| `capability.go` | DEPRECATED | Old Attenuation/Resource/Capability model |
-| `verifier.go` | DEPRECATED | JWT parsing, old proof chain format |
-| `source.go` | DEPRECATED | JWT token creation with MPC |
-| `vault.go` | PARTIAL | VaultCapability needs Policy migration |
+| File | Status | Description |
+|------|--------|-------------|
+| `ucan.go` | ✅ Complete | Type re-exports, Sonr commands, pre-parsed constants |
+| `policy.go` | ✅ Complete | PolicyBuilder fluent API, Sonr-specific policy helpers |
+| `delegation.go` | ✅ Complete | DelegationBuilder fluent API, Sonr delegation helpers |
+| `invocation.go` | ✅ Complete | InvocationBuilder fluent API, Sonr invocation helpers |
+| `types.go` | ✅ Complete | ValidationError, Capability, ExecutionResult, Sonr types |

-### Reference Implementation (Already Compliant)
+### Dependencies Added

-These files are already aligned with v1.0.0-rc.1:
+- `github.com/ucan-wg/go-ucan v1.1.0` - Official UCAN library
+- `github.com/ipld/go-ipld-prime v0.21.0` - IPLD encoding
+- `github.com/MetaMask/go-did-it v1.0.0-pre1` - DID handling (indirect)
+- `github.com/ipfs/go-cid v0.5.0` - Content addressing (indirect)

- `src/ucan.ts` - TypeScript types with envelope format
- `internal/codec/ucan-schemas.json` - JSON Schema definitions
+### Deleted (Deprecated JWT-based)
+
+- ~~`jwt.go`~~ - Removed
+- ~~`capability.go`~~ - Removed
+- ~~`verifier.go`~~ - Removed
+- ~~`source.go`~~ - Removed
+- ~~`internal/crypto/mpc/spec/`~~ - Entire directory removed

 ### 1.1 Core Data Structures

- [ ] Create `internal/crypto/ucan/types.go` - v1.0.0-rc.1 types
-  - [ ] `DelegationPayload` struct (iss, aud, sub, cmd, pol, nonce, meta, nbf, exp)
-  - [ ] `InvocationPayload` struct (iss, sub, aud, cmd, args, prf, meta, nonce, exp, iat, cause)
-  - [ ] `Delegation` type as `[Signature, DelegationSigPayload]` tuple
-  - [ ] `Invocation` type as `[Signature, InvocationSigPayload]` tuple
-  - [ ] `Task` struct (sub, cmd, args, nonce)
-  - [ ] `ReceiptPayload` struct (iss, ran, out, fx, meta, iat)
-  - [ ] `RevocationPayload` struct
+- [x] Create `internal/crypto/ucan/types.go` - v1.0.0-rc.1 types
+  - [x] Re-export `Delegation` and `Invocation` from go-ucan
+  - [x] `Task` struct (sub, cmd, args, nonce)
+  - [x] `ReceiptPayload` struct (iss, ran, out, fx, meta, iat)
+  - [x] `RevocationPayload` struct
+  - [x] `ValidationError` with error codes matching TypeScript
+  - [x] `Capability` struct (sub, cmd, pol)
+  - [x] `ExecutionResult[T, E]` generic type
+  - [x] Sonr-specific types: `VaultCapability`, `DIDCapability`, `DWNCapability`

- [ ] Create `internal/crypto/ucan/policy.go` - Policy Language
-  - [ ] `PolicyStatement` union type
-  - [ ] `EqualityStatement` - `["==", selector, value]` / `["!=", selector, value]`
-  - [ ] `InequalityStatement` - `[">", selector, number]` etc.
-  - [ ] `LikeStatement` - `["like", selector, glob]`
-  - [ ] `NotStatement` - `["not", statement]`
-  - [ ] `AndStatement` / `OrStatement` - logical connectives
-  - [ ] `AllStatement` / `AnyStatement` - quantifiers
-  - [ ] `Selector` parser (jq-inspired: `.foo`, `.bar[0]`, `.items[-1]`)
-  - [ ] `GlobPattern` matcher
+- [x] Create `internal/crypto/ucan/policy.go` - Policy Language
+  - [x] `PolicyBuilder` fluent API with all operators
+  - [x] `Equal`, `NotEqual` - equality statements
+  - [x] `GreaterThan`, `LessThan`, etc. - inequality statements
+  - [x] `Like` - glob pattern matching
+  - [x] `Not`, `And`, `Or` - logical connectives
+  - [x] `All`, `Any` - quantifiers
+  - [x] Sonr helpers: `VaultPolicy`, `DIDPolicy`, `ChainPolicy`, `AccountPolicy`

- [ ] Create `internal/crypto/ucan/command.go` - Command types
-  - [ ] `Command` type with validation (must start with `/`, lowercase, no trailing slash)
-  - [ ] Standard commands: `/crud/*`, `/msg/*`, `/ucan/revoke`, `/wasm/run`
-  - [ ] Custom Sonr commands: `/vault/*`, `/did/*`, `/dwn/*`
+- [x] Create `internal/crypto/ucan/ucan.go` - Command types
+  - [x] `Command` type re-exported from go-ucan
+  - [x] Sonr commands: `/vault/*`, `/did/*`, `/dwn/*`, `/ucan/revoke`
+  - [x] Pre-parsed command constants: `VaultRead`, `VaultWrite`, `DIDUpdate`, etc.
+  - [x] `CommandSubsumes()` helper using go-ucan's `Covers()` method

 ### 1.2 Envelope Format & Encoding

- [ ] Create `internal/crypto/ucan/envelope.go` - Envelope handling
-  - [ ] `UCANEnvelope[P]` generic type as `[Signature, {h: VarsigHeader} & P]`
-  - [ ] Encode envelope to CBOR (requires `github.com/fxamacker/cbor/v2`)
-  - [ ] Decode envelope from CBOR
-  - [ ] DAG-JSON encoding for interop
-  - [ ] CID computation (DAG-CBOR codec, SHA-256 multihash, base58btc)
+- [x] Envelope handling via go-ucan library
+  - [x] `ToSealed()` method produces DAG-CBOR bytes + CID
+  - [x] `ToDagCbor()`, `ToDagJson()` encoding methods
+  - [x] CID computation handled by go-ucan

- [ ] Create `internal/crypto/ucan/varsig.go` - Varsig v1 headers
-  - [ ] Varsig header encoding/decoding
-  - [ ] Algorithm metadata extraction
-  - [ ] Support Ed25519, P-256, secp256k1
+- [x] Varsig support via go-ucan library
+  - [x] Ed25519, P-256, secp256k1 via `go-did-it/crypto`

 ### 1.3 Delegation Operations

- [ ] Create `internal/crypto/ucan/delegation.go` - Delegation creation/validation
-  - [ ] `NewDelegation(issuer, audience, subject, cmd, policy, exp, meta)` builder
-  - [ ] Sign delegation with issuer private key
-  - [ ] Validate delegation signature
-  - [ ] Validate delegation payload (temporal, structural)
-  - [ ] Extract `Capability` from delegation (sub + cmd + pol)
+- [x] Create `internal/crypto/ucan/delegation.go` - Delegation creation/validation
+  - [x] `DelegationBuilder` fluent API
+  - [x] `NewDelegation`, `NewRootDelegation`, `NewPowerlineDelegation` re-exports
+  - [x] `BuildSealed(privKey)` for signing
+  - [x] Sonr helpers: `NewVaultDelegation`, `NewDIDDelegation`, `NewDWNDelegation`
+  - [x] Temporal options: `ExpiresAt`, `ExpiresIn`, `NotBefore`, `NotBeforeIn`

 ### 1.4 Invocation Operations

- [ ] Create `internal/crypto/ucan/invocation.go` - Invocation creation/validation
-  - [ ] `NewInvocation(issuer, subject, cmd, args, proofs, exp)` builder
-  - [ ] Sign invocation with invoker private key
-  - [ ] Validate invocation signature
-  - [ ] Validate proof chain (CID references to delegations)
-  - [ ] Evaluate policies against invocation args
+- [x] Create `internal/crypto/ucan/invocation.go` - Invocation creation/validation
+  - [x] `InvocationBuilder` fluent API
+  - [x] `NewInvocation` re-export
+  - [x] `BuildSealed(privKey)` for signing
+  - [x] Proof chain management: `Proof()`, `Proofs()`
+  - [x] Sonr helpers: `VaultReadInvocation`, `VaultSignInvocation`, `DIDUpdateInvocation`

 ### 1.5 Policy Evaluation Engine

- [ ] Create `internal/crypto/ucan/eval.go` - Policy evaluation
-  - [ ] `EvaluatePolicy(policy Policy, args Arguments) (bool, error)`
-  - [ ] Selector resolution against IPLD data
-  - [ ] Equality comparison (deep IPLD equality)
-  - [ ] Numeric comparisons
-  - [ ] Glob pattern matching for `like` operator
-  - [ ] Logical connectives (`and`, `or`, `not`)
-  - [ ] Quantifiers (`all`, `any`) over collections
+> Note: go-ucan provides `ExecutionAllowed()` on invocations which validates proofs and evaluates policies.
+
+- [x] Policy evaluation via go-ucan's `invocation.ExecutionAllowed(loader)`
+- [ ] Create `internal/crypto/ucan/eval.go` - Additional evaluation helpers (if needed)
+  - [ ] Custom selector resolution for Sonr-specific args
+  - [ ] Caching layer for repeated evaluations

 ### 1.6 Proof Chain Validation

- [ ] Create `internal/crypto/ucan/chain.go` - Chain validation
-  - [ ] Resolve CID to Delegation (requires delegation store)
-  - [ ] Validate chain continuity (child.iss == parent.aud)
-  - [ ] Validate capability attenuation (child.cmd subsumes parent.cmd)
-  - [ ] Validate policy attenuation (child.pol more restrictive than parent.pol)
-  - [ ] Validate temporal bounds (child.exp <= parent.exp)
-  - [ ] Check revocation status for all chain members
+> Note: go-ucan handles chain validation internally via `ExecutionAllowed()`.
+
+- [x] Chain validation via go-ucan library
+- [ ] Create `internal/crypto/ucan/store.go` - Delegation store
+  - [ ] Implement `delegation.Loader` interface
+  - [ ] `GetDelegation(cid.Cid) (*delegation.Token, error)`
+  - [ ] Cache loaded delegations for performance

 ### 1.7 Revocation

+- [x] `RevocationInvocation()` helper in `invocation.go`
 - [ ] Create `internal/crypto/ucan/revocation.go` - Revocation handling
-  - [ ] `NewRevocation(revoker, delegation_cid)` builder
-  - [ ] Validate revoker is in delegation's issuer chain
-  - [ ] Store revocation in database
-  - [ ] Query revocation status by CID
+  - [ ] Revocation store implementation
+  - [ ] `IsRevoked(cid.Cid) (bool, error)` query
+  - [ ] Integration with chain validation

 ### 1.8 Database Integration

@@ -141,20 +143,18 @@ These files are already aligned with v1.0.0-rc.1:
  - [ ] `InsertInvocation`, `GetInvocationByCID`
  - [ ] `InsertRevocation`, `IsRevoked`, `GetRevocationsByDelegation`

-### 1.9 Migration from Old Format
+### 1.9 MPC Signing Integration

- [ ] Create migration script for existing UCAN data (if any)
- [ ] Remove deprecated files after migration complete:
-  - [ ] `jwt.go` - Remove entirely
-  - [ ] `capability.go` - Replace with policy-based capabilities
-  - [ ] `verifier.go` - Replace with envelope-based verification
-  - [ ] `source.go` - Replace with envelope-based token creation
+- [ ] Create `internal/crypto/ucan/signer.go` - MPC key integration
+  - [ ] Implement `crypto.PrivateKeySigningBytes` interface for MPC
+  - [ ] Sign delegations with MPC key shares
+  - [ ] Sign invocations with MPC key shares

 ### 1.10 Testing

- [ ] Unit tests for policy evaluation
- [ ] Unit tests for envelope encoding/decoding
- [ ] Unit tests for chain validation
+- [ ] Unit tests for builders (DelegationBuilder, InvocationBuilder)
+- [ ] Unit tests for policy helpers
+- [ ] Unit tests for Sonr-specific invocations
 - [ ] Interoperability tests against TypeScript implementation
 - [ ] Test vectors from UCAN spec

@@ -478,23 +478,26 @@ These files are already aligned with v1.0.0-rc.1:

 ## Priority Order

-1. **CRITICAL (Spec Compliance)**
-   - UCAN v1.0.0-rc.1 Migration (Section 1)
-   - Core data structures (1.1)
-   - Envelope format (1.2)
-   - Delegation operations (1.3)
-   - Policy evaluation (1.5)
+1. **CRITICAL (Spec Compliance)** - ✅ Core Complete
+   - ~~UCAN v1.0.0-rc.1 Migration (Section 1)~~ ✅ Core types, builders, policies done
+   - ~~Core data structures (1.1)~~ ✅ Using go-ucan v1.1.0
+   - ~~Envelope format (1.2)~~ ✅ Handled by go-ucan
+   - ~~Delegation operations (1.3)~~ ✅ DelegationBuilder complete
+   - ~~Invocation operations (1.4)~~ ✅ InvocationBuilder complete
+   - Database integration (1.8) - Next priority
+   - MPC signing integration (1.9) - Next priority

 2. **High Priority (Core Functionality)**
   - Database Serialization (3.1, 3.2)
   - Credential Creation (6.2, 4.7)
   - Key Share Actions (4.1)
   - Account Actions (4.6)
+   - UCAN Database Integration (1.8)

 3. **Medium Priority (Authorization)**
-   - Invocation operations (1.4)
-   - Proof chain validation (1.6)
-   - Revocation (1.7)
+   - Delegation store (1.6)
+   - Revocation store (1.7)
+   - MPC Signing (1.9)
   - Encryption Strategy (2.1, 2.2)

 4. **Lower Priority (Enhancement)**
@@ -506,13 +509,39 @@ These files are already aligned with v1.0.0-rc.1:

 ---

+## Completed Items
+
+### UCAN v1.0.0-rc.1 Core (January 2025)
+
+The following was completed using `github.com/ucan-wg/go-ucan v1.1.0`:
+
+- ✅ Type re-exports from go-ucan (Delegation, Invocation, Command, Policy)
+- ✅ Sonr command constants (/vault/*, /did/*, /dwn/*)
+- ✅ DelegationBuilder fluent API with Sonr-specific helpers
+- ✅ InvocationBuilder fluent API with Sonr-specific helpers
+- ✅ PolicyBuilder fluent API with all operators
+- ✅ Sonr policy helpers (VaultPolicy, DIDPolicy, ChainPolicy)
+- ✅ ValidationError types matching TypeScript definitions
+- ✅ Capability, ExecutionResult, and related types
+
+### Deleted (Deprecated JWT-based)
+
+- ✅ Deleted `jwt.go` - Old JWT token handling
+- ✅ Deleted `capability.go` - Old Attenuation/Resource/Capability model
+- ✅ Deleted `verifier.go` - Old JWT verification
+- ✅ Deleted `source.go` - Old JWT token creation
+- ✅ Deleted `internal/crypto/mpc/spec/` - Old MPC JWT integration
+- ✅ Removed `github.com/golang-jwt/jwt/v5` dependency
+
+---
+
 ## Deprecated Items (Removed)

 The following items from the previous TODO have been removed as they reference the **deprecated JWT-based UCAN format**:

- ~~Section 4.1 "Token Validation" - JWT parsing~~ -> Replaced by envelope validation (1.2, 1.3)
- ~~Section 4.2 "Capability Verification" - `can`/`with` format~~ -> Replaced by policy evaluation (1.5)
- ~~Section 4.3 "Proof Chain Validation" - JWT proof strings~~ -> Replaced by CID-based chain (1.6)
+- ~~Section 4.1 "Token Validation" - JWT parsing~~ -> Replaced by go-ucan validation
+- ~~Section 4.2 "Capability Verification" - `can`/`with` format~~ -> Replaced by policy evaluation
+- ~~Section 4.3 "Proof Chain Validation" - JWT proof strings~~ -> Replaced by CID-based chain
 - ~~Section 3.2 "UCAN Token Actions" - Old format~~ -> Replaced by v1.0.0-rc.1 actions (4.2)
 - ~~Section 3.3 "Delegation Actions" - Old delegation model~~ -> Merged into Section 1 and 4.2