Memory lifecycle management

Memories are not permanent. They have a lifecycle: creation, active use, consolidation, archival, and deletion. Astrocyte manages this lifecycle with policy-driven automation and compliance-grade controls.

This maps to Principle 7 (Pruning / phagocytosis) - structured forgetting is a first-class concern, not an afterthought.

1. Lifecycle states

Created → Active → Consolidated → Archived → Deleted
   │         │          │             │
   │         │          │             └── TTL expiry, compliance purge, manual
   │         │          └── Merged into observations, source facts optionally archived
   │         └── Being recalled, contributing to reflects
   └── Just stored, not yet recalled

1.1 State definitions

State	Description	Storage
Active	Recently created or recently recalled. Fully indexed and searchable.	Hot storage (primary vector/graph store)
Consolidated	Merged into higher-order observations. Source facts may remain active or be archived.	Hot storage (observations active, source facts optionally archived)
Archived	No longer indexed for search. Retained for audit, compliance, or potential reactivation.	Cold storage (if supported by provider)
Deleted	Permanently removed. Irreversible.	Removed from all stores

2. TTL policies

Time-to-live policies automatically transition memories through lifecycle states.

lifecycle:
  ttl:
    # Archive memories that haven't been recalled in N days
    archive_after_days: 90

    # Delete archived memories after N days
    delete_after_days: 365

    # Never auto-delete memories with these tags (compliance hold)
    exempt_tags: ["legal_hold", "compliance"]

    # Per-fact-type TTLs (override global)
    fact_type_overrides:
      observation: null              # Observations never expire
      experience: 180               # Experiences archive after 180 days
      world: 365                    # World facts archive after 365 days

2.1 Recall freshness tracking

The pipeline tracks when each memory was last recalled (via metadata). Memories that are never recalled are candidates for archival. Memories that are frequently recalled stay active indefinitely.

# Internal metadata updated on every recall hit
{
  "_last_recalled_at": "2026-04-01T10:00:00Z",
  "_recall_count": 7,
  "_created_at": "2026-01-15T10:30:00Z",
}

3. Compliance operations

# Delete all memories in a bank (convenience method)
await brain.clear_bank("user-123")

# Equivalent to:
await brain.forget("user-123", scope="all")

Note: scope="all" requires admin permission when access control is enabled. The clear_bank() method is the recommended public API for bank-wide deletion. For Tier 1 pipelines, this paginates through VectorStore.list_vectors() and deletes all vectors in batches.

When compliance=True:

All memories are permanently deleted (not archived)
Embeddings are removed from vector stores
Entity references are cleaned up in graph stores
An audit event is emitted (see section 5)
The operation is idempotent (safe to retry)

3.2 Selective forget

# Forget specific memories by ID
await brain.forget("user-123", memory_ids=["mem-abc", "mem-def"])

# Forget by tag and/or date
await brain.forget(
    "user-123",
    tags=["sensitive"],
    before_date=datetime(2026, 1, 1),
)

3.3 Legal hold

Memories under legal hold are exempt from TTL expiry and cannot be deleted via normal forget() calls:

# Place a bank under legal hold (synchronous)
brain.set_legal_hold(
    bank_id="user-123",
    hold_id="case-2026-001",
    reason="Litigation hold per legal request #LH-2026-001",
)

# Release hold (synchronous)
brain.release_legal_hold(
    bank_id="user-123",
    hold_id="case-2026-001",
)

While under hold:

TTL policies are suspended for the bank
forget() calls return LegalHoldActive error
Retain and recall continue normally
The hold is logged in the audit trail

4. Consolidation lifecycle

Consolidation (see built-in-pipeline.md section 5) creates observations from raw facts. The lifecycle integration:

Pre-consolidation: raw facts are Active
Consolidation runs: observations are created from clusters of related facts
Post-consolidation: observations are Active, source facts can be:
- Kept active (default) - both observation and source facts are searchable
- Archived - source facts move to cold storage, observations remain searchable
- Deleted - source facts removed, observations are the only record

lifecycle:
  consolidation:
    source_fact_policy: keep_active     # "keep_active" | "archive" | "delete"
    min_facts_for_consolidation: 5      # Don't consolidate until N related facts exist
    consolidation_schedule: "0 3 * * *" # Cron: 3am daily

5. Audit trail

All lifecycle transitions are recorded as audit events.

@dataclass
class AuditEvent:
    event_type: str                      # "created", "recalled", "archived", "deleted", "legal_hold", etc.
    bank_id: str
    memory_ids: list[str] | None
    actor: str                           # "system:ttl", "system:consolidation", "user:api", "compliance:forget"
    reason: str | None
    timestamp: datetime
    metadata: dict[str, Any] | None      # Additional context

5.1 Audit event types

Event	Actor	Description
`memory.created`	`user:api`	Memory stored via retain
`memory.recalled`	`user:api`	Memory included in recall results
`memory.archived`	`system:ttl`	TTL policy moved memory to archive
`memory.deleted`	`system:ttl`	TTL policy permanently deleted memory
`memory.deleted`	`compliance:forget`	Compliance purge deleted memory
`memory.consolidated`	`system:consolidation`	Memories merged into observation
`bank.legal_hold.set`	`user:api`	Legal hold placed on bank
`bank.legal_hold.released`	`user:api`	Legal hold released
`bank.created`	`user:api` or `system:template`	New bank created
`bank.deleted`	`user:api`	Bank and all memories deleted

5.2 Audit storage

Audit events are:

Emitted as OTel span events (see policy-layer.md)
Optionally written to an audit log (file, database, or external system via hooks - see event-hooks.md)
Never stored in the memory provider itself (the audit trail must survive provider changes)

lifecycle:
  audit:
    enabled: true
    sink: file                           # "file" | "webhook" | "otel_only"
    file_path: ./audit/astrocyte.audit.jsonl
    retention_days: 2555                 # 7 years for compliance

6. Lifecycle execution

6.1 Scheduled (background)

Status: The lifecycle.scheduler and lifecycle.audit config sections are design targets — not yet implemented in config.py. Currently, lifecycle runs are triggered on-demand via brain.run_lifecycle().

TTL checks and consolidation run on a schedule (planned):

lifecycle:
  scheduler:
    enabled: true
    ttl_check_schedule: "0 2 * * *"      # 2am daily
    consolidation_schedule: "0 3 * * *"  # 3am daily

6.2 On-demand

# Manually trigger lifecycle (TTL check + consolidation)
await brain.run_lifecycle(bank_id="user-123")

Note: run_ttl_check() and run_consolidation() as separate methods are a design target. Currently, run_lifecycle() handles both.

6.3 Provider interaction

Tier 1 (Storage): Astrocyte orchestrates lifecycle by calling retrieval SPI methods (delete, update metadata).
Tier 2 (Memory Engine): Astrocyte delegates lifecycle to the memory engine if it supports it (supports_consolidation in capabilities). Falls back to Astrocyte-managed lifecycle if not.