Skip to main content

Processing Checkpoints

Blue documents evolve as they process events, changing their state—and therefore their BlueId—while maintaining their fundamental identity. Processing checkpoints capture this temporal dimension, serving as formal markers in a document's ongoing journey.

The Need for State Tracking​

Consider a document that processes events from a timeline:

name: Simple Counter
counter: 0
contracts:
  incrementChannel:
    type: Timeline Channel
    account: user@example.com

  counterWorkflow:
    type: Sequential Workflow
    channel: incrementChannel
    steps:
      - name: Update Counter
        type: Update Document
        changeset:
          - op: replace
            path: /counter
            val: ${document('/counter') + 1}

As this document processes events, its state changes:

  • After one event: counter: 1
  • After two events: counter: 2
  • After three events: counter: 3

Each state change creates a new BlueId—a new content hash representing the document's new reality. Without some record of which events created this state, processors would have no way to know where in the event stream this document currently exists.

The Processing Checkpoint Contract​

The Processing Checkpoint contract serves as a formal record of a document's position in time:

name: Simple Counter
counter: 2
contracts:
  incrementChannel:
    type: Timeline Channel
    account: user@example.com

  counterWorkflow:
    type: Sequential Workflow
    channel: incrementChannel
    steps:
      - name: Update Counter
        type: Update Document
        changeset:
          - op: replace
            path: /counter
            val: ${document('/counter') + 1}

  checkpoint:
    type: Processing Checkpoint
    # Implementation details follow

This contract is optional, but when present, processors are expected to automatically update it after processing each event, maintaining an accurate record of the document's temporal position.

Common Checkpoint Types​

The Blue language allows various checkpoint implementations. Here are the most common types:

Channel Event Checkpoint​

This approach records the BlueId of the last processed event from each channel:

checkpoint:
  type: Channel Event Checkpoint
  lastEvents:
    incrementChannel:
      blueId: 7UEBwTmRMfQ92rGt4vHkzPa8Ypd5KJsLNcA3FV6xDqbn # Last processed event

This approach:

  • Provides exact tracking of which events were processed
  • Works reliably across decentralized processors
  • Supports multi-channel documents with independent event streams

For documents with multiple channels, all are tracked independently:

checkpoint:
  type: Channel Event Checkpoint
  lastEvents:
    userChannel:
      blueId: 7UEBwTmRMfQ92rGt4vHkzPa8Ypd5KJsLNcA3FV6xDqbn
    paymentChannel:
      blueId: CgJ83PcZwuAHboEL6AjrbJsxTWRTPzY8WR8Y2zAMXRbyd
    weatherChannel:
      blueId: JXiEdbLToPRWNQEekNxrxfQWBG8wmfjEqugPEEXByMYW

Timestamp Checkpoint​

For some scenarios, a timestamp-based checkpoint is used:

checkpoint:
  type: Timestamp Checkpoint
  timestamp: '2023-10-15T14:30:00Z' # Last processed event timestamp

Important considerations for timestamp checkpoints:

  • The timestamp must be deterministic (typically derived from event timestamps)
  • All processors must interpret time identically
  • Precision must be sufficient to distinguish between events

Timestamp checkpoints work best in controlled environments where:

  • Events have reliable timestamps
  • Event ordering by time is consistent
  • Time interpretation is standardized across processors

The River of Time: Document Identity Across States​

In traditional databases, identity is imposed externally: a record with ID 12345 remains "the same record" regardless of how its contents change. But in a content-addressed system like Blue, the fundamental question becomes more profound: what makes a document "the same document" when its content—and thus its BlueId—changes over time?

The answer lies in the Processing Checkpoint contract, which creates a formal history of causality. This history binds different document states together, creating a continuous identity through time:

# The document in its past (BlueId: a1b2c3...)
name: Simple Counter
counter: 0
contracts:
  # Channel and workflow definitions
  checkpoint:
    type: Channel Event Checkpoint
    lastEvents:
      incrementChannel: { blueId: null } # No events processed yet
# The document in its present (BlueId: d4e5f6...)
name: Simple Counter
counter: 2
contracts:
  # Channel and workflow definitions
  checkpoint:
    type: Channel Event Checkpoint
    lastEvents:
      incrementChannel: { blueId: 8Y2zAMXRbydQj83PcZwuAHboEL6AjrbJsxTWRTPzY }

These two documents have different BlueIds because their content has changed, yet they represent the same logical entity at different points in its lifecycle. The checkpoint creates a causal connection between them, showing how one evolved into the other.

This is more than just a technical feature—it's a formal model of identity through change. Like the philosophical Ship of Theseus which remains "the same ship" even as all its planks are replaced over time, a Blue document remains "the same document" through its processing history despite having completely different content and BlueId.

The Three Temporal Dimensions​

The Processing Checkpoint gives every document three temporal dimensions:

1. Verifiable Past​

The checkpoint's record of processed events creates a verifiable history. Any processor can:

  • Validate that the document legitimately evolved from a known earlier state
  • Reproduce the document's current state by replaying events
  • Verify the integrity of document processing

2. Known Present​

The checkpoint itself represents the document's "now"—its current position in the event timeline. This gives processors:

  • A clear understanding of what state the document represents
  • Confidence in which events have been incorporated
  • A foundation for consistent behavior across systems

3. Processable Future​

By knowing exactly which events have been processed, the checkpoint enables:

  • Precise continuation of processing from the current state
  • Clear criteria for what constitutes a "new" event
  • Orderly evolution to future states

Automatic Checkpoint Management​

Blue processors update checkpoints automatically after each event is processed:

// Process an event
ProcessingResult result = processor.processEvent(event);
Node updatedDoc = result.getUpdatedDocument();

// The checkpoint is automatically updated
Node checkpoint = updatedDoc.getAsNode("/contracts/checkpoint/lastEvents/incrementChannel");
String lastEventId = checkpoint.getBlueId();
assert lastEventId.equals(event.getBlueId());

Next Steps​

Now that you understand how documents maintain their identity through time, let's explore how Blue handles multiple event channels in the Events from Multiple Sources section.