Skip to main content

Circular References

This page explores how Blue handles circular references between documents while maintaining the integrity of content-addressing.

The Object Reference Problem​

In programming, circular references are common and useful:

Person alice = new Person("Alice");
Dog pirate = new Dog("Pirate");

// Create circular reference
alice.setPet(pirate);
pirate.setOwner(alice);

In memory, this works fine - objects can point to each other. But when we need to save these objects, we face a challenge: how do we represent this circular relationship in a serialized form?

The Content-Addressing Challenge​

This challenge becomes particularly complex in Blue's content-addressed system:

  • Document A's BlueId depends on its content, including any reference to Document B
  • Document B's BlueId depends on its content, including any reference to Document A

This creates a paradox: how can we calculate stable BlueIds when each document's identity depends on the other? Traditional content-addressing systems often can't handle circular references, requiring workarounds that compromise the system's integrity.

The Two Circular Reference Problems​

Blue solves two related problems:

  1. Storage Problem: How to serialize and store objects that already have circular references
  2. Evolution Problem: How to work with objects that develop circular references over time

Let's examine both solutions.

Direct Circular References: Combined BlueId Calculation​

For storing objects with existing circular references, Blue provides a mechanism to calculate consistent BlueIds for interconnected documents.

When saving circular references (like our Person and Dog example), documents initially use special this#n references:

- name: Person
  pet:
    type:
      blueId: 'this#1'  # References the Dog document at position 1

- name: Dog
  owner:
    type:
      blueId: 'this#0'  # References the Person document at position 0
  breed:
    type: Text

This special syntax allows to calculate a single master BlueId for the entire set of interconnected documents. The number after this# indicates the document's position in the sorted document set.

Once the master BlueId is calculated, each document receives a final identifier derived from this master BlueId and its position. If the master BlueId is "12345", the individual documents would be identified as "12345#0" and "12345#1".

These final BlueIds can be used in other documents:

name: Alice
type:
  blueId: '12345#0'  # References the Person document
pet:
  name: Pirate
  breed: German Shepherd
anotherPet:
  type: 
    blueId: '12345#1'  # References the Dog document

How Combined BlueId Calculation Works​

When saving objects with circular references (like our dog.setOwner(owner) and owner.setPet(dog) example), Blue follows this process:

  1. Identify the Document Set: Identify all documents that form a mutually referential set

    Example: In our case, we identify the Person and Dog documents as mutually referential

  2. Break the Cycle: Replace all circular references with a placeholder (44 zeros)

    Example: Person's reference to Dog becomes pet.type.blueId: 00000000000000000000000000000000000000000000

    Example: Dog's reference to Person becomes owner.type.blueId: 00000000000000000000000000000000000000000000

  3. Individual Calculation: Calculate preliminary BlueIds for each document in isolation

    Example: Person might get a temporary BlueId of 7UEBwTmRMfQ92rGt4vHkzPa8Ypd5KJsLNcA3FV6xDqbn

    Example: Dog might get a temporary BlueId of 2qCKS6yS11cYGabM3nPXsbd6DYK4m6WNrEqGGyySDCaD

  4. Establish Order: Sort documents alphabetically by their preliminary BlueIds

    Example: Since 2qCKS6... comes before 7UEBwT... alphabetically, Dog would be position #0 and Person would be position #1

  5. Reference Resolution: Replace placeholder values with the appropriate this#n references

    Example: In Dog, replace placeholder with this#1 (pointing to Person)

    Example: In Person, replace placeholder with this#0 (pointing to Dog)

  6. Final Assignment: Create a list containing all documents in their sorted order and calculate the BlueId of this entire list

    Example: Create a list [Dog, Person] and calculate its BlueId as "12345", making the final BlueIds Dog="12345#0" and Person="12345#1"

This process allows Blue to maintain the content-addressing principle even with circular references, by treating interconnected documents as a single computational unit with a shared master BlueId.

Temporal Evolution into Circular References​

Consider this common programming pattern that creates circular references:

Person alice = new Person("Alice");             // Alice with no pet
Dog pirate = new Dog("Pirate", alice);          // Pirate with Alice as owner
alice.setPet(pirate);                           // Alice with Pirate as pet - now circular

This pattern creates a challenge in Blue: how do document processors handle documents that develop circular references through events over time?

How Document Processors Handle Evolving Circular References​

Let's walk through how this Java example translates to Blue documents and processing:

  1. First, we create Alice's document:
# Alice (initial state - no pet)
name: Alice
type: Person
contracts:
  aliceTimelineChannel: # A channel for events about Alice
  petUpdateWorkflow: # A workflow that handles adding pets, connected to this channel
  processSubnodes: # Process any documents in the /pet path

  1. We start a document processor for Alice's document. At this point, the processor:

    • Reads Alice's contracts
    • Expects events on Alice's timeline channel
    • Notes it should process any document at /pet (but there isn't one yet)

  2. Next, we create Pirate's document:

# Pirate (references Alice's initial state)
name: Pirate
type: Dog
owner:
  blueId: HVkRPb4L3EQey6rdwypi7na7ATz5P3PLH8Q3PSy2WQ94  # BlueId of Alice (initial state)
contracts:
  bobTimelineChannel: # Sample channel for demo purposes
  processSubnodes: # Process any documents in the /owner path

  1. We then add this document to Alice's timeline.

  2. The document processor:

    • Receives the new event on Alice's timeline
    • Processes it through the petUpdateWorkflow
    • Updates Alice's document, adding Pirate as her pet
    • Updates the root document's checkpoint, marking that it has processed this event

  3. After this update, the processor has the following state:

    • It tracks the root document (Alice) with a checkpoint after the pet-adding event
    • It identifies the document at /pet path (Pirate) for processing
    • It adds Pirate's timeline to its tracked channels (with no checkpoint) for the /pet node
    • It discovers Pirate has a processSubnodes contract for /owner
    • It adds Alice's timeline to track for the /pet/owner path (with no checkpoint)

  4. The processor now tracks three document-channel combinations:

    • Channel A: Alice's timeline with checkpoint after the pet-adding event (for root Alice)
    • Channel B: Pirate's timeline with no checkpoint (for /pet node)
    • Channel C: Alice's timeline with no checkpoint (for /pet/owner node)

  5. The next event received by the processor comes from Channel C (Alice's timeline with no checkpoint):

    • This is the same pet-adding event already processed for root Alice, but not yet for the /pet/owner node
    • The processor applies this event to the Alice document at /pet/owner
    • This updates the earlier Alice to also have Pirate as her pet
    • The checkpoint for Channel C advances to match the checkpoint of Channel A

  6. The document processor realizes the /pet/owner node now has the same checkpoint as the ROOT node and instead of repeating step 6 for /pet/owner, it internally marks them as identical. This creates the circular reference pattern we need.

  7. When this document is later converted to objects, we achieve our desired circular reference structure - Alice has Pirate as her pet, and Pirate has Alice as its owner.

This approach resolves the apparent paradox through normal event processing. No special circular reference handling is needed - the standard processing of events in checkpoint order naturally synchronizes different instances of the same document, allowing circular references to form correctly.

The Risk of Infinite Loops​

With circular document references, you must be careful to avoid creating infinite event loops:

- name: Person
  pet:
    type:
      blueId: 'this#1'  # References the Dog document at position 1
  contracts:
    docChannel: # listen to /pet/a changes
    workflow:   # put b into root document with same value as /pet/a

- name: Dog
  owner:
    type:
      blueId: 'this#0'  # References the Person document at position 0
  breed:
    type: Text
  contracts:
    docChannel: # listen to /owner/b changes
    workflow:   # put a into root document with same value as /owner/b

This is like a scenario where calling a method on Person triggers a method on Dog, which then triggers the same method on Person again. If a circular event chain like this occurs:

  1. Person's workflow triggers when Dog's a field changes
  2. Person updates its b field in response
  3. Dog's workflow triggers when Person's b field changes
  4. Dog updates its a field in response
  5. Person's workflow triggers again, creating an infinite loop

When this happens, the document processor will terminate processing and place the document in a "Terminated due to Fatal Error" state. This state is irrecoverable - the document cannot be further processed or updated.

Just as with traditional programming, it's essential to design Blue documents carefully to avoid circular event chains that could trigger infinite loops.