RFC: NBSON Storage Format and PRPH Operations

Status: Draft
Version: 0.1
Date: November 2025
Authors: BeTTY Protocol Working Group

Abstract

This document specifies:

Together, these technologies provide the storage foundation for the BeTTY protocol, enabling encrypted-at-rest storage with selective access patterns.

1. Introduction

In Phil Alden Robinson's Sneakers (1992), mathematician Gunter Janek (played by Donal Logue) says of his work in cryptography:

There exists an intriguing possibility for a far more elegant solution... we might induce homomorphisms from the principal ordinance of each of these fields...

Using this as inspiration, NBSON uses an elegant solution to induce pseudo-homomorphisms by using the ordinance of each field in a newline-delimited, encrypted at rest binary document format.

1.1 Motivation

Modern distributed systems require storage solutions that balance several competing concerns:

Traditional approaches force a choice: either encrypt everything and lose functionality, or leave data unencrypted to enable operations. True homomorphic encryption is computationally expensive; NBSON and PRPH provide a more efficient shortcut.

1.2 Goals

This specification aims to provide:

  1. Encrypted Storage - All content encrypted at rest by default
  2. Structured Access - Line-delimited format enables random access
  3. Blind Operations - Write without reading encrypted content
  4. Minimal Overhead - Operations complete in single-digit milliseconds
  5. Standard Compatibility - Works with existing encryption tools

1.3 Document Structure

This RFC is organized into two major parts:

Part I: NBSON Storage Format (Sections 2-6)

Part II: PRPH Operations (Sections 7-11)

1.4 Terminology

NBSON
A line-delimited binary storage format for JSON-LD documents
PRPH
Poor Richard's Pseudo-Homomorphism - Operations on encrypted data without full decryption
Almanack
First line of NBSON file mapping keys to byte offsets
Blind Append
Adding content to encrypted container without reading existing content
Blind Edit
Modifying specific field without decrypting entire document
KEBAC
Keyed Encryption-Based Access Control - BeTTY's permission system

PART I: NBSON STORAGE FORMAT

2. File Format Specification

2.1 Overall Structure

NBSON files consist of a line-delimited structure:

Line 0: ALMANACK
Line 1+: CONTENT_LINES
EOF

Line Delimiters:

2.2 Almanack Format

The first line (Line 0) contains a JSON object mapping keys to line numbers:

{
  "routing": 1,
  "meta": 2,
  "content": 3,
  "queue": [4, 5, 6]
}

Properties:

Dot Notation Support:

The almanack MAY use dot notation for nested keys:

{
  "betty.topic": 1,
  "betty.rev": 2,
  "meta.tags": 3,
  "content.title": 4,
  "content.body": 5
}

This enables more granular PRPH operations on nested fields.

Additionally, if the line becomes unmanageably long with key defintions, a subset MAY be broken out into its own almanack-type line referenced in the first line and re-attached to a keymap by document post-processing:

{
  "betty.topic": 1,
  "betty.rev": 2,
  "meta": 3,
  "content": 7
}
14t4awgeQ3af4SXGrw35346W
14t4awgeQ3af4SXGrw35346W
{"almanack":{"headline":4,"tags":5,"link":6}}
"GNU COPYLEFT NOTICE"
["open-source", "gnu", "licenses"]
"https://www.gnu.org/licenses/copyleft.en.html"
"Copyleft is a general method for making a program (or other work) free..."

2.3 Content Lines

Each line after the almanack contains a value, which may be:

Example File Structure:

Line 0: {"routing": 1, "meta": 2, "payload": 3}
Line 1: [encrypted_routing_data]
Line 2: [encrypted_meta_data]
Line 3: [encrypted_payload_data]
EOF

2.4 Encryption Layers

NBSON supports three encryption strategies:

Full Encryption - Entire file encrypted (index + content):

Line 0: [encrypted_index]
Line 1+: [encrypted_content]

Selective Encryption - Index plaintext, content encrypted:

Line 0: {"routing": 1, "meta": 2, "payload": 3}  [PLAINTEXT]
Line 1+: [encrypted_content]

Layered Encryption - Different keys for index vs content:

Line 0: [index_encrypted_to_group_key]
Line 1+: [content_encrypted_to_owner_key]

The encryption strategy is determined by:

  1. Document permissions (betty.permissions)
  2. Server configuration (nbson.index_encryption)
  3. User preferences during document creation

3. Compression

3.1 Compression Algorithm

NBSON uses gzip compression by default (configurable levels 0-9):

Original JSON → BSON binary → gzip compression → encryption

Compression Levels:

3.2 Compression Benefits

Typical Savings:

The performance impact should be negligible compared to encryption overhead.

3.3 Configuration

Compression is configured per-directory via .nbaccess files or globally:

{
  "compression": 6,
  "compress_index": false,
  "compression_algorithm": "gzip"
}

4. Storage Operations

4.1 Write Operation

Algorithm:

1. Validate document structure
2. Compute topic hash (with salt if present)
3. Compute revision hash
4. Generate index line
5. Compress each value
6. Encrypt according to permissions
7. Write index line
8. Write content lines
9. Sync to disk

Pseudocode:

def write_nbson(document, file_path, owner_key):
    # Compute hashes
    topic = hash_content(document.content, document.meta.salt)
    rev = hash_document(document, document.meta.salt)
    
    # Build almanack
    almanack = {}
    line_num = 1
    for key in document.keys():
        if key != "betty" and key != "meta":
            almanack[key] = line_num
            line_num += 1
    
    # Write index (line 0)
    write_line(file_path, 0, json.dumps(almanack))
    
    # Write content lines
    for key, line_num in almanack.items():
        value = document[key]
        compressed = gzip.compress(bson.encode(value))
        encrypted = encrypt(compressed, owner_key)
        write_line(file_path, line_num, encrypted)

4.2 Read Operation

Algorithm:

1. Read almanack (decrypt if necessary)
2. Parse key-to-line mapping
3. For each requested key:
   a. Seek to line offset
   b. Read encrypted line
   c. Decrypt with private key
   d. Decompress
   e. Parse BSON to JSON
4. Reconstruct document

Pseudocode:

def read_nbson(file_path, private_key, keys=None):
    # Read and decrypt almanack
    index_line = read_line(file_path, 0)
    almanack = json.loads(decrypt(index_line, private_key))
    
    # Read requested keys (or all if keys=None)
    document = {}
    target_keys = keys or almanack.keys()
    
    for key in target_keys:
        line_num = almanack[key]
        encrypted_line = read_line(file_path, line_num)
        compressed = decrypt(encrypted_line, private_key)
        bson_data = gzip.decompress(compressed)
        document[key] = bson.decode(bson_data)
    
    return document

4.3 Update Operation

Full Update (Upsert):

1. Read existing document
2. Modify content
3. Compute new revision hash
4. Archive old version
5. Write new version
6. Update topic pointer

Partial Update (Blind Edit via PRPH):

1. Read almanack only (e.g., with head -n1)
2. Locate target key offset
3. Encrypt new value
4. Overwrite at offset
5. Update index hash

5. Directory Configuration

5.1 .nbxs Configuration Files

Each directory can contain a .nbxs file defining:

{
  "compression": 6,
  "encryption": {
    "default_recipients": ["node@localhost"],
    "require_encryption": true,
    "almanack_encryption": "selective"
  },
  "context": "betty://schema/article/v1",
  "indexing": {
    "auto_index": true,
    "fields": ["author", "created", "tags"]
  },
  "prph_write": 2
}

Key Settings:

5.2 Directory Hierarchy

.nbxs files are inherited hierarchically:

~/.betty/storage/
├── .nbxs                    # Global settings
├── articles/
│   ├── .nbxs                # Article-specific settings
│   ├── article1.nbson
│   └── article2.nbson
└── private/
    ├── .nbxs                # Private content settings
    └── secret.nbson

Child directories inherit and override parent settings.

PART II: PRPH OPERATIONS

7. Core Concepts

7.1 The Key Insight

PRPH exploits three properties of NBSON files:

  1. Line-delimited structure - File boundaries known without decryption
  2. Index accessibility - First line maps keys to byte offsets
  3. Append-only semantics - New content extends existing data

Analogy: A locked filing cabinet where you can:

7.2 Security Model

PRPH maintains confidentiality through:

Threat Model:

Not Protected Against:

7.3 Permission Requirements

PRPH operations respect KEBAC permissions:

Operation Read (4) Write (2) Index Access Use Case
Blind Append No Yes Optional Anonymous submissions
Blind Edit No Yes Required Status updates
Full Read Yes No Required Normal reading
Upsert Yes Yes Required Normal updates

8. Blind Append Operation

8.1 Purpose

Add content to encrypted container without reading existing content.

8.2 Algorithm

1. Read almanack line (decrypt if necessary)
2. Identify target structure (queue, directory, etc.)
3. Encrypt new content to owner's public key
4. Append encrypted content at EOF
5. Update almanack line with new offset
6. Write updated almanack

8.3 Pseudocode

def blind_append(file_path, new_content, target_key, owner_pubkey):
    # Read and decrypt almanack
    almanack = read_index_line(file_path)
    
    # Find target key offset
    current_offsets = almanack.get(target_key, [])
    
    # Encrypt new content
    compressed = gzip.compress(bson.encode(new_content))
    encrypted_content = encrypt(compressed, owner_pubkey)
    
    # Append to file
    new_offset = append_to_file(file_path, encrypted_content)
    
    # Update index
    if isinstance(current_offsets, list):
        almanack[target_key].append(new_offset)
    else:
        almanack[target_key] = [current_offsets, new_offset]
    
    # Write updated almanack
    write_index_line(file_path, almanack)

8.4 Complexity

Time: O(1) - constant time regardless of file size
I/O: 2 reads + 2 writes (index read/write, content append)
Encryption: 1 asymmetric encryption operation

8.5 Example Use Case

Anonymous Tip Queue:

// Initial state (encrypted to alice@example)
{
  "queue": [1, 2]  // Two existing messages
}

// Bob appends without reading existing tips
blind_append(
    file="tips.nbson",
    new_content={"msg": "Anonymous tip", "timestamp": "..."},
    target_key="queue",
    owner_pubkey=alice_pubkey
)

// Updated almanack
{
  "queue": [1, 2, 3]  // Three messages now
}

Security Properties:

9. Blind Edit Operation

9.1 Purpose

Modify specific field without decrypting entire document.

9.2 Algorithm

1. Read and decrypt almanack line
2. Locate target key offset
3. Encrypt new value to owner's public key
4. Overwrite at specified offset
5. Update content hash in almanack

9.3 Pseudocode

def blind_edit(file_path, key_path, new_value, owner_pubkey, almanack_key):
    # Read and decrypt almanack (requires almanack key)
    almanack = decrypt(read_index_line(file_path), almanack_key)
    
    # Navigate to target key
    offset = resolve_key_path(almanack, key_path)
    
    # Encrypt new value
    compressed = gzip.compress(bson.encode(new_value))
    encrypted_value = encrypt(compressed, owner_pubkey)
    
    # Overwrite at offset
    write_at_offset(file_path, offset, encrypted_value)
    
    # Update almanack hash
    almanack[key_path + "_hash"] = hash(encrypted_value)
    write_index_line(file_path, encrypt(almanack, almanack_key))

9.4 Complexity

Time: O(log n) for key path resolution, O(1) for write
I/O: 2 reads + 2 writes (index read/write, field overwrite)
Encryption: 1 asymmetric encryption operation

9.5 Example Use Case

Status Update:

// Document with readable index, encrypted content
{
  "status": {"offset": 1, "hash": "abc123"},
  "content": {"offset": 2, "hash": "def456"}
}

// Moderator updates status without reading content
blind_edit(
    file="post.nbson",
    key_path="status",
    new_value="published",
    owner_pubkey=admin_pubkey,
    almanack_key=moderator_group_key
)

// Almanack updated with new hash
{
  "status": {"offset": 1, "hash": "xyz789"},  // Updated
  "content": {"offset": 2, "hash": "def456"}  // Unchanged
}

Security Properties:

10. Configuration

10.1 PRPH Mode Settings

Configuration Key: prph_write (integer 0-5)

Mode Behavior Use Case
0 Disabled Maximum security, all operations require full decryption
1 Error-only Log PRPH attempts but fail operations
2 Blind-append Allow append operations without read (recommended)
3 Forkable topic Create new topic on write failure with read access
4 PRPH error-only Log errors but don't block operations
5 PRPH writable Full support, writable with almanack key only

Recommended Settings:

10.2 Index Encryption Policy

Configuration Key: almanack_encryption (string)

Policy Almanack State Almanack Key Use Case
none Plaintext None Maximum PRPH capability
symmetric Encrypted Shared secret Group collaboration
asymmetric Encrypted Owner's key Private documents

11. Security Considerations

11.1 Information Leakage

File Size Analysis:

Timing Analysis:

Access Patterns:

11.2 Attack Vectors

Malicious Appends:

Almanack Corruption:

Replay Attacks:

11.3 Key Management

Almanack Keys:

Content Keys:

Key Rotation:

11.4 Cryptographic Considerations

Algorithm Selection:

Key Strength:

Random Number Generation:

12. Implementation Guidelines

12.1 Required Components

Storage Engine:

Encryption Layer:

Permission System:

12.2 Testing Requirements

Functional Tests:

Security Tests:

Performance Tests:

12.3 Error Handling

Common Errors:

{
  "error": "PRPHDisabled",
  "message": "PRPH operations disabled",
  "current_mode": 0,
  "required_mode": 2
}
{
  "error": "AlmanackKeyRequired",
  "message": "Almanack key required for blind edit",
  "operation": "blind_edit",
  "key_path": "status"
}
{
  "error": "InsufficientPermissions",
  "message": "Write permission required",
  "current": "r--",
  "required": "-w-"
}
{
  "error": "AlmanackCorrupted",
  "message": "Index hash mismatch",
  "expected": "abc123",
  "actual": "def456",
  "recovery": "Restore from backup or rebuild index"
}

13. Use Cases

13.1 Anonymous Tip Submission

Scenario: Public can submit tips to investigative journalists

Setup:

{
  "betty": {
    "owner": "journalist@news.org",
    "permissions": {
      "@world": 3  // Write + index, no read
    }
  },
  "queue": []
}

Flow:

  1. Anonymous user submits tip via blind append
  2. Tip encrypted to journalist's key
  3. User cannot read other tips
  4. Journalist decrypts all tips with private key

Benefits:

13.2 Collaborative Moderation

Scenario: Moderators can update post status without reading content

Setup:

{
  "betty": {
    "owner": "admin@forum.org",
    "permissions": {
      "@moderators": 2  // Write only
    }
  },
  "content": {
    "status": "pending",
    "body": "[sensitive content]"
  }
}

Flow:

  1. Moderator receives almanack key (shared group key)
  2. Updates status field via blind edit
  3. Cannot read post body (lacks owner's private key)
  4. Admin reviews with full access

Benefits:

13.3 Encrypted Audit Logging

Scenario: Applications log events they cannot read

Setup:

{
  "betty": {
    "owner": "security@company.org",
    "permissions": {
      "@applications": 2  // Write only
    }
  },
  "queue": []
}

Flow:

  1. Application generates audit event
  2. Blind appends to encrypted log
  3. Application cannot read previous entries
  4. Security team reviews with private key

Benefits:

13.4 Dead Drop Communication

Scenario: Whistleblower drops information without knowing recipient

Setup:

{
  "betty": {
    "owner": "recipient@secure.org",
    "permissions": {
      "@world": 2  // Write only, not indexed
    }
  },
  "queue": []
}

Flow:

  1. Whistleblower appends encrypted document
  2. Document not searchable (no index permission)
  3. Recipient polls dead drop location
  4. Only recipient can decrypt contents

Benefits:

14. Comparison with Alternatives

14.1 True Homomorphic Encryption

Feature PRPH Homomorphic Encryption
Computation on encrypted data No Yes
Complexity Low Very High
Use cases Append, edit Mathematical operations
Maturity Stable Research/early adoption
Resource requirements Minimal Significant

When to use PRPH: Fast operations, simple access patterns, mature deployment
When to use HE: Mathematical computation required, performance acceptable

14.2 Secure Multi-Party Computation

Feature PRPH SMPC
Multiple parties Optional Required
Network overhead Low High
Trust model Key-based Quorum-based
Setup complexity Simple Complex

When to use PRPH: Single owner, multiple writers, low latency
When to use SMPC: Distributed trust required, no single owner

14.3 Traditional Access Control Lists

Feature PRPH/NBSON Traditional ACLs
Encryption at rest Always Optional
Write without read Yes Requires special implementation
Permission enforcement Cryptographic Server-side
Offline capability Yes No
Scalability High Varies

When to use PRPH/NBSON: Strong encryption required, offline operation, federated system
When to use ACLs: Performance critical, trusted server, centralized system

15. Future Directions

15.1 Potential Enhancements

Selective Decryption:

Quantum-Resistant Algorithms:

Zero-Knowledge Proofs:

Streaming Operations:

15.2 Open Questions

Concurrent Modifications:

Revocation:

Cross-System Interoperability:

Performance Optimization:

16. References

16.1 Normative References

[RFC2119] Bradner, S., "Key words for use in RFCs to Indicate Requirement Levels", BCP 14, RFC 2119, March 1997.

[BSON] "BSON - Binary JSON", http://bsonspec.org/

[GZIP] Deutsch, P., "GZIP file format specification version 4.3", RFC 1952, May 1996.

16.2 Cryptographic References

[OpenPGP] Callas, J., et al., "OpenPGP Message Format", RFC 4880, November 2007.

[Ed25519] Josefsson, S. and I. Liusvaara, "Edwards-Curve Digital Signature Algorithm (EdDSA)", RFC 8032, January 2017.

16.4 BeTTY Protocol Documents

Appendix A: Complete Example

A.1 Scenario: Anonymous Feedback System

Initial Document:

{
  "betty": {
    "topic": "feedback-abc123",
    "owner": "manager@company.org",
    "permissions": {
      "@employees": 3  // Write + index, no read
    }
  },
  "meta": {
    "tags": ["feedback", "anonymous"],
    "salt": "company-feedback-2025"
  },
  "queue": []
}

NBSON File Structure (Initial):

Line 0: {"queue": []}                            [PLAINTEXT ALMANACK]
EOF

Employee Submits Feedback (blind append):

feedback = {
    "timestamp": "2025-11-14T10:00:00Z",
    "department": "Engineering",
    "message": "Suggestion for improvement..."
}

# Blind append without reading existing feedback
blind_append(
    file_path="feedback-abc123.nbson",
    new_content=feedback,
    target_key="queue",
    owner_pubkey=manager_pubkey
)

Resulting File Structure:

Line 0: {"queue": [1]}                           [PLAINTEXT ALMANACK]
Line 1: [encrypted_feedback_1]                   [ENCRYPTED to manager]
EOF

Second Employee Submits:

feedback2 = {
    "timestamp": "2025-11-14T11:00:00Z",
    "department": "Marketing",
    "message": "Another suggestion..."
}

blind_append(
    file_path="feedback-abc123.nbson",
    new_content=feedback2,
    target_key="queue",
    owner_pubkey=manager_pubkey
)

Updated File:

Line 0: {"queue": [1, 2]}                        [PLAINTEXT ALMANACK]
Line 1: [encrypted_feedback_1]                   [ENCRYPTED to manager]
Line 2: [encrypted_feedback_2]                   [ENCRYPTED to manager]
EOF

Manager Reads All Feedback:

# Full read access with private key
document = read_nbson(
    file_path="feedback-abc123.nbson",
    private_key=manager_privkey
)

for item in document["queue"]:
    print(f"{item['department']}: {item['message']}")

Output:

Engineering: Suggestion for improvement...
Marketing: Another suggestion...

A.2 Security Analysis

User Capabilities:

Manager Capabilities:

Attacker with File Access:

Appendix B: Configuration Examples

B.1 High Security Configuration

{
  "compression": 9,
  "encryption": {
    "default_recipients": ["admin@secure.org"],
    "require_encryption": true,
    "index_encryption": "asymmetric",
    "require_signatures": true
  },
  "prph_write": 0,
  "hash_verification": true,
  "audit_logging": true,
  "file_permissions": "600"
}

Use Case: Classified documents, medical records, financial data

Characteristics:

B.2 Collaborative Workspace Configuration

{
  "compression": 6,
  "encryption": {
    "default_recipients": ["team@company.org"],
    "require_encryption": true,
    "index_encryption": "symmetric",
    "group_key": "team-2025-key"
  },
  "prph_write": 5,
  "hash_verification": true,
  "indexing": {
    "auto_index": true,
    "fields": ["author", "created", "modified", "status"]
  }
}

Use Case: Team document collaboration, shared knowledge base

Characteristics:

B.3 Public Service Configuration

{
  "compression": 4,
  "encryption": {
    "default_recipients": ["service@public.org"],
    "require_encryption": true,
    "index_encryption": "none"
  },
  "prph_write": 3,
  "hash_verification": false,
  "indexing": {
    "auto_index": true,
    "public_fields": ["title", "created", "tags"]
  },
  "rate_limiting": {
    "blind_append": "100/hour",
    "blind_edit": "10/hour"
  }
}

Use Case: Anonymous submission forms, public feedback, tip lines

Characteristics:

B.4 Archive Storage Configuration

{
  "compression": 9,
  "encryption": {
    "default_recipients": ["archive@org.org"],
    "require_encryption": true,
    "index_encryption": "asymmetric"
  },
  "prph_write": 0,
  "hash_verification": true,
  "immutable": true,
  "checksum_algorithm": "sha256",
  "retention": {
    "verify_interval": "monthly",
    "backup_copies": 3
  }
}

Use Case: Long-term archives, compliance storage, historical records

Characteristics:

Appendix C: Glossary

Blind Append
Adding content to encrypted container without reading existing content
Blind Edit
Modifying specific field without decrypting entire document
BSON
Binary JSON - compact binary representation of JSON documents
Content-Addressable
Identifier derived from content hash, ensuring immutability
EOF
End of File marker
Index Line
First line of NBSON file mapping keys to byte offsets
KEBAC
Keyed Encryption-Based Access Control - cryptographic permission system
NBSON
Nota Betty, Son - line-delimited binary storage format for BeTTY
PRPH
Poor Richard's Pseudo-Homomorphism - operations on encrypted data without full decryption
Salt
Random string appended to content before hashing to ensure unique topic identifiers
Topic
Content-addressable identifier for a document stream
Revision
Specific version of a document identified by full document hash
.nbxs
Directory configuration file controlling NBSON behavior (similar to .htaccess)

Appendix D: Comparison Matrix

D.1 Storage Format Comparison

Feature NBSON JSON Files Database IPFS
Encryption at Rest Built-in Optional Optional No
Random Access Yes (via index) No Yes No
Compression Built-in External Optional Built-in
Blind Operations Yes (PRPH) No No No
Human Readable Partial Yes No No
Content Addressed Yes No No Yes
Version Control Built-in External Optional Implicit
File Size Overhead 5-10% 0% Varies 20-30%

D.2 Operation Performance Comparison

Operation NBSON+PRPH Encrypted JSON Database IPFS
Write 5-10ms 10-20ms 1-5ms 50-200ms
Read (full) 5-15ms 10-30ms 1-5ms 50-200ms
Read (partial) 3-8ms 10-30ms 1-3ms 50-200ms
Blind Append 3-5ms N/A N/A N/A
Blind Edit 4-8ms N/A N/A N/A
Search Via index Full scan Indexed DHT lookup

Authors' Addresses

The BeTTY Project
https://betty.land
Email: betty-protocol@betty.land

Copyright (C) The BeTTY Project (2025). All Rights Reserved.

This document and translations of it may be copied and furnished to others, and derivative works that comment on or otherwise explain it or assist in its implementation may be prepared, copied, published and distributed, in whole or in part, without restriction of any kind, provided that the above copyright notice and this paragraph are included on all such copies and derivative works.

This document itself may not be modified in any way, such as by removing the copyright notice or references to the BeTTY Project, except as needed for the purpose of developing BeTTY standards in which case the procedures for copyrights defined in the BeTTY Process must be followed, or as required to translate it into languages other than English.

The limited permissions granted above are perpetual and will not be revoked by the BeTTY Project or its successors or assigns.

This document and the information contained herein is provided on an "AS IS" basis and THE BETTY PROJECT DISCLAIMS ALL WARRANTIES, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE INFORMATION HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE.

Acknowledgments

NBSON and PRPH build upon ideas from:

Special thanks to:

End of RFC

Comments and feedback welcome at: betty-protocol@betty.land