🔍 Code Extractor

A repair tool that fixes corrupted size entries in reMarkable cloud's root.docSchema file by recalculating correct document sizes from their component schemas.

Purpose

This class provides functionality to diagnose and repair corrupted entries in the reMarkable cloud storage's root.docSchema file. It identifies documents with suspicious size values (2247, 2246, 2715 bytes), recalculates their correct sizes by summing component sizes from individual document schemas, and uploads the corrected root.docSchema back to the reMarkable cloud. The tool preserves the original version number and maintains all folder and special entries unchanged while fixing only corrupted document entries.

Source Code

class CorrectedRootDocSchemaRepair:
    """Corrected version of the repair tool"""
    
    def __init__(self):
        # Load auth session
        from auth import RemarkableAuth
        auth = RemarkableAuth()
        self.session = auth.get_authenticated_session()
        
        if not self.session:
            raise RuntimeError("Failed to authenticate with reMarkable")
        
        self.base_dir = Path(__file__).parent
        self.backup_dir = self.base_dir / "docschema_repair"
        self.backup_dir.mkdir(exist_ok=True)
        
        print("🔧 CORRECTED reMarkable Root DocSchema Repair Tool")
    
    def get_current_root_content(self) -> Tuple[str, str, Dict]:
        """Get current root.docSchema content and metadata"""
        # Get root info
        root_response = self.session.get("https://eu.tectonic.remarkable.com/sync/v4/root")
        root_response.raise_for_status()
        root_data = root_response.json()
        
        # Get content
        root_content_response = self.session.get(f"https://eu.tectonic.remarkable.com/sync/v3/files/{root_data['hash']}")
        root_content_response.raise_for_status()
        root_content = root_content_response.text
        
        return root_content, root_data['hash'], root_data
    
    def identify_corrupted_entries(self, root_content: str) -> Dict[str, List]:
        """Identify which entries need size correction"""
        lines = root_content.strip().split('\n')
        original_version = lines[0]  # KEEP ORIGINAL VERSION!
        entries = lines[1:]
        
        print(f"📊 Original version: {original_version} (will preserve this)")
        print(f"📊 Total entries: {len(entries)}")
        
        # Suspicious sizes that indicate corruption
        suspicious_sizes = {'2247', '2246', '2715'}
        
        categorized = {
            'original_version': original_version,
            'good_entries': [],      # Entries with reasonable sizes
            'corrupted_entries': [], # Entries with suspicious small sizes
            'folders': [],           # All folder types (type 1 and 2)
            'special': []            # Other special entries
        }
        
        for i, line in enumerate(entries):
            if ':' in line:
                parts = line.split(':')
                if len(parts) >= 5:
                    entry = {
                        'line_number': i + 1,
                        'hash': parts[0],
                        'uuid': parts[2], 
                        'type': parts[3],
                        'size': parts[4],
                        'full_line': line
                    }
                    
                    # Categorize by type and size
                    if entry['type'] in ['1', '2']:  # Folders (both types)
                        categorized['folders'].append(entry)
                        print(f"📁 Folder (type {entry['type']}): {entry['uuid'][:8]}... size {entry['size']}")
                    
                    elif entry['type'] in ['4', '5']:  # Documents (PDF/Notebook)
                        if entry['size'] in suspicious_sizes:
                            categorized['corrupted_entries'].append(entry)
                            print(f"🔧 CORRUPTED document: {entry['uuid'][:8]}... size {entry['size']} (needs fixing)")
                        else:
                            categorized['good_entries'].append(entry)
                            print(f"✅ Good document: {entry['uuid'][:8]}... size {entry['size']}")
                    
                    else:
                        categorized['special'].append(entry)
                        print(f"❓ Special entry type {entry['type']}: {entry['uuid'][:8]}... size {entry['size']}")
        
        print(f"\n📊 Classification Summary:")
        print(f"   📁 Folders: {len(categorized['folders'])}")
        print(f"   ✅ Good documents: {len(categorized['good_entries'])}")
        print(f"   🔧 Corrupted documents: {len(categorized['corrupted_entries'])}")
        print(f"   ❓ Special entries: {len(categorized['special'])}")
        
        return categorized
    
    def fix_corrupted_document(self, entry: Dict) -> str:
        """Fix a single corrupted document by recalculating its correct size"""
        doc_hash = entry['hash']
        doc_uuid = entry['uuid']
        
        print(f"\n🔧 Fixing document {doc_uuid[:8]}...")
        print(f"   Current size: {entry['size']}")
        
        try:
            # Fetch document's docSchema
            doc_response = self.session.get(f"https://eu.tectonic.remarkable.com/sync/v3/files/{doc_hash}")
            doc_response.raise_for_status()
            doc_content = doc_response.text
            
            # Parse components and sum their sizes
            lines = doc_content.strip().split('\n')
            if len(lines) < 2:
                print(f"   ❌ Invalid docSchema - keeping original")
                return entry['full_line']
            
            total_component_size = 0
            component_count = 0
            
            for line in lines[1:]:  # Skip version header
                if ':' in line:
                    parts = line.split(':')
                    if len(parts) >= 5:
                        comp_size = int(parts[4])
                        total_component_size += comp_size
                        component_count += 1
            
            # The correct size is the sum of all component sizes
            correct_size = total_component_size
            
            print(f"   📦 Components: {component_count}")
            print(f"   📊 Correct size: {correct_size}")
            
            # Rebuild the line with correct size
            parts = entry['full_line'].split(':')
            parts[4] = str(correct_size)  # Replace size field
            fixed_line = ':'.join(parts)
            
            print(f"   ✅ Fixed: {entry['size']} → {correct_size}")
            return fixed_line
            
        except Exception as e:
            print(f"   ❌ Error fixing document: {e}")
            print(f"   ⚠️  Keeping original line")
            return entry['full_line']
    
    def rebuild_corrected_root(self, categorized: Dict) -> str:
        """Rebuild root.docSchema with all entries (fixed sizes where needed)"""
        print(f"\n🏗️  Rebuilding root.docSchema...")
        
        # Start with ORIGINAL version (not "43"!)
        lines = [categorized['original_version']]
        
        # Add folders (unchanged)
        for folder in categorized['folders']:
            lines.append(folder['full_line'])
        
        # Add good documents (unchanged)
        for doc in categorized['good_entries']:
            lines.append(doc['full_line'])
        
        # Add fixed corrupted documents
        for corrupted_doc in categorized['corrupted_entries']:
            fixed_line = self.fix_corrupted_document(corrupted_doc)
            lines.append(fixed_line)
        
        # Add special entries (unchanged)
        for special in categorized['special']:
            lines.append(special['full_line'])
        
        new_content = '\n'.join(lines)
        
        print(f"\n✅ Rebuilt root.docSchema:")
        print(f"   📊 Version: {categorized['original_version']} (preserved)")
        print(f"   📊 Total entries: {len(lines) - 1}")
        print(f"   📁 Folders: {len(categorized['folders'])}")
        print(f"   ✅ Good documents: {len(categorized['good_entries'])}")
        print(f"   🔧 Fixed documents: {len(categorized['corrupted_entries'])}")
        print(f"   ❓ Special entries: {len(categorized['special'])}")
        print(f"   📏 Content size: {len(new_content)} bytes")
        
        return new_content
    
    def preview_changes(self, dry_run: bool = True) -> bool:
        """Preview what changes will be made"""
        print(f"\n🔍 {'DRY RUN' if dry_run else 'LIVE RUN'} - Root DocSchema Repair")
        print("=" * 60)
        
        try:
            # Get current state
            root_content, current_hash, root_data = self.get_current_root_content()
            
            print(f"📋 Current root.docSchema:")
            print(f"   Hash: {current_hash}")
            print(f"   Size: {len(root_content)} bytes")
            
            # Identify corrupted entries
            categorized = self.identify_corrupted_entries(root_content)
            
            if len(categorized['corrupted_entries']) == 0:
                print(f"\n✅ No corrupted entries found - no changes needed!")
                return True
            
            # Build corrected version
            corrected_content = self.rebuild_corrected_root(categorized)
            
            # Save preview
            preview_file = self.backup_dir / f"corrected_preview_{int(time.time())}.txt"
            with open(preview_file, 'w') as f:
                f.write(corrected_content)
            print(f"📄 Preview saved to: {preview_file}")
            
            if dry_run:
                print(f"\n🔍 DRY RUN COMPLETE - No changes made to server")
                return True
            
            # Upload corrected version
            return self.upload_corrected_root(corrected_content)
            
        except Exception as e:
            print(f"❌ Repair failed: {e}")
            return False
    
    def upload_corrected_root(self, corrected_content: str) -> bool:
        """Upload the corrected root.docSchema"""
        print(f"\n⬆️  Uploading corrected root.docSchema...")
        
        try:
            # Calculate new hash
            new_hash = hashlib.sha256(corrected_content.encode()).hexdigest()
            print(f"   📊 New hash: {new_hash}")
            
            # Upload content
            upload_response = self.session.put(
                f"https://eu.tectonic.remarkable.com/sync/v3/files/{new_hash}",
                data=corrected_content.encode(),
                headers={'Content-Type': 'text/plain'}
            )
            
            if upload_response.status_code in [200, 202]:
                print(f"   ✅ Content uploaded ({upload_response.status_code})")
                
                # Update root pointer
                root_update_response = self.session.put(
                    "https://eu.tectonic.remarkable.com/sync/v4/root",
                    json={'hash': new_hash}
                )
                
                if root_update_response.status_code in [200, 202]:
                    print(f"   ✅ Root updated ({root_update_response.status_code})")
                    print(f"   🎉 REPAIR COMPLETED SUCCESSFULLY!")
                    return True
                else:
                    print(f"   ❌ Root update failed: {root_update_response.status_code}")
                    return False
            else:
                print(f"   ❌ Content upload failed: {upload_response.status_code}")
                return False
                
        except Exception as e:
            print(f"   ❌ Upload failed: {e}")
            return False

Parameters

Parameter Details

__init__: No parameters required. The constructor automatically authenticates with reMarkable cloud services, creates a backup directory for repair operations, and initializes the session for API communication.

Return Value

Instantiation returns a CorrectedRootDocSchemaRepair object with an authenticated session ready to perform repair operations. Key method returns: get_current_root_content() returns tuple of (content_string, hash_string, metadata_dict); identify_corrupted_entries() returns dict with categorized entries; fix_corrupted_document() returns corrected line string; rebuild_corrected_root() returns complete corrected content string; preview_changes() and upload_corrected_root() return boolean success status.

Class Interface

Methods

Attributes

Dependencies

• requests
• pathlib
• typing
• json
• time
• hashlib

Required Imports

import json
import time
import hashlib
from pathlib import Path
from typing import Dict, List, Tuple, Any
import requests

Conditional/Optional Imports

These imports are only needed under specific conditions:

from auth import RemarkableAuth

Usage Example

# Basic usage with dry run
repair_tool = CorrectedRootDocSchemaRepair()

# Preview changes without modifying server
success = repair_tool.preview_changes(dry_run=True)

# If preview looks good, apply changes
if success:
    success = repair_tool.preview_changes(dry_run=False)
    if success:
        print('Repair completed successfully')

# Advanced usage: manual inspection
content, hash_val, metadata = repair_tool.get_current_root_content()
categorized = repair_tool.identify_corrupted_entries(content)
print(f'Found {len(categorized["corrupted_entries"])} corrupted entries')

# Build corrected version without uploading
corrected = repair_tool.rebuild_corrected_root(categorized)

Best Practices

• Always run preview_changes(dry_run=True) before applying actual changes to inspect what will be modified
• The tool automatically creates backups in the 'docschema_repair' directory - review these before and after repairs
• Ensure stable network connection as the tool makes multiple API calls to reMarkable cloud services
• The class preserves the original version number from root.docSchema - do not manually modify this
• Only documents with suspicious sizes (2247, 2246, 2715) are modified; folders and other entries remain unchanged
• If authentication fails during __init__, a RuntimeError is raised - handle this appropriately
• The repair process fetches individual document schemas to recalculate sizes - this may take time for many corrupted entries
• Review console output carefully as it provides detailed logging of each operation and categorization
• The tool uses SHA256 hashing to generate new content hashes - ensure hashlib is available
• Backup files are timestamped and stored locally for recovery if needed

Tags

Similar Components

AI-powered semantic similarity - components with related functionality:

• class RootDocSchemaRepair 93.1% similar

  A repair tool for fixing corrupted root.docSchema entries in reMarkable cloud storage by recalculating document sizes and rebuilding the schema.

  From: /tf/active/vicechatdev/e-ink-llm/cloudtest/fix_root_docschema.py

• function main_v64 74.4% similar

  Entry point function that orchestrates a repair process for a corrupted reMarkable root.docSchema file by running a dry-run analysis first, then optionally applying the repair based on user confirmation.

  From: /tf/active/vicechatdev/e-ink-llm/cloudtest/fix_root_docschema.py

• function repair_system 70.1% similar

  Emergency repair function that resets a reMarkable cloud sync system by creating and uploading an empty root.docSchema file, then updating the root hash to restore system functionality.

  From: /tf/active/vicechatdev/e-ink-llm/cloudtest/repair_system.py

• class RootCleaner 67.7% similar

  A class that completely clears the reMarkable cloud's root.docSchema file, removing all document references while maintaining the proper file structure and version.

  From: /tf/active/vicechatdev/e-ink-llm/cloudtest/clear_root_docschema.py

• function main_v79 62.6% similar

  Interactive command-line tool that runs a schema repair process with a dry-run preview before applying changes to the root document schema.

  From: /tf/active/vicechatdev/e-ink-llm/cloudtest/corrected_repair.py