Overview

The DCBCache system provides high-performance in-memory caching for DCBDB (Data Control Block Database) entries, eliminating expensive database lookups that were taking 2.8+ seconds per query. This implementation can reduce DCBDB lookup times from seconds to milliseconds, providing significant performance improvements for online COBOL applications.

Problem Statement

Before DCBCache

• Each DCBDB lookup required a database query: SELECT DSNAME FROM "DCBDB_prod" WHERE DSNAME = ?
• Individual queries were taking 2,843ms (2.8+ seconds) due to:
  • Primary key index was present, but high latency to database
  • Frequent lookups for the same datasets
  • No caching mechanism in place
• Performance profiling showed DCBDB queries as top performance bottleneck

After DCBCache

• First access loads entire DCBDB table into memory (typically 50-200ms for 12,000-50,000 entries)
• Subsequent lookups are served from memory cache (<1ms)
• Cache hit rates of >95% expected after initial load

Architecture

Components

1. DCBCacheEntry.java - Immutable data holder for DCBDB table rows
2. DCBCache.java - Main cache manager with thread-safe operations
3. DCBDB.java - Modified to use cache when enabled (backwards compatible)
4. JMX MBean - Monitoring and management interface

Design Principles

• Backwards Compatibility: Caching disabled by default (dcbdb.cache.enabled=false)
• Thread-Safe: Uses ConcurrentHashMap and atomic operations
• Lazy Loading: Cache loads on first access, not at startup
• Memory Efficient: Stores only essential DCBDB columns
• Observable: Full JMX monitoring and statistics

Configuration

Properties

Enabling the Cache

# In deploy.properties or as JVM system properties:

# Enable DCB caching (REQUIRED - default is false)
dcbdb.cache.enabled=true

# Optional: Disable database writes for online applications
dcbdb.noupdate=true

# Optional: Enable debug logging
logdcb=true

Usage Patterns

Development/Testing Environment

# Enable caching for performance testing
dcbdb.cache.enabled=true
logdcb=true

Production Online Application

# Enable caching and disable writes
dcbdb.cache.enabled=true
dcbdb.noupdate=true

Batch Processing Environment

# Keep caching disabled for backwards compatibility
# (or enable if performance testing shows benefits)
dcbdb.cache.enabled=false

Implementation Details

Cache Loading Process

1. Trigger: First call to DCBCache.getCachedEntry() when cache enabled
2. Query: SELECT DSNAME, LRECL, DSORG, ... FROM "DCBDB_prod"
3. Storage: All entries loaded into ConcurrentHashMap<String, DCBCacheEntry>
4. Key Format: DSNAME converted to uppercase for consistent lookup
5. Thread Safety: Double-checked locking pattern for initialization

Memory Usage

• Typical Installation: ~12,000 entries ≈ 2-4 MB RAM
• Large Installation: ~50,000 entries ≈ 8-15 MB RAM
• Entry Size: ~150-300 bytes per DCB entry (varies by content)

Performance Characteristics

JMX Monitoring

MBean ObjectName

com.heirloomcomputing.ecs.isamsql:type=DCBCache

Available Attributes

Available Operations

JConsole Usage

1. Connect JConsole to your JVM
2. Navigate to: MBeans → com.heirloomcomputing.ecs.isamsql → DCBCache
3. View Attributes tab for real-time statistics
4. Use Operations tab to manage cache

Code Integration

DCBDB.java Changes

The cache is integrated into DCBDB with minimal code changes:

// In loadDCB() method - try cache first
DCBCacheEntry cacheEntry = DCBCache.getCachedEntry(datasetName, connection);
if (cacheEntry != null) {
    // Cache hit - convert to properties and return
    properties = cacheEntry.toProperties(datasetName);
    merge(properties);
    setDefaultAttributes();
    return; // Skip database query
}
// Cache miss - proceed with original database query

Backwards Compatibility

When dcbdb.cache.enabled=false (default):

• DCBCache.getCachedEntry() returns null immediately
• DCBCache.exists() returns false immediately
• All cache operations become no-ops
• DCBDB behaves exactly as before
• Zero performance overhead

Console Output Examples

Cache Disabled (Default)

DCBCache: Cache DISABLED (dcbdb.cache.enabled=null)

Cache Enabled

DCBCache: Cache ENABLED (dcbdb.cache.enabled=true)
DCBCache: Successfully registered MBean with name: com.heirloomcomputing.ecs.isamsql:type=DCBCache
DCBCache: Loaded 12487 entries in 127ms
DCBCache: HIT for CUSTOMER.MASTER
DCBCache: HIT for INVOICE.DETAIL

Statistics Output

DCB Cache Statistics:
  Cache Enabled: true
  Entries: 12487
  Cache Hits: 15234
  Cache Misses: 145
  Hit Rate: 99.06%
  Load Time: 127 ms
  Last Load: Wed Sep 11 11:45:32 UTC 2025
  No-Update Mode: true
  Initialized: true

Deployment Strategy

Phase 1: Safe Deployment

1. Deploy code with cache disabled (default behavior)
2. Verify existing functionality unchanged
3. No performance impact, full backwards compatibility

Phase 2: Performance Testing

1. Enable caching in development/test environments:

   dcbdb.cache.enabled=true
   logdcb=true

2. Run performance tests and validate improvements
3. Monitor JMX metrics for hit rates and load times

Phase 3: Production Rollout

1. Enable in production for online applications:

   dcbdb.cache.enabled=true
   dcbdb.noupdate=true  # Prevent writes in online apps

2. Monitor via JMX for performance gains
3. Gradually enable for other workloads as appropriate

Troubleshooting

Cache Not Loading

• Check: dcbdb.cache.enabled=true is set
• Check: Database connection available for bulk load
• Look for: "DCBCache: Loaded X entries" in logs

Poor Hit Rates

• Expected: >95% hit rate after warmup period
• Check: Application accessing same datasets repeatedly
• Monitor: JMX HitRate attribute over time

Memory Issues

• Monitor: JVM heap usage after cache load
• Estimate: ~300 bytes per entry (12K entries ≈ 4MB)
• Consider: Adjust heap size if needed for large installations

Performance Issues

• Initial Load: First access will include load time (50-500ms)
• Subsequent: Should be <1ms per lookup
• Database: Verify database performance for initial bulk load

Security Considerations

• No Authentication: Cache assumes trusted JVM environment
• JMX Access: Secure JMX access as per organizational policies
• Memory Contents: Cache contains dataset metadata (non-sensitive)
• Database Credentials: Uses existing DCBDB connection credentials

Future Enhancements

Potential Improvements

1. TTL/Expiration: Add time-based cache expiration
2. Selective Loading: Load subsets based on usage patterns
3. Change Detection: Detect database changes and refresh cache
4. Metrics Integration: Export metrics to monitoring systems
5. Configuration UI: Web-based cache management interface

Performance Optimizations

1. Compression: Compress cache entries to reduce memory usage
2. Partitioning: Split cache by dataset patterns for large installations
3. Preloading: Load cache at application startup for predictable performance

Testing

Unit Tests

• Cache loading and lookup operations
• Thread safety under concurrent access
• Property disable/enable functionality
• MBean registration and operations

Integration Tests

• DCBDB integration with cache enabled/disabled
• Database connection handling
• Memory usage under various dataset sizes
• JMX monitoring functionality

Performance Tests

• Compare lookup times before/after cache
• Measure cache load times for different dataset sizes
• Validate hit rates under realistic workloads
• Memory usage profiling

Conclusion

The DCBCache implementation provides a significant performance optimization for DCBDB operations while maintaining full backwards compatibility. By caching the entire DCBDB table in memory, applications can achieve >99% reduction in lookup times after the initial cache load.

The solution is production-ready with comprehensive monitoring, proper error handling, and a phased deployment approach that ensures zero risk to existing functionality.