Logging#

OpenSTEF uses Python’s standard logging library to provide information about its operations. By default, OpenSTEF configures a null handler, giving you complete control over how logging is handled in your application.

Overview#

OpenSTEF follows these logging principles:

  • No default output: OpenSTEF uses NullHandler by default, so no log messages appear unless you configure logging

  • Standard library: Uses Python’s built-in logging module for consistency

  • Hierarchical loggers: Package-level and module-level loggers allow granular control

  • Structured context: Log messages include contextual information through extras

Basic Configuration#

To see OpenSTEF log messages, configure logging in your application:

import logging

# Basic configuration - shows INFO level and above
logging.basicConfig(
    level=logging.INFO,
    format='%(asctime)s - %(name)s - %(levelname)s - %(message)s'
)

# Now OpenSTEF operations will produce log output
# TODO: Update with actual OpenSTEF classes when implemented
from openstef_models import LinearForecaster
forecaster = LinearForecaster()  # Will log initialization details

Log Levels#

OpenSTEF uses standard Python logging levels:

  • DEBUG: Detailed diagnostic information, typically only of interest when diagnosing problems

  • INFO: General information about program execution

  • WARNING: Something unexpected happened, but the software is still working

  • ERROR: A serious problem occurred that prevented a function from completing

  • CRITICAL: A very serious error occurred that may prevent the program from continuing

Example configuration for different use cases:

import logging

# Development: See everything
logging.basicConfig(level=logging.DEBUG)

# Production: Important messages only
logging.basicConfig(level=logging.WARNING)

# Data science workflows: Informational messages
logging.basicConfig(level=logging.INFO)

Logger Hierarchy#

OpenSTEF loggers follow Python’s hierarchical naming convention. You can control logging at different levels of granularity:

Package level control#

Control logging for entire OpenSTEF packages:

import logging

# Disable all openstef-models logging
logging.getLogger('openstef_models').setLevel(logging.CRITICAL)

# Show only warnings from openstef-beam
logging.getLogger('openstef_beam').setLevel(logging.WARNING)

# Enable debug mode for specific package
logging.getLogger('openstef_models').setLevel(logging.DEBUG)

Module level control#

Control logging for specific modules:

import logging

# Only show errors from the presets module
logging.getLogger('openstef_models.presets').setLevel(logging.ERROR)

# Debug feature engineering specifically
logging.getLogger('openstef_models.transforms').setLevel(logging.DEBUG)

Advanced Configuration#

For more advanced logging configurations like custom formatters, file handlers, rotating logs, and other features, refer to the official Python logging documentation.

Contextual Information#

OpenSTEF includes contextual information in log messages to help with debugging and monitoring. Many log messages include extra fields that provide additional context:

# Example of structured log output (when using appropriate formatters)
2025-01-20 14:30:25 - openstef_models.training - INFO - Model training started
    extra_info: {
        'model_type': 'XGBoostForecaster',
        'training_samples': 8760,
        'features': ['temperature', 'humidity', 'hour_of_day'],
        'horizon': 24
    }

This contextual information is particularly useful when:

  • Debugging model training issues

  • Monitoring model performance in production

  • Analyzing feature engineering pipelines

  • Tracking data processing workflows

Integration Examples#

Jupyter Notebooks#

Configure logging for interactive data science work:

import logging

# Configure for notebook use
logging.basicConfig(
    level=logging.INFO,
    format='%(name)-20s | %(levelname)-8s | %(message)s',
    force=True  # Override any existing configuration
)

# Now OpenSTEF operations will show progress
# TODO: Update with actual OpenSTEF classes when implemented
from openstef_models import load_sample_data, XGBoostForecaster

data = load_sample_data()  # Shows data loading progress
model = XGBoostForecaster()
model.fit(data)  # Shows training progress

Structured logging with structlog#

If you’re using structlog in your application, you can configure OpenSTEF to work with it by integrating structlog with Python’s standard logging:

import logging
import structlog

# Configure structlog to integrate with standard logging
structlog.configure(
    processors=[
        structlog.stdlib.filter_by_level,
        structlog.stdlib.add_logger_name,
        structlog.stdlib.add_log_level,
        structlog.stdlib.PositionalArgumentsFormatter(),
        structlog.processors.StackInfoRenderer(),
        structlog.processors.format_exc_info,
        structlog.processors.UnicodeDecoder(),
        structlog.processors.JSONRenderer()
    ],
    context_class=dict,
    logger_factory=structlog.stdlib.LoggerFactory(),
    wrapper_class=structlog.stdlib.BoundLogger,
    cache_logger_on_first_use=True,
)

# Configure standard logging
logging.basicConfig(
    format="%(message)s",
    level=logging.INFO,
)

# Now OpenSTEF logs will be processed by structlog
# TODO: Update with actual OpenSTEF classes when implemented
from openstef_models import create_forecaster
forecaster = create_forecaster()

For more advanced structlog configurations and features, see the structlog standard library integration guide.

Standard logging setup#

For most applications, a simple standard logging configuration is sufficient:

import logging

# Configure root logger
logging.basicConfig(
    level=logging.INFO,
    format='%(asctime)s - %(name)s - %(levelname)s - %(message)s',
    datefmt='%Y-%m-%d %H:%M:%S'
)

# Optional: Adjust OpenSTEF package levels
logging.getLogger('openstef_models').setLevel(logging.INFO)
logging.getLogger('openstef_beam').setLevel(logging.WARNING)

# Now use OpenSTEF with logging
# TODO: Update with actual OpenSTEF classes when implemented
from openstef_models import LinearForecaster
forecaster = LinearForecaster()  # Will log initialization details

Troubleshooting#

No log output appearing#

If you’re not seeing any OpenSTEF log messages:

  1. Check if logging is configured: OpenSTEF uses NullHandler by default

  2. Verify log levels: Ensure your handler level isn’t too restrictive

  3. Check logger hierarchy: Parent logger settings can override child settings

import logging

# Debug logging configuration
logger = logging.getLogger('openstef_models')
print(f"Logger level: {logger.level}")
print(f"Effective level: {logger.getEffectiveLevel()}")
print(f"Handlers: {logger.handlers}")
print(f"Parent handlers: {logger.parent.handlers}")

Too much log output#

If OpenSTEF is producing too many log messages:

import logging

# Reduce OpenSTEF verbosity
logging.getLogger('openstef_models').setLevel(logging.WARNING)
logging.getLogger('openstef_beam').setLevel(logging.WARNING)

# Or disable specific noisy modules
logging.getLogger('openstef_models.transforms').setLevel(logging.ERROR)

Performance considerations#

Logging can impact performance in tight loops. OpenSTEF follows best practices:

  • Log messages are not formatted unless actually output

  • Debug logging is conditionally executed

  • Structured logging uses lazy evaluation

You can further optimize by:

import logging

# Set appropriate levels to avoid unnecessary processing
logging.getLogger('openstef_models').setLevel(logging.WARNING)

# Use logging filters for fine-grained control
class PerformanceFilter(logging.Filter):
    def filter(self, record):
        # Skip debug messages during performance-critical sections
        return record.levelno >= logging.INFO

logging.getLogger('openstef_models').addFilter(PerformanceFilter())

Best Practices#

For library users:

  1. Configure logging early: Set up logging before importing OpenSTEF modules

  2. Use appropriate levels: INFO for general monitoring, DEBUG for troubleshooting

  3. Leverage hierarchical control: Use package/module-level logger configuration

  4. Integrate with your existing setup: OpenSTEF works with any Python logging configuration

Example complete setup:

import logging

def setup_openstef_logging(level=logging.INFO):
    """Set up logging for OpenSTEF integration."""

    # Basic configuration
    logging.basicConfig(
        level=level,
        format='%(asctime)s - %(name)-25s - %(levelname)-8s - %(message)s',
        datefmt='%Y-%m-%d %H:%M:%S'
    )

    # Optional: Fine-tune specific packages
    logging.getLogger('openstef_models').setLevel(level)
    logging.getLogger('openstef_beam').setLevel(logging.WARNING)  # Less verbose

# Use in your application
setup_openstef_logging(level=logging.INFO)

# Now use OpenSTEF with proper logging
# TODO: Update with actual OpenSTEF classes when implemented
from openstef_models import create_forecaster
forecaster = create_forecaster()