patroni.log module
Patroni logging facilities.
Daemon processes will use a 2-step logging handler. Whenever a log message is issued it is initially enqueued in-memory and is later asynchronously flushed by a thread to the final destination.
- class patroni.log.PatroniFileHandler(filename: str, mode: Optional[int])
Bases:
logging.handlers.RotatingFileHandler
Wrapper of
RotatingFileHandler
to handle permissions of log files.- __init__(filename: str, mode: Optional[int]) None
Create a new
PatroniFileHandler
instance.- Parameters
filename – basename for log files.
mode – permissions for log files.
- _open() _io.TextIOWrapper
Open a new log file and assign permissions.
- Returns
the resulting stream.
- class patroni.log.PatroniLogger
Bases:
threading.Thread
Logging thread for the Patroni daemon process.
It is a 2-step logging approach. Any time a log message is issued it is initially enqueued in-memory, and then asynchronously flushed to the final destination by the logging thread.
See also
QueueHandler
: object used for enqueueing messages in-memory.- Variables
DEFAULT_TYPE – default type of log format (
plain
).DEFAULT_LEVEL – default logging level (
INFO
).DEFAULT_TRACEBACK_LEVEL – default traceback logging level (
ERROR
).DEFAULT_FORMAT – default format of log messages (
%(asctime)s %(levelname)s: %(message)s
).NORMAL_LOG_QUEUE_SIZE – expected number of log messages per HA loop when operating under a normal situation.
DEFAULT_MAX_QUEUE_SIZE – default maximum queue size for holding a backlog of log messages that are pending to be flushed.
LOGGING_BROKEN_EXIT_CODE – exit code to be used if it detects(
5
).log_handler – log handler that is currently being used by the thread.
log_handler_lock – lock used to modify
log_handler
.
- DEFAULT_FORMAT = '%(asctime)s %(levelname)s: %(message)s'
- DEFAULT_LEVEL = 'INFO'
- DEFAULT_MAX_QUEUE_SIZE = 1000
- DEFAULT_TRACEBACK_LEVEL = 'ERROR'
- DEFAULT_TYPE = 'plain'
- LOGGING_BROKEN_EXIT_CODE = 5
- NORMAL_LOG_QUEUE_SIZE = 2
- __init__() None
Prepare logging queue and proxy handlers as they become ready during daemon startup.
Note
While Patroni is starting up it keeps
DEBUG
log level, and writes log messages through a proxy handler. Once the logger thread is finally started, it switches from that proxy handler to the queue based logger, and applies the configured log settings. The switching is used to avoid that the logger thread prevents Patroni from shutting down if any issue occurs in the meantime until the thread is properly started.
- _close_old_handlers() None
Close old log handlers.
Note
It is used to remove different handlers that were configured previous to a reload in the configuration, e.g. if we are switching from
PatroniFileHandler
to class:~logging.StreamHandler and vice-versa.
- _get_formatter(config: Dict[str, Any]) logging.Formatter
Returns a logging formatter based on the type of logger in the given configuration.
- Parameters
config –
log
section from Patroni configuration.- Returns
A
logging.Formatter
object that can be used to format log records.
- _get_json_formatter(logformat: Union[List[Union[str, Dict[str, Any], Any]], str, Any], dateformat: Optional[str], static_fields: Dict[str, Any]) logging.Formatter
Returns a logging formatter that outputs JSON formatted messages.
Note
If
pythonjsonlogger
library is not installed, prints an error message and returns a plain log formatter instead.- Parameters
logformat – Specifies the log fields and their key names in the JSON log message.
dateformat – The format of the timestamp in the log messages.
static_fields – A dictionary of static fields that are added to every log message.
- Returns
A logging formatter object that can be used to format log records as JSON strings.
- _get_plain_formatter(logformat: Union[List[Union[str, Dict[str, Any], Any]], str, Any], dateformat: Optional[str]) logging.Formatter
Returns a logging formatter with the specified format and date format.
Note
If the log format isn’t a string, prints a warning message and uses the default log format instead.
- Parameters
logformat – The format of the log messages.
dateformat – The format of the timestamp in the log messages.
- Returns
A logging formatter object that can be used to format log records.
- _is_config_changed(config: Dict[str, Any]) bool
Checks if the given config is different from the current one.
- Parameters
config –
log
section from Patroni configuration.- Returns
True
if the config is changed,False
otherwise.
- reload_config(config: Dict[str, Any]) None
Apply log related configuration.
Note
It is also able to deal with runtime configuration changes.
- Parameters
config –
log
section from Patroni configuration.
- run() None
Run logger’s thread main loop.
Keep consuming log queue until requested to quit through
None
special log record.
- update_loggers(config: Dict[str, Any]) None
Configure custom loggers’ log levels.
Note
It creates logger objects that are not defined yet in the log manager.
- Parameters
config –
dict
object with custom loggers configuration, is set either from:log.loggers
section of Patroni configuration; orfrom the method that is trying to make sure that the node name isn’t duplicated (to silence annoying
urllib3
WARNING’s).
- Example
update_loggers({'urllib3.connectionpool': 'WARNING'})
- class patroni.log.ProxyHandler(patroni_logger: patroni.log.PatroniLogger)
Bases:
logging.Handler
Handle log records in place of pending log handlers.
Note
This is used to handle log messages while the logger thread has not started yet, in which case the queue-based handler is not yet started.
- Variables
patroni_logger – the logger thread.
- __init__(patroni_logger: patroni.log.PatroniLogger) None
Create a new
ProxyHandler
instance.- Parameters
patroni_logger – the logger thread.
- emit(record: logging.LogRecord) None
Emit each log record that is handled.
Will push the log record down to
handle()
method of the currently configured log handler.- Parameters
record – the record that was emitted.
- class patroni.log.QueueHandler
Bases:
logging.Handler
Queue-based logging handler.
- Variables
queue – queue to hold log messages that are pending to be flushed to the final destination.
- _put_record(record: logging.LogRecord) None
Asynchronously enqueue a log record.
- Parameters
record – the record to be logged.
- _try_to_report_lost_records() None
Report the number of log messages that have been lost and reset the counter.
Note
It will issue an
WARNING
message in the logs with the number of lost log messages.
- emit(record: logging.LogRecord) None
Handle each log record that is emitted.
Call
_put_record()
to enqueue the emitted log record.Also check if we have previously lost any log record, and if so, log a
WARNING
message.- Parameters
record – the record that was emitted.
- patroni.log._type(value: Any) str
Get type of the value.
- Parameters
value – any arbitrary value.
- Returns
a string with a type name.
- patroni.log.debug_exception(self: logging.Logger, msg: object, *args: Any, **kwargs: Any) None
Add full stack trace info to debug log messages and partial to others.
Handle
exception()
calls for self.Note
If self log level is set to
DEBUG
, then issue aDEBUG
message with the complete stack trace;- If self log level is
INFO
or higher, then issue anERROR
message with only the last line of the stack trace.
- If self log level is
- Parameters
self – logger for which
exception()
will be processed.msg – the message related to the exception to be logged.
args – positional arguments to be passed to
debug()
orerror()
.kwargs – keyword arguments to be passed to
debug()
orerror()
.
- patroni.log.error_exception(self: logging.Logger, msg: object, *args: Any, **kwargs: Any) None
Add full stack trace info to error messages.
Handle
exception()
calls for self.Note
By default issue an
ERROR
message with the complete stack trace. If you do not want to show the complete stack trace, call withexc_info=False
.
- Parameters
self – logger for which
exception()
will be processed.msg – the message related to the exception to be logged.
args – positional arguments to be passed to
error()
.kwargs – keyword arguments to be passed to
error()
.