logo
Splunk Team

how to output logs with python logging in a click cli?

  • Last Update :
  • Techknowledgy :

Here's an example of how I usually do it:

import click
from loguru
import logger

@click.command()
@click.option('--count',
   default = 1, help = 'Number of greetings.')
@click.option('--name', prompt = 'Your name', help = 'The person to greet.')
def hello(count, name):
   ""
"Simple program that greets NAME for a total of COUNT times."
""
for x in range(count):
   logger.info(f "That's it, beautiful and simple logging! - Counter: {x}")
click.echo('Hello %s!' % name)

if __name__ == '__main__':
   hello()
❯
python loguru - click - cli - test.py--count = 3
Your name: Geraldo
2020 - 06 - 10 20: 02: 48.297 | INFO | __main__: hello: 11 - That 's it, beautiful and simple logging! - Counter: 0
Hello Geraldo!
   2020 - 06 - 10 20: 02: 48.297 | INFO | __main__: hello: 11 - That 's it, beautiful and simple logging! - Counter: 1
Hello Geraldo!
   2020 - 06 - 10 20: 02: 48.297 | INFO | __main__: hello: 11 - That 's it, beautiful and simple logging! - Counter: 2
Hello Geraldo!
I had to manually set
logging.basicConfig(level = logging.INFO)
Example
import click
import logging

logging.basicConfig(level = logging.INFO)

@click.command()
def cli():
   logging.info("it works")
gives
$ mycli
INFO: root: it works
I was looking to get logs when a click handler is called (command name and parameters). I end up with this snippet:
from functools
import wraps
import logging
import click

logger = logging.getLogger(__name__)

def click_log(fn):
   @wraps(fn)
def wrapper( * args, ** kwargs):
   context = click.get_current_context()
logger.info(f "{context.command.name}(**{context.params})")
return fn( * args, ** kwargs)
return wrapper

@click.command()
@click.option('--count',
   default = 1, help = 'Number of greetings.')
@click.option('--name', prompt = 'Your name',
   help = 'The person to greet.')
@click_log
def hello(count, name):
   ""
"Simple program that greets NAME for a total of COUNT times."
""
for x in range(count):
   click.echo(f "Hello {name}!") # you can use logger.info

def configure_logger():
   logging.basicConfig(level = "DEBUG")

configure_logger()

if __name__ == '__main__':
   hello()
It gives you the following output:
python test_click.py--count 3
Your name: ok
INFO: __main__: hello( ** {
   'count': 3,
   'name': 'ok'
})
Hello ok!
   Hello ok!
   Hello ok!

Suggestion : 2
if creating a logger with logging.getLogger, I have to create a handler for my logger explicitly.,I was looking to get logs when a click anycodings_logging  handler is called (command name and anycodings_logging  parameters). I end up with this snippet:,A third option--creating a handler and anycodings_click  setting loglevels for the handler and the anycodings_click  logger-- works: ,The logger content doesn't appear in my anycodings_click  console. Maybe it needs a handler to anycodings_click  explicitly send log messages to stdout? 
I've written a python cli with the 'click' anycodings_click  library. I'd like to also use python's anycodings_click  built-in logging module for logging to the anycodings_click  console. But I've struggled getting the anycodings_click  logging messages to the console. I tried a anycodings_click  very simple approach:
logger = logging.getLogger(__name__)

@click.command()
def cli():
   logger.setLevel("INFO")
logger.info("Does this work?")
print("done.")
The logger content doesn't appear in my anycodings_click  console. Maybe it needs a handler to anycodings_click  explicitly send log messages to stdout? 
logger = logging.getLogger(__name__)

@click.command()
def cli():
   handler = logging.StreamHandler(sys.stdout)
handler.setFormatter(logging.Formatter('%(asctime)s %(message)s'))
handler.setLevel("INFO")
logger.addHandler(handler)
logger.info("Does this work?")
print("done.")
A third option--creating a handler and anycodings_click  setting loglevels for the handler and the anycodings_click  logger-- works: 
logger = logging.getLogger(__name__)

@click.command()
def cli():
   logger.setLevel("INFO")
handler = logging.StreamHandler(sys.stderr)
handler.setFormatter(logging.Formatter('%(asctime)s %(message)s'))
handler.setLevel("INFO")
logger.addHandler(handler)
logger.info("Does this work?")
print("done.")
Here's an example of how I usually do anycodings_logging  it:
import click
from loguru
import logger

@click.command()
@click.option('--count',
   default = 1, help = 'Number of greetings.')
@click.option('--name', prompt = 'Your name', help = 'The person to greet.')
def hello(count, name):
   ""
"Simple program that greets NAME for a total of COUNT times."
""
for x in range(count):
   logger.info(f "That's it, beautiful and simple logging! - Counter: {x}")
click.echo('Hello %s!' % name)

if __name__ == '__main__':
   hello()
⠝ ¯ python loguru - click - cli - test.py--count = 3
Your name: Geraldo
2020 - 06 - 10 20: 02: 48.297 | INFO | __main__: hello: 11 - That 's it, beautiful and simple logging! - Counter: 0
Hello Geraldo!
   2020 - 06 - 10 20: 02: 48.297 | INFO | __main__: hello: 11 - That 's it, beautiful and simple logging! - Counter: 1
Hello Geraldo!
   2020 - 06 - 10 20: 02: 48.297 | INFO | __main__: hello: 11 - That 's it, beautiful and simple logging! - Counter: 2
Hello Geraldo!
I had to manually set
logging.basicConfig(level = logging.INFO)
Example
import click
import logging

logging.basicConfig(level = logging.INFO)

@click.command()
def cli():
   logging.info("it works")
gives
$ mycli
INFO: root: it works
I was looking to get logs when a click anycodings_logging  handler is called (command name and anycodings_logging  parameters). I end up with this snippet:
from functools
import wraps
import logging
import click

logger = logging.getLogger(__name__)

def click_log(fn):
   @wraps(fn)
def wrapper( * args, ** kwargs):
   context = click.get_current_context()
logger.info(f "{context.command.name}(**{context.params})")
return fn( * args, ** kwargs)
return wrapper

@click.command()
@click.option('--count',
   default = 1, help = 'Number of greetings.')
@click.option('--name', prompt = 'Your name',
   help = 'The person to greet.')
@click_log
def hello(count, name):
   ""
"Simple program that greets NAME for a total of COUNT times."
""
for x in range(count):
   click.echo(f "Hello {name}!") # you can use logger.info

def configure_logger():
   logging.basicConfig(level = "DEBUG")

configure_logger()

if __name__ == '__main__':
   hello()
It gives you the following output:
python test_click.py--count 3
Your name: ok
INFO: __main__: hello( ** {
   'count': 3,
   'name': 'ok'
})
Hello ok!
   Hello ok!
   Hello ok!

Suggestion : 3
Click-log: Simple and beautiful logging for click applications,Click-log: Simple and beautiful logging for click applications
click-log
License
Getting started
API
Classes


Indices and tables

,Assuming you have this Click application:,Set up the default handler (ClickHandler) and formatter
(ColorFormatter) on the given logger.
@click.command()
def cli():
   click.echo("Dividing by zero.")

try:
1 / 0
except:
   click.echo("ERROR: Failed to divide by zero.")
@click.command()
@click.option('--quiet',
   default = False, is_flag = True)
def cli(quiet):
   if not quiet:
   click.echo("Dividing by zero.")

try:
1 / 0
except:
   click.echo("ERROR: Failed to divide by zero.")
import logging
logger = logging.getLogger(__name__)
# More setup
for logging handlers here

@click.command()
@click.option('--quiet',
   default = False, is_flag = True)
def cli(quiet):
   if quiet:
   logger.setLevel(logging.ERROR)
else:
   logger.setLevel(logging.INFO)

   ...
import logging
logger = logging.getLogger(__name__)
click_log.basic_config(logger)

@click.command()
@click_log.simple_verbosity_option(logger)
def cli():
   logger.info("Dividing by zero.")

try:
1 / 0
except:
   logger.error("Failed to divide by zero.")
Dividing by zero.
error: Failed to divide by zero.

Suggestion : 4
In this article I describe how to properly set up logging in a Python command line
interface (or CLI), so that we can set up a industry-ready command line program.,It can be confusing sometimes to add logging to command line interfaces programs, though.
Above we showed how to do this.,Novice Python programmers (or veteran Python programmers) may be a bit frustrated
configuring logging for a command line application. Unfortunately, while Python’s logging
module is highly configuragle, this power comes at the cost of clarity.,In this tutorial we showed how to properly configure logging in Python. What Python
lacks in clarity it makes up for in granularity. Python’s logging module is one of the
most powerful out there, and there are many modules you can incorporate to add
formatters, handlers, and more!
Python is often used as a scrpting language for command line utilities. The basic template
for such a program looks like this:
def main():
   print("Hello World!")

if __name__ == '__main__':
   main()
The program is run like this:
$ python hello_world.py
Hello World!
or like this:
$ python - m hello_world
Hello World!
This program has an optional argument name, which will be printed in place of World,
if given.
$ python - m hello
Hello, World!

   $ python - m hello Jerry
Hello, Jerry!
Here’s the code for the program:
import sys
from datetime
import datetime, timedelta
import argparse
import logging

# Only used
for type hinting.
import typing as T

parser = argparse.ArgumentParser()
parser.add_argument("name", nargs = "?",
   default = "World")
parser.add_argument("-o", "--offset", type = int,
   default = 0)

# Create a new logger
log = logging.getLogger(__name__)

def say_hello(name: T.Any, hour_offset: int):
   # Boundary checking.
# NOTE: typically we 'd want to raise an exception here,
# such as a RuntimeException.However, to demonstrate logging,
   # we 'll just log an error, then exit gracefully.
# NOTE: Actually, more than likely the proper way to do this
   # would be to do bounds checking via argparser.
   if abs(hour_offset) > 23:
   log.error("`hour_offset` must be less than 24!")
sys.exit(1)
curr_time = datetime.now()
check_time = curr_time + timedelta(hours = hour_offset)
log.debug("Current time: %s", curr_time.strftime("%c"))
log.debug("Offset in hours: %s", hour_offset)
log.debug("Time to check: %s", check_time)
if check_time.hour > 16:
   print(f "Closing time, {name}. I'm done for the day.")
return
if check_time.hour > 12:
   print(f "Good afternoon, {name}. How was lunch?")
return
print(f "{name}, I need coffee.")
return

def main(args):
   hour_offset = int(args.offset)
name = args.name
say_hello(name, hour_offset)

if __name__ == '__main__':
   args = parser.parse_args()
main(args)

Suggestion : 5

                  
  Jan 6, 2021

                ,
        Released: 
  Jan 6, 2021

      ,
          Uploaded 
  Jan 6, 2021

          
          
          py2
          
          py3
          
          
        ,
          Uploaded 
  Jan 6, 2021

          
          
          source
          
          
        
Assuming you have this Click application:
import click

@click.command()
def cli():
   click.echo("Dividing by zero.")

try:
1 / 0
except ZeroDivisionError:
   click.echo("ERROR: Failed to divide by zero.")
Ignore the application's core functionality for a moment. The much more pressing question here is: How do we add an option to not print anything on success? We could try this:
import click

@click.command()
@click.option('--quiet',
   default = False, is_flag = True)
def cli(quiet):
   if not quiet:
   click.echo("Dividing by zero.")

try:
1 / 0
except ZeroDivisionError:
   click.echo("ERROR: Failed to divide by zero.")
Wrapping if-statements around each echo-call is cumbersome though. And with that, we discover logging:
import logging
import click

logger = logging.getLogger(__name__)
# More setup
for logging handlers here

@click.command()
@click.option('--quiet',
   default = False, is_flag = True)
def cli(quiet):
   if quiet:
   logger.setLevel(logging.ERROR)
else:
   logger.setLevel(logging.INFO)
#...
The output will look like this:
Dividing by zero.
error: Failed to divide by zero.
You can customize click styles for each log level with
style_kwargs keyword argument of basic_config function.
import logging
import click_logging

logger = logging.getLogger(__name__)
style_kwargs = {
   'error': dict(fg = 'red', blink = True),
   'exception': dict(fg = 'red', blink = True),
   'critical': dict(fg = 'red', blink = True)
}
click_logging.basic_config(logger, style_kwargs = style_kwargs)

Suggestion : 6
If we change the logging level, then we can change the information sent to the
log. For example, if we want more information:,Note that the formatting of logging messages for final output to logs is
completely independent of how an individual logging message is constructed.
That can still use %-formatting, as shown here:,While the above examples use print() to show how the formatting works, you
would of course use logger.debug() or similar to actually log using this
approach.,You can then specify, in a logging configuration passed to dictConfig(),
that a logging handler be created by calling this function:
import logging
import auxiliary_module

# create logger with 'spam_application'
logger = logging.getLogger('spam_application')
logger.setLevel(logging.DEBUG)
# create file handler which logs even debug messages
fh = logging.FileHandler('spam.log')
fh.setLevel(logging.DEBUG)
# create console handler with a higher log level
ch = logging.StreamHandler()
ch.setLevel(logging.ERROR)
# create formatter and add it to the handlers
formatter = logging.Formatter('%(asctime)s - %(name)s - %(levelname)s - %(message)s')
fh.setFormatter(formatter)
ch.setFormatter(formatter)
# add the handlers to the logger
logger.addHandler(fh)
logger.addHandler(ch)

logger.info('creating an instance of auxiliary_module.Auxiliary')
a = auxiliary_module.Auxiliary()
logger.info('created an instance of auxiliary_module.Auxiliary')
logger.info('calling auxiliary_module.Auxiliary.do_something')
a.do_something()
logger.info('finished auxiliary_module.Auxiliary.do_something')
logger.info('calling auxiliary_module.some_function()')
auxiliary_module.some_function()
logger.info('done with auxiliary_module.some_function()')
import logging

# create logger
module_logger = logging.getLogger('spam_application.auxiliary')

class Auxiliary:
   def __init__(self):
   self.logger = logging.getLogger('spam_application.auxiliary.Auxiliary')
self.logger.info('creating an instance of Auxiliary')

def do_something(self):
   self.logger.info('doing something')
a = 1 + 1
self.logger.info('done doing something')

def some_function():
   module_logger.info('received a call to "some_function"')
2005 - 03 - 23 23: 47: 11, 663 - spam_application - INFO -
   creating an instance of auxiliary_module.Auxiliary
2005 - 03 - 23 23: 47: 11, 665 - spam_application.auxiliary.Auxiliary - INFO -
   creating an instance of Auxiliary
2005 - 03 - 23 23: 47: 11, 665 - spam_application - INFO -
   created an instance of auxiliary_module.Auxiliary
2005 - 03 - 23 23: 47: 11, 668 - spam_application - INFO -
   calling auxiliary_module.Auxiliary.do_something
2005 - 03 - 23 23: 47: 11, 668 - spam_application.auxiliary.Auxiliary - INFO -
   doing something
2005 - 03 - 23 23: 47: 11, 669 - spam_application.auxiliary.Auxiliary - INFO -
   done doing something
2005 - 03 - 23 23: 47: 11, 670 - spam_application - INFO -
   finished auxiliary_module.Auxiliary.do_something
2005 - 03 - 23 23: 47: 11, 671 - spam_application - INFO -
   calling auxiliary_module.some_function()
2005 - 03 - 23 23: 47: 11, 672 - spam_application.auxiliary - INFO -
   received a call to 'some_function'
2005 - 03 - 23 23: 47: 11, 673 - spam_application - INFO -
   done with auxiliary_module.some_function()
import logging
import threading
import time

def worker(arg):
   while not arg['stop']:
   logging.debug('Hi from myfunc')
time.sleep(0.5)

def main():
   logging.basicConfig(level = logging.DEBUG, format = '%(relativeCreated)6d %(threadName)s %(message)s')
info = {
   'stop': False
}
thread = threading.Thread(target = worker, args = (info, ))
thread.start()
while True:
   try:
   logging.debug('Hello from main')
time.sleep(0.75)
except KeyboardInterrupt:
   info['stop'] = True
break
thread.join()

if __name__ == '__main__':
   main()
   0 Thread - 1 Hi from myfunc
   3 MainThread Hello from main
   505 Thread - 1 Hi from myfunc
   755 MainThread Hello from main
   1007 Thread - 1 Hi from myfunc
   1507 MainThread Hello from main
   1508 Thread - 1 Hi from myfunc
   2010 Thread - 1 Hi from myfunc
   2258 MainThread Hello from main
   2512 Thread - 1 Hi from myfunc
   3009 MainThread Hello from main
   3013 Thread - 1 Hi from myfunc
   3515 Thread - 1 Hi from myfunc
   3761 MainThread Hello from main
   4017 Thread - 1 Hi from myfunc
   4513 MainThread Hello from main
   4518 Thread - 1 Hi from myfunc
import logging

logger = logging.getLogger('simple_example')
logger.setLevel(logging.DEBUG)
# create file handler which logs even debug messages
fh = logging.FileHandler('spam.log')
fh.setLevel(logging.DEBUG)
# create console handler with a higher log level
ch = logging.StreamHandler()
ch.setLevel(logging.ERROR)
# create formatter and add it to the handlers
formatter = logging.Formatter('%(asctime)s - %(name)s - %(levelname)s - %(message)s')
ch.setFormatter(formatter)
fh.setFormatter(formatter)
# add the handlers to logger
logger.addHandler(ch)
logger.addHandler(fh)

# 'application'
code
logger.debug('debug message')
logger.info('info message')
logger.warning('warn message')
logger.error('error message')
logger.critical('critical message')

Suggestion : 7
That is all there is to it. In Python, __name__ contains the full name of the current module, so this will simply work in any module.,Python comes with a logging module in the standard library that provides a flexible framework for emitting log messages from Python programs. This module is widely used by libraries and is the first go-to point for most developers when it comes to logging.,This article covers the basics of using the standard logging module that ships with all Python distributions. After reading this, you should be able to easily integrate logging into your Python application.,Applications should configure logging as early as possible, preferably as the first thing in the application, so that log messages do not get lost during startup.
To emit a log message, a caller first requests a named logger. The name can be used by the application to configure different rules for different loggers. This logger then can be used to emit simply-formatted messages at different log levels (DEBUG, INFO, ERROR, etc.), which again can be used by the application to handle messages of higher priority different than those of a lower priority. While it might sound complicated, it can be as simple as this:
import logging
log = logging.getLogger("my-logger")
log.info("Hello, world")
The only responsibility modules have is to make it easy for the application to route their log messages. For this reason, it is a convention for each module to simply use a logger named like the module itself. This makes it easy for the application to route different modules differently, while also keeping log code in the module simple. The module just needs two lines to set up logging, and then use the named logger:
import logging

log = logging.getLogger(__name__)

def do_something():
   log.debug("Doing something!")
In the case of running Python in containers like Docker, logging to standard output is also often the easiest move as this output can be directly and easily managed by the container itself.
import logging
import os

logging.basicConfig(level = os.environ.get("LOGLEVEL", "INFO"))

exit(main())
The alternative is to send it directly to syslog. This is great for older operating systems that don’t have systemd. In an ideal world this should be simple, but sadly, Python requires a bit more elaborate configuration to be able to send unicode log messages. Here is a sample implementation.
import logging
import logging.handlers
import os

class SyslogBOMFormatter(logging.Formatter):
   def format(self, record):
   result = super().format(record)
return "ufeff" + result

handler = logging.handlers.SysLogHandler('/dev/log')
formatter = SyslogBOMFormatter(logging.BASIC_FORMAT)
handler.setFormatter(formatter)
root = logging.getLogger()
root.setLevel(os.environ.get("LOGLEVEL", "INFO"))
root.addHandler(handler)

try:
exit(main())
except Exception:
   logging.exception("Exception in main()")
exit(1)
Here’s a sample implementation.
import logging
import logging.handlers
import os

handler = logging.handlers.WatchedFileHandler(
   os.environ.get("LOGFILE", "/var/log/yourapp.log"))
formatter = logging.Formatter(logging.BASIC_FORMAT)
handler.setFormatter(formatter)
root = logging.getLogger()
root.setLevel(os.environ.get("LOGLEVEL", "INFO"))
root.addHandler(handler)

try:
exit(main())
except Exception:
   logging.exception("Exception in main()")
exit(1)

Suggestion : 8
Published: April 11, 2019
import logging

def word_count(myfile):
   logging.basicConfig(level = logging.DEBUG, filename = 'myapp.log', format = '%(asctime)s %(levelname)s:%(message)s')
try:
# count the number of words in a file and log the result
with open(myfile, 'r') as f:
   file_data = f.read()
words = file_data.split(" ")
num_words = len(words)
logging.debug("this file has %d words", num_words)
return num_words
except OSError as e:
   logging.error("error reading the file")[...]
2019 - 03 - 27 10: 49: 00, 979 DEBUG: this file has 44 words
2019 - 03 - 27 10: 49: 00, 979 ERROR: error reading the file
logger = logging.getLogger(__name__)
# lowermodule.py
import logging

logging.basicConfig(level = logging.DEBUG, format = '%(asctime)s %(name)s %(levelname)s:%(message)s')
logger = logging.getLogger(__name__)

def word_count(myfile):
   try:
   with open(myfile, 'r') as f:
   file_data = f.read()
words = file_data.split(" ")
final_word_count = len(words)
logger.info("this file has %d words", final_word_count)
return final_word_count
except OSError as e:
   logger.error("error reading the file")[...]

# uppermodule.py

import logging
import lowermodule

logging.basicConfig(level = logging.DEBUG, format = '%(asctime)s %(name)s %(levelname)s:%(message)s')
logger = logging.getLogger(__name__)

def record_word_count(myfile):
   logger.info("starting the function")
try:
word_count = lowermodule.word_count(myfile)
with open('wordcountarchive.csv', 'a') as file:
   row = str(myfile) + ',' + str(word_count)
file.write(row + '\n')
except:
   logger.warning("could not write file %s to destination", myfile)
finally:
logger.debug("the function is done for the file %s", myfile)
2019 - 03 - 27 21: 16: 41, 200 __main__ INFO: starting the
function
2019 - 03 - 27 21: 16: 41, 200 lowermodule INFO: this file has 44 words
2019 - 03 - 27 21: 16: 41, 201 __main__ DEBUG: the
function is done
for the file myfile.txt
2019 - 03 - 27 21: 16: 41, 201 __main__ INFO: starting the
function
2019 - 03 - 27 21: 16: 41, 202 lowermodule ERROR: [Errno 2] No such file or directory: 'nonexistentfile.txt'
2019 - 03 - 27 21: 16: 41, 202 __main__ DEBUG: the
function is done
for the file nonexistentfile.txt
[loggers]
keys = root

[handlers]
keys = fileHandler

[formatters]
keys = simpleFormatter

[logger_root]
level = DEBUG
handlers = fileHandler

[handler_fileHandler]
class = FileHandler
level = DEBUG
formatter = simpleFormatter
args = ("/path/to/log/file.log", )

[formatter_simpleFormatter]
format = % (asctime) s % (name) s - % (levelname) s: % (message) s


Trending Technology 


Most popular in python