how to capture command-line output with python logging?

# You can use following commands to run any shell command.I have used them on ubuntu.

import os
os.popen('your command here').read()

# Note: This is deprecated since python 2.6.Now you must use subprocess.Popen.Below is the example

import subprocess

p = subprocess.Popen("Your command", shell = True, stdout = subprocess.PIPE, stderr = subprocess.PIPE).communicate()[0]
print p.split("\n")

import subprocess
process = subprocess.Popen(['echo', 'More output'],
   stdout = subprocess.PIPE,
   stderr = subprocess.PIPE)
stdout, stderr = process.communicate()
stdout, stderr

>>> subprocess.check_output(['ls', '-l'])
b 'total 0\n-rw-r--r--  1 memyself  staff  0 Mar 14 11:04 files\n'

from subprocess
import Popen, PIPE
path = "echo"
pipe = Popen(path, stdout = PIPE)
text, err = pipe.communicate()
if (pipe.returncode == 0)
   print(text, err)

Suggestion : 2

Display console output for ordinary usage of a command line script or program,Report events that occur during normal operation of a program (e.g. for status monitoring or fault investigation),Report suppression of an error without raising an exception (e.g. error handler in a long-running server process),logging.error(), logging.exception() or logging.critical() as appropriate for the specific error and application domain

import logging
logging.warning('Watch out!') # will print a message to the console
logging.info('I told you so') # will not print anything

WARNING: root: Watch out!

import logging
logging.basicConfig(filename = 'example.log', encoding = 'utf-8', level = logging.DEBUG)
logging.debug('This message should go to the log file')
logging.info('So should this')
logging.warning('And this, too')
logging.error('And non-ASCII stuff, too, like Øresund and Malmö')

DEBUG: root: This message should go to the log file
INFO: root: So should this
WARNING: root: And this, too
ERROR: root: And non - ASCII stuff, too, like Øresund and Malmö

--log = INFO

getattr(logging, loglevel.upper())

Suggestion : 3

To output logs from your function code, you can use the print method, or any logging library that writes to stdout or stderr. The following example logs the values of environment variables and the event object.,In the same command prompt, use the base64 utility to decode the logs. The following example shows how to retrieve base64-encoded logs for my-function.,This page describes how to produce log output from your Lambda function's code, or access logs using the AWS Command Line Interface, the Lambda console, or the CloudWatch console.,The AWS CLI is an open-source tool that enables you to interact with AWS services using commands in your command line shell. To complete the steps in this section, you must have the following:

import os

def lambda_handler(event, context):
   print('## ENVIRONMENT VARIABLES')
print(os.environ)
print('## EVENT')
print(event)

START RequestId: 8 f507cfc - xmpl - 4697 - b07a - ac58fc914c95 Version: $LATEST
# # ENVIRONMENT VARIABLES
environ({
   'AWS_LAMBDA_LOG_GROUP_NAME': '/aws/lambda/my-function',
   'AWS_LAMBDA_LOG_STREAM_NAME': '2020/01/31/[$LATEST]3893xmpl7fac4485b47bb75b671a283c',
   'AWS_LAMBDA_FUNCTION_NAME': 'my-function',
   ...
})
# # EVENT {
   'key': 'value'
}
END RequestId: 8 f507cfc - xmpl - 4697 - b07a - ac58fc914c95
REPORT RequestId: 8 f507cfc - xmpl - 4697 - b07a - ac58fc914c95 Duration: 15.74 ms Billed Duration: 16 ms Memory Size: 128 MB Max Memory Used: 56 MB Init Duration: 130.49 ms
XRAY TraceId: 1 - 5e34 a614 - 10 bdxmplf1fb44f07bc535a1 SegmentId: 07 f5xmpl2d1f6f85 Sampled: true

The following example shows how to retrieve a log ID from the LogResult field for a function named my-function.

aws lambda invoke-- function -name my - function out--log - type Tail

You should see the following output:

{
   "StatusCode": 200,
   "LogResult": "U1RBUlQgUmVxdWVzdElkOiA4N2QwNDRiOC1mMTU0LTExZTgtOGNkYS0yOTc0YzVlNGZiMjEgVmVyc2lvb...",
   "ExecutedVersion": "$LATEST"
}

In the same command prompt, use the base64 utility to decode the logs. The following example shows how to retrieve base64-encoded logs for my-function.

aws lambda invoke-- function -name my - function out--log - type Tail\
   --query 'LogResult'--output text | base64 - d

You should see the following output:

START RequestId: 57 f231fb - 1730 - 4395 - 85 cb - 4 f71bd2b87b8 Version: $LATEST "AWS_SESSION_TOKEN": "AgoJb3JpZ2luX2VjELj...", "_X_AMZN_TRACE_ID": "Root=1-5d02e5ca-f5792818b6fe8368e5b51d50;Parent=191db58857df8395;Sampled=0"
",ask/lib:/opt/lib",
END RequestId: 57 f231fb - 1730 - 4395 - 85 cb - 4 f71bd2b87b8
REPORT RequestId: 57 f231fb - 1730 - 4395 - 85 cb - 4 f71bd2b87b8 Duration: 79.67 ms Billed Duration: 80 ms Memory Size: 128 MB Max Memory Used: 73 MB

The cli-binary-format option is required if you're using AWS CLI version 2. To make this the default setting, run aws configure set cli-binary-format raw-in-base64-out. For more information, see AWS CLI supported global command line options.

#!/bin/bash

aws lambda invoke-- function -name my - function --cli - binary - format raw - in - base64 - out--payload '{"key": "value"}'
out
sed - i '' - e 's/"//g'
out
sleep 15
aws logs get - log - events--log - group - name / aws / lambda / my - function --log - stream - name stream1--limit 5

In the same command prompt, macOS and Linux users may need to run the following command to ensure the script is executable.

chmod - R 755 get - logs.sh

Suggestion : 4

Setting capturing methods or disabling capturing,How to capture stdout/stderr output Default stdout/stderr/stdin capturing behaviour Setting capturing methods or disabling capturing Using print statements for debugging Accessing captured output from a test function ,You can influence output capturing mechanisms from the command line:,To temporarily disable capture within a test, both capsys and capfd have a disabled() method that can be used as a context manager, disabling capture inside the with block:

pytest - s # disable all capturing
pytest--capture = sys # replace sys.stdout / stderr with in -mem files
pytest--capture = fd # also point filedescriptors 1 and 2 to temp file
pytest--capture = tee - sys # combines 'sys'
and '-s', capturing sys.stdout / stderr
# and passing it along to the actual sys.stdout / stderr

# content of test_module.py

def setup_function(function):
   print("setting up", function)

def test_func1():
   assert True

def test_func2():
   assert False

$ pytest
=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-7.x.y, pluggy-1.x.y
rootdir: /home/sweet/project
collected 2 items

test_module.py .F [100%]

================================= FAILURES =================================
________________________________ test_func2 ________________________________

def test_func2():
> assert False
E assert False

test_module.py:12: AssertionError
-------------------------- Captured stdout setup ---------------------------
setting up <function test_func2 at 0xdeadbeef0001>
   ========================= short test summary info ==========================
   FAILED test_module.py::test_func2 - assert False
   ======================= 1 failed, 1 passed in 0.12s ========================

def test_myoutput(capsys): # or use "capfd"
for fd - level
print("hello")
sys.stderr.write("world\n")
captured = capsys.readouterr()
assert captured.out == "hello\n"
assert captured.err == "world\n"
print("next")
captured = capsys.readouterr()
assert captured.out == "next\n"

def test_disabling_capturing(capsys):
   print("this output is captured")
with capsys.disabled():
   print("output not captured, going directly to sys.stdout")
print("this output is also captured")