Use Application Insights for Python with Custom Log Levels

Today we want to have a look at Python and Application Insights to centralize and unify our log streams.
Furthermore, we will talk about how to create custom log levels and ensure the correct log severity in the Application Insights portal.

Introduction

While working with Python you will reach the point to think about your logging infrastructure.
In the beginning, a simple console logging will be sufficient but soon you will hit a wall.
For example, when logging your Azure ML Model Deployment.

Application Insights, provided by Azure, is an great log aggregator as it makes collection logs, traces and exceptions a breeze.
Additionally, you get a rich dashboard and query capabilities for free and don’t have to reinvent the wheel.

We will cover the following general topics today:

• Perquisites for logging with Python to Application Insights
• How to log to Application Insights

Also, a bit more exotic ones will be covered:

• How to introduce a custom log level
• How to ensure that the correct log severity is sent to Application Insights

Let’s dive right into it!

Create Application Insights in Azure

At first, you need an actual Application Insights instance in your Azure subscription.
Please refer to the official documentation to set things up.

Setup Application Insights SDK for Python

The SDK by Microsoft is no longer maintained, but they recommend using the SDK provided by Open Census.

This package is easily installed using pip:

pip install opencensus-ext-azure

Initialize the logger module

Now we are ready to initialize our first logger module, which will log straight to Azure!
Nothing special here unless the instantiation of the AzureLogHandler and the call to logger.addHandler(handler).

Your application insights key can be obtained by navigating to our application insights resource.
You will find your key at the top right of the initial screen.

import logging from logging 
import Logger from opencensus.ext.azure.log_exporter 
import AzureLogHandler 

application_insights_connection_string = 'YOUR_APPLICATION_INSIGHTS_CONNECTION_STRING' 
handler = AzureLogHandler(connection_string=application_insights_connection_string) 
logger = logging.getLogger() 
logger.addHandler(handler)

Log your first traces

Now you can use your newly instantiated logger object to log traces using the familiar python logging interface.

logger.info('info') 
logger.warning('warning')

For an in depth guide please refer to the corresponding section in the Python documentation.

Log your first exception

Hopefully, you will never need this but in case 😉

try:
  ...
except Exception:
   logger.exception('Exception')

Keep in mind, that logger.exception expects to be called in an exception handler.

The need for custom log levels

Looking at the python documentation for logging, we recognize the following predefined log levels:

• NOTSET
• DEBUG
• INFO
• WARNING
• ERROR
• CRITICAL

So, if you want to log what’s going on in your application, which level would you choose?
Personally, I would tend to INFO.

After running the application for some time, you may visit your Application Insights portal.
There you may be surprised by the amount log messages.
This flood is caused by python packages which log (in my opinion) way to verbose to the INFO log level like azure-storage-queue.

The obvious solution would be to just log at the WARNING level, what leaves a bitter aftertaste for me.

But there’s an better solution.

Introduce a custom log level

Let’s introduce a custom log level and call it APPLICATION.

To archive this, we need the following ingredients:

• an integer representation of our APPLICATION log level
• an string representation of our APPLICATION log level
• configure our logger to suppress log messages with a lower level than APPLICATION

This can be archived with a few lines of python:

application_log_level = logging.INFO + 1 
logging.addLevelName(application_log_level, 'APPLICATION') 
logger.setLevel(application_log_level)

To ease the interface for our new log level, we will add an application() method to the logger object:

def log_application_message(self, message, *args, **kwargs):
  if self.isEnabledFor(application_log_level):
    self._log(application_log_level, message, args, **kwargs)

logging.Logger.application = log_application_message

Now we can utilize our new log level conveniently:

logger.application('I have an higher log level then INFO')

Messed up severity level in Application Insights

Everything’s fine now? Unfortunately not.

Navigating to your Application Insights portal, you will find your APPLICATION level messages.
The level field of the custom properties section will be set correctly.

But the severity level in Application Insights will be messed up:
Instead of APPLICATION, it will be categorized as warning!

This will be quite annoying, if you really want to find some warnings.

So we need a way to:

• send only our APPLICATION messages (and higher levels) to Application Insights
• block the noise generated by third party python packages
• ensure the right severity (Information) in Application Insights

Filters to the rescue

The python logging module allows to hook up filters in the processing chain.
These are normally reserved to filter specific log messages (for example, based on an string comparison).

So, this mechanism could also be used to get rid of the generated noise but shows the following downsides:

• for each log message, a string comparison must be executed
• you must know each noise message in advance or update the filter iteratively.

Personally, this is not something i would like to do.

Fortunately, filters can also manipulate log messages (contextual filters).
Of course, this should be done with care.

But for our issue, this is quite helpful.

We will add a filter, which resets the log level to Information before sending to Azure.
This will ensure the correct severity level in the Application Insights Portal.

This filter must be an class which implements a filter() method and inherits from logging.Filter, as shown below:

class ApplicationInsightsFilter(logging.Filter):
    def filter(self, record):
        if record.levelno == application_log_level:
            record.levelno = logging.INFO

        return True

Notify the return value of True to ensure, that all log messages are passed through.

To register the filter to our logging chain, we only need a call to addFilter()

handler.addFilter(ApplicationInsightsFilter())

Summary

Wrapping up, we learned how to

• set up a basic logging infrastructure using Application Insights
• filter out the noise generated by third party packages
• ensure the correct severity level in Application Insights

I hope, you learned something new and thank you for reading!