Real-Time Logging for Embedded Systems and IoT

Logging is the way to monitor the state of any machine. As per analytics, there are 21.5 billion interconnected devices in the world which are growing by 9% every year. Since each device is smart enough to perform its targeted application, what about the failure of those devices? Is there any monitoring system of that device? This leads to logging information in the proper manner.

What is logging?

Logging is capturing real-time updates of any system and storing or properly delivering that information. Another question arises: How is logging generated and how do we access it?

Mostly all Embedded systems are comprised of a Linux kernel and the system is programmed to store all real-time updates to its logging directory.

logging in the Linux system — Logging directory in Linux system

As shown above, there are a lot of log files there. The system stores them to capture all updates into files; a different application may be programmed to store their logging info to their separate directory.

Way to store logging information

Syslog is the general standard for logging systems and program messages in the Linux environment. This service constitutes the system log daemon, where any program can do its logging (debug, security, normal operation) along with the Linux kernel messages.

#include <syslog.h>

Whenever any log is sent to the system, it also needs additional information like who has sent that log and what kind of log info it is, what kind of program is logging the message.

Types of messages based on program

LOG_AUTH
    security/authorization messages
LOG_AUTHPRIV
    security/authorization messages (private)
LOG_CRON
    clock daemon (cron and at)
LOG_DAEMON
    system daemons without separate facility value
LOG_FTP
    ftp daemon
LOG_KERN
    kernel messages (these can't be generated from user processes)
LOG_LOCAL0 through LOG_LOCAL7
    reserved for local use
LOG_LPR
    line printer subsystem
LOG_MAIL
    mail subsystem
LOG_NEWS
    USENET news subsystem
LOG_SYSLOG
    messages generated internally by syslogd(8)
LOG_USER (default)
    generic user-level messages
LOG_UUCP
    UUCP subsystem

Types of messages based on level

LOG_EMERG
    system is unusable
LOG_ALERT
    action must be taken immediately
LOG_CRIT
    critical conditions
LOG_ERR
    error conditions
LOG_WARNING
    warning conditions
LOG_NOTICE
    normal, but significant, condition
LOG_INFO
    informational message
LOG_DEBUG
    debug-level message

Other options used along with logging

LOG_CONS
    Write directly to the system console if there is an error while sending to the system logger.
LOG_NDELAY
    Open the connection immediately.
LOG_NOWAIT
    Don't wait for child processes that may have been created while logging the message.
LOG_ODELAY
    The converse of LOG_NDELAY; opening of the connection is delayed until syslog() is called. (Default)
LOG_PERROR
    Also log the message to stderr.
LOG_PID
    Include the caller's PID with each message.

Application

Logging information by default consists of Timestamp, User information, program name, then logging information — although it can be customized. Below is an example that will store log within the system log. It will store two pieces of information:
1. System information
2. Hello World from Main!

<time_stamp> <user_name> <program_name>: <logging information>

#include <stdlib.h>
#include <stdio.h>
#include <syslog.h>
/**
*
*           https://errbits.com
*           Subscribe for exciting stuff!
*
**/
int main (int argc, char *argv[])
{
   //Buffer to store system name
   char str[100];
   //pipe stream to or from a process
   FILE *cmd = popen("uname -a", "r");
   //read output of cmd and store
   while(fgets(str, 100, cmd)!=NULL)
   pclose(cmd);
   //system information
   syslog(LOG_INFO,"%s",str);
   //custom message
   syslog(LOG_DEBUG,"Hello World from Main!");
   closelog();
}

syslog logging in the Linux system — syslog output