Streaming io.Connect Desktop Logs to a Remote Server (Splunk) with a Custom log4js UDP Appender

io.Connect Desktop
,
1

The example below comes from a pre-10.0 setup and uses the logger.json file - see the section for 10.0 and later at the end of this article.

The io.Connect Desktop logging mechanism is based on log4js-node, and its appenders and categories are defined in an external JSON configuration file. This means you can add your own log appender to forward io.Connect logs to a remote collector such as Splunk, without a change to io.Connect Desktop itself.

Q: Is there a built-in Splunk integration?

No. There is no Splunk-specific appender shipped with io.Connect Desktop. You forward logs to Splunk by adding a custom log4js appender to the io.Connect Desktop logging configuration. More about logs here: Understanding logs in io.Connect

Q: How do I add a custom UDP appender that forwards logs to Splunk?

The setup below wraps the log4js-udp-appender package so that you can add your own logic before the event is sent.

  1. npm install log4js-udp-appender in the folder where you are hosting it e.g.: \Desktop\assets\log4js-udp-appender.

  2. Set an index.js file in the same folder with the following code:

const createUdpAppender = require("log4js-udp-appender");

function customUdpAppender(config, layouts) {
  const udpAppender = createUdpAppender.configure(config, layouts);

  const appender = (loggingEvent) => {
    // Add your custom logic here
    udpAppender(loggingEvent);
  };

  appender.shutdown = (done) => {
    udpAppender.shutdown(done);
  };

  return appender;
}

function configure(config, layouts) {
  // Append the custom appender to the log4js configuration
  config.layout.tokens.user = process.env.USER || process.env.USERNAME || 'unknown';
  return customUdpAppender(config, layouts);
}

exports.configure = configure;
  1. In the logger.json located in Desktop\config, configure your custom appender as needed:
{
    "appenders": {
        "splunk": {
            "type": "%GDDIR%/assets/log4js-udp-appender",
            "host": "127.0.0.1",
            "port": 12345,
            "level": "info",
            "layout": {
                "type": "pattern",
                "tokens": {
                  "user": "%USERNAME%",
                  "appName": "MyApp",
                  "env": "%GLUE-ENV%"
                },
                "pattern":"%d{ISO8601_WITH_TZ_OFFSET} [app=%x{appName}] [env=%x{env}] [user=%x{user}] [host=%h] %p:[%f{1}-%M] %m %n"
            },
            "endMsg": "\t"
        },
        "out": {
            "type": "stdout"
        }
    },
  "categories": {
        "default": {
            "appenders": [
                "splunk"
            ],
            "level": "info"
        }
    }

Note on the environment variables in the example. %GDDIR% and %GLUE-ENV% are legacy environment variables. They are still supported, but if running io.Connect Desktop 10.0 please migrate to %IO_CD_ROOT_DIR% and %IO_CD_ENV%. See Configuration — Overview.

Q: Which logs get forwarded?

The logging configuration has two parts: appenders (where log entries go) and categories (the different logs). Each category can have one or more appenders. You can either add your custom appender to an existing category’s appender list, or replace the appenders of a category entirely.

Q: Can I forward the logs of one specific app only?

Yes. Since io.Connect Desktop 9.5, you can define a separate log appender per app. Define the appender, then create a category named after your app and assign the appender to it.

{
    "appenders": {
        "my-app-appender": {
            "type": "file",
            "filename": "%GLUE-USER-DATA%/logs/my-app.log",
            "maxLogSize": 10485760,
            "backups": 5,
            "minLevel": "info",
            "keepFileExt": true,
            "compress": true
        }
    }
}

{
    "categories": {
        "my-app": {
            "appenders": [
                "my-app-appender"
            ],
            "level": "trace"
        }
    }
}

Q: How do I control the log level of what is forwarded?

Set the level property on the category to control its logging threshold. The UDP example above also sets "level": "info" on the appender.

Q: Where is the format of what is sent coming from?

The format is controlled by the log4js layout pattern and tokens on the appender. This is standard log4js, not io.Connect-specific: see the log4js Layouts and Writing Appenders documentation.

io.Connect Desktop 10.0 and later

The example above uses logger.json, which is the pre-10.0 location of the logging configuration.

  • logger.json is deprecated as of io.Connect Desktop 10.0. Custom log appenders and categories are now defined under the "configuration" property of the "logging" top-level key in system.json:
{
    "logging": {
        "configuration": {
            "appenders": {},
            "categories": {}
        }
    }
}

See the io.Connect Desktop 10.0 changelog.

Alternative on 10.1 and later: io.Insights log publishing

Since io.Connect Desktop 10.1, the Logs module of io.Insights bridges the existing io.Connect logging mechanisms with the OpenTelemetry log exporter, so platform and client app logs can be published as standard OpenTelemetry log records to an OpenTelemetry-compatible backend. See io.Insights — Logs.

See also