o.a.s.t.ShellBolt log level is always INFO

[mstrucken]

Example bolt written in Python (multilang/resources/log_level.py):

import asyncio
import storm

class LogLevelTestBolt(storm.BasicBolt):
    async def initialize(self, conf, context):
        storm.logInfo("Python bolt starting...")

        storm.logWarn("This is a sample warning")

    async def process(self, tup):
        storm.logError("Error processing tuple with python...")

async def main():
    bolt = LogLevelTestBolt()
    await bolt.run()

asyncio.run(main())

A minimal bolt that extends the ShellBolt used to execute the Python bolt above:

package com.stockpulse.stockstorm.logging;

import org.apache.storm.task.ShellBolt;
import org.apache.storm.topology.IRichBolt;
import org.apache.storm.topology.OutputFieldsDeclarer;

public class PythonLogLevelBolt extends ShellBolt implements IRichBolt {
    public PythonLogLevelBolt() {
        super("python", "log_level.py");
    }

    @Override
    public void declareOutputFields(OutputFieldsDeclarer declarer) {

    }

    @Override
    public java.util.Map<String, Object> getComponentConfiguration() {
        return null;
    }
}

Output:

16:30:51.806 [main] WARN  o.a.s.s.o.a.z.s.ServerCnxnFactory - maxCnxns is not configured, using default value 0.
16:30:57.247 [Thread-37-python-logging-executor[2, 2]] INFO  o.a.s.t.ShellBolt - Launched subprocess with pid 17336
16:30:57.248 [Thread-37-python-logging-executor[2, 2]] INFO  o.a.s.t.ShellBolt - Start checking heartbeat...
16:30:57.248 [Thread-39] INFO  o.a.s.t.ShellBolt - ShellLog pid:17336, name:python-logging Python bolt starting...
16:30:57.248 [Thread-39] INFO  o.a.s.t.ShellBolt - ShellLog pid:17336, name:python-logging This is a sample warning
16:30:57.251 [Thread-39] INFO  o.a.s.t.ShellBolt - ShellLog pid:17336, name:python-logging Error processing tuple with python...

Expected output:

16:30:51.806 [main] WARN  o.a.s.s.o.a.z.s.ServerCnxnFactory - maxCnxns is not configured, using default value 0.
16:30:57.247 [Thread-37-python-logging-executor[2, 2]] INFO  o.a.s.t.ShellBolt - Launched subprocess with pid 17336
16:30:57.248 [Thread-37-python-logging-executor[2, 2]] INFO  o.a.s.t.ShellBolt - Start checking heartbeat...
16:30:57.248 [Thread-39] INFO  o.a.s.t.ShellBolt - ShellLog pid:17336, name:python-logging Python bolt starting...
16:30:57.248 [Thread-39] WARN  o.a.s.t.ShellBolt - ShellLog pid:17336, name:python-logging This is a sample warning
16:30:57.251 [Thread-39] ERROR o.a.s.t.ShellBolt - ShellLog pid:17336, name:python-logging Error processing tuple with python...

[mstrucken]

The essential code from multilang/resources/storm.py:

def sendMsgToParent(msg):
    print(json_encode(msg))
    print("end")
    sys.stdout.flush()

def log(msg, level=2):
    sendMsgToParent({"command": "log", "msg": msg, "level":level})

def logTrace(msg):
    log(msg, 0)

def logDebug(msg):
    log(msg, 1)

def logInfo(msg):
    log(msg, 2)

def logWarn(msg):
    log(msg, 3)

def logError(msg):
    log(msg, 4)

The json encoded messages:

{"command": "log", "msg": "Python bolt starting...", "level": 2}
{"command": "log", "msg": "This is a sample warning", "level": 3}
{"command": "log", "msg": "Error processing tuple with python...", "level": 4}

A single message is deserialized using org.apache.storm.multilang.JsonSerializer:

    public ShellMsg readShellMsg() throws IOException, NoOutputException {
        JSONObject msg = (JSONObject)this.readMessage();
        ShellMsg shellMsg = new ShellMsg();
        String command = (String)msg.get("command");
		
	// ...	
		
        if (command.equals("log")) {
            Object logLevelObj = msg.get("level");
            if (logLevelObj != null && logLevelObj instanceof Long) {
                long logLevel = (Long)logLevelObj;
                shellMsg.setLogLevel((int)logLevel);
            }
        }

        return shellMsg;
    }

But logLevelObj is not an instance of Long but an instance of Integer.

[rzo1]

Thanks for the report. Do you want to create a PR for it?

[mstrucken]

To be a little more relaxed about the deserialized numeric type, the check could be changed as follows:

if (command.equals("log")) {
	Object logLevelObj = msg.get("level");
	if (logLevelObj != null && logLevelObj instanceof Number) {
		int logLevel = ((Number) logLevelObj).intValue();
		shellMsg.setLogLevel(logLevel);
	}
}

[mstrucken]

Will create a PR for this, @rzo1