Add local Botkit CMS plugin to load a local scripts file

[etiennellipse]

This adds a version of the Botkit CMS plugin that works by loading a local scripts file, without the need to run a Botkit CMS web app. It fixes howdyai/botkit-cms#24, providing a replacement for this with Botkit 4.5+.

I decided to put this in the botkit-plugin-cms package, this way both plugins expose the exact same API and share the same logic when relevant.

To use this in your bot, just:

• Install botkit-cms inside your bot project: npm install botkit-cms --save
• Add this snippet to your bot startup script:

import { BotkitCmsLocalPlugin } from "botkit-plugin-cms";
const cms = require("botkit-cms")();

// Load the plugin
controller.usePlugin(new BotkitCmsLocalPlugin({
    cms,
    path: `${__dirname}/../scripts.json`,
}));

controller.ready(() => {
    // Same handler as for the existing CMS plugin which start dialog when a trigger is hit
    controller.on("message", async (bot, message) => {
        const results = await controller.plugins.cms.testTrigger(bot, message);
        return results === false;
    });
});

The existing hooks all works with this too; before, after and onChange, the exact same way documented here.

I also brought back some the methods exposed by the existing system: getScripts, getScript and getScriptById.

I would like some feedback on it. If it seems acceptable, I will:

• Update the botkit-plugin-cms README with how to get up and running with this
• Send a PR to remove botkit_mutagen.js from botkit-cms, since it does not work anymore with Botkit 4 and update the README as well to point to this package

[maximebeaudoin]

Yes please !

[benbrown]

I will review ASAP.

[etiennellipse]

@benbrown Great! I fiddled a bit with the naming and refactored to make some parts reusable, so if you have any preference to keep things more consistent just let me know.

[InnoraG]

@etiennellipse , is it supposed to be backward compatible or is it changing the behavior of the previous uselocalstudio features ?
Because uselocalstudio features is working well finally...

[etiennellipse]

@InnoraG It's actually much better ;-)

For me, useLocalStudio did not work at all using Botkit 4.6 and the latest Botkit CMS. I was getting the errors described in howdyai/botkit-cms#24 (and then some more).

The old API exposed by useLocalStudio is: https://github.com/howdyai/botkit-cms/blob/master/src/botkit_mutagen.js

The one exposed by my PR combines the methods exposed by https://github.com/howdyai/botkit/pull/1851/files#diff-e54cf01839fc7d00bea99a8fc031c461 and the before, after and onChange events described in https://github.com/howdyai/botkit/pull/1851/files#diff-4ed859ea1c3038679871d0807ae3ba47.

So no, it is not the exact same; you can see that the evaluateTrigger signature is different; although mine is simpler and I think the return value is the same. This should make the change easy. I cannot guarantee it behaves the same though, since the old simply does not work.

My focus was actually to make it work exactly like the API of the "regular" Botkit CMS plugin. This makes switching from one deployment model to the other seamless.

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[etiennellipse]

It's not stale. I'll fix the conflicts.

@benbrown Did you have any time to look at it? Any comments?

[InnoraG]

Team, this was my native working solution fixing howdyai/botkit-cms#24 without any change in lib, works in latest version 4.6+

Tricks is to deactivate botkit-plugin-cms one time initial fetching since scripts file is local :

const { BotkitCMSHelper } = require(`botkit-plugin-cms`);
const scripts = require(`./cms-scripts/scripts.json`);
    // Mutation of apiRequest for local script usage
    BotkitCMSHelper.prototype.apiRequest = function apiRequest() {
      return new Promise(resolve => {
        resolve(scripts);
      });
    };


  controller.usePlugin(
    new BotkitCMSHelper({
      uri: process.env.cms_uri, // "http://localhost:3000/"
      token: process.env.cms_token, // "dontguessit"
    })
  );

Then load into DialogSet using native botkit-cms loadScriptsFromFile, same process/function is used by botkit-cms to render scripts file on its GUI :

  const localcms = require(`botkit-cms`)();

  localcms.useLocalStudio = function useLocalStudio(botkit) {
    const mutagen = require(`./own_botkit_mutagen.js`); // .studio. replaced with .plugins.cms.
    return mutagen(localcms, botkit);
  };
  localcms.useLocalStudio(controller);
  localcms.loadScriptsFromFile(`./cms-scripts/scripts.json`);

Usage is same :

controller.ready(() => {
    // Same handler as for the existing CMS plugin which start dialog when a trigger is hit
    controller.on("message", async (bot, message) => {
        const results = await controller.plugins.cms.testTrigger(bot, message);
        return results === false;
    });
});

It appears keeping handling of local scripts to botkit-cms itself instead of botkit-plugin-cms is future proof since will natively sync and evolve with the cms...
This also has no required modification to any libs

[etiennellipse]

@InnoraG It's a great workaround, and I thank you for it. I started with your solution; in fact I wanted to apply it so that everybody could use it easily. I tried to modify the CMS to have something that works natively with Botkit 4.5+. While doing so, I discovered that the botkit-cms-plugin does some magic of its own, like binding events and such, which would have been lost. Also, modifying the CMS itself would impact backward-compatibility, so it would have needed to be mitigated somehow; either by having the major version bumped, or by creating 2 "mutagen" implementations that support different botkit versions...

What I am offering here is a working plugin that anybody can integrate within their code in 12 lines, almost the same way they would integrate with the Botkit CMS API, and that makes sure both implementations are aligned. Also, by bringing the code in the botkit repository, we make sure it stays aligned with botkit itself. It's easy to use, the API is consistent and it does not impact the CMS itself. If the CMS format or API changes, the existing plugin (the one calling the web API) will need to change, so will this one. This way, they are bundled together, which makes a lot of sense.

[benbrown]

Thank you for your continued efforts on this @etiennellipse.

I am going to try to get this validated and merged into the next release, and if not, a quick follow up release.

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[etiennellipse]

Not stale! Still waiting for review tho ;-)

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[vaustymenko]

@benbrown, have you had a chance to review this one?

Our team is evaluating usage of botkit but I don't see how can we use it if we can't load cms content in production from files. Search for loading local files eventually leads to this thread. Maybe I'm missing something.

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[etiennellipse]

Not stale! @benbrown, any comments?

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[etiennellipse]

:-(

[maximebeaudoin]

Bring back this go !

[vasani-arpit]

@InnoraG, Can you elaborate more on your solution? I tried to follow what you have mentioned. did not worked for me.

I want to run botkit using json file without having bot-cms running.

based on what I have understood. by mutating apiRequest function it will solve the getScripts()'s problem. But what about evaluateTrigger()? It is being called when following from bot.js is being executed

controller.on('message,direct_message', async (bot, message) => {
        let results = false;
        results = await controller.plugins.cms.testTrigger(bot, message); // <-- this line calls  evaluateTrigger()
        console.log(results)
        return results === false
});

Thanks in advance.

[InnoraG]

Hi @vasani-arpit & team

Sorry for bit delay
Attached is minimal working setup with local script.json file
minimal_botkit+cms.zip

-extract in root folder of a working bot
-put script in ./cms-scripts/scripts.json
-run node .\bot_copy.js
-test with emulator

Below is startup flow showing botkit:cms Dialogs loaded from Botkit CMS plus emulator first conversationUpdate activity :

Please test and feedback

[vasani-arpit]

Hi @InnoraG Thanks for the reply. I tried your solution ran into error in BotFrameworkAdapter.processActivity(): 500 ERROR
can you provide me your email? or can you reach out to me on [email protected] ? I have few things which I need to run by you. I will not take too much of your time and I am willing to pay for your time you spend helping me with my requirements.

[jpfcabral]

Any update to this issue on v4.15? I still trying to use local scripts.json on botkit