Scripting languages

[BearishSun]

bs::f currently comes with an automated script binding generator that generates bindings for the Mono runtime and outputs C# binding code. This binding generator should be extended to support Lua, Python and Java. Scripting plugins that know how to communicate between bs::f and runtimes for those languages should also be added.

The scripting API might need to be extended so an app can be created fully from a scripting language. Currently the systems allows the user to create custom components, in which a majority of engine functionality is exposed. But initial start-up and window creation is not currently exposed.

An alternative approach to scripting language support is to use the script binding generator to generate SWIG interface, which can then output bindings for a variety of languages. However SWIG only allows for one-way method calls, making some bs::f operations like auto-serialization not possible. However this approach does allow the user to just use bs::f as a plain library in their scripting language.

It remains to be seen which approach is best.

[jonesmz]

I hate myself for suggesting this, but you might consider adding javascript to the list of supported scripting languages. It's hugely popular, as sad as that makes me.

[jayrulez]

@jonesmz is it popular among game developers?

[jonesmz]

@jayrulez No idea! I certainly hope not. But it is hugely popular in the general programming world.

Consider also issue #18 , which includes soft plans to support WebAssembly / Emscripten. Maybe scripting support wouldn't have any influence on that PR, maybe it would.

But keeping Javascript support in mind (Either because it'll be supported, or because it'll explicitly not be) would be meaningful when considering the intersection between this issue and #18

[BearishSun]

The Emscripten route will end up just using JavaScript natively as the entire engine gets converted to js / webasm. So I'd probably wait to see how that goes first, and then evaluate whether we also want javascript as scripting language for non-web as well (most likely yes, but maybe majority of js users just want a web version).

In any case it will certainly be nice to be accessible to such a big user base.

[jeebus3000]

Hi!
Haxe support would be nice.
In a form of hxcpp externs. With a tool like UnrealHxGenerator - https://github.com/proletariatgames/unreal.hx/tree/development/Haxe/BuildTool for example.

[BearishSun]

It sounds interesting. I'll consider it but from a quick look it seems Haxe would require significantly different approach to the languages above, since it cross-compiles to other languages rather than running a virtual machine (but correct me if I'm wrong, I just took a quick look).

bsf relies on communicating with the scripting VM in order to add various functionality like serialization, scanning for dependencies, etc. (basically it needs the language to have some kind of reflection). So if I do decide to go with that approach some core concepts about the API would need to change, which I haven't really had a chance to properly think about.

[jeebus3000]

I think, Haxe externs fit the alternative approach you described. Two-way binding (bfs <-> hxcpp) is also possible with CFFI Prime. Haxe experts give some insights on this -- snowkit/hxcpp-guide#1
Also, they (named experts) came up with a scaffolding template for C++ externs -- https://snowkit.github.io/linc/
If you consider to implement haxe externs, it would be nice if it comply with a proposed structure (for the sake of standardization).

[BearishSun]

I'm not seeing any usage of C++ calling into Haxe in the CFFI/CFFI Prime examples. I did find an example of it being done elsewhere, by passing a callback from Haxe to C++, and then invoking the callback. So I guess it's technically possible. It remains to be seen how flexible that approach is and if there are any limitations to types being passed. I'm a bit wary about potential limitations since it doesn't appear to be something that's commonly used.

[christianclavet]

Currently for me, C# used a main scripting language is fine, as an alternative scripting language, I would use LUA as I used it when working on IRB and when I worked at Gameloft on Dungeon Hunter II. Lua is still used a lot in games. But for now for me, C# is ok, I don't feel the need to try to implement lua.

Checked a look on HAXE, and it could help support lots of Scripting language at once! Don't know if it would integrate well with what is already there...

[severin-lemaignan]

With Blender 2.8 removing altogether the Blender's embedded game engine, I'm one of possibly many people who would certainly give a bsf a good try as a prototyping platform if it could provide good quality Python bindings.

In that respect, being able to create and launch an application directly from Python would be a very welcomed feature.

[jeebus3000]

@christianclavet proletariatgames integrated Haxe with Unreal Engine; they use it in production -- https://www.proletariat.com/our-games/ how cool is that?
@severin-lemaignan did you hear about http://armory3d.org/ (some say it's a perfect replacement for BGE)?

[severin-lemaignan]

@jeebus3000 I know Armory3D quite well actually (Haxe again ;-) ), but we have a massive code base in Python, and after tinkering a bit, I do not quite see how I could efficiently reuse that Python codebase if I were to use Armory...

[BearishSun]

I am going to focus on finishing up C# support first, since that is mostly done already. So it might take me a bit to get around to Python or Lua. Since this seems to be a popular topic, if someone does want to take on the task themselves take a look at the following:

• bsfMono plugin - we need an equivalent for Lua/Python to interact with their runtime. This plugin is used by the auto-generated scripting interop code (still part of the https://github.com/BearishSun/BansheeEngine repo, as SBansheeEngine, but should eventually be moved)
• Script binding generator tool (https://github.com/BearishSun/BansheeSBGen) needs to be updated so it generates bindings in the target language. Currently it generates C#, but it shouldn't be too hard to use that as a basis and add a generator for another language.

[kattkieru]

When you say Python, do you mean at least 3.6+ or are you interested in targeting 2.7x for compatibility with DCCs until VFXPlatform 2020?

[BearishSun]

Undecided at the moment, but since most of the framework tends to focus on newer technologies, quite likely we'll go with the most recent version.

[kattkieru]

Awesome. Do you have any thoughts on how you'd want Python component evaluation would work with the current threaded task system?

[BearishSun]

I'm not that familiar with Python so I'm not sure what you mean by component evaluation.

The task system is currently not exposed through the script API. Most higher level languages have their own functionality for handling threading, so the preference is to use those directly. If needed it can eventually be exposed.

[ethan-tqa]

One alternative that you can consider is Wren: https://github.com/munificent/wren. Otherwise you won't go wrong with Lua honestly.

[BearishSun]

I don't have an ETA for it yet, but I'm hoping I can squeeze it in soon. Hopefully sometime around v1.1 release, which should happen near the end of the year.

[Milerius]

For lua you probably want to look into sol2 -> https://github.com/ThePhD/sol2
For python you probably want to look into pybind11 -> https://github.com/pybind/pybind11
Other scripting language such as ChaiScript -> http://chaiscript.com/

I'm using sol2 in my own engine for registering type in lua environnement -> https://github.com/Milerius/shiva/blob/master/modules/lua/shiva/lua/lua_helpers.hpp#L19

I'm sure that you can do something similar with you're RTTR system

[BearishSun]

sol2 looks nice. If you are ever interested in doing script binding for for bsf let me know.

[Milerius]

I will do it anyway for my game engine, i'm using lua as scripting langage and i plan to use bsf, so i will probably do it and then make a pr when this will work

[Milerius]

@BearishSun Can you explain how work the

automated script binding generator

So i can start to investigate for integrate sol2 and lua bindings ?

[BearishSun]

First grab BansheeSBGen source and build it:

• https://github.com/BearishSun/BansheeSBGen
• The readme includes a short guide on how to do it, optionally you can also take a look a the CI files for steps it uses to build it

You can take a look at Source/CMake/GenerateScriptBindings.cmake on how the tool gets executed. This functionality is currently only used with Banshee as a whole and I have not tried running it on bsf alone. There might be minor tweaks required.

Some script bindings aren't auto-generated and use custom wrappers. Those are provided in the Banshee project under Source/Scripting/SBansheeEngine. This library should be moved to bsf. The auto-generated files from SBGen also end up as part of this project. This code should ideally remain identical regardless of which scripting language is used.

SBansheeEngine must be linked with a particular scripting backend. Currently there is only bsfMono available. Lua backend should be implemented using the same interface as bsfMono.

Finally, BansheeSBGen will need to be modified so it generates Lua code. The tool parses C++ and outputs a set of structures that are then used for generating C++ glue code (i.e. code in SBansheeEngine) and C# code. For Lua the parser and the C++ generator shouldn't need to change. You'll need to add code to replace C# generation with Lua generation, as well as a command-line flag to pick which one to use.

Some code isn't auto-generated. Stuff like GUI I haven't updated yet to use the auto-generator. Certain types like Vector3 are implemented directly in script code for performance reasons. And there are a few classes that are very specific in how they work, that they cannot be easily auto-generated. Those will need to be written manually in Lua (there are other options but those would add to an already complicated task).

It's a big task, but hopefully this gives you some information on where to start.

[blockspacer]

cling based C++ as scripting language / hot code reload
Why? Able to run C++ script in runtime or compile it for max speed ( as in example https://github.com/derofim/cling-cmake )
Also: May be used to improve modules compile time

HOT code reload
possible approaches:

store app state
fix cling undo for files
https://root-forum.cern.ch/t/loading-unloading-class-as-interpreted-macro-in-cling-multiple-times/32976/2

execute cling code to change callbacks & variables
nested cling::Interpreter with multiple cling::MetaProcessor
IDK how to do it, but you can create child cling::Interpreter