Python object macro support

[kirsle]

It'd be nice if rivescript-go were able to parse and run Python object macros for RiveScript bots, by using the Python C API.

There are two projects I found so far: sbinet/go-python and qur/gopy. They both only support Python 2 so far, but that will work for now.

I did some experimenting and came up with the following Go code that demonstrates the key pieces of functionality needed: dynamically parse a Python function, call the function giving it an array of string arguments, and retrieve the result of the function as a Go string.

package main

import (
    "fmt"
    "github.com/sbinet/go-python"
)

func init() {
    err := python.Initialize()
    if err != nil {
        panic(err.Error())
    }
}

func main() {
    // The source code of the python function we wanna be able to call.
    pycode := `
def test(rs, args):
    print "Test works"
    return "Forwards: {}\nBackwards: {}".format(
        " ".join(args),
        " ".join(args[::-1]),
    )
`

    // The []string to use as the 'args' param to `def test()`
    args := StringList_ToPython("Hello", "world")
    defer args.DecRef() // Always do this so Python can count references well.

    // To load the function you can simply eval the code in the global scope:
    python.PyRun_SimpleString(pycode)

    // Get the main module's dictionary so we can get a reference to our
    // function back out of it.
    main_module   := python.PyImport_AddModule("__main__")
    main_dict     := python.PyModule_GetDict(main_module)
    test_function := python.PyDict_GetItemString(main_dict, "test")

    // The tuple of (rs, args) arguments to pass to the function.
    // This tuple is the *args in Python lingo.
    test_args := python.PyTuple_New(2)
    defer test_args.DecRef()
    python.PyTuple_SetItem(test_args, 0, python.Py_None)
    python.PyTuple_SetItem(test_args, 1, args)

    // Call the actual Python function now. Functions return a *PyObject, and
    // we can cast it back to a string.
    returned := test_function.CallObject(test_args)
    result := python.PyString_AsString(returned)

    // Print the result of the function.
    fmt.Println("Result:", result)
}

// StringList_ToPython is a helper function that simply converts a Go []string
// into a Python List of the same length with the same contents.
func StringList_ToPython(items... string) *python.PyObject {
    list := python.PyList_New(len(items))

    for i, item := range items {
        python.PyList_SetItem(list, i, python.PyString_FromString(item))
    }

    return list
}

This code lets us:

• Dynamically provide Python source for a function definition.
• Prepare the Python *args tuple, converting Go strings into Python strings for the args argument (which is a list(str) type)
• Call the function and gets its (string) result.

The other huge TODO is the rs parameter to the Python function: the above code sends in a NoneType, but IRL it would need to provide an object with at least a subset of the RiveScript API (most importantly, functions like rs.current_user() and rs.set_uservar() & friends). I imagine to expose the full RiveScript API to Python I'd need to write a whole wrapper class that translates all the arguments to/from Python types, but I'll probably just focus on the aforementioned useful functions to start out with.

[rmasci]

Wow. This is old but I overcame this issue five years ago -- sorry I didn't see this until now. My rivescript looks like this:

+ tell me what time it is
- LocalCommand "date"

+ run a custom script to get * from the database
- LocalCommand /var/tmp/bin/mycustomscript -u root -d Opetion1 -m <star1>

When the chatbot sees the first word of the response is a LocalCommand (You can change the name of this keyword in the settings file). It knows to exec everything after this. THis can be easily executed a better way than I did originally by using github.com/bitfield script:
<determine if firstword is 'LocalCommand' then run:

cmd := strings.Split(returnMessage, " ")[1:] // get everything after LocalCommand
returnMessage ,err:= script.Exec(cmd).String()

This allows my chatbot users to create commands in ANY language they prefer, or use standard Linux commands. I also have a version of this for 'RemoteCommand' that executes them on other servers using SSH.

[kirsle]

@rmasci I've done similar before, like this example to run Perl object macros for the Python version of RiveScript: https://github.com/aichaos/rivescript-python/tree/master/eg/perl-objects

The various RiveScript versions have a SetHandler() function to register custom programming language hooks for non-native languages (up to the programmer to implement how those hooks work). The Python module also has a JavaScript example for bots that run out of a web browser environment (so the JS macros are run using embedded <script> tags in the user's own web browser): https://github.com/aichaos/rivescript-python/tree/master/eg/js-objects

Being that Go sits at a close level to C and can embed CPython (or Perl or other C-based languages) it may be possible to get a wide variety of languages "natively" supported without shell commands calling out to third party scripts to bridge the gap. Though on that latter idea, I was once playing with the idea of defining a 'standard' interface for all RiveScript libraries to be able to interact with any third party language (e.g., by a standard format for input/output similar to the CGI standard for web scripts) -- with the idea that hacks like that perl-objects example didn't need to be done in an ad-hoc basis by individual developers but that somebody could take a Perl wrapper (e.g.) and use it on any of the 5 RiveScript libraries with built-in support for the protocol.

In more recent years I have some further ideas that could be used:

• Google's Starlark language is a Python-like syntax implemented in native Go: https://github.com/google/starlark-go It's not exactly Python but is familiar enough for users who would like to program object macros in Python.
• Traefik has developed an interpreted version of Go: https://github.com/traefik/yaegi Currently RiveScript-go can't have > object * go in-line macros because Go can't evaluate itself at runtime and Yaegi seems to be at least a 99% compatible Go interpreted language which could support in-line Go object macros.
• Goja is a better JavaScript interpreter for Go than otto is and supports a lot of ES6 syntax: https://github.com/dop251/goja RiveScript-Go currently uses otto for its JavaScript interpreter, but it's only old ES5 syntax and quirky at that (not all ES5 compatible code works with otto - try changing the type of a variable and you get a runtime exception, try initializing a variable as null and then assigning a type later, etc.).