Skip to content
This repository has been archived by the owner on Mar 8, 2020. It is now read-only.

Latest commit

 

History

History
191 lines (126 loc) · 8.18 KB

getting-started.md

File metadata and controls

191 lines (126 loc) · 8.18 KB
Raw

Getting Started

Using the online web client

The easiest way to get started with Babelfish is to try the online playground where you can write or paste your code and run the parser to see the generated UAST.

Installing bblfshd locally

After playing with the web client, you will probably want to get Babelfish running locally. The first thing to do for that is to setup and run the bblfshd command. Once the server is running, you can connect to it using any of the available clients.

Prerequisites

Running with Docker (recommended)

The easiest way to run the bblfshd daemon is using Docker. You can run it in one of those modes:

  • Embedded drivers mode, meaning that recommended drivers will be included in the Docker image. Installing more drivers is similar to stateless mode.
  • Stateless mode, meaning that all installed drivers will be wiped out once you remove the container.
  • Stateful mode, using a Docker volume to store part of the container internal filesystem and thus add persistence.

Embedded drivers

For embedded drivers mode, run the following command:

$ docker run -d --name bblfshd --privileged -p 9432:9432 bblfsh/bblfshd:latest-drivers

Stateless mode

For stateless mode, run the following command:

$ docker run -d --name bblfshd --privileged -p 9432:9432 bblfsh/bblfshd:latest

Using a volume

This is an alternative to stateless mode. Depending on your operating system, you can use a Docker volume or a normal local filesystem directory.

Docker volume (Linux and macOS)

First, create a volume with the command:

$ docker volume create bblfshd-cache

Then you can run the daemon with this command:

$ docker run -d --name bblfshd --privileged -p 9432:9432 -v bblfshd-cache:/var/lib/bblfshd bblfsh/bblfshd:latest

Volume mapped to a local directory (Linux only)

In this case, just specify the local directory in the -v parameter when running the daemon:

$ docker run -d --name bblfshd --privileged -p 9432:9432 -v /var/lib/bblfshd:/var/lib/bblfshd bblfsh/bblfshd:latest

Notes on command line parameters to "docker run"

The only mandatory flag is --privileged. bblfshd uses containers itself to run language drivers, which currently requires it to run in privileged mode.

Exposing the port (9432) with -p 9432:9432 makes it easier to connect to the gRPC server from outside the container.

If you are behind an HTTP or HTTPS proxy server, for example in corporate settings, you will need to add the HTTP_PROXY, HTTPS_PROXY, and NO_PROXY environment variables in the docker run command to configure HTTP or HTTPS proxy behavior.

$ docker run -d --name bblfshd --privileged -p 9432:9432 -e
HTTP_PROXY="http://proxy.example.com:80/" -v bblfshd-cache:/var/lib/bblfshd bblfsh/bblfshd

If your system uses SELinux (like Fedora, Red Hat or CentOS among other Linux distributions) you'll need to install a policy module. You can find instructions about it in the bblfshd README.

Testing the daemon

If everything worked, docker logs bblfshd should output something like this:

[2019-05-01T17:58:29Z]  INFO bblfshd version: v2.12.1 (build: 2019-04-10T16:05:44+0000)
[2019-05-01T17:58:29Z]  INFO initializing runtime at /var/lib/bblfshd
[2019-05-01T17:58:29Z]  INFO server listening in 0.0.0.0:9432 (tcp)
[2019-05-01T17:58:29Z]  INFO control server listening in /var/run/bblfshctl.sock (unix)

Installing the drivers

Now we need to install the driver images into the daemon. You can install the official images with the following command:

$ docker exec -it bblfshd bblfshctl driver install --recommended

If you want to install a driver for a single language, you can do so by specifying the driver URI to bblfshctl:

docker exec -it bblfshd bblfshctl driver install docker://bblfsh/python-driver:latest

You can check the installed versions executing:

$ docker exec -it bblfshd bblfshctl driver list
+------------+------------------------------------------+---------+--------+----------+--------+-------------+----------------------+
|  LANGUAGE  |                  IMAGE                   | VERSION | STATUS | CREATED  |   OS   |     GO      |        NATIVE        |
+------------+------------------------------------------+---------+--------+----------+--------+-------------+----------------------+
| python     | docker://bblfsh/python-driver:latest     | v2.3.0  | beta   | 8 months | alpine | 1.10-alpine | python:3.6-alpine    |
| java       | docker://bblfsh/java-driver:latest       | v2.2.0  | beta   | 8 months | alpine | 1.10-alpine | openjdk:8-jre-alpine |
+------------+------------------------------------------+---------+--------+----------+--------+-------------+----------------------+

Testing the drivers

To test the driver you can execute a parse request to the server with the bblfshctl parse command, and an example contained in the Docker image:

$ docker exec -it bblfshd bblfshctl parse /opt/bblfsh/etc/examples/python.py

Use this only for testing the installation; if you want to do any real parsing with files in your local filesystem you should use one of the clients which would also allow you to run XPath queries over the results.

Running standalone

A standalone distribution of bblfshd and bblfshctl can be found at the GitHub release page. This will contain a single binary that can be run anywhere but it depends on the ostree library as explained in the readme.

bblfshd is only provided for Linux distributions, since it relies on Linux containers to run language drivers. And bblfshctl can be found for Windows, macOS and Linux.

Using bblfshctl

The bblfshd daemon comes with a command line tool called bblfshctl, which can be used to monitor and manage the daemon.

If you are using the Docker image then the command line tool is provided with the bblfsh/bblfshd image and can be used with a docker exec command.

$ docker exec -it bblfshd bblfshctl --help
Usage:
  bblfshctl [OPTIONS] <command>

Help Options:
  -h, --help  Show this help message

Available commands:
  driver     Manage drivers: install, remove and list
  instances  List the driver instances running on the daemon
  parse      Parse a file and prints the UAST or AST
  status     List all the pools of driver instances running

Driver management

The bblfshd's drivers can be installed, updated and removed with the driver command and its subcommands. Remember to prefix all these commands with docker exec -it bblfshd if you are running bblfshd from a Docker image.

Installing all official drivers:

$ bblfshctl driver install --recommended

Replacing an installed driver with a specific version:

$ bblfshctl driver install python bblfsh/python-driver:v2.3.0 --update

Listing all available drivers:

$ bblfshctl driver list
+------------+------------------------------------------+---------+--------+----------+--------+-------------+----------------------+
|  LANGUAGE  |                  IMAGE                   | VERSION | STATUS | CREATED  |   OS   |     GO      |        NATIVE        |
+------------+------------------------------------------+---------+--------+----------+--------+-------------+----------------------+
| python     | docker://bblfsh/python-driver:latest     | v2.3.0  | beta   | 8 months | alpine | 1.10-alpine | python:3.6-alpine    |
| java       | docker://bblfsh/java-driver:latest       | v2.2.0  | beta   | 8 months | alpine | 1.10-alpine | openjdk:8-jre-alpine |
+------------+------------------------------------------+---------+--------+----------+--------+-------------+----------------------+