All versions of this manual
X

 Linkurious administration manual

Welcome to the Linkurious administrator documentation. This documentation will help you install, run and customize Linkurious.

Architecture overview

Linkurious is a three-tier application.

The presentation layer is a Web application. It uses our graph visualization library, Ogma, to allow rich interactions with the graph. It also provides a user interface to enable data administration and collaboration among end users.

The presentation layer communicates with the logic layer via a JSON-based REST API. Custom presentation layer application can be developed on top of the logic layer.

The logic layer is a NodeJS-based server. It provides a unified REST API to read, write and search into graph databases from multiple vendors (Neo4j, JanusGraph, DataStax Enterprise Graph, AllegroGraph and Stardog). It implements also a security layer with modular authentication that enables role-based access control policies. It can be connected to multiple graph databases at the same time and offers high-level APIs for collaborative exploration of graphs: users can create, share and publish graph visualizations, and multiple users can edit graph data.

Administrators can control it from its REST API for easy automation and deployment.

Multiple external authentication providers are supported (LDAP, Microsoft Active Directory, Microsoft Azure Active Directory, Google Suite, OpenID Connect, SAML2 / ADFS).

The data layer supports several graph databases, as well as indexation engines.

 Getting started: Technical requirements

Linkurious is a Web-application server. It needs to be installed on a server and can then be accessed by multiple users using their Web browser.

Linkurious Server

Technical requirements for the machine used to install the Linkurious Web-application server:

Hardware

For a standard installation of Linkurious, the recommended hardware is:

Linkurious requires a 64-bit system to run.

Please keep in mind that these technical requirements are for Linkurious server only. For hardware requirements regarding your graph database, please refer to these guides:

Elasticsearch

Linkurious includes an embedded Elasticsearch instance for search capabilities. Please keep in mind that this embedded instance will only work for smaller graphs (less than 50M nodes + edges). For larger graphs, you will need to deploy an Elasticsearch cluster. Please refer to Elasticsearch's hardware requirements guide for details.

Operating System

Linkurious server can be deployed on the following platforms:

Java

Linkurious embedded Elasticsearch engine requires Java JRE (or JDK) 7+.

You can download the latest JRE from Oracle's website.

Installation instructions:

The JAVA_HOME environment variable should be set.

SQLite and GLIBC 2.14

Linkurious uses an embedded SQLite store for user-data persistence. This database requires GLIBC >= 2.14. Some older Linux distributions don't have this version of GLIBC available. You can check the version available on your system on http://distrowatch.com.

If SQLite does not work on your system, please refer to the user-data store documentation section to learn how to use an alternative database.

Linkurious Client

Technical requirements for users that access Linkurious with their Web browser:

Hardware

Hardware requirements of the Linkurious Web client vary with the size of the visualized graphs. For up to 500 nodes and edges in a single visualization, we recommend to use a machine with 4 GB RAM, and 2 CPU cores @ 1.6 Ghz.

The minimal screen resolution is 1024 x 768.

Web Browser

End-users will access Linkurious through a Web browser. All modern browsers are supported:

 Getting started: Downloading

Where to download from

The latest version of Linkurious can be downloaded from at https://linkurio.us/my-account/.

Log in with the username and password created during the purchase process. You will then see the downloadable files:

Archive content

The ZIP file contains:

Staying up-to-date

Please see our documentation on how to update Linkurious.

 Getting started: Installing

Linux systems

  1. Unzip Linkurious archive: > unzip linkurious-linux-v2.6.7.zip
  2. Enter the Linkurious folder: > cd linkurious-linux
  3. Check the configuration file at linkurious-linux/data/config/production.json (see how to configure a data-source)
  4. Make sure that Java JRE 7+ is installed (type java -version in a terminal), if needed, install Java

See how to start Linkurious on Linux.

Windows systems

  1. Unzip Linkurious archive (right-click on the file, then "Extract all")
  2. Enter the linkurious-windows folder
  3. Check the configuration file at linkurious-windows/data/config/production.json (see how to configure a data-source)
  4. Make sure that Java JRE 7+ is installed (see Java JDK requirements)

See how to start Linkurious on Windows.

Mac OS X systems

  1. Unzip Linkurious archive: > unzip linkurious-osx-v2.6.7.zip
  2. Enter the Linkurious folder: > cd linkurious-osx
  3. Check the configuration file at linkurious-osx/data/config/production.json (see how to configure a data-source)
  4. Make sure that Java JRE 7+ is installed (type java -version in a terminal), if needed, install Java

See how to start Linkurious on Mac OS X.

Docker Linux

  1. Unzip the docker image: > gunzip linkurious-docker-v2.6.7.tar.gz
  2. Load the docker image > docker load < linkurious-docker.tar

See how to start Linkurious with docker.

Install as a service

In order to run Linkurious automatically when the operating system starts, it is possible to install Linkurious as a system service on Linux, Mac OS X and Windows.

Open the administration menu by running menu.sh, menu.bat or menu.sh.command in the linkurious folder. Click on Install Linkurious as a system service.

 Getting started: Starting Linkurious

Linux systems

To start Linkurious, run the start.sh script in the linkurious-linux directory.

Alternatively, run the menu.sh script and click Start Linkurious.

By default, Linkurious server will listen for connection on port 3000. However, some firewalls block network traffic ports other than 80 (HTTP). See the Web server configuration documentation to learn how to make Linkurious listen on port 80.

Windows systems

To start Linkurious, run the start.bat script in the linkurious-windows directory.

Alternatively, run the menu.bat script and click Start Linkurious.

The firewall of Windows might ask you to authorize connections to Linkurious. If so, click on Authorize access.

Content of the linkurious-windows directory:

Linkurious starting up on Windows:

Mac OS X systems

To start Linkurious, run the start.sh.command script in the linkurious-osx directory.

Alternatively, run the menu.sh.command script and click Start Linkurious.

Docker Linux

  1. Port configuration

The Linkurious docker image exposes the ports 3000 and 3443 for http and https connections respectively. These ports should we mapped on the host machine to allow user connections.

Please visit the docker documentation to learn how publish the ports of a container.

  1. Volume configuration

The Linkurious docker image use 2 volumes:

These volumes should be maintained when upgrading Linkurious.

Please visit the docker documentation to learn how the configure volumes.

To create a Linkurious docker container, please use the docker run command. Here is an example:

 docker run -d \
     -p 3000:3000  \
     -v /path/to/my/linkurious/volume:/data \
     -v /path/to/my/elasticsearch/volume:/elasticsearch \
     linkurious:2.6.7

 Getting started: Stopping Linkurious

Linux systems

Run the stop.sh script in the linkurious-linux directory.

Alternately, run menu.sh and click Stop Linkurious.

Windows systems

Run the stop.bat script in the linkurious-windows directory.

Alternately, run menu.bat and click Stop Linkurious.

Mac OS X systems

Run the stop.sh.command script in the linkurious-osx directory.

Alternately, run menu.sh.command and click Stop Linkurious.

 Getting started: Configure Linkurious

To edit the Linkurious configuration, you can either edit the configuration file located at linkurious/data/config/production.json or use the Web user-interface:

Using an administrator account, access the Admin > Configuration menu to edit the Linkurious configuration:

Some configuration change requires a restart to be applied. Linkurious will notify you about it and offer you to restart from the Web UI only if you made the changes from the Web UI itself. If you modified the production.json file manually, changes won't get applied immediately and you will need to restart Linkurious.

Configuration keys are divided by category. When you are finished changing, click Save.

Password fields will always be hidden from the Web UI, but they can be edited.

 Getting started: Release notes

Changelog for Linkurious Enterprise v2.6.7

The release v2.6.7 "Galactic Gingerbread" closes 3 issues.

A total of 237 issues were closed in all v2.6.x versions.

Client changes

New features (7)

Enhancements (58)

Bugs fixes (87)

Server changes

New features (7)

Enhancements (42)

Bugs fixes (36)

API Changes (6)

type AccessLevel = "readable" | "editable" | "writable";
 
interface SchemaResult {
  // access-level for any category/type (for write-all and admins)
  any: {
    access: AccessLevel;
  };
 
  // access-level for specific categories/types
  results: Array<{
    name: string;
    access: AccessLevel;
    count?: number;
    properties?: Array<{
      key:string;
      count?:number;
      type?:string;
    }>;
  }>;
}
{
  "type": "node" | "edge",
  "totalHits": number,
  "results": Node[] | Edge[]
}

The output, where before was in the format:

  {
    "nodes": Array<{..., edges: Edge[], ...}>
  }

Now it's in the format:

  {
    "nodes": Node[],
    "edges": Edge[]
  }

The edges are no longer contained in each node. To retrieve the nodes from the edges, it's necessary to look at the source and target property of the edge that contain the IDs of the nodes.

A new option "edgesTo", has been added to the APIs to allow to specify the list of node ids to fetch the edges towards to. The same option was available only in the Expand API and was called "visibleNodeIds", now not available anymore.

REST API documentation

For more details about the API changes, please refer to the updated REST API documentation.

 Getting started: Graph DB Feature Map

Features per graph database vendor

Feature \ Vendor Neo4j JanusGraph JanusGraph on IBM Compose DataStax Enterprise Graph
Full-text search
Graph styles customization
Graph filtering
Graph editing
Access rights management
Custom graph queries
Custom query templates
Shortest path analysis
Alerts
Feature \ Vendor Cosmos DB Stardog AllegroGraph
Full-text search
Graph styles customization
Graph filtering
Graph editing
Access rights management
Custom graph queries
Custom query templates
Shortest path analysis
Alerts

Feature details

 Troubleshooting: Checking Linkurious status

Process monitoring

Linkurious starts 3 separate processes when launched:

Check if these processes are alive by opening the menu from the Linkurious directory (see how to open it on each operating system below):

Linux systems

Run menu.sh. Alternately, run menu.sh status.

Windows systems

Run menu.bat. Alternately, run menu.bat status.

Mac OS X systems

Run menu.sh.command. Alternately, run menu.sh.command status.

API status

The status of the API can be retrieved using a browser or a command line HTTP client like cURL.

To retrieve the API status, send a GET request to http://127.0.0.1:3000/api/status (replace 127.0.0.1 and 3000 with the actual host and port of your server).

// example response 
{
  "status": {
    "code": 200,
    "name": "initialized",
    "message": "Linkurious ready to go :)",
    "uptime": 8633
  }
}

API version

To retrieve the API status, send a GET request to http://127.0.0.1:3000/api/version (replace 127.0.0.1 and 3000 with the actual host and port of your server).

// example response 
{
  "tag_name": "2.6.7",
  "name": "Brilliant Burrito",
  "prerelease": false,
  "enterprise": true
}

 Troubleshooting: Reading the logs

The logs of the application are located in linkurious/data/manager/logs folder:

For a structured version of the Linkurious server logs in JSONL format, see the linkurious/data/logs folder:

 Troubleshooting: Getting support

If you need to get support regarding an issue or you have a question, please contact us at support@linkurio.us.

If the issue is technical, send us an automatically generated report via e-mail. You can download the report from the Web user interface via the Admin > Data menu:

At the end of the page, click Download Report:

 FAQ

Going to production

What should I do before going to production?

1. Ready your graph database for production

Consult with your vendor to make sure that your graph database is installed on appropriate hardware and configured for better performances:

Make sure that your graph database is secure:

2. Ready Elasticsearch for production

Keep in mind that Linkurious Enterprise can be used without Elasticsearch, see search options.

If you are using Linkurious Enterprise with Elasticsearch

3. Ready your user-data store for production

By default, SQLite is used for the user-data store. SQLite is not recommended for production environment: switch to MySQL/MariaDB/MSSQL instead.

Schedule regular backups of the user-data store:

Make sure your user-data-store database is secure

If you need high-availability, set up replication

4. Ready Linkurious Enterprise itself for production

How can Fault tolerance be achieved?

Linkurious Enterprise can be set up with a backup instance to allow for continuity of service when the main server crashes.

For this setup:

A reverse proxy is then configured to send requests to the backup server when the main server is down. If you are using nginx, this sample configuration can be used:

http {
    # define the "backend" upstream
    upstream backend {
        # main server
        server linkurious-main.example.com;
 
        # backup server
        server linkurious-backup.example.com backup;
    }
 
    # redirect all queries to the "backend" upsteam
    server {
        location / {
            proxy_pass http://backend;
        }
    }
}

See nginx documentation for more details.

Fault-tolerance diagram

Security

Where is the user-data store located?

The user-data store database (containing visualizations, saved queries, user, groups, etc) is stored in a SQL database.

By default, this database is an SQLite database (located at linkurious/data/database.sqlite). In production, the use of a MySQL/MariaDB/MSSQL database is recommended. These databases can be located on a remote server.

Is the user-data store encrypted?

The default user-data store (SQLite) is not encrypted.

Encryption is available with the following vendors:

Is it possible to delete the SQLite user-data store when using an external database?

Yes, when using an external user-data store (e.g. MariaDB, MySQL or MSSQL), the SQLite files can be deleted.

What kind of information is stored in the configuration file?

The configuration file contains all configurable options, as well as the configuration options of all configured data sources (e.g. MySQL host/port/username/encrypted password; Neo4j URL/username/encrypted password; Elasticsearch URL/username/encrypted password, etc). All passwords/secrets in the configuration file are encrypted before storage.

The configuration file, like the rest of the data folder, should be considered private and not be readable by anyone other than the Linkurious Enterprise service account.

How are application secrets stored?

All application secrets stored by Linkurious Enterprise (Neo4j credentials, MySQL credentials, Elasticsearch credentials, SSL certificate passphrase, etc.) are encrypted using the AES-256-CTR algorithm.

How are user credentials stored?

User passwords are strongly hashed before being stored in the database. Passwords for LDAP and other external authentication solutions are not stored at all.

Where is the audit trail stored?

The audit trail files are generated in linkurious/data/audit-trail by default. This path can be set in the audit trail configuration.

Does enabling the audit trail require additional security measures?

The audit trail contains sensitive information and should be secured. It should be owned and readable only by the Linkurious Enterprise service account.

How can the data directory be secured?

The data directory contains logs, configuration files, and, if enabled, audit trails. This information is sensitive, and the directory should be owned and readable only by the Linkurious Enterprise service account

What is a service account and why should I use one?

A service account is an operating system user account with restricted privileges that is used only to run a specific service and own it data related to this service. Service accounts are not intended to be used by people, except for performing administrative operations. Access to service accounts is usually tightly controlled using privileged access management solutions.

Service accounts prevent other users and services from reading or writing to sensitive files in the directories that they own, and are themselves prevented from reading and writing to other parts of the file system where they are not owners.

Can Kerberos be used for single sign-on?

We do not support Kerberos as of now (but we support many other SSOs).

What do the log files contain?

Linkurious Enterprise creates three types of logs:

How can the communication with an LDAP server be secured?

If your LDAP server supports secure LDAP, use the “ldaps://” protocol in your LDAP configuration.

How can Elasticsearch be secured?

If you need authentication and transport layer security for Elasticsearch:

Miscellaneous

Can I use Linkurious Enterprise without Elasticsearch?

Most graph vendors support search strategies other than Elasticsearch. See dails on our search options page.

Can I use a custom tile server in geospatial mode?

Yes. See the geospatial configuration options for further details.

What are the command lines utilities to administrate Linkurious Enterprise?

Will enabling the audit trail impact performance?

Depending on the configuration options specified, enabling the audit trail can have an impact on performance. See the audit trail documentation for details.

 Importing graph data

Before starting to explore the content of your graph database using Linkurious, you need to import data in your graph database.

The following sections will help you import data into your graph:

  1. Neo4j
  2. JanusGraph
  3. AllegroGraph
  4. DataStax Enterprise Graph
  5. Stardog
  6. Cosmos db

If you already have data in your graph database or you don't need to import anything, see how to configure your graph database with Linkurious.

 Importing graph data: Neo4j

Linkurious relies on Neo4j to store the data. The data importation is thus not handled by our solution. Many options exist to import data into Neo4j:

Finally, if you want to quickly try Linkurious Enterprise with an example dataset, you can download a Neo4j-compatible example dataset.

 Importing graph data: JanusGraph

Please refer to the JanusGraph online documentation for details on how to load data into JanusGraph.

 Importing graph data: AllegroGraph

Please refer to the AllegroGraph online documentation for details on how to load data into AllegroGraph.

 Importing graph data: DataStax Enterprise Graph

Please refer to the DataStax Enterprise Graph online documentation for details on how to load data into DataStax Enterprise Graph.

 Importing graph data: Stardog

Please refer to the Stardog online documentation for details on how to load data into Stardog.

 Importing graph data: Cosmos db

Please refer to the Cosmos db online documentation for details on how to load data into Cosmos db.

 Updating Linkurious

We release a new version of Linkurious every couple of month on average. New releases include new features, improvements, and bug fixes.

The following pages will help you check for updates, back-up Linkurious before updating, and update Linkurious.

 Updating Linkurious: Checking for updates

About menu

Using an administrator account, access the Your Username > About menu to open the Linkurious info:

Click Check for updates to see if you are using the latest version.

Public version API

Alternatively, you can check at http://linkurio.us/version/linkurious-enterprise.json

// example response 
{
  "tag_name": "v2.6.7", // latest version of Linkurious 
  "message": null,
  "url": "https://linkurio.us/my-account/" // where to download the latest version from 
}

 Updating Linkurious: Backing up your data

Follow these steps to perform a backup of your Linkurious data:

  1. Stop Linkurious
  2. Back-up of the linkurious/data folder
  3. Back-up the user-data store
  4. Start Linkurious

Please note that this procedure does not back-up your graph data, but only your Linkurious configuration and user-data (visualizations, users, etc.).

 Updating Linkurious: Update procedure

If you follow this procedure, you will be able to update Linkurious to a newer version without loosing your configuration and user-data store. You data will be automatically migrated to the newer version.

Before updating, make sure that you have backed-up your data in case something unexpected happens.

During the update procedure, if Linkurious is running, it will be stopped. You will need to re-start Linkurious after the update procedure.

  1. Download the newer version you wish to install (see how to download Linkurious)
  2. Copy linkurious-xxx-v2.6.7.zip in your current Linkurious directory (along the start stop and update scripts)
  3. Run the update script (Linux: update.sh, OSX: update.sh.command, Windows: update.bat)
  4. Done! You can restart Linkurious.

Troubleshooting

If the update script fails, please check the update log located at linkurious/data/update.log for details.

 Configuring data-sources

By default, Linkurious is configured to connect to a Neo4j database at 127.0.0.1:7474.

Supported vendors

Linkurious can connect to some of the the most popular graph databases:

Multi-database support

Linkurious is able to connect to several graph databases at the same time and lets you switch from one database to another seamlessly.

Edit the data-source configuration

You can configure your data-sources via the Web user interface or directly on the linkurious/data/config/production.json file.

Using the Web user interface

Using an administrator account, access the Admin > Data menu to edit the current data-source configuration:

Edit the data-source configuration to connect to your graph database:

Submit the changes by hitting the Save configuration button.

Using the configuration file

Edit the configuration file located at linkurious/data/config/production.json.

See details for each supported graph database vendor:

 Configuring data-sources: Neo4j

Neo4j is supported since version 2.1.5.

Configuration

To edit the Neo4j data-source configuration, you can either use the Web user-interface or edit the configuration file located at linkurious/data/config/production.json.

Example configuration:

{
  "dataSources": [
    {
      "graphdb": {
        "vendor": "neo4j",
        "url": "http://127.0.0.1:7474/",
        "user": "myNeo4jUser",
        "password": "nyNeo4jPassword"
      },
      "index": {
        "vendor": "neo4jSearch"
      }
    }
  ]
}

Since Neo4j version 3.0, Linkurious can connect to Neo4j via the Bolt protocol. To do so, you need to enable the protocol in your Neo4j configuration file. If Linkurious is connected over HTTP/S, it will try to automatically upgrade the connection to Bolt. The HTTP/S protocol is still required to perform a small subset of operations.

Supported graphdb options with Neo4j:

Search with Neo4j

In order to have full-text search, you can choose among the following options:

Neo4j credentials

If you just installed Neo4j, these steps will help you create credentials:

  1. Launch the Neo4j server
  2. Open your Web browser at http://127.0.0.1:7474
  3. Follow the instructions to create a new username and password

Alternatively, you can disable credentials in Neo4j by editing the Neo4j configuration at neo4j/conf/neo4j.conf by uncommenting the following line:

dbms.security.auth_enabled=false

 Configuring data-sources: JanusGraph

JanusGraph is supported since version 0.1.1.

Configuration

To edit the JanusGraph data-source configuration, you can either use the Web user-interface or edit the configuration file located at linkurious/data/config/production.json.

Example configuration:

{
  "dataSources": [
    {
      "graphdb": {
        "vendor": "janusGraph",
        "url": "ws://127.0.0.1:8182/",
        "graphAlias": 'graph1',
        "traversalSourceAlias": "g1"
      },
      "index": {
        "vendor": "janusGraphSearch",
        "create": true
      }
    }
  ]
}

Supported graphdb options for JanusGraph:

There are 3 alternatives to configure JanusGraph:

Note that only one of these alternatives can be used.

Dynamic Graphs

You can access your dynamically created graphs using the graph and traversal bindings references. Simply set graphAlias to <graph.graphname> and traversalSourceAlias to <graph.graphname>_traversal.

Visit the JanusGraph Documentation for more information on how to set up dynamic graphs.

JanusGraph 0.3.x

Linkurious DOES NOT support JanusGraph versions above 0.3.0 with the default configuration.

However, you can connect to a JanusGraph 0.3.x instance by switching the GraphSON serializer from version 3 to version 2.

To do this, follow these steps:

  1. Open the gremlin server configuration file, generally located at $INSTALL_DIR/conf/gremlin-server/gremlin-server.yaml
  2. Remove or comment the following line { className: org.apache.tinkerpop.gremlin.driver.ser.GraphSONMessageSerializerV3d0, config: { ioRegistries: [org.janusgraph.graphdb.tinkerpop.JanusGraphIoRegistry] }}
  3. Restart JanusGraph.

Search with JanusGraph

In order to have full-text search, you can choose among the following options:

JanusGraph For Compose

Linkurious can connect to your JanuGraph instances deployed on Compose.

Configuration

Example configuration:

{
"dataSources": [
  {
    "graphdb": {
      "vendor": "janusGraphForCompose",
      "url": "wss://xyz.composedb.com:16916",
      "user": "admin",
      "password": "XYZ",
      "graphName": "myGraph"
    },
    "index": {
      "vendor": "elasticSearch",
      "host": "127.0.0.1",
      "port": 9201
    }
  }
]
}

Supported graphdb options for JanusGraph For Compose:

 Configuring data-sources: AllegroGraph

AllegroGraph is supported since version 6.0.0.

Configuration

To edit the AllegroGraph data-source configuration, you can either use the Web user-interface or edit the configuration file located at linkurious/data/config/production.json.

Example configuration:

{
  "dataSources": [
    {
      "graphdb": {
        "vendor": "allegroGraph",
        "url": "http://127.0.0.1:10035/",
        "repository": "myGraph"
      },
      "index": {
        "vendor": "allegroGraphSearch"
      }
    }
  ]
}

Supported graphdb options for AllegroGraph:

Search with AllegroGraph

In order to have full-text search, you can choose among the following options:

 Configuring data-sources: DataStax Enterprise Graph

DataStax Enterprise Graph is supported since version 5.1.3.

Prerequisites

In order to use DSE Graph with Linkurious is necessary that the option graph.tx_autostart is set to true on your current database.

Enable it via the DSE gremlin console by typing:

:remote config alias g <graphName>.g
graph.schema().config().option('graph.tx_autostart').set('true')

If you are going to use any search index other than the one provided directly from DataStax Enterprise Graph, it is necessary to set to true also the option graph.allow_scan.

:remote config alias g <graphName>.g
graph.schema().config().option('graph.allow_scan').set('true')

Configuration

To edit the DataStax Enterprise Graph data-source configuration, you can either use the Web user-interface or edit the configuration file located at linkurious/data/config/production.json.

Example configuration:

{
  "dataSources": [
    {
      "graphdb": {
        "vendor": "dse",
        "url": "ws://127.0.0.1:8182/",
        "graphName": "myGraph"
      },
      "index": {
        "vendor": "dseSearch"
      }
    }
  ]
}

Supported graphdb options for DataStax Enterprise Graph:

Search with DataStax Enterprise Graph

In order to have full-text search, you can choose among the following options:

 Configuring data-sources: Stardog

Stardog is supported since versions 5.0.0.

Configuration

To edit the Stardog data-source configuration, you can either use the Web user-interface or edit the configuration file located at linkurious/data/config/production.json.

Example configuration:

{
  "dataSources": [
    {
      "graphdb": {
        "vendor": "stardog",
        "url": "http://127.0.0.1:10035/",
        "repository": "myGraph"
      },
      "index": {
        "vendor": "stardogSearch"
      }
    }
  ]
}

Supported graphdb options for Stardog:

Search with Stardog

In order to have full-text search, it's required to configure a search index in Stardog.

Reasoning

Reasoning can be enabled on pattern queries with the directive #REASONING=TRUE. You can add this directive at the beginning of the query to enable reasoning on that query.

Example pattern query with reasoning:

#REASONING=TRUE
select ?s {?s a <http://linkurio.us/organization>}

 Configuring data-sources: Cosmos DB

Cosmos DB is supported by Linkurious.

Configuration

To edit the Cosmos DB data-source configuration, you can either use the Web user-interface or edit the configuration file located at linkurious/data/config/production.json.

Example configuration:

{
  "dataSources": [
    {
      "graphdb": {
        "vendor": "cosmosDb",
        "url": "https://your-service.gremlin.cosmosdb.azure.com:443/",
        "database": "your-graph-database",
        "collection": "your-collection",
        "primaryKey": "your-account-primary-key"
      },
      "index": {
        "vendor": "azureSearch",
        "url": "https://your-search-service.search.windows.net",
        "apiKey": "your-search-service-admin-api-key",
        "nodeIndexName": "your-node-index",
        "edgeIndexName": "your-edge-index"
      }
    }
  ]
}

Supported graphdb options for Cosmos DB:

Search with Cosmos DB

In order to have full-text search, you can choose among the following options:

 Configuring data-sources: Alternative IDs

When you save a visualization in Linkurious, only the node and edge identifier are persisted in the user-data store, along with position and style information. When a visualization is loaded, the node and edge identifiers are used to reload the actual node and edge data from the graph database.

If you need to re-generate your graph database from scratch, the graph database will probably generate new identifiers for all nodes and edges, breaking all references to nodes and edges in existing visualizations.

Using a property as a stable identifier

You can configure Linkurious to use a node or edge property as stable identifiers. Once set-up, Linkurious will use the given property as identifier instead of using the identifiers generated by the database.

Thank to this strategy, visualizations will be robust to graph re-generation.

Notice that the properties used as identifier should be indexed by the database to allow for a fast lookup by value.

Alternative identifiers configuration

To use alternative node and edge identifiers, edit your data-source database configuration in the configuration file (linkurious/data/config/production.json):

Example of alternative identifier configuration with Neo4j:

{
  "dataSources": [
    {
      "graphdb": {
        "vendor": "neo4j",
        "url": "http://127.0.0.1:7474/",
        "alternativeNodeId": "STABLE_NODE_PROPETY_NAME",
        "alternativeEdgeId": "STABLE_EDGE_PROPETY_NAME"
      }
      // [...] 
    }
  ]
}

Stable sourceKey

Linkurious generates a unique identifier for data-source, based on internal information from the data-source. This data-source identifiers (called the sourceKey) is used to identify all user-data (visualizations etc.) that belongs to a data-source.

When re-generating a Neo4j graph database, for example, the sourceKey will change. In order to avoid breaking all references to existing visualizations, it is possible to set manually the sourceKey of a data-source.

Before re-generating the graph database, go to the Admin > Sources menu:

In the sources administration panel, find the Key value for your data-source (e.g. 1c3490bd) and copy it.

Then, edit the configuration file (linkurious/data/config/production.json) and set the key manualSourceKey for your data-source:

{
  "dataSources": [
    {
      "manualSourceKey": "1c3490bd",
      "graphdb": {
        "vendor": "neo4j",
        "url": "http://127.0.0.1:7474/"
      }
      // [...] 
    }
  ]
}

 Configuring data-sources: Advanced settings

The following advanced data-source settings applies to all data-sources.

To change them, see how to configure Linkurious.

General settings

Search engine settings

Graph exploration settings

Additional Certificate Authorities

Password obfuscation

 Search index

Linkurious allows you to search your graph using natural full-text search.

In order to offer the search feature out-of-the-box, Linkurious ships with an embedded Elasticsearch server. This option allow for zero-configuration deployment in many cases.

Indexing your graph data

By default, Linkurious uses Elasticsearch for search. This options requires Linkurious to index your graph database, which technically means that Linkurious will feed the whole content of the graph database to Elasticsearch to make it searchable. The time required to index the graph database increases with the size of the graph and this solution has limits in its scalability.

Indexation typically happens at speeds between 2000 and 20000 nodes or edges per second, depending on the number of properties for nodes and edges, and hardware performances.

Embedded Elasticsearch

By default, Linkurious ships with an embedded Elasticsearch server (version 1.4.5). This server only listens for local connections on a non-default port (it binds to 127.0.0.1:9201), for security reasons and to avoid collisions with existing servers.

Use your own Elasticsearch

It is possible to use your own Elasticsearch cluster for performances reasons. Linkurious supports Elasticsearch v1.x and v2.x. See details about Elasticsearch configuration options.

Search scalability and alternatives to Elasticsearch

Using Elasticsearch is convenient but may not fit cases where the graph database is big (more than a couple million nodes and edges) and is regularly modified from outside Linkurious, which required to re-index the whole database.

In order to offer a scalable search feature on big graphs, Linkurious offers alternatives search solution:

Edit the search configuration

You can configure your search engines via the Web user interface or directly on the linkurious/data/config/production.json file.

Using the Web user interface

Using an administrator account, access the Admin > Data menu to edit the current data-source configuration:

Edit the search engine configuration to connect to your graph database:

Submit the changes by hitting the Save configuration button.

Using the configuration file

Edit the configuration file located at linkurious/data/config/production.json.

See details for each supported search connector:

 Search index: Search Option Comparison

Feature Description
Onboarding Does this search option require additional configuration, or can it be used out-of-the-box with the associated graph database?
Fast indexation How fast is indexation? Note that this is a relative metric; while some search options may be faster than others, speed will depend on the complexity of your data model and your hardware limitations.
Automatic index sync Are changes made to the graph DB propagated to the index automatically?
Search scalability How well search queries perform for large graph databases? The actual upper limit on the performance of a given option will vary from vendor to vendor.
Advanced search If advanced search features are available, such as numerical and date range search operators.

Neo4j

Onboarding Fast indexation Automatic index sync Search scalability Advanced search
Embedded Elasticsearch Plug-and-play No No Won't scale beyond ~100M nodes Yes (requires configuration)
External Elasticsearch (v2+) Requires Elasticsearch installation and configuration No No Yes (by adding hardware to Elasticsearch cluster) Yes (requires configuration)
Neo4j-to-Elasticsearch Requires Elasticsearch installation and configuration + Neo4j plugin installation and configuration Yes (up to 10,000 nodes/second) Yes Yes (by adding hardware to Elasticsearch cluster) Yes
Neo4j Search (pre-3.4) Requires configuration in Neo4j (properties to index must be specified) Yes Yes Limited No

JanusGraph

Onboarding Fast indexation Automatic index sync Search scalability Advanced search
JanusGraph Search Requires configuration in JanusGraph No Yes Yes No
Embedded Elasticsearch Plug-and-play No No Won't scale beyond ~100M nodes Yes (requires configuration)
External Elasticsearch (v2+) Requires Elasticsearch installation and configuration No No Yes (by adding hardware to Elasticsearch cluster) Yes (requires configuration)

JanusGraph on Compose.com

Onboarding Fast indexation Automatic index sync Search scalability Advanced search
Embedded Elasticsearch Plug-and-play No No Won't scale beyond ~100M nodes Yes (requires configuration)
External Elasticsearch (v2+) Requires Elasticsearch installation and configuration No No Yes (by adding hardware to Elasticsearch cluster) Yes (requires configuration)

DataStax Enterprise Graph

Onboarding Fast indexation Automatic index sync Search scalability Advanced search
DSE Search Plug-and-play Yes Yes Yes No
Embedded Elasticsearch Plug-and-play No No Won't scale beyond ~100M nodes Yes (requires configuration)
External Elasticsearch (v2+) Requires Elasticsearch installation and configuration No No Yes (by adding hardware to Elasticsearch cluster) Yes (requires configuration)

AllegroGraph

Onboarding Fast indexation Automatic index sync Search scalability Advanced search
AllegroGraph Search Plug-and-play Yes Yes Yes No
Embedded Elasticsearch Plug-and-play No No Won't scale beyond ~100M nodes Yes (requires configuration)
External Elasticsearch (v2+) Requires Elasticsearch installation and configuration No No Yes (by adding hardware to Elasticsearch cluster) Yes (requires configuration)

Stardog

Onboarding Fast indexation Automatic index sync Search scalability Advanced search
Stardog Search Plug-and-play Yes Yes Yes No

Cosmos DB

Onboarding Fast indexation Automatic index sync Search scalability Advanced search
AzureSearch Requires AzureSearch setup (easy) Yes Yes Yes No
Embedded Elasticsearch Plug-and-play No No Won't scale beyond ~1M nodes Yes (requires configuration)
External Elasticsearch (v2+) Requires Elasticsearch installation and configuration No No Won't scale beyond ~1M nodes Yes (requires configuration)

 Search index: Neo4j

The neo4jSearch connector is a solution for full-text search with Neo4j.

Neo4j search integration

Linkurious can use the builtin search indices managed by Neo4j itself. You can either use the Web user-interface or edit the configuration file located at linkurious/data/config/production.json to set the index.vendor property to the value neo4jSearch.

Configuration

To edit the Neo4j data-source configuration, you can either use the Web user-interface or edit the configuration file located at linkurious/data/config/production.json.

Example configuration:

{
  "dataSources": [
    {
      "graphdb": {
        "vendor": "neo4j",
        "url": "http://127.0.0.1:7474/",
        "user": "myNeo4jUser",
        "password": "nyNeo4jPassword"
      },
      "index": {
        "vendor": "neo4jSearch",
        "batchSize": 4000000,
        "numberOfThreads": 32,
        "categoriesToIndex": ["Person", "Company"],
        "edgeTypesToIndex": ["Works for"]
      }
    }
  ]
}

Supported index options with Neo4jSearch:

Setting batchSize and numberOfThreads to higher values will decrease the required time to index the whole dataset. Suggested values for the batch size are 10000000 (default value) and multiples of it. Regarding the number of threads, the suggested value is the number of cores of the machine where Neo4j is installed.

Configure Neo4j to create a search index

  1. Edit the Neo4 configuration at neo4j/conf/neo4j.conf and add the following lines at the beginning of the file:
dbms.auto_index.nodes.enabled=true
# node properties that will be searchable (SET YOUR OWN) 
dbms.auto_index.nodes.keys=name,description,firstName,lastName,email,content
 
# Add the following lines if you want to index relationships as well 
dbms.auto_index.relationships.enabled=true
# relationship properties that will be searchable (SET YOUR OWN) 
dbms.auto_index.relationships.keys=description,content
  1. Restart Neo4j

If you came upon "The index can't be overwritten to be lower case"

It means that Neo4j already has a node_auto_index index with a different configuration. Unfortunately, Linkurious is not able to delete or replace the configuration on this index. A manual procedure on your dataset will be necessary.

Please be sure to have a backup of your data before performing the following actions.

  1. Stop Neo4j
  2. Open your Neo4j graph database folder (by default it is in your Neo4j folder, in data/databases/graph.db)
  3. Within your database folder, open your database's index folder: index/lucene
  4. Within index/lucene, delete the node folder
  5. Within index/lucene, delete the relationship folder
  6. Run the Neo4j shell (from the Neo4j folder, run ./bin/neo4j-shell --path ./data/databases/graph.db/, check that the database folder is correct)
  7. In the Neo4j shell, execute these commands:
  1. Start Neo4j, you are done!

 Search index: Neo4j to Elasticsearch

Neo4j-to-elasticsearch is a Neo4j plugin that enables automatic synchronization between Neo4j and Elasticsearch. This means that all changes to Neo4j are automatically propagated to Elasticsearch.

Resources for supported versions of neo4j-to-elasticsearch

Install neo4j-to-elasticsearch

Follow these steps to install the plugin:

  1. Download the GraphAware framework JAR
    • Choose a version A.B.C.x where A.B.C matches your Neo4j version and x is 44 or later
  2. Download the neo4j-to-elasticsearch JAR
    • Choose a version A.B.C.x.y where A.B.C matches your Neo4j version and x.y is 44.8 or later
  3. Copy graphaware-server-community-all-A.B.C.x.jar and graphaware-neo4j-to-elasticsearch-A.B.C.x.y.jar to your neo4j/plugins directory
  4. Add the following lines to the beginning of your Neo4 configuration file (neo4j/conf/neo4j.conf):
    com.graphaware.runtime.enabled=true
    com.graphaware.module.ES.1=com.graphaware.module.es.ElasticSearchModuleBootstrapper
    com.graphaware.module.ES.uri=HOST_OF_YOUR_ELASTICSEARCH_SERVER
    com.graphaware.module.ES.port=PORT_OF_YOUR_ELASTICSEARCH_SERVER
    com.graphaware.module.ES.mapping=AdvancedMapping
    com.graphaware.module.ES.keyProperty=ID()
    com.graphaware.module.ES.retryOnError=true
    com.graphaware.module.ES.asyncIndexation=true
    com.graphaware.module.ES.initializeUntil=2000000000000
     
    # Set "relationship" to "(false)" to disable relationship (edge) indexation. 
    # Disabling relationship indexation is recommended if you have a lot of relationships and don't need to search them. 
    com.graphaware.module.ES.relationship=(true)
     
    com.graphaware.runtime.stats.disabled=true
    com.graphaware.server.stats.disabled=true
  5. Restart Neo4j
  6. Once Neo4j has finished indexing the data, remove the following line (and only this line) from neo4j/conf/neo4j.conf:
    com.graphaware.module.ES.initializeUntil=2000000000000

Note regarding initializeUntil

The initializeUntil specification is used to trigger the indexation of existing Neo4j data in Elasticsearch. This is because com.graphaware.module.ES.initializeUntil must be set to a number slightly higher than a Java call to System.currentTimeInMillis() would normally return when the module is started. Thus, the database will be (re-)indexed only once, and not with every subsequent restart.

In other words, re-indexing will happen if System.currentTimeInMillis() < com.graphaware.module.ES.initializeUntil.

Integrate with Linkurious

Once the neo4j-to-elasticsearch plugin is installed, you need to change the relevant data-source configuration to use neo2es as its search index vendor.

You can either use the Web user-interface or edit the configuration file located at linkurious/data/config/production.json to set the index.vendor property to the value neo2es.

Troubleshooting and Optimization

For smaller indexation tasks, neo4j-to-elasticsearch can be used straight out of the box. Larger indexes are trickier. If you find that indexation is failing to complete, or that search is unusually slow, you will need to settle for partial indexation in order to keep Elasticsearch usable.

Partial indexation can be configured in your Neo4j configuration file (neo4j/conf/neo4j.conf) by specifying a subset of your graph to index. You can do this by selecting which types of nodes and relationships to keep and which properties to index on these nodes.

(For a detailed list of configuration options, please consult Neo4j-to-elasticsearch's official documentation, available at Graphaware's Github repo. This guide will focus on a few use cases that should be adaptable to a wide variety of graph models.)

Configuration Options

Partial indexation is handled by four options which can be added to your Neo4j configuration file:

com.graphaware.module.ES.node=...
com.graphaware.module.ES.node.property=...
com.graphaware.module.ES.relationship=...
com.graphaware.module.ES.relationship.property=...

com.graphaware.module.ES.node and com.graphaware.module.ES.relationship control which nodes and relationships to index. com.graphaware.module.ES.node.property and com.graphaware.module.ES.relationship.property control which properties of these nodes and relationships to index.

Each of these lines is followed by one or more parameters. Parameters are boolean expressions. They can be chained together using standard logical operators && (AND) and || (OR). The most basic parameters are (true) and (false). They will tell Elasticsearch either to index everything (the default behavior) or to index nothing. Note that the parentheses are required in order to force Elasticsearch to ignore these nodes while loading your database into the index.

To reiterate, if we add the line

com.graphaware.module.ES.relationship=(false)

to our configuration file, Elasticsearch will not index any of the relationships in our database.

The next step up is parameter functions. These allow for more complex inclusion and exclusion rules. For both nodes and relationships, the following two functions are available:

Additionally, there are functions specific to nodes and to relationships. For nodes, those that are useful for partial indexation are:

And for relationships, they are:

(You can find a full list of functions in Graphaware's inclusion policies maintained on their Github repo.)

Example

Say we have a due diligence database containing people, banks, account numbers, addresses, telephone numbers, and email addresses. It's a large database -- several hundred million nodes -- so to fully index the graph would be a lengthy process, and may ultimately be unnecessary if we are only interesting in full-text search on a restricted set of node and relationship properties.

The first question we should ask is about this data we are interested in. Maybe as part of our hypothesis about the data, we want to focus on a certain subset of connections that we believe form patterns of interest.

Let's say that we want to focus on identifying information only -- we think that there are cases where this information is shared by multiple individuals, for instance, and we're interested in analyzing them. The nodes of interest to us will therefore be those nodes which represent pure identifiers and not entities themselves -- addresses, telephone numbers, account numbers, and email addresses. We can tell Elasticsearch to index these and only these by adding the following line to our Neo4j configuration file:

com.graphaware.module.ES.node=hasLabel('Address') || hasLabel('Telephone') || hasLabel('Email') || hasLabel('Account')

(com.graphaware.module.ES.node=!hasLabel('Person') && !hasLabel('Bank') will also work.)

And since we aren't including people or banks, we also want to focus on the relationships relevant to our nodes of interest:

com.graphaware.module.ES.relationship=isType('HAS_ADDRESS') || isType('HAS_PHONE') || isType('HAS_EMAIL') || isType('HAS_ACCOUNT')

If we want to be even more specific, we can select only those properties which are relevant to our inquiry by adding com.graphaware.module.ES.node.property to our config. We follow this with a list of keys, or node property names, (separated by || (OR) statements) that we want to include in our index:

com.graphaware.module.ES.node.property=key == 'address1' || key == 'city' || key == 'state' || key == 'number' || key == 'email' || key == 'accountNumber'

And if we only want to index nodes and NOT relationships, we can disable relationship indexation completely:

com.graphaware.module.ES.relationship=(false)

Since relationships are often more numerous than nodes, excluding them from our index can significantly reduce its storage footprint.

What if we finish our initial analysis and conclude that we need to know more about the financial institutions in our graph? We want to add banks to our index, but we don't need to search everything that is stored on them. Furthermore, we're only interested in banks in a certain region -- Europe, say. We can add the right banks back by modifying our original directive to read:

com.graphaware.module.ES.node=hasLabel('Address') || hasLabel('Telephone') || hasLabel('Email') || hasLabel('Account') || (hasLabel('Bank') && getProperty('bankRegion', 'None') == 'Europe')

And we can add a few bank properties like this:

com.graphaware.module.ES.node.property=key == 'address1' || key == 'city' || key == 'state' || key == 'number' || key == 'email' || key == 'accountNumber' || key == 'bankName' || key == 'bankIdentifier' || key == 'bankRegion'

Keep in mind that these keys will be indexed for every node on which they appear. If banks have a state property, for example, it will be added to the index for state. It's worth remembering this when constructing your data model. Namespace collisions can prove computationally costly.

Summary

Neo4j-to-Elasticsearch, in combination with partial indexation strategies, can be a very efficient way of handling index synchronization for large graphs. Even if your graph is small enough to index in full, it's worth considering to what extent it may grow. By examining your problem space and choosing only a subset of the information available to you for your traversal needs, you may save yourself future headaches and maximize the efficiency of your graph.

 Search index: JanusGraph

The janusGraphSearch connector is the recommended solution for full-text search for graphs with more than 100,000 nodes and edges.

JanusGraph search integration

Linkurious can use the builtin search indices managed by JanusGraph itself. You can either use the Web user-interface or edit the configuration file located at linkurious/data/config/production.json to set the index.vendor property to the value janusGraphSearch.

Configuration

To edit the JanusGraph data-source configuration, you can either use the Web user-interface or edit the configuration file located at linkurious/data/config/production.json.

Example configuration:

{
  "dataSources": [
    {
      "graphdb": {
        "vendor": "janusGraph",
        "url": "ws://127.0.0.1:8182/",
        "configuration": {
          "storage.backend": "cassandra",
          "storage.hostname": "127.0.0.1"
        }
      },
      "index": {
        "vendor": "janusGraphSearch",
        "create": true
      }
    }
  ]
}

Global vs Local configuration

The JanusGraph instance used by Linkurious can be configured via the configuration field as illustrated in the example above. Using this field, you can provide Linkurious a valid set of JanusGraph configuration options.

However, it's important to note that these are local configuration options. As opposed to the global configuration options, these options are only applicable if they have a MASKABLE or LOCAL mutability level. See Mutability Levels in JanusGraph.

If there are conflicts between the LOCAL and GLOBAL configuration values, JanusGraph may refuse to start or simply ignore the local configuration options and fallback on the globally managed values. This will likely produce unexpected results as Linkurious will be running with a different configuration than the one provided.

You can check the value of a GLOBAL configuration option with the Management API using the gremlin console, e.g.:

mgmt = graph.openManagement()
// Get the value of the configuration option 'index.search.backend' 
mgmt.get('index.search.backend') //elasticsearch 

The GLOBAL options can only be changed through the Management API of JanusGraph. In addition, changing GLOBAL_OFFLINE options requires restarting the cluster.

For example, if you want to change the index backend configuration:

// Open the management API 
mgmt = graph.openManagement()
// Force close all opened instances 
mgmt.getOpenInstances().each {if (!it.contains('current')) mgmt.forceCloseInstance(it)}
// Apply the changes 
mgmt.commit()
// Open the management API (All but the current instance are closed) 
mgmt = graph.openManagement()
// Change the GLOBAL_OFFLINE configuration option 
mgmt.set('index.search.backend', 'solr')
// Apply the changes 
mgmt.commit()

Solr Configuration

To use Solr as an index backend for JanusGraphSearch, you first need to create a Solr collection for indexing the nodes. A second collection must be created if you want to index the edges as well.

The collections should be named linkurious_index and linkurious_index_edges for nodes and edges respectively. Please refer to the Solr configuration overview on how to create a collection.

Here an example on how to provide Solr configuration information to Linkurious.

Example configuration:

{
  "dataSources": [
    {
      "graphdb": {
        "vendor": "janusGraph",
        "url": "ws://127.0.0.1:8182/",
        "configuration": {
          "storage.backend": "cassandra",
          "storage.hostname": "127.0.0.1",
          "index.search.backend": "solr",
          "index.search.solr.mode": "cloud",
          "index.search.solr.zookeeper-url": "localhost:9983"
        }
      },
      "index": {
        "vendor": "janusGraphSearch",
        "create": true
      }
    }
  ]
}

Linkurious will not override globally managed configuration values.

Supported index options with JanusGraphSearch:

Create a search index in JanusGraph

Linkurious will automatically create a search index in JanusGraph if the index option "create" is true. However, Linkurious can also use a mixed search index that already exist in JanusGraph.

Please refer to the JanusGraph documentation on how to create mixed indices if you want to create one directly.

 Search index: AllegroGraph

The allegroGraphSearch connector is the recommended solution for full-text search for graphs with more than 1,000,000 triples.

AllegroGraph search integration

Linkurious can use the builtin search indices managed by AllegroGraph itself. You can either use the Web user-interface or edit the configuration file located at linkurious/data/config/production.json to set the index.vendor property to the value allegroGraphSearch.

See the AllegroGraph documentation to learn more.

If you already have a search index in AllegroGraph, the following step is not required.

Create a search index in AllegroGraph

  1. Go to the repository page on AllegroGraph WebView (by default at http://127.0.0.1:10035)
  2. Press Manage free-text indices
  3. Create a new free-text index, the name is irrelevant
  4. The default options for the newly created index should be enough.

You must ensure that Objects under Fields to index is highlighted.

 Search index: DataStax Enterprise Graph

The dseSearch connector is the recommended solution for full-text search for graphs with more than 100,000 nodes and edges.

DataStax Enterprise Graph Search integration

Linkurious can use the builtin search indices managed by DataStax Enterprise Graph itself. You can either use the Web user-interface or edit the configuration file located at linkurious/data/config/production.json to set the index.vendor property to the value dseSearch.

See the DataStax Enterprise Graph documentation to learn more.

If you already have a search index in DataStax Enterprise Graph, the following step is not required.

Create a search index in DataStax Enterprise Graph

DSE Search in DataStax Enterprise's built-in search engine. You can learn more about DSE Search from DataStax' documentation.

Index with DSE Search

Please refer to the DataStax Enterprise Graph documentation on how to create new indices, including search indices.

Follow these steps to use DSE Search and integrate it with Linkurious:

  1. Connect via gremlin to DSE Graph
  2. For each node label you want to index execute this command:
schema.vertexLabel("MY_NODE_LABEL").index("search").search()
.by("MY_NODE_PROPERTY_1").asText()
.by("MY_NODE_PROPERTY_2").asText()
.by("MY_NODE_PROPERTY_3").asText()
.by("MY_NODE_PROPERTY_N").asText()
.add();

Every property key that you want to index has to appear in this command. You can add as many properties as you want. We recommend to index only what you actually need to.

We recommend to create asText() search indices instead of asString() ones. Both types of indices are supported by Linkurious, but the latter doesn't support OR search queries. If you don't need this feature, asString() indices will actually compute search queries faster. It's also possible to mix asText() and asString() indices to find a balance of features and performances.

 Search index: Stardog

The stardogSearch connector is the only solution for full-text search in Stardog.

Stardog search integration

Linkurious can use the builtin search indices managed by Stardog itself. You can either use the Web user-interface or edit the configuration file located at linkurious/data/config/production.json to set the index.vendor property to the value stardogSearch.

See the Stardog documentation to learn more.

If you already have a search index in Stardog, the following step is not required.

Create a search index in Stardog

Under the configuration page of the repository, ensure that search.enabled is ON.

 Search index: Azure search

Azure search is the recommended full text search solution for Cosmos DB.

Azure search integration

Linkurious requires an index on nodes to perform a search. If you do not have a configured index yet, you can create one via the azure portal.

Additionally, you can create an index on edges if you want search them as well with Linkurious.

Configuration

To edit the AzureSearch data-source configuration, you can either use the Web user-interface or edit the configuration file located at linkurious/data/config/production.json.

Example configuration:

{
  "dataSources": [
    {
      "graphdb": {
        "vendor": "cosmosDb",
        "url": "https://your-service.gremlin.cosmosdb.azure.com:443/",
        "database": "your-graph-database",
        "collection":  "your-collection",
        "primaryKey": "your-account-primary-key"
      },
      "index": {
        "vendor": "azureSearch",
        "url": "https://your-search-service.search.windows.net",
        "apiKey": "your-search-service-admin-api-key",
        "nodeIndexName": "your-node-index-name",
        "edgeIndexName": "your-edge-index-name"
      }
    }
  ]
}

Supported index options with Azure search:

Please refer to the Azure search online documentation for details on how to load data into Azure search.

 Search index: Properties disclosure

By default, all node and edge properties are available to users of Linkurious Enterprise. You can make some properties unavailable. Unavailable properties will no longer be accessible or searchable.

Open the Admin > Data menu, and scroll down to the Node properties section. All properties are marked as available in the following example.

 Search index: Configuring Elasticsearch

Elasticsearch is supported from version 1.4.0 (using the elasticSearch connector) and for version 2.x and version 5.x (using the elasticSearch2 connector).

Elasticsearch 6 is not yet supported in Linkurious 2.6.7.

Embedded Elasticsearch

Linkurious ships with an embedded Elasticsearch server (version 1.4.5).

ATTENTION: The internal Elasticsearch is not intended to be used for graph databases > 50,000,000 nodes. Though indexation and search performance are ultimately dependent on hardware limitations, it has been configured to prevent horizontal scaling and so is not an efficient choice for large DBs. It is meant instead as a quick indexation strategy for POCs or small deployments.

To use the Linkurious embedded Elasticsearch instance, set the following index configurations keys:

Example configuration:

{
  "dataSources": [
    {
      "graph": {
        "vendor": "neo4j"
        "url": "http://127.0.0.1:7474"
      },
      "index": {
        "vendor": "elasticSearch",
        "host": "127.0.0.1",
        "port": 9201
      }
    }
  ]
}

Configuring Elasticsearch v1.x

Search connector elasticSearch supports the following options:

Example configuration:

{
  "dataSources": [
    {
      "graph": {
        "vendor": "neo4j"
        "url": "http://127.0.0.1:7474"
      },
      "index": {
        "vendor": "elasticSearch",
        "host": "192.168.1.80",
        "port": 9200,
        "dynamicMapping": false
      }
    }
  ]
}

Configuring Elasticsearch v2.x

Search connector elasticSearch2 supports the following options:

Example configuration:

{
  "dataSources": [
    {
      "graph": {
        "vendor": "neo4j"
        "url": "http://127.0.0.1:7474"
      },
      "index": {
        "vendor": "elasticSearch2",
        "host": "192.168.1.122",
        "port": 9200,
        "skipEdgeIndexation": true
      }
    }
  ]
}

 Search index: Using Elasticsearch on AWS

By default, Linkurious ships with an embedded Elasticsearch instance which works out-of-the-box by default. The embedded Elasticsearch instance will work well for average to large database sizes, but for search-heavy use-cases or very large databases, configuring your own Elasticsearch cluster might be necessary.

An easy way to deploy an easy-to-scale Elasticsearch cluster yourself is to use Amazon Web Services (AWS).

Please follow these steps to create a configure your AWS Elasticsearch cluster with Linkurious:

Create your AWS account

Visit the Amazon Web Services website and create your account (or log in if you already have one).

Create a new cluster

Visit the Amazon Elasticsearch Service page, log-in and follow the steps to create an Elasticsearch cluster:

  1. Select Services > Elasticsearch Service

  2. Click get started

  3. Name your cluster (1) and select the Elasticsearch version 2.x (2), click Next

  4. Select the instance type, the number of instances and the number of dedicated masters in your cluster (3), depending on your database's size

  5. Configure the access policy for your cluster. Use access from specific IP (4) and enter the public IP address of your Linkurious server (5)

  6. Review your configuration and confirm the creation of the cluster

  7. Wait until the cluster is deployed (usually less than an hour)

  8. When your cluster is deployed, copy the Endpoint host name

  9. Configure Elasticsearch as explained here

 User-data store

Linkurious uses an SQL database to store user-data. The database contains:

By default, the user-data store is a SQLite file-based database. This makes Linkurious easy to deploy.

For deployment at scale (more than a couple users), we recommend switching to one of the supported server-based databases:

Configure with SQLite

SQLite if the default user-data store of Linkurious.

"db": {
    "name": "linkurious",
    "options": {
      "dialect": "sqlite"
      "storage": "server/database.sqlite"
    }
}

Configure with MySQL

"db": {
    "name": "linkurious",
    "username": "MYSQL_USER_NAME",
    "password": "MYSQL_PASSWORD",
    "options": {
      "dialect": "mysql",
      "host": "MYSQL_HOST",
      "port": 3306
    }
}

Configure with Microsoft SQL Server

"db": {
    "name": "linkurious",
    "username": "MSSQL_USER_NAME",
    "password": "MSSQL_PASSWORD",
    "options": {
      "dialect": "mssql",
      "host": "MSSQL_HOST",
      "port": 1433
    }
}

Configure with MariaDB

"db": {
    "name": "linkurious",
    "username": "MARIADB_USER_NAME",
    "password": "MARIADB_PASSWORD",
    "options": {
      "dialect": "mariadb",
      "host": "MARIADB_HOST",
      "port": 3306
    }
}

 User-data store: Migrating to MySQL

The default storage system for Linkurious is SQLite.

Configuring Linkurious to work with MySQL is a really easy procedure if you don't need to migrate data from SQLite.

Migrating data from SQLite to MySQL is possible but it is a procedure we would recommend only if restarting from scratch with MySQL is not a viable option.

This is the procedure to dump data from SQLite:

sqlite3 ./linkurious/data/server/database.sqlite .dump > export.sql

This is the procedure to import data into MySQL:

mysql -u MY_MYSQL_USER -p -h 127.0.0.1 linkurious < export.sql

The SQL file format from SQLite and MySQL is not exactly the same. If you need help with the procedure please contact us at support@linkurio.us.

 User-data store: Backing-up your store

If you are using the SQLite database (by default), you only need to follow the standard Linkurious backup procedure.

If you are using another database to store the Linkurious user-data, please refer to one of the following guides:

 Web server

The web server of Linkurious delivers the application to end users through HTTP/S. It is configured in the server configuration key within the configuration file (linkurious/data/config/production.json):

General

Within the server key:

Some firewalls block network traffic ports other than 80 (HTTP). Since only root users can listen on ports lower than 1024, you may want reroute traffic from 80 to 3000 as follows:

sudo iptables -t nat -A PREROUTING -p tcp --dport 80 -j REDIRECT --to-port 3000

If you use SSL, you can add a second rule to redirect 3443 to 443:

sudo iptables -t nat -A PREROUTING -p tcp --dport 443 -j REDIRECT --to-port 3443

Serving Linkurious under a custom path

Within the server key:

In some cases, you may want to host Linkurious on a path other than root for a particular domain. For example, if you want Linkurious to be reachable at http(s)://HOST:PORT/linkurious you should set baseFolder equal to linkurious.

Link generation

Within the server key:

In some cases, Linkurious needs to generate links to itself (for example when generating a link to a widget). For that, the server needs to know its public domain and port to generate those links.

The public port can be different from the actual port if you use traffic rerouting (using a firewall or a reverse-proxy). In the example above (traffic rerouting), the actual HTTP port (listenPort) is 3000, but the public HTTP port (publicPortHttp) is 80.

Cookies

Within the server key:

Cross-origin resource sharing (CORS)

Within the server key:

Image cross-origin (client-side)

Within the ogma.settings.render key:

SSL

Within the server key:

External communications with the Linkurious server can be secured using SSL without installing third-party software.

If the Linkurious server, graph database, and the search index are installed on different machines, we recommend using secure communication channels between these machines (e.g. HTTPS or WSS). Please refer to the data-source documentation and search index documentation to learn how to enable HTTPS.

To use custom Certificate Authorities (CA), please check how to use additional Certificate Authorities in Linkurious.

 Access control

Configuring access control

The access control is configured within the access configuration key in the configuration file (linkurious/data/config/production.json):

Local vs. external authentication

To access Linkurious when authRequired is true, users need accounts in Linkurious. Administrators can create accounts directly in Linkurious (see how to create users) or rely on an external authentication service.

Linkurious supports the following external authentication services:

If your company uses an authentication service that Linkurious does not support yet, please get in touch.

If you enable an SSO capable authentication service (OAuth/OpenID Connect or SAML2), your users won't need to login directly in Linkurious but, instead, by clicking the SSO button they will be redirected to the identity provider for authentication.

 Access control: Enabling authentication

We strongly advise you to enable user authentication to secure the access to your data once Linkurious Enterprise is deployed on a server. This will allow you to enforce the limit of authenticated users with regards to your license terms.

By default, user authentication is disabled and all actions are performed under the special account named "Unique User". The unique user has unrestricted access and does not require a password, so anyone can access the platform.

To enable authentication use the Web user interface via the Admin > Users menu:

The following screen will be prompted if authentication is disabled. Click Enable Authentication.

Create an admin account and click Save and enable.

 Access control: Guest mode

What is the Guest mode

The Guest mode is a way to share graphs with people who do not have an account on Linkurious Enterprise.

Key characteristics:

Standard user interface:

Guest mode user interface:

Enabling the Guest mode

By default, the Guest mode is disabled. Enabling the Guest mode is done on a per-source basis so that you can manage the access rights associated with the Guest mode at the source level.

Enabling it is a 3-steps process:

  1. In the Configuration screen (Menu > Admin > Configuration) activate the Guest mode
  2. A new user is created in the user list with the email guest@linkurio.us. By default it is not associated with any user group. Assign the Guest user to the Read-Only group, or any custom group if you need to restrict its rights (see the security warnings below).
  3. Repeat step 2 for each source on which you want to enable the Guest mode as its access rights are managed on a source per source basis.

Once enabled, the Guest mode is available at http://<linkurious>/guest (replace <linkurious> with the actual host and port of your server).

Security warnings

All people who have access to /guest will be able to browse the whole database (in read-only mode), even if nobody has explicitly shared a specific visualization with them. You may want to check who has access to /guest before enabling the Guest mode.

If you have assigned the Guest user to the Read Only built-in user group, all node categories and edge types will be available to see by the people who can access /guest. If the database contains sensitive data, you should create a custom group with limited rights on the data and assign the Guest user to that group.

Even if the Guest user is assigned to a group that has Write or Delete permissions, Guest mode users will not be able to write or edit data.

When initialized with a visualization, the Guest mode allows the user to expand the graph beyond the content of that visualization. Consider restricting the Guest user access rights if that is an issue.

Populating the Guest mode workspace

Accessing /guest directly will return an empty workspace.

Using parameters in the URL you can populate the Guest mode workspace with:

The Guest user sees the current state of a visualization. Any change to a visualization from an authenticated user will be automatically applied to the public link shared to the Guest user.

 Access control: Users, groups and rights

Linkurious relies on a role-based access control model based on node-categories and edge-types:

When users belong to several groups, it can happen that many access-rights are defined for a node-category (or edge-type). In this case, the more permissive access-right is granted.

Access-right example

Creating users

Only when authentication is enabled, use the Web user interface via the Admin > Users menu:

From this page, you can see all the users in Linkurious (both local and external users) and their groups in the current data-source:

To create a new user, click Create User.

Assigning users to groups

Click edit button next to a user. The following screen will appear:

Any user can belong to at most 1 built-in group, no limitations occurs for custom groups. Click + to add a new group to the user.

Any user can't edit its own groups.

Built-in groups

Linkurious defines several built-in groups. To learn more about them, click the inspect button next to a built-in group.

To create a new group, click Create Group.

Create a custom group

Define a name for the custom group. You can hover on the options to learn more about them.

On the next page, you can select the access-right for any given node-category:

Password hashing

Passwords of local users are hashed with the PBKDF2 algorithm and the following parameters:

External users

When using an external source for authentication, users are automatically created in Linkurious when they connect for the first time.

These shadow-users allow to store Linkurious specific data such as user preferences, groups and visualizations.

Passwords of external users are never stored inside Linkurious.

Group mapping

If an external source already organizes users in groups, it's possible to use this information to map automatically external groups to Linkurious groups. To do so, you have to set the access.externalUsersGroupMapping configuration key to be an object with the external group IDs as keys and the internal group IDs as values.

For example, if we want to provide group mapping for Microsoft Active Directory:

{ // under the access configuration key 
  // ... 
  "externalUsersGroupMapping": {
    "Administrators": 1 // any Active Directory admin is a Linkurious admin 
  }
  // ... 
}

For some identity providers the external group IDs is an actual name, for others is an ID:

To exclude some groups of users from logging in into Linkurious, set up a list of authorized groups in the configuration key access.externalUsersAllowedGroups.

{ // under the access configuration key 
  // ... 
  "externalUsersAllowedGroups": [
    "CN=Administrators,CN=Users,DC=linkurious,DC=local",
    "CN=Analysts,CN=Users,DC=linkurious,DC=local"
  ]
  // ... 
}

 Access control: LDAP / Active Directory

If Linkurious is connected to an LDAP service, users will be authenticated using the external service at each log-in.

If you have a LDAP service running in your network, you can use it to authenticate users in Linkurious.

Contact your network administrator to ensure that the machine where Linkurious is installed can connect to the LDAP service.

OpenLDAP

For OpenLDAP compatible providers, add an ldap section inside the access configuration.

Allowed options in access.ldap:

The bindDN and bindPassword are optional. If specified they will be used to bind to the LDAP server.

Example LDAP configuration:

"access": {
  // [...] 
  "ldap": {
    "enabled": true,
    "url": "ldap://ldap.forumsys.com:389",
    "bindDN": "cn=read-only-admin,dc=example,dc=com",
    "bindPassword": "password",
    "baseDN": ["dc=example,dc=com"],
    "usernameField": "uid",
    "emailField": "mail",
    "groupField": "group"
  }
}

Active Directory

For Microsoft Active Directory, add a msActiveDirectory section inside the access configuration.

Allowed options in access.msActiveDirectory:

Users can authenticate with their userPrincipalName or their sAMAccountName.

Use the domain configuration key to avoid your users to specify the domain part of their userPrincipalName. Use the netbiosDomain configuration key to avoid your users to specify the netbios domain part of their sAMAccountName.

Example Active Directory configuration:

"access": {
  // [...] 
  "msActiveDirectory": {
    "enabled": true,
    "url": "ldaps://ldap.lks.com:636",
    "baseDN": "dc=ldap,dc=lks,dc=com",
    "domain": "ldap.lks.com",
    "netbiosDomain": "LINKURIO",
    "tls": {
      "rejectUnauthorized": true
    }
  }
}

In alternative is possible to use your on premises Active Directory in conjunction with Azure Active Directory to provide SSO to your users. Please refer to Prerequisites for Azure AD Connect for more information and to SSO with Azure AD to know how to setup Azure AD as an identity provider.

 Access control: SSO with Azure AD

Linkurious supports Microsoft Azure Active Directory as an external authentication provider.

Configuration

To set up Linkurious authentication with Microsoft Azure Active Directory, follow these steps:

  1. Create a new app called Linkurious in Azure Active Directory on Azure Portal
  2. From the Azure Portal, obtain the following parameters:
    • authorizationURL, e.g. https://login.microsoftonline.com/60d78xxx-xxxx-xxxx-xxxx-xxxxxx9ca39b/oauth2/authorize
    • tokenURL, e.g. https://login.microsoftonline.com/60d78xxx-xxxx-xxxx-xxxx-xxxxxx9ca39b/oauth2/token
    • clientID, e.g. 91d426e2-xxx-xxxx-xxxx-989f89b6b2a2
    • clientSecret, e.g. gt7BHSnoIffbxxxxxxxxxxxxxxxxxxtyAG5xDotC8I=
    • tenantID, (optional, required only for group mapping) e.g. 60d78xxx-xxxx-xxxx-xxxx-xxxxxx9ca39b
  3. Add an oauth2 section inside the access section in linkurious/data/config/production.json

Example access.oauth2 configuration with Microsoft Azure Active Directory:

"access": {
  // [...] 
  "oauth2": {
    "enabled": true,
    "provider": "azure",
    "authorizationURL": "https://login.microsoftonline.com/XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX/oauth2/authorize",
    "tokenURL": "https://login.microsoftonline.com/XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX/oauth2/token",
    "clientID": "XXXXXXXX-XXX-XXXX-XXXX-XXXXXXXXXXXX",
    "clientSecret": "XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX",
    "azure": {
      "tenantID": "XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX"
    }
  }
}

 Access control: SSO with Google

Linkurious supports Google Suite (a.k.a. Google Apps) as an external authentication provider (with Single Sign-On).

Since Google Suite implements the OpenID Connect standard, it can be configured as an OpenID Connect provider.

Configuration

To set up Linkurious authentication with Google Suite, follow these steps:

  1. Create the credentials on your Google Developers console.
  2. From the portal, obtain the following parameters:
    • authorizationURL, e.g. https://accounts.google.com/o/oauth2/v2/auth
    • tokenURL, e.g. https://www.googleapis.com/oauth2/v4/token
    • clientID, e.g. 1718xxxxxx-xxxxxxxxxxxxxxxx.apps.googleusercontent.com
    • clientSecret, e.g. E09dQxxxxxxxxxxxxxxxxSN
  3. Add an oauth2 section inside the access section in linkurious/data/config/production.json

To limit the access to the Google accounts from your Google Suite domain, use the hd query parameter in the authorizationURL with your domain as value.

Example access.oauth2 configuration with Google Suite:

"access": {
  // [...] 
  "oauth2": {
    "enabled": true,
    "provider": "openidconnect",
    "authorizationURL": "https://accounts.google.com/o/oauth2/v2/auth?hd=YOUR_GSUITE_DOMAIN.COM",
    "tokenURL": "https://www.googleapis.com/oauth2/v4/token",
    "clientID": "XXXXXXXXXX-XXXXXXXXXXXXXXXX.apps.googleusercontent.com",
    "clientSecret": "XXXXXXXXXXXXXXXXXXXXXXX"
  }
}

 Access control: SSO with OpenID Connect

Linkurious supports any OpenID Connect compatible provider as external authentication providers.

What is OpenID Connect?

OpenID Connect is an identity layer on top of the OAuth2 protocol. It allows applications (like Linkurious) to verify the identity of End-User based on the authentication performed by an Authorization Server, as well as to obtain basic profile information about the End-User in an interoperable manner.

Configuration

To set up Linkurious authentication with an OpenID Connect provider, you need to obtain the following parameters from the provider:

Example access.oauth2 configuration with any OpenID Connect provider:

"access": {
  // [...] 
  "oauth2": {
    "enabled": true,
    "provider": "openidconnect",
    "authorizationURL": "https://accounts.google.com/o/oauth2/v2/auth",
    "tokenURL": "https://www.googleapis.com/oauth2/v4/token",
    "clientID": "XXXXXXXXXX-XXXXXXXXXXXXXXXX.apps.googleusercontent.com",
    "clientSecret": "XXXXXXXXXXXXXXXXXXXXXXX"
  }
}

Group mapping in OIDC

To set up group mapping in OpenID Connect is necessary to specify additional configuration keys:

For example if you want to set up OIDC with Okta:

"access": {
  // [...] 
  "oauth2": {
    "enabled": true,
    "provider": "openidconnect",
    "authorizationURL": "https://XXXXXXXXXX.oktapreview.com/oauth2/v1/authorize",
    "tokenURL": "https://XXXXXXXXXX.oktapreview.com/oauth2/v1/token",
    "clientID": "XXXXXXXXXXXXXXXXXXXXXXX",
    "clientSecret": "XXXXXXXXXXXXXXXXXXXXXXX",
    "openidconnect": {
      "userinfoURL": "https://XXXXXXXXXX.oktapreview.com/oauth2/v1/userinfo",
      "scope": "openid profile email groups",
      "groupClaim": "groups"
    }
}

 Access control: SSO with SAML2 / ADFS

Linkurious supports any SAML2 compatible provider as external authentication providers.

Configuration

To set up Linkurious authentication with a SAML2 provider, you need to obtain the following parameters from the provider:

groupAttribute is the attribute of the SAML response containing the array of groups a user belongs to.

Example access.saml2 configuration with any SAML2 provider:

"access": {
  // [...] 
  "saml2": {
    "enabled": true,
    "url": "https://example.com/adfs/ls",
    "identityProviderCertificate": "/Users/example/linkurious/saml.cert",
    "groupAttribute": "Groups"
  },
}

Assertion consumer service

To complete the login process, you need to configure your identity provider to return the SAML response to Linkurious at the following URL: http(s)://HOST:PORT/api/auth/sso/return.

ADFS Configuration

In particular, ADFS (Active Directory Federation Services) is a SAML2 provider that offers Single-Sign-On towards an Active Directory service.

To set up Linkurious authentication with ADFS, Linkurious has to be configured as a Relying Party Trust in ADFS.

 Alerts

Alerts are a way to watch your graph database for specific patterns and be notified when such pattern appears in the data. A simple triage interface let the users navigate through pattern matches and flag them as confirmed or dismissed.

Alerts are a supported only for Neo4j in Linkurious 2.6.7.

If you are interested in alerts from other graph vendors, please get in touch.

How alerts work

To create an alert, an administrator needs to:

Using the Run query button, you can preview results for the current query:

Additional columns can be configured using explicitly returned values, as done here with n.length.

Clicking Next, a secondary configuration page will ask for the name of the alert and the frequency at which the query will be run.

After configuring and enabling an alert, it can be accessed by users in the Alerts panel:

Opening an alerts lists all recent matches, sortable by any column:

Clicking on a match opens it for details:

 Alerts: Configuration

The maximum number of matches and the maximum execution time can be set in linkurious/data/config/production.json under the alerts key:

alerts: {
  maxMatchesLimit: 5000,
  maxRuntimeLimit: 600000,
  maxConcurrency: 1
}

The maxConcurrency key indicates how many alert queries can be run concurrently. This is useful when you configure multiple alerts and want to prevent them from running at the same time to limit load on the graph database server.

 Visualizations appearance

Linkurious allows you to modify the visualizations appearence by modifying the ogma.options settings.

To edit the Ogma settings, you can either use the Web user-interface or edit the configuration file located at linkurious/data/config/production.json.

Example configuration:

{
  "renderer": "webgl",
  "options": {
    "styles": {
      "node": {
        "nodeRadius": 5,
        "shape": "circle",
        "text": {
          "minVisibleSize": 24,
          "maxLineLength": 35,
          "backgroundColor": null,
          "font": "roboto",
          "color": "#000",
          "size": 14
        }
      },
      "edge": {
        "edgeWidth": 1,
        "shape": "arrow",
        "text": {
          "minVisibleSize": 4,
          "maxLineLength": 35,
          "backgroundColor": null,
          "font": "roboto",
          "color": "#000",
          "size": 14
        }
      }
    },
    "interactions": {
      "zoom": {
        "modifier": 1.382
      },
      "pan": {
      },
      "rotation": {
        "enabled": false
      }
    },
    "backgroundColor": "rgba(240, 240, 240, 0)"
  }
}

Supported ogma.options settings are available here.

In addition to what you find in the Ogma documentation, Linkurious allows the following extra configuration keys:

 Visualizations appearance: Default styles

In Linkurious Enterprise you can customize the default visual aspect of nodes and edges for new visualizations; your users can then jump head first into the exploration of the data.

Styles can be configured individually by each user using the Design panel. Default values can be configured for all users by an administrator.

Styles belong to one particular data-source. To set the default styles of a data-source, you have two options:

By default, every node category has a pre-assigned color.

Inside Default Styles, the nodes and the edges sections define the default styles for nodes and edges respectively.

A style rule has the following elements:

For example:

{
  "index": 3,
  "itemType": "COMPANY",
  "type": "is",
  "input": ["properties", "name"],
  "value": "linkurious",
  "style": {
    "color": "blue"
  }
}

The above rule will apply the style {"color": "blue"} to all nodes with category "COMPANY" where the name is "linkurious".

index has to be unique. It is currently required for technical reasons. This will be made more user-friendly in future releases.

Selectors

The selector is used to specify to which items the style is applied to. For example, you can configure all the "COMPANY" founded more than 12 years ago to have a particular style. To do so, we use a style rule with a range type and with value:

{
  ">": 12
}

The overall style rule will look like the following (assuming we want to color the nodes in red):

{
  "index": 3,
  "type": "range",
  "itemType": "COMPANY",
  "input": ["properties", "age"],
  "value": {
    ">": 12
  },
  "style": {
    "color": "red"
  }
}

For range queries you can use one or more among the following operators: >, <, >=, <=.

Supported selectors

{
  "type": "is",
  "input": ["properties", "name"],
  "value": "linkurious",
  // .. 
}

Additionaly to type, input and value you must always specify itemType to filter by node category or edge type except if type is any.

Styles

Colors

Set under the style property key an object with one key, color, e.g:

"style": {
  "color": "blue" // or "#0000FF", "rgba(0, 0, 255, 1)" 
}

The color style for nodes and edges has the same format.

Sizes

For nodes, set under the style property key an object with one key, size, e.g:

"style": {
  "size": "220%"
}

For edges it is quite similar: set under the style property key an object with one key, width, e.g:

"style": {
  "width": "220%"
}

Shapes

Set under the style property key an object with one key, shape.

For nodes, set the shape of the node. Possible values are: "circle" (default), "cross", "diamond", "equilateral", "square" or "star".

"style": {
  "shape": "star" // "circle", "cross", "diamond", "equilateral", "square" or "star" 
}

For edges, set the shape of the edge. Possible values are: "arrow" (default), "dashed", "dotted", "line" or "tapered".

"style": {
  "shape": "dotted" // "arrow", "dashed", "dotted", "line" or "tapered" 
}

Custom node icons

You can host your custom icons in Linkurious Enterprise itself by storing them in the folder located at linkurious/data/server/customFiles/icons.

Users will find them in the Design panel:

If you want to edit style rules manually, the style rules to access these images would look like:

"style": {
  "image": {
    "url": "/icons/company.png"
  }
}

Advanced custom node icons

Nodes can be filled with an image if one of their property is an URL to an image. Available image formats are PNG, JPG, GIF, or TIFF.

The following style will set an image:

Example:

"style": {
  "image": {
    "url": "http://example.com/img/company.png"
  }
}

To assign dynamically an image to a node, for example if the logo is stored in a node property called "logo_url", you just need to set the following style:

"style": {
  "image": {
    "url": {
      "type": "data",
      "path": [
       "properties",
       "logo_url" // change it to the property key where your image urls are stored 
      ]
    }
  }
}

If you want to resize your images in a node you can you use the additional properties scale, fit and tile, e.g.:

"style": {
  "image": {
    "url": ... // one of the above 
    "scale": 0.8, // scale the image in the node 
    "fit": false, // if true, fill the node with the image 
    "tile": false // if true, repeat the image to fill the node 
  }
}

Editing the default styles in the data-source page does automatically change captions for existing users for newly created visualizations. Existing visualizations are not touched.

 Visualizations appearance: Default captions

In Linkurious Enterprise, node and edge captions are the texts displayed next to a node or an edge.

Captions, as for Styles, can be configured individually by each user using the Design panel.

As for Styles, an administrator can set the default values from its own workspace or by editing them from the data-source configuration page.

Inside Default Captions, the nodes and the edges sections define the default captions for nodes and edges respectively.

Each of these two sections is an object where the keys are node categories or edge types and the values are objects with the following keys:

Example:

"defaultCaptions": {
  "nodes": {
    "CITY": {
      "active": true,
      "displayName": true,
      "properties": ["name"]
    },
    "COMPANY": {
      "active": true,
      "displayName": false,
      "properties": ["name", "country"]
    }
  },
  "edges": {
    "INVESTED_IN": {
      "active": true,
      "displayName": true,
      "properties": ["funded_month"]
    }
  }
}

Editing the default captions in the data-source page does automatically change captions for existing users for newly created visualizations. Existing visualizations are not touched.

 Visualizations appearance: Geographic tiles

Linkurious supports displaying nodes with geographic coordinates (latitude and longitude) on a map.

Users are able to switch a visualization to geo mode when geographic coordinates are available on at least one nodes of the visualization. The map tiles layer used in geo mode can be customized by users.

By default, Linkurious comes pre-configured with several geographical tile layers. Administrators change the available geographical tile layers by editing the leaflet section in the configuration file (linkurious/data/config/production.json). The leaflet key is an array of geographical tile layer configurations. Each entry has the following attributes:

Geographical tile layers and overlay layers can be found at https://leaflet-extras.github.io/leaflet-providers/preview/.

Example configuration:

"leaflet": [
  {
    "overlay": true,
    "name": "Stamen Toner Lines",
    "thumbnail": "",
    "urlTemplate": "http://stamen-tiles-{s}.a.ssl.fastly.net/toner-lines/{z}/{x}/{y}.png",
    "attribution": "Map tiles by <a href="http://stamen.com">Stamen Design</a>, <a href="http://creativecommons.org/licenses/by/3.0">CC BY 3.0</a> &mdash; Map data &copy; <a href="http://www.openstreetmap.org/copyright">OpenStreetMap</a>", 
    "subdomains": "abcd",
    "id": null,
    "accessToken": null,
    "minZoom": 2,
    "maxZoom": 20
  },
  {
    "name": "MapBox Streets",
    "thumbnail": "/assets/img/MapBox_Streets.png",
    "urlTemplate": "https://api.tiles.mapbox.com/v4/{id}/{z}/{x}/{y}.png?access_token={accessToken}",
    "attribution": "Map data &copy; <a href="http://openstreetmap.org">OpenStreetMap</a>, <a href="http://creativecommons.org/licenses/by-sa/2.0/">CC-BY-SA</a>, Imagery &copy <a href="http://mapbox.com">Mapbox</a>", 
    "subdomains": null,
    "id": "mapbox.streets",
    "accessToken": "pk.eyJ1Ijoic2hleW1hbm4iLCJhIjoiY2lqNGZmanhpMDAxaHc4bTNhZGFrcHZleiJ9.VliJNQs7QBK5e5ZmYl9RTw",
    "minZoom": 2,
    "maxZoom": 20
  }
]

 Audit trail

Audit trails are detailed logs about the operations performed on your graph databases by your users. Because they have the potential to take up a substantial amount of memory depending on the number of users and the operations performed, they are disabled by default.

Configuration

Audit trails can be enabled and configured in linkurious/data/config/production.json. They are found under the auditTrail key, which contains the following options:

Performances

Enabling the audit trail can impact performances negatively.

Here are options to consider to improve performances:

 Audit trail: Log format

The audit trail log files contain JSON lines in JSONL format.

You can easily bind a log-management system like Logstash to interpret them.

Each log line contains:

Lines are written to the log as follows:

{"mode":"WRITE","date":"2017-01-09T17:34:07.446Z","user":"simpleUser@example.com","sourceKey":"e8890b53","action":"createEdge","params":{"createInfo":{"source":4328,"target":4332,"type":"ACTED_IN","data":{"tata":"toto"}}},"result":{"edge":{"id":5958,"data":{"tata":"toto"},"type":"ACTED_IN","source":4328,"target":4332}}}
{"mode":"READ","date":"2017-01-09T17:34:07.478Z","user":"simpleUser@example.com","sourceKey":"e8890b53","action":"getNode","params":{"id":4330},"result":{"node":{"id":4330,"data":{"tagline":"Welcome to the Real World","title":"The Matrix","released":1999,"nodeNoIndexProp":"foo"},"categories":["Movie","TheMatrix"]}}}
{"mode":"READ","date":"2017-01-09T17:34:07.507Z","user":"simpleUser@example.com","sourceKey":"e8890b53","action":"getEdge","params":{"edgeId":5950},"result":{"edge":{"id":5950,"data":{"edgeNoIndexProp":"bar","roles":["Neo"]},"type":"ACTED_IN","source":4313,"target":4330}}}
{"mode":"READ WRITE","date":"2017-01-09T17:34:12.253Z","user":"user@linkurio.us","sourceKey":"e8890b53","action":"rawQuery","params":{"query":"MATCH (n:Person) RETURN n","dialect":"cypher"},"result":{"nodes":[{"id":4357,"data":{"born":1967,"name":"Andy Wachowski"},"categories":["Person"],"edges":[]},{"id":4359,"data":{"born":1967,"name":"Carrie-Anne Moss"},"categories":["Person"],"edges":[]},{"id":4360,"data":{"born":1954,"name":"James Cameron"},"categories":["Person"],"edges":[]},{"id":4361,"data":{"born":1964,"name":"Keanu Reeves"},"categories":["Person"],"edges":[]},{"id":4362,"data":{"born":1965,"name":"Lana Wachowski"},"categories":["Person"],"edges":[]},{"id":4364,"data":{"born":1901,"name":"Phillip Cameron"},"categories":["Person"],"edges":[]},{"id":4365,"data":{"born":1976,"name":"Sam Worthington"},"categories":["Person"],"edges":[]}]}}

Each line is a JSON objects in the following format (with logResult set to true):

{
  "mode": "WRITE",
  "date": "2017-01-09T17:34:07.446Z",
  "user": "simpleUser@example.com",
  "sourceKey": "e8890b53",
  "action": "createEdge",
  "params": {"createInfo":{"source":4328,"target":4332,"type":"ACTED_IN","data":{"tata":"toto"}}},
  "result": {"edge":{"id":5958,"data":{"foo":"bar"},"type":"BAZ","source":4328,"target":4332}}
}

The params key contains the parameters of the operation being performed. In this case, the user has run as createEdge action; params then contains the source and target IDs of the edge, as well as the ege type and a data key which holds the properties set on that edge.

result contains a JSON representation of the edge produced by the action. As can be seen, there is a substantial amount of duplication between the information in params and in data. This may not be a consideration with operations on single nodes, but with larger collections can mean that log size increases substantially. Consider the example below:

{
  "mode": "READ WRITE",
  "date": "2017-01-09T17:34:12.289Z",
  "user": "user@linkurio.us",
  "sourceKey": "e8890b53",
  "action": "rawQuery",
  "params": {
    "query": "MATCH (n1)-[r:DIRECTED]->(n2) RETURN n1, r",
    "dialect": "cypher"
  },
  "result":{"nodes":[{"id":4357,"data":{"born":1967,"name":"Andy Wachowski"},"categories":["Person"],"edges":[{"id":6009,"data":{},"type":"DIRECTED","source":4357,"target":4366},{"id":6010,"data":{},"type":"DIRECTED","source":4357,"target":4367},{"id":6011,"data":{},"type":"DIRECTED","source":4357,"target":4368}]},{"id":4358,"data":{"tagline":"Return to Pandora","title":"Avatar","released":1999},"categories":["Avatar","Movie"],"edges":[{"id":6034,"data":{},"type":"DIRECTED","source":4360,"target":4358}]},{"id":4360,"data":{"born":1954,"name":"James Cameron"},"categories":["Person"],"edges":[{"id":6034,"data":{},"type":"DIRECTED","source":4360,"target":4358}]},{"id":4362,"data":{"born":1965,"name":"Lana Wachowski"},"categories":["Person"],"edges":[{"id":6020,"data":{},"type":"DIRECTED","source":4362,"target":4366},{"id":6021,"data":{},"type":"DIRECTED","source":4362,"target":4367},{"id":6022,"data":{},"type":"DIRECTED","source":4362,"target":4368}]},{"id":4366,"data":{"tagline":"Welcome to the Real World","title":"The Matrix","released":1999,"nodeNoIndexProp":"foo"},"categories":["Movie","TheMatrix"],"edges":[{"id":6020,"data":{},"type":"DIRECTED","source":4362,"target":4366},{"id":6009,"data":{},"type":"DIRECTED","source":4357,"target":4366}]},{"id":4367,"data":{"tagline":"Free your mind","title":"The Matrix Reloaded","released":2003},"categories":["Movie","TheMatrixReloaded"],"edges":[{"id":6021,"data":{},"type":"DIRECTED","source":4362,"target":4367},{"id":6010,"data":{},"type":"DIRECTED","source":4357,"target":4367}]},{"id":4368,"data":{"tagline":"Everything that has a beginning has an end","title":"The Matrix Revolutions","released":2003},"categories":["Movie","TheMatrixRevolutions"],"edges":[{"id":6022,"data":{},"type":"DIRECTED","source":4362,"target":4368},{"id":6011,"data":{},"type":"DIRECTED","source":4357,"target":4368}]}]}
}

In this case, we've used a raw query to read nodes from the database without making any changes to them. The results of the query are returned to us in result, and include a substantial number of nodes and edges. To maximize the usefulness of audit trails and to minimize their footprint, it might be advisable to exclude unnecessary data, including passive queries such as these. Disabling logResult and setting mode to "w" (logging only operations which perform write operations to the database) is one strategy for accomplishing this.