Installing CloudShark via Docker

CloudShark in Docker

Starting with CloudShark 3.9.0, users are able to deploy in a Docker environment. This document provides an overview of the environment and a sample docker-compose.yaml file that may be used as a starting point for your own deployment. These settings are intended to be used as an example for your container management platform of choice.

The CloudShark Base Image

All of the CloudShark services run on top of our base image. A download link to this image must be provided to you by Technical Support. Once you have downloaded it, it can be loaded into your docker environment.

$ docker load < cloudshark-3.9.0-05192021.tar.gz

The following services are run from this one image:

  • web
  • auto-importer
  • search-worker
  • traceframe

Shared Volumes

The CloudShark services all must share the following volume in order for them to store and have access to the underlying pcap data.

  • /usr/cloudshark/data

Additionally, the license file and an auto-import directory should be made available across all the CloudShark services:

  • /usr/cloudshark/etc/cloudshark.lic
  • /autoimport

Environment Variables

The CloudShark services use the following environment variables to control how they talk to each other, and how they connect to the required 3rd-party services outlined below.

DATABASE_URL=mysql://<user>:<password>@<db-addr>/<db-name>
REDIS_URL=redis://<redis-addr>/
MEMCACHE=<memcache-addr>:11211
TRACEFRAME=<traceframe-addr>:6783

3rd-Party / Open Source Services

The following 3rd-party or Open Source services are required for CloudShark to run. These may be replaced with vendor-specific instances that are compatible. For example, you may replace MariaDB with Amazon AuroraDB as long as it is compatible with MariaDB/MySQL.

  • memcache:1.6.9
  • redis:3.2
  • mariadb:10

The database service must have a database created that has full read-write permissions for the user specified in the environment variable above.

Caveats

There are a few known issues and environment-specific things that need to be done outside of this guide.

Auto-Importer

After adding or removing an Auto-Import location from within the CloudShark web interface, the `auto-import` service must be restarted from the container management system for the changes to take effect.

Periodic Jobs

In order for Auto-Delete and Quota Reset to happen automatically, periodic jobs must be scheduled to run from outside the container. These are currently not available and are returning in CloudShark 3.9.1.

Sample docker-compose.yaml

Below is our sample docker-compose.yaml file showing how to configure all of these services together. While Docker does not recommend using compose in production, it still serves as a concrete, easy to translate example of how all the service interact.

Note, we've used YAML anchors to extract the shared environment and volumes sections.

version: "3.8"
x-env:
  &default-env
    - DATABASE_URL=mysql://root:qacafe123@mariadb/cloudshark
    - REDIS_URL=redis://redis/
    - TRACEFRAME=traceframe:6783
    - MEMCACHE=memcached:11211

x-volumes:
  &default-volumes
      - ${PWD}/data:/usr/cloudshark/data
      - ${PWD}/license/cloudshark.lic:/usr/cloudshark/etc/cloudshark.lic
      - ${PWD}/autoimport:/autoimport

services:
  web:
    depends_on:
      - mariadb
      - memcached
      - redis
      - traceframe
    image: cloudshark:3.9.0
    ports:
      - "9292:9292"
    environment: *default-env
    volumes: *default-volumes

  memcached:
    image: memcached:1.6.9

  redis:
    image: redis:3.2

  mariadb:
    image: mariadb:10
    volumes:
      - ${PWD}/db:/var/lib/mysql
    environment:
      - MYSQL_ROOT_PASSWORD=qacafe123
      - MYSQL_DATABASE=cloudshark

  search-worker:
    image: cloudshark:3.9.0
    environment: *default-env
    volumes: *default-volumes
    working_dir: /usr/cloudshark/app
    entrypoint: ["/usr/cloudshark/ruby/bin/ruby", "/usr/cloudshark/app/bin/search-worker"]

  auto-importer:
    image: cloudshark:3.9.0
    environment: *default-env
    volumes: *default-volumes
    working_dir: /usr/cloudshark/app
    entrypoint: ["/usr/cloudshark/app/bin/autoimport", "--sock", "web:9292"]

  traceframe:
    image: cloudshark:3.9.0
    expose:
      - "6783"
    volumes: *default-volumes
    entrypoint: ["/usr/cloudshark/bin/tf_service", "-sharkd", "/usr/cloudshark/bin/sharkd", "-path", "/usr/cloudshark/data", "-profiles", "/usr/cloudshark/data", "-idle", "55s", "-addr", ":6783"]

Note

If you are using the above with docker-compose, the first time you start the environment, you will need to run the MariaDB container first by itself.

$ docker-compose up --no-start
$ docker-compose start mariadb

Once it is running, the default database name will be automatically created. Then you can start the rest:

$ docker-compose up