GREASE Administration: Up & Running¶

Initial Installation¶

Via PyPi: The Traditional Way¶

GREASE is built like any other traditional python software, as a package. This means many things but luckily for the systems administrator it means we traverse the typical PyPi pipeline allowing for you to specify your version and allow for updates to be pretty easy. Here is the installation steps:

Run pip install tgt_grease
Setup your configuration file following

From Source on GitHub: Because you’re cool like that¶

GREASE is developer friendly! We’re always looking for new ways to bring joy to our guests as well as our developers. To install GREASE via source follow these steps:

Download the repo either via Git or HTTP from GitHub
Enter the project folder you created
Run python setup.py install
- NOTE: We recommend for all use cases to use a virtual environment
Setup your configuration file

Understanding & Configuring Your System¶

There are multiple definitions for configuration in GREASE. The primary ones are:

Node Configuration: This refers to the local server’s configuration for things like MongoDB credentials & resource limits
Cluster Configuration: This refers to the configuration of Job Servers inside of the cluster
Prototype Configuration: These are the configurations for prototypes, things such as sourcing & detection

Node Configuration¶

Node configuration is the local file the running instance uses to execute GREASE operations. It is stored in a file called grease.conf.json. This is stored in the GREASE directory. On Unix-like operating systems is found at /opt/grease/ and for Windows C:\\grease\\. You can override this behavior using the environment variable GREASE_DIR.

The default configuration looks like this:

{
    'Connectivity': {
        'MongoDB': {
            'host': 'localhost',
            'port': 27017
        }
    },
    'Logging': {
        'mode': 'filesystem',
        'verbose': False,
        'trace': False,
        'foreground': False,
        'file': Configuration.greaseDir + 'log' + os.sep + 'grease.log'
    },
    'Notifications': {
        'HipChat': {
            'enabled': False,
            'token': None,
            'room': None
        }
    },
    'Configuration': {
        'dir': Configuration.greaseDir + 'etc' + os.sep
    },
    'Sourcing': {
        'dir': Configuration.greaseDir + 'etc' + os.sep,
        'source': None,
        'config': None,
        'mock': False
    },
    'Import': {
        'searchPath': [
            'tgt_grease.router.Commands',
            'tgt_grease.enterprise.Prototype',
            'tgt_grease.management.Commands',
            'tgt_grease.enterprise.Sources',
            'tgt_grease.enterprise.Detectors',
            'tgt_grease.core',
            'tgt_grease'
        ]
    },
    "NodeInformation": {
        "ResourceMax": 95,
        "DeduplicationThreads": 150
    },
    "Additional": {}
}

Lets go through each key and the properties below, what they control and some values you may want to use.

Connectivity: This is the store for details around connectivity
- MongoDB: This key is the key used to find connection details about the central database
  
  Key value type default
  
  host str localhost
  
  port int 27017
  
  username str
  
  password str
  
  db str
Logging: Logging configuration information
- mode: only supports filesystem logging currently to the log file
- verbose: Can either be True or False. Setting it to True would print any message where the verbose flag was passed. Note, the only internal system of GREASE that utilizes verbose is deduplication. The rest is in tracing
- trace: Can either be True or False. This enables tracing from within GREASE. This will show a “stream of consciousness” in the log files.
- foreground: Can either be True or False. True would print log messages to stdout as well as a log file
- file: Log file to write messages to
Notifications: Stores information about notification channels. All channels will need at least one key, “enabled” with a boolean True/False value to enable or disable the channel. All other keys are dependent on the notification channel
Configuration: This section contains information about this node’s prototype configurations
- dir: A directory string on where to load configurations from
Sourcing: This section contains information about this node’s sourcing prototype configuration
- dir: A directory string on where to load configurations from
- source: A string defaulted to null that if provided sourcing will focus only on prototype configurations from that source to get source data from
- config: A string defaulted to null that if provided sourcing will focus only that prototype configuration
- mock: A boolean value which when enabled will attempt to source mocking data dependent from the prototype configurations
Import: This section holds information about the import system
- searchPath: A list of strings of packages to attempt loading commands from
NodeInformation: This section controls how GREASE performs on the Node
- ResourceMax: Integer that GREASE uses to ensure that new jobs or processes are not spun up if memory or CPU utilization exceed this limit
- DeduplicationThreads: This integer is how many threads to keep open at one time during deduplication. On even the largest source data sets the normal open threads is 30 but this provides a safe limit at 150 by default
Additional: Unused currently but can be used for additional user provided configuration

Cluster Configuration¶

Cluster configuration is stored in the MongoDB collection JobServer. Check the The GREASE Data Models for more information about what is stored here.

Prototype Configuration¶

Prototype configuration is stored in the MongoDB collection Configuration, in the filesystem or located in the package. Check the The GREASE Data Models for more information about what is stored here and the schema.

Installing the Daemon¶

Installing the GREASE daemon is super easy. Be sure you are logged in or running a console with administrative privileges. Now install the daemon on all supported platforms by running grease daemon install. On Unixlike systems you should now have a Systemd Service installed with the service file being stored at /etc/systemd/system/grease.service and for Launchd at /Library/LaunchDaemons/net.grease.daemon.plist. For windows you will have a new service installed.

You can now control the operations of your cluster!

Key	value type	default
host	str	localhost
port	int	27017
username	str
password	str
db	str