Workbench API BETA (2.9.39-hotfix-02)

Introduction

URL: https://workbench.expel.io/api/v2

The Workbench Public API gives organizations the means to access and manipulate their Expel Workbench data using custom API clients. Want to extract information into your reporting system? Use this API! Have some innovative data science idea for analyzing Expel alert evidence? Use this API! Want to write your own custom browser plugin to make your team's analysts more efficient? Use this API!


The majority of the Workbench API implements the jsonapi spec.

jsonapi-server Post Processing

Only the routes that do not follow the JSON:API spec have been defined in the 'CUSTOM ROUTES' section below.

Authentication

User authentication

User authentication is accomplished using the /login API route. This route is used to pass user authentication credentials to the API. If authentication is successful, the /login returns a temporary (default: 13 hours) Bearer Token (access_token) that can be used to access the API.

The request body requires a flat json document with three properties: username, password, and an otp.

    curl 'https://workbench.expel.io/api/v2/login' \
        -X POST \
        -H 'Content-Type: application/json' \
        --data-binary '{ "username": "sample_user@expel.io", "password": "secret", "otp": "123456" }'
    {
      "access_token": "<access_token>",
      "created_at": 1532537770,
      "expires_in": 46800,
      "token_type": "bearer",
      "user_id": "0796cba8-3c96-4984-8d51-22222381f870",
      "username": "sample_user@expel.io",
      "role": "expel_admin",
      "organization_id": "2ca75135-5a84-4298-b8ba-b5e0fd6fb576",
      "realm": "public"
    }

Note: as an alternative to the otp property, you can pass an X-ExpelInc-Otp http request header with the otp token. This is useful when interfacing with single login frameworks

The otp property or X-ExpelInc-Otp header must be populated with the correct 6 digit two-factor authentication code (e.g. Google Auth code) for authentication to succeed. If the Workbench API is not given an OTP token, authentication will fail with a 401 with the response header X-ExpelInc-Otp set to required.

After authentication, the rest of the API may be accessed using the Bearer Token returned in the access_token field. Pass that token back to the API in the Authorization header using Bearer as the credential type.

    curl 'https://workbench.expel.io/api/v2/expel_alerts' \
        -H 'Authorization: Bearer <access_token>'

Browser authentication

Browser authentication uses the same user authentication mechanism described in the previous sections of this document.
However, if you wish for the Workbench API to set a browser cookie in addition to a Bearer Token, you can pass the
optional X-Accepts-Auth-Cookie http request header set to true.

    curl 'https://workbench.expel.io/api/v2/login' \    
        -D - -o /dev/null \    
        -X POST \    
        -H 'Content-Type: application/json' \    
        -H 'X-Accepts-Auth-Cookie: true' \    
        --data-binary '{ "username": "sample_user@expel.io", "password": "secret", "otp": "123456" }'    
    . . .    
    Set-Cookie: token=<access_token>; Path=/; Expires=Thu, 26 Jul 2018 06:05:41 GMT; HttpOnly; Secure    
    . . .    

That cookie will then be used to authenticate all future API calls in a browser. The cookie will expire in tandem with
the access_token.

API key authentication

API keys are obtained through your Expel Engagement Manager. The API key amounts to a Bearer access_token without an expiration that is tied to a specific API client (as opposed to a specific user). This is particularly useful for daemon services, automated scripts, ETL jobs, etc.
API key Bearer tokens are used in the same manner as the temporary user Bearer tokens as described in the previous sections of this document.

    curl 'https://workbench.expel.io/api/v2/expel_alerts' \
        -H 'Authorization: Bearer <api_key>'

CRUD operations

GET /:resource

See jsonapi spec for more details of request and response formatting of this request.

Use this route to query or list records of a particular resource type (e.g. investigations, expel_alerts, security_devices, etc). All common query parameters may be used on this route. See Common Query Parameters for information on the available query parameters.

    # querying the expel_alerts resource
    curl 'https://workbench.expel.io/api/v2/expel_alerts` \
        -H 'Authorization: Bearer <access_token>'

GET /:resource/:id

See jsonapi spec for more details of request and response formatting of this request.

Use this route to retrieve a specific document. The include query parameter may be used on this route.

    # Get a specific investigation, and include its related expel_alerts in the response.
    curl 'https://workbench.expel.io/api/v2/investigations/10e7bfac-27ff-4381-a0ec-5c479b73342b?include=expel_alerts' \
        -H 'Authorization: Bearer <access_token>'

POST /:resource

See jsonapi spec for more details of request and response formatting of this request.

Use this route to create a new document of a particular resource type.

    # Create a new user account
    curl 'https://workbench.expel.io/api/v2/user_accounts' \
        -X POST \
        -H 'Authorization: Bearer <access_token>' \
        -H 'Content-Type: application/json' \
        --data-binary '{
            "data": {
                "type": "user_accounts",
                "attributes": {
                    "display_name": "Sample User",
                    "email": "sample_user@expel.io",
                    "role": "organization_analyst"
                }
            }
        }'

PATCH /:resource/:id

See jsonapi spec for more details of request and response formatting of this request.

Use this route to update fields and/or overwrite relationships on a specific document. The include query parameter may be used on this route to include related resources in the response.

    # Change the title of an investigation AND set (overwrite!) the expel_alerts relationship
    curl 'https://workbench.expel.io/api/v2/investigations/155901f0-d5b1-4158-a1a1-85370c6db07f' \
        -X PATCH
        -H 'Authorization: Bearer <access_token>' \
        -H 'Content-Type: application/json' \
        --data-binary '{
            "data": {
                "type": "investigations",
                "attributes": {
                    "title": "Sample Investigation"
                },
                "relationships": {
                    "expel_alerts": {
                        "data": [
                            { "id": "d3d4edaa-8acc-40f5-bc0a-5d40c910cac0", "type": "expel_alerts" },
                            { "id": "80f72c0f-6f03-421c-8436-61fc9ca6c356", "type": "expel_alerts" }
                        ]
                    }
                }
            }
        }'  

DELETE /:resource/:id

See jsonapi spec for more details of request and response formatting of this request.

Use this route to delete a specific document.

    # Delete a timeline entry 
    curl 'https://workbench.expel.io/api/v2/timeline_entries/ddd488d8-d0e4-4cbe-bfba-aa3e75a9d103' \
        -X DELETE
        -H 'Authorization: Bearer <access_token>'

Relationships

GET /:resource/:id/:relation

See jsonapi spec for more details of request and response formatting of this request.

Use this route to query or list records related to a particular known record. All common query parameters may be used on this route. See Common Query Parameters for information on the available query parameters.

    # querying all expel_alerts related to investigation with the id of 6e8def26-a645-49b6-9688-ed2fd5385726
    curl 'https://workbench.expel.io/api/v2/investigations/6e8def26-a645-49b6-9688-ed2fd5385726/expel_alerts` \
        -H 'Authorization: Bearer <access_token>'

This route is used to query actual related resource records and only offers a GET method. It's behavior is similar to the GET /:resource route. If you are looking to get information about (or modify) the actual relationship links between records, see the various GET|POST|PATCH|DELETE /:resource/:id/relationship/:relationship routes.

GET /:resource/:id/relationships/:relation

See jsonapi spec for more details of request and response formatting of this request.

Use this route to return the meta information about a specific relationship. This route does NOT return the actual related records (see GET /:resource/:id/:relation for that). You can use the ?include=<relation> query parameter to get the records returned in this call, but this is not recommended as the include query parameter is resource intensive, is not paginated, and does not support filtering/sorting. Instead, use this route to discover links and meta information about the relationship.

    # querying information about the vendor_alerts relationship of a specific expel_alert
    curl 'https://workbench.expel.io/api/v2/expel_alerts/27340b1e-6f3f-43f1-9ba5-d7c1d9d0dcae/relationships/vendor_alerts` \
        -H 'Authorization: Bearer <access_token>'

POST /:resource/:id/relationships/:relation

See jsonapi spec for more details of request and response formatting of this request.

Use this route to append new records to an existing relationship on a specific resource.


    # see what expel_alerts are related to a specific investigation
    curl 'http://workbench.expel.io/api/v2/investigations/70f5c8c9-5b7f-4295-a1e6-0e6e4dfa8c6e/expel_alerts?fields%5Bexpel_alerts%5D=id' \
        -H 'Authorization: Bearer <access_token>' \
        | jq '.data[] | .id'
    "a61863e5-e15f-4586-9612-a40bf7e2ab64"
    "20bce40b-654a-4830-91a2-701524b0668a"

    # Append two new expel_alerts to an investigation
    curl 'https://workbench.expel.io/api/v2/investigations/70f5c8c9-5b7f-4295-a1e6-0e6e4dfa8c6e/relationships/expel_alerts' \
        -X POST \
        -H 'Authorization: Bearer <access_token>' \
        -H 'Content-Type: application/json' \
        --data-binary '{
            "data": [
                { "type": "expel_alerts", "id": "6ff0cc08-23b2-4afd-9e98-6aa072c621d5" },
                { "type": "expel_alerts", "id": "6084b93c-506e-4594-87e9-1c0198045b55" }
            ]
        }'

    # check to see if the relationships were appended.  Note that the expel_alerts that were already related to this 
    # investigation are still there.
    curl 'http://workbench.expel.io/api/v2/investigations/70f5c8c9-5b7f-4295-a1e6-0e6e4dfa8c6e/expel_alerts?fields%5Bexpel_alerts%5D=id' \
        -H 'Authorization: Bearer <access_token>' \
        | jq '.data[] | .id'
    "a61863e5-e15f-4586-9612-a40bf7e2ab64"
    "20bce40b-654a-4830-91a2-701524b0668a"
    "6ff0cc08-23b2-4afd-9e98-6aa072c621d5"
    "6084b93c-506e-4594-87e9-1c0198045b55"

PATCH /:resource/:id/relationships/:relation

See jsonapi spec for more details of request and response formatting of this request.

Use this route to overwrite a relationship of a specific resource.


    # see what expel_alerts are related to a specific investigation
    curl 'http://workbench.expel.io/api/v2/investigations/70f5c8c9-5b7f-4295-a1e6-0e6e4dfa8c6e/expel_alerts?fields%5Bexpel_alerts%5D=id' \
        -H 'Authorization: Bearer <access_token>' \
        | jq '.data[] | .id'
    "a61863e5-e15f-4586-9612-a40bf7e2ab64"
    "20bce40b-654a-4830-91a2-701524b0668a"

    # Overwrite the expel_alerts related to this investigation
    curl 'https://workbench.expel.io/api/v2/investigations/70f5c8c9-5b7f-4295-a1e6-0e6e4dfa8c6e/relationships/expel_alerts' \
        -X PATCH \
        -H 'Authorization: Bearer <access_token>' \
        -H 'Content-Type: application/json' \
        --data-binary '{
            "data": [
                { "type": "expel_alerts", "id": "6ff0cc08-23b2-4afd-9e98-6aa072c621d5" },
                { "type": "expel_alerts", "id": "6084b93c-506e-4594-87e9-1c0198045b55" }
            ]
        }'

    # check to see if the relationships were appended.  Note that the expel_alerts that were already related to this 
    # investigation have been overwritten!
    curl 'http://workbench.expel.io/api/v2/investigations/70f5c8c9-5b7f-4295-a1e6-0e6e4dfa8c6e/expel_alerts?fields%5Bexpel_alerts%5D=id' \
        -H 'Authorization: Bearer <access_token>' \
        | jq '.data[] | .id'
    "6ff0cc08-23b2-4afd-9e98-6aa072c621d5"
    "6084b93c-506e-4594-87e9-1c0198045b55"

This route can also be used to set parent relationships. Note that when setting a parent relationship like this, "data" is NOT an array, since there can only ever be a single value.

    # Establish a parent security_device for an existing security_device (self-referencing m:1 relationship)

    curl 'https://workbench.expel.io/api/v2/security_devices/4819bb1e-bad4-4b9b-a154-7a9aac9654a2/relationships/parent_security_device' \
        -X PATCH \
        -H 'Authorization: Bearer <access_token>' \
        -H 'Content-Type: application/json' \
        --data-binary '{
            "data": { "type": "security_device", "id": "1dd4ee04-792a-4052-9968-66cf347fe1b7" }
        }'

DELETE /:resource/:id/relationships/:relation

See jsonapi spec for more details of request and response formatting of this request.

Use this route to unlink specific records in a relationship. Note that unlike traditional HTTP DELETE calls, this call requires a body. Only the records listed in the body will be removed from the relationship.

Tip: If you want to remove all records from the relationship, use PATCH /:resource/:id/relationships/:relation with an empty array!

    # see what expel_alerts are related to a specific investigation
    curl 'http://workbench.expel.io/api/v2/investigations/70f5c8c9-5b7f-4295-a1e6-0e6e4dfa8c6e/expel_alerts?fields%5Bexpel_alerts%5D=id' \
        -H 'Authorization: Bearer <access_token>' \
        | jq '.data[] | .id'
    "a61863e5-e15f-4586-9612-a40bf7e2ab64"
    "20bce40b-654a-4830-91a2-701524b0668a"
    "6ff0cc08-23b2-4afd-9e98-6aa072c621d5"
    "6084b93c-506e-4594-87e9-1c0198045b55"

    # Remove two expel_alerts from this investigation
    curl 'https://workbench.expel.io/api/v2/investigations/70f5c8c9-5b7f-4295-a1e6-0e6e4dfa8c6e/relationships/expel_alerts' \
        -X DELETE \
        -H 'Authorization: Bearer <access_token>' \
        -H 'Content-Type: application/json' \
        --data-binary '{
            "data": [
                { "type": "expel_alerts", "id": "6ff0cc08-23b2-4afd-9e98-6aa072c621d5" },
                { "type": "expel_alerts", "id": "6084b93c-506e-4594-87e9-1c0198045b55" }
            ]
        }'

    # check to see if the relationships were appended.  Note that the expel_alerts that were already related to this 
    # investigation are still there.
    curl 'http://workbench.expel.io/api/v2/investigations/70f5c8c9-5b7f-4295-a1e6-0e6e4dfa8c6e/expel_alerts?fields%5Bexpel_alerts%5D=id' \
        -H 'Authorization: Bearer <access_token>' \
        | jq '.data[] | .id'
    "a61863e5-e15f-4586-9612-a40bf7e2ab64"
    "20bce40b-654a-4830-91a2-701524b0668a"

Common query parameters

?include=<relation>[,<relation>[...]]

The include query parameter allows you to specify which relationship records you want included in the response. This is useful when you are querying for a record, and want to resolve specific relationship data without making multiple calls.

    # request a specific investigation, and include that investigation's organization and create/update actors in the response.
    curl 'http://workbench.expel.io/api/v2/investigations/aa8492a2-b1f2-4ccb-a4dd-ff76d4903ee9?include=organization,created_by,updated_by' \
        -H 'Authorization: Bearer <access_token>'

Using the include query parameter will have the following effects on the API's response:

  1. The data element of the included 1:m ("has many") relationships will be populated with the array of id's
     { 
         "data": {
             . . . 
             "relationships": { 
                 "<relation>": {
                     . . . 
                     "data": [ 
                         { "type": "<resource>", "id": "<uuid>" },
                         { "type": "<resource>", "id": "<uuid>" },
                         . . .
                     ]
                 }
             }
         },
         "included": [
           . . .
         ]
     }
  2. The included array at the top level of the response will include the records references in the relationships sections across all returned records.
         { 
             "data": {
                 . . .
             },
             "included": [
                 {
                     "type": "<resource>",
                     "id": "<uuid>"
                     "attributes": {
                         . . .
                     },
                     "relationships": {
                         . . .
                     }
                 },
                 {
                     "type": "<resource>",
                     "id": "<uuid>"
                     "attributes": {
                         . . .
                     },
                     "relationships": {
                         . . .
                     }
                 },
                 . . .
             ]
         }

Important performance limitations

  • The include query parameter can be resource intensive.
  • The records returned by the include query parameter cannot be paginated. It's all or nothing.
  • The records returned by the include query parameter cannot be filtered or sorted.

Because of these limitations, it is not recommended to use include when listing multiple records (GET /:resource) that are likely to have a large number of related records (e.g. /investigations?include=expel_alerts or /expel_alerts?include=vendor_alerts). Instead, in cases like these, it may be more efficient to query the relationships you're interested in using separate calls or using includes only for specific records.


    # query all investigations and return all their expel_alerts (NOT RECOMMENDED!  all expel_alerts associated with all
    # investigations returned for each page will be returned without pagination)
    curl 'http://workbench.expel.io/api/v2/investigations?include=expel_alerts
                -H 'Authorization: Bearer <access_token>' 

    # query all expel alerts for a specific investigation using includes (warning, no pagination on the expel_alerts)
    curl 'http://workbench.expel.io/api/v2/investigations/aa8492a2-b1f2-4ccb-a4dd-ff76d4903ee9?include=expel_alerts
            -H 'Authorization: Bearer <access_token>' 

    # query all expel alerts for a specific investigation using relationships (with pagination, filtering, and sorting available)
    curl 'http://workbench.expel.io/api/v2/investigations/aa8492a2-b1f2-4ccb-a4dd-ff76d4903ee9/expel_alerts
                -H 'Authorization: Bearer <access_token>'

?sort=[+|-]<field>[,[+|-]<field>[...]]

The sort query parameter allows you to sort by a particular attribute of a resource. Each field may be prefixed by a + or - to signify ascending or descending sorts respectively.

    # sort investigations ascending by created_at date.
    curl 'http://workbench.expel.io/api/v2/investigations?sort=created_at
        -H 'Authorization: Bearer <access_token>'

    # sort investigations descending by created_at date
    curl 'http://workbench.expel.io/api/v2/investigations?sort=-created_at
            -H 'Authorization: Bearer <access_token>'

    # sort investigations by title, and then descending by created_at date
    curl 'http://workbench.expel.io/api/v2/investigations?sort=title,-created_at
            -H 'Authorization: Bearer <access_token>'

Note:

  • Some columns are derived from other data in the system, and may not be sortable.
  • There is currently no default sort on any resource, so sorting is highly recommended when paginating data.

?page[limit|offset]=<value>

?filter[<field>]=[<operator>]<value>

?flag[<variable>]=<value>

In addition to filter query parameter, as specified by the jsonapi spec, our API supports a custom API query parameter of flags that allows callers to pass variables to the backend. Flags are defined on a resource by resource basis, and will alter the behavior of a give API call.

flag[scope]

The scope flag enables callers to specify scopes for resources backed by Sequelize. When adding a scope that should be accessible from the API, after adding the scope to the scopes object in the resource model definition, add the scope's name to the api_scopes array to allow it to be accessible via API. You can specify the scope as a comma-delimited string (e.g. flag[scope]=firstScope,secondScope) or as an array (e.g. flag[scope][]=firstScope&flag[scope][]=secondScope).

Examples

  • remediation_actions?flag[scope]=assigned_to_expel_but_no_user_assigned: Remediation actions where the assigned_to_org_id === EXPEL_ORGANIZATION_ID && assigned_to_user_id IS NULL.
  • expel_alerts?flag[scope]=assigned_to_expel_but_no_user_assigned: Expel alerts where assigned_to_org_id === EXPEL_ORGANIZATION_ID && assigned_to_user_id IS NULL.
  • investigations?flag[scope]=assigned_to_expel_but_no_user_assigned: Investigations where assigned_to_org_id === EXPEL_ORGANIZATION_ID && assigned_to_user_id IS NULL.
  • investigative_actions?flag[scope]=assigned_to_expel_but_no_user_assigned: Investigative actions where assigned_to_org_id === EXPEL_ORGANIZATION_ID && assigned_to_user_id IS NULL. Will check analysis_assigned_to_org_id/analysis_assigned_to_user_id if the status of the action is in an analysis state (e.g. !RUNNING).

Other scopes

  • expel_alerts
    • assigned_to_organization: Expel alerts where assigned_to_org_id === organization_id.
    • assigned_to_organization_user: Expel alerts where assigned_to_org_id === organization_id AND assigned_to_user_id IS NOT NULL.
    • assigned_to_organization_but_no_user_assigned: Expel alerts where assigned_to_org_id === organization_id AND assigned_to_user_id NOT NULL.
    • assigned_to_expel: Expel alerts where assigned_to_org_id === EXPEL_ORGANIZATION_ID.
    • assigned_to_user_account: Expel alerts where assigned_to_org_id === EXPEL_ORGANIZATION_ID AND assigned_to_user_id IS NOT NULL.
    • assigned_to_expel_but_no_user_assigned: Expel alerts where assigned_to_org_id === EXPEL_ORGANIZATION_ID AND assigned_to_user_id NOT NULL.
    • assigned_to_user: Expel alerts assigned to the calling user (e.g. "Assigned to me")
  • expel_alert_grid
    • assigned_to_organization: Expel alerts (grid view) where assigned_to_org_id === organization_id.
    • assigned_to_organization_user: Expel alerts (grid view) where assigned_to_org_id === organization_id AND assigned_to_user_id IS NOT NULL.
    • assigned_to_organization_but_no_user_assigned: Expel alerts (grid view) where assigned_to_org_id === organization_id AND assigned_to_user_id NOT NULL.
    • assigned_to_expel: Expel alerts (grid view) where assigned_to_org_id === EXPEL_ORGANIZATION_ID.
    • assigned_to_user_account: Expel alerts (grid view) where assigned_to_org_id === EXPEL_ORGANIZATION_ID AND assigned_to_user_id IS NOT NULL.
    • assigned_to_expel_but_no_user_assigned: Expel alerts (grid view) where assigned_to_org_id === EXPEL_ORGANIZATION_ID AND assigned_to_user_id NOT NULL.
    • assigned_to_user: Expel alerts (grid view) assigned to the calling user (e.g. "Assigned to me")
  • investigations
    • assigned_to_organization: Investigations where assigned_to_org_id === organization_id.
    • assigned_to_organization_user: Investigations where assigned_to_org_id === organization_id AND assigned_to_user_id IS NOT NULL.
    • assigned_to_organization_but_no_user_assigned: Investigations where assigned_to_org_id === organization_id AND assigned_to_user_id NOT NULL.
    • assigned_to_expel: Investigations where assigned_to_org_id === EXPEL_ORGANIZATION_ID.
    • assigned_to_user_account: Investigations where assigned_to_org_id === EXPEL_ORGANIZATION_ID AND assigned_to_user_id IS NOT NULL.
    • assigned_to_expel_but_no_user_assigned: Investigations where assigned_to_org_id === EXPEL_ORGANIZATION_ID AND assigned_to_user_id NOT NULL.
    • assigned_to_user: Investigations assigned to the calling user (e.g. "Assigned to me")
  • remediation_actions
    • assigned_to_organization: Actions where assigned_to_org_id === organization_id.
    • assigned_to_organization_user: Actions where assigned_to_org_id === organization_id AND assigned_to_user_id IS NOT NULL.
    • assigned_to_organization_but_no_user_assigned: Actions where assigned_to_org_id === organization_id AND assigned_to_user_id NOT NULL.
    • assigned_to_expel: Actions where assigned_to_org_id === EXPEL_ORGANIZATION_ID.
    • assigned_to_user_account: Actions where assigned_to_org_id === EXPEL_ORGANIZATION_ID AND assigned_to_user_id IS NOT NULL.
    • assigned_to_expel_but_no_user_assigned: Actions where assigned_to_org_id === EXPEL_ORGANIZATION_ID AND assigned_to_user_id NOT NULL.
    • assigned_to_user: Actions assigned to the calling user (e.g. "Assigned to me")
  • investigative_actions
    • completed_auto_actions: Actions that have result_task_id IS NOT NULL
    • assigned_to_organization: Actions where assigned to an organization.
    • assigned_to_organization_user: Actions where assigned to an organization and a user.
    • assigned_to_organization_but_no_user_assigned: Actions assigned to an organization but not yet assigned to a user.
    • assigned_to_expel: Actions where assigned to EXPEL_ORGANIZATION_ID.
    • assigned_to_user_account: Actions where assigned to EXPEL_ORGANIZATION_ID and a user.
    • assigned_to_expel_but_no_user_assigned: Actions assigned to EXPEL_ORGANIZATION_ID but not yet assigned to a user.
    • assigned_to_user: Actions assigned to the calling user (e.g. "Assigned to me")
    • is_assigned_to_a_user: Actions where current_assigned_actor.actor_type === "user".
    • is_assigned_to_an_organization: Actions where current_assigned_actor.actor_type === "organization".

/asset_groups?flag[exclude_expired_groups]

The following two flags are supported on the /asset_groups

  • exclude_expired_groups: Return only asset_groups where the current date/time (i.e. now) is within the start/end range (i.e. return only groups that are NOT expired). Must be a boolean value (true/false). Defaults to false.
  • at_time: Instead of using the current date/time (i.e. now) for the exclude_expired_groups flag, use the specified date/time (ISO-8601 format). Defaults to current date/time (i.e. now).

Examples

  • /asset_groups?flag[exclude_expired_groups]=true: returns only un-expired asset groups based on the current date/time
  • /asset_groups?flag[exclude_expired_groups]=true&flag[at_time]=2017-11-01T00:00:00.000Z: returns only asset groups that were not expired on November 1st, 2017 at midnight UTC time.

CSV upload

With the exception of non-jsonapi resources and tasking resources, you can upload csv files to the jsonapi resources exposed by this API.

Content-Type

There are two ways to import csv:

  • Option A: text/csv

    If you POST with Content-Type of text/csv, the request body must contain the csv text. Note that field overrides are NOT supported when posting directly as text/csv

  • Option B: multipart/form-data

    This is useful for browsers. Create an HTML form with a file input field with the name csv. NOTE: You can add other fields to the form, and these fields will override the values of the csv import (useful for establishing relationships), but these fields MUST COME SEQUENTIALLY BEFORE THE csv FIELD.

Field headers

The csv MUST contain field headers in the first row. The field headers should match the attribute and relationship names documented in the API; However; the csv upload is more forgiving in its syntax. The field names when uploading a csv are case-insensitive, and spaces MAY be used in lieu of underscores.

Example: Both of the following evaluate to the same field names

  • event,comment,is_selected,event_date,investigation
  • Event, Comment, Is Selected, Event Date, Investigation

Field mapping

Sometimes a 3rd party csv doesn't have headers that match the headers the API expects. To resolve these issues, the request body can contain an attribute called fields. The value must be formatted as a JSON array of strings and/or objects. If provided, then the only fields explicitly added to the fields array will be included in the import. All other fields will be ignored. To map custom csv field names to API field names, use an object in the format of {"from": "csvField", "to": "apiField"}, The fields request field MUST COME SEQUENTIALLY BEFORE THE csv FIELD.

Example: adding the fields attribute will rename the incoming csv fields to fields accepted by the API

  • fields=[ "event", { "from": "message", "to":"comment" }, { "from": "datetime", "to": "event_date"} ]

If there are conflicts (i.e. two csv fields map to the same API field name), then the "winner" will be determined by the order of the fields in the field map, with the first one "winning". NOTE: Only the headers of the csv are checked... the API does NOT check the value of individual rows.

Example: In this example, the message_1 field will be mapped to the event API field. The message_2 field will be ignored. This is because message_2 comes after the message_1 fields map. If the csv had not included a "message_1" header, then the "message_2" header would be used. Think of "message_2' as a "fallback".

  • fields=[ { "from": "message_1", "to":"event" }, { "from": "message_2", "to": "event"} ]
    message_1,message_2
    "i win", "i lose"

Relationships

For relationships, use the relationship name as the field name.

  • Values for Belongs To (aka Many-To-One) relationships should contain a single guid.
  • Values for Has Many (aka One-To-Many or Many-To-Many) relationships are a semi-colon delimited list of guids.

Examples

  • Upload using Content-Type text/csv with a Many-To-One column included.

    curl -X POST 'http://localhost:7550/api/v2/timeline_entry' -H 'Content-Type: text/csv' --data-binary @- <<-EOF
    event,comment,is_selected,event_date,investigation
    "Lateral movement","Type 3 Network Logon",true,"2018-01-15T21:14:35.000Z",c7195a33-7e66-49d3-a59c-2810f224d03e
    "Exploit execution","Process: ""powershell.exe""",true,"2018-01-15T21:14:36.000Z",c7195a33-7e66-49d3-a59c-2810f224d03e
    "Beacon deployment","C:\Users\admin\appdata\local\temp\payload.exe",false,"2018-01-15T20:14:35.000Z",c7195a33-7e66-49d3-a59c-2810f224d03e
    "MIMIKATZ","C:\Users\admin\appdata\local\temp\1.exe",true,"2018-01-15T20:14:35.000Z",c7195a33-7e66-49d3-a59c-2810f224d03e
    EOF
  • Upload using Content-Type text/csv with a Many-To-Many column included.

    curl -X POST 'http://localhost:7550/api/v2/expel_alerts' -H 'Content-Type: text/csv' --data-binary @- <<-EOF
    alert_type,expel_name, Vendor Alerts
    CLOUD,Test Alert 1,7962ae2-8b8c-4371-9fc8-376300d3d6e0;1b842323-8f54-4958-b14c-af384dd345cf;
    NETWORK,Test Alert 2,
    SIEM,Test Alert 3,
    ENDPOINT,Test Alert 4,
    EOF
  • Upload using Content-Type multipart/form-data, overriding the investigation relationship via a form field.

       <!DOCTYPE html>
       <html lang="en">
       <head>
           <meta charset="UTF-8">
           <title>Upload Example</title>
       </head>
       <body>
    
       <!-- NOTE there are loads of libraries that do this stuff for you, but this example is shown in raw HTML5 and
            javascript to as a proof of concept -->
    
       <!-- start with an input field so that the user can select a file -->
       <input type="file" name="csv" id="csv" accept=".csv">
    
       <button id="go">Submit</button>
    
       <script>
           document.getElementById('go').onclick = function () {
    
               const csv = document.getElementById('csv');
    
               // using XHR2 and FormData to send the file so that we can set the authorization headers before
               // sending the file.
               const fd = new FormData();
               // NOTE: field overrides MUST BE appended BEFORE the file field is appended.
               // in this case, we'll override the investigation_id on all imported records so that
               // all the timeline entry rows are associated with with my investigation.
               fd.append("investigation", "c7195a33-7e66-49d3-a59c-2810f224d03e");
    
               // now we can append the file!
               fd.append("csv", csv.files[0]);
    
               const xhr = new XMLHttpRequest();
               xhr.open('POST', '/api/v2/timeline_entry', true);
               // Set headers after request is open
               // replace the <mytoken_here> placeholder with an actual token
               xhr.setRequestHeader('Authorization', 'Bearer <mytoken_here>');
               // Send the form. Bask in the glory of a successfully uploaded csv file!
               xhr.send(fd);
           };
    
       </script>
       </body>
       </html>
  • Upload using Content-Type multipart/form-data, overriding the fields AND applying a field map (using curl).

     curl -X POST \
        http://localhost:7550/api/v2/timeline_entry \
        -H 'authorization: Bearer ...' \
        -F 'fields=[ "event", { "from": "message", "to":"event" }, { "from": "datetime", "to": "event_date"} ]' \
        -F investigation=cc916c39-d099-4761-a4a8-3d7790d88a93 \
        -F created_by=d5758a91-4f09-4f77-886c-959983e9a533 \
        -F csv=@entries.csv

CSV download

You can also download csv files from some resource endpoints. We currently only support exporting from vendor_alerts and investigations. The signature for exporting csv is GET /api/v2/[resource]/export. The endpoint also supports query filter params: to (datetime), from (datetime) and organization_id (guid) which can be used to further narrow down the result set.

Examples

  • Exporting resources

      curl -X GET 'http://localhost:7550/api/v2/vendor_alerts/export'
      curl -X GET 'http://localhost:7550/api/v2/investigations/export'
  • Exporting with filter params

      curl -X GET 'http://localhost:7550/api/v2/vendor_alerts/export?filter[to]=2015-01-01'
      curl -X GET 'http://localhost:7550/api/v2/investigations/export?filter[from]=2011-01-13'
      curl -X GET 'http://localhost:7550/api/v2/vendor_alerts/export?filter[organization_id]=c7195a33-7e66-49d3-a59c-2810f224d03e'

Server sent events

Server sent events are a means to stream events from the API server to a client application (such as a browser) over an persistent tcp connection (see EventSource spec for more details).

If you are connecting to an sse stream from a browser, there are some shortcomings in the EventSource spec that require the use of a polyfill (Mainly, that the EventSource implementation on most browsers do not support setting headers). To allow API clients to subscribe to events relevant to their organization scope, a polyfill is required that supports setting request headers (specifically, the Authorization header). In our testing, we used the polyfill provided by https://github.com/EventSource/eventsource.

Also, please note that each SSE connection will consume one of the 3 to 10 total http connections allocated to each distinct domain by each of the major browsers. As an alternative, the Workbench API also supports websockets. See Websockets below.

Subscribing to events is as simple as calling GET /api/v2/sse with the proper Authorization headers.

  1. GET /api/v2/sse

    In a browser (using the eventsource polyfill)

    <script src=<