Nightfall Dlp Configuration

Hi there!

I want to talk today about DLP Nightfall. The Nightfall DLP Action scans your code commits upon Pull Request for sensitive information - like credentials & secrets, PII, credit card numbers & more - and posts review comments to your code hosting service automatically. The Nightfall DLP Action is intended to be used as a part of your CI to simplify the development process, improve your security, and ensure you never accidentally leak secrets or other sensitive information via an accidental commit. I tried to implement DLP with Github Actions and want to share my results

Github Action

Nightfall gives GitHub Action which can be used to scan Pull Request and Push in the branch. First you need to create a .github\workflows folder in the root of the repository and create a dlp.yml file in this folder. The file name can be any.

name: nightfalldlp
on:
  push:
    branches:
      - master
  pull_request:
jobs:
  run-nightfalldlp:
    name: nightfalldlp
    runs-on: ubuntu-latest
    steps:
      - name: Checkout Repo Action
        uses: actions/checkout@v2

      - name: nightfallDLP action step
        uses: nightfallai/nightfall_dlp_action@v0.0.7
        env:
          NIGHTFALL_API_KEY: ${{ secrets.NIGHTFALL_API_KEY }}
          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
          EVENT_BEFORE: ${{ github.event.before }}

In the on section it is specified that to start scanning on push in master branch and on Pull Request. Action has two steps, the first is to clone the repository the second is to run the scanner. Several env variables are required for the scanner to work. GITHUB_TOKEN is used so that the scanner can add comments to the PR it should have value secrets.GITHUB_TOKEN

EVENT_BEFORE is required if the scanner starts at a push event. And its value must be github.event.before NIGHTFALL_API_KEY the last required variable, you can get it by registering at nightfall.ai. Once NIGHTFALL_API_KEY is received it needs to be added to GitHub Secrets

Nightfall Configuration

Now when i finished with GitHub Action I need to create a configuration file for Nightfall. To do this, create a folder named .nightfalldlp and the file config.json in it. List of detectors that support nightfall.

{
  "detectors": [
    "CREDIT_CARD_NUMBER",
    "PHONE_NUMBER",
    "API_KEY",
    "CRYPTOGRAPHIC_KEY",
    "RANDOMLY_GENERATED_TOKEN",
    "US_SOCIAL_SECURITY_NUMBER",
    "AMERICAN_BANKERS_CUSIP_ID",
    "US_BANK_ROUTING_MICR",
    "ICD9_CODE",
    "ICD10_CODE",
    "US_DRIVERS_LICENSE_NUMBER",
    "US_PASSPORT",
    "EMAIL_ADDRESS",
    "IP_ADDRESS"
  ]
}

My configuration file looks shorter because the full one generates a lot of false-positive alarms. Example

Example of my file:

{
  "detectors": [
    "API_KEY",
    "CRYPTOGRAPHIC_KEY",
    "RANDOMLY_GENERATED_TOKEN",
    "EMAIL_ADDRESS",
    "IP_ADDRESS"
  ]
}

Now it’s all you need to push configuration for scanner to start. Even with this configuration, the scanner gives a lot of false-positive alarm results For example, I get such a error Suspicious content detected (3:***, type IP_ADDRESS) on "arn:aws:s3:::usershome/{{name}}-{{data}}/*" Or Suspicious content detected (ke********, type RANDOMLY_GENERATED_TOKEN) on "Resource": "arn:aws:kms:us-east-1:000000000:key/6a2b4d78-oy69-44ab-ce3d-43faca87fd14". Despite the false-positive part, the scanner allows you to find cluttered passwords and api keys