This add-on is operated by Probe.ly
Intuitive and easy-to-use automated webapp vulnerability scanner
Last updated 15 May 2019
The Probe.ly add-on is currently in beta.
Table of Contents
Probe.ly is a vulnerability scanner add-on that automates your security testing.
Probe.ly performs continuous scanning of your web applications and lets you efficiently manage the lifecycle of the vulnerabilities it finds. It also provides guidance on how to fix any identified vulnerabilities (including snippets of code or configuration).
Because it is a developer-centric product, Probe.ly’s vulnerability guidance is tailored to developers and relevant to the technologies you use. With Probe.ly, you can also add multiple Environment targets and scan them using different scanning profiles. This means that you can scan your testing environment with more aggressive and intrusive scans, and you can run safer scans against your production app. Integrations for Slack and Jira are also available.
In addition to its web interface, Probe.ly provides all features through an API. By using it, developers can fully integrate their security testing with their CI/CD pipelines. To learn more about using the API, visit the Probe.ly developers page.
Provisioning the add-on
Probe.ly can be attached to a Heroku application via the CLI:
A list of all plans available can be found here.
$ heroku addons:create probely -----> Adding probely to sharp-mountain-4005... done, v18 (free)
For more information on the features available within the Probe.ly dashboard, please see the docs at Probe.ly’s help site.
The Probe.ly dashboard allows you to access essential stats and information about the security of your target, configure settings, view detailed results and findings, manage vulnerabilities, and start or schedule scans.
The dashboard gives you an overview of your target’s security status. There are 6 different sections of the dashboard that will help you oversee your security testing process with Probe.ly:
- Risk trend
- Average time to fix
- Severity trend
- Top 5 vulnerabilities
- Next scheduled scans
- Previous scans
Sections that aren’t self-explanatory are covered in more detail below.
This section displays a graph that shows the change in your risk score over time. The risk score is a value that is calculated from the number of active vulnerabilities, their types, and their severity. This value is most useful for performing comparisons between different periods of time. This way, you’ll have an overall idea of how your current risk level compares to previous periods.
Average time to fix
This section indicates how much time you take, on average, to fix a certain type of vulnerability. This helps you understand when you have high-severity vulnerabilities that remain open for a long time.
Probe.ly divides vulnerabilities into 3 different severity categories based on a combination of:
- The damage they might cause if exploited
- Their chance of being exploited
- Their difficulty to exploit (low, medium, or high)
The severity trend shows the change in the number of vulnerabilities from each category over time. A good first priority is to reduce the red line (high severity) to zero, and then work your way to fixing the remaining vulnerabilities from there.
The settings on Probe.ly’s dashboard can be used to configure Probe.ly in a way that works best for you and your website. You can use the setting to configure both scanners and integrations.
If your app is protected by either Basic Authentication or form-based authentication, you can configure Probe.ly to access your app with it.
For form-based authentication, you need to specify the URL of the form (it can be in a different domain than your app) and the name of the fields (attribute
name) required for login, with the matching values (attribute
value). Usually these are the username and password fields.
Add site to target
Here you can add extra hosts that are required by your app. This is most commonly necessary if your app is a Single Page Application (SPA) that uses an API hosted on a different domain (for instance,
You can add this domain to the list, and Probe.ly’s crawler will honour any request there, triggered by your app. Probe.ly will not scan that domain, we just use it in the spider so we can discover more content in the app.
Here you can add relative paths that Probe.ly’s crawler should visit even if they are not linked to.
A scan profile is a set of pre-defined options that affect what is scanned and how. Depending on your subscription, you can choose between the following profiles:
Each profile scans your website with a different level of thoroughness. You can learn more about the differences between scanning profiles here.
Set a custom header that needs to be sent in all requests. For instance, an
Set a custom cookie that needs to be sent in all requests.
The Integrations tab allows you to configure integrations with Slack (available now) and Jira (coming soon), and to use the Probe.ly API to integrate with your tools (such as CI).
To integrate the API with your tools, create a new API key by naming it and clicking the Generate new key button. You can learn more about using the API on our developers page.
You can configure Probe.ly to notify a Slack channel when certain events occur, such as when a new scan starts or a new vulnerability is found. To do that through the dashboard, you can enter your Slack channel’s URL and choose what do you want to be notified about.
Probe.ly’s dashboard also includes a vulnerability manager. Use it to perform actions such as:
- Initiate a retest
- Assign a vulnerability to a user in your company
- Label a vulnerability
- Declare a vulnerability invalid or accept its associated risk
Retesting is an easy and quick way to check if you properly fixed a specific vulnerability. After you are done fixing a vulnerability and run a retest, Probe.ly will scan only for that specific security issue and show you if it’s properly fixed. If it is, the vulnerability automatically gets a value of fixed, and the fix is recorded in its history log.
Assign to a user
If you use Probe.ly as part of a team, this action is extremely useful for tracking responsibility. You can assign vulnerabilities to a specific user in your company, which will help you organize your work better as a team. Simply go under the Action section of a vulnerability in your findings, scroll over Assign, and chose the person you would like to assign the vulnerability to.
Using Probe.ly, you can also label vulnerabilities. This way you will recognize vulnerabilities more easily. To label a vulnerability, go to its page (click on it in the Findings page), click Edit on the label’s tab, and write your label.
Declaring a vulnerability invalid/accepting the risk of a vulnerability
If you believe a security issue to be a false positive, or if it’s otherwise not relevant for you, you can declare it invalid. Probe.ly will analyze the issue and try to not flag similar vulnerabilities.
Also, if a vulnerability is of low risk for you and it is not beneficial for you to fix it, you can always just accept the risk.
In the Findings page you can filter vulnerabilities by their values or the action you’ve taken on them. You can filter them by severity, the person they are assigned to, by their current state (fixed, not fixed, accepted risk, invalid), or by the label you gave them. You can also search for vulnerabilities using the search bar right next to the filters.
A vulnerability’s page
By clicking on a certain vulnerability in the findings, you access its page. The vulnerability page provides you with a short description of the vulnerability and some other valuable information. There you can find tailored instructions on how to fix the vulnerability, the evidence that Probe.ly found, the request that we made to expose the vulnerability, and the response of your app. That way fixing the vulnerability will be a lot easier for you since you get a full overview of the whole process. You can also add a note to yourself or your team in the activity log.
You can run and schedule scans under the Scan section of the dashboard. There you can also see your scheduled and recent scans. For each scan, you can get the PDF report by clicking on the PDF file logo to the far right. Using the dashboard you can only schedule reoccurring scans on a monthly, weekly or daily basis. In order to schedule reoccurring scans in a more detailed time period (e.g. every 3 weeks) you can use the API.
Accessing the dashboard
You can access the dashboard via the CLI:
$ heroku addons:open probely Opening probely for sharp-mountain-4005
or by visiting the Heroku Dashboard and selecting the application in question. Select Probe.ly from the Add-ons menu.
In case something doesn’t work as expected, please contact Support
Migrating between plans
Application owners should carefully manage the migration timing to ensure proper application function during the migration process. Keep in mind that scheduled scans might be affected by the migration (especially when downgrading to the Starter Plan).
To learn more about our plans you can check out our pricing page. By choosing an annual billing method you save 20%, and have the option to pay by invoice.
heroku addons:upgrade command to migrate to a new plan.
$ heroku addons:upgrade probely:newplan -----> Upgrading probely:newplan to sharp-mountain-4005... done, v18 ($49/mo) Your plan has been updated to: probely:newplan
Removing the add-on
You can remove Probe.ly via the CLI:
This will destroy all associated data and cannot be undone!
$ heroku addons:destroy probely -----> Removing probely from sharp-mountain-4005... done, v20 (free)
All Probe.ly support and runtime issues should be submitted via one of the Heroku Support channels. Any non-support related issues or product feedback is welcome at email@example.com, or using our Intercom chat on our website.