Introduction
When a discovery ius run, the relevant discovery scan option is chosen and those settings used by Nmap to scan the target devices. If no option set is chosen, the default configuration item (discovery_default_scan_option) is selected and used.
Starting with Open-AudIT 2.3.2 we have introduced sets of preconfigured options for running the discovery scan, these pre-configured options allow a range of Nmap scan options. More detail is here: New Discovery Options
As at 3.3.0 we have introduced a "filtered|open" option to discovery scan options with a default of 'y'. Previously we used the "filtered" column to check for open|filtered. This change aligns the discovery scan options with Nmap return strings.
How Does it Work?
When a discovery ius run, the relevant discovery scan option is chosen and those settings used by Nmap to scan the target devices. If no option set is chosen, the default configuration item (discovery_default_scan_option) is selected and used.
If a device is individually discovered using the "Discover Device" link on the device details page, we first check if this device has been discovered previously (by Discoverey) and if so, use the discovery options from that scan. If it has not been previously discovered, we revert to the configuration item discovery_default_scan_option the settings.
Creating a Discovery Scan Options entry
Discovery Scan Options are just another item collection. Enterprise users can create, read, update and delete entries as required. Professional users can read all entries, but not create new entries, update existing entries or delete entries. Community users have no GUI that allows access to this collection.
Roles contain the definitions for users that are allowed to CRUD these items (regardless of license). Just like Baselines, for example.
The attributes for discovery scan options are as below.
| Attribute | Description |
|---|---|
| id | The standard ID of an auto-incrementing integer. |
| name | The standard 'name' of a collection item. Ideally should be unique. |
| org_id | The Org that owns this entry. |
| description | The standard description field. |
| options | Unused at present. Options fields within the Open-AudIT schema are generally designed to hold a stringified JSON object. |
| edited_by | The user that created or last edited this entry. |
| edited_date | The standard date/time this entry was created or last edited. |
| ping | Must Respond To Ping. If set, Nmap will fist attempt to send and listen for an ICMP response. If the device does not respond, no further scanning will occur. Previously a device did not have to respond to a ping for Open-AudIT to continue scanning. |
| service_version | Use Service Version Detection. When a detected port is detected as open, if set to 'y', Nmap will query the target device in an attempt to determine the version of the service running on this port. This can be useful when identifying unclassified devices. This was not previously used. |
| open|filtered | An open|filtered port is considered open (and will trigger device detection). Previously, Open-AudIT considered an Nmap response of "open|filtered" as a device responding on this port. This has caused some customers issues where firewalls respond on behalf of a non-existing device, and hence cause false positive device detection. We now have this attribute available to set per scan. |
| filtered | A filtered port is considered open (and will trigger device detection). |
| timing | The standard Nmap timing options. Previously set at T4 (aggressive). |
| nmap_tcp_ports | Top Nmap TCP Ports. The top 10, 100, 1000 ports to scan as per Nmaps "top ports" options. Previously we scanned the Top 1000 ports (the Nmap standard). |
| nmap_udp_ports | Top Nmap UDP Ports. The top 10, 100, 1000 ports to scan as per Nmaps "top ports" options. Previously we scanned UDP 161 (snmp) only. |
| tcp_ports | Custom TCP Ports. Any specific ports we would liuke scanned in addition to the Top TCP Ports. Comma seperated, no spaces. |
| udp_ports | Custom UDP Ports. Any specific ports we would liuke scanned in addition to the Top UDP Ports. Comma seperated, no spaces. |
| The below fields can be overwritten by an individual discovery, while still "using" a discovery_scan_options item for these if they're not set in the discovery. | |
| timeout | Timeout per Target. Wait for X seconds for a target response. |
| exclude_tcp | Exclude any ports listed from being scanned. Comma seperated, no spaces. |
| exclude_udp | Exclude any ports listed from being scanned. Comma seperated, no spaces. |
| exclude_ip | Exclude IP Addresses (individual IP - 192.168.1.20, ranges - 192.168.1.30-40 or subnets - 192.168.1.100/30) listed from being scanned. Comma seperated, no spaces. |
| ssh_port | Scan for this port(s) and if detected open, use this port for SSH communication. This is added to the list of Custom TCP POrts above, so there is no need to include it in that listr as well. Comma seperated, no spaces. |
Database Schema
CREATE TABLE `discovery_scan_options` (
`id` int(10) unsigned NOT NULL AUTO_INCREMENT,
`name` varchar(100) NOT NULL DEFAULT '',
`org_id` int(10) unsigned NOT NULL DEFAULT '1',
`description` text NOT NULL,
`ping` enum('','y','n') NOT NULL DEFAULT 'y',
`service_version` enum('','y','n') NOT NULL DEFAULT 'n',
`open|filtered` enum('','y','n') NOT NULL DEFAULT 'n',
`filtered` enum('','y','n') NOT NULL DEFAULT 'n',
`timeout` int(10) unsigned NOT NULL DEFAULT '0',
`timing` int(1) unsigned NOT NULL DEFAULT 4,
`nmap_tcp_ports` int(10) unsigned NOT NULL DEFAULT '0',
`nmap_udp_ports` int(10) unsigned NOT NULL DEFAULT '0',
`tcp_ports` text NOT NULL,
`udp_ports` text NOT NULL,
`exclude_tcp_ports` text NOT NULL,
`exclude_udp_ports` text NOT NULL,
`exclude_ip` text NOT NULL,
`ssh_ports` text NOT NULL,
`options` text NOT NULL,
`edited_by` varchar(200) NOT NULL DEFAULT '',
`edited_date` datetime NOT NULL DEFAULT '2000-01-01 00:00:00',
PRIMARY KEY (`id`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8;
API / Web Access
You can access the /discovery_scan_options collection using the normal Open-AudIT JSON based API. Just like any other collection. Please see the API documentation for further details.
Access is provided as part of a roles permissions. Discovery Scan Options is a standard resource and can have create, read, update and delete permissions (by Enbterprise licensed customers).
The API routes below are usable from both a JSON Restful API and the web interface. The Web application routes are specifically designed to be called from the web interface (a browser).
API Routes
Request Method | ID | Action | Resulting Function | Permission Required | URL Example | Notes |
|---|---|---|---|---|---|---|
| POST | n | create | discovery_scan_options::create | /discovery_scan_options | Insert a new discovery_scan_options entry. | |
| GET | y | read | discovery_scan_options::read | /discovery_scan_options/{id} | Returns a discovery_scan_options details. | |
| PATCH | y | update | discovery_scan_options::update | /discovery_scan_options/{id} | Update an attribute of a discovery_scan_options entry. | |
| DELETE | y | delete | discovery_scan_options::delete | /discovery_scan_options/{id} | Delete a discovery_scan_options entry. | |
| GET | n | collection | discovery_scan_options::read | /discovery_scan_options | Returns a list of discovery_scan_options. |
Web Application Routes
Request Method | ID | Action | Resulting Function | Permission Required | URL Example | Notes |
|---|---|---|---|---|---|---|
| GET | n | create | create_form | discovery_scan_options::create | /discovery_scan_options/create | Displays a standard web form for submission to POST /discovery_scan_options. |
Introduction
As at Open-AudIT 2.3.2 and later, we have introduced some easy to use and extremely powerful options for discovering devices. These options centre around directing Nmap on how to discover devices.
We have grouped these options into what we're calling Discovery Scan Options. We ship seven different groups of options (items) by default that cover the common use-cases.
This benefits Community, Professional and Enterprise customers.
Feature Availability
Feature availability is dependent on license type as per the table below.
| Feature | Community | Professional | Enterprise |
|---|---|---|---|
| Match Rules - set default for all discoveries | y | y | y |
| Discovery Scan Options - set default for all discoveries | y | y | y |
| Discovery Scan Options - read | y | y | |
| Discovery Scan Options - set per discovery | y | y | |
| Discovery Scan Options - create, read, update, delete | y | ||
| Discovery Scan Options - Custom per Discovery | y | ||
| Discovery Scan Options - Exclude IP, range, subnet per discovery | y | ||
| Discovery Scan Options - Exclude ports per discovery | y | ||
| Discovery Scan Options - Set device timeout, per discovery | y | ||
| Discovery Scan Options - Custom SSH port per discovery | y | ||
| Match Rules - set per discovery | y |
Discovery Scan Types
The Discovery Scan Options we ship are detailed in the table below. As above, Enterprise users can create more of these or edit the shipped items.
| Attribute | UltraFast | SuperFast | Fast | Medium (Classic)1 | Medium | Slow | UltraSlow |
|---|---|---|---|---|---|---|---|
| Approximate time in seconds for remote IP scan | 1 | 5 | 40 | 90 | 100 | 240 | 1200 |
| Must Respond to Ping | y | y | y | n | y | y | n |
| Use Service Version Detection | n | n | n | n | n | y | y |
| Consider Filtered Ports as Open | n | n | n | y | n | y | y |
| Timing | T4 | T4 | T4 | T4 | T4 | T3 | T2 |
| Top Nmap TCP Ports | 10 | 100 | 1000 | 1000 | 1000 | 1000 | |
| Top Nmap UDP Ports | 10 | 100 | 100 | 100 | 1000 | ||
| Custom TCP Ports | 22,135,62078 | 62078 | 62078 | 62078 | 62078 | 62078 | 62078 |
| Custom UDP Ports | 161 | 161 | |||||
| Exclude TCP Ports | |||||||
| Exclude UDP Ports | |||||||
| Timeout per Host | |||||||
| Exclude IP (address, range, subnet) | |||||||
| Custom SSH Port |
1The item for Medium (Classic) is similar to the Nmap for Discovery setting available in Open-AudIT 2.3.2.
Check the wiki here for a deeper look at Discovery Scan Options.
Example Scanning Improvement
We have a customer who is running discovery on a /22. The scan time to complete when using the original (hard set) options, prior to 2.3.2 was 29 hours. Using 2.3.2's UltraFast option, that scan now takes less than 10 minutes. To say they are impressed would be an understatement! They are now left with a smaller set of unknown devices that they can run a more detailed audit against. And remember, if the audited device is a computer, you will have a list of open ports derived from Netstat, anyway - possibly saving another audit cycle.
Use Cases
Handling Duplicate Serials
Recently we had cause to scan a subnet that was made up of virtual Cisco networking devices. These devices all happened to have identical serial numbers. Using the Match Rules per Discovery (available to Enterprise users) we were able to tweak the ruleset for this discovery only, without affecting other discoveries that rely upon matching a serial number. This ability solved a long-standing issue of working around a less than ideal setup on a network. A serial number, by definition, should be unique.
Filtered Ports
Networks respond differently depending on how they're configured. Some routers and/or firewalls can respond "on behalf" of IPs on the other side of their interfaces to the Open-AudIT Server. It is quite common to see Nmap report a probe for SNMP (UDP port 161) to respond as open|filtered for devices that do and do not exist. This is misleading as there is no device at that IP, yet it ends up with a device entry in the database. 99.9% of the time, it is not Open-AudIT, nor even Nmap, but the network causing this issue. Now that we have the options to treat open|filtered ports as either open or closed, we can eliminate a lot of this confusion. Enterprise users even have the option to change this on a per discovery basis (more than just using the Medium (Classic) item, as above).
Discovery Enterprise Options
The screenshot below is the Open-AudIT discovery page where all the audit configuration is set. I've added ample notes in the page explaining all the options making the tool easy to use for less technical staff.
Click to enlarge.
Check the wiki for a more detailed explanation about Discoveries
Display Improvements
As well as the functional improvements to discovery, we have also revised the Discovery Details page. We have sections for Summary, Details, Devices, Logs and IP Addresses. The Devices section, in particular, is now much more useful. We have added a new type of Unclassified to the list and we use this when we have more than just an IP and/or name for the device. For instance, we may know it's IP, name and the fact that it has port 135 open. This at least is a good indication that the device is likely a Windows machine. So we know "something". More than just "there is something at this IP". That is now an Unclassified device. We still support Unknown devices as always - for those devices we really know nothing about. An example of this screen is below. We also provide a quick link to creating credentials when a service (SSH, WMI, SNMP) has been identified, but we were not able to authenticate to it.
We think these display improvements will go a long way to assisting you to remove any Unknown or Unclassified devices that are on your network.
Click to enlarge.
Wrap Up
This new functionality makes Open-AudIT a powerful and easy to use discovery solution while providing great flexibility for advanced users.
I hope you enjoy the new features as much as our test customers and I do.
Mark Unwin.