John Gudmundson, Senior Product Marketing ManagerOctober 24, 2019
NetFlow Is No Longer Adequate
Metadata is defined as “data about data.” You can learn a
lot about an environment with this information. Imagine you’re in counterintelligence
and trying to prevent espionage. You don’t need to know the actual content of suspected
foreign operatives’ communications you could obtain via a FISA warrant; just
knowing who talks to whom and where, for how long, how often, time and day,
device type and so on can give substantial clues. In fact, given the exabytes
of unified communications packets flowing through servers worldwide, having a summary
of the data would make it easier for you to identify and proactively prevent national
In the IT world there exists advanced metadata powered by deep packet inspection that provides abridged and context-aware information about raw packets based on Layers 3-7. Such insights can be applied to obtain application behavior. Not to catch a spy, but perhaps earn a promotion. Organizations can acquire critical details pertaining to flows, reduce false positives by separating signals from noise, identify nefarious data extraction, and accelerate threat detection and remediation through proactive, real-time traffic monitoring as well as troubleshooting forensics.
Application metadata can supply security tools with thousands of attributes that shed light on security status and help prevent breaches — over three orders of magnitude beyond the “five-tuple” offered by simple NetFlow that only provides minimal Layer 2-4 data. Lower layers provide details on what devices are involved and how much data; however, nothing useful beyond basic network management. Obtaining advanced metadata doesn’t have to devour resources from your infrastructure devices such as routers. Example application metadata attributes can include:
Identification: Social media user, file and
video names, SQL requestsHTTP: URL identification, command response codes
DNS parameters: Request/response, queries and
device identifiersIMAP and SMTP email-based communications with
sender and receiver addressesService identification: Audio, video, chat and
file transfers for VoIP and messaging
SIEM solutions use this information to correlate and analyze log data from servers and security appliances. Network security tools leverage these attributes to deliver the insight and analytics needed to manage the opportunities and risks associated with a digital transformation. SecOps administrators can automate detection of anomalies in the network and stop cyberthreats that overcome perimeter or endpoint protection.
Show Me the Money
Advanced metadata can be used in a variety of security use
cases. The principal deployment is in providing metadata to SIEM tools for
security analysis. Brief teasers of the issues that can potentially indicate
security breaches and how to identify them include:
Identify data exfiltration. Evaluate the volume and type of DNS requests, including non-standard ports, at various domain levels, including DNS queries involving entropy, statistics, outliers and record types. This data can reveal DNS tunneling in the network and help establish the legitimacy of domains.Detect suspicious network activity. This involves identifying command-and-control attacks using machine learning. Admins can determine whether a domain is legitimate or was generated using a botnet-controlled domain generating algorithm (DGA). SecOps can verify authenticity by leveraging external sources such as VirusTotal. Dashboards of interest here include the total unique domains seen on the network and those predicted to be legitimate versus DGA generated.Uncover distrustful remote connections. This scenario helps in identifying suspicious SSH, RDP and Telnet remote connections. This is done by looking at leading indicators such as bandwidth usage, longevity of these connections, IP reputation and geolocation. This can help uncover the detection of unauthorized external remote connections used for data exfiltration.Recognize dubious user activity. Uncover potential brute-force attacks on your network involving high-privileged users who are logging in from unauthorized systems or have an unusually high level of login activity. Such events suggest user credentials may have been compromised. Another example involves multiple logins from different locations by the same user.Locate weak ciphers. This can help ensure security compliance. Ideally clients and servers should only employ the strongest cipher suites available and negotiate to one of these during the TLS handshake, but this is not always the case. Metadata can reveal all TLS connections with weak ciphers, along with the applications and systems hosting these apps. Time window analysis. Allows IT to derive an end-to-end picture of various security events. This is supported by leveraging metadata to look at Kerberos, SMB, DNS and HTTP use. By isolating their prior and post-protocol activities that led up to an incident, security breach origins can be found, and administrators can find all activities of a particular host in a given time frame.HTTP client errors. Additional suspicious activity can be identified by analyzing HTTP client errors, including the number of HTTP response code errors relative to the total number of codes. Also, the distribution of these errors and the clients seeing these codes can give insight into malicious activity. This also provides details about client IP and the number of errors it has encountered and can indicate a hacker is trying a brute-force attack and getting 401 errors.Detect rogue DNS and DHCP servers. Attackers could host shadow IT within your network for diverting traffic and launching man-in-the-middle attacks. In this use case, you evaluate the following details about DNS and DHCP servers in your network: the total number of DNS servers and list of rogue DNS servers; the total number of DHCP servers and list of rogue DHCP Servers; and the list of trusted/publicly known servers.Identification of expired TLS certificates. These certs help facilitate encryption and authentication and are effectively mandatory for web servers. Without them visitors will quickly move on. With validity dates becoming shorter, some only for a few weeks, it is imperative to find those that are defunct. There are several attributes to spot these and create real-time alerts. You can also detect use of non-trusted or self-signed certificates for SSL-decrypted traffic, which could indicate nefarious activity.
Sounds Intriguing — Must Be Complicated to Implement
Hardly. Just a straightforward add-on license from Gigamon that runs on our
We recently introduced Application Metadata Intelligence (AMI), which dramatically extends basic NetFlow, not only to Layer 7, but generates over 7000 attributes! In addition to the app aware attribute examples mentioned above, dozens of elements each, on mainstream apps such as Facebook, YouTube, Gmail and Yahoo and protocols including DNS, FTS, IMAP and SIP are available.
AMI is tightly woven with leading SIEM and other security solutions. It strategically sits in the middle between the networking and compute infrastructure and these advanced tools and provides details specific to apps and protocols in play. Only in this way can you truly optimize the breadth/depth and accuracy of your security appliances and forensically pinpoint causes of security fissures. The AMI Dashboard allows granular selection of numerous metadata elements on a per-app and protocol basis.
AMI empowered partners
can take advantage of AMI to add more value to their tools. Our solution works
out of the box with Splunk and QRadar where connectors have been developed. Their
reports and accompanying dashboards are fully customizable. Reports can be produced
with the type of data you’re looking for.
Essentially, any type of security analytics tools can receive and use application metadata (app metadata for short). This involves developing an adapter that “parses” the incoming CEF or IPFIX to identify and understand the metadata. These are not complex undertakings and several vendors have them. Several other ecosystem partners include:
Countermeasures: Artificial intelligence (AI)-powered threat hunting. They
combine penetration testing best practices, app metadata and AI to reduce the
time and effort required in threat hunting.Spirent:
Lab-to-live testing. They use app metadata to replicate exact application and
user loads for mission-critical, high-volume application testing.Plixer:
Unified SecOps and NetOps. App metadata reduces risk, lowers operational cost,
and improves efficiency for their unified Scrutinizer platform.FireEye.
Helix cloud-hosted security operations. Use metadata to provide Helix the raw
material for a host of extremely valuable security investigation and
operational use cases.NetFlow
Auditor: Network forensics. App metadata assists big data and machine
learning to correlate threat intelligence and identify cyberthreats and
What About Automation and Proactively Stopping Bad Stuff?
Retroactively looking at forensic data, isolating and
remediating lapses in counterintelligence security and building a prosecutorial
case against cyber spies is great. However, what would be even better is automating
the use of metadata to boost security. Let’s say you wanted to take certain
attributes, such as filename in combination with username, and send an alert in
real time to some tool to block specific downloads based on those attributes. Or
if an email has an attachment with a particular file type, can metadata about
this file be used to automatically generate alerts that will ensure the file is
sent to a sandboxing tool for analysis prior to opening? Can you do that?
Yes, you most certainly can. Implementing this dynamic security, to no surprise, involves a SIEM. The SIEM interacts with our centralized management tool, GigaVUE Fabric Manager (FM), to make changes to traffic flows based on an anomaly. One such example is our app for Splunk Base:
Gigamon Adaptive Response Application for Splunk provides their administrators with alert actions to be taken on Gigamon Visibility nodes via FM. These actions can be bound to correlation searches on Splunk Enterprise Security for automated response or executed on an ad-hoc basis with notable events. This app leverages Splunk’s adaptive response framework and uses a RESTful API to integrate with FM to perform response actions on Gigamon Visibility Nodes. Actions can include redirecting certain apps or flows to specific security tools such as advanced threat protection or secure email gateways for “enhanced interrogation” (no, not water boarding)!
In other words, a nice real-time feedback loop between
Gigamon and Splunk.
Gigamon Application Metadata Intelligence Benefits
Integration with Gigamon Application Visualization, Application Filtering and Fabric Manager solutionsBroad and deep insights into protocols, applications and user behavior with 7000-plus Layer 3-7 attributes spanning 3,000 cloud and client-server appsSupported by connectors for SIEM tools (Splunk and QRadar) and out of the box by Gigamon-empowered third-party platforms for simplified and rapid assimilation of AMI metadata Assist tools in ensuring resource security by viewing and potentially automatically blocking actions such as select social media users, SQL queries and requested file/video namesEnable security tools to better visualize and measure performance, troubleshoot issues, spot security events and improve their effectivenessSecure communication links by observing broad Layer 7 metadata such as HTTP codes and DNS and preventing malicious commands and directives
If you are involved with security operations and want to attain deeper insights into the new paradigm of application metadata and what it entails, please check out this great demo. Or read the very interesting Application Intelligence solution brief and the AMI data sheet.
Hear from our experts on the latest trends and best practices to optimize your network visibility and analysis.
Continue the Discussion
People are talking about this in the Gigamon Community’sApplication Intelligence group. Share your thoughts today.