High-Volume Data Analysis
Network Control Data — What is it and Why is it Important?
Conversation with Tom Tortolani, VP Product Management, Skytide
Q: What is network control data and why is it important?
Control data is literally the “bits & bytes” that surround the content delivered across a network, made up of the details of the transaction itself—from beginning to end. It can provide the context of the actual content delivery process. For example, control data can identify who accesses the network content, how often it was accessed, download times, originating and ending points, and even how it is used. Control data is generated as result of the content’s journey—the “network exhaust”—and holds a wealth of information related to the specifics of every network-related transaction.
Q: What type of content is included?
Networks are transmitting a wide range of content, with the diversity and volume of this content on a sharp increase. Popular consumer-oriented content ranges from video, flash, video, music, and HTTP files, to interactive chat sessions. Corporate entities are more interested in moving online text and display ads as well as video. And, every industry has content specific to that business. An energy company may be moving packets of energy metering data along with actual transmission of power, while financial institutions perform real-time trading submissions and process online applications. The list of content is virtually endless.
The one common element is that a network—the Internet or an internal, private network—is increasingly the critical transmission source for content that drives these businesses forward. As the critical nature of the network to business success increases, so does the difficulty in accessing and understanding the details of this network traffic in a timely, meaningful way. This poses a mission-critical problem for today’s businesses.
Q: Why is it difficult for companies to analyze this network control data?
Traditional BI tools that perform analytics atop relational data stored in a database or data warehouse break down when faced with this type of data. The data is coming across much too fast, at too high a volume, and in such complex, diverse and changing formats that it becomes a very costly, labor-intensive and time consuming process. More often than not, this valuable control data is either stored and ignored, or simply deleted.
Q: What specific intelligence can be gained from analyzing network control data?
The data can be applied to various business areas. It can provide greater visibility into how content is being used as well as network performance metrics—all over time and by unique segments. For example, analysis of network control data could provide historical views of the number of requests for individual or groups of content; aggregate results of time to download content; reveal average performance per transaction; identify unique users, etc.
Without the ability to review the data over time, it is impossible to understand the complexities of any content access patterns or to verify network performance delivery. Business are then left to base decisions on an incomplete picture of how its services are being promoted, sold, delivered and used.
Q: What is one example of how network control data analysis can benefit an organization?
The ability to analyze IP addresses alludes many organizations, primarily due to the extremely high number of individual IP addresses present on any network—millions of IP addresses generating network traffic resulting in petabytes of network control data daily—along with data format complexity. This is the type of data that cannot be analyzed with any conventional tool; traditional analytical tools break down when they have to process data segmentation across millions of buckets such as IP addresses.
The Skytide Analytical Platform™ gives users the ability to aggregate IP addresses and perform analysis across unique segments, allowing users to track access to network content—video, or even applications—by individual requestor, by region, all over time. Add to this the power of Skytide to associate an IP address to a defined entity, such as company, and the value increases as users can now track content patterns by named customers, prospects, or even competitors.
The results from this type of analysis can be immediately applied to tactical and strategic actions. Perhaps marketing messages need to be changed, sales activity concentrated on an underperforming region, or a new competitive product offering reviewed more closely. Comprehensive analysis of the IP address history can provide valuable insights into strategic and tactical business decisions across the organization.
Q: Why can Skytide perform this type of analysis?
Skytide offers several technological breakthroughs in how it makes possible high volume network control data intelligence. First, Skytide links directly to data sources, in this case IP address log files, without requiring the data be put into a relational structure. This represents a significant time savings versus traditional BI systems.
Additionally, Skytide uses XML as a common file format layer, making it possible to easily correlate and join multiple file formats together. In the previous example, Skytide can match IP addresses to known entities, thus analysis reveals usage by specific organizations. Skytide also persists aggregated results along with associated data in its analytical models that are only a small fraction of a size of source data. This severely reduces storage requirements, and speeds processing time. Skytide also provides flexible ad-hoc query options which can quickly be adapted to using drag & drop GUIs to accept new data sources or new queries.
Find out More
For more information about how Skytide can solve your high volume data analytics challenges please contact Skytide at info@skytide.com or call 650-292-1900.