Data Collection Tuning -Default Real-Time & Long-Term Config
Posted: Wed Jun 23, 2010 3:02 pm
CySight creates the long-term and the real-time data bases simultaneously. Granularity of real-time and long-term data can be changed to suit your needs.
The Real-time and Long-term Data collections are a completely separate process that aggregate according to rules set in “Data Collection Tuning”. Multiple rules can be set to reduce granularity where flows become excessive when using the full flow Professional license.
Real-time is all NetFlow fields, one minute granularity. Typically the real-time is used for trouble shooting, forensics, alerting, threat detection, etc. The default retention for real-time is 7 days. The default retention for long-term is 12 months. You may retain real-time and long-term data for as long as you like, subject to disk storage.
After changing the rollover retention period of real-time or long-term data policies click Confirm then click Apply Now and confirm
Long-term data is for trending, capacity planning, etc where the focus is on volume, usage by application, account, interface etc. Such trending analysis usually doesn't require granularity down to individual IP address. That granularity is provided (by default) in the real-time data.
We aggregate IP address, high port, and other elements to slim down the long-term database to allow very long retention, and speedy reporting over long time duration reporting periods (weeks, months etc).
The Long-Term archive is also used to provide an archive of data aggregated for specific trending needs such as capacity planning or 95th percentile. Long-term aggregates by default by class C to provide an efficient default archive collection for long-term trending. When IP Allocations are set the long-term default rules are automatically changed to keep the start of the Location Range.
The default setting of long-term is to retain Interfaces, Accounts, Costs Centres, QoS values and Selected Ports. The current setting provides the start of a Network Range Allocation as the IP Source / Dest and the unallocated IP is set to 0.0.0.0. Therefore data in the Long-term provides speedy analysis and capacity to analyze trends.
Long-Term data is by default in 1 hour increments but can be changed to increments of 5/10/15/60 minutes.
Real-time and long-term archives retention policy can be changed to suit your needs and disk space and license profile.
You can drill down from Long-Term to Real-Time data as long as the data exists in the Real-Time archive (based on retention/rollover period setting) you can view the detail of the flows.
The great thing with Auditor is that you have complete control over these granularities. If you want to maintain IP address granularity in long-term you can. On the other hand, it may be better to achieve that granularity by extending the real-time retention time.
We set the defaults for what works in the general sense.
We can help you to determine the best trade-offs among granularity, storage capacity, system performance etc.
The Real-time and Long-term Data collections are a completely separate process that aggregate according to rules set in “Data Collection Tuning”. Multiple rules can be set to reduce granularity where flows become excessive when using the full flow Professional license.
Real-time is all NetFlow fields, one minute granularity. Typically the real-time is used for trouble shooting, forensics, alerting, threat detection, etc. The default retention for real-time is 7 days. The default retention for long-term is 12 months. You may retain real-time and long-term data for as long as you like, subject to disk storage.
After changing the rollover retention period of real-time or long-term data policies click Confirm then click Apply Now and confirm
Long-term data is for trending, capacity planning, etc where the focus is on volume, usage by application, account, interface etc. Such trending analysis usually doesn't require granularity down to individual IP address. That granularity is provided (by default) in the real-time data.
We aggregate IP address, high port, and other elements to slim down the long-term database to allow very long retention, and speedy reporting over long time duration reporting periods (weeks, months etc).
The Long-Term archive is also used to provide an archive of data aggregated for specific trending needs such as capacity planning or 95th percentile. Long-term aggregates by default by class C to provide an efficient default archive collection for long-term trending. When IP Allocations are set the long-term default rules are automatically changed to keep the start of the Location Range.
The default setting of long-term is to retain Interfaces, Accounts, Costs Centres, QoS values and Selected Ports. The current setting provides the start of a Network Range Allocation as the IP Source / Dest and the unallocated IP is set to 0.0.0.0. Therefore data in the Long-term provides speedy analysis and capacity to analyze trends.
Long-Term data is by default in 1 hour increments but can be changed to increments of 5/10/15/60 minutes.
Real-time and long-term archives retention policy can be changed to suit your needs and disk space and license profile.
You can drill down from Long-Term to Real-Time data as long as the data exists in the Real-Time archive (based on retention/rollover period setting) you can view the detail of the flows.
The great thing with Auditor is that you have complete control over these granularities. If you want to maintain IP address granularity in long-term you can. On the other hand, it may be better to achieve that granularity by extending the real-time retention time.
We set the defaults for what works in the general sense.
We can help you to determine the best trade-offs among granularity, storage capacity, system performance etc.