Data Security

Dodging the Data Observability Trap

Three pipelines

Guest blog courtesy of LimaCharlie.

When you buy an observability pipeline solution (or anything else), it should solve more problems than it creates. Ideally, it should alleviate telemetry fragmentation and visibility issues without causing additional headaches. Yet, there is an entire observability pipeline market that thrives on transforming difficulties of one type into another.

For example, many SOCs suffer from alert fatigue due to managing dozens of independent security tools. They see the potential value in having one large observability pipeline handling all of their telemetry. Their goal is to simplify and consolidate the data they need into a more manageable framework. Sounds like a great plan, right?

When SOCs begin investigating popular observability pipeline solutions, a few things become startlingly apparent:

  • Consolidating telemetry often requires additional infrastructure. Setting up additional infrastructure is another demand placed upon your team that will detract from their core business mission. Maintaining this new infrastructure is a task that someone will have to shoulder in addition to their current duties. Your journey toward less work begins with tackling significantly more work.
  • Putting all the data into a single pipeline doesn’t change the nature of the data. If the SOC was overwhelmed by false positives coming from dozens of tools, it is now exhausted by them coming from a single source.
  • Vendor lock-in adds another layer of complexity, contract negotiations, limitations, and friction to an already difficult security process.
  • At this point, SOC managers must weigh the benefits of buying an observability pipeline solution against the costs. Which pain points will be alleviated and what new ones will arise? Even this calculation does not encompass the larger picture.  Some of the challenges of onboarding the observability pipeline will be heaped upon other departments. For example, consider vendors who, for reasons known only to themselves, tie their product charges to an arbitrary credit system.

    For example, there are data observability vendors who charge X credits per GB of storage. They will also roll over 20% of your credits at the end of the month. How much is a credit? What Monex index tracks the value and exchange rates of these credits? How many accountants will you need to hire to do the credit-to-real-money conversion each month simply to pay for your services? Fortunately, you can avoid this headache entirely by selecting a simpler and more effective option.

    The LimaCharlie SecOps Cloud Platform uses a vendor-neutral, API-first approach to data observability. This means there is no vendor lock-in, no risk of a partner becoming a competitor, and users can freely integrate any security resources (via API) through the platform. The SCP begins by normalizing and consolidating data from disparate security tools into an observability pipeline. This serves as the starting point from which security analysts can launch a wide range of additional security operations.

    For example, the SCP features bi-directionality which allows two-way communication with native telemetry sources. This capability allows analysts to automate and streamline detection and response processes. If suspicious activity appears in the pipeline the SCP can easily return a paybook response to the original data source. In the example below MS Office 365 reports an account login from a suspicious location. A SOC analyst using the SCP has already crafted an automated response that immediately locks the account and relays this information back to Office 365.

     Bi-directionality transforms an observability pipeline into a detection and response pipeline

    Taking this example a step further, it is easy to imagine how bi-directionality and automation can be used to improve other security operations as well. By writing automated responses to the source of an alert you can take your detection and response game to a whole new level.

    The benefits of the SCP extend to data storage costs as well. Users are given a year of free telemetry storage. This makes it possible to keep all of the organization’s telemetry while only forwarding relevant data to SIEMs for further analysis. Adopting the SCP gives an organization a low-friction and interactive observability pipeline and instant SIEM savings.

    Best of all, any expenses incurred by using the SCP are transparent, pay-per-use, and payable in standard currencies. Your accounting department won’t struggle converting credits to tokens to Big Mac coupons or any other arbitrary medium of exchange to reconcile their books. Adding unnecessary complexity to one’s observability pipeline just doesn’t make sense - but that’s just my two Cribl credits.

    You can skip this ad in 5 seconds

    Cookies

    This website uses cookies to improve your experience, provide social media features and deliver advertising offers that are relevant to you.

    If you continue without changing your settings, you consent to our use of cookies in accordance with our privacy policy. You may disable cookies.