20 October 2014

Salesforce Application Monitoring with Event Log Files



Have you ever:
  • wondered how to make your sales and support reps more successful?
  • wanted to track the adoption of projects that you roll out on the Salesforce platform like S1, Chatter, or the Clone This User app from Arkus?
  • wanted to find out which apex classes are succeeding and how long it takes for your Visualforce pages to render in production?
  • find out why some reports run slower than others?
  • needed to audit when ex-employees leave the company with your customer list?
Application Monitoring using Event Log Files, new in the Winter '15 release, enables all of these use cases and many, many more using an easy to download, file based API to extract Salesforce app log data.

When we started building this feature, which has been in pilot for over a year, we talked with a lot of customers who wanted access to our server logs for a variety of use cases.

What we heard from many of those customers was that they wanted to easily integrate the log data from all of their organizations with their back-end reporting and audit systems so they could drill down into the day-to-day detail. As a result, you won't find a user interface within setup to access these files; everything is done through the API in order to make integration easy.

The idea behind this feature is simple. Everyday, Salesforce generates a massive amount of app log data on our servers.

Our app logs do not contain customer data but instead contain metadata about the events happening in an organization. As a result, we store a report id rather than a report name or an account id instead of an account name. This obfuscation of the data enables customer's to normalize ids into names for records.

Every night, we ship these logs to a Hadoop server where we map reduce over the log data to create Event Log Files for organizations enabled with the feature.

As a result, every morning, a customer can download the previous day's log events in the form of CSV (comma separated values) files using the API. We chose CSV as a file format because it's easy to use when migrating data between systems.

Once you have this file, you can easily integrate it with a data warehouse analytics tool, build an app on top of a platform like force.com, or buy an ISV (Independent Software Vendor) built app to analyze and work with the data.

To make it easy to try this feature out and build new kinds of apps, we are including one day of data retention for all Developer Edition organizations. That means if you have a Developer Edition organization already, just log into it using the API and you'll have access to the previous day's worth of log data. If you don't already have one, just go to http://developerforce.com/signup to get your free Developer Edition org.

Application monitoring at Salesforce with Event Log Files has just made auditing and tracking user adoption easier than ever before.



Icons by DryIcons

5 comments:

  1. Thanks for sharing, Adam! Do you mind posting more details on the steps you took to extract / load the event log data into Wave during the dreamforce session you did on Event Logs & Wave? Thanks!

    ReplyDelete
    Replies
    1. Hi dallasDeuce - That's a great idea! I'll work on a blog post on how I did it. Thanks for reading!

      Delete
  2. Will you guys create a mechanism to view the output of System.debug in realtime in my console (where I can stream that data)? Without this I cannot readily debug any Apex code.

    ReplyDelete
    Replies
    1. Hi Mike - I believe Josh Kaplan answered this question for you but let me know if you need more details. thanks!

      Delete