23 March 2015

Event Monitoring Quick Start Guide or How to get from zero to dashboard in 10 minutes or less

So you are trying Event Monitoring out or perhaps you've purchased the add-on. Now what?

That was exactly what happened the other day when a customer asked me where the check box in setup was to get started with Event Monitoring.

Unfortunately, he stumbled upon two critical points when getting started with Event Monitoring:
  1. It's an API only feature. There is no check box in setup.
  2. It's just log data. What you do with that data is up to you whether you store it for a long time or you analyze it with a dashboard.
But it dawned on me that what the customer really needed was a quick start guide - from zero to dashboard in ten minutes or less.

Below is a set of steps to help get started quickly regardless of whether you are trying it out or you are ready to implement.

Steps to getting started

1. What should you expect to get for your edition?

  • Enterprise, Unlimited, Performance Edition: Login/Logout log lines for free with 1 day data retention
  • Enterprise, Unlimited, Performance Edition: full 28 log lines for add-on price with 30 day data retention
  • Developer Edition: full 28 log lines for free with 1 day data retention
That means that pretty much any organization that has access to our API has some form of Event Log Files already provisioned, even if it's only the Login and Logout log file types. Knowing what edition you have is important because it's what determines what you can access using the API. However, if you don't have a production organization and still want to try it out, sign up for Developer Edition which is free.

Troubleshooting tip: You won't see any Event Log File records for the first 24 hours from when it is provisioned. So if you don't see any records at first, try again tomorrow.

2.  What permissions do you need to access Event Log Files?

You need at least the following permissions:
  • API Enabled
  • View Event Log Files (However, any user with View All Data automatically gets access as well)
To add them, go to Setup > Administer > Manage Users > Permission Sets to create a new permission set with these two permissions.

To assign them to your user,  click on the Manage Assignments button in the permission set you just created and click Add Assignments to find your user and assign them to the permission set you just created. 

Troubleshooting tip: if you don't have the ability to create, edit, and assign a permission set, talk with a system administrator who does. 

3. Now you have access, but how do you actually access Event Log Files?

Remember, there is no check box in setup. You have to use the API.

I typically try new things out in the API using the workbench which works on any platform and provides access to a Swiss Army knife of great API features.

Once you login to workbench, the first thing to verify is that you have EventLogFile data. Go to queries > SOQL Query and from the object drop down, select EventLogFile.

To verify that you have some data, run the following query:

SELECT count() FROM EventLogFile

Troubleshooting tips:
  • if you can't login, you don't have API access and need to go back to step 2.
  • if you don't see EventLogFile from the object drop down, you don't have access and need to go back to step 2.
  • if you get 0 records returned from the query, you don't have any data yet and you should plan on trying again tomorrow.

4. Now that you have data, how do you view it?

While in workbench, go to utilities > REST Explorer and enter the following query into the text box:


You should get some records back. Expand one of the records and click on the LogFile link attribute.

Copy everything in double quotes and pasted into a text editor like Notepad or Sublime. If you save that content with a '.csv' file extension, you now have a CSV file with your log data.

Troubleshooting tips:
  • if workbench times out, try a smaller file. Login as and Report Export tends to render in workbench. URI, Visualforce, Apex, and API tend to be too large.

5. Now you've seen the data, how do you download it when there's a lot of it?

That's where an automation script works great. I've written several blog posts to get started with writing an automation script:

You can also use a middleware provider like Cast Iron or Informatica. Just make sure they can deserialize base64 (Blob) content and/or handle CSV data.

Troubleshooting tips:
  • if you only have a Windows machine, I recommend using the python script since the other scripts are optimized for Linux and Mac operating systems.
  • if you run into some problems, comment on this blog post and I'll try to help out.

6. Now you have the data locally, but how do you make it look good?

To make it look good, you need a visualization layer. Event Monitoring doesn't come with one by default; however, there are a number of add-on tools that you can use. Below are some great ISVs (independent software vendors) who already build on top of Event Monitoring:
  • Salesforce Analytics Cloud and the Wave Platform
  • Splunk
  • New Relic
  • Fairwarning
  • ezCloudAudit
  • SkyHigh Networks
  • Cloudlock
That doesn't mean you can't use other tools like Tableau or Qlik.  As long as they can handle CSV data, they can visualize Event Monitoring data.

In case you don't have access to any of these, I recommend a free service like Plot.ly or Raw Designs.

In the Visualizing Identity Fraud Using Login History blog post, I discussed the Raw app from Density Designs.

To use Raw, just paste the CSV data you downloaded in step 4 and pick from one of their great visualizations like a Circular Dendrogram.

When you map your dimensions, pick USER_ID and URI to get a sense of who is download what reports.

And finally visualize your data.

Event Monitoring enables organizations to have self-service access to the app logs for a variety of use cases. What are you going to do with the data once you have it?

10 March 2015

ElfPy - a Tasty Little Script for Downloading Event Log Files on Multiple Platforms

We launched Event Monitoring with Event Log Files last November. Since then, we've had a lot of customers sign up to download their log files.

But recently, one of those customers reached out with a problem. He was a Salesforce administrator on a Windows machine. That meant he had to write code or use one of my stock Bash shell scripts. Most administrators aren't used to writing code and Windows as a platform doesn't support Bash shell scripts without a lot of elbow grease, duct tape, and spit.

To fix this, I created a Python script that runs on multiple platforms including Windows, Linux, and Mac OSX. You can download the script from my Github repo.  You will need to install Python version 2.7.9 on your machine first; however, that's far easier than trying to get a Bash shell script working with Cygwin.

Why python? Because it's easy to learn, it's easy to read, it has incredible library support, and most importantly, it supports multiple platforms including some I haven't even heard of!

To run the script, open a terminal (or cmd on Windows) and type:

python elf.py

or if you have multiple versions of Python, including 2.7.9, installed already:

python2.7 elf.py

The script is very simple and takes four prompts:
  1. Username
  2. Password (hidden)
  3. Date range
  4. Download directory
Beyond that, it's as easy as running the script from a terminal or command prompt and you're ready to start downloading Event Log Files on Mac, Linux, or Windows.