08 December 2014

Event Monitoring + Salesforce Wave = BFF

At Dreamforce 2014, I led a session entitled Audit Analytics that described the integration between Event Monitoring and Project Wave.

Combing the two solutions is a no brainer. Event Log Files generates CSV files on a daily basis. The Wave platform makes sense of CSVs for business analysts. 

While you can watch the video at http://bit.ly/DF14AuditAnalytics, there are a couple of tips, tricks, and best practices I want to share when using Event Log Files with the Wave platform:
  1. Consider storage requirements. Event data isn't like CRM data - there's a lot more of it. One org I work with logs approximately twenty million rows of event data per day using Event Log Files. That's approximately 600 million rows per month or 3.6 billion every half year. That means you will need to consider what data you import and how you manage that data over time.
  2. Understand your schema. There are tons of great use cases that Event Log Files solve; however, the secret sauce here is understanding what's possible already. Download a sample of files and take a look in Excel or run the head command in your terminal (i.e. head -n 2 VisualforceRequest-2014-10-21.csv) to get a sense of the kinds of lenses and dashboards you want to create. Read more about the lexicon of possible field values in the Event Log File Field Lexicon blog posting.
  3. You should convert the TIMESTAMP field in each log file to something that Wave can understand and trend in their timeline graphs. Event Log Files provides an epoch style TIMESTAMP (i.e. 20140925015802.062) rather than date format (i.e. 2014-09-25T01:58:02Z). I usually build this transformation into the download process. Read more about this transformation process with my Working with Timestamps in Event Log Files blog posting.
  4. You should de-normalize Ids into Name fields where possible. For instance, instead of uploading just USER_ID, you should also upload USER_NAME so that it's more human readable. If you don't do this before you upload the data, you can always use SAQL to help normalize name fields. Read more about using pig and data pipelines to de-normalize data before importing it into Wave with the Hadoop and Pig come to the Salesforce Platform with Data Pipelines blog posting.
  5. Merge files across days to reduce the number of datasets you have to manage (i.e. awk -F ',' 'FNR > 1 {print $0}' new_* > merged_file.csv) rather than treating each day of log files as a new dataset.
  6. Import your data using the dataset loader from Github: https://github.com/forcedotcom/Analytics-Cloud-Dataset-Utils/releases. This is the easiest way to automate dataset creation and management.
Combining large scale event data about the operational health of your organization with the power of an incredible visualization platform has the ability to change how you separate truth from fiction with your users.


  1. Very good write-up. I certainly love this website. asurveyzone.com

  2. I need to thank you for this very good read and i have bookmarked to check out new things from your post. Thank you very much for sharing such a useful article and will definitely saved and revisit your site.
    Data Science Course

  3. Wow, happy to see this awesome post. I hope this think help any newbie for their awesome work and by the way thanks for share this awesomeness, i thought this was a pretty interesting read when it comes to this topic. Thank you..
    Artificial Intelligence Course

  4. Thank a lot. You have done excellent job. I enjoyed your blog . Nice efforts
    Data Science Certification in Hyderabad

  5. Your site is truly cool and this is an extraordinary moving article and If it's not too much trouble share more like that. Thank You..
    Digital Marketing Course in Hyderabad

  6. I really appreciate the kind of topics you post here. Thanks for sharing us a great information that is actually helpful. Good day!

    DevOps Training in Hyderabad