28 July 2015

Who stole the cookie from the cookie jar?

Sample Visualforce Page using Google Charting API
Have you ever needed to track what users view, not just what they change? Have you ever had security, risk, or legal ask for a report on user activity for audit or regulatory reasons? Have you ever needed to track user's actions down to the device level so that activity on the phone, tablet, and web desktop are tracked separately?

Starting with the Summer '15 release, we're introducing key data leakage detection information through a pilot program. The goal of the pilot is to enable customers to query specific data leakage use cases in near real-time for analysis purposes.

The initial pilot of the Data Leakage Detection pilot tracks SOQL queries in near real-time from the SOAP, REST, and Bulk APIs. Because greater than half of all data accessed on the platform is performed via these APIs, organizations can gain greater insights into:

  • Who saw what data
  • When they saw that data
  • Where they accessed the data
  • What fields they accessed
  • How long a query took
  • How many records they accessed

When combined with the Login Forensics pilot, you can also track every query back to a unique login to identify anomalies in user behavior.

Each event consists of key information about the API transaction including:

  • AdditionalInfo
  • ApiType
  • ApiVersion
  • Client
  • ElapsedTime
  • EventTime
  • Id
  • LoginHistoryId
  • ObjectType
  • Operation
  • RowsProcessed
  • Soql
  • SourceIp
  • UserAgent
  • UserId
  • Username

This means you can find out who (e.g. Username), saw what (e.g. Soql) including sensitive PII (Personally Identifiable Information) fields, how much (e.g. RowsProcessed), how long (e.g. ElapsedTime), when (e.g. EventTime), how (e.g. UserAgent), and from where (e.g. SourceIp). Plus, you can correlate all of this information back to the original Login (e.g. LoginHistoryId) to profile user behavior and disambiguate between legitimate and questionable activity beyond the login.

Once the pilot is enabled in your organization, you can visualize a set of events using the sample Visualforce page with Google Charting API from my Github repository.

To learn more specifics about the Data Leakage Detection Pilot functionality, read the pilot tip sheet and to participate in the pilot, please contact your account executive or customer support rep.

13 July 2015

Track your Apex limits in production using Apex Limit Events

Salesforce Apex is the backbone of the programmatic platform. With it you can push the customization of any org beyond the button click realm.

Example of an app you can build with the
Apex Limit Events Pilot
Apex runs in a multitenant environment. The Apex runtime engine strictly enforces limits to ensure that runaway Apex doesn’t monopolize shared resources. If some Apex code ever exceeds a limit, the associated governor issues a run-time exception that cannot be handled. Apex limits are defined in the Salesforce documentation.

I wrote about tracking Apex limits in a blog post last May (That which doesn't limit us makes us stronger). As a force.com developer, you have the ability to instrument your Apex code with System.Limits() methods that allow you to compare how much you're using against the ceiling of what's allowed by any limit.

However, the more instrumentation you add, the more opportunity you have for code and performance issues. It's the Heisenberg concept - that which you try to measure affects the measurement. And even when you do try to track limits in this fashion, the results are stored in developer oriented tools like the Debug Logs or the Force.com Console logs which are really only meant to be used in sandbox. Bu what about DevOps? How can they monitor the health of their developer's code in production?

As a reaction to this, we are introducing the Apex Limit Events pilot program in Summer '15.  The goal of this pilot is to enable operations to monitor their production instances in near real-time and with automated hourly roll-up aggregate metrics that tell you the state and health of your Apex according to their limits.

Each event consists of key information about the Apex execution in the context of a limit including:
  • EntryPointId
  • EntryPointName
  • EntryPointType
  • EventTime
  • ExecutionIdentifier
  • Id
  • LimitType
  • LimitValue
  • NamespacePrefix
  • UserId
  • Username
Each hourly metric consists of key aggregate information including:
  • Distinct Number of Apex Transactions
  • Distinct Number of Apex Transactions With Limits Exceeding 60% Threshold
  • Distinct Number of Apex Transactions With Limits Exceeding 60% Threshold By Entry Point Name
  • Distinct Number of Apex Transactions With Limits Exceeding 60% Threshold By Limit Type
  • Average Limit Value By Entry Point Name
  • Average Limit Value By Limit Type
Apex limit events is accessible via the Salesforce public API through the apexlimitevent sobject. The only user interface is an org preference in setup that an administrator can control to enable or disable the feature within their org.

To help developers and operations bootstrap their efforts to take advantage of this feature, I built a Github repo including a:

1. Visualforce page and tab integrating Apex limit events with the Google Charting API
2. LimitsTest Apex class that exposes a REST API webservice you can use to intentionally exceed limits and load data into the apexlimitevents sobject
3. ApexLimitJob Apex class that can be used with scheduled Apex to generate limits
4. limitsHammer python code that makes it easy to generate limits data

To participate in the pilot, contact salesforce support or your account executive. For more information about the pilot, check out the tip sheet.