24 March 2014

Using a bash shell script to solve basic administrative tasks

In my ideal world, there would be a simple yet powerful user interface for everything I needed to do.

However, while waiting for my grand utopian vision to come true, I'm still left finding anyway possible to solve the everyday problems that I have.

In particular, I have repeatable administrative tasks that I need to automate. For instance, the ability to mass export, dedupe, download, update, and delete data from my org.

So when there isn't a user interface for me to use, I resort to using the API and great tools like Workbench and Data Loader for all of my administrative needs. But even these API tools have their limitations, in particular automating tasks.

Recently, while trying to automate the download of very large CSV files, I turned to a  tool from a former lifetime, bash shell scripts. While not ideal compared to using user interfaces or API based tools, bash shell is exceedingly efficient when performing rudimentary, repetitive, and ridiculously simple tasks.

However, I ran into a problem trying to use a common bash app, curl, with the salesforce REST API. To use curl with the REST API, I need to use oauth. While oauth is incredibly powerful and secure, it turned out to be a little more difficult to use with curl than I originally expected.

With oauth, a user logs into salesforce from a client app, curl in this case. After logging in, the client app receives a security access token. With this access token, the app can continue to make API calls on behalf of the user. So to use the REST API, I needed an access token.

The problem was that oauth provides that access token in an embedded JSON response that has to be parsed.


While playing around with a variety of techniques to parse JSON, I found a great reference to pulling out the access token using regular expressions (regex).

if [[ "$response" =~ (\"access_token\"):\"(.+)\" ]]; then
    access_token="${BASH_REMATCH[2]}"
After that, including the access token in subsequent REST API calls was easy.
curl https://${instance}.salesforce.com/services/data/v29.0/query?q=Select+Id+From+Account -H "Authorization: Bearer ${access_token}" -H "X-PrettyPrint:1"

I created a github repo called curlREST to help bootstrap the creation of these bash shell scripts in the future. If you have the need to mass manage data easily, I definitely recommend checking it out. With very few changes, you can quickly automate the most simple administrative tasks.

Sometimes it's about using a powerful and simple user interface. But always, it's about solving the problems before us by any means possible.

10 March 2014

A very little key [prefix] will open a very heavy door




Key prefixes hold the proverbial keys to salesforce object kingdom. Key prefixes are the beginning of every unique, immutable id.  Because of their simplicity, its easy to not only identify objects based on their id, it's also possible to interpret the actions that a user performed on that record.

For instance, an id like 001D000000IBVzo/e?retURL=%2F001D000000IBVzo in the browser address bar or a log file indicates someone edited an account record on the na1 instance and was redirected to the detailed view of the record after saving it. There are auto-magical things you can learn from links in general and that magic begins with a key prefix.

There are many uses for key prefixes:
  1. they help in the construction of custom links
  2. they can be used when creating Visualforce pages or Apex code
  3. they are essential to performing data extractions, transformations, and loads using the API
  4. they are used in identifying user activity in the logs
Its with the last use case that I spend a lot of my time. A key prefix allows me to skim across the URI log lines and look for particular objects that a user interacts with so I can better find that needle in the haystack.

The easiest way to find out which key prefix maps to which object is by using a tool like the workbench and the describeSObject() method. After you log into workbench, go to Info > Standard & Custom Objects and pick an object from the pick list.




This method is also the best way to discover a custom object's key prefix since those will be unique to each org, where as standard objects like Account will always be the same across orgs.

You can also obtain a key prefix using Apex. There's a great explanation of this in the Apex help docs with the introduction of the Id.getSObjectType() method.

And finally, when push comes to shove, you can navigate around the app and take a look in the address bar at the resulting ids. Below is a list of some of my favorite ones I obtained either by navigating the app or from other blog sites like Mohammad Usman's sfdchack.blogspot.com or Daniel Ballinger's awesome list on fishofprey.com:


Key Prefix Object Type
'03G' Account Criteria Share
'001' Account
'00r' Account Share
'007' Activity
'04m' Additional Directory Number
'01p' Apex Class
'07L' Apex Debug Log
'707' Apex Job
'01q' Apex Trigger
'099' Apex Component
'07L' Apex Debug Log
'07M' Apex Test Result
'0DS' App Menu Item
'806' Approval
'04i' Approval Request
'ka0' Article
'02i' Asset
'08A' Scheduled Report
'00P' Attachment
'01m' Business Hours
'019' Business process
'00U' Calendar
'04v' Call Center
'701' Campaign
'00v' Campaign Member
'01Y' Campaign Member Status
'08s' Campaign Share
'07O' Canvas
'500' Case
'00a' Case Comment
'01n' Case Share
'010' Case Solution
'01n' Case Share
'02o' Category Data
'02n' Category Node
'5CS' Chat Session
'0F9' Chatter Group
'0FB' Chatter Group Member
'0ca' Chatter Activity
'00a' Comments
'09a' Community
'003' Contact
'02Z' Contact Role
'03s' Contact Share
'069' Content
'800' Contract
'811' Contract Line Item
'00b' Custom Button or Link
'01N' Custom S-Control
'02F' Custom Field Map
'02U' Custom Page
'070' Custom Report Type
'03f' Custom Setup
'02c' Custom Share
'0A7' Custom Share Row Cause
'01r' Custom Tab
'01Z' Dashboard
'0FL' Dashboard Refresh
'02f' Delegated Administration Group
'02h' Delegated Administration Group Assignment
'02g' Delegated Administration Group Member
'00C' Delete Event
'02d' Division
'015' Document
'05X' Document Entity Map
'0I4' My Domain
'091' Email Service
'093' Email Services Address
'018' Email Status
'00X' Email Template
'02s' Email Message
'058' Entity Subscription
'017' Entity History
'00U' Event
'020' Event Attendee
'0AT' Event Log File
'0IM' Export
'0D7' Feed Comment
'0F7' Feed Post
'0D6' Feed Tracked Change
'07F' Feed Favorite
'0I0' Feed Like
'0II' Feed Mention
'09B' Feed Poll Vote
'737' Field History
'1HA' Field History Archive
'0IX' Field Set
'00B' Filter
'022' Fiscal Year Settings
'00l' Folder
'01d' Folder Group
'0AF' Folder Share
'608' Forecast Share
'00A' Forecast Item
'0J9' Forecast Quota
'00G' Group
'011' Group Member
'00G' Group
'02E' Help
'0C0' Holiday
'087' Idea
'00a' Idea Comment
'0Bg' Idea Theme
'0ET' Import
'kA#' Knowledge Article
'00h' Layout
'00Q' Lead
'01o' Lead Share
'016' Letter Head
'0Ya' Login History
'02x' Login Hours
'710' Login IP
'01H' Mail merge Template
'07A' Mass Mail
'063' Mobile Configuration
'086' Mobile Device
'0D5' News Feed
'002' Note
'0JH' Notification
'888' OAuth
'110' Object Permissions
'006' Opportunity
'00J' Opportunity Competitor
'008' Opportunity History
'00k' Opportunity Product
'00t' Opportunity Share
'02r' Opportunity Big Deal Alert
'00K' Opportunity Contact Role
'00k' Opportunity Line Item
'802' Order Item
'801' Orders
'0D2' Org Wide Email Address
'00D' Organization
'04l' Outbound Message
'02l' Outbound Queue
'00l' Partner
'026' Period
'0PS' Permission Set
'0Pa' Permission Set Assignment
'0PL' Permission Set License
'2LA' Permission Set License Assignment
'060' Portal
'061' Portal Account
'067' Portal Member
'01u' Price Book Entry
'01s' Price Book2
'04g' Process Instance
'04h' Process Instance Step
'01t' Product
'00e' Profile
'01k' Field Level Security
'01G' Profile Layout
'01P' Profile Tab
'021' Quantity Forecast
'04X' Quantity Forecast History
'906' Question
'03g' Queue
'0Q0' Quote
'012' Record Type
'00O' Report
'08e' Scheduled Job
'0DM' Site
'04n' Softphone Layout
'501' Solution
'081' Static Resource
'00T' Task
'005' User
'100' User License
'03u' User Preference
'0D5' User Profile Feed
'00E' User Role
'099' Visual force component
'066' Visual force Page
'083' Vote

The most awesome Chris Peterson also pointed out that the instance mapping (the fourth place in the unique id) can be found in the source code for the workbench app.

Key prefixes open many heavy doors. Which ones will it open for you?


04 March 2014

Auto-magical things you can do with salesforce links


Ever notice how the URL in the address bar of your browser can be used to do auto-magical things in the app? 

I've been spending a lot of time in the logs looking at URLs and having to interpret what the links actually mean. But understanding the salesforce link structure opens many different doors for an admin including creating custom links, passing parameters into pages and reports, creating formula fields, bookmarking, and other general hackery.  

Below are tips and tricks to master the salesforce URL link structure. Most of the following is from Travis Bryant and Keith Torluemke who put together an incredible tip sheet that I've edited here.

All the links listed here will only work if the user has already logged in to salesforce, and the valid session id is stored as a cookie. If there is no session id, these links will not be accessible and the user will be directed to the login page.

All URLs accessed for salesforce will start with the structure https://<server>.salesforce.com/, where <server> is the particular instance of the salesforce service that the customer's organization is hosted on. For a full list of servers, you can take a look at the salesforce trust site. Example values for <server> include:


  • na1
  • na2
  • na3
  • na4
  • ap
  • emea

You can also have a custom domain name like https://salesforcehacker.my.salesforce.com using the My Domain feature. Otherwise, the first part of the address will typically be the server where your org resides.

When creating a link that will be used within the application, for instance as a Custom Link, the https://<server>.salesforce.com is not needed. All that is necessary is a relative URL which starts with a "/", as in "/015300000007epj" which could be used to access a particular record.

The next part of the URL will typically reference an object type by key prefix, such as Accounts, Contacts, Opportunities, etc... Some common object key prefixes include:


  • 001 - Accounts
  • 003 - Contacts
  • 006 - Opportunities
  • 015 - Documents
  • 500 - Cases
  • 00O - Reports
  • 00Q - Leads
  • 01Z - Dashboards
  • 701 - Campaigns

Standard objects have object key prefixes that are consistent across orgs. Custom objects have a key prefix as well, but it is based on a particular org implementation, thus custom links would most likely break upon sharing between orgs. The easiest way you can obtain a key prefix is by performing a describeSObject() on the particular object using the API (i.e. using Workbench) or with the .getKeyPrefix() method in Apex.

View Links

These URLs are used to view various screens within the application


Action
Description
Relative URL
Example
App Home Page
Go to the overall home
/home/home.jsp
https://na1.salesforce.com/home/home.jsp (Homepage)
View Object Home Page (Tab)
Go to the homepage for the object’s tab
/<object key prefix>
https://na1.salesforce.com/001/o (Account Home Page)
View All Objects
Go to the last list view for that object
/<object key prefix>
https://na1.salesforce.com/003 (View Contacts)
View Object/Record
Open an individual record’s detail page
/<id>
https://na1.salesforce.com/00130000003DFea   
(Individual Account)
Download Doc
Download a file from the doc tab
/servlet/servlet.FileDownload?file=<id>
https://na1.salesforce.com/servlet/servlet.FileDownload?file=01530000000BCh5
*Download Content
Download a file from Chatter Files
/sfc/servlet.shepherd/version/download/<id>
https://na1.salesforce.com/sfc/servlet.shepherd/version/download/06830000003ILyw
Mini-Page Layout View
See a minimalist view of the record
/<id>/m
https://emea.salesforce.com/00120000002vqFD/m
Print View
Print a page
/<id>/p
https://emea.salesforce.com/00120000002vqFD/p

* - When downloading content, multiple files can be downloaded as a zip. Use this syntax:

/sfc/servlet.shepherd/version/download/cvId00/cvId01/...?  

where ContentVersion ids are separated by forward slash. (Hack from crop1645 in comments below)
Edit Links

These links are used to go to edit object screens (new, edit, clone, or add related task).


Action
Description
Relative URL
Example
Create New Object/Record
Go to the new record screen for that object
/<object key prefix>/e?retURL=<PreviousURL>
https://na1.salesforce.com/001/e?retURL=https://emea.salesforce.com/001/e?retURL=%2F001%2Fo
Edit Object/Record
Go to the edit record screen for a specific object/record
/<id>/e?retURL=<PreviousURL>
https://emea.salesforce.com/00120000003vqFD/e?retURL=%2F00120000003vqFD
Clone Object/Record
Go to the clone record screen for that object/record
/<id>/e?retURL=<PreviousURL>&clone=1
https://emea.salesforce.com/00120000003vqFD/e?retURL=%2F00120000003vqFD&clone=1
Add a Related Task
Schedule a task associated to a particular object/record
/<object key prefix>/e?retURL=<PreviousURL>&what_id=<id>
https://emea.salesforce.com/00U/e?retURL=%2F00620000002EV2x&what_id=00620000002EV2x
Transfer a Record
Transfer Ownership (affects sharing)
/<id>/a
https://emea.salesforce.com/00120000003vqFD/a


Link Structure Key:
  • <server> - Name of the salesforce instance (e.g. na1, na2, ap, emea, etc…)
  • <object key prefix> - The key prefix of the type of object (e.g. 001, 003, 005)
  • <id> - unique record id of the particular record or object including key prefix and server identification
  • <PreviousURL> - this is the relative (meaning, starting with “/” instead of the fully qualified domain name “https://na1.salesforce.com”) URL of where you’d like to return after completing the edit. This could be any of the View Links listed above. If the return link is not included, salesforce returns the homepage. Note: the use of “%2F” is code for the character “/”, which is needed to create the return link correctly.
  • what_id - this is the id that the task should be associated to, whether Account, Contact, Opportunity, etc…


Creating New Objects and Pre-Populating Fields


Using one of the Edit Links listed above, a custom link can be created which pulls data from a record and populates another record. For example, a custom link can be created from an Account page which creates a case and pre-populates some of the case fields with data from the Account record.


Relative URL
Example
/<object key prefix>/e?retURL=<PreviousURL>&<field id1>=<value>&<field...>
/500/e?retURL=%2F500%2Fo&cas8={!Account_Customer_Priority}


Link Structure Key:
  • <object key prefix> - The key prefix of the type of object (e.g. 001, 003, 005)
  • <field id1> - this is the identifier of the field in the html of the Edit object page. To get this id, View Source of the edit object page, and search on the Field Label. The control (radio button, check box, select box, etc…) will have an id attribute. The value for this id attribute is what you need to add to the custom link. Note: be careful when viewing the source of a page as salesforce reserves the right to change the page attributes in future releases which can break automation built around those attributes.
  • <value> - this is the value you will pass into the field. This could be a literal value (e.g. “1” or “High”), or a merge field from the record you’re coming from (e.g. {!Account_Customer_Priority}).


Passing Parameters to a Report

Using a custom link, a user can launch to a report and pass parameters to make the report unique.


Relative URL
Example
/<report object id>?pv0=<par1value>&pv1=<par2value>&pv2...
https://na1.salesforce.com/00O30000001CrJx?pv0={!Account_Name}


Link Structure Key:
  • <report object id> - id of the specific report to run
  • pv0 - list each parameter passed to the report (e.g. pv0, pv1, pv2, etc…)
  • <par1value> - specific value for the parameter, which could be a merge field (e.g. {!Account_Name}).


Understanding the salesforce URL link structure can open doors to salesforce customizations. Don't be afraid to experiment and let me know how it goes! Happy linking!!