But recently, one of those customers reached out with a problem. He was a Salesforce administrator on a Windows machine. That meant he had to write code or use one of my stock Bash shell scripts. Most administrators aren't used to writing code and Windows as a platform doesn't support Bash shell scripts without a lot of elbow grease, duct tape, and spit.
To fix this, I created a Python script that runs on multiple platforms including Windows, Linux, and Mac OSX. You can download the script from my Github repo. You will need to install Python version 2.7.9 on your machine first; however, that's far easier than trying to get a Bash shell script working with Cygwin.
Why python? Because it's easy to learn, it's easy to read, it has incredible library support, and most importantly, it supports multiple platforms including some I haven't even heard of!
To run the script, open a terminal (or cmd on Windows) and type:
python elf.py
or if you have multiple versions of Python, including 2.7.9, installed already:
python2.7 elf.py
The script is very simple and takes four prompts:
- Username
- Password (hidden)
- Date range
- Download directory
Beyond that, it's as easy as running the script from a terminal or command prompt and you're ready to start downloading Event Log Files on Mac, Linux, or Windows.
Hi Adam -- I don't see any other comments -- how can this be possible?!
ReplyDeleteThanks so much for this -- I'm having some difficulties in my first attempt to run, but will do some more diagnosis before bothering you further. It can't be because I downloaded the default, Python 2.7.10 instead of your cited 2.7.9, can it? Can the dot version be that different?
If I've piqued your curiosity, here's the output I'm getting:
C:\Users\David\Downloads>python elf.py
Username:
dbrenn23@optonline.net.legacy1
Password:
Using user inputed username: dbrenn23@optonline.net.legacy1
Traceback (most recent call last):
File "elf.py", line 236, in
download_elf()
File "elf.py", line 112, in download_elf
access_token, instance_url = login()
File "elf.py", line 94, in login
res = urllib2.urlopen(req)
File "C:\Python27\lib\urllib2.py", line 154, in urlopen
return opener.open(url, data, timeout)
File "C:\Python27\lib\urllib2.py", line 437, in open
response = meth(req, response)
File "C:\Python27\lib\urllib2.py", line 550, in http_response
'http', request, response, code, msg, hdrs)
File "C:\Python27\lib\urllib2.py", line 475, in error
return self._call_chain(*args)
File "C:\Python27\lib\urllib2.py", line 409, in _call_chain
result = func(*args)
File "C:\Python27\lib\urllib2.py", line 558, in http_error_default
raise HTTPError(req.get_full_url(), code, msg, hdrs, fp)
urllib2.HTTPError: HTTP Error 400: Bad Request
C:\Users\David\Downloads>
I plead Sunday-Morning-Before-9-itis. Log indicates that I didn't put in my API Token. . Tried appending token to password - first attempt a no-go, but may still be user/typo error. If not, good opp for me to learn more about Python by building a separate Token prompt into your script. I'll be back...
DeleteThis comment has been removed by the author.
ReplyDeleteHi Adam,
ReplyDeleteTried to download the logs using your script with Python 279. It will not log me into Salesforce neither do I see a rejection in the SF login history with my user. The error python throws is:
Traceback (most recent call last):
File "elf.py", line 236, in
download_elf()
File "elf.py", line 112, in download_elf
access_token, instance_url = login()
File "elf.py", line 94, in login
res = urllib2.urlopen(req)
File "C:\Localdata\Programma's\Python279\lib\urllib2.py", line 154, in urlopen
return opener.open(url, data, timeout)
File "C:\Localdata\Programma's\Python279\lib\urllib2.py", line 431, in open
response = self._open(req, data)
File "C:\Localdata\Programma's\Python279\lib\urllib2.py", line 449, in _open
'_open', req)
File "C:\Localdata\Programma's\Python279\lib\urllib2.py", line 409, in _call_chain
result = func(*args)
File "C:\Localdata\Programma's\Python279\lib\urllib2.py", line 1240, in https_open
context=self._context)
File "C:\Localdata\Programma's\Python279\lib\urllib2.py", line 1197, in do_open
raise URLError(err)
urllib2.URLError:
Regards,
Jos
Hi Jos,
DeleteSorry you're encountering this error. I didn't put enough error handling in the original script. Couple of suggestions:
1. Even though you're not getting a login history failure row, try logging in as an API user with a profile that has ip restrictions: 0.0.0.0 - 255.255.255.255. I wouldn't keep this setting permanent but it will tell us if it's a security token issue.
2. Add print res and print req statements into the login block
3. Implement try catch in login block
If that doesn't work, we can set up some time to talk via goto or google hangout to review what's happening. Thanks!
AT
Thanks for the post. No issues for me after I updated my PATH environment variable on my Windows machine. Can you offer any suggestions on how to automate running this script in windows? How can we store the password securely?
ReplyDeleteI don't know of any good job schedulers on Windows unfortunately. I have a sample script for Linux to hide a password but not sure how on Windows unfortunately. Hope someone in the community who knows Windows better might have a good answer.
ReplyDeleteThis comment has been removed by the author.
DeleteThis comment has been removed by the author.
DeleteThanks for your reply to that. I am "Unknown" .. I may switch gears then to Linux. Can you provide that script? And I guess i could schedule the script via cron, right?
DeleteI usually use crontab on Linux. It's pretty easy to find tutorials online for it. And it's easier than Launchd on Mac for scheduled jobs imho...
DeleteI scheduled it to run using the Task Scheduler. This link might help anyone who hasn't figured it out : http://desktop.arcgis.com/en/arcmap/10.3/analyze/executing-tools/scheduling-a-python-script-to-run-at-prescribed-times.htm
DeleteThis comment has been removed by the author.
ReplyDeleteThanks for writing this script. I notice that it downloads the login and logout activities for all users. Does it also download other activities like "Exported Reports"?
ReplyDeleteIt does but only in Developer Edition orgs or if you have purchased the full Event Monitoring product offering for your production org where we support 32 event types.
DeleteHi Adam,
ReplyDeleteThanks for the script, it's perfect!
But sometimes it downloads just first four event types and nothing more.
That's what it downloaded yesterday when I selected last 4 days:
2016-03-17-ApexCallout.csv
2016-03-17-ApexExecution.csv
2016-03-17-ApexSoap.csv
2016-03-17-API.csv
2016-03-17-AsyncReportRun.csv
2016-03-18-ApexCallout.csv
2016-03-18-ApexExecution.csv
2016-03-18-ApexSoap.csv
2016-03-18-API.csv
2016-03-18-AsyncReportRun.csv
2016-03-19-ApexCallout.csv
2016-03-19-ApexExecution.csv
2016-03-19-ApexSoap.csv
2016-03-19-API.csv
2016-03-19-AsyncReportRun.csv
2016-03-20-ApexCallout.csv
2016-03-20-ApexExecution.csv
2016-03-20-ApexSoap.csv
2016-03-20-API.csv
2016-03-20-AsyncReportRun.csv
Could you please point on the possible solution?
Thank you!
we only generate files for event types where there's activity. For instance, if you didn't run any reports for a day, we wouldn't generate a report file.
DeleteIs it possible, you haven't done anything via the UI in the org for several days?
Hi Adam,
DeleteThanks for your reply. The main thing is that there were other event types, I'm sure because I've checked them through heroku event log browser.
I absolutely have no clue why it happens :(
Hi Adam,
ReplyDeleteI'm having trouble Finding my Client id and Client secret in SFDC. Also, for password, I' putting in password and token. Is that right ? I'm getting a 400 error.
Thank in advance fro your reply.
Regards,
Bill
Hi Adam,
ReplyDeleteI've got the client id and secret set up but am still getting this error:
Traceback (most recent call last):
File "D:\SFDCMonitor\elf.py", line 232, in
download_elf()
File "D:\SFDCMonitor\elf.py", line 124, in download_elf
res = urllib2.urlopen(req)
File "C:\Python27\lib\urllib2.py", line 154, in urlopen
return opener.open(url, data, timeout)
File "C:\Python27\lib\urllib2.py", line 437, in open
response = meth(req, response)
File "C:\Python27\lib\urllib2.py", line 550, in http_response
'http', request, response, code, msg, hdrs)
File "C:\Python27\lib\urllib2.py", line 475, in error
return self._call_chain(*args)
File "C:\Python27\lib\urllib2.py", line 409, in _call_chain
result = func(*args)
File "C:\Python27\lib\urllib2.py", line 558, in http_error_default
raise HTTPError(req.get_full_url(), code, msg, hdrs, fp)
HTTPError: HTTP Error 400: Bad Request
Hi Bill,
DeleteThanks for posting. Did you resolve this? One potential reason may be because of the Security Token? Did you try adding that into the password flow for the user? Thanks!
AT
Hi Adam,
ReplyDeleteI am not able to download data for Event Login.
When i have inserted username & password, it is giving me below error.
Using user inputed username: acutedge@cssny.org
check point
Traceback (most recent call last):
File "elf.py", line 239, in
download_elf()
File "elf.py", line 115, in download_elf
access_token, instance_url = login()
File "elf.py", line 97, in login
res = urllib2.urlopen(req)
File "C:\Python27\lib\urllib2.py", line 154, in urlopen
return opener.open(url, data, timeout)
File "C:\Python27\lib\urllib2.py", line 435, in open
response = meth(req, response)
File "C:\Python27\lib\urllib2.py", line 548, in http_response
'http', request, response, code, msg, hdrs)
File "C:\Python27\lib\urllib2.py", line 473, in error
return self._call_chain(*args)
File "C:\Python27\lib\urllib2.py", line 407, in _call_chain
result = func(*args)
File "C:\Python27\lib\urllib2.py", line 556, in http_error_default
raise HTTPError(req.get_full_url(), code, msg, hdrs, fp)
urllib2.HTTPError: HTTP Error 400: Bad Request
Please give any solution.
Hi Arvind - may need som emore information. From the traceback, all I know is that the login failed, not why. If you can't get this to work, you can also try the elf browser (https://trailhead.salesforce.com/modules/event_monitoring/units/event_monitoring_download) which bootstrap creates a bash file for the same purposes. thanks!
DeleteAny luck on solving for this? I'm getting the same error. when trying to download data.
DeleteHey Adam, Don't the log files change with each release? Is there a good way to parse the file to insert into Postgre? If the log files are constantly changing, then the Postgre schema would be broken with every release. Any suggestions on getting the log file data to Wave?
ReplyDeleteThere are two fields on event log file object that provide field name and data type. We created them so that you could introspect the schema in sandbox prior to a major release to check for changes.
ReplyDeleteAdam,
ReplyDeleteI do not know scripting but trying to use this and getting error:
Traceback (most recent call last):
File "C:\Users\esssrv\elf.py", line 239, in
download_elf()
File "C:\Users\esssrv\elf.py", line 115, in download_elf
access_token, instance_url = login()
File "C:\Users\esssrv\elf.py", line 97, in login
res = urllib2.urlopen(req)
File "C:\Python27\lib\urllib2.py", line 154, in urlopen
return opener.open(url, data, timeout)
File "C:\Python27\lib\urllib2.py", line 435, in open
response = meth(req, response)
File "C:\Python27\lib\urllib2.py", line 548, in http_response
'http', request, response, code, msg, hdrs)
File "C:\Python27\lib\urllib2.py", line 473, in error
return self._call_chain(*args)
File "C:\Python27\lib\urllib2.py", line 407, in _call_chain
result = func(*args)
File "C:\Python27\lib\urllib2.py", line 556, in http_error_default
raise HTTPError(req.get_full_url(), code, msg, hdrs, fp)
urllib2.HTTPError: HTTP Error 400: Bad Request
C:\Users\esssrv>
Adam or anyone, is there a way we can select just select event types?
DeleteAttempted to update the script, but had no luck only pulling 2 events types that my company would like to retain. Currently running into a memory error when pulling APEX Triggers that I am not able to get around.
At the start of July the script was pulling multiple days worth of values before I would run into this error.
Hi Mike,
ReplyDeleteI adjusted the script in a way that might help you.
With these lines you can select timing and event type at prompt:
if len(day) < 1:
day = 'Last_n_Days:2'
print 'Using default date range: ' + day + '\n'
else:
print 'Using user inputed date range: ' + day + '\n'
et = raw_input('\nType: \n')
# query Ids from Event Log File
url = instance_url+'/services/data/v33.0/query?q=SELECT+Id+,+EventType+,+Logdate+From+EventLogFile+Where+LogDate+='+ day +'+and+EventType+='+ et
Hope that helps.
Jos
Nice!
DeleteHi Adam,
ReplyDeleteI am also facing same issue
File "elf.py", line 236, in
download_elf()
File "elf.py", line 112, in download_elf
access_token, instance_url = login()
File "elf.py", line 94, in login
res = urllib2.urlopen(req)
File "C:\Python27\lib\urllib2.py", line 154, in urlopen
return opener.open(url, data, timeout)
File "C:\Python27\lib\urllib2.py", line 437, in open
response = meth(req, response)
File "C:\Python27\lib\urllib2.py", line 550, in http_response
'http', request, response, code, msg, hdrs)
File "C:\Python27\lib\urllib2.py", line 475, in error
return self._call_chain(*args)
File "C:\Python27\lib\urllib2.py", line 409, in _call_chain
result = func(*args)
File "C:\Python27\lib\urllib2.py", line 558, in http_error_default
raise HTTPError(req.get_full_url(), code, msg, hdrs, fp)
urllib2.HTTPError: HTTP Error 400: Bad Request
What i noticied is it's taking user name in my case & password even if i try to type it is taking blank
I had problems with v 2.7 and the TLS upgrade in orgs. Have you tried upgrading to v 3.4 of python? It may help and only require minor refactor.
DeleteHello Adam,
ReplyDeleteI'm also getting the "HTTPError: HTTP Error 400: Bad Request" error. I have Python 2.7.9 installed.
I tried attaching my two factor code to my password since my API logins require two factor, but it wasn't working. Perhaps I'm misunderstanding how the API logins work. Any ideas?
Thanks for your help!
Hi Adam,
DeleteYou have to create a Connected App under Setup->Apps and copy the Consumer key and Consumer secret in lines 41 and 42 of the python script. You should then be able t run it fine
This comment has been removed by the author.
ReplyDeleteHi
ReplyDeleteYou have to create a Connected App under Setup->Apps and copy the Consumer key and Consumer secret in lines 41 and 42 of the python script. You should then be able t run it fine