Versions Compared

Version Old Version 1 New Version Current
Changes made by

Juliane Wetzel

Juliane Wetzel

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

Table of Contents

Info
titleINFO
Salesforce is an enterprise cloud computing firm. It specializes in social and mobile cloud technologies, including sales and CRM applications helping companies connect with customers, partners, and employees.

Enabling TLS 1.1

As of July 22, 2017, Salesforce is disabling their use of TLS version 1.0. If you are running Datameer X using Java 7, you need to manually update to TLS 1.1 or 1.2. First, you need to update TLS with Java 7. Then, add the following properties to the Custom Properties section of your Salesforce import job or in the Custom Properties section on the Hadoop Cluster page of the Admin tab.

For Tez, add the following properties:

tez.am.launch.cmd-opts="-Dhttps.protocols=TLSv1.1,TLSv1.2"
tez.task.launch.cmd-opts="-Dhttps.protocols=TLSv1.1,TLSv1.2"

For Spark, add the following properties:

as.spark.launcher.spark-submit-opts="-Dhttps.protocols=TLSv1.1,TLSv1.2"
spark.executor.extraJavaOptions="-Dhttps.protocols=TLSv1.1,TLSv1.2"
spark.driver.extraJavaOptions="-Dhttps.protocols=TLSv1.1,TLSv1.2"

For MapReduce, add the following properties:

mapreduce.map.java.opts="-Dhttps.protocols=TLSv1.1,TLSv1.2"


...

Prerequisites


Info
titleINFO
Make sure you have your Salesforce instance configured accordingly. Datameer will retrieve the data with an OAuth token - without revealing a username and a password to Datameer X.

To prepare your Salesforce instance:

  1. Login as an Admin or ask an authorized person to execute the setup.
  2. Create a new 'Connected App'. The steps differ depending on whether you us an old or a new Salesforce Interface:
    - Old UI: "Built" → "Create" → "Apps" → "Connected Apps" → "New"
    - New UI: "Platform Tools" → "Apps" → "Connected Apps" → "App Manager" → "New Connected App"
  3. Switch to the "API (Enable OAuth Settings)" section and tick the "Enable OAuth Settings" checkbox.  

  4. Set the Callback URLs (Datameer Application Server URL):

    Code Block
    http://<datameer application server hostname>:<port>/properties/external-callback-button
https://<datameer application server hostname>:<port>/properties/external-callback-button


  5. Add the following OAuth scopes:
    - Full Access
    - Manage Use Data via Web Browsers
    - Platform Requests at Any Time
  6. Leave all the rest default parameters and save the settings.
  7. Open this Connected App settings and pick up its client ID and client secret values. 

To prepare your Datameer X instance:

  1. Open the 'live.properties" file under '/<Datameer installation folder>/conf/live.properties' and add the following lines at the end:

    Code Block
    #Salesforce Connectetd Apps details for get OAuth token
system.property.webservice.salesforce.oauth.client-id=<client-id of the Saleforce Connected App>
system.property.webservice.salesforce.oauth.client-secret=<client-secret of the Saleforce Connected App>


  2. Restart Datameer X. 

Configuring Salesforce as a Connection

  1. Click the "+

...

  1. " button and select "Connection" or right-click in the

...

  1. File Browser and select 

...

  1. "Create New" → "Connection". The "New Connection" tab appears in the menu bar.
    Image Added

  2. Select "Salesforce" from the drop-down and confirm with "Next". The type is displayed in the drop-down.
    Image Added

  3. Select the web service protocol to be used from the drop-down.
    Image Added

  4. Enter the host address and the port, where the web service is running on.

    Image Added

  5. Enter the root path.

    Image Added

  6. To authenticate, click on “Authorize Datameer to retrieve data”. You are then taken to

...

  1. the http://salesforce.com portal to generate an OAuth access token for Datameer. The OAuth code is automatically transferred to the appropriate field.

...


  1. Image Added

  2. Confirm with “Next”.

    Image Added

  3. If needed, enter a description and confirm with "Next". The 'Save Connection' dialog opens.
    Image Added

  4. Select the folder to save the connection, enter a name in "Save as" and confirm with "Save". The connection is saved. Configuring the Salesforce connection is finished. 
    Image Added

Importing Data with the Salesforce Connector 

Applicable Custom Properties

Info
titleINFO

You can use custom properties to modify the importing process to your needs.


Custom PropertyExample ValueDefault ValueDescription
datameer.plugin-webservice.read-timeout.ms120000 (~2 minutes)120000
  • sets a HTTP response data read timeout
datameer.salesforce.maxrecords-per-result-request600000determined by the Salesforce API endpoint
  • specifies the maximum number of records within a batch
datameer.salesforce.max-parallel-result-request21
  • specifies the maximum number of batches, that are fetched in parallel

Importing Data with the Salesforce Connector

  1. Click the + (plus) button and select Import Job or right-click in the browser and select Create new > Import Job.     

  2. Click Select Connector, choose the Salesforce connector, and click 

...

  1. Next.

...

  2. Enter a search Salesforce Query (SOQL) and click 

...

  1. Next.

...

  2. Review the defined fields and click 

...

  1. Next.

  2. Review the schedule and advanced properties for the job and click 

...

  1. Next.

  2. Create a description for the import job and click Save.