For example, a simple integration might periodically poll Salesforce for new leads. When a Salesforce connection finds a new lead it can send a message that contains the details to a Knative channel. Multiple simple integrations can obtain messages from that same Knative channel. Each integration can perform different actions on the lead data.
See the following topics for details:
Enabling connections to Knative resources
You must enable Knative connections before you can add them to simple integrations.
Knative connections might have been enabled when Syndesis was
installed. To check this, view the content of the syndesis
custom resource.
If the camelk
and knative
add-ons specify enabled: true
then Knative
connections are already enabled.
-
The
oc
client tool is installed in, and connected to, the OCP cluster in which Syndesis is running. -
Syndesis is running on OpenShift Container Platform on-site in a customer-managed environment.
-
The Syndesis runtime is Camel K.
-
In the cluster in which Syndesis is installed, each Knative resource that you want an integration to connect to is also installed. Before you can install Knative resources, you must install the OpenShift Serverless Operator and the Knative Eventing Operator.
-
You have permission to install Syndesis.
-
Log in to OpenShift with an account that has permission to install Syndesis. For example:
$ oc login -u developer -p developer
-
Switch to the project in which Syndesis is running, for example:
$ oc project my-integration-project
-
Edit the
syndesis
custom resource:-
Invoke the following command, which typically opens the resource in an editor:
oc edit syndesis
-
Update the resource to specify the following:
spec: addons: camelk: enabled: true knative: enabled: true
-
Save the resource.
Saving this change to the
syndesis
custom resource triggerssyndesis-operator
, which is responsible for installing Syndesis, to redeploysyndesis-server
.If
camelk
andknative
are already enabled, you can close the file. Connections to Knative resources are enabled.
-
In the Syndesis console, on the Connections page, a Knative connection is now available. You can add the same Knative connection to any number of simple integrations. Configuration of the connection action, specifies the Knative resource that the connection accesses.
Subscribing to a Knative channel to obtain messages
To trigger integration execution upon receiving a message from a Knative channel, add a Knative connection as a simple integration’s start connection. An integration that starts this way automatically scales down to 0 pods when no messages have been received from the channel for a specified number of minutes. The default is 1 minute.
This automatic scaling saves resources, which is an advantage over simple integrations that start with other kinds of connections, such as a connection to an AMQ broker.
-
Connections to Knative resources are enabled.
-
The Knative resource that you want to obtain messages from is in the same cluster as Syndesis
-
You know the name of the Knative channel that you want the connection to obtain messages from.
-
In Syndesis, in the left panel, click Integrations.
-
Click Create Integration.
-
On the Choose a connection page, click the Knative connection.
-
Select the Receive from Channel action.
-
Click in the Channel name field to display a list of available Knative channels.
-
Select the Knative channel name for the resource that you want to obtain messages from.
-
Click Next to specify the Receive from Channel action’s output type. This is the type that the connection passes to the next step in the integration.
-
In the Select Type field, if the data type does not need to be known, accept Type specification not required and click Next. You do not need to follow the rest of these instructions.
However, a structured data type is recommended. For example, if you want to map the connection output in a data mapper step then you must specify the data type. The data mapper cannot display fields for unstructured data.
To specify the data type, click in the Select Type field and select one of the following as the schema type:
-
JSON schema is a document that describes the structure of JSON data. The document’s media type is
application/schema+json
. -
JSON instance is a document that contains JSON data. The document’s media type is
application/json
. -
XML schema is a document that describes the structure of XML data. The document’s file extension is
.xsd
. -
XML instance is a document that contains XML data. The document’s file extension is
.xml
.
-
-
In the Definition input box, paste a definition that conforms to the schema type you selected. For example, if you select JSON schema then you would paste the content of a JSON schema file, which has a media type of
application/schema+json
. -
In the Data Type Name field, enter a name that you choose for the data type. For example, if you are specifying a JSON schema for vendors then you might specify
Vendor
as the data type name.You will see this data type name when you are creating or editing an integration that uses the connection for which you are specifying this type. Syndesis displays the type name in the integration visualization and in the data mapper.
-
In the Data Type Description field, provide information that helps you distinguish this type. This description appears in the data mapper when you hover over the step that processes this type.
-
Click Next.
-
Syndesis prompts you to add the simple integration’s finish connection.
During execution, each time the selected Knative channel receives a new message, the Knative connection passes the message to the next step in the integration,
Add all needed connections to the integration. Then add a data mapper step
after the Knative connection that starts the integration. Map the source
message
field to a target field in the next connection.
Sending messages to a Knative channel
In a simple integration, a Knative connection can send a message to a Knative channel in the middle of a flow or to finish a simple integration. To do this, add a Knative connection to the middle of a flow or as a simple integration’s finish connection.
-
Knative connections are enabled.
-
The Knative resource that you want to send messages to is installed in the same cluster as Syndesis.
-
You know the name of the Knative channel that you want the connection to send messages to.
-
You are creating or editing a flow and Syndesis is prompting you to add to the integration. Or, Syndesis is prompting you to choose a finish connection.
-
On the Add to Integration page, click the plus sign where you want to add the connection. Skip this step if Syndesis is prompting you to choose the finish connection.
-
On the Choose a connection page, click the Knative connection.
-
Select the Send to Channel action.
-
Click in the Channel name field to display a list of the Knative channels that are available in the cluster that is running Syndesis.
-
Select the channel that you want the integration to send messages to.
-
Click Next to specify the Send to Channel action’s input type. This is the type that the integration sends to the Knative channel.
-
In the Select Type field, if the data type does not need to be known, or if the output from the previous integration step is the expected type, accept Type specification not required and click Next. You do not need to follow the rest of these instructions.
However, a structured data type is recommended. For example, if you want to map the connection input in a data mapper step then you must specify the data type. The data mapper cannot display fields for unstructured data.
To specify the data type, click in the Select Type field and select one of the following as the schema type:
-
JSON schema is a document that describes the structure of JSON data. The document’s media type is
application/schema+json
. -
JSON instance is a document that contains JSON data. The document’s media type is
application/json
. -
XML schema is a document that describes the structure of XML data. The document’s file extension is
.xsd
. -
XML instance is a document that contains XML data. The document’s file extension is
.xml
.
-
-
In the Definition input box, paste a definition that conforms to the schema type you selected. For example, if you select JSON schema then you would paste the content of a JSON schema file, which has a media type of
application/schema+json
. -
In the Data Type Name field, enter a name that you choose for the data type. For example, if you are specifying a JSON schema for vendors then you might specify
Vendor
as the data type name.You will see this data type name when you are creating or editing an integration that uses the connection for which you are specifying this type. Syndesis displays the type name in the integration visualization and in the data mapper.
-
In the Data Type Description field, provide information that helps you distinguish this type. This description appears in the data mapper when you hover over the step that processes this type.
-
Click Next.
-
In the integration visualization, the connection appears where you added it. During execution, this connection sends a message to the selected Knative channel.
Add all needed connections to the integration. Then add a data mapper
step just before the Knative connection that sends messages to a Knative channel.
Map a source field to the Knative connection target message
field.
Exposing a simple integration as a Knative service
In a simple integration, the start connection can be a Knative connection that exposes the integration as a Knative service. An integration that starts this way automatically scales down to 0 pods when there have been no calls to the service for a specified number of minutes. The default is 1 minute.
This automatic scaling saves resources, which is an advantage over simple integrations that start with other kinds of connections, such as a connection to an AMQ broker.
-
Connections to Knative resources are enabled.
-
Knative is installed in the same cluster that Syndesis is running in.
-
In Syndesis, in the left panel, click Integrations.
-
Click Create Integration.
-
On the Choose a connection page, click the Knative connection.
-
Select the Expose as Service action.
Syndesis prompts you to add the simple integration’s finish connection.
When Syndesis publishes the integration, the Camel K runtime creates a
Knative serving.knative.dev
service. This service:
-
Is an alternative to a Kubernetes deployment object
-
Can automatically scale to 0 after a period of no activity
-
Should not be confused with the Kubernetes Service
Publication makes the integration available to be called at the root path ( / ). Clients can invoke execution of the integration by calling it as an HTTP endpoint and passing data as needed. Typically, the passed data is in JSON format.
In this release, it is expected that the primary use for exposing an integration as a Knative service is that other Syndesis integrations call that service. In this scenario, the integration that calls the service selects the service name and does not need to specify a path.
Calling a Knative service from a simple integration
In a simple integration, a Knative connection can call a Knative service in the middle of the integration or to finish the integration. To do this, add a Knative connection to the middle of a flow or as a simple integration’s finish connection.
-
Knative connections are enabled.
-
You know the name of the Knative service that you want the connection to call. The service is running in the same cluster as Syndesis.
-
You are creating or editing a flow and Syndesis is prompting you to add to the integration. Or, Syndesis is prompting you to choose a finish connection.
-
On the Add to Integration page, click the plus sign where you want to add the connection. Skip this step if Syndesis is prompting you to choose the finish connection.
-
On the Choose a connection page, click the Knative connection.
-
Select the Call Service action.
-
Click in the Service name field to display a list of the Knative services that are available in the cluster that is running Syndesis.
-
Select the service that you want the integration to call.
If you exposed an integration as a Knative service and that is the service that you want to call, the name of the service is similar, if not identical, to the name of the integration.
-
Click Next to specify the Call Service action’s input type. This is the type that the integration sends to the Knative service.
-
In the Select Type field, if the data type does not need to be known, or if the output from the previous integration step is the expected type, accept Type specification not required and click Next. You do not need to follow this subset of instructions.
A structured data type is recommended. For example, if you want to map the connection input in a data mapper step then you must specify the data type. The data mapper cannot display fields for unstructured data.
To specify the data type, click in the Select Type field and select one of the following as the schema type:
-
JSON schema is a document that describes the structure of JSON data. The document’s media type is
application/schema+json
. -
JSON instance is a document that contains JSON data. The document’s media type is
application/json
. -
XML schema is a document that describes the structure of XML data. The document’s file extension is
.xsd
. -
XML instance is a document that contains XML data. The document’s file extension is
.xml
.
-
-
In the Definition input box, paste a definition that conforms to the schema type you selected. For example, if you select JSON schema then you would paste the content of a JSON schema file, which has a media type of
application/schema+json
. -
In the Data Type Name field, enter a name that you choose for the data type. For example, if you are specifying a JSON schema for vendors then you might specify
Vendor
as the data type name.You will see this data type name when you are creating or editing an integration that uses the connection for which you are specifying this type. Syndesis displays the type name in the integration visualization and in the data mapper.
-
In the Data Type Description field, provide information that helps you distinguish this type. This description appears in the data mapper when you hover over the step that processes this type.
-
Click Next.
-
-
Specify the Call Service action’s output type.
This is the type that the Knative service returns to the integration when the Knative connection is a middle connection.
Repeat the previous subset of instructions to specify the output type. Or, if the Knative service response is empty, accept Type specification not required.
-
Click Next to add the connection.
In the integration visualization, the connection appears where you added it.
During execution, this connection calls the specified Knative service. The call includes the output from the previous integration step (the step before this Knative connection). If this Knative connection is a middle connection, the connection passes the service output to the next step in the integration.
Add any other needed connections to the integration. If you specified input and/or output types, add data mapper steps before and/or after the Knative connection and map fields as needed.
Connecting to Apache Kudu
Apache Kudu is a columnar storage manager developed for the Apache Hadoop platform. A Syndesis integration can connect to a Kudu data store to scan a table, which returns all records in the table to the integration, or to insert records into a table. Details are in the following topics:
Creating a connection to an Apache Kudu data store
In an integration, to obtain records from or insert records into a Kudu table, create a connection to a Kudu master host and then add that connection to an integration.
-
You must know the IP address or the hostname for the Kudu master host that you want to connect to.
-
You must know the port that Kudu is listening on.
-
In Syndesis, in the left panel, click Connections to display any available connections.
-
Click Create Connection to display connectors.
-
Click the Apache Kudu connector.
-
To configure the connection:
-
In the Address of Kudu master host field, enter the hostname or the IP address of the Kudu master host.
-
In the Port to establish connection to field, enter the port that Kudu is listening on. The default is
7051
.
-
-
Click Next.
-
In the Name field, enter your choice of a name that helps you distinguish this connection from any other connections. For example, you might enter
Kudu North
. -
In the Description field, optionally enter any information that is helpful to know about this connection.
-
Click Save to see that the connection you created is now available. If you entered the example name, you would see that Kudu North appears as a connection that you can choose to add to an integration.
Triggering an integration when scanning returns records from a Kudu table
To trigger execution of an integration upon obtaining data from a Kudu table, add a Kudu connection to a simple integration as its start connection. When the integration is running, the Kudu connection scans the table that you specified at the interval that you specified, obtains all records in the table, and passes a collection of the records to the next step in the integration.
A Kudu connection can obtain data from only one table. Between scans, if there are no changes to the data in the table that the connection scans, then the next scan returns the same data as the previous scan.
-
You created a Kudu connection.
-
The table that you want to obtain records from exists.
-
In the Syndesis panel on the left, click Integrations.
-
Click Create Integration.
-
On the Choose a connection page, click the Kudu connection that you want to use to start the integration.
-
On the Choose an action page, select the Scan a Kudu table action.
-
In the Table field, enter the name of the table that you want to obtain records from.
-
In the Period field, accept the default of one minute, or enter the interval at which you want the connection to scan the table and return the records that are in the table.
-
Click Next.
The connection is the simple integration’s start connection.
Add the integration’s finish connection and any other connections that you want to include in the integration. When the integration contains all the connections that are needed, consider whether you need to split the collection of records that the Kudu connection returns. If you want to execute integration steps for each record that you obtained from the Kudu table, then after the Kudu connection, add a split step. Also, you probably need to follow the Kudu connection with a data mapping step that maps data obtained from Kudu to fields in subsequent connections in the integration.
Inserting records into a Kudu table
In an integration, you can add records to a Kudu table in the middle of a flow or to finish a simple integration. To do this, add a Kudu connection to the middle of a flow or as a simple integration’s finish connection.
-
You created a Kudu connection.
-
You are creating or editing an integration and Syndesis is prompting you to add to the integration. Or, Syndesis is prompting you to choose a finish connection.
-
The table that you want to add records to exists.
-
On the Add to Integration page, click the plus sign where you want to add the connection. Skip this step if Syndesis is prompting you to choose the finish connection.
-
Click the Kudu connection that you want to use. Note that when a Kudu connection inserts data, the connection does not return anything.
-
On the Choose an action page, select Insert data into a Kudu table.
-
To configure the action, in the Table field, specify the name of the table to add records to.
It is important for you to have an understanding of how the Kudu table that you are adding records to is set up. For example, a Kudu table that you are adding records to might have a unique key. If you try to add a record that contains a key value that is already in the table, the Kudu connection does not add that record.
-
Click Next.
The connection appears in the integration visualization where you added it.
Consider whether you need to split a collection of records into individual records that a Kudu connection can add to a table. To do this, add a split step to the integration. The split step executes the subsequent steps in the integration once for each record. Also, you probably need a data mapper step before a Kudu connection that adds records to a table.
Connecting to MongoDB
MongoDB is a distributed NoSQL database. A Syndesis integration can obtain content from a MongoDB database or update content in a MongoDB database. To do this, create a MongoDB connection, and add it to an integration.
Details for connecting to a MongoDB database are in the following topics:
Creating a connection to a MongoDB database
Create a MongoDB connection so that you can connect to a MongoDB database in an integration.
-
You must know the host name for the MongoDB database that you want to connect to.
-
You must have a user name and password that authorizes access to the MongoDB database.
-
In Syndesis, in the left panel, click Connections to display any available connections.
-
Click Create Connection to display connectors.
-
Click the MongoDB connector.
-
Configure the connection:
-
In the Database host(s) field, specify one or more MongoDB database hosts that this connection can access. Separate multiple specifications of
host:port
with commas. Replacehost
with a host name, a virtual host name, or an IP address. For example:mongodb0.example.com:1234,mongodb1.example.com:1234
. -
In the Username field, enter a user name that is authorized to access the database.
-
In the Password field, enter the password for the user name that you specified.
-
Optional. In the Administration database field, if the database has an administration database, enter the name of the MongoDB administration database that is used to grant access permission. A MongoDB administrator should let you know whether the database that you want to access has an administration database.
-
Optional. In the Replica set name field, specify the name of the MongoDB replica set (cluster) that the connection can access.
-
In the Database field, enter the name of the MongoDB database that contains the collection(s) that you want the connection to read.
-
-
Click Validate. Syndesis immediately tries to validate the connection and displays a message that indicates whether validation is successful. If validation fails, revise the input parameter(s) and try again.
-
When validation is successful, click Next.
-
In the Name field, enter your choice of a name that helps you distinguish this connection from any other connections. For example, you might enter
MongoDB North
. -
In the Description field, optionally enter any information that is helpful to know about this connection.
-
Click Save to see that the connection you created is now available. If you entered the example name, you would see that MongoDB North appears as a connection that you can choose to add to an integration.
Add your MongoDB connection to an integration.
Triggering an integration when a MongoDB collection is updated
To trigger execution of an integration when a document is added to a MongoDB collection, add a MongoDB connection to a simple integration as its start connection. When the integration is running, the MongoDB connection monitors the database collection that you specified when you created the integration. Insertion of a new document into the collection that the connection is configured to monitor triggers execution of the integration. The MongoDB connection returns the new document to the integration and this data is available to the next step in the integration.
-
You created a MongoDB connection.
-
You know the name of the MongoDB collection that you want the connection to read.
-
If the collection that you want the connection to monitor is a capped collection, then you must also know:
-
The name of the MongoDB collection field that tracks incoming documents.
-
Whether persistent tracking is enabled for the database that contains the collection.
-
How to specify tail tracking information related to the data that you want the connection to read. MongoDB connections can use tail tracking information to resume an action after an unexpected connection termination. Tail tracking ensures that when the connection resumes its activity, it does not return duplicate data to the integration nor does it drop any data that needs to be returned to the integration.
-
-
In the Syndesis panel on the left, click Integrations.
-
Click Create Integration.
-
On the Choose a connection page, click the MongoDB connection that is configured to access the database that you want the integration to obtain updates from.
-
On the Choose an action page, select one of these actions:
-
Retrieve documents (stream) is typically the action that you want to select. It retrieves data from non-capped collections.
-
Retrieve documents (tail) is the action for retrieving data from capped collections.
-
-
Configure the action:
-
In the Collection name field, enter the name of the MongoDB collection that you want the connection to read.
If you are configuring the Retrieve documents (stream) action, this is the only required parameter. Go to step 6. If you are configuring the Retrieve documents (tail) action, follow the remaining substeps.
-
In the Collection field used to track incoming documents field, enter the name of the relevant MongoDB collection field. This field contains data that can be sorted, such as a date or an ID.
-
Select the Persistent tracking enabled checkbox if persistent tracking is enabled for the database that you want the connection to read.
-
Optional. In the Unique id to identify this tracking process field, specify a unique ID if there are multiple consumers that monitor the same collection. For more information, see the Camel documentation for Tailable Cursor Consumer.
-
Optional. In the DB used to store tail tracking field, enter the name of the MongoDB database that contains tail tracking information.
-
Optional. In the Collection used to store tail tracking field, enter the name of the collection that contains tail tracking information.
-
Optional. In the Field used to store tail tracking field, enter the name of the field that stores tail tracking information for the collection that this connection is reading.
To identify the tail tracking field, you can specify the tail tracking database or tail tracking collection. With one of these pieces of information, the connection can find the tail tracking field.
-
-
Click Next to specify the action’s output type.
-
In the Select Type field, if the data type does not need to be known, accept Type specification not required and click Next. You do not need to follow the rest of these instructions.
However, a structured data type is recommended. For example, if you want to map the connection output in a data mapper step then you must specify the data type. The data mapper cannot display fields for unstructured data.
To specify the data type, click in the Select Type field and select one of the following as the schema type:
-
JSON schema is a document that describes the structure of JSON data. The document’s media type is
application/schema+json
. -
JSON instance is a document that contains JSON data. The document’s media type is
application/json
. -
XML schema is a document that describes the structure of XML data. The document’s file extension is
.xsd
. -
XML instance is a document that contains XML data. The document’s file extension is
.xml
.
-
-
In the Definition input box, paste a definition that conforms to the schema type you selected. For example, if you select JSON schema then you would paste the content of a JSON schema file, which has a media type of
application/schema+json
. -
In the Data Type Name field, enter a name that you choose for the data type. For example, if you are specifying a JSON schema for vendors then you might specify
Vendor
as the data type name.You will see this data type name when you are creating or editing an integration that uses the connection for which you are specifying this type. Syndesis displays the type name in the integration visualization and in the data mapper.
-
In the Data Type Description field, provide information that helps you distinguish this type. This description appears in the data mapper when you hover over the step that processes this type.
-
Click Next.
The integration now has a start connection and Syndesis is prompting you to choose the finish connection.
During execution, the connection returns the inserted document in JSON format.
Operating on documents in a MongoDB collection
In an integration, you can search for, update, delete, count, or add a document to a MongoDB collection. To do this, add a MongoDB connection to the middle of a flow or as a simple integration’s finish connection.
-
You created a MongoDB connection.
-
You know the name of the MongoDB collection that you want the connection to operate on.
-
You are creating or editing an integration and Syndesis is prompting you to add to the integration. Or, Syndesis is prompting you to choose a finish connection.
-
On the Add to Integration page, click the plus sign where you want to add a MongoDB connection. Skip this step if you are adding a simple integration’s finish connection.
-
Click the MongoDB connection that you want to use.
-
On the Choose an action page, select the action that you want the connection to perform.
-
To configure the action, in the Collection name field, enter the name of the collection that you want the connection to operate on.
If you selected the Insert action:
-
You are done configuring the connection.
-
Ensure that the data that the integration passes to this MongoDB connection contains JSON notation that specifies the document to add to the collection.
-
Click Next to add this connection to the integration.
-
-
For all other actions, the following table provides information about what to specify in the second field for configuring the action.
The second field is a JSON notation expression that identifies the document(s) to operate on and possibly, details for the operation. If you plan to map an incoming source field to an identifier in the expression, specify
:#variable-name
in the expression.For completeness, the Insert action is described in this table as well.
Action What it does Parameter field Returns Count
Counts the number of documents that satisfy the count expression that you specify.
In the Count expression field, specify a JSON filter expression in the form of
{"test":"unit"}
. The action counts the number of documents that have atest
attribute value ofunit
.An instance of
java.lang.Long
that specifies the number of documents that were found.Delete
Removes one or more documents from the collection.
In the Delete expression field, specify a JSON expression in the form of
{"test":"unit"}
. The operation removes each document whosetest
attribute has a value ofunit
.An instance of
java.lang.Long
that specifies the number of documents that were deleted.Find
Looks for all documents that match the filter expression that you specify.
In the Filter expression field, specify a JSON notation string that identifies one or more documents.
JSON notation that specifies a list (an array). Each list element identifies a document that matches the filter expression and provides the document content. If no documents are found, the action returns an empty list.
You must be familiar with the data model of the collection. If the collection adheres to a JSON schema, you must be familiar with this schema because the returned list elements adhere to this schema. For additional information, see MongoDB Data Models.
To use output data from the Find action in a subsequent data mapper step, the collection must adhere to a JSON schema. This enables the data mapper to display data fields. For more information, see MongoDB documentation for JSON schema validation.Insert
Adds one or more documents to the collection.
There is no action configuration parameter for inserting a document. The data coming in to the MongoDB connection must contain JSON notation that specifies one ore more documents to add to the collection.
JSON notation that specifies a list (an array). Each list element is a document that this action inserted.
Update
Updates one or more documents in the collection.
In the Filter criteria field, specify an array of 2 arguments. The first argument is a filter expression that identifies the documents to update. The second argument is a MongoDB update statement that specifies how to update the documents. Update statement format is described in https://docs.mongodb.com/manual/reference/method/db.collection.update/. For example:
[{"_id":11},{$set: {"test":"updated!"}}]
.An instance of
java.lang.Long
that specifies the number of documents that were updated.Upsert
Updates one or more documents in the collection or creates a document if no documents match the Upsert expression.
In the Upsert expression field, specify JSON notation that identifies one or more documents. The action uses this expression to detect any matching documents. If the action finds a matching document(s) in the collection, it updates it. If the action does not find a matching document, the action creates a new document. The data coming into the MongoDB connection must contain JSON notation that specifies:
* An
_id
field to identify whether it is an insert or an update operation.* The content to add or update.
An instance of
com.mongodb.client.result.UpdateResult
that specifies either the number of updated documents or the ID of the new, inserted document.Each action returns a response. If this is a finish connection, then the response is lost, but you can add a Log step to capture the response.
-
Click Next.
-
Optional. If this is a finish connection, add a Log step to finish the integration instead. Configure the Log step to capture the response.
The connection appears in the integration visualization where you added it.
Add any other needed connections to the flow. Add any other steps that
process data. Complete the integration by adding any needed data mapping steps.
When you configured the MongoDB connection action, if you specified any
:#variable-name
identifiers in an action expression, be sure to map a
source field to the MongoDB connection variable field.
Connecting to MQTT
MQ Telemetry Transport (MQTT) is a lightweight, machine-to-machine, internet of things, connectivity protocol. In an integration, you can obtain messages from or publish messages to an MQTT broker. To do this, create a connection to the MQTT broker of interest and then add that connection to an integration flow. Details are in the following topics:
Creating a connection to an MQTT broker
In an integration, to obtain messages from or publish messages to an MQTT broker, create a connection to the MQTT broker of interest and then add that connection to an integration.
You must know the URL for the MQTT broker that you want to connect to.
-
In Syndesis, in the left panel, click Connections to display any available connections.
-
Click Create Connection to display connectors.
-
Click the MQTT Message Broker connector.
-
To configure the connection:
-
In the MQTT broker URL field, enter the location of the MQTT broker that you want to send data to or obtain data from. This is the only required field.
-
In the User Name field, optionally enter the user name for the MQTT account whose credentials you want to use to access the broker.
-
In the Password field, if you specified a user name, then specify the password associated with that account.
-
In the Client ID field, optionally enter the ID that allows connections to close and reopen without missing messages. The connection must subscribe to or publish to a topic.
-
-
Click Validate. Syndesis immediately tries to validate the connection and displays a message that indicates whether validation is successful. If validation fails, revise the input parameters and try again.
-
If validation is successful, click Next.
-
In the Name field, enter your choice of a name that helps you distinguish this connection from any other connections. For example, you might enter
MQTT West
. -
In the Description field, optionally enter any information that is helpful to know about this connection.
-
Click Save to see that the connection you created is now available. If you entered the example name, you would see that MQTT West appears as a connection that you can choose to add to an integration.
Obtaining a message from an MQTT broker to trigger integration execution
To trigger execution of an integration based on receiving a message from an MQTT broker, add an MQTT connection as the start connection. When the integration is running, the MQTT connection continuously watches for messages on the MQTT queue or topic that you specify. When the connection finds a message, it passes it to the next step in the integration. An MQTT connection handles one message at a time.
You created an MQTT connection.
-
In the Syndesis panel on the left, click Integrations.
-
Click Create Integration.
-
On the Choose a connection page, click the MQTT connection that you want to use to start the integration.
-
On the Choose an action page, select the Subscribe action to receive messages from the queue or topic that you specify.
-
In the MQTT queue/topic name field, enter the name of the queue or topic to subscribe to in order to receive data.
-
Click Next to specify the action’s output type.
-
In the Select Type field, if the data type does not need to be known, accept Type specification not required and click Next. You do not need to follow the rest of these instructions.
However, a structured data type is recommended. For example, if you want to map the connection output in a data mapper step then you must specify the data type. The data mapper cannot display fields for unstructured data.
To specify the data type, click in the Select Type field and select one of the following as the schema type:
-
JSON schema is a document that describes the structure of JSON data. The document’s media type is
application/schema+json
. -
JSON instance is a document that contains JSON data. The document’s media type is
application/json
. -
XML schema is a document that describes the structure of XML data. The document’s file extension is
.xsd
. -
XML instance is a document that contains XML data. The document’s file extension is
.xml
.
-
-
In the Definition input box, paste a definition that conforms to the schema type you selected. For example, if you select JSON schema then you would paste the content of a JSON schema file, which has a media type of
application/schema+json
. -
In the Data Type Name field, enter a name that you choose for the data type. For example, if you are specifying a JSON schema for vendors then you might specify
Vendor
as the data type name.You will see this data type name when you are creating or editing an integration that uses the connection for which you are specifying this type. Syndesis displays the type name in the integration visualization and in the data mapper.
-
In the Data Type Description field, provide information that helps you distinguish this type. This description appears in the data mapper when you hover over the step that processes this type.
-
Click Next.
The integration now has a start connection and Syndesis is prompting you to choose the finish connection.
Publishing a message to an MQTT broker
In an integration, you can publish a message to an MQTT broker in the middle of a flow or to finish a simple integration. To do this, add an MQTT connection to the middle of a flow or as the integration’s finish connection.
-
You created an MQTT connection.
-
You are creating or editing a flow and Syndesis is prompting you to add to the integration. Or, Syndesis is prompting you to choose a finish connection.
-
On the Add to Integration page, click the plus sign where you want to add an MQTT connection. Skip this step if you are adding a simple integration’s finish connection.
-
Click the MQTT connection that you want to use to publish a message.
-
On the Choose an action page, select Publish.
-
In the MQTT queue/topic name field, specify the name of the queue or topic to publish the message to.
-
Click Next to specify the action’s input type.
-
In the Select Type field, if the data type does not need to be known, accept Type specification not required and click Next. You do not need to follow the rest of these instructions.
However, a structured data type is recommended. For example, if you want to map the connection input in a data mapper step then you must specify the data type. The data mapper cannot display fields for unstructured data.
To specify the data type, click in the Select Type field and select one of the following as the schema type:
-
JSON schema is a document that describes the structure of JSON data. The document’s media type is
application/schema+json
. -
JSON instance is a document that contains JSON data. The document’s media type is
application/json
. -
XML schema is a document that describes the structure of XML data. The document’s file extension is
.xsd
. -
XML instance is a document that contains XML data. The document’s file extension is
.xml
.
-
-
In the Definition input box, paste a definition that conforms to the schema type you selected. For example, if you select JSON schema then you would paste the content of a JSON schema file, which has a media type of
application/schema+json
. -
In the Data Type Name field, enter a name that you choose for the data type. For example, if you are specifying a JSON schema for vendors then you might specify
Vendor
as the data type name.You will see this data type name when you are creating or editing an integration that uses the connection for which you are specifying this type. Syndesis displays the type name in the integration visualization and in the data mapper.
-
In the Data Type Description field, provide information that helps you distinguish this type. This description appears in the data mapper when you hover over the step that processes this type.
-
Click Next.
The connection appears in the integration visualization where you added it.
Connecting to OData
Open Data Protocol (OData) is a standard for building and consuming REST APIs. An OData service provides services to clients, such as Syndesis, through OData-compliant HTTP requests. A Syndesis integration can obtain entities from an OData service, and can update, create, or delete entities managed by an OData service. To do this, create an OData connection, and add it to an integration.
It is assumed that you are familiar with the OData specification.
Details for connecting to an OData service are in the following topics:
Creating a connection to an OData service
In an integration, to obtain, update, create, or delete entities that are managed by an OData service, you must first create a connection to that OData service.
-
You must know the base URL for the OData service that you want to connect to.
-
If that service uses authentication then you must have the required credentials, and, if required, an SSL certificate. However, most OData services are public and do not require authentication.
-
In Syndesis, in the left panel, click Connections to display any available connections.
-
Click Create Connection to display connectors.
-
Click the OData connector.
-
To configure the connection:
-
In the Service Root URL field, enter the base URL for the OData service that you want to access.
-
In the User Name field, if the service uses basic authentication, then enter your user name.
-
In the Password field, if the service uses basic authentication, then enter your password.
-
In the Server Certificate field, if the service requires it, paste the content of an SSL certificate.
Typically, a public OData service does not require a certificate in addition to the certificates provided by the browser. However, for an internal OData service, you might have created your own SSL certificate and self-signed it. Since this certificate is not trusted by a certificate authority, connections to your OData service would fail. To enable connections, paste the self-signed certificate here.
One way to get the certificate is to load the service in a browser. The steps after that depend on which browser you are using, but it will be something like the following: click the Not secure padlock symbol next to the address bar, then click View Certificate, export the displayed certificate to a file, copy the certificate, and paste it into this field.
-
-
Click Validate. Syndesis immediately tries to validate the connection and displays a message that indicates whether validation is successful. If validation fails, revise the input parameter(s) and try again.
-
When validation is successful, click Next.
-
In the Name field, enter your choice of a name that helps you distinguish this connection from any other connections. For example, you might enter
OData North
. -
In the Description field, optionally enter any information that is helpful to know about this connection.
-
Click Save to see that the connection you created is now available. If you entered the example name, you would see that OData North appears as a connection that you can choose to add to an integration.
Add your OData connection to an integration.
Triggering an integration when polling returns data from an OData service
To trigger execution of an integration upon obtaining data from an OData service, add an OData connection to a simple integration as its start connection. When the integration is running, the OData connection polls the service at intervals that you specify. When the connection finds data that satisfies the connection configuration, the connection passes the data to the next step in the integration.
You created an OData connection.
-
In the Syndesis panel on the left, click Integrations.
-
Click Create Integration.
-
On the Choose a connection page, click the OData connection that you want to use to start the integration.
-
On the Choose an action page, select the Read action.
-
Configure the Read action:
-
In the Resource Collection field, select the name of the OData resource that you want to query. Syndesis fetches data from the OData service to provide a list of available resource collections.
-
In the Entity Key Predicate field, to obtain a particular entity, identify the entity that you want by specifying its key predicate. For example, specify something like
UserName='Bob'
orCategories(1)
. To obtain multiple entities, leave this field blank. -
In the Query Options field, enter a query that you want to apply to the resource. Use OData syntax. For example,
$filter=startswith(name, 'N')
returns a message for each entity in the resource that has aname
field that starts withN
.You can specify both Entity Key Predicate and Query Options. If you do, the OData service obtains the specified entity and applies the query to that entity. For example, suppose that you set Entity Key Predicate to
UserName='russellwhyte'/Emails
and you set Query Options to$filter=contains($it,'example')
. The connection creates a request that looks something like this:https://services.odata.org/TripPinRESTierService/People(UserName='russellwhyte')/Emails?$filter=contains($it, 'example')
The service returns all email addresses for Russell Whyte that contain the domain
example
. -
Select the Filter Old Results checkbox to obtain a particular message only once.
-
Select the Split Results checkbox if you want the connection to return individual messages rather than a collection of messages.
Syndesis also provides discrete split and aggregate steps, which you can add to a flow. If you want to process individual messages in one or more steps and then aggregate the messages into a collection, do not select the Split Results checkbox. Instead, leave the checkbox empty and then add a split step to the flow after this connection. A split step is required if you want an aggregate step in the flow.
-
In the Interval Before Polling Starts field, accept the default of 1 second, or enter the length of time that you want to elapse before the connection starts to poll the OData service.
-
In the Delay field, accept the default of 30 seconds, or enter the interval at which you want the connection to poll the OData service.
-
In the Backoff Idle Threshold field, accept the default of 1 or enter an integer that indicates the number of consecutive polls that can return no data. After this number of polls, the connection increases the interval between subsequent polls. The connection determines the new length of the polling interval by multiplying the Delay value by the Backoff Multiplier value.
For example, suppose that the polling interval (the Delay value) is the default of 30 seconds, Backoff Idle Threshold is set to 5, and Backoff Multiplier is set to 12. After 5 consecutive polls that return no data, the connection waits 360 seconds (30 x 12) before polling again. The connection continues to poll every 360 seconds until a poll returns data. After a poll returns data, the connection resumes polling every 30 seconds.
-
In the Backoff Multiplier field, accept the default of 1 or enter an integer that indicates the multiplier for increasing the polling interval if the value set for the Backoff Idle Threshold is reached.
If you accept the default of 1 for the Backoff Multiplier, then the connection continues to poll at the specified interval no matter how many consecutive polls return no results.
The values that you specify for Backoff Idle Threshold and Backoff Multiplier are useful for reducing CPU overhead since the connection can automatically poll less often during quiet periods.
-
-
Click Next.
The integration now has a start connection and Syndesis is prompting you to choose the finish connection.
During execution, what the connection returns depends on what you specified in the Entity Key Predicate and Query Options fields. The OData connection can return:
-
A collection of entities or a collection of entity properties
For example, this might be all Person entities in the resource, or perhaps all Age properties for all Person entities. The connection returns the collection in one message. Syndesis executes each subsequent step in the flow once for the collection. However, when you configure the Read action, if you select Split Results, then the connection returns each entity or each property in its own message. Syndesis executes each subsequent step in the flow once for each message.
-
An entity or an entity property
For example, this might be the Person entity whose UserName property is Bob, or perhaps the Age property for the Person entity whose UserName is Bob. The connection returns the entity or the entity property in a message that it passes to the next step in the flow.
Add the integration’s finish connection and any other connections that you want to include in the integration. When the integration contains all the connections that are needed, if the OData connection returns a collection, consider whether you need to add a split step after the OData connection. An integration usually needs to map the data returned by the OData connection to fields that a subsequent connection in the flow can use. Sometimes you can map a collection, but more often you need to split the collection in order to map to the target fields.
After the OData connection, add a data mapper step to the flow. Exactly where in the flow depends on what you want the flow to do. For example, after the OData connection, you might add a basic filter step and then add a data mapper step.
Updating, creating, and deleting data that is managed by an OData service
In an integration, you can update a resource that is managed by an OData service in the middle of a flow or to finish a simple integration. To do this, add an OData connection to the middle of a flow or as a simple integration’s finish connection.
-
You created an OData connection.
-
You are creating or editing an integration and Syndesis is prompting you to add to the integration. Or, Syndesis is prompting you to choose a finish connection.
-
On the Add to Integration page, click the plus sign where you want to add an OData connection. Skip this step if you are adding a simple integration’s finish connection.
-
Click the OData connection that you want to use. Note that when an OData connection updates or deletes an entity, the connection does not return anything. When the OData connection creates an entity, the connection returns the new entity.
-
On the Choose an action page, select the action that you want the connection to perform:
-
Create adds an entity to an OData resource.
-
Delete removes an entity from an OData resource.
-
Update changes an entity that is in an OData resource.
-
-
Configure the action by selecting the OData resource that you want to update.
It is important for you to have an understanding of how the OData resource that you are updating is set up. The backing data source for an OData service determines the rules for updates and resolving conflicts. For example, suppose an OData connection tries to create a new entity but an entity with that predicate key already exists. The OData service might overwrite the existing entity, or it might update some fields in the existing entity, or it might ignore the operation. It is up to you to know how the OData service behaves in this situation.
-
Click Next.
The connection appears in the integration visualization where you added it.
Add a data mapper step before the OData connection. You must map source fields that provide the data that is required to create a new entity, to update an entity, or to delete an entity. See Mapping integration data to fields for the next connection.
Connecting to REST APIs
In an integration, to connect to a REST API, you must have created a connector for that API by uploading an OpenAPI document that describes the API. See Integrating Applications, Adding and managing REST API client connectors.
When a connector for the REST API you want to connect to is available in Syndesis, the steps for connecting to that REST API are:
Registering Syndesis as a REST API client
Before Syndesis creates a REST API client connector, it prompts you to indicate the API’s security requirements. For APIs that use the OAuth protocol, when Syndesis creates the connector it also adds an entry for the API to the Syndesis Settings page. This is where you provide the API client ID and the API client secret that authorize Syndesis to access the API.
If the API you want to connect to does not use OAuth, skip this section and see Creating a REST API client connection.
You must know the URL for the OAuth custom application setting page for the REST API that you want to connect to.
-
In Syndesis:
-
On the Settings page, near the top, to the right of the callback URL, click to copy the callback URL for your Syndesis environment to the clipboard. You will need this URL later in this procedure.
-
Look for the name of the API you want to connect to and click it to display its client ID and client secret fields.
-
-
In another browser window, you must register Syndesis as an OAuth client of the API you want to connect to. The exact steps for doing this vary for each API service. Typically, the API service provides an OAuth custom application setting page. Go to that page and do the following:
-
Provide the Syndesis callback URL, which you copied at the beginning of this procedure.
-
Respond to any other prompts that require your input.
-
Obtain the client ID and client secret that the API service assigns to your Syndesis environment.
-
-
Return to the Syndesis Settings page entry for the API service you are registering with and do the following:
-
Paste the assigned client ID.
-
Paste the assigned client secret.
-
Click Save.
-
Creating a REST API client connection
In an integration, to connect to a REST API, create a connection to that REST API, which you can then add to any number of integrations.
-
You created a connector for the REST API that you want to connect to.
-
If the REST API uses the OAuth protocol then you registered you Syndesis environment as a client application that can access that REST API.
-
In Syndesis, in the left panel, click Connections to display any available connections.
-
Click Create Connection to display available connectors.
-
Click the connector for the API that you want to create a connection for.
-
Respond to prompts for additional information. The definition of the API determines what Syndesis prompts for.
Typically, an API service has security requirements and Syndesis prompts for security input according to the particular security type that the OpenAPI document defines. The security type can be one of the following:
Security type
Syndesis displays
HTTP Basic Authentication
Input fields for user name and password.
OAuth 2.0
A button for you to click so that Syndesis can verify its registration credentials for connecting to the API. These credentials were provided when the connector was created.
API Key
Input field for an API key, which you must obtain from the API service.
-
In the Name field, enter your choice of a name that helps you distinguish this connection from any other connections.
-
In the Description field, optionally enter any information that is helpful to know about this connection.
-
Click Save to see that you can now choose the connection you created and add to an integration.
Adding an API client connection to an integration
In an integration, to connect to a REST API, add a step that connects to that REST API.
In this release, a connection to a REST API can be a step that is in the middle of a flow or it can be the finish connection in a simple integration. In other words, a connection to a REST API cannot be a start connection.
-
You created a connection to the REST API.
-
You are creating or editing a flow.
-
The flow already has its start connection.
-
Syndesis is prompting you to select a finish connection or to add to the integration.
-
On the Add to Integration page, click the plus sign where you want to add the connection. Skip this step if Syndesis is prompting you to choose the finish connection.
-
On the page that displays available steps, click the REST API connection that you want to add to the flow.
-
Select the action that you want the connection to perform. The actions that are available are based on the resource operations specified in the OpenAPI document that was uploaded to Syndesis and that describes the API that you are connecting to.
-
Depending on the action you select, enter any parameters that Syndesis prompts for.
-
Click Next.
The connection appears in the integration visualization where you added it.
Connecting to Salesforce
In an integration, to connect to Salesforce, you must register your Syndesis environment as a Salesforce client application. This authorizes Syndesis to access Salesforce. With registration in place, you create a Salesforce connection, which you can then add to any number of integrations. For details, see the following topics:
Registering Syndesis as a Salesforce client application
In an integration, to connect to Salesforce, the first thing you must do is register your Syndesis environment as a client application that can access Salesforce. This lets you create any number of integrations that connect to Salesforce. In other words, you need to register a particular Syndesis environment with Salesforce only once.
In each Syndesis environment, there can be only one registration of Syndesis as a Salesforce client application. However, while each Salesforce connection uses the same registration, it can use different user credentials.
You are logged in to Syndesis.
-
In Syndesis:
-
In the left panel, click Settings.
-
On the Settings page, near the top, to the right of the callback URL, click to copy the callback URL for your Syndesis environment to the clipboard. You will need this URL toward the end of this procedure.
-
Click the Salesforce entry to display the Client ID and Client Secret fields.
-
-
In another browser tab, log in to your Salesforce account and follow the steps below to create a connected app. These instructions assume that you are using the Salesforce Classic user interface. To switch from the Salesforce Lightning Experience interface, click your profile icon and select Switch to Salesforce Classic. For additional information, see the Salesforce documentation for Create a Connected App.
-
In Salesforce, in the upper right, click Setup.
-
In the left panel, select Build > Create > Apps.
-
Scroll down to Connected Apps and click New.
-
Enter the required information and then select Enable OAuth Settings.
-
In the Callback URL field, paste your Syndesis URL, which you copied at the beginning of this procedure. For example:
https://app-proj912876.7b63.fuse-ignite.openshiftapps.com/api/v1/credentials/callback
. -
For OAuth Scopes, add:
-
Access and manage your data (api)
-
Allow access to your unique identifier (openid)
-
Perform requests on your behalf at any time (refresh_token, offline_accesss)
-
-
Select Configure ID token and then Include Standard Claims.
-
Scroll down and click Save.
-
Scroll up to see that Salesforce indicates a short wait:
-
Click Continue.
-
Copy the consumer key that Salesforce provides.
-
-
Return to your Syndesis Settings page and paste the Salesforce-provided consumer key into the Syndesis Salesforce Client ID field.
-
Return to Salesforce and copy the consumer secret that Salesforce provides.
-
Return to your Syndesis Settings page and paste the Salesforce-provided consumer secret into the Syndesis Salesforce Client Secret field.
-
Click Save.
-
Click the Salesforce entry to collapse it.
Creating a Salesforce connection
To create an integration that accesses data in Salesforce, you must first create a Salesforce connection. After you create a Salesforce connection, you can use it in multiple integrations.
-
You are logged in to Syndesis.
-
Syndesis is open in a web browser.
-
You must have already registered your Syndesis environment as an application that can access Salesforce.
-
You added the Salesforce client ID and client secret that you received after registration to the Syndesis Settings page.
Be sure to wait 2 - 10 minutes after registering your Syndesis installation as a Salesforce client before you try to create a Salesforce connection.
-
In Syndesis, in the left panel, click Connections to display available connections.
-
Click Create Connection to display the available connectors. A connector is a template for creating one or more connections.
-
Click the Salesforce connector.
-
Click Connect Salesforce to display a Salesforce authorization page. You might need to log in to Salesforce before you see the authorization page.
If Connect Salesforce does not appear, then your Syndesis environment is not registered as a Salesforce client application. See Registering Syndesis as a Salesforce client application. When you try to create a Salesforce connection and your Syndesis environment is not registered as a Salesforce client application, then Syndesis displays multiple fields that prompt for authorization information. While you can create a Salesforce connection by entering values in these fields, it is not recommended.
The following error indicates that Salesforce does not have the correct Syndesis callback URL:
error=redirect_uri_mismatch&error_description=redirect_uri%20must%20match%20configuration
If you get this error message, then in Salesforce, ensure that the Syndesis callback URL is specified according to the instructions in Registering Syndesis as a Salesforce client application.
-
Click Allow to return to Syndesis.
-
In the Name field, enter your choice of a name that helps you distinguish this connection from any other connections. For example, enter
SF Connect 1
. -
In the Description field, optionally enter any information that is helpful to know about this connection. For example, enter
Sample Salesforce connection that uses my Salesforce login credentials.
-
Click Save to see that the connection you created is now available. If you entered the example name, you would see that SF Connect 1 appears as a connection that you can choose to add to an integration.
Adding a Salesforce connection to an integration
In an integration, you can connect to Salesforce in the middle of a flow or to start or finish a simple integration. To do this, add a Salesforce connection to the integration.
-
You created a Salesforce connection.
-
You are creating or updating an integration.
-
Syndesis is prompting you to choose a start connection, or to choose a finish connection, or to add to the integration.
-
If you are adding to the Salesforce connection to the middle of a flow, click the plust sign where you want to add the connection. Skip this step if Syndesis is prompting for a start or finish connection.
-
Click the Salesforce connection that you want to add to the flow. When the integration uses the connection that you select to connect to Salesforce, Syndesis uses the credentials defined in that connection.
-
Select the action that you want the selected connection to perform. Each Salesforce connection that you add to a flow performs only the action you choose.
-
Specify the Salesforce object that the action operates on, for example, it might be a contact, lead or price book entry. Click in the Object field to select from a list of Salesforce objects or enter the name of the object.
-
Click Next to add the connection to the flow.
Connecting to SAP Concur
SAP Concur manages business expenses, travel, and invoices. In an integration, you can connect to SAP Concur and perform any one of a large variety of SAP Concur actions. For example, suppose that you store project codes in a SQL database and you need the same project codes in SAP Concur so that expenses can be associated with projects. You can create an integration that connects to your SQL database and runs a stored procedure that obtains any new project codes. The integration can then connect to SAP Concur and upload the new project codes to the appropriate list.
See the following topics:
User roles for connecting to SAP Concur
SAP Concur has an implementation site for development work and a production site for live use. The implementation site simulates the production site. This lets you create and refine an integration before you use it in production with the live SAP Concur site.
At any one time, a Syndesis environment can connect to the SAP Concur implementation site or the SAP Concur production site. In other words, a particular Syndesis environment does not support connections to both SAP Concur sites at the same time. Each Syndesis environment requires its own credentials for accessing each SAP Concur site and the credentials for the implementation site are different from the credentials for the production site.
Consequently, the expectation is that someone in a system administrator-like role would do the following:
-
Obtain two sets of OAuth credentials from SAP Concur:
-
One set for access to the SAP Concur implementation site
-
Another set for access to the SAP Concur production site
-
-
Configure the SAP Concur connector for access to the appropriate SAP Concur site.
-
Create a SAP Concur connection.
A business user can then create an integration that uses the SAP Concur connection.
How to Obtain SAP Concur implementation site OAuth credentials
To obtain credentials for accessing the SAP Concur implementation site, contact SAP Concur directly and tell them:
-
You want to register Syndesis as a new app that is authorized to access the SAP Concur implementation site.
-
You want a new set of OAuth credentials for their implementation site.
-
The credentials must include the
LIST
scope. -
Optionally, to obtain permission to perform actions other than the list actions, ask for other scopes.
-
The OAuth grant type for these credentials should be
Authorization Grant Flow
.To use a REST client, such as Postman, to send a request to the SAP Concur implementation site, you need credentials with an OAuth grant type of
Password
. -
The Syndesis callback URL for the environment that that you want credentials for, which is something like this:
https://syndesis.my-minishift.syndesis.io/api/v1/credentials/callback
In this URL,
syndesis.my-minishift.syndesis.io
matches the OpenShift route to your Syndesis environment.
The response from SAP Concur should provide:
-
A username and password for logging in to the SAP Concur implementation site.
-
Client ID and client secret values that you will specify in the Syndesis development environment Settings page to configure the Syndesis SAP Concur connector.
How to obtain SAP Concur production site OAuth credentials
To obtain OAuth credentials that authorize Syndesis to access the SAP Concur production site, go to the SAP Concur Developer Center Getting Started page and follow the instructions there.
Configuring the SAP Concur connector
To connect to SAP Concur in an integration, you must configure the Syndesis SAP Concur connector. You can then use the connector to create a connection to SAP Concur.
-
You know whether you want to configure the SAP Concur connector to access the SAP Concur implementation site or the SAP Concur production site.
-
You obtained OAuth credentials that authorize your Syndesis environment to access the appropriate SAP Concur site.
-
In Syndesis, in the left panel, click Settings.
-
On the Settings page, expand the SAP Concur entry.
-
In the Client ID field, enter the client ID that you received from SAP Concur.
-
In the Client Secret field, enter the client secret that you received from SAP Concur.
Syndesis populates the other fields.
-
Click Save.
Syndesis immediately tries to validate the configuration. If validation is not successful, correct the input values and try again. If validation is successful, you can create a SAP Concur connection, which you can add to an integration.
-
Click the SAP Concur entry to collapse it.
Create a SAP Concur connection.
Creating a SAP Concur connection
To connect to SAP Concur in an integration, you must first create a SAP Concur connection, which you can then add to any number of integrations. When you create a SAP Concur connection, you authorize the connection to access SAP Concur with the login credentials that you enter when you create the connection.
-
In the Syndesis environment in which you are creating the connection, you must have configured the SAP Concur connector.
-
You should know whether the connector is configured for access to the SAP Concur implementation site or the SAP Concur production site.
-
You must have a user name and password for logging in to the SAP Concur site for which the SAP Concur connector is configured.
-
In Syndesis, in the left panel, click Connections to display any available connections.
-
Click Create Connection to display the available connectors. A connector is a template that you use to create one or more connections.
-
Click the SAP Concur connector.
-
In the Configure Connection page, click Connect SAP Concur, which takes you to the SAP Concur site for which the connector is configured.
If Connect SAP Concur does not appear, then the SAP Concur connector in your Syndesis environment is not configured. See Configuring the SAP Concur connector.
If
redirect_uri is improper or not previously registered
appears, then the SAP Concur connector configuration is incorrect. Return to the Syndesis Settings page to update and validate the connector configuration. -
On the SAP Concur OAuth site:
-
Enter the email address for the SAP Concur account that you want this connection to use to access SAP Concur.
-
Ensure that Username is selected.
-
Click Continue.
-
Enter the SAP Concur password associated with the email address.
-
Click Sign In, which returns you to Syndesis.
-
-
In the Syndesis Name field, enter your choice of a name that helps you distinguish this connection from any other connections. For example, enter
SAP Concur Test1
. -
In the Description field, optionally enter any information that is helpful to know about this connection. For example, enter
Sample SAP Concur connection to the implementation site.
-
Click Save to see that the connection you created is now available. If you entered the example name, you would see that SAP Concur Test1 appears as a connection that you can choose to add to an integration.
Adding a SAP Concur connection to an integration
You can add a SAP Concur connection to the middle of a flow or as the finish connection in a simple integration. A connection to SAP Concur can perform any one of 85 actions.
-
You created a SAP Concur connection.
-
You are creating or editing a flow.
-
Syndesis is prompting you to add to the integration, or to choose the finish connection in a simple integration.
-
On the Add to Integration page, click the plus sign where you want to add the connection. Skip this step if Syndesis is prompting you to choose the finish connection.
-
Click the SAP Concur connection that you want to add to the flow. When the integration uses the connection that you select, Syndesis uses the credentials defined in that connection to connect to SAP Concur.
-
Select the action that you want the connection to perform. Each SAP Concur connection that you add to a flow performs only the action you choose.
For details about SAP Concur actions, visit the SAP Concur developer center and expand v3.0.
-
Click Next to add the connection to the flow.
The connection appears in the integration visualization where you added it.
Example of mapping integration data fields to SAP Concur fields
In most integrations, you need to add a data mapping step before a connection to SAP Concur. In the data mapping step, you map data fields that are available from previous integration steps to SAP Concur action parameters. In the data mapper, the Target panel displays the SAP Concur parameters for the action that you selected when you added the SAP Concur connection to the integration.
Suppose that an integration starts by executing a SQL stored procedure that obtains new project codes. The integration finishes by adding new project codes to SAP Concur. The following procedure provides an example of a data mapper step before the SAP Concur connection. The integration visualization displays and Syndesis is prompting you to add to the integration.
-
In the integration visualization, click the plus sign that is before the SAP Concur finish connection.
-
Click Data Mapper. When the data fields appear, the Sources panel on the left displays the fields that are available from the database connection. In this example, the source fields include
concur-list-id
andproject-code
. The Target panel on the right displays the SAP Concurid
andcontent
parameters for the update list action, which the connection performs. -
Map the
concur-list-id
source field to theid
target field. -
Map the
project-code
source field to thecontent
target field. -
In the upper right, click Done to add the data mapper step to the integration.
Connecting to ServiceNow
An integration can retrieve records from a ServiceNow table or add records to a ServiceNow import set, which ServiceNow uses to update a table. In an integration, to connect to ServiceNow, create a ServiceNow connection and then add that connection to an integration. For details, see:
Creating a ServiceNow connection
In an integration, to connect to your company’s ServiceNow instance, you must create a ServiceNow connection.
-
A ServiceNow administrator at your site must have created a ServiceNow account for you.
-
You must know the URL for your ServiceNow instance and your ServiceNow user name and password.
-
If the ServiceNow administrator also created a ServiceNow client ID and client secret for you, you must know these values.
-
In Syndesis, in the left panel, click Connections to display any available connections.
-
Click Create Connection to display Syndesis connectors.
-
Click the ServiceNow connector.
-
Configure this ServiceNow connection:
-
In the Instance Name field, enter the name of the ServiceNow instance that you want to obtain records from or copy records to. For example, if the URL for your ServiceNow instance is
https://abc12345.service-now.com
, the instance name isabc12345
. -
In the User Name field, enter your ServiceNow user name.
-
In the User Password field, enter your ServiceNow password.
-
Optionally, in the Client ID field, enter your client ID if you received one from your ServiceNow administrator.
-
Optionally, in the Client Secret field, enter your client secret if you received one from your ServiceNow administrator.
-
Click Validate. Syndesis immediately tries to validate the connection and displays a message that indicates whether validation is successful. If validation fails, update the configuration details as needed and try again.
-
When validation is successful, click Next.
-
-
In the Name field, enter a name that helps you distinguish this connection from other connections. For example, enter
ServiceNow Con 1
. -
In the Description field, optionally enter any information that is helpful to know about this connection. For example, enter
ServiceNow connection that uses administrative credentials.
-
Click Save to see that the connection you created is now available. If you entered the example name, you would see that ServiceNow Con 1 appears as a connection that you can choose to add to an integration.
Obtaining records from ServiceNow to trigger integration execution
To trigger execution of an integration upon receiving records from ServiceNow, add a ServiceNow connection to a simple integration as its start connection.
-
You created a ServiceNow connection.
-
You know the name of the table that you want to retrieve records from.
-
You should have already defined a ServiceNow query that obtains only the records that you want. You can find information about ServiceNow queries here: ServiceNow encoded queries.
-
In the Syndesis panel on the left, click Integrations.
-
Click Create Integration.
-
On the Choose a connection page, click the ServiceNow connection that you want to use to start the integration.
-
On the Choose an action page, select Retrieve Record to obtain records from a ServiceNow table that you select.
-
Configure the Retrieve Record action:
-
Click in the Table name field and select the table that you want to obtain records from.
-
In the Query to filter the result set field, enter a ServiceNow query. For example, consider the
state=1^impact=2
query on a table that keeps records about incidents. This query returns records for new incidents (state=1
) that have a medium impact (impact=2
). If you do not enter a query and the records in the table do not change, the connection obtains the same records every time. -
In the Limit of elements per page field, enter the maximum number of records that you want the connection to obtain. In this release, you must enter a value, the recommendation is to enter
1000
or less, and pagination is not supported. -
In the Period field, indicate how often you want to obtain records. The default is every minute. In other words, by default, Syndesis executes this integration every 60 seconds.
-
Click Next.
-
Syndesis generates a JSON schema that defines the structure of the obtained records. This enables you to add a data mapping step before the next connection in the integration if data mapping is needed.
Syndesis prompts you to choose a finish connection.
Creating an import set in ServiceNow
In a Syndesis integration, a ServiceNow connection cannot directly update a ServiceNow table. To update ServiceNow data in an integration, a prerequisite is an import set that stages updates for a ServiceNow table.
Often, the easiest way to create an import set is for a ServiceNow administrator to create a ServiceNow inbound web service. This implicitly creates a ServiceNow import set. The import set is based on the ServiceNow target table that the administrator selects when creating the web service. A ServiceNow connection updates this import set, and ServiceNow uses the import set to update the corresponding ServiceNow table.
You must have ServiceNow administrative privileges.
-
In ServiceNow, search for
web services
. -
In the results, under System Web Services > Inbound, click Create New.
-
In the Create Web Service page:
-
In the Label field, specify the display name for the web service. The web service’s import set will also have this name.
-
In the Target table field, select the ServiceNow table that this service updates.
-
Select Copy fields from target table. This creates an import set that has the same schema as the target table. The import set is a staging area for updating the target table.
-
Select Create transform map. This map enables ServiceNow to copy and transform data from the import set to the target table.
-
-
Click Create.
-
In the next display, under Related Links, click Auto Map Matching Fields to display a list of fields in the target table that you selected.
-
Select one target table field to be the correlation (or coalesce) field.
When the value of the correlation field in a record in the import set matches the value of a correlation field in the target table, ServiceNow updates that record in the target table rather than creating a new record.
-
Select Coalesce to identify the field that you just selected as the correlation field. Without a correlation field, ServiceNow adds each record in the import set to the target table.
-
Click Update.
-
Confirm that the transform map was created:
-
In ServiceNow, search for
transform map
. -
Under System Import Sets, click Transform Maps to display a list of transform maps.
-
In this list, confirm that there is a transform map whose name is the value that you specified for the label of the web service that you created.
-
Copying records to ServiceNow during or to finish an integration
You can copy records to ServiceNow in the middle of a flow or to finish a simple integration. To do this, add a ServiceNow connection to the middle of a flow or as a simple integration’s finish connection.
-
You created a ServiceNow connection.
-
You are creating or editing a flow and Syndesis is prompting you to add to the integration. Or, Syndesis is prompting you to choose a finish connection.
-
You know the name of the ServiceNow import set that you want to add records to. Your ServiceNow administrator can help you identify the appropriate import set. If the import set does not exist, see Creating an import set in ServiceNow.
-
The ServiceNow import set is configured to handle the addition of records.
-
On the Add to Integration page, click the plus sign where you want to add the connection. Skip this step if Syndesis is prompting you to choose the finish connection.
-
Click the ServiceNow connection that you want to add to the integration.
-
Select the Add Record action.
-
Click in the Import Set field and select the import set that you want to add records to.
-
Click Next.
Syndesis adds the connection to the flow.
Typically, a data mapper step is needed before this connection. In the integration visualization, Syndesis displays on the connection. In most integrations, you need to map a source field to the target field used as the correlation field in the ServiceNow table.
Example Salesforce to ServiceNow integrations
This example describes two simple integrations:
-
One integration obtains new cases from Salesforce and adds them as incidents to ServiceNow.
-
The other integration obtains updated cases from Salesforce and updates a corresponding incident in ServiceNow.
For both integrations, a prerequisite is for a ServiceNow administrator to create an inbound web service. This implicitly creates a ServiceNow import set, which is required for an integration to add data to ServiceNow. For this example:
-
The name of the web service is Cases from Salesforce.
-
The service’s target table is Incidents. This table contains the data to be updated.
-
The import set has the same schema as the Incidents table because of the selection of Copy fields from target table. The name of the import set is also Cases from Salesforce.
-
ServiceNow can copy and transform data from the Cases from Salesforce import set to the Incidents target table because of the selection of Create transform map.
-
In the import set, the correlation field is
correlation_Id
because of the selection of thecorrelation_Id
field followed by selection of Coalesce.
In Syndesis, you create the Salesforce and ServiceNow connections.
The two integrations are the same except for the action performed by the Salesforce connection, as noted in the following steps. To create the integrations, you would perform these steps twice, that is, once to create each integration:
-
Add the Salesforce connection as the start connection.
-
To create the integration that obtains new cases, select the On create action.
-
To create the integration that obtains updated cases, select the On update action.
-
-
Select the Case object for the action to operate on. This Salesforce connection returns new Case objects or updated Case objects according to the chosen action.
-
Add the ServiceNow connection as the finish connection.
-
Select the Create Record action, and then select Cases from Salesforce as the import set that the integration updates.
This ServiceNow connection adds new or updated Salesforce cases to the Cases from Salesforce import set.
-
Add a data mapper step that maps:
-
Salesforce case
id
to the ServiceNow import set’scorrelation_Id
-
Salesforce
subject
to the ServiceNowshort_description
-
Salesforce
description
to the ServiceNowdescription
-
-
Give the integration a name and then publish it.
-
For the integration that obtains new cases, use On SF Create Case.
-
For the integration that obtains updated cases, On SF Update Case.
-
When both integrations are running, you can confirm that the integrations work:
-
In Salesforce, create a case.
-
In Syndesis, view the summary for the On SF Create Case integration. Click its Activity tab to see that Syndesis executed the integration once.
-
In ServiceNow, view the Incidents table. You should see a new incident that has the subject and description that you specified in Salesforce.
-
Back in Salesforce, update the case that you just created by changing its subject.
-
In Syndesis, view the summary for the On SF Update Case integration. Click its Activity tab to see that Syndesis executed the integration once.
-
In ServiceNow, view the Incidents table and expand the entry for the incident that was previously new. You should see that this incident has an updated
short_description
value. ServiceNow checks the import set entry for the value of itscorrelation_Id
. If this value already exists in the Incidents table, ServiceNow updates the incident that has that value.
Connecting to Slack
As a business user, you can create an integration that connects to Slack. A connection to Slack can do either one of the following:
-
Trigger execution of a simple integration when a Slack channel that you specify receives a message. The integration passes the message to the next step in the flow. For example, you can monitor a Slack channel for keyword instances such as product names. Upon finding messages that contain those product names, the integration can notify the appropriate contact in a Gmail connection.
-
Deliver a message to a particular user or to a channel. For example, this behavior is useful when an integration downloads a file from an FTP server and processes it in some way. An integration flow can notify a Slack channel or user that the process was successful.
To connect to Slack in an integration, create a Slack connection and then add the connection to an integration flow. Details are in the following topics:
Creating a Slack connection
In an integration, a Slack connection can retrieve messages from a channel that you specify or send a message to a channel or user. You can use the same Slack connection in any number of integrations.
-
You created a Slack app for connecting Syndesis to Slack.
-
You can obtain the Slack webhook URL for your Slack app.
-
You can obtain a legacy token that authorizes access to your Slack app.
-
In Syndesis, in the left panel, click Connections to display any available connections.
-
Click Create Connection to display Syndesis connectors.
-
Click the Slack connector.
-
In the Slack webhook URL field, enter the webhook URL for your Slack app.
-
In the Token for accessing Slack API field, enter the Slack-provided token, which you can obtain from Slack.
-
Optionally, enter values for additional parameters:
-
In the Sending username for messages field, enter the user name that the bot has when it sends messages to Slack.
-
In the Message avatar emoji field, specify one emoji, which the bot uses as the message avatar when it sends a message. Examples:
:smile:
,:wave:
,:question:
.Search the web for emoji cheat sheet to see some emoji codes. -
In the Message avatar icon URL field, specify the URL of the avatar that the bot uses when it sends messages to Slack.
If you specify an emoji and an icon URL, then the integration uses the icon URL. If you specify neither an emoji nor an icon URL, then the message is sent without an avatar.
-
-
Click Validate. Syndesis immediately tries to validate the connection and displays a message that indicates whether validation is successful. If validation fails, revise the connection configuration values and try again.
-
If validation is successful, click Next.
-
In the Name field, enter your choice of a name that helps you distinguish this connection from any other connections. For example, enter
Slack for Tracking Company Sales
. -
In the Description field, optionally enter any information that is helpful to know about this connection.
-
Click Save to see the updated list of available connections, including the connection that you just created. If you entered the example name, you would see that Slack for Tracking Company Sales appears as a connection that you can choose to add to an integration. You can add this connection to any number of integrations.
Adding a Slack connection to trigger integration execution upon receiving messages
A Slack connection that starts an integration triggers execution of the integration when the connection finds messages on a Slack channel that you specify.
-
You created a Slack connection.
-
You are creating an integration and Syndesis is prompting you to choose how you want to start the integration.
-
Click the Slack connection that you want to use to start the integration.
-
Select the Read Messages action.
-
In the Channel field, specify the Slack channel from which you want to obtain messages.
-
In the Delay field, accept the default of 500 milliseconds as the time that elapses between polls for messages. Or, to specify a different interval for checking for messages, enter a number and select its time unit.
-
In the Maximum Messages to Retrieve field, accept the default of 10 or indicate the maximum number of messages that the connection can return for each poll. If the connection finds more than this number of unread messages, then it returns the most recent messages. The connection returns a particular message only once.
For example, if you accept the default of 10 and there are 15 messages at the time of a poll then the connection returns the 10 most recent messages. At the next poll, the connection checks for messages that have a time stamp that is after the most recent message that the connection previously returned. This means that the 5 messages that were not returned in the first example poll are never returned.
When a Slack connection returns more than one message, the integration processes the messages as a batch. In other words, Syndesis executes the integration once for the batch.
-
Click Next to add the connection to the integration.
Adding a Slack connection to send a message to a Slack channel or user
In an integration, you can send a message to a Slack channel or Slack user in the middle of a flow or to finish a simple integration. To do this, add a Slack connection to the middle of a flow or as a simple integration’s finish connection.
-
You created a Slack connection.
-
You are creating or editing a flow and Syndesis is prompting you to add to the integration. Or, Syndesis is prompting you to choose a finish connection.
-
On the Add to Integration page, click the plus sign where you want to add the connection. Skip this step if Syndesis is prompting you to choose the finish connection.
-
Click the Slack connection that you want to add to the flow.
-
Select the action that you want the connection to perform.
-
Select Username to send a message to one user. To configure this action, in the User name field, specify the Slack name of the user to send the message to.
-
Select Channel to publish a message on a channel. To configure this action, in the Channel field, specify the channel to publish the message to.
-
-
Click Next to add the connection to the flow.
-
Optionally, add additional connections to the flow. Whether additional connections are needed depends on what you want the flow to do. The important point is to add all connections before you continue.
-
Add a data mapper step just before the Slack connection that you added in this procedure. In the mapping step, map a string from a data mapping source to the Slack message field. This string should contain the message that you want to send to the Slack user or channel. See Adding a data mapper step.
Connecting to SQL databases
In an integration, you can connect to any of the following types of SQL databases:
-
Apache Derby
-
MySQL
-
PostgreSQL
Create a connection to the database that you want to access in an integration. Then create an integration and add that database connection to the integration.
To connect to other types of databases, you must upload a JDBC driver for that database.
See the following topics for details:
Creating a database connection
You create a separate connection for each database that you want to connect to in an integration. You can use the same connection in multiple integrations.
A database connection operates on a database table that you specify or invokes a stored procedure that you specify.
The database table or the stored procedure must exist when an integration connects to the database.
-
Ensure that the JDBC driver for the database that you want to connect to is on your classpath. If you uploaded a JDBC driver library extension to connect to a proprietary database, then the upload process puts the driver on your classpath. See Creating JDBC driver library extensions.
-
In Syndesis, in the left panel, click Connections to display any available connections.
-
Click Create Connection to display Syndesis connectors.
-
Click the Database connector.
-
Configure the connection by entering:
-
In the Connection URL field, enter the JDBC URL for the database that you want to connect to. For example, enter
jdbc:postgresql://ignite-db1234/sampledb
. -
In the Username field, enter the name of the account that you want to use to access the database. Ensure that the specified password and user name are for the same account.
-
In the Password field, enter the password associated with the user account you want to use to access the database.
-
In the Schema field, enter the name of the schema for the database. How you specify the database schema varies for each type of database. Details are in the next section: How to specify the schema in a database connection.
-
-
Click Validate. Syndesis tries to validate the connection and displays a message that indicates whether validation is successful. If validation fails, revise the configuration details as needed and try again.
-
If validation is successful, click Next.
-
In the Name field, enter your choice of a name that helps you distinguish this connection from any other connections. For example, enter
PostgreSQL DB 1
. -
In the Description field, optionally enter any information that is helpful to know about this connection. For example, enter
Sample PostgreSQL connection that uses my login credentials.
-
Click Save to see that the connection you created is now available. If you entered the example name, you would see that PostgreSQL DB 1 appears as a connection that you can choose to add to an integration.
How to specify the schema in a database connection
In Syndesis, when you create a database connection, the user interface prompts you to specify the database’s schema. How you specify the schema varies for each database. The table below shows what each database requires.
| Database | Example connection configuration | Notes |
|---|---|---|
Apache Derby |
Connection URL (JDBC URL): |
For testing purposes only. The default username is |
MS SQLServer |
Connection URL (JDBC URL): |
Upload the driver by using the extension mechanism. Then create a connection. Connection verification fails if you have not already uploaded the driver. You must specify the same schema at the end of the connection URL and in the Schema field. |
MySQL |
Connection URL (JDBC URL): |
MySQL does not support schemas. When you create a connection to a MySQL database, leave the Schema field blank. MySQL expects all configuration in the connection (JDBC) URL. |
Oracle DB |
Connection URL (JDBC URL): |
Use the Syndesis
extension mechanism.
to upload an Oracle database driver.
Then create the connection. Connection verification fails if the driver has
not been uploaded. |
PostgreSQL |
Connection URL (JDBC URL): |
You must specify the database in the connection (JDBC) URL. T If you want to use a schema you must specify it in the Schema field. A query can refer to a table in the form database.schema.table. When a reference specifies only a table name, the connection first searches the schema that you specify when you configure the connection. If the table is not found, then the connection searches public resources for the specified table. For more details, see PostgreSQL schema documentation. |
Teiid |
Connection URL (JDBC URL): |
The virtual database must have already been created, for example by using
the
Data Virtualization
feature. In the connection URL, replace |
Obtaining database records to trigger integration execution
To trigger execution of an integration based on the result of invoking a SQL statement or a SQL stored procedure, choose a database connection as the integration’s start connection.
You created a database connection.
-
In the Syndesis panel on the left, click Integrations.
-
Click Create Integration.
-
On the Choose a connection page, click the database connection that you want to use to start an integration.
-
On the Choose an action page, select one of the following:
-
Periodic SQL invocation obtains data by periodically invoking the SQL statement that you specify.
-
Periodic stored procedure invocation obtains data by periodically invoking the stored procedure that you specify or select.
-
-
If you selected Periodic SQL invocation, in the Query field, enter a SQL
SELECT
statement or some other standard SQL statement that obtains one or more records. For example:SELECT * from my_db_table
. The database table that contains the data you want must already exist.If you selected Periodic stored procedure invocation, in the Procedure name field, select or enter the stored procedure to invoke to obtain the data of interest. The stored procedure you specify must already exist. The database administrator should have created any stored procedures you need to use in an integration.
-
In the Period field, enter an integer and indicate whether the unit is milliseconds, seconds, minutes, hours, or days. For example, if you specify
5 minutes
then the connection invokes the specified statement or stored procedure every five minutes. -
Click Next.
SQL statements that you enter in the Query field must be standard SQL constructs. Syndesis does not recognize or parse SQL extensions such as Procedural Language/PostgreSQL (PL/pgSQL) or Oracle Procedural Language Extensions to SQL (PL/SQL). However, you can use SQL extensions within stored procedures.
Syndesis tries to validate the connection, which includes checking that a specified SQL statement is syntactically correct and confirming that the statement or stored procedure target data exists. If verification is successful then Syndesis adds the start connection to the integration. If verification fails then Syndesis displays a message about the problem. Update your input as needed and try again.
About parameter placeholders and values in SQL statements that update data
A database connection that is in the middle of a flow, or that finishes a simple integration, can execute a SQL statement that updates data. When you add a database connection to an integration, you can specify placeholder parameters in the SQL statement to be executed. During execution, the connection can perform a batch update when the input parameter values are in a collection.
When you create or edit an integration and you add a connection that updates a database, you can specify placeholder parameters in the SQL statement that the connection executes at runtime or there can be placeholders in the stored procedure that will be executed at runtime. For example:
INSERT INTO TODO(task, completed) VALUES(:#param_1, :#param_2)
DELETE FROM TODO WHERE task LIKE :#param_3
To specify the values of these placeholders, add a data mapping step
to the flow before the database connection. In the data mapping
step, map the appropriate source data fields to the target data
fields, for example, map source data to the :#param_1
, :#param_2
, and
:#param_3
target fields. See
Adding a data mapper step.
At runtime, when a database connection executes a SQL statement that
inserts, deletes, or updates data, the SQL statement being executed often
specifies one or more
placeholder parameters, for example, :#task
. When it does,
the input to the connection can be a single set of parameters
or a collection of parameter sets, where each element in a set
defines a value for the corresponding placeholder parameter.
The input to the database connection comes from an earlier connection in the flow, for example, an HTTP request (Webhook) or a request sent to the REST API service for an API provider integration. When the input is a collection, the connection uses batch mode to update the table. For example, consider this SQL statement:
insert into todo (task) values (:#task)
If the input collection contains 3 values, the connection inserts three records, one for each value. Now consider a SQL statement with two placeholder parameters:
insert into todo (task, completed) values (:#task, :#completed)
In the input to the connection that executes this SQL statement, a data shape specification could specify a single element, for example:
{"task": "write some docs", "completed": 0}
Or it could specify a collection, for example:
[{"task": "write doc", "completed": 1}, {"task": "publish doc", "completed": 0}]
With a collection as input, the connection executes the insert operation once for each set of parameter values. In this example, the connection adds two records to the table, one for each set of parameter values.
Accessing a database in the middle or to complete an integration
In an integration, you can access a database in the middle of a flow or to finish a simple integration. To do this, add a database connection to the middle of a flow or as a simple integration’s finish connection.
-
You created a database connection.
-
You are creating or editing a flow and Syndesis is prompting you to add to the integration. Or, Syndesis is prompting you to choose a finish connection.
-
On the Add to Integration page, click the plus sign where you want to add the connection. Skip this step if Syndesis is prompting you to choose the finish connection.
-
Click the database connection for the database that you want to access.
-
On the Choose an action page, select one of the following:
-
Invoke SQL operates on data by executing the SQL statement you specify.
-
Invoke stored procedure operates on data by invoking the stored procedure that you specify or select.
-
-
If you selected Invoke SQL, in the SQL Statement field:
-
For a middle connection, enter a SQL
SELECT
statement that obtains one or more records or enter a SQLINSERT
,UPDATE
, orDELETE
statement that operates on one or more records. The database table that contains the data must already exist. -
For a finish connection, enter a SQL
INSERT
,UPDATE
orDELETE
statement to operate on one or more records. -
In the Batch update field, accept No, which is the default, or select Yes.
The setting of Batch update affects connection behavior when the input to this action is a collection, and the SQL statement inserts, deletes, or updates records. The default behavior (Batch update is No) is that the connection accepts only individual objects and executes the SQL statement once for each object. When Batch update is Yes, the connection accepts a collection as the input to the action. The connection executes the SQL statement once and uses all collection items as input to a batch update operation.
If you selected Invoke stored procedure, in the Procedure name field, select or enter the name of the stored procedure to invoke to operate on the data of interest. The stored procedure you specify must already exist. The database administrator should have created any stored procedures you need to use in an integration.
-
-
Click Next.
Syndesis tries to validate the connection, which includes checking that a specified SQL query is syntactically correct and confirming that the query or stored procedure target data exists. If verification is successful then Syndesis adds the connection to the flow. If verification fails then Syndesis displays a message about the problem. Update your input as needed and try again.
About parameter placeholders and values in SQL statements that update data.
Connecting to proprietary databases
To connect to a proprietary SQL database, the main tasks that must be accomplished are as follows:
-
A developer creates a library extension that contains the JDBC driver for the database that you want to access in an integration. See Creating JDBC driver library extensions.
-
The developer provides a
.jar
file that contains the library extension. -
You upload that
.jar
file to Syndesis. See Making extensions available. -
You create a connection to your database by selecting the Syndesis Database connector and specifying the connection URL for your database. See Creating a database connection.
-
In an integration, you add the connection to your database. See Starting an integration by accessing a database or Accessing a database in the middle or to complete an integration.
Connecting to Telegram
As a business user, you can create an integration that connects to Telegram. A connection to Telegram can do either one of the following:
-
Trigger execution of a simple integration when a Telegram bot that you specify receives a message. The integration passes the message to the next step in the flow. For example, after a simple integration receives a message there might be a filter step that watches for keyword instances such as product names. Upon finding messages that contain those product names, the integration can notify the appropriate contact in a Gmail connection or in a different Telegram connection.
-
Deliver a message to a particular Telegram chat. For example, this behavior is useful when an integration downloads a file from an FTP server and processes it in some way. You can add a Telegram connection that notifies a Telegram chat that the process was successful.
To connect to Telegram in an integration, create a Telegram connection and then add the connection to an integration flow. Details are in the following topics:
Creating a Telegram connection
In an integration, a Telegram connection can receive messages that have been sent to a particular chat. A chat bot that you create forwards the messages to Syndesis. A Telegram connection can also send messages to chats that the connection has access to. You can use the same Telegram connection:
-
In any number of integrations that use the connection to send messages to a chat.
-
In addition, you can use it in one integration that uses the connection to receive messages from a Telegram chat.
In other words, you cannot use the same Telegram connection in more than one integration if you want that connection to receive Telegram chat messages in both integrations. This is a Telegram limitation.
-
In Telegram, you created a bot for the chat that you want to obtain messages from and send messages to. For help with this, see Bots, an introduction for developers.
-
You obtained a bot authorization token from the Telegram BotFather.
-
In Syndesis, in the left panel, click Connections to display any available connections.
-
Click Create Connection to display Syndesis connectors.
-
Click the Telegram connector.
-
In the Authorization Token field, enter the Telegram authorization token for the bot that you want to get messages from and send messages to.
-
Click Next.
-
In the Name field, enter your choice of a name that helps you distinguish this connection from any other connections. For example, enter
Telegram for Product Mentions
. -
In the Description field, optionally enter any information that is helpful to know about this connection.
-
Click Save to see the updated list of available connections, including the connection that you just created. If you entered the example name, you would see that Telegram for Product Mentions appears as a connection that you can choose to add to an integration. You can add this connection to any number of integrations.
Adding a Telegram connection to trigger integration execution upon receiving messages
A Telegram connection that starts an integration triggers execution of the integration each time that the connection receives a message from a Telegram chat bot that you specify.
-
You created a Telegram connection.
-
You are creating a simple integration and Syndesis is prompting you to choose how you want to start the integration.
-
Click the Telegram connection that you want to use to start the integration.
-
Select the Read Messages action to receive all messages that are sent to the chat bot.
The connection configuration identifies the chat bot that you want to receive messages from. No other configuration is required.
Adding a Telegram connection to send a message to a Telegram chat
In an integration, you can send a message to a Telegram chat in the middle of a flow or to finish a simple integration. To do this, add a Telegram connection to the middle of a flow or as a simple integration’s finish connection.
-
You created a Telegram connection.
-
You are creating or editing a flow and Syndesis is prompting you to add to the integration. Or, Syndesis is prompting you to choose a finish connection.
-
On the Add to Integration page, click the plus sign where you want to add the connection. Skip this step if Syndesis is prompting you to choose the finish connection.
-
Click the Telegram connection that you want to add to the flow.
-
Select the Send a Text Message action.
-
In the Chat Id field, enter the ID for the Telegram chat that you are sending a message to.
If you leave this field blank, and if there is an earlier Telegram connection that receives messages, then Syndesis uses the chat ID that is in an incoming message.
To obtain the ID for a chat:
-
In Telegram, add a bot to the chat that you want to send a message to. The BotFather response includes a bot access token.
-
Copy the access token to the clipboard.
-
In Telegram, send a message directly to that chat. Do not use the bot to send the message.
-
Read messages that are sent to that chat by invoking a
curl
command in the following format. Replacebot-access-token
with the access token in the clipboard.curl https://api.telegram.org/bot-access-token/getUpdates\?offset\=0
The response is in JSON format. The JSON
chat
object contains the chat ID.
-
-
Click Next to add the connection to the flow.
Connecting to Twitter
To connect to Twitter in an integration, the main steps are:
You need a Twitter developer account to authorize access from your
Syndesis environment to Twitter. If you do not already have a Twitter
developer account,
go to https://apps.twitter.com, sign in to Twitter, and click
Apply for a developer account. Obtaining the developer account is
quick when you provide a professional email address,
such as<EMAIL_ADDRESS>rather than a personal address such as<EMAIL_ADDRESS>.
Registering Syndesis as a Twitter client application
In an integration, to connect to Twitter, the first thing you must do is register your Syndesis environment as a client application that can access Twitter. This lets you create any number of integrations that connect to Twitter. In other words, you need to register a particular Syndesis environment with Twitter only once.
In each Syndesis environment, there can be only one registration of Syndesis as a Twitter client application. However, while each Twitter connection uses the same registration, it can use different user credentials.
You are logged in to Syndesis.
-
In Syndesis:
-
In the left panel, click Settings.
-
On the Settings page, near the top, to the right of the callback URL, click to copy the callback URL for your installation of Syndesis to the clipboard. You will need this URL later in this procedure.
-
Click the Twitter entry to display the Consumer API Key and Consumer API Secret Key fields.
-
-
In another browser tab, go to your Twitter developer account, sign in if you need to, and do the following:
-
In the upper right, next to the name of your developer project, click the down caret and click Apps.
-
In the upper right, click Create an app.
-
In the App name field, enter a name such as My Syndesis Client.
-
In the Application description field, enter a tip such as For leveraging tweets.
-
In the Website URL field, paste the URL that you copied at the beginning of this procedure and remove
api/v1/credentials/callback
from the end of the URL. -
Select Enable Sign in with Twitter.
-
In the Callback URLs field, paste the URL again. It should be something like this:
https://app-proj912876.7b63.fuse-ignite.openshiftapps.com/api/v1/credentials/callback
.You can skip the next few fields.
-
Tell us how this app will be used requires a 100-character description. For convenience, you can copy and paste:
This is a demo application so that I can evaluate how to integrate Twitter with my enterprise applications.
-
Click Create.
-
Click Create again to confirm that you reviewed the developer terms.
-
At the top of the page for the app you just created, click the Keys and tokens tab.
-
Copy the consumer API key.
-
-
Return to your Syndesis Settings page and paste the Twitter consumer API key into the Syndesis Twitter Consumer API Key field.
-
Return to the Twitter Keys and tokens tab and copy the consumer API secret key.
-
Return to your Syndesis Settings page and paste the Twitter consumer API secret key into the Syndesis Twitter Consumer API Secret Key field.
-
Click Save.
-
Click the Twitter entry to collapse it.
Creating a Twitter connection
To create an integration that obtains data from Twitter, you must first create a Twitter connection. After you create a Twitter connection, you can use it in any number of integrations.
-
You are logged in to Syndesis.
-
Syndesis is open in a web browser.
-
You registered your Syndesis environment as an application that can access Twitter.
-
You added the Twitter consumer API key and consumer API secret key that you received after registration to the Syndesis Settings page.
-
In Syndesis, in the left panel, click Connections to display any available connections.
-
Click Create Connection to display the available connectors. A connector is a template that you use to create one or more connections.
-
Click the Twitter connector.
-
Click Connect Twitter to display a Twitter authorization page. You might need to log in to Twitter before you see the authorization page.
If Connect Twitter does not appear, then your Syndesis environment is not registered as a Twitter client application. See Registering Syndesis as a Twitter client application. When you try to create a Twitter connection and your Syndesis environment is not registered as a Twitter client application, then Syndesis displays multiple fields that prompt for authorization information. While you can create a Twitter connection by entering values in these fields, it is not recommended.
-
Click Authorize app to return to Syndesis.
-
In the Name field, enter your choice of a name that helps you distinguish this connection from any other connections. For example, enter
Twitter Connect 1
. -
In the Description field, optionally enter any information that is helpful to know about this connection. For example, enter
Sample Twitter connection that uses my Twitter login credentials.
-
Click Save to see that the connection you created is now available. If you entered the example name, you would see that Twitter Connect 1 appears as a connection that you can choose to add to an integration.
Adding a Twitter connection to trigger integration execution
In a simple integration, the start connection can be a Twitter connection that triggers execution of the integration when it finds certain tweets or direct messages.
-
You created a Twitter connection.
-
You are creating or editing a simple integration and Syndesis is prompting you to choose the start connection.
-
Click the Twitter connection that starts the integration. When the integration uses the selected connection to connect to Twitter, Syndesis uses the credentials defined in that connection.
-
Select the action that you want the connection to perform. A Twitter connection that you add to an integration performs only the action that you choose:
-
Mention monitors Twitter for tweets that contain the Twitter handle for the Twitter account that the connection is authorized to access. Upon finding such a tweet, the connection returns it to Syndesis, which triggers execution of the integration.
-
Retrieve periodically polls Twitter for direct messages sent to the Twitter account that the Twitter connection is authorized to access. Upon finding such messages, the connection returns them to Syndesis, which triggers execution of the integration.
-
Search periodically polls Twitter for tweets that match criteria that you specify. Upon finding such tweets, the connection returns them to Syndesis, which triggers execution of the integration.
-
-
Configure the action that you selected:
-
Mention does not require any configuration.
-
Retrieve
-
Count has a default of 100 and you should not need to change this value, which affects internal behavior. However, do not set this value to less than 50 because doing so might decrease performance.
The Retrieve action always returns all previously unreturned direct messages received in the last 30 days. If you do not select Ignore direct messages previously found, then the Retrieve action returns all messages that have been received in the last 30 days.
-
Delay is the period of time between polls. Twitter enforces rate limits, so you should not poll too often. The default interval is 5 seconds.
-
Ignore direct messages previously found returns only new messages when selected.
-
-
Search
-
Delay is the period of time between polls. Twitter enforces rate limits, so you should not poll too often. The default interval is 5 seconds.
-
Ignore tweets previously found returns only new tweets when selected.
-
Query is a Twitter-formatted search expression that specifies the tweets that you want the connection to return. The following table provides examples of what you can enter. More details are in this Twitter document for search operators.
Query
Results
My Product
Obtains tweets that contain both
My
andProduct
, but not necessarilyMy Product
."My Product”
Obtains tweets that contain an instance of
My Product
.My OR Product
Obtains tweets that contain an instance of
My
orProduct
or an instance of each one.My -Product
Obtains tweets that contain an instance of
My
and do not contain an instance ofProduct
.#MyProduct
Obtains tweets that contain the
MyProduct
hashtag.
-
-
-
Click Next to add the connection to the integration.
The integration now has a start connection and Syndesis is prompting you to choose the finish connection.
Adding a Twitter connection that sends a direct message
In the middle of a flow, or to finish a simple integration, you can add a Twitter connection that sends a direct message to a Twitter user. A direct message is a private message that goes to only the specified user.
-
You created a Twitter connection.
-
You are creating or editing a flow and Syndesis is prompting you to add to the integration. Or, for a simple integration, Syndesis is prompting you to choose a finish connection.
-
On the Add to Integration page, click the plus sign where you want to add a Twitter connection.
-
Click the Twitter connection that you want to add to the integration. When the integration uses the selected connection to connect to Twitter, Syndesis uses the credentials defined in that connection.
-
Select the Send action.
-
In the Default Message field, enter the message that you want the connection to send when a data mapper step does not map the direct message content from a previous step to this Twitter connection. While you must enter a message when you configure this Twitter connection, a message that you map from a previous step has precedence over the message that you specify here.
-
In the User field, enter the Twitter handle of the user who you want to send a direct message to. For example, both Aslan and @Aslan are correct.
-
Click Next to add the connection to the integration.
The connection appears in the middle or at the end of an integration visualization.