The Global Observer is an integration method that allows sending an external system all the data in the application as a series of messages as data changes, using a generic integration message queue.
If enabled, Researcher Solutions generates a message every time any object is created, updated or deleted in the system. This includes workflow transitions, submission of a form, editing a record, adding a record, etc.
Message format
Messages are formatted as JSON documents which contain zero or more messages, and a token which identifies the set of messages. If you use the polling API, this token is used to mark the messages as read.
Inspect the queue
The message queue can be inspected in the UI. It is found in Tools → Integrations → Integration message queues → Global observation.
From here, you can see all messages ever generated by the system and which ones have been consumed by the integration layer. By clicking on the Info... link, you can also see the content of any message.
Errors in integrations
Any errors, such as HTTP transport errors, can be viewed and acknowledged in the admin interface: Tools → Integrations → Errors from integrations.
Default message format
The default format for messages is a JSON document containing zero or more messages, along with a token which can be used to mark messages as read.
The top level properties are:
version | Format version. |
token | An opaque token representing the messages included in this response. |
messages | An array of zero or more messages. |
Within the messages array, each entry will include the properties:
id | The message ID. |
datetime | The date and time in ISO8601 format when the message was generated. |
The messages will include other properties, but these will all be specific to your application and the integration.
Message IDs may not be sequential, and you can’t use them to check that all messages have been received.
Message delivery
Messages are offered up for consumption by the customer’s integration layer in one of two ways:
- Customer polls for new messages on a schedule for new messages and marks them as read (default)
- See documentation: Message queue polling API
- Researcher Solutions push messages to a customer-defined endpoint
- See documentation: Message queue push to remote HTTPS endpoint
Message queue polling API
The polling API enables your systems to pull messages from the Haplo application. This is our recommended method.
The Global observation page (referred to in section “Inspect the queue” above) will give you two URLs: Poll URL and Mark URL.
The Poll URL is used to fetch the unread messages from the queue. This includes a token, which is then POSTed to the Mark URL to mark the messages as read and ensure they are not sent again.
This two step process ensures that transport errors will not result in messages being lost.
Create an API key
Polling requests are authenticated with an API key. Click Create API key... on the queue information page.
To authenticate the request, use HTTP Basic authentication with the username haplo and the API key as the password.
Request unread messages
To fetch a small number of the oldest unread messages, you should make a simple authenticated GET request to the Poll URL. Here’s an example script using a curl command:
#!/bin/sh
set -e
API_KEY=01234567890123456789
SERVER=example.haplo.train-au.cayuse.com
curl -X GET --user haplo:${API_KEY} https://${SERVER}/api/haplo-integration-messages/poll/{QUEUE}/fetchImportant: Replace QUEUE in the URL with the name of the queue. You can find the exact URL to use from the queue information page.
To read all messages in a queue, repeat the requests to the poll and mark endpoints until the unreadCount is zero.
Mark messages as read
The response to the Poll URL will include a token. This must be posted back to the server to mark the messages received as read, so they will not be included the next time the application is polled.
The token is opaque and should not be interpreted by your integration code, as its format may change at any time.
Your integration should extract the token from the response, then POST to the Mark URL with the token as the token parameter. An example curl script is:
#!/bin/sh
set -e
API_KEY=01234567890123456789
SERVER=example.haplo.train-au.cayuse.com
curl -X POST --data-urlencode "token=MSwy8w099=" --user haplo:${API_KEY} https://${SERVER}/api/haplo-integration-messages/poll/{QUEUE}/markImportant: Replace QUEUE in the URL with the name of the queue. You can find the exact URL to use from the queue information page.
The response to the Mark request will be JSON with the structure below:
{
"markedCount":2,
"unreadCount":1
}Configuration for different environments
You will probably need to write your integration so that it can be used with test and live environments.
The only things which will differ between environments will be the hostname in the URLs and the API key.
Message queue push to remote HTTPS endpoint
If your application pushes messages to an HTTPS endpoint on your systems, the configuration is specific to your application. We’ll provide details on how to configure the HTTPS endpoint.
Options include:
- Sending messages immediately they’re generated
- Sending batches of messages on a schedule
- HTTP Basic authentication
- SSL client certificates
Note that only https: endpoints are supported. Cayuse does not support unencrypted protocols.
Reliability
When the remote endpoint returns a 200 OK response, the messages will be marked as sent.
Any other status code or transport error will not mark them as sent, and the messages will be resent at some point in the future.
Admin user interface
The admin user interface will include a button to send all messages to your endpoint, rather than waiting for a schedule. This is useful when you have marked a message for resending.