Data Streaming

Data Streaming gives you the autonomy to configure your streaming services independently. With Data Streaming, terabytes of data can be continuously captured and sent so that you can base your business decisions on dynamically updated information.

The Data Streaming screen lists your Delivery Streams in cards that display the following information:

  • Delivery Stream name;

  • Whether it is active or inactive;

  • Destination: the destination of the delivery stream (Datadog, AWS S3, or Dynatrace);

  • Creation Date: date you first configured the delivery stream.

  • By enabling the export of traces, there may be an increase in the charges for these services, which is not related to Sensedia.

  • Activation or deactivation of Data Streaming may take up to 15 minutes.

Configuring a new delivery stream

You have these destination options for the delivery stream:

  • For AWS# (JSON): The traces will be exported as a compressed gzip object.

  • Each exported object can contain multiple traces.

Due to the encryption of the access keys for Data Streaming, they are not displayed on the screen and cannot be retrieved after they are saved.

If you cannot create, edit, or delete a Data Stream, please check your access permissions.

Datadog

To create a new delivery stream with Datadog as destination, follow the steps below:

  1. Access Data Streaming from the left menu or click the card located on your Analytics home screen.

  2. Click + NEW DELIVERY STREAM.
    detail of screen with new delivery stream button

  3. Complete the fields:

    detail of the new datadog delivery stream creation screen

    • Destination: Select Datadog;

    • Delivery stream name: Define a name for the delivery stream. Do not use spaces or special characters;

    • Site: Select the Datadog site to which your metrics and traces will be sent;

    • API key: Enter your API key. This information can be found in Datadog under Organization Settings  API Keys.

  4. Click ACTIVATE to confirm activation of the Delivery stream.
    delivery stream activation confirmation screen

To export traces to Datadog, you will need the APM Host license. For more information, see the Datadog billing model.



AWS S3 (Open Telemetry)

To create a new delivery stream with AWS S3 as destination, follow the steps below:

  1. Access Data Streaming from the left menu or click the card located on your Analytics home screen.

  2. Click + NEW DELIVERY STREAM.
    detail of screen with new delivery stream button

  3. Complete the fields:

    detalhe da tela de criação de streaming para aws

    • Destination: Select AWS S3;

    • Delivery stream name: Define a name for the delivery stream. Do not use spaces or special characters;

    • AWS Region: Select the region where your AWS S3 is hosted;

    • Bucket name: Enter an AWS S3 bucket to which the data streams will be sent. You must be the owner of this bucket in order to have the required permissions to use it as destination;

    • Key: Enter the access key ID for your resource;

      IMPORTANT:The access key you provide must have the required permissions on the AWS S3 bucket — PutObject, GetObject, and AbortMultipartUpload — when creating the Data Streaming.

    • Secret: Enter the secret access key.

  4. Click ACTIVATE to confirm activation of the Delivery stream.
    delivery stream activation confirmation screen


Dynatrace

To create a new delivery stream with Dynatrace as the destination, follow the steps below:

  1. Access the Data Streaming screen via the left menu or the card on the Analytics home screen.

  2. Click the + NEW DELIVERY STREAM button.
    detail of screen with new delivery stream button

  3. Fill in the fields:

    fields to configure Dynatrace

    • Destination: select Dynatrace;

    • Delivery stream name: define a name for the delivery stream. Avoid using spaces or special characters;

    • Active Gate URL: Provide the URL for your Dynatrace ActiveGate instance. Ensure the URL is accessible and correctly configured to receive the exported data.

    • API access token: Enter a valid API token for the Dynatrace destination. The token must have the necessary permissions to authenticate and send data to the specified destination.

      Ensure that the token has permissions for ingesting OpenTelemetry traces, metrics, and logs.

  4. Click ACTIVATE to confirm the activation of the Delivery stream.
    delivery stream activation confirmation screen

Only one destination can be configured when exporting data to Dynatrace.


OTLP Datadog Agent

To create a new delivery stream with OTLP (Datadog Agent) as the destination, follow the steps below:

  1. Access the Data Streaming screen via the left menu or the card on the Analytics home screen.

  2. Click the + NEW DELIVERY STREAM button.
    detail of screen with new delivery stream button

  3. Complete the fields:

    detail of the new OTLP Datadog Agent delivery stream creation screen

    • Destination: Select OTLP (Datadog Agent);

    • Delivery stream name: Define a name for the delivery stream. Avoid using spaces or special characters;

    • Endpoint: Provide the endpoint of your Datadog Agent. The endpoint must be accessible and correctly configured to receive the exported data.

      The connection uses the gRPC protocol. Make sure to use an endpoint that is compatible with this protocol.

  4. Click CREATE to confirm the activation of the Delivery stream.
    delivery stream activation confirmation screen


AWS S3 (JSON)

The configuration for sending data to AWS S3 in JSON is performed by support.
Follow the steps below to open a ticket:

  1. Access the Data Streaming screen via the left menu or the card on your Analytics home screen.

  2. Click the + NEW DELIVERY STREAM button.
    detail of the screen with the new delivery stream button

  3. Select AWS S3 (JSON) in Destination.

  4. With the information listed below, click the CREATE TICKET button.
    You will be redirected to the Zendesk ticket creation screen.

    • Delivery stream name: define a name for the delivery stream, without spaces or special characters;

    • AWS region: the region where the bucket is hosted;

    • Bucket name: name of the AWS S3 bucket to which the data streams will be sent. You must be the owner of this bucket to allow it to be used as a destination;

    • Root path (optional): root path of the bucket where the data will be stored.

    • Key/secret or role ARN:

      • If you provide a key (access key ID), also provide the secret (secret access key) associated with it.

      • If you provide a role ARN, it is not necessary to provide the key and secret.

        Either the access key ID or the role ARN you provide must have the required permissions on the AWS S3 bucket — PutObject, GetObject, and AbortMultipartUpload — when creating the Data Streaming.

    • Partition (partition): the AWS S3 partition where the data will be stored, which can be by year, month, day, and hour.

Information about Data Transfer has been removed from Data Streaming.


Editing delivery stream settings

To change the Datadog website or API Key — for Datadog as destination; or to change the bucket, AWS Region, access key or secret — for AWS S3 as destination, follow the steps below:

  1. Access Data Streaming from the left menu or click the card located on your Analytics home screen.

  2. Find the Delivery Stream you want to edit. You can search by name or part of it.

  3. Click pencil icon representing edit action.

  4. Change the information and click SAVE.

Once changed, the previous settings cannot be recovered.

Deleting a delivery stream

To delete a delivery stream, click trash can icon representing the delete action.

  • Deleted delivery streams cannot be recovered. It may take several minutes for the deletion to take place.

  • Some delivery streams can only be deleted by the support team. If the delete button is disabled, please open a ticket.
Thanks for your feedback!
EDIT

Share your suggestions with us!
Click here and then [+ Submit idea]