File Storage and Transfer

Azure Storage Blob Service

  • Description: Azure Storage Blob Service is used to store and retrieve blobs (binary large objects) from the Azure Storage Blob service using Azure APIs v12.

  • Example

    • The flow begins with a POST request to the /hello endpoint.

    • The azure-storage-blob component is used to connect to the myblobcontainer container in the camelstorageaccount01 account.

    • Parameters:

      • credentialType: "SHARED_ACCOUNT_KEY": specifies that authentication will use the shared account key.

      • operation: "listBlobs": defines that the performed operation will be listing the blobs in the container.

      • accessKey: "{{AWS_ACCESS_KEY}}": specifies the access key for authentication, referenced in a variable.

    • After listing the blobs, the flow logs information using the myLogger identifier.

Script

Diagram

 
- from:
    uri: "rest:post:/hello"
    steps:
    - to:
        uri: "azure-storage-blob://camelstorageaccount01/myblobcontainer"
        parameters:
          credentialType: "SHARED_ACCOUNT_KEY"
          operation: "listBlobs"
          accessKey: "{{AWS_ACCESS_KEY}}"
    - to:
        uri: "log:myLogger"
component azure storage blob service

AWS S3 Storage Service

  • Description: the AWS S3 Storage Service component allows you to interact with AWS’s Amazon S3 (Simple Storage Service) and manage data in S3 buckets directly through integration routes.

  • Example

    • The flow starts with a GET request to the /file/retrieve endpoint.

    • The connector retrieves a file from the S3 bucket.

      • uri: defines the bucket (name or ARN).

      • The parameters include:

        • fileName: defines the path to the file in the bucket (e.g., path/to/myfile.txt).

        • accessKey and secretKey: authentication credentials to access the bucket, referenced as environment variables.

    • Once the file is successfully retrieved, its content is logged.

Script

Diagram

 
- from:
    uri: "rest:get:/file/retrieve"
    steps:
      - to:
          uri: "aws2-s3:bucketNameOrArn"
          parameters:
            fileName: "path/to/myfile.txt"
            accessKey: "{{ACCESS_KEY}}"
            secretKey: "{{SECRET_KEY}}"
      - log:
          message: "File retrieved successfully: ${body}"
component aws3 connector

FTP/FTPS

  • Description: the FTP component provides access to remote file systems using FTP and SFTP protocols. It operates in passive mode only.

The FTPS is a secure version of FTP that adds support for SSL/TLS to encrypt the connection.

  • Example

    • The flow starts by downloading a file from an FTP server. The server is specified by the ftp_host variable.

    • The file content is transformed by replacing accented characters 'ã', 'â', 'á', 'à', and 'ä' with 'a'.

    • The modified file is then uploaded to the same or a different directory on the FTP server.

Script

Diagram

 
- from:
    id: ftp-input
    uri: "ftp:{{ftp_host}}/download"
    parameters:
      fileName: "{{ftp_filename_extension_input}}"
      password: "{{ftp_password}}"
      username: "{{ftp_username}}"
      delay: 10000
      passiveMode: true
    steps:
      - transform:
          simple: ${bodyAs(String).replaceAll('[ãâáàä]', 'a')}
      - to:
          id: ftp-output
          uri: "ftp:{{ftp_host}}/upload"
          parameters:
            fileName: "{{ftp_filename_extension_output}}"
            password: "{{ftp_password}}"
            username: "{{ftp_username}}"
            passiveMode: true
component ftp connector

Google Storage

  • Description: the Google Storage component allows integration with the Google Cloud Storage service, facilitating the storage and retrieval of objects in Google Cloud buckets. As a connector, it can be configured to upload files, list objects, and remove items from buckets. The integration supports sending different types of files and allows for header customization, overwrite control, and error handling.

  • Example

    • The flow receives a GET request.

    • It connects to Google Cloud Storage using the service account credentials and deletes the specified object in the bucket.

    • Then, it records in the log that the object was deleted.

Script

Diagram

 
- from:
    uri: rest:get:/simple
    steps:
      - to:
          uri: google-storage:my bucket
          parameters:
            serviceAccountKey: my account key
            objectName: object name
            operation: deleteObject
      - log:
          message: object deleted
component google storage

SFTP

  • Description: the SFTP component provides access to remote file systems through FTP and SFTP protocols. It operates only in passive mode.

  • Example

    • The flow in the example is configured to be triggered by a Quartz scheduler, which fires every minute.

    • When the trigger is activated, the message body is set to Hello World.

    • The flow then sends this message as a file named helloworld.txt to an SFTP server at demo.wftpserver.com, on port 2222, in the /upload directory.

Script

Diagram

 
- from:
    uri: "quartz://ipaas/trigger"
    parameters:
      cron: "* * * * *"
    steps:
    - setBody:
        constant: "Hello World"
    - to:
        uri: "sftp://demo.wftpserver.com:2222/upload"
        parameters:
          username: "demo"
          password: "demo"
          fileName: "helloworld.txt"
          passiveMode: true
component sftp
Thanks for your feedback!
EDIT

Share your suggestions with us!
Click here and then [+ Submit idea]