Request Syntax URI Request Parameters Request Body Response Syntax Response Elements Errors See Also

CreateKxCluster

Creates a new kdb cluster.

Request Syntax


POST /kx/environments/environmentId/clusters HTTP/1.1
Content-type: application/json

{
   "autoScalingConfiguration": { 
      "autoScalingMetric": "string",
      "maxNodeCount": number,
      "metricTarget": number,
      "minNodeCount": number,
      "scaleInCooldownSeconds": number,
      "scaleOutCooldownSeconds": number
   },
   "availabilityZoneId": "string",
   "azMode": "string",
   "cacheStorageConfigurations": [ 
      { 
         "size": number,
         "type": "string"
      }
   ],
   "capacityConfiguration": { 
      "nodeCount": number,
      "nodeType": "string"
   },
   "clientToken": "string",
   "clusterDescription": "string",
   "clusterName": "string",
   "clusterType": "string",
   "code": { 
      "s3Bucket": "string",
      "s3Key": "string",
      "s3ObjectVersion": "string"
   },
   "commandLineArguments": [ 
      { 
         "key": "string",
         "value": "string"
      }
   ],
   "databases": [ 
      { 
         "cacheConfigurations": [ 
            { 
               "cacheType": "string",
               "dataviewName": "string",
               "dbPaths": [ "string" ]
            }
         ],
         "changesetId": "string",
         "databaseName": "string",
         "dataviewConfiguration": { 
            "changesetId": "string",
            "dataviewName": "string",
            "dataviewVersionId": "string",
            "segmentConfigurations": [ 
               { 
                  "dbPaths": [ "string" ],
                  "onDemand": boolean,
                  "volumeName": "string"
               }
            ]
         },
         "dataviewName": "string"
      }
   ],
   "executionRole": "string",
   "initializationScript": "string",
   "releaseLabel": "string",
   "savedownStorageConfiguration": { 
      "size": number,
      "type": "string",
      "volumeName": "string"
   },
   "scalingGroupConfiguration": { 
      "cpu": number,
      "memoryLimit": number,
      "memoryReservation": number,
      "nodeCount": number,
      "scalingGroupName": "string"
   },
   "tags": { 
      "string" : "string" 
   },
   "tickerplantLogConfiguration": { 
      "tickerplantLogVolumes": [ "string" ]
   },
   "vpcConfiguration": { 
      "ipAddressType": "string",
      "securityGroupIds": [ "string" ],
      "subnetIds": [ "string" ],
      "vpcId": "string"
   }
}

URI Request Parameters

The request uses the following URI parameters.

environmentId

A unique identifier for the kdb environment.

Length Constraints: Minimum length of 1. Maximum length of 32.

Pattern: ^[a-z0-9]+$

Required: Yes

Request Body

The request accepts the following data in JSON format.

azMode

The number of availability zones you want to assign per cluster. This can be one of the following

SINGLE – Assigns one availability zone per cluster.
MULTI – Assigns all the availability zones per cluster.

Type: String

Valid Values: SINGLE | MULTI

Required: Yes

clusterName

A unique name for the cluster that you want to create.

Type: String

Length Constraints: Minimum length of 3. Maximum length of 63.

Pattern: ^[a-zA-Z0-9][a-zA-Z0-9-_]*[a-zA-Z0-9]$

Required: Yes

clusterType

Specifies the type of KDB database that is being created. The following types are available:

HDB – A Historical Database. The data is only accessible with read-only permissions from one of the FinSpace managed kdb databases mounted to the cluster.
RDB – A Realtime Database. This type of database captures all the data from a ticker plant and stores it in memory until the end of day, after which it writes all of its data to a disk and reloads the HDB. This cluster type requires local storage for temporary storage of data during the savedown process. If you specify this field in your request, you must provide the savedownStorageConfiguration parameter.
GATEWAY – A gateway cluster allows you to access data across processes in kdb systems. It allows you to create your own routing logic using the initialization scripts and custom code. This type of cluster does not require a writable local storage.
GP – A general purpose cluster allows you to quickly iterate on code during development by granting greater access to system commands and enabling a fast reload of custom code. This cluster type can optionally mount databases including cache and savedown storage. For this cluster type, the node count is fixed at 1. It does not support autoscaling and supports only SINGLE AZ mode.
Tickerplant – A tickerplant cluster allows you to subscribe to feed handlers based on IAM permissions. It can publish to RDBs, other Tickerplants, and real-time subscribers (RTS). Tickerplants can persist messages to log, which is readable by any RDB environment. It supports only single-node that is only one kdb process.

Type: String

Valid Values: HDB | RDB | GATEWAY | GP | TICKERPLANT

Required: Yes

releaseLabel

The version of FinSpace managed kdb to run.

Type: String

Length Constraints: Minimum length of 1. Maximum length of 16.

Pattern: ^[a-zA-Z0-9._-]+$

Required: Yes

vpcConfiguration

Configuration details about the network where the Privatelink endpoint of the cluster resides.

Type: VpcConfiguration object

Required: Yes

autoScalingConfiguration

The configuration based on which FinSpace will scale in or scale out nodes in your cluster.

Type: AutoScalingConfiguration object

Required: No

availabilityZoneId

The availability zone identifiers for the requested regions.

Type: String

Length Constraints: Minimum length of 8. Maximum length of 12.

Pattern: ^[a-zA-Z0-9-]+$

Required: No

cacheStorageConfigurations

The configurations for a read only cache storage associated with a cluster. This cache will be stored as an FSx Lustre that reads from the S3 store.

Type: Array of KxCacheStorageConfiguration objects

Required: No

capacityConfiguration

A structure for the metadata of a cluster. It includes information like the CPUs needed, memory of instances, and number of instances.

Type: CapacityConfiguration object

Required: No

clientToken

A token that ensures idempotency. This token expires in 10 minutes.

Type: String

Length Constraints: Minimum length of 1. Maximum length of 36.

Pattern: .*\S.*

Required: No

clusterDescription

A description of the cluster.

Type: String

Length Constraints: Minimum length of 1. Maximum length of 1000.

Pattern: ^[a-zA-Z0-9\_\-\.\s]+$

Required: No

code

The details of the custom code that you want to use inside a cluster when analyzing a data. It consists of the S3 source bucket, location, S3 object version, and the relative path from where the custom code is loaded into the cluster.

Type: CodeConfiguration object

Required: No

commandLineArguments

Defines the key-value pairs to make them available inside the cluster.

Type: Array of KxCommandLineArgument objects

Required: No

databases

A list of databases that will be available for querying.

Type: Array of KxDatabaseConfiguration objects

Required: No

executionRole

An IAM role that defines a set of permissions associated with a cluster. These permissions are assumed when a cluster attempts to access another cluster.

Type: String

Length Constraints: Minimum length of 1. Maximum length of 1024.

Pattern: ^arn:aws[a-z0-9-]*:iam::\d{12}:role\/[\w-\/.@+=,]{1,1017}$

Required: No

initializationScript

Specifies a Q program that will be run at launch of a cluster. It is a relative path within .zip file that contains the custom code, which will be loaded on the cluster. It must include the file name itself. For example, somedir/init.q.

Type: String

Length Constraints: Minimum length of 1. Maximum length of 255.

Pattern: ^[a-zA-Z0-9\_\-\.\/\\]+$

Required: No

savedownStorageConfiguration

The size and type of the temporary storage that is used to hold data during the savedown process. This parameter is required when you choose clusterType as RDB. All the data written to this storage space is lost when the cluster node is restarted.

Type: KxSavedownStorageConfiguration object

Required: No

scalingGroupConfiguration

The structure that stores the configuration details of a scaling group.

Type: KxScalingGroupConfiguration object

Required: No

tags

A list of key-value pairs to label the cluster. You can add up to 50 tags to a cluster.

Type: String to string map

Map Entries: Maximum number of 50 items.

Key Length Constraints: Minimum length of 1. Maximum length of 128.

Key Pattern: ^(?!aws:)[a-zA-Z+-=._:/]+$

Value Length Constraints: Minimum length of 1. Maximum length of 256.

Value Pattern: ^[a-zA-Z0-9+-=._:@ ]+$

Required: No

tickerplantLogConfiguration

A configuration to store Tickerplant logs. It consists of a list of volumes that will be mounted to your cluster. For the cluster type Tickerplant, the location of the TP volume on the cluster will be available by using the global variable .aws.tp_log_path.

Type: TickerplantLogConfiguration object

Required: No

Response Syntax


HTTP/1.1 200
Content-type: application/json

{
   "autoScalingConfiguration": { 
      "autoScalingMetric": "string",
      "maxNodeCount": number,
      "metricTarget": number,
      "minNodeCount": number,
      "scaleInCooldownSeconds": number,
      "scaleOutCooldownSeconds": number
   },
   "availabilityZoneId": "string",
   "azMode": "string",
   "cacheStorageConfigurations": [ 
      { 
         "size": number,
         "type": "string"
      }
   ],
   "capacityConfiguration": { 
      "nodeCount": number,
      "nodeType": "string"
   },
   "clusterDescription": "string",
   "clusterName": "string",
   "clusterType": "string",
   "code": { 
      "s3Bucket": "string",
      "s3Key": "string",
      "s3ObjectVersion": "string"
   },
   "commandLineArguments": [ 
      { 
         "key": "string",
         "value": "string"
      }
   ],
   "createdTimestamp": number,
   "databases": [ 
      { 
         "cacheConfigurations": [ 
            { 
               "cacheType": "string",
               "dataviewName": "string",
               "dbPaths": [ "string" ]
            }
         ],
         "changesetId": "string",
         "databaseName": "string",
         "dataviewConfiguration": { 
            "changesetId": "string",
            "dataviewName": "string",
            "dataviewVersionId": "string",
            "segmentConfigurations": [ 
               { 
                  "dbPaths": [ "string" ],
                  "onDemand": boolean,
                  "volumeName": "string"
               }
            ]
         },
         "dataviewName": "string"
      }
   ],
   "environmentId": "string",
   "executionRole": "string",
   "initializationScript": "string",
   "lastModifiedTimestamp": number,
   "releaseLabel": "string",
   "savedownStorageConfiguration": { 
      "size": number,
      "type": "string",
      "volumeName": "string"
   },
   "scalingGroupConfiguration": { 
      "cpu": number,
      "memoryLimit": number,
      "memoryReservation": number,
      "nodeCount": number,
      "scalingGroupName": "string"
   },
   "status": "string",
   "statusReason": "string",
   "tickerplantLogConfiguration": { 
      "tickerplantLogVolumes": [ "string" ]
   },
   "volumes": [ 
      { 
         "volumeName": "string",
         "volumeType": "string"
      }
   ],
   "vpcConfiguration": { 
      "ipAddressType": "string",
      "securityGroupIds": [ "string" ],
      "subnetIds": [ "string" ],
      "vpcId": "string"
   }
}

Response Elements

If the action is successful, the service sends back an HTTP 200 response.

The following data is returned in JSON format by the service.

autoScalingConfiguration

The configuration based on which FinSpace will scale in or scale out nodes in your cluster.

Type: AutoScalingConfiguration object

availabilityZoneId

The availability zone identifiers for the requested regions.

Type: String

Length Constraints: Minimum length of 8. Maximum length of 12.

Pattern: ^[a-zA-Z0-9-]+$

azMode

The number of availability zones you want to assign per cluster. This can be one of the following

SINGLE – Assigns one availability zone per cluster.
MULTI – Assigns all the availability zones per cluster.

Type: String

Valid Values: SINGLE | MULTI

cacheStorageConfigurations

The configurations for a read only cache storage associated with a cluster. This cache will be stored as an FSx Lustre that reads from the S3 store.

Type: Array of KxCacheStorageConfiguration objects

capacityConfiguration

A structure for the metadata of a cluster. It includes information like the CPUs needed, memory of instances, and number of instances.

Type: CapacityConfiguration object

clusterDescription

A description of the cluster.

Type: String

Length Constraints: Minimum length of 1. Maximum length of 1000.

Pattern: ^[a-zA-Z0-9\_\-\.\s]+$

clusterName

A unique name for the cluster.

Type: String

Length Constraints: Minimum length of 3. Maximum length of 63.

Pattern: ^[a-zA-Z0-9][a-zA-Z0-9-_]*[a-zA-Z0-9]$

clusterType

Specifies the type of KDB database that is being created. The following types are available:

HDB – A Historical Database. The data is only accessible with read-only permissions from one of the FinSpace managed kdb databases mounted to the cluster.
RDB – A Realtime Database. This type of database captures all the data from a ticker plant and stores it in memory until the end of day, after which it writes all of its data to a disk and reloads the HDB. This cluster type requires local storage for temporary storage of data during the savedown process. If you specify this field in your request, you must provide the savedownStorageConfiguration parameter.
GATEWAY – A gateway cluster allows you to access data across processes in kdb systems. It allows you to create your own routing logic using the initialization scripts and custom code. This type of cluster does not require a writable local storage.
GP – A general purpose cluster allows you to quickly iterate on code during development by granting greater access to system commands and enabling a fast reload of custom code. This cluster type can optionally mount databases including cache and savedown storage. For this cluster type, the node count is fixed at 1. It does not support autoscaling and supports only SINGLE AZ mode.
Tickerplant – A tickerplant cluster allows you to subscribe to feed handlers based on IAM permissions. It can publish to RDBs, other Tickerplants, and real-time subscribers (RTS). Tickerplants can persist messages to log, which is readable by any RDB environment. It supports only single-node that is only one kdb process.

Type: String

Valid Values: HDB | RDB | GATEWAY | GP | TICKERPLANT

code

Type: CodeConfiguration object

commandLineArguments

Defines the key-value pairs to make them available inside the cluster.

Type: Array of KxCommandLineArgument objects

createdTimestamp

The timestamp at which the cluster was created in FinSpace. The value is determined as epoch time in milliseconds. For example, the value for Monday, November 1, 2021 12:00:00 PM UTC is specified as 1635768000000.

Type: Timestamp

databases

A list of databases that will be available for querying.

Type: Array of KxDatabaseConfiguration objects

environmentId

A unique identifier for the kdb environment.

Type: String

Length Constraints: Minimum length of 1. Maximum length of 32.

Pattern: ^[a-z0-9]+$

executionRole

An IAM role that defines a set of permissions associated with a cluster. These permissions are assumed when a cluster attempts to access another cluster.

Type: String

Length Constraints: Minimum length of 1. Maximum length of 1024.

Pattern: ^arn:aws[a-z0-9-]*:iam::\d{12}:role\/[\w-\/.@+=,]{1,1017}$

initializationScript

Type: String

Length Constraints: Minimum length of 1. Maximum length of 255.

Pattern: ^[a-zA-Z0-9\_\-\.\/\\]+$

lastModifiedTimestamp

The last time that the cluster was modified. The value is determined as epoch time in milliseconds. For example, the value for Monday, November 1, 2021 12:00:00 PM UTC is specified as 1635768000000.

Type: Timestamp

releaseLabel

A version of the FinSpace managed kdb to run.

Type: String

Length Constraints: Minimum length of 1. Maximum length of 16.

Pattern: ^[a-zA-Z0-9._-]+$

savedownStorageConfiguration

Type: KxSavedownStorageConfiguration object

scalingGroupConfiguration

The structure that stores the configuration details of a scaling group.

Type: KxScalingGroupConfiguration object

status

The status of cluster creation.

PENDING – The cluster is pending creation.
CREATING – The cluster creation process is in progress.
CREATE_FAILED – The cluster creation process has failed.
RUNNING – The cluster creation process is running.
UPDATING – The cluster is in the process of being updated.
DELETING – The cluster is in the process of being deleted.
DELETED – The cluster has been deleted.
DELETE_FAILED – The cluster failed to delete.

Type: String

statusReason

The error message when a failed state occurs.

Type: String

Length Constraints: Minimum length of 1. Maximum length of 250.

Pattern: ^[a-zA-Z0-9\_\-\.\s]+$

tickerplantLogConfiguration

A configuration to store the Tickerplant logs. It consists of a list of volumes that will be mounted to your cluster. For the cluster type Tickerplant, the location of the TP volume on the cluster will be available by using the global variable .aws.tp_log_path.

Type: TickerplantLogConfiguration object

volumes

A list of volumes mounted on the cluster.

Type: Array of Volume objects

Array Members: Minimum number of 0 items. Maximum number of 5 items.

vpcConfiguration

Configuration details about the network where the Privatelink endpoint of the cluster resides.

Type: VpcConfiguration object

Errors

For information about the errors that are common to all actions, see Common Errors.

AccessDeniedException

You do not have sufficient access to perform this action.

HTTP Status Code: 403

AccessDeniedException

You do not have sufficient access to perform this action.

HTTP Status Code: 403

ConflictException

There was a conflict with this action, and it could not be completed.

HTTP Status Code: 409

InternalServerException

The request processing has failed because of an unknown error, exception or failure.

HTTP Status Code: 500

LimitExceededException

A service limit or quota is exceeded.

HTTP Status Code: 400

ResourceNotFoundException

One or more resources can't be found.

HTTP Status Code: 404

ThrottlingException

The request was denied due to request throttling.

HTTP Status Code: 429

ValidationException

The input fails to satisfy the constraints specified by an AWS service.

HTTP Status Code: 400