ecs_composex_msk_cluster package

Submodules

ecs_composex_msk_cluster.msk_cluster module

class ecs_composex_msk_cluster.msk_cluster. MskCluster ( name , definition , module , settings ) [source]

Bases: NetworkXResource

Class to manage MSK Cluster resource

property cluster_uuid
init_outputs ( ) [source]

Placeholder method

to_ecs ( settings , modules , root_stack = None ) [source]

Maps a database service to ECS services

Return type :

None

ecs_composex_msk_cluster.msk_cluster. define_msk_clusters_mappings ( module , settings ) [source]
ecs_composex_msk_cluster.msk_cluster. describe_kafka_cluster ( cluster , account_id , resource_id ) [source]
Return type :

dict

ecs_composex_msk_cluster.msk_cluster_conditions module

ecs_composex_msk_cluster.msk_cluster_ecs module

ecs_composex_msk_cluster.msk_cluster_ecs. handle_kafka_iam_permissions ( resource , settings ) [source]

ecs_composex_msk_cluster.msk_cluster_ecs_iam module

Manages the IAM Permissions & SCRAM configuration for authentication to the Kafka cluster resources

ecs_composex_msk_cluster.msk_cluster_ecs_iam. handle_cluster_permissions ( resource , settings , cluster_def , statement , cluster_arn_prefix ) [source]

Handle Cluster permissions

Return type :

None

ecs_composex_msk_cluster.msk_cluster_ecs_iam. handle_group_iam_policy ( resource , groups_def , statement , cluster_arn_prefix ) [source]

Creates the IAM policies for MSK IAM connectivity to Kafka Topics

Return type :

None

ecs_composex_msk_cluster.msk_cluster_ecs_iam. handle_iam_kafka_resources_access ( resource , settings , family , family_target_def ) [source]

Creates IAM Policies for access to MSK resources [topic, group, cluster, transactional-id]

Return type :

None

ecs_composex_msk_cluster.msk_cluster_ecs_iam. handle_topic_iam_policy ( resource , topics_def , statement , cluster_arn_prefix ) [source]

Creates the IAM policies for MSK IAM connectivity to Kafka Topics

Return type :

None

ecs_composex_msk_cluster.msk_cluster_ecs_iam. handle_transactional_iam_policy ( resource , transactional_def , statement , cluster_arn_prefix ) [source]

Creates the IAM policies for MSK IAM connectivity to Kafka Topics

Return type :

None

ecs_composex_msk_cluster.msk_cluster_logging module

ecs_composex_msk_cluster.msk_cluster_logging. add_log_group ( cluster , template ) [source]

Creates a new Log Group for the cluster and configures the property for it.

Return type :

None

ecs_composex_msk_cluster.msk_cluster_logging. handle_logging ( cluster , cluster_template , settings ) [source]
Return type :

None

ecs_composex_msk_cluster.msk_cluster_module module

ecs_composex_msk_cluster.msk_cluster_params module

ecs_composex_msk_cluster.msk_cluster_stack module

Main module.

class ecs_composex_msk_cluster.msk_cluster_stack. XStack ( title , settings , module , ** kwargs ) [source]

Bases: ComposeXStack

Class to handle MSK resources

ecs_composex_msk_cluster.msk_cluster_template module

class ecs_composex_msk_cluster.msk_cluster_template. MskClusterStack ( name , stack_template , cluster , top_stack , ** kwargs ) [source]

Bases: ComposeXStack

Class to represent an MSK Cluster’ Stack

ecs_composex_msk_cluster.msk_cluster_template. build_msk_clusters ( module , msk_top_stack ) [source]

Creates a new MSK cluster from properties Each MSK cluster gets its own template & own stack to allow for better re-usability.

ecs_composex_msk_cluster.msk_cluster_template. set_instance_type ( cluster , cluster_stack ) [source]

Checks the instance type value. Replaces it with parameter. Updates parameter value if valid, uses Default if not

Return type :

None

ecs_composex_msk_cluster.msk_cluster_template. set_msk_cluster_template ( module , cluster ) [source]
Return type :

Template

ecs_composex_msk_cluster.msk_cluster_template. validate_cluster_version ( cluster , input_version ) [source]

Validates the kafka version

Return type :

Parameter

ecs_composex_msk_cluster.msk_configuration module

MSK Configuration

class ecs_composex_msk_cluster.msk_configuration. MskConfiguration ( title , ** kwargs ) [source]

Bases: Configuration

static config_dict_to_string ( config ) [source]
Return type :

str

default_configurations : dict = {'auto.create.topics.enable': False, 'default.replication.factor': 3, 'min.insync.replicas': 2, 'num.io.threads': 8, 'num.network.threads': 5, 'num.partitions': 1, 'num.replica.fetchers': 2, 'replica.lag.time.max.ms': 30000, 'socket.receive.buffer.bytes': 102400, 'socket.request.max.bytes': 104857600, 'socket.send.buffer.bytes': 102400, 'unclean.leader.election.enable': True, 'zookeeper.session.timeout.ms': 18000}

ecs_composex_msk_cluster.msk_sg_ingress module

ecs_composex_msk_cluster.msk_sg_ingress. allow_clients_sg_ingress ( cluster ) [source]
Return type :

list [ If ]

ecs_composex_msk_cluster.msk_sg_ingress. handle_msk_auth_parameters ( cluster ) [source]
Return type :

None

ecs_composex_msk_cluster.msk_sg_ingress. handle_msk_auth_settings ( cluster ) [source]
ecs_composex_msk_cluster.msk_sg_ingress. str_to_bool ( input_var ) [source]
Return type :

bool

ecs_composex_msk_cluster.msk_storage_scaling module

Configure MSK Cluster autoscaling

ecs_composex_msk_cluster.msk_storage_scaling. add_storage_scaling ( cluster , template ) [source]

Adds storage autoscaling to the MSK Cluster. Application Autoscaling will add capacity as the data on the cluster grows. However, it won’t scale in (remove capacity). This feature is mostly aimed at ensuring there is always storage available on the Kafka cluster

Return type :

None

ecs_composex_msk_cluster.msk_storage_scaling. set_scalable_target ( cluster , storage_scaling ) [source]
Return type :

Optional [ ScalableTarget ]

ecs_composex_msk_cluster.msk_storage_scaling. set_scaling_policy ( cluster , storage_scaling , scaling_target ) [source]
Return type :

ScalingPolicy

Module contents

Top-level package for ECS Compose-X - MSK Cluster.