Connect - Connector Flashcards
(23 cards)
Connect: labels or annotations
annotation
Connector: labels or annotations
labels
Debezium: label or annotation
labels
Transformation: labels or annotation
labels
Connect annotations
annotations:
___strimzi.io/use-connector-resources: ‘true’
Connect abbreviation
bbc virtual reality
Expand bbc virtual reality
Elements of connect spec:
build
bootstrapServer
config
version
replicas
Connect’s build element
build:
___output:
____ image: ‘quay.io/USERNAME/filewatch-:latest’
____ pushSecret: filewatch-connect-push-secret
____ type: docker
___plugins:
____ - artifacts:
_____ - type: tgz
_____ url: https://repo1.maven.org/maven2/or.gz
____ name: filewatch-source-connector
Connect config
config. storage.topic: filewatch-connect-cluster-configs
offset. storage.topic: filewatch-connect-cluster-offsets
status. storage.topic: filewatch-connect-cluster-status
key. converter: org.apache.kafka.connect.storage.StringConverter
value. converter: org.apache.kafka.connect.storage.StringConverter
config. storage.replication.factor: 1
offset. storage.replication.factor: 1
status. storage.replication.factor: 1
Connector abreviation
cct
Expand cct
class
config
taskMax
File source complete connector

Create connector from CLI
curl -X POST \
http: //localhost:8083/connectors \
- H ‘Content-Type: application/json’ \
- d ‘{ “name”: “connector-name”, “config”: {}}’
Config elements for creating connector from CLI
“connector.class”: “‘$connector_class’”,
“tasks.max”: “1”,
“topics”: “github-events”,
“camel.sink.endpoint.hostAddresses”: “elasticsearch-es-http:9200”,
“camel.sink.endpoint.indexName”: “github_events”,
“camel.sink.endpoint.operation”: “Index”,
“camel.sink.path.clusterName”: “elasticsearch”,
“key.converter”: “org.apache.kafka.connect.storage.StringConverter”,
“value.converter”: “org.apache.kafka.connect.storage.StringConverter”
Spec of camel github soucrce connector

Debezium connector

Connector transform fields
transforms: Mask,Insert
transforms. Mask.type: org.apache.kafka.connect.transforms.MaskField$Value
transforms. Mask.fields: ssn
transforms. Insert.type: org.apache.kafka.connect.transforms.InsertField$Value
transforms. Insert.static.field: data-source
transforms. Insert.static.value: database
JDBC Connector

Event router transform
transforms: EventRouter
transforms. EventRouter.type: io.debezium.transforms.outbox.EventRouter
transforms. EventRouter.table.fields.additional.placement: type:header:eventType
transforms. EventRouter.route.topic.replacement: ${routedByValue}.events
Standard JDBC connector config elements
database. hostname: postgresql-repair-requests
database. port: 5432
database. user: postgres
database. password: postgres
database. dbname: waterleakdb
database. server.name: waterleakdb1
Alternative to bbc vr
bic vr
Expand bic vr
build
image
config
version
replica
Standard config elements for connect
config. storage.topic: my-connect-cluster-configs
offset. storage.topic: my-connect-cluster-offsets
status. storage.topic: my-connect-cluster-status
key. converter: org.apache.kafka.connect.storage.StringConverter
value. converter: org.apache.kafka.connect.storage.StringConverter
config. storage.replication.factor: 1
offset. storage.replication.factor: 1
status. storage.replication.factor: 1