Ask Your Question
2

Is it not possible for TaskManager to register with JobManager's Resource Manager while using Apache Flink with Docker Swarm?

asked 2021-08-08 11:00:00 +0000

djk gravatar image

edit retag flag offensive close merge delete

1 Answer

Sort by ยป oldest newest most voted
1

answered 2022-12-03 08:00:00 +0000

qstack gravatar image

It is possible for TaskManager to register with JobManager's Resource Manager while using Apache Flink with Docker Swarm. In fact, this is a common use case for deploying Apache Flink on Docker Swarm. To achieve this, you need to use Docker Swarm's overlay network mode to allow communication between the containers running TaskManager and JobManager. You also need to configure the Flink resource manager explicitly to use Docker Swarm mode. This can be done by setting the Flink configuration parameters:

resourcemanager.slot.request.timeout: <timeout-value>
containerized.heap-cutoff-ratio: <ratio-value>
containerized.heap-cutoff-min: <min-value>
containerized.heap-cutoff-max: <max-value>

These parameters specify the timeout value for slot requests, the heap cutoff ratio, and the minimum and maximum heap cutoff values. With these configurations in place, TaskManagers can successfully register with the Resource Manager of the JobManager, and tasks can be executed on the Docker Swarm cluster.

edit flag offensive delete link more

Your Answer

Please start posting anonymously - your entry will be published after you log in or create a new account. This space is reserved only for answers. If you would like to engage in a discussion, please instead post a comment under the question or an answer that you would like to discuss

Add Answer


Question Tools

Stats

Asked: 2021-08-08 11:00:00 +0000

Seen: 15 times

Last updated: Dec 03 '22