Configure Caching Agents

Caching agents in Spinnaker

Learn how to configure caching agents in Spinnaker™ to improve Clouddriver performance. Refer the Clouddriver Caching Agents on the Spinnaker page for detailed content on caching agents.

How does Spinnaker discover and caches infrastructure elements?

Depending on the size and complexity of your infrastructure, you may need to increase Clouddriver's CPU and memory limits, increase the number of running Clouddriver instances, or both.

The following configuration settings affect the behavior of the caching agents and can be used to adjust them depending on the size of the infrastructure being cached. If using the Spinnaker Operator, these settings are available under the key .spec.spinnakerConfig.profiles.clouddriver of SpinnakerService manifest. If using Halyard, they available in .hal//profiles/clouddriver-local.yml.

SQL global caching agents configuration

KeyDescriptionDefault Value

sql.scheduler.enabled

Enable or disable the sql scheduler

False

sql.agent.max-concurrent-agents

Indicates the maximum amount of agents to run at the same time when using the sql scheduler

100

sql.agent.disabled-agents

List of agent names to disable from running

(Blank)

sql.agent.enabled-pattern

Regex of agent names enabled for running

.*

sql.agent.release-threshold-ms

Maximum amount of time for releasing agent locks after they finish running. If this value is higher than agent’s execution cycle, the agents keep running immediately after finishing

500

sql.agent.agent-lock-acquisition-interval-seconds

How often the scheduler checks for the next batch of agents to run

1

sql.agent.poll.interval-seconds

Default time for how often to run caching agents

30

sql.agent.poll.error-interval-seconds

Default time for when to run caching agents after an execution fails

30

sql.agent.poll.timeout-seconds

Maximum time to hold a lock of an agent execution. If an agent takes longer to finish than this, it’s possible to have concurrent executions of the same agent

300

See the Configure Clouddriver to use a SQL Database page for how to configure Clouddriver.

Redis global caching agents configuration

KeyDescriptionDefault Value

redis.scheduler.enabled

Enable or disable the redis scheduler

True

redis.agent.max-concurrent-agents

Indicates the maximum amount of agents to run at the same time when using the redis scheduler

1000

redis.agent.enabled-pattern

Regex of agent names enabled for running

.*

redis.agent.agent-lock-acquisition-interval-seconds

How often the scheduler checks for the next batch of agents to run

1

redis.poll.interval-seconds

Default time for how often to run caching agents

30

redis.poll.error-interval-seconds

Default time for when to run caching agents after an execution fails

30

redis.poll.timeout-seconds

Maximum time to hold a lock of an agent execution. If an agent takes longer to finish that this, it’s possible to have concurrent executions of the same agent

300

Issues to consider:

  • Some agents may not run in some versions of Spinnaker when using the SQL scheduler, which is a known issue.

  • The max-concurrent-agents setting is directly related to how much CPU and Clouddriver memory requires to cache infrastructure. Higher values make each replica consume more resources. Higher values also make it possible for Clouddriver to reduce the time spent caching all infrastructure.

Last updated