Configure Caching Agents

Caching agents in Spinnaker

Learn how to configure caching agents in Spinnaker™ to improve Clouddriver performance. Refer the Clouddriver Caching Agents on the Spinnaker page for detailed content on caching agents.

How does Spinnaker discover and caches infrastructure elements?

Depending on the size and complexity of your infrastructure, you may need to increase Clouddriver's CPU and memory limits, increase the number of running Clouddriver instances, or both.

The following configuration settings affect the behavior of the caching agents and can be used to adjust them depending on the size of the infrastructure being cached. If using the Spinnaker Operator, these settings are available under the key .spec.spinnakerConfig.profiles.clouddriver of SpinnakerService manifest. If using Halyard, they available in .hal//profiles/clouddriver-local.yml.

SQL global caching agents configuration

Key	Description	Default Value
sql.scheduler.enabled	Enable or disable the sql scheduler	False
sql.agent.max-concurrent-agents	Indicates the maximum amount of agents to run at the same time when using the sql scheduler	100
sql.agent.disabled-agents	List of agent names to disable from running	(Blank)
sql.agent.enabled-pattern	Regex of agent names enabled for running	.*
sql.agent.release-threshold-ms	Maximum amount of time for releasing agent locks after they finish running. If this value is higher than agent’s execution cycle, the agents keep running immediately after finishing	500
sql.agent.agent-lock-acquisition-interval-seconds	How often the scheduler checks for the next batch of agents to run	1
sql.agent.poll.interval-seconds	Default time for how often to run caching agents	30
sql.agent.poll.error-interval-seconds	Default time for when to run caching agents after an execution fails	30
sql.agent.poll.timeout-seconds	Maximum time to hold a lock of an agent execution. If an agent takes longer to finish than this, it’s possible to have concurrent executions of the same agent	300

See the Configure Clouddriver to use a SQL Database page for how to configure Clouddriver.

Redis global caching agents configuration

Key	Description	Default Value
redis.scheduler.enabled	Enable or disable the redis scheduler	True
redis.agent.max-concurrent-agents	Indicates the maximum amount of agents to run at the same time when using the redis scheduler	1000
redis.agent.enabled-pattern	Regex of agent names enabled for running	.*
redis.agent.agent-lock-acquisition-interval-seconds	How often the scheduler checks for the next batch of agents to run	1
redis.poll.interval-seconds	Default time for how often to run caching agents	30
redis.poll.error-interval-seconds	Default time for when to run caching agents after an execution fails	30
redis.poll.timeout-seconds	Maximum time to hold a lock of an agent execution. If an agent takes longer to finish that this, it’s possible to have concurrent executions of the same agent	300

Issues to consider:

• Some agents may not run in some versions of Spinnaker when using the SQL scheduler, which is a known issue.

• The max-concurrent-agents setting is directly related to how much CPU and Clouddriver memory requires to cache infrastructure. Higher values make each replica consume more resources. Higher values also make it possible for Clouddriver to reduce the time spent caching all infrastructure.