Best practices to deploy Kapacitor's TICK scripts

InfluxDB 1 Kapacitor
1

Hello!

We are planning to create quite a lot of tickscripts and use them on several (dozens) installations. What are the best practices to manage and deploy in production Kapacitos’s tickscripts?

2

I would say standard configuration management solutions like Chef or Ansible can do that.

3

I am running kapacitor in a docker container, in fact one for each deployment/stack, so many instances. What I did is push them to a file store and have a companion service that polls the file store for changes. If the files change then that companion service (bash script) updates the task. I cheated a bit as I encode some stuff in comments like the task type i.e. batch or stream.

Nice thing is that on startup of the container, it pulls the tasks and away it goes. No chef/puppet etc. I can terminate containers, they can crash and they will be rescheduled to run somewhere else.

Works really well for me.

I am using an S3 bucket as the storage.

#!/bin/bash
exec 2>&1
NEWLINE='
'
S3_BUCKET="${TASK_S3_BUCKET:-s3:/foobar}"
TASKS_DIR="/opt/tasks"
POLL_PERIOD=30
KAPACITOR_PORT=9092

kapacitor_url="http://localhost:${KAPACITOR_PORT}"
[[ "${KAPACITOR_HTTPS_ENABLED}" == "true" ]] && kapacitor_url="https://localhost:${KAPACITOR_PORT}"

function update_tasks {
	add_task_regex="^[+~] (.*)$"
	del_task_regex="^[-] (.*)$"
	while read line; do
			if [[ "${line}" =~ ${add_task_regex} ]]; then
				tickscript="${BASH_REMATCH[1]}"
				taskname=${BASH_REMATCH[1]##*/}
				taskname=${taskname%%.tick}
				echo "Adding/Updating Task '${taskname}' using '${tickscript}'"

				# Pull dbrp and tasktype from config
				dbrp=$(grep 'DBRP:.*' < "${tickscript}")
				dbrp=${dbrp#*:}
				dbrp=${dbrp%% *}
				[[ -z "${dbrp}" ]] && dbrp="ion.default"

				task_type=$(grep 'Type:.*' < "${tickscript}")
				task_type=${task_type#*:}
				task_type=${task_type%% *}
				[[ -z "${task_type}" ]] && task_type="stream"

				kapacitor -skipVerify -url "${kapacitor_url}" define "${taskname}" -type "${task_type}" -tick "${tickscript}" -dbrp "${dbrp}"
				# Reload or enable the task
				kapacitor -skipVerify -url "${kapacitor_url}" show "${taskname}" | grep -q 'Status:' | grep -q 'disabled'
				if [[ ${?} == 0 ]]; then
					kapacitor -skipVerify -url "${kapacitor_url}" enable "${taskname}"
				else
					kapacitor -skipVerify -url "${kapacitor_url}" reload "${taskname}"
				fi

				# Disable the task if config overrides
				grep -q 'Disabled:True' < "${tickscript}"
				if [[ $? == 0 ]]; then
					echo "Disabling Task '${taskname}'"
					kapacitor -skipVerify -url "${kapacitor_url}" disable "${taskname}"
				fi
			fi
			if [[ "${line}" =~ ${del_task_regex} ]]; then
				taskname=${BASH_REMATCH[1]##*/}
				taskname=${taskname%%.tick}
				echo "Removing Task '${taskname}'"
				kapacitor -skipVerify -url "${kapacitor_url}" delete tasks "${taskname}"
			fi

	done <<< "${1}"
}

function sync_tasks {
	processed_changes=""
	current_tasks=$(ls ${TASKS_DIR})

	download_regex="^download: (.*) to (.*)$"
	delete_regex="^delete: (.*)$"

	changes=$(aws s3 sync --delete "$S3_BUCKET" "$TASKS_DIR" | tr '\r' '\n')
	if [[ $? != 0 ]]; then
		echo "${changes}"
		return 1
	fi
	# Process the changes to determine if this was update or deletion
	while read line; do
		if [[ "${line}" =~ ${download_regex} ]]; then
			if [[ -e "/${BASH_REMATCH[2]}" ]]; then
				processed_changes+="~ /${BASH_REMATCH[2]}${NEWLINE}"
			else
				processed_changes+="+ /${BASH_REMATCH[2]}${NEWLINE}"
			fi
		fi
		[[ "${line}" =~ ${delete_regex} ]] && processed_changes+="- /${BASH_REMATCH[1]}${NEWLINE}"
	done <<< "${changes}"
	echo "${processed_changes}"
	return 0
}


## Main
[[ ! -d "${TASKS_DIR}" ]] && mkdir "${TASKS_DIR}"

while [ true ]; do

	# Sync and get changes.
	changed_tasks=$(sync_tasks)
	rc=${?}

	if [[ -n "${changed_tasks}" ]]; then
		echo "Changes: "
		echo "${changed_tasks}"
	else
		echo "No Changes"
	fi

	#Update the tasks
	if [[ "${rc}" == 0 && -n "${changed_tasks}" ]]; then
		update_tasks "$changed_tasks"
	fi

	sleep "${POLL_PERIOD}"
done
2 Likes
4

Hey!

@sigterm you may want to take a look at kapacitor-unit (GitHub - gpestana/kapacitor-unit: Testing framework for Kapacitor TICKscripts) . It’s a framework for automating TICK script tests and helps you to keep the tasks behaving as expected when things change (data, tick scripts, etc…)

Gonçalo