Introduction
I have written a few posts about different aspects of Azure Data Factory. I use it as the main workhorse of my data integration and ETL projects. One major drawback I have found with Azure Data Factory is the scheduling system, it’s not as flexible as I and many others would like it to be. With that being said there are certainly ways to adapt and get more control of an Azure Data Factory pipeline execution. In my post Starting an Azure Data Factory Pipeline from C# .Net, I outline the need to kick off a pipeline after a local job has completed and how this can be attained by utilizing the SDK to programmatically set the pipelines Start/End dates. You may not have that requirement specifically, but let’s say you want to only run a pipeline during the weekday or another specific schedule, this can be accomplished by utilizing the same code from my prior post and scheduling a local console app. However, I thought it would be more fun to utilize Azure Functions to kick off a pipeline on a weekday schedule to provide a fully cloud based solution. Continue reading “Running an Azure Data Factory Pipeline on a Weekday Schedule Using an Azure Function”