By Brendan Woodell
In a recent project, our team decided that we wanted the ability to reconfigure the application settings of a running service without having to restart it. This isn’t a particularly special case, but it was new for us.
As with most of our C# applications, we provide configuration files that our code deserializes into a C# class to control how the application behaves. For example, a configuration setting might inform the application of how often to run a process on a timer. If we wanted to change this config value from a 60 minute interval to a 120 minute interval we would have to stop the service, edit the configuration file, and restart the service. Instead we want to simply edit the configuration without stopping the service, monitor the log for a log message confirming the change has taken affect, then go about our day with no interruption.
We initally turned to the IOptionsMonitor interface to detect specific changes and pass them to the appropriate places. For example, say there is a section of a config.json file we care about specifically.
{
"TimerConfig": {
"IntervalInMinutes": 60
}
}Configure the Worker host to inject this configuration with our other services:
public static IHostBuilder CreateHostBuilder(string[] args) =>
Host.CreateDefaultBuilder(args)
.UseSystemd()
.ConfigureServices((hostContext, services) =>
{
services.Configure<TimerConfiguration>(hostContext.Configuration.GetSection("TimerConfig"));
services.AddHostedService<Worker>();
});Now apply the IOptionsMonitor to this configuration in the Worker class and set up a Provider class to pass references of the latest values where we need them:
static MyScheduledProcessHandler _processHandler;
private TimerConfigurationProvider _timerConfigProvider { get; set; }
public Worker(IOptionsMonitor<TimerConfiguration> appConfig, IConfiguration configuration)
{
_logger = StartLogging();
_configuration = configuration;
StartTimerConfigurationProvider(appConfig.CurrentValue);
appConfig.OnChangeDelayed(TimerConfigurationListener);
}
//Create whatever Process Handler I'm using on a timer
protected override async Task ExecuteAsync(CancellationToken cancellationToken)
{
_processHandler = new MyScheduledProcessHandler(_timerConfigProvider, _logger);
}
//Validate and Set initial Timer Config provider
private void StartTimerConfigurationProvider(TimerConfiguration appConfig)
{
_logger.LogDebug("Beginning Config Validation");
var errors = appConfig.Validate();
if (errors.Count > 0)
{
foreach (var error in errors)
{
_logger.LogError(error);
}
throw new AggregateException(TimerConfiguration.ValidationFailed, errors.Select(x => new Exception(x)));
}
else
{
_logger.LogDebug("Config Validation Succeeded");
_timerConfigProvider = new TimerConfigurationProvider(appConfig, _logger);
}
}
//Validate and refresh new config values in the Timer Config provider
private void TimerConfigurationListener(TimerConfiguration appConfig)
{
_logger.LogDebug("Config Update Detected");
var errors = appConfig.Validate();
if (errors.Count > 0)
{
foreach (var error in errors)
{
_logger.LogError(error);
}
_logger.LogError(TimerConfiguration.ConfigUpdateFailed);
}
else
{
_logger.LogDebug("Config Validation Succeeded");
_timerConfigProvider.RefreshSettings(appConfig);
}
}The Provider class contains private properties that are only changed via RefreshSettings(), and public properties that safely read them. This makes it easy to pass around and trust that the values I read are the most accurate and up-to-date. Upon changing TimerConfig.IntervalInMinutes from 60 to 120, I can see that setting validated, applied, and used going forward in the logs.
Great! Problem solved right?
Not quite…
Can we trust the users?
The obvious answer is No. While we expect the users of our application to be Power Users and understand how configuration works, we cannot expect them to never make a mistake. Let’s say that a user comes along and decides that TimerConfig.IntervalInMinutes is best set to the value of "Thirty". That’s not great. Our TimerConfiguration is specifically looking for a integer when it deserializes. What happens to the application?
Fortunately, and unfortunately, nothing at all. The IOptionsMonitor does not recognize the changed object as valid, and therefore does not attempt to refresh the settings. No log message is written, no changes are made, the application keeps chugging along with the original settings. That may be acceptable in some cases, quietly protecting the users from themselves is a valid design choice. However, in this case, we wanted to alert the user to their mistake and give them the tools to fix it without assistance. IOptionsMonitor will not suffice in this case, we need to dive a little deeper into the chain-of-events that happens once a change to the config.json is saved.
Enter the FileSystemWatcher
Rather than waiting for our application to deserialize changes into something consumable, we can try to detect changes to a file and write our own code to validate and handle updates from there. This doesn’t require much change, as System.IO.FileSystemWatcher is easy to substitute in:
...
private FileSystemWatcher _fileSystemWatcher;
public Worker(IConfiguration configuration)
{
_logger = StartLogging();
_configuration = configuration;
var timerConfig = _configuration.GetSection("TimerConfig").Get<TimerConfiguration>();
var path = Path.GetDirectoryName(Assembly.GetExecutingAssembly().Location);
_logger.LogDebug($"Config FileWatcher folder Path: {path}");
_fileSystemWatcher = new FileSystemWatcher(path, ""config.json");
_fileSystemWatcher.Changed += OnChanged;
_fileSystemWatcher.Deleted += OnDeleted;
_fileSystemWatcher.EnableRaisingEvents = true;
StartTimerConfigurationProvider(timerConfig);
}
...
//no necessary changes here
private void StartTimerConfigurationProvider(TimerConfiguration appConfig)
protected override async Task ExecuteAsync(CancellationToken cancellationToken)
...
private void OnChanged(object sender, FileSystemEventArgs e)
{
try
{
_fileSystemWatcher.EnableRaisingEvents = false;
if (e.ChangeType != WatcherChangeTypes.Changed)
{
return;
}
_logger.LogDebug("Config Update Detected");
var timerConfiguration = new ConfigurationBuilder().SetBasePath(Path.GetDirectoryName(Assembly.GetExecutingAssembly().Location)).AddJsonFile("config.json").Build().GetSection("TimerConfig").Get<TimerConfiguration>();
var errors = timerConfiguration.Validate();
if (errors.Count > 0)
{
foreach (var error in errors)
{
_logger.LogError(error);
}
_logger.LogError(TimerConfiguration.ConfigUpdateFailed);
}
else
{
_logger.LogDebug("Config Validation Succeeded");
_timerConfigProvider.RefreshSettings(timerConfiguration);
}
}
catch(Exception ex)
{
_logger.LogError($"Config Update Failed with Error '{ex.Message}'");
}
finally
{
_fileSystemWatcher.EnableRaisingEvents = true;
}
}
//Handle the file being deleted
private void OnDeleted(object sender, FileSystemEventArgs e)
{
_logger.LogError("Config Update Detected, Config file DELETED");
}
...
Great! Now if a user changes the TimerConfig to something that is invalid our OnChanged function will fall into the catch block. When the user inspects the logs to find out why their change hasn’t taken effect, the problem will be obvious, and hopefully the solution will to. That saves a call to the On-Call phone and makes everyone happier for it. The Pros and Cons list between the two solutions is small. They both get you to the same place, but one requires a bit more hand-holding in exhange for greater customization and safety. Which one you choose will simply depend on the use-case and what level of support is necessary.