The Core Technologies Blog

Our Software // Windows Services // 24×7 Operation


How to Survive Automatic Updates when Running 24/7 with AlwaysUp

How to Survive Automatic Updates when Running 24/7

Automatic software updates: An overview

For Windows users, software updates are a regular part of life. Vendors like Microsoft, Adobe and Oracle are constantly tweaking their software, which then inundate us with “helpful” nudges. This prompt from printer manufacturer Brother International is a prime example:

Update Brother printer software

Most of the time, you have to install software updates manually. That is, you must take deliberate action to approve and apply the upgrade. For example, in the screenshot above, the new printer driver will not be installed if we never click the “Update” button.

But while manual updates remain the norm, an increasing number of vendors refuse to be held hostage by reluctant users. They choose to deploy software updates automatically, on their own schedule, without securing explicit consent. Products like Dropbox, OneDrive, and Box Drive are members of that new “vendor knows best” group. All apply updates whenever they like.

Yet despite the faint whiff of authoritarianism, automatically installing software has many benefits. To begin with, busy users don’t need to be remember to check for updates or be interrupted to install new packages.

And most important of all, automatic updates have the power to stamp out critical vulnerabilities — quickly and efficiently.

However, those positives are only appreciated when the updated application works as expected, with no surprises. From our own experience, there is nothing more infuriating than software that breaks itself without our consent! 😡

The problem with automatic updates in a 24/7 environment

24/7 Operation

For mission-critical applications that must operate continuously, updates introduce significant risk. Each change to the application (or the supporting operating system) has the potential to cripple the software — and send blood pressures skyrocketing.

The best practice is to update systems in a controlled manner. Experts agree that it’s safest to make changes during a scheduled maintenance window, when customers are aware that there may be a disruption of service.

After applying updates, staff test and certify the upgraded system. If the team discovers a formidable problem, they roll the system back to the pre-update state to resume service.

Needless to say, automatic updates are the antithesis to a controlled upgrade process.

Indeed, automatic software updates will leave you wondering:

  • For how long will the software be unavailable during the update?

  • What happens if the upgrade fails?

  • How quickly will you be able to restore service if the new software doesn’t work?

Those are very difficult questions to answer when the upgrade process is out of your control.

Running 24/7 with AlwaysUp is at odds with automatic updates

Windows Service trouble

If you’re using AlwaysUp to run an application 24/7, you will likely run into trouble if that application insists on randomly updating itself.

To explain why, let’s examine how most automatic software updates work (without AlwaysUp involved).

Once per day:

  1. The application:

    1. Checks online and realizes that an update is required

    2. Downloads the latest package from the servers

    3. Launches a “helper” application to perform the upgrade

    4. Exits (because integral components cannot be updated while the application is running)

  2. The helper:

    1. Upgrades the application, adding and replacing executable files as necessary

    2. Restarts the application

    3. Cleans up and exits

Unless interrupted, the new version of the application continues to run until the next update is detected and the process above repeats.

Unfortunately the update process can go off the rails when AlwaysUp is ensuring that your application is running 24/7.

Specifically, the trouble starts when the application exits (step 1d). In response, AlwaysUp springs into action and quickly re-launches the application.

And when the helper tries to apply the upgrades (step 2a), it cannot update the executable files because they are in use by the (once again) running application. Ultimately, the helper fails, potentially leaving the application’s components in an inconsistent state. That could spell disaster for a business-critical application!

In fact, one of our customers recently experienced this peculiar predicament when running Dropbox as a Windows Service with AlwaysUp. After a failed auto-update, Dropbox simply refused to copy files to and from the cloud. Fortunately, a technician resumed normal operation by rebooting the server — but only after the customer’s pipeline had been idle for several, costly hours. It was not a good day. 🙁

The best solution: Turn off automatic updates

No automatic updates

Naturally, the best solution for business-critical applications is to avoid automatic software upgrades entirely.

And if you’re lucky, your application provides a simple mechanism to disable auto-updates.

For instance, you can easily turn off automatic update checks for the Java Virtual Machine from the Java Control Panel:

Turn off Java check for updates

For other applications, you may have to cripple (or remove) one or more scheduled tasks that periodically check for updates.

In particular, disabling the OneDrive Standalone Update Task will prevent OneDrive from trying to download and install the latest changes each day:

Disable OneDrive Standalone Update Task

If you can’t disable automatic updates, configure AlwaysUp to tolerate them

To avoid drama with mandatory auto-updates, please make the following two changes to your AlwaysUp configuration:

1. Introduce a delay before AlwaysUp restarts your application

By default, AlwaysUp restarts your application immediately after it stops. But as we have pointed out, that rapid re-launch can cause headaches for the update process.

To get around that problem, instruct AlwaysUp to restart your application after waiting for a few minutes. Doing so should allow the update process to finish.

To make that change:

  1. Edit your application in AlwaysUp.

  2. Switch to the Restart tab.

  3. Check the Not immediately and After boxes and specify a suitable delay.
    Five minutes should be enough for most automatic upgrades to complete:

Delay application restarts in AlwaysUp

Note however — this delay applies to all situations, not just for auto-upgrades. That is, if your application exits or crashes for any reason, AlwaysUp will pause before restarting it. Hopefully that behavior is acceptable in your environment.

2. Empower AlwaysUp to retake control after an automatic upgrade

After AlwaysUp waits for the update to complete, it will launch a new copy of the application. But that launch can fail if the application is already running.

To remove that potential hazard, we must instruct AlwaysUp to terminate all other copies of the application prior to starting its own copy.

To do so:

  1. Edit your application in AlwaysUp.

  2. Switch to the Startup tab.

  3. Check the Stop all copies of the application running on this computer and Also whenever the application is restarted boxes:

Stop all application instances in AlwaysUp

This adjustment will allow AlwaysUp to “take control” of your application after the upgrade concludes.

Best of luck with your 24/7 environment! 😀

Posted in AlwaysUp | Tagged , , , | Leave a comment

Essential Windows Services: EventLog / Windows Event Log

EventLog Service

What is the Windows Event Log (EventLog) service?

The EventLog service manages event logs — repositories of events generated by services, scheduled tasks and applications working closely with the Windows operating system.

The service’s display name is Windows Event Log and it runs inside the service host process, svchost.exe. By default, the service is set to start automatically when your computer boots:

EventLog Windows Service

You can use the Windows Event Viewer to browse the event logs managed by the service. For example, here are some of the records captured in the Windows Security event log:

Event Viewer: Windows Security log

What happens if I stop EventLog?

You may find it virtually impossible to stop the Windows Event Log service.

That’s because the service supports several important system services. You can see that list on the service’s Dependencies tab:

Windows Event Log Dependencies

And because of those dependency relationships, attempting to stop EventLog triggers a “cascade” that causes all dependent services to stop too. Here you can see Windows alerting us of that situation:

EventLog service: Stop dependents

But after we clicked “Yes”, Windows failed to stop EventLog and the dependent services! A peculiar error was returned:

Error stopping the EventLog service

We tracked the issue to “Network List Service” (netprofm). That service refused every attempt to stop it, consistently failing with the error above. And since we could not stop “Network List Service”, we could not stop EventLog either.

Is it OK to disable the Windows Event Log service?

No — it’s not safe to disable the Windows Event Log service.

Indeed, in the very description of the service, Microsoft warns:

Stopping this service may compromise security and reliability of the system.

That advice makes sense because EventLog provides essential support for Windows Services, scheduled tasks, and other background programs. Those components typically run “headless”, without a user interface, and rely on the event logs to record important events.

If the EventLog service stops, those background components will have no way to chronicle their activities. There would be an ominous gap in the operating system’s low-level records.

With that in mind, it’s easy to see why the EventLog service is an alluring target for attackers looking to compromise a system. Once the service has been crippled, vital forensics records may not be captured and intruders could operate with impunity.

Questions? Problems?

If you would like to know more about the Windows Event Log service, or you have a specific problem, please feel free to get in touch. We will do our best to help you!

Posted in Windows Services | Tagged , , , | Leave a comment

Q&A: How do I Change the Name of my Windows Service?

Q&A: How do I Change the Name of my Windows Service
We manage a third-party package that installs a Windows Service with a generic name. I keep having to explain what it means to my team/customers, which can be a pain. How do I change the name of the service to include our company name?

— Garth P

Hi Garth.

A service actually has two names — the “Service Name” and the “Display Name”.

Both names are prominently displayed in the Services utility:

Windows Service Names

The Service Name

The Service Name is the unique identifier of the service. As a result, no two services can have the same Service Name (even with differing case).

And Windows restricts the Service Name in a couple other ways too, specifically:

  1. Its maximum length is 256 characters.

  2. Forward-slash (/) and back-slash (\) characters are not permitted.

So the Service Name is the “real” name of the Windows Service.

All command line programs — including the built-in NET and SC utilities — will accept the Service Name as an identifier of the service. And the Service Name identifies the service in the Windows Registry (in < HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services).

However, because Service Names are used in command line operations, they tend to be terse, technical and downright cryptic. To illustrate, here the Service Names of five important services installed on Windows 10:

  1. wuauserv

  2. W32Time

  3. ProfSvc

  4. Wlansvc

  5. lmhosts

Only the Windows OS geeks among us will be able to guess what those services do! 🙂

The Display Name

To counteract the caginess of the Service Name, the Display Name records a “friendlier” value, mainly for user interface programs to identify the service. For example, here are the corresponding Display Names of the services listed above:

  1. Windows Update

  2. Windows Time

  3. User Profile Service

  4. WLAN AutoConfig

  5. TCP/IP NetBIOS Helper

More meaningful, right? When picking from a list, it’s almost always better to show the Display Name instead of the Service Name. That’s what the Services utility does.

Note that the length of the Display Name is also limited to 256 characters but all characters (including forward-slash and back-slash) are allowed.

And perhaps surprisingly, there is no issue with two services having the same Display Name. However, that can be troublesome for lists that rely on Display Name, as we see here with Services:

Two service with the same Display Name

So which name would you like to change? The Service Name or the Display Name? Or both?

How to change the Service Name

Unfortunately Microsoft has not provided a great way (i.e. an API) for changing the Service Name. No operating system tools will allow you to change that value — not SC, Services or any other application.

However it is possible to adjust the Service Name — provided that you don’t mind working with the Windows Registry.

To change the Service Name:

  1. Open the Registry Editor.

    To do so, press the Windows + R keys to open the Run window, and then type regedit.

  2. On the left, navigate to the section where Windows Services are recorded:

    HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\

    Open the Services key to reveal all the services installed on your computer. The list is sorted alphabetically by Service Name:

    Registry Editor: Services key
  3. Find and highlight your service in the list.

  4. Right-click the entry and choose Rename:

    Registry Editor: Rename the service
  5. Enter the new name. Remember: limit the length to 256 characters; forward-slashes and back-slashes are not allowed.

  6. Reboot your computer. The new Service Name will not take effect until Windows restarts.

How to change the Display Name

Here are a three ways to change a service’s Display Name.

1. Using the SC command

Run this command from an elevated command prompt:

sc config <SERVICE-NAME> DisplayName= "Your new display name"

For example, we have updated the Display Name of the Spooler service:

Spooler service Display name updated

2. With PowerShell Set-Service

If you prefer PowerShell, the Set-Service cmdlet will do the trick:

Set-Service -Name <SERVICE-NAME> -DisplayName "Your new display name"

As you can see, PowerShell works just as smoothly as SC:

Spooler service Display name updated with PowerShell

However, be sure to run from a PS prompt with administrative rights, to avoid the dreaded “access denied” error.

3. Updating the service’s registry key

Finally, you can roll up your sleeves and modify the registry value directly.

DisplayName is a REG_SZ (string) value contained in the service’s registry key:

Registry: Spooler service Display Name

Double-click the “DisplayName” to set a new value:

Edit DisplayName

The new value will take effect after a reboot.

Best of luck with your customers!

Posted in Windows Services | Tagged , , , , , , , | Leave a comment

Q&A: How do I Restart a Windows Service when an Event is Reported?

Q&A: How do I Restart a Windows Service when an Event is Logged?
  We have a service that should restart whenever another service writes a particular event to the Event Viewer. How can I do that?

— Russell

Hi Russell.

To create that workflow, you can introduce an “Event Trigger” with the Windows Task Scheduler. However, that will only work if your triggering event is well defined, with a unique, non-zero identifier.

For example, event 1026 — reported when a .NET application crashes — is an excellent candidate:

Event 1026: .NET Runtime Error

However, you are out of luck if your entry has a generic ID or is 0, as pictured here:

Events with ID 0

Unfortunately there is no good way to create a trigger based on a poorly defined event.

How to setup an “Event Trigger” Task that restarts your Windows Service

Assuming that your event has a unique ID, here’s the step-by-step process to recycle your service when the event arrives:

  1. First, create a batch file that restarts your service.

    The file should contain two commands — one to stop your service and another to start it again, like this:

    NET STOP <Service-Name>
    NET START <Service-Name>

    <Service-Name> is the name of your Windows Service.

    For example, here is the batch file we created to recycle the Print Spooler service:

    Restart service batch file
  2. Launch the Windows Event Viewer.

    To do so, press the Windows + R keys to open the Run window, and then type eventvwr.msc.

  3. In the Event Viewer, navigate to find and highlight an instance of the entry that should trigger your service to restart.

    To illustrate, we’ll work with event 1026 in the Application log:

    Event Viewer: Application event 1026
  4. Right-click the event’s row and select Attach Task To This Event:

    Attach a task to the event

    That will launch the Create Basic Task Wizard, which is part of the Task Scheduler.

  5. In the wizard, give your task a friendly name. We’re called ours “Restart Service When Event 1026 Occurs”:

    Create event trigger task: Name

    Click Next to continue.

  6. Confirm the details of your trigger and click Next to move on:

    Create event trigger task: Event details
  7. Choose Start a program and click Next to continue:

    Create event trigger task: Start a program
  8. Enter the full path to the batch file you created in step 1.

    Create event trigger task: Enter batch file

    Click Next once done.

  9. Review the summary and make sure everything looks good. This is your last chance to go back and make changes before creating the task.

    Check the Open the Properties dialog for this task when I click Finish box. We’ll need to tweak a few settings to ensure that your service restarts smoothly from the batch file.

    Create event trigger task: Summary
  10. In the Properties window, affirm a couple of options:

    Run whether user is logged on or not. Otherwise, your service will only restart when you’re logged on.

    Run with highest privileges. Necessary because restarting your service likely requires administrative rights.

    Create event trigger task: Properties

    Click OK to continue.

  11. As a security precaution, please enter your password:

    Create event trigger task: Password
  12. And finally, the Event Viewer confirms that it has created your new task:

    Create event trigger task: Confirmed

    Click OK to wrap up.

That’s it!

The next time the event enters the Windows Event Logs, the Task Scheduler will run the batch file, which will restart your service.

Managing the new “Event Trigger” Task

The new task will reside in the Event Viewer Tasks section of the Task Scheduler:

Task Scheduler: New Event Viewer task

You can make adjustments to the task there.

One final Task Scheduler tip: If you see a link labeled Enable All Tasks History in the Actions section on the right, be sure to click it:

Task Scheduler: Enable All Tasks History

That setting tells Task Scheduler to keep a record every time a task runs. And with that tracking in place, after your event has been reported you can check the task’s History tab to confirm that the task ran and your batch file executed successfully.

Bonus #1: Send an email alert when the service restarts

Would you like to be notified whenever the event appears and the service restarts?

If so, you should update your script to call a command line tool that will deliver an email message.

We recommend using a PowerShell script, or SwithMail, as described in this article on sending email without Outlook.

Bonus #2: Use ServicePilot to restart a “slow” service that takes a while to stop

The NET command can run into trouble if your service takes more than 30 seconds to stop. In that scenario, NET STOP gives up and the subsequent NET START will fail because the service isn’t in the right state. The end result is a dead service!

In that case, we recommend using our free ServicePilot utility instead of the NET command.

For example, here is our script that replaces NET with ServicePilot:

Restart the service with ServicePilot

Notice the option to give the service up to 120 seconds to stop. You should specify a duration that works for your service.

All the best!

Posted in Windows Services | Tagged , , , , | Leave a comment

Q&A: Why Doesn’t AlwaysUp Restart my Application?

Q&A: Why Doesn't AlwaysUp Restart my Application?
  My company uses AlwaysUp to run 4 applications on our lab server. Every few weeks, one of the applications stops running and I have to log in and restart it. Why doesn’t AlwaysUp automatically restart it?

Here is what I see in the logs:

AlwaysUp not restarting: Activity

— Sandy

Hi Sandy, thanks for reaching out. The whole point of using AlwaysUp is to keep your application running 24/7/365 so you are right to be puzzled!

Before digging into the details, let’s review the basics of AlwaysUp.

How AlwaysUp works (a quick summary)

When you configure an application in AlwaysUp, a true Windows Service is created. Let’s call it the AlwaysUp service.

For example, if you setup Dropbox in AlwaysUp, you will see an AlwaysUp service called “Dropbox (managed by AlwaysUpService)” in the Services utility:

Dropbox (managed by AlwaysUpService)

When your computer boots, the AlwaysUp service:

  1. Starts before anyone logs in

  2. Launches your application

  3. Constantly monitors your application, quickly restarting it if it crashes or dies for any reason.

The AlwaysUp service is the key to running your application 24×7.

(If you are curious to find out more about the inner workings of AlwaysUp, check out our detailed How AlwaysUp Works explainer.)

What your logs tell us

The activity report shows AlwaysUp running your application continuously for a few days. Then at 7:03:11 AM, your application suddenly stops. And soon after, the AlwaysUp service stops as well.

Your application is not crashing or failing in any way. No errors or warnings were reported and nothing out of the ordinary occurred.

From that benign sequence, we can conclude that someone (or something) intentionally stopped your AlwaysUp service.

And with the service stopped, the only way to restart your application is to restart the AlwaysUp service.

Who stopped your AlwaysUp Windows Service?

Unfortunately the logs don’t say who (or what) stopped the service.

However, there are at least 4 ways that the service could have been stopped:

  1. From the Services application.

    For example, clicking the Stop button will terminate the AlwaysUp Dropbox service:

    Stop the service
  2. With the NET STOP command.

  3. With the SC STOP command.

  4. From an application leveraging the Windows Services API.

Do you know who would stop the service?

Are there any batch files that manipulate services?

Watch out for maintenance scripts and other background tasks.

Investigate with Windows Service Auditor

If it’s time to put your detective hat on and figure out who’s stopping the service, our free Windows Service Auditor tool should be able to help. It will record all operations performed on the AlwaysUp service.

After downloading and starting Windows Service Auditor, enable extended auditing for the AlwaysUp service. Periodically review the service’s security events, watching for “Stop the service” operations, like this:

Windows Service Auditor: Stop event

Good hunting!

Keep the service running with Service Protector

If you are unable to prevent the service from shutting down, consider deploying Service Protector as a second layer of defense.

Install Service Protector, select Protector > Add and choose your AlwaysUp service from the list:

Select your AlwaysUp service

After you save, Service Protector will babysit the AlwaysUp service — to quickly restart it if someone stops it:

Service Protector protecting the AlwaysUp service

Hopefully you will get to the bottom of this unwelcome behavior soon. Please be sure to get in touch if you notice any errors or warnings in the activity reports.

Posted in AlwaysUp | Tagged , , , , , | Leave a comment