The Core Technologies Blog

Our Software // Windows Services // 24×7 Operation


Q&A: Why doesn’t Srvany Close my Application when I Stop the Service?

Q&A: Why doesn't Srvany close my Java application when I stop the service?
  We use Srvany to run our Java application as a service. It starts fine but when we stop the service our Java application does not close. We have to kill the java.exe process in Task Manager. That’s not supposed to happen, right? Is there a registry setting that we are missing that will shut down our application properly when we stop the service?

— Angela

Hi Angela.

As you probably know, Srvany is the original service wrapper. Its job is to accept commands from the Windows Service Control Manager (SCM) and take appropriate action on your java application. Simple, right?

Well, maybe not. Let’s examine what Microsoft’s service wrapper does when it receives the most important SCM commands — “start service” and “stop service”.

How Srvany handles the “Start Service” command

When you start your service — from the Services application, NET START, or SC START — the SCM immediately launches a fresh instance of srvany.exe and notifies it of the start request.

In response, Srvany:

  1. Informs the SCM that the service is starting.

  2. Starts the program configured to run as a service. The full command line is read from this registry value:

    HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\<Your Service>\Parameters\Application

If Srvany fails to start the application (which will happen if the application/path doesn’t exist), Srvany will:

  1. Inform the SCM that the service has stopped.

  2. Exit, ending the srvany.exe process.

On the other hand, if it successfully launches the application, Srvany will:

  1. Inform the SCM that the service is now running.

  2. Continue to run, listening for subsequent commands from the SCM.

Testing service start with Notepad

To confirm this behavior, we installed a new service (with Instsrv) and configured it to run the Windows Notepad text editor:

Notepad Windows Service installed with Srvany

We started the service.

With the help of Microsoft’s excellent Process Explorer, here is what the process tree looked like after a few seconds. As expected, there was a srvany.exe process that had spawned a notepad.exe child process:

Srvany running Notepad as a service

And Notepad was happily running in the background, on the isolated Session 0 desktop.

How Srvany handles the “Stop Service” command

When you attempt to stop your service — from the Services application, NET STOP, or SC STOP — the SCM immediately notifies the associated Srvany process of the stop request.

In response, Srvany will:

  1. Inform the SCM that the service is stopping.

  2. Close the process/application that it started.

  3. Inform the SCM that the service is stopped.

  4. Exit, ending the srvany.exe process.

When we stopped the Notepad service, Notepad.exe was terminated as expected.

But what happens when running a Java application/service?

Since stopping your Java application didn’t go smoothly, we decided to dig into that specific scenario.

We installed a new service and configured it to launch a Java JAR package:

Java/JAR Windows Service installed

When we started the service, we saw srvany.exe launch java.exe. No surprises there:

Srvany running Java as a service

And when we stopped the service, the java.exe process ended and Srvany exited — all good.

So how come it isn’t working for you?

What about a Java application started from a batch file?

After some head scratching, we realized something important. Many of our customers running Java as a service with AlwaysUp don’t run java directly. Instead, they start java via a batch file because it gives them the opportunity to set important environment variables in advance. Could that be an issue?

To answer that question, we created a simple batch file that launched java and installed a new service to run the batch file:

Batch File Windows Service installed

We started the service. Srvany launched the batch file (cmd.exe), which in turn launched Java — all as intended:

Srvany running the batch file (and Java) as a service

However, when tried to stop the service, something unexpected happened. The service stopped and srvany.exe and cmd.exe closed, but java.exe did not exit! The Java process remained running, even after the service had transitioned to the stopped state. It was exactly as you described.

So from these tests, it seems that Srvany will terminate the process it launched (i.e. its direct child process) but will not terminate any descendant processes.

Do you think this is what you are experiencing? If so, please read on for a couple of potential solutions.

Solution #1: Run the Java executable directly from Srvany

Instead of starting Java from a batch file, let Srvany run your Java.exe command line itself. As we have shown above, Srvany is able to terminate Java when it launches it directly.

However, this option may be impractical if your batch file performs lots of setup. But if the batch file focuses on setting environment variables (e.g. CLASSPATH), you can get around that by:

  1. Permanently setting the environment variables in a specific user account, and

  2. Running Java in that account (by specifying the user’s details on the service’s Log On tab).

Solution #2: Install your Java application as a service with AlwaysUp instead of Srvany

Alternatively, if this is a professional setting and a commercial option is acceptable, you can replace Srvany with our AlwaysUp utility.

When you stop a service created by AlwaysUp, all descendant processes are terminated. That is, AlwaysUp will close cmd.exe, java.exe — and any other processes that your Java application spawns. You will never have a situation where your service is stopped but some processes remain alive.

Please review the benefits to using AlwaysUp instead of Srvany to see if you should make the switch.

Best of luck managing your Java application!

Posted in Srvany | Tagged , , , , , | Leave a comment

Service Protector 7.0: Informative Email Alerts, Sanity Check Options and More

What's new in Service Protector 7.0

Are you responsible for a temperamental Windows Service? If so, you should definitely check out the latest version of Service Protector — the easiest way to achieve 100% uptime today.

Here’s what’s new in this release:

Email alerts include recent activity

Customers who have configured email alerts will notice that messages now contain the service’s last five events from the Windows Event Logs. The idea is to provide helpful context when something unusual happens, to avoid you having to log on to interrogate Service Protector’s reports.

Here is what an email with the new Recent Activity section looks like:

Service Protector email alerts include recent activity

Delay the initial Sanity Check when detecting service problems

A customized sanity check is an excellent way to extend failure detection and automatically restart a faltering service. With a sanity check, you can probe network connectivity, check for a “stale” output file, and much more — whatever you like!

Service Protector version 7 allows you to delay the first sanity check. This is useful when your service takes a while to get ready — either at boot or after it has been restarted.

The new delay settings appear on the Configure Sanity Check window:

Service Protector: Configure Sanity Check

Full compatibility with Windows 10 20H2

Microsoft published Windows 10, version 20H2 in October 2020.

From the release notes, 20H2 doesn’t include significant changes to the Windows Services infrastructure. The update focused mostly on end-user improvements for the Edge browser, task tray notifications and the like.

Nevertheless, our team tested Service Protector 7.0 extensively on the new version of Windows 10. We’re pleased to report that no problems were detected and Service Protector remains fully compatible with all versions of Windows 10.

Service Protector is compatible with Windows 10 20H2

As usual, please review the release notes for the full list of features, fixes and improvements included in this release.

Upgrading to Service Protector 7

If you purchased Service Protector version 6 (after February 2019) you can upgrade to version 7 for free. Simply download and install “over the top” to preserve your existing services and all settings. Your registration code will continue to work.

If you bought version 5 or earlier (before February 2019), you will need to upgrade to use version 7. Please purchase upgrades here — at a 50% discount.

See the full upgrade policy for additional details.

Enjoy!

Posted in Service Protector | Tagged , , , , | Leave a comment

Q&A: Does AlwaysUp work in High Availability Failover Clusters?

Does AlwaysUp work with High Availability Failover Clustering?
My team inherited a cluster of 4 Windows Server 2012 R2 machines running a legacy finance application. At a time only one instance of the application should be up and running as the active/primary instance.

The application is cluster-unaware so we set it up with the Generic Application type. Most days it works fine but about once or twice per month the EXE stops responding and no failover happens. Someone has to log in and kill it to trigger failover, which is just plain silly.

I see that your AlwaysUp may be able to better manage the program. My question is, can your product work to control and maintain the supervision of these instances?

— Andrei

Hi Andrei.

Several of our corporate customers have deployed AlwaysUp in Windows clusters. They tell us that, despite having no specific features that target cluster management, AlwaysUp works very well in that context.

In situations like yours where 100% uptime of a particular application is important, AlwaysUp adds an efficient line of defense — one that complements traditional multi-machine failover. Here’s how that works.

Application resilience: AlwaysUp protects against application failures

AlwaysUp’s job is to ensure that your application is always running. If your application crashes, AlwaysUp will automatically restart it.

But AlwaysUp provides much more than basic crash protection. It can operate proactively, rooting out problematic situations before they metastasize into full blown failures.

For example, you can have AlwaysUp quickly recycle your application if it:

  • Monopolizes the CPU for too long;

  • Consumes too much RAM;

  • Fails to respond properly to network/web requests;

  • Stops writing to a log file.

In it helps, you can even have AlwaysUp restart your finance program once a week during off-hours — to fend off mysterious lock-ups and other unpleasant instabilities.

Less downtime when your application fails

Most importantly, your system/service will likely experience less downtime when AlwaysUp is the first line of defense. Instead of waiting for the cluster failure to be detected and the switchover to the backup server to occur, your application will quickly bounce back on the active server. That rapid resolution can shave many precious seconds off your recovery time!

System resilience: Clustering protects against catastrophic failures

While AlwaysUp is able to cure many application failures, there are a range of deeper problems that it cannot solve. For example, when:

  • The machine loses power;

  • The server’s operating system crashes;

  • The network experiences an outage;

  • A critical hardware component (motherboard, hard drive, etc.) malfunctions.

For those dicey situations — where a server has been compromised — your failover cluster setup will save the day.

Configure a “Generic Service” instead of a “Generic Application”

Since you set up a Generic Application resource type to monitor your important program, you should remove that and replace it with a Generic Service that monitors the Windows Service created by AlwaysUp:

Select Generic Service in the High Availability Wizard

That change will enable your cluster to fail over whenever AlwaysUp protection stops — not when your legacy application fails. That is an important distinction.

If your application is named “Legacy Finance App” in AlwaysUp, select the Windows Service called “Legacy Finance App (managed by AlwaysUpService)”. You can find out more about the service created by AlwaysUp on the Frequently Asked Questions (FAQ) page.

Best of luck with your legacy application!

Posted in AlwaysUp | Tagged , , , , , | Leave a comment

3 Proven Ways to Send Email from a Windows Service (Without Outlook)

Sending Email from a Windows Service

Not all Outlook functions work from a Windows Service

Calling Outlook from a Windows Service can be problematic. Even though many operations work fine, Microsoft has issued some pointed advice for customers looking to run any Office application in the background in Session 0:

All current versions of Microsoft Office were designed, tested, and configured to run as end-user products on a client workstation. They assume an interactive desktop and user profile. They do not provide the level of reentrancy or security that is necessary to meet the needs of server-side components that are designed to run unattended.

Microsoft does not currently recommend, and does not support, Automation of Microsoft Office applications from any unattended, non-interactive client application or component (including ASP, ASP.NET, DCOM, and NT Services), because Office may exhibit unstable behavior and/or deadlock when Office is run in this environment.

If you are building a solution that runs in a server-side context, you should try to use components that have been made safe for unattended execution. Or, you should try to find alternatives that allow at least part of the code to run client-side. If you use an Office application from a server-side solution, the application will lack many of the necessary capabilities to run successfully.

Very disappointing!

So instead of calling Outlook, which may be unreliable when run in the context of a service, look to one of these alternative solutions instead:

Solution #1: Have your Windows Service call PowerShell to send basic email

If you don’t want to install any third-party utilities, you can leverage Microsoft’s ubiquitous PowerShell utility to deliver your messages. And to help, we’ve created a simple script that, given eight required parameters, will send an email to any address:

<#
   send-email.ps1
   
   Summary: Sends email.

   Usage: 
      send-email.ps1 <FROM-EMAIL> <TO-EMAIL> <SUBJECT> <BODY> <SMTP-SERVER> <USERNAME> <PASSWORD>

      Example:
         send-email.ps1 "admin@coretechnologies.com" "alerts@coretechnologies.com" "Server down" 
           "Server 'FileServer1' is down!" "smtp.gmail.com" 587 "admin@coretechnologies.com" "pwd74YKYRO"

   2020, Core Technologies Consulting, LLC (https://coretechnologies.com)
#>

if ($args.Count -ne 8) {
   Write-Host "Passed" $args.Count "parameters; expected 8."
   Write-Host "Usage:"
   Write-Host "send-email.ps1 <FROM-EMAIL> <TO-EMAIL> <SUBJECT> <BODY> <SMTP-SERVER> <SMTP-PORT> <USERNAME> <PASSWORD>"
   exit 1
}

$fromEmail = $args[0]
$toEmail = $args[1]
$subject = $args[2]
$body = $args[3]
$smtpServer = $args[4]
$smtpPort = $args[5]
$userName = $args[6]
$password = $args[7]

$smtpClient = New-Object Net.Mail.SmtpClient($smtpServer, $smtpPort)
$smtpClient.EnableSsl = $true
$smtpClient.Credentials = New-Object System.Net.NetworkCredential($userName, $password)

$smtpClient.Send($fromEmail, $toEmail, $subject, $body)

You can download the PowerShell script here. Please feel free to adapt it for your needs.

Your email provider will determine the SMTP server and port number you should use. For example, if your provider is Gmail, the SMTP server is “smtp.gmail.com” and the port is 587.

To invoke the script from an application, run the PowerShell executable with the -File option. Specify the full path to the script along with the eight required parameters.

For example, if you’ve saved the script in C:\Utilities and you’re sending via Gmail, your command line will look like this:

powershell.exe -File "C:\Utilities\send-email.ps1" from@coretechnologies.com to@coretechnologies.com "Server Down Alert" "Server 'FileServer1' is down!" smtp.gmail.com 587 from@coretechnologies.com "PWD8581JG$"

Watch out for quotes in the subject and body and escape accordingly!

Solution #2: Use SwithMail to deliver complex messages from your Windows Service

SwithMail is a free, no-nonsense utility that can send very detailed email messages. It supports all the important messaging options, including:

  • HTML formatting
  • Multiple attachment files
  • CC and BCC recipients
  • “ReplyTo” configuration

After downloading the SwithMail zip file and extracting its contents to a suitable location, double-click SwithMail.exe to reveal the comprehensive command line:

SwithMail command line usage

Simply specify the options you need. For example, here is a sample command line that sends the same message as the PowerShell script above:

SwithMail.exe /Silent /FromAddress from@CoreTechnologies.com /ToAddress to@coretechnologies.com /Subject "Server down" /HTML /Body "Server <b>FileServer1</b> is down!" /Server smtp.gmail.com /Port 587 /SSL /Username from@coretechnologies.com /Password "PWD8581JG$"

If you prefer, you can compose an XML file with all the details and pass to SwithMail instead, like this:

SwithMail.exe /XML "C:\Your-SwithMail-Settings.xml"

If you’re having trouble configuring SwithMail, add the /Log parameter and a path to a log file. Check the file for error messages after a failed run.

Solution #3: Update your Windows Service code to send email directly

If you have access to the service’s source code, your best option may be to include code to send email using the SMTP classes built into your programming language.

For example, if you are using C#, we recommend incorporating the System.Net.Mail.MailMessage and System.Net.Mail.SmtpClient classes. They are very easy to use.

Here is some sample C# code (error handling omitted for clarity):

// Compose the message.
MailMessage mailMessage = new MailMessage();
mailMessage.From = new MailAddress("from@coretechnologies.com");
mailMessage.To.Add(new MailAddress("to@coretechnologies.com"));
mailMessage.Subject = "Server down";
mailMessage.Body = "Server <b>FileServer1</b> is down!";
mailMessage.BodyFormat = MailFormat.Html;
// Construct the SMTP object that will send the message.
smtpClient = new SmtpClient("smtp.gmail.com", 587);
smtpClient.EnableSsl = true;
smtpClient.Credentials = new System.Net.NetworkCredential("from@coretechnologies.com", "PWD8581JG$");
// Send the message!
smtpClient.Send(mailMessage);

Get in touch if you need help sending email from your Windows Service

Hopefully one of these three solutions, which don’t involve Outlook, will work from your service. If not — or if you have questions about the methods outlined above — please don’t hesitate to reach out to our support team. We’re here to help!

Posted in Windows Services | Tagged , , , , , | Leave a comment

Q&A: What changed with my Windows Services?

Q&A: What changed with my Windows Services?
  How can I tell if someone updated the services on our Windows 2019 server? Do you have any tools for that?

— Sheldon P.

Hi Sheldon.

Since Windows Services run with high privileges, it’s very important to keep an eye on them. And because of their inherent power, services are a prized target for bad actors looking to hack your system.

Indeed, 2020’s SolarWinds supply chain exploit — one of the worst attacks in the past decade — featured a rogue Windows Service depositing malware in the background. A periodic review of the list of services could have identified the compromise months earlier.

Anyway, our free Windows Service Auditor is an excellent monitoring tool that can help you in your situation. Follow these instructions to keep a watchful eye on your mission-critical servers.

1. Download & run Windows Service Auditor

Windows Service Auditor is portable application, meaning that you don’t need to install it. Simply download the executable file and place it in a folder where you can easily find it.

Double-click the file to start it. In a few seconds, a window listing all your Windows Services will appear:

Windows Service Auditor

2. Update your computer’s security policy to allow advanced auditing

By default, Windows does not keep track all changes made to Windows Services. That capability must be enabled via advanced security audit policies. Specifically, you need to watch for:

Windows Service Auditor makes it easy to enable that auditing in your local policy. To do so, open the Application menu and ensure that the Enable Local Audit Policy entry is checked:

Enable Local Audit Policy settings

3. Enable auditing for important Windows Services, to track who starts/stops/changes them

Do you care about the activities of a specific Windows Service? Even though we have enabled advanced auditing in step 2, you must enable auditing for each service that you would like to monitor.

To enable auditing of a service in Windows Service Auditor, highlight the service and check the Selected Service > Enable Auditing menu entry:

Enable service auditing

With auditing in place for a service, the Windows Event logs will record an event whenever someone attempts to start, stop or modify the service. And to save you from hours of digging through the Event Viewer, Windows Service Auditor will collect those records in the lower Events panel:

Examining the Windows Update service

4. Capture a baseline snapshot of all services running on your machine

To capture a snapshot of all the services running on your computer:

  1. Start Windows Service Auditor;

  2. Select All Services > Export (XML);

  3. Choose a file name where the services should be saved.

The file will contain an XML record for each service installed on your computer:

Windows Service Auditor: All services XML export

5. Compare future snapshots to the baseline, to identify changes

Whenever you want to check if any services have changed, you should:

  1. Create a new snapshot XML file, as described in the previous section;

  2. Using your favorite text comparison tool, compare the new snapshot to the baseline you established in the previous section.

The text comparison tool will highlight all changes that have taken place in between the snapshots.

We recommend using WinMerge — a free, mature text differencing tool for Windows.

For example, we established a baseline snapshot on December 29. On December 31, we wanted to see what changed with services so we took another snapshot. Afterwards, comparing the two snapshots with WinMerge identified 8 differences, including one showing that the TrustedInstaller service was stopped:

Compare service snapshots with WinMerge

Best of luck managing your system!

Posted in Windows Services | Tagged , , , , | Leave a comment