The Core Technologies Blog

Our Software // Windows Services // 24×7 Operation


4 Affordable Tools to Monitor & Bulletproof your Web Application/Server

Monitor Web Application/Server

Does your application accept HTTP requests?

Are you responsible for a mission-critical application that features a HTML, XML or JSON web interface?

And can you visit a URL (such as “http://yoursite.com:3232/status”) to confirm that your web application is working well?

If so, website monitoring tools — which periodically “ping” your site and confirm that requests are being accepted — can significantly improve your application’s reliability. If you’re not already leveraging such tools, do yourself a favor and grab one today!

Improving uptime: Detection vs. Remediation

Some tools focus on detection. Like the mythical canary in the coal mine, their primary job is to recognize that there is a problem and quickly inform you. They can give you a meaningful head start on resolving the problem — before your important customers start complaining.

But while detection is clearly an advantage, the gold standard is remediation. A quick, automatic remedy minimizes downtime and reduces (or eliminates) stress for the humans involved. Let the machines fix the problem while you eat dinner with your family! 🙂

With that important distinction in mind, let’s review a few inexpensive tools/services that we have used in our professional lives, and a couple that our company has developed for Windows.

1. Pingdom (Detection only)

Starting at $10/month, Pingdom offers a basic monitoring service that checks your URL every minute. If your site is unreachable, it will immediately email you to let you know.

Here are the options when adding a new URL check:

Pingdom URL Check: Settings

Notice that you can configure the check to be performed from multiple geographies. That is a welcome feature if you have customers accessing your site from around the globe.

The URL check’s alerting settings are equally intuitive:

Pingdom URL Check: Alerting

And alerts can be sent via email or SMS/text:

Pingdom User Alert Settings

We use Pingdom to babysit a handful of customer sites/applications. All in all, it’s a reliable detection tool that quickly shouts whenever there is a problem.

2. SiteUptime (Detection only)

SiteUptime is another excellent monitoring tool. It works much like Pingdom, but with a few extra capabilities.

Adding a URL to check is straightforward. Only the basics are required:

SiteUptime Monitor Settings

Things get interesting when you delve into the advanced settings. For example, you can choose the city from where the checks should originate:

SiteUptime Monitor Advanced Settings

Alerts can be sent by email, sms/text or by a phone call:

SiteUptime Monitor Advanced Settings

PagerDuty is also supported.

You can use SiteUptime entirely free (with 1 monitor, 30 minute check interval) but the professional-grade service starts at $10/month. Check it out!

3. AlwaysUp (Detection & Remediation for Windows Applications)

While Pingdom and SiteUptime are affordable, easy to setup services, their effect is limited to failure detection. Once a problem has been identified, they cannot help you take action to resolve the interruption.

That shortcoming is understandable. After all, those services work from the “outside”; they do not know anything about your web application (other than the URL you supplied).

On the other hand, our AlwaysUp utility — designed to run any application in the background as a Windows Service — can do more because it has control of your web application’s process. And at a one-time cost of $50 per server, it is a bargain.

When configured with the check web server sanity check, AlwaysUp will improve reliability by:

  1. Checking the URL every few seconds or minutes (configurable)

  2. Automatically restarting the application (or rebooting the machine) if the URL does nor respond or returns a 5XX error

  3. Sending you an email to let you know what happened

AlwaysUp: Check web server sanity check

The result is that your web application will be down only for a few seconds or minutes.

Note, however, that AlwaysUp works on Windows only. It is not an option for Linux web applications.

4. Service Protector (Detection & Remediation for Windows Services)

If your application runs as a Windows Service, then Service Protector should be your tool of choice for improved reliability.

Like AlwaysUp, it will leverage the check web server sanity check to:

  1. Check the URL periodically

  2. Automatically restart the service (or rebooting the machine) if the URL times out or returns an error

  3. Email you about the issue, to keep you in the loop

You can protect an unlimited number of services on a single server for $70. If you use the product for just a single year, that works out to $0.20 per day. Peace of mind, on the cheap!

Honorable mention (Detection only)

We’ve heard great things about another three site monitoring tools but we have never tried them. If you not on Windows and are interested in detection alone, they are definitely worth a look:

Enjoy!

Posted in Software | Tagged , , , , , | Leave a comment

Q&A: How do I Remove an AlwaysUp “At-Login” Startup Task?

  At some point I must have setup a command line to start my application in AlwaysUp when I log in. But I do not need to do that any longer. I deleted the configuration in AlwaysUp but I do not know how to review and delete what I did in the command line. So this error comes up every time I log in:

AlwaysUp Command Line Error

Can you help me please?

— Nic

Hi Nic. The attempt to launch an obsolete AlwaysUp service is coming from one of several places. Please check the following:

Is there a shortcut in your Windows Startup folder?

Your Windows Startup folder holds shortcuts to applications that should be started when you log in. You can find it here:

C:\Users\YOUR-USER-NAME\AppData\Roaming\Microsoft\Windows\Start Menu\Programs\Startup

(You can find your user name by running whoami at a command prompt.)

Navigate to that folder in File Explorer. Do you have any shortcuts that launch an AlwaysUp application? Look for the AlwaysUp icon:

For example, here is a shortcut that restarts Dropbox in the current session sitting in the startup folder:

Dropbox Service Shortcut in the Startup Folder

If you find a shortcut for your bogus application, please delete it. That will eliminate your problem.

If you don’t see an appropriate shortcut, note that on Windows 10, there is another startup folder for all users:

C:\ProgramData\Microsoft\Windows\Start Menu\Programs\StartUp

Please check for an AlwaysUp shortcut there as well.

If you still don’t find any shortcuts, let’s move on to the more obscure locations.

Is there a scheduled task?

The Task Scheduler is a powerful Windows component that can launch applications in response to specific trigger events. It is possible that someone created a task that launches your application/service whenever you log in. Let’s see if we can find such a task.

  1. Start the Windows Task Scheduler by running taskschd.msc. (You can also find it in Control Panel by searching for “Schedule tasks”).

  2. In the Task Scheduler window that comes up, select Task Scheduler (Local) on the left and scroll down to the Active tasks section in the center panel:

    Task Scheduler Active Tasks
  3. Browse the list of active tasks. Look for any mention of your application, AlwaysUp or anything else relevant to your situation. The Triggers column should say At log on, perhaps for your user account.

    For example, here we have identified an AlwaysUp task that restarts Plex Media Server in the current session (i.e. visibly on your desktop) whenever you log on:

    Plex Media Server Task
  4. Let’s confirm that this is the offending task.

    Double-click on the task to move to its location. And double-click the entry to open its properties.

    Switch to the Actions tab and examine the actions. An AlwaysUp task will point to the AlwaysUp executable and reference the application name in the optional arguments, as shown here for Plex:

    AlwaysUp Scheduled Task Action

    Also look for a batch file with a single command launching AlwaysUp.exe.

  5. Once you are comfortable that you have identified the obsolete task, dismiss all sub-windows and return to the main Task Scheduler window. Right-click the task and select Delete to remove it from your system:

    Delete Plex Scheduled Task
  6. Still nothing? OK, let’s bring out the big guns…

What does Autoruns show?

Autoruns is a free administrative tool created by Microsoft. Its purpose is to identify every single application that Windows launches automatically — either when the computer boots or when you log in.

To check for an AlwaysUp task with Autoruns:

  1. Download Autoruns. Extract the zip file to a folder on your hard drive (e.g. C:\Apps\Autoruns).

  2. Start Autoruns as an administrator. (The tool needs admin rights to help you eliminate unwanted startup programs.)

  3. Wait a few seconds for Autoruns to collect its information.

  4. The Everything tab lists all applications, services and scripts launched at startup. Scroll through the list and see if you see any AlwaysUp tasks. You can also use the search function.

    Autoruns: All Startup Programs

    In any case, be sure to omit services. Those will be listed in the HKLM\System\CurrentControlSet\Services section. Those are not the problem.

  5. If you find an AlwaysUp task, you can delete (or disable) it from Autoruns.

    But please proceed with caution. Make 100% sure that you are taking the right action! You don’t want to inadvertently eliminate a vital operation.

Still stumped? Please get in touch for a few more tips

For additional guidance, please send an email to our support team detailing everything you have tried. Please include an export of your Autoruns data (created by selecting File > Save from the menu) so we can see what is automatically loaded onto your system. We’ll help you get the job done!

Posted in AlwaysUp | Tagged , , , , , | Leave a comment

Q&A: Why did Dropbox Stop Copying/Synchronizing my Files?

Q&A: Dropbox stopped copying my files
My client who has AlwaysUp for Dropbox running on his server informed me that his files stopped syncing.

I created a test file in their folder that is supposed to be syncing and sure enough, it was not working. I checked the service in AlwaysUp and it was running. I tried restarting it and still nothing. I stopped the service and opened Dropbox and it is now syncing.

Please inform me of what I need to do from here. Thanks.

— Kurt

Hi Kurt, sorry to hear of the trouble!

In response, our technical team took the latest version of Dropbox (Stable Build 105.4.651) for a test drive on our Windows Server 2019 machine. Sure enough, file synchronization stalled when running as a Windows Service in Session 0.

Our first instinct was to apply the time-tested advice from this troubleshooting article. We carefully ensured that:

  1. Dropbox.exe wasn’t updating itself (because frequent automatic updates can thwart 24/7 operation)

  2. The Dropbox sync folder was accessible

  3. Dropbox was not trying to get our attention (by showing a notification or confirmation prompt)

Yet after doing all that the problem remained!

So, we rolled up our technical sleeves, unplugged the phone, paused the email and dived into Dropbox. Here’s what we observed.

Dropbox synchronized files at first but stalled after a few minutes

Our testers employ a simple test plan to validate basic file synchronization functionality. It goes like this:

  1. Start Dropbox from AlwaysUp (in Session 0)

  2. Wait 1-2 minutes, to allow the program to initialize and get its bearings

  3. Copy a new file into the PC’s Dropbox folder

  4. Wait up to 1 minute, for Dropbox.exe to recognize the new file and copy it up to the cloud

  5. Check that the new file is listed on Dropbox.com

When we followed the process, step 5 failed. The new file did not appear on the website.

So, we tried again. This time, we noticed something interesting. Dropbox synchronized all pending files when it started, but it would not copy the new file we added. Apparently, synchronization worked at the outset but “broke” after a few minutes. Bizarre!

Upon closer examination — using Microsoft’s excellent Process Explorer — we observed the Dropbox application’s network connections behaving strangely. Let me explain.

A minute after Dropbox starts, the application opens several connections back to “home base”. Here you can see six sockets in the ESTABLISHED state linked to multiple cloud servers on the 162.125.*.* network:

Dropbox TCP/IP connections

A few minutes later, five of the connections have vanished and the final link is dead on its feet (in the terminal CLOSE_WAIT state):

Dropbox TCP/IP connections - Closed

At this point, Dropbox is unable to synchronize any files because it doesn’t have a functioning connection to the back-end servers.

Applying the “QT_OPENGL” fix restored synchronization

About a year ago, we experienced a puzzling situation where Dropbox would hang when started as a Windows Service. This post documents the problem and this post describes the fortuitous fix.

Back then, Dropbox wouldn’t process any files at all — very different from today’s situation, where the program starts strong but quickly goes AWOL.

Nevertheless, a very similar fix works!

Once we set the “QT_OPENGL” system environment variable to “software” (as described below), Dropbox seemed to maintain its server connections indefinitely.

To verify, we wrote a simple script that adds a new file to the Dropbox sync folder every 30 seconds. We let it run for six hours. During that time, Dropbox was able to upload each file to the cloud and every one showed up properly on the website.

Here is a screenshot of the test while in progress:

Dropbox copy test in progress

How to apply the “QT_OPENGL” fix

Please perform this step-by-step process to resume your client’s 24/7 service:

  1. Start a Windows command prompt as an Administrator (an elevated command prompt)

  2. Run this command:

    SETX /M QT_OPENGL software

    SETX /M QT_OPENGL software
  3. Close the command prompt.

  4. Stop and start your Dropbox entry in AlwaysUp.

You can confirm that the fix was applied by looking for the QT_OPENGL value in your system environment variables (available through the Control Panel):

QT_OPENGL System Environment Variable

Please be sure to get in touch if you notice any other misbehavior!

Posted in AlwaysUp | Tagged , , , , | 1 Comment

Essential Windows Services: LanmanServer / Server

Server Service

What is the LanmanServer (Server) service?

The LanmanServer service allows your computer to share files and printers with other devices on your network.

The service’s display name is Server and it runs inside the service host process, svchost.exe. By default, the service is set to start automatically when your computer boots:

LanmanServer Windows Service

What happens if I stop the LanmanServer service?

There will be a couple of potentially significant consequences if you stop the Server service.

1. File sharing will not work

Is your computer hosting a shared folder? If so, that folder will no longer be available to network computers once you stop LanmanServer.

Mapped drives pointing to your computer’s shared folders will be inaccessible and remote users will experience abrupt errors like this:

Mapped drive not accessible

In addition, all sharing controls will disappear from your computer’s File Explorer windows.

For example, the drive properties window has a Sharing tab while LanmanServer is running:

The Sharing tab is accessible

But once the service stops, the Sharing tab vanishes:

The Sharing tab has vanished

Of course, only remote users will be impacted. The underlying files and folders will still be available on the computer hosting the drive.

2. Remote printing will not work

Is your computer sharing its printer with other computers? If so, the service being down will affect folks printing from those other computers.

Specifically, Windows will tell them that the remote printer is “Offline”. All their print requests will enter the print queue, patiently waiting for the printer to come back online:

The printer is offline and jobs are queued

Note that local printing (from the computer with the printer attached) will still work. Only remote users/computers will be affected.

Is it safe to disable LanmanServer?

Yes, it should be safe to disable the service — provided that your computer is not sharing files or printers. As mentioned above, those services won’t operate without the Server service.

Note that Internet-based networking features (for example, accessing web sites from your browser) will still be available when the service is disabled. In theory, many non-networked or home computers will do just fine without LanmanServer always running.

And because LanmanServer doesn’t have any dependencies, disabling it shouldn’t prevent any other services from starting up:

LanmanServer has no dependencies

However, despite the above evidence to the contrary, Microsoft recommends keeping the Server service enabled. Unfortunately they don’t provide a reason or explanation!

Please proceed with caution if you decide to disable the LanmanServer service.

Questions? Problems?

If you would like to know more about the Server service, or you have a specific problem, please feel free to get in touch. We will do our best to help you!

Posted in Windows Services | Tagged , , , | Leave a comment

Q&A: I created a Windows Service with SC. Why won’t it Start?

Q&A: SC Windows Service won't start
  I installed my program as a Windows Service using the SC command. The result was “CreateService SUCCESS”, and my service is listed with all the others so I know it worked.

But whenever I try to start the service, the NET command fails with “The service is not responding to the control function”. What does that mean? Why can’t I start the service?

— Marc B.

Hi Marc.

You are running into a common problem with the SC utility. While SC will happily install any script, batch file or executable as a service, the service it creates may not be usable!

Let’s dig into the details.

SC is not picky; it will install anything as a service

As you have discovered, the “SC CREATE” command will create a new Windows Service. The command line must look like this:

SC CREATE <SERVICE-NAME> binpath= "<FULL-PATH-TO-PROGRAM>"

(Yes — the space after “binpath=” is required!)

Apparently all you need to specify is a friendly name and the full path to your program for SC to work it’s magic. Simple, right?

For example, to install the Windows Notepad application as a service called “NotepadService”, run this command from an administrative command prompt:

SC CREATE NotepadService binpath= "C:\Windows\Notepad.exe"

SC will confirm success:

SC creates a new service

And the new service will be listed in the Services application:

New service in services.msc

But there are limits to what SC will do. Even as it dutifully performs the basic operations — creating the necessary entries in the Windows Registry — SC will not validate the path/program in any way. This means that:

  1. SC will succeed even if the program does not exist. You can give it a bogus path and see:

    SC installs non-existent program

    However, the following error pops up when you try to start the “BadPath” service installed above:

    You can't start a service with a bad path
  2. SC will succeed even if the program cannot run. For example, here we have installed a ridiculous service where the path is an MP3 music file:

    SC creates an MP3 service

    Indeed, as an MP3 is not a valid application, it is no surprise that attempting to start the service results in failure:

    You can't start a MP3 file as a service
  3. So, as we have shown, SC is firmly focused on basic installation — not operation. The fact that a service was created with SC does not mean that the service can ever start or run.

    Only a “true” Windows Service Application can be started as a service

    Even though SC will install any program as a service, the reality is that only specially constructed executables can actually operate as a service. We call those Service Applications.

    Service applications implement the Windows Services API — a set of operations used to control and monitor services. Windows uses the API to interact with Service Applications.

    The API is quite extensive. With it, Windows can command any Service Application to:

    • start running
    • stop running
    • report status
    • pause
    • resume running
    • prepare for system shutdown

    Unfortunately regular desktop applications (like Microsoft Word, Google Chrome and Adobe Acrobat Reader) do not support the Windows Services API. They are not Service Applications. There is simply no way for Windows to directly control or interrogate any of those desktop programs as a service.

    Non-Service Applications will fail to start, citing error 1053

    To illustrate, let’s take Dropbox — a leading cloud storage provider. The company produces Dropbox.exe, a desktop application that synchronizes files between your PC and the cloud. Dropbox.exe is not a service application.

    As you can see, SC has no problem installing Dropbox as a service:

    SC creates Dropbox service

    But when you try to start the Dropbox Service from the Services application, a curious scenario plays out:

    1. Windows starts Dropbox.exe.

      Here you can see Dropbox (and its two helper processes) running in Process Explorer:

      Dropbox.exe has been started
    2. Next, the Service Control window will report that “Windows is attempting to start” the service:

      Windows is attempting to start the Dropbox service
    3. And after about 30 seconds, the Dropbox.exe processes vanish from the system and Error 1053 is raised:

      Dropbox service failed to start with Error 1053

    Even though Windows kicked off Dropbox.exe as intended, the process was terminated in less than 40 seconds. A clear failure.

    And the story is the same with other regular, non-service applications as well. They all misfire in the same way.

    Why did the Dropbox service fail to start?

    The problem occurs in step 2.

    After launching Dropbox.exe (step 1), Windows waits for the new process to report its status and confirm that the service has started properly. However, since Dropbox does not support the Windows Services API, it fails to respond to that service-specific status check.

    And after a few seconds of lingering, Windows eventually gives up. It terminates the Dropbox.exe process (and all its sub-processes) and reports that “the application did not respond to the start or control request in a timely fashion” (i.e. error 1053).

    Use a “Service Wrapper” to install a regular program as a Windows Service

    As demonstrated above, you cannot simply install a regular application with SC and expect the service to work. You need help from a special “go-between”.

    A service wrapper is a Service Application that starts and stops another application in response to important events. When a service wrapper receives a Windows Services API request, it acknowledges the request and performs the appropriate action on the application it is managing.

    For example, let’s say that we have a service wrapper configured to manage Notepad. When the service wrapper receives a “start the service” request, it would start a new instance of Notepad.exe and let Windows know that the service was started successfully.

    And when the service wrapper receives a “stop the service” demand, it would terminate the Notepad process it started before exiting itself.

    Perhaps the best known service wrapper is Microsoft’s Srvany. Srvany is free and functional, but has some significant shortcomings.

    Alternatively, our AlwaysUp utility is a professional service wrapper. It’s not free, but is robust and mature, and makes running your application in the background as a Windows Service super easy.

Posted in Windows Services | Tagged , , , , | Leave a comment