Wednesday, April 30, 2008

Ensuring only one instance of an application can be run and then breaking that approach


A question posted on a DL was as follows

"I have an application that allows only one instance on the desktop, how do I force it to open multiple instance for some testing purposes".

Disclaimer: Note that this is kinda hacky and only relevant if you are a tester trying to break something. This shouldn't be treated as a way to achieve anything productive.

Since the user didn't explain which approach is used to ensure only one instance is allowed I'll try listing down the methods I know off and how I can break them.

Using named Mutex:
This is the more resilient approach taken by applications like Windows Media player. Here the application tries to create a named Mutex with sufficiently complicated name to avoid collision. E.g. and then try to acquire the mutex. In case it fails to acquire it then an instance is already running and it closes itself. If it acquired the mutex it means it is the first instance and it continues normally. An approach is outlined here.

Even though this seems to be the most robust solution this can be made to fail as well. If an user has sufficient permission he can do a CloseHandle on a Mutex handle created by some other application as well. For this you don't even need to write code. Do the following

  1. Download and install Process Explorer.
  2. Launch it (might need elevation on Vista)
  3. Have an application that uses this approach like my application named SingleInstanceApp.exe. Try launching a second instance and it will fail to open it.
  4. Click on the app finder toolbar button in Process Explorer and drag it to SingleInstanceApp window. This will get this application selected in process explorer.
  5. Select View->Lower Pane View ->Handles or simply Ctrl+H
  6. This will show all the handles created by SingleInstanceApp.
  7. Scroll down looking for the mutexes. At the end the screen looks something as follows Capture
  8. Right click on the mutex and choose Close Handle.
  9. Now try creating the second instance and it will open fine.

Enumerate the names of applications running and see if the app is already running:
This uses a combination of Process.GetCurrentProcess and Process.GetProcessByName(), see an implementation here.

This is easy to break. To do a DOS (Deinal of Service or DOS) just  create another application with the name that the app expects and per-launch it. To open two instances copy the application to another name and open that first and then the original application.

Enumerate Windows

This involves iterating through all the open windows in the system using EnumWindows and then seeing if a Window with a given name or classname is already present. Since the system doesn't guarantee uniqueness in either of the text or class name this can be broken in a similar approach to the first one. However, there are some complications to it as it is difficult to change window text from outside. However, a combination of code injection and SetWindowText win32 API should work.

No comments: