Herbi Web Keys Visual Studio 2010 project
The Visual Studio 2010 project for the Herbi Web Keys app was created from the template for Windows Forms apps.
It has a pre-build step of:
"%PROGRAMFILES%\Microsoft SDKs\Windows\v7.0A\bin\tlbimp.exe" %windir%\system32\UIAutomationCore.dll /out:..\interop.UIAutomationCore.dll"
This makes it straightforward for a C# app to call into the native-code Windows UI Automation API. Given that the build step uses a hard-coded path, that path might need to be updated in order to get the project to build on your computer.
The files of most interest are listed below.
This file contains all the interop code to allow me to call all the Win32 APIs that I leverage from my C# code.
This file contains the keyboard hook which allows me to react to key presses while the app's running. If a key press is unrelated to the app, I allow it to pass through unaffected by the hook. If the key press is related to the app, I post a message to the main app form, and prevent the original event from being processed further.
This file contains some regular UI-related action, for example, button and link click handlers. It also processes the message posted from the keyboard hook. If it's appropriate to only describe what action a key relates to, then I call the SpeakAsync() method of the System.Speech.Synthesis.SpeechSynthesizer.
If I do want to take some action in response to the key press, I call my PerformAction() action. Beneath this, I sometimes call SendInput() to simulate keyboard input to control either a screen reader or browser. The file also has the code for using UI Automation to invoke different elements in the browser UI. WebKeys.cs is the only C# file that references the interop.UIAutomationCore.dll assembly that was generated by the pre-build step mentioned above.
Note that when the solution is built, a number of build warnings relating to arguments in UIA methods calls "cannot be marshaled by the runtime marshaler". These warnings can be ignored.
Download the VS 2010 solution
The solution can be downloaded from Herbi Web Keys Visual Studio 2010 project.
The files included in the zip file downloaded are:
The following known issues exist with the app today.
1. If keyboard focus is in an edit control on the web page, this can prevent some of the keyboard shortcuts from working.
2. When the app is waiting for the UI to react some some action, it simply calls Thread.Sleep(), and assumes that the UI it was awaiting for appears, rather than programmatically checking that the UI's appeared during the wait.
3. Sometimes if Internet Explorer is brought into the foreground as part of showing the Favorites menu, the menu doesn't appear. If that happens, then the key to show the Favorites menu must be pressed again.