Hi,
In the last article, you saw how overlay classes work, as well as how track items in Studio are defined and used. We'll continue our tour of track items by looking at column navigation feature and how it is used in various places. But before we get into that, we need to talk about how NVDA knows how certain commands apply in specific situations via method resolution order.
Method resolution order: locating commands and defining command scope
If multiple classes (objects) are defined, especially if inheritance is involved, it becomes hard to determine which method is which and where various methods are defined. This becomes complicated when two or more classes inherit from a single parent, or multiple inheritance is in use (Python supports both scenarios).
One way Python solves this is through Method Resolution Order (MRO). Simply put, when a method is to be used, it first looks at whether this method is defined in the object it is dealing with, and if not, will consult the parent of this object.
For example, suppose we have a list box that defines a scroll method, and a custom list box widget inherits from this list box (in effect, custom list box is a list box). To make matters slightly complicated, let's say the scroll method is not defined (not really defined) in the custom list box. Then when the user scrolls through the custom list box, Python will see that the custom list box does not have the scroll method, so it'll look at the parent (original list box) and use its scroll method (in this case, yes, Python will use the parent's scroll method).
In terms of NVDA, method resolution order comes in handy when dealing with overlay classes. This has wide ranging consequences, including ability to limit where certain commands can be used to not defining a command (setting the script bound to gesture to None), effectively forcing NVDA to look up a given gesture from the base class (parent). If NVDA cannot locate the command in question, it'll pass this to Windows, which then sends the command to the active program.
Finishing the puzzle: MRO and Studio's track items
As described in the previous article, Studio app module defines two classes for track items: one for Studio 5.0x and another for 5.10. In reality, all that matters is the former, with the 5.10 class providing custom routines on top of the 5.0x track item class. In case of MRO, Studio 5.10 will be consulted if 5.10 is in use, then NVDA will consult Studio track item. For 5.0x, only original track item will be consulted.
The contents of the track item class (appModules.splstudio.SPLTrackItem) are as follows:
* initOverlayClass: This is run when the track item is first encountered. This method checks if Track Dial (see below) is active, and if so, assigns left and right arrow keys to Track Dial functions.
* reportFocus: This is called when reporting track items to you (broadcaster). It's main job is to see if custom column order is defined (see below) and builds needed pieces if column order is specified.
* Track Dial methods: Various methods and scripts used by Track Dial are defined, including announcing column information, handling leftmost column and so on.
In addition, Studio 5.10 track item (appModules.splstudio.SPL510TrackItem) includes a script to announce changed state when SPACE is pressed (this is done by speaking and/or brailling track name (obj.name)).
Birth of Track Dial: from hesitation to possibilities
As I was writing the add-on, one of the top suggestions I received was ability to use enhanced arrow keys feature to review columns. This feature allows broadcasters using screen reader scripts to use arrow keys to review column information such as artist, duration and so on. As of time of this writing, all three screen reader scripts support this feature.
At first, I told broadcasters that this wasn't possible. My impression back then (summer 2014) was that I had to manipulate track description text (obj.description) to enable this possibility. But seeing how other screen readers implement this convinced me that it might be possible to implement this for NVDA users, thus I started researching this in fall 2014.
I started by looking for patterns in description text that could be used to give users an impression that column navigation was active (when it was not). I studied how Python handles regular expressions and manipulated substrings (str.find and slicing) with no satisfactory results. Then in 2015, while working on improving support for Studio's own stream encoder, I noticed that the encoder entries were SysListView32 objects (NVDAObjects.IAccessible.SysListView32). Careful study of this object, especially a method to retrieve column content, gave me an idea as to how to bring Track Dial to life.
The magic behind Track Dial: SysListView32 controls
SysListView32 controls are lists with items organized into columns. For example, certain apps use these controls to arrange entries into columns, such as in certain table-based apps, and in case of studio, displaying various status about encoders.
When these controls are encountered, NVDA allows you to use table navigation commands (Control+Alt+arrows) to navigate between columns, provided that there are child objects (columns) exposed by the accessibility API implementation in use. Table navigation commands are supplied by another class (behaviors.RowWithFakeNavigation; the behaviors mix-in includes things NVDA should perform in various scenarios, including terminal input and output, editable text handling (via a dedicated module) and so on). This is all possible thanks to a method in SysListView32 class (NVDA Core) that allows one to retrieve column text, and this became the engine for Track Dial and other column navigation facilities in the Studio add-on (I say "add-on" because column navigation is used by both Studio and Track Tool).
The column text retrieval routine (which lives in SysListView32 and a copy lives in splstudio.splmisc module) is as follows:
1. Features requiring text from a specific column will call a private function in splstudio.splmisc module, which will take the current object and the column index as parameters.
2. The column retriever first looks up the handle for the process where the control lives, then creates a place holder for the buffer to hold the column content.
3. Next, it looks at the size of the underlying sysListView32 control (ctypes.sizeof) and asks Windows to allocate storage for an internal SysListView32 control via kernel32.dll's VirtualAllocEx. This is needed to store the resulting column text. Same is done for another place holder to store the actual column text by calling VirtualAllocEx.
4. The retriever then creates an internal SysListView32 control used as a place holder to store column text, then asks Windows to tell the process where the column text lives to reveal the column text for the specified column (first calls WriteProcessMemory, sens a message via user32.dll's SendMessage to retrieve text length, then uses ReadProcessMemory to retrieve the actual column text if there is something to read). Once the column text is revealed, the retriever stores the text inside the text buffer (ctypes.create_unicode_buffer, allocated to store the resulting text).
5. Lastly, the retriever frees resources (VirtualFreeEx) and returns the just retrieved column text, which can be used by routines requesting this.
Column retrieval and navigation routines: four scenarios, one servant
The above routine is used not only in Track Dial, but is also employed by three other routines: Column Search, column announcement order and Track Tool app module. Let's tour how these four friends use this routine in more detail.
Track Dial: Navigating columns in track items
The column retriever routine is just one of the activities performed during Track Dial, and to see the beauty of this feature, assign a command to toggle Track Dial (you need to focus on the track item before opening Input Gestures dialog, as Track Dial is used by track items alone). Once you assign a command to toggle Track Dial and toggle this on, Studio will set a flag indicating that Track Dial is on, which causes left and right arrow keys to be assigned to column navigation commands (this flag is stored as part of the add-on configuration database). If you tell NVDA to play beeps for status announcements (see previous article), NVDA will play a high beep. Once you are done with Track Dial, press the just assigned command to turn off Track Dial, at which point left and right arrow keys return to their original functions, a low beep will be heard (if told to do so) and Track Dial flag will be cleared.
When navigating columns, NVDA will check if you are at the edge of the track row, and if so, it will play a beep and repeat the last column text. If not, NVDA will look at the column you wish to navigate to (stored in the app module), then it'll call the column text retriever to retrieve the column text.
To handle differences between Studio 5.0x and 5.10, each track item class informs NVDA as to how leftmost column should be handled. For Studio 5.0x, leftmost column is artist field (obj.name will be checked), while track checked status is "shown" in Studio 5.10 (obj.name will be announced).
Custom column announcement order: What to announce and how
Track Dial routine also allowed another top requests to come to life: column announcement order. This allows you (broadcaster) to hear columns in specific order and to exclude certain columns from being announced. This is housed in reportFocus method in the main track item class.
In order to use this, you must tell NVDA to not use screen order (add-on settings dialog). Then open column announcement dialog and check the columns you wish to hear, then use the columns list to set column announcement order. The column announcement order is a list box with two buttons: move up and down.
Once column order and included columns are defined, NVDA will use this information to build track item description text. This is done by repeatedly calling the column retriever routine for columns you wish to hear, then using the column order you defined to build parts of the description text (a combination of a list and str.join is used).
For example, if NVDA is told to announce title and artist (in that specific order), NVDA will first locate title, then will add artist information. This is then presented as, "Title: some title, Artist: some artist".
Column Search: Finding text in specific columns
In the previous article, I mentioned that a single dialog performs double duty when talking about Track Finder. We'll now tour the other side of the coin: Column Search.
Column Search dialog adds a second control to Track Finder: a list of columns. Once text is entered to be searched in a column, NVDA will use trackFinder routine (discussed earlier) to locate text in specific columns (I mentioned that trackFinder routine takes column as the argument, and this is where this argument comes in handy). The routine is same as Track Finder except it uses the above column retriever routine to locate column text, and once search is done, it'll either move you to a track item or present an error dialog.
So what causes one dialog to present both Track Finder and Column Search dialog? It's all thanks to the arguments passed into the find dialog constructor. The signature is:
splmisc.SPLFindDialog(parent, obj, text, title, columnSearch=False)
The last argument (columnSearch) determines which version of the dialog to present. The object (obj) is needed to tell NVDA where to begin the search and to call the track finder routine defined in the object's app module.
Track Tool: one routine, multiple app modules
Column retriever routine is not only employed by Studio app module, but is also used in Track Tool app module (part of the studio add-on). Track Tool's use of column retriever include Track Dial for Track Tool (same routine as the Studio app module and Studio must be running to use it) and announcing column information (Control+NVDA+1 through 8).
Conclusion
Of all the features in the StationPlaylist Studio add-on, column navigation is one of my favorites (besides Cart Explorer and encoder support and others). I enjoyed working with this routine and learned a few things about Windows API, as well as open possibilities not previously explored before, such as Track Tool and Column Search. I hope that you'll find column navigation commands to be useful in your broadcasts.
I would like to take this time to answer a question posed by some users and developers: Can NVDA be ported to other operating systems? No. The above column retriever routine is a prime example why this cannot be done easily: different operating systems use different API's, and porting NonVisual Desktop Access to other operating systems will involve significant architectural changes to use the new API's. In case of ReactOS, this isn't possible, as there are no stable foundation from which NVDA screen reader can exercise its full rights: accessibility API's are needed, stable driver development framework is needed, ability to run a program as a service must be ready and so on. Add to the fact that we have several add-ons relying on Windows API (including StationPlaylist Studio add-on) and you'll see the huge work involved in an attempt to port NVDA to other operating systems.
I think we talked about column routines too much today. There is another important concept we need to talk about next time: threads and how it helps Studio perform some of its important tasks: library scan and microphone alarm.
References:
1. Method Resolution Order, The History of Python, june 23, 2010. http://python-history.blogspot.com/2010/06/method-resolution-order.html
2. List View, Microsoft Developer Network: https://msdn.microsoft.com/en-us/library/windows/desktop/bb774737(v=vs.85).aspx
3. List View Messages, Microsoft Developer Network: https://msdn.microsoft.com/en-us/library/windows/desktop/ff485961(v=vs.85).aspx
4. List View Item structure, Microsoft Developer Network: https://msdn.microsoft.com/en-us/library/windows/desktop/bb774760(v=vs.85).aspx
5. VirtualAllocEx (kernel32.dll) reference (Windows API): https://msdn.microsoft.com/en-us/library/windows/desktop/aa366890(v=vs.85).aspx
6. VirtualFreeEx (kernel32.dll) reference (Windows API): https://msdn.microsoft.com/en-us/library/windows/desktop/aa366894(v=vs.85).aspx
7. WriteProcessMemory (kernel32.dll) reference (Windows API): https://msdn.microsoft.com/en-us/library/windows/desktop/ms681674(v=vs.85).aspx
8. ReadProcessMemory (kernel32.dll) reference (Windows API): https://msdn.microsoft.com/en-us/library/windows/desktop/ms680553(v=vs.85).aspx
9. Ctypes (Python documentation, Python Software Foundation): https://docs.python.org/2/library/ctypes.html
No comments:
Post a Comment