Hi,
When you use the Studio add-on for NVDA, you may have noticed that you can perform certain commands while focused on track items, and that the command to find tracks is same as that of find command in web browsers. If you are curious about these, then this installment will let you see how it works. But first, we need to go over some more facts about NVDA screen reader, this time we'll talk about objects.
Important facts about NVDA's use of objects
One of the questions I and other add-on authors and NVDA developers received was, "what exactly are objects and how are they used in NVDA?" In programming, an object is instance of the object definition coming to life (this definition, called a "class", defines how certain things behave and how information can be retrieved from this object by other parts of the program; the programming paradigm that uses classes and related concepts is termed "object-oriented programming"). For example, someone may say, "build me a phone book", and a programmer will think about how phone book entries are stored and come up with a "phone book" (an array of phone entries), all done via objects.
In graphical user interfaces (GUI's), an object is a more technical term for controls (sometimes called widgets). This includes windows, form fields, links, documents and so on. A control (object) can convey information such as state of the control, location, color and so on (the control contains both visible and internal attributes that can be used by other programs).
In NVDA world, all screen elements (controls) are objects. As such, when dealing with objects, NVDA uses accessibility API's to obtain needed information. To provide consistent user experience, differences between accessibility API's (IAccessible, UI Automation, Java Access Bridge and so on) are checked and provides a mechanism to announce same information across controls implemented using different frameworks. For example, when a check box is checked, NVDA will say "checked"" - NVDA will know if you checked this box because the underlying accessibility API informs NVDA of this change, and the same information is spoken regardless of whether it is dealing with IAccessible, UIA and so on.
Currently, NVDA can work with IAccessible, User Interface Automation (UIA), Java Access Bridge (JAB) and others (WAI ARIA is supported). Support modules for these API's lives in source/NVDAObjects directory of the NVDA Core source code.
Overlay classes: Customizing built-in objects
If NVDA was limited to using its own object handlers, we would be limited to information that is correctly exposed by accessibility API's (no app modules at all). But why is that NVDA can announce extra information for some controls and comes with various app modules for different applications? This is done through overlay classes - custom objects and their handlers built on top of API classes (built-in objects).
In essence, overlay classes are subclasses of stable API classes (subclasses are specialist classes deriving (inheriting) from one or more parent classes). This allows custom (overlay) objects to provide extra properties, ranging from control-specific commands to removing certain properties. For example, here's how NVDA's way of announcing toast notifications (Windows 8.x and 10) works:
1. Toasts are notifications from apps, and they are UIA objects (NVDAObjects.UIA.Toast).
2. When events fired by toasts are received by NVDA, it'll check to make sure it is dealing with toast notifications.
3. When NVDA is dealing with toasts, it'll perform what it is told to do by toast objects (announce toasts provided by "report help balloons" is checked from Object Presentation dialog).
Why do objects and overlay classes matter in Studio app module?
Some readers might ask this question after reading the above section on overlay classes. I had to introduce overlay classes because they are important in Studio app module: track items in playlist Viewer are overlay classes. In fact, there are two of them: track items for studio 5.0x (appModules.splstudio.SPLTrackItem) and 5.10 (appModules.splstudio.SPL510TrackItem; in case of 5.10, it derives its power from track item class for Studio 5.0x, which in turn is powered by IAccessible).
These classes were born when I started working on Studio 5.10 support in 2014. Because Studio 5.10 uses a different way of showing track properties, I had to come up with a way to take care of them. Adding to the urgency was the fact that Studio 5.10 uses check marks to indicate whether a track is selected for playback (Studio 5.0x and earlier uses check boxes), and when check marks are checked in Studio 5.10, NVDA would not announce newly checked state, fixed by defining a routine to be used when SPACE is pressed (via an overlay class). In addition, Track Finder (see below) was sensitive to object description changes, I modified it to account for differences between Studio versions.
Then in 2015, when I was designing Track Dial (next article), I thought about scope of this feature. I thought, "if I let this be invoked from everywhere, it could lead to issues such as errors and attempting to use Track Dial from somewhere other than track items". Then I thought, "perhaps I should limit this feature to main playlist viewer at the cost of making sure I identify track items correctly". Given that I had experience with overlay classes and since there was already an overlay class for Studio 5.10 track items, I decided to go with the latter option, which led to defining a new overlay class for Studio 5.0x track items and letting 5.10 track items inherit from this new class.
Track items overview
Each track item in Studio's playlist viewer consists of a row of columns (6 for Studio 5.0x and earlier, 18 for 5.10). As far as NVDA is concerned, it is an overlay class that provides a number of services, including:
* Track Dial toggle and announcing column information.
* Announcing columns in specific order (see the next article on importance of column navigation).
* For studio 5.10, a routine to handle when check marks are checked (when you check a track by pressing SPACE, NVDA will announce the newly checked state and will update the braille display accordingly).
We'll come back to track items when talking about columns in the next article. For now, let's move onto a related feature in Studio app module that works with track items: Track Finder.
Track Finder: Locating tracks given a search string
Track Finder allows you to search for tracks with the given artist or song title. This is done by performing a "linear search" - examining one track item to the next until the search term is found. This feature was partly inspired by similar features in other screen readers and NVDA's own find facility (cursorManager.FindDialog and its friends).
Track finder is not limited to searching for artist or title: a variation of this dialog (called Column Search) allows you to search for text in specific columns such as duration, file name and so on. In reality, it is a single dialog (splmisc.SPLFindDialog) that presents two dialogs (does this sound familiar?). For now, we'll talk about how the original Track Finder works (stay tuned for the next article to learn more about Column Search).
Original track Finder: commands, routines and controls
To use Track Finder, press Control+NVDA+F (wait, I saw this command before). For anyone who are accustomed to NVDA's browse mode, this command would be familiar: find text in webpages. This command performs activities similar to alarm dialogs (see previous articles): after conditions are checked (making sure you are in playlist viewer and you have added at least one track) and setting required flags, NVDA opens Track Finder dialog where you can enter a search term. Click OK, and NVDA will then go through tracks (via trackFinder method) looking for search term in track descriptions (in case you are searching for artist in Studio 5.0x, NVDA will also check the name of the check box, as this holds artist name). Once a track with the search term is found, NVDA will move focus to it (wx.widget.SetFocus), and if not, an error dialog will be shown.
Two other commands are used as part of Track Finder: Find next and previous, assigned to NVDA+F3 and NVDA+Shift+F3, respectively (they also come from browse mode). When these commands are invoked, it'll check if you have searched for a term before, and if not, it'll open Track Finder dialog. If you have searched for a term before, NVDA will perform linear search with search direction specified (trackFinder method in the app module takes various parameters, and one of them is search direction).
The signature of trackFinder method (linear search routine) in the Studio app module is:
trackFinder(self, text, obj, directionForward=True, column=None)
Text is the search term, obj is where the search should begin, direction specifies search direction and column is used if Column Search is used (searching for text in specific columns).
Conclusion
You learned a lot in this article: objects, overlay classes, track items and Track Finder. You also saw how an add-on writer will design a feature and how a design philosophy influences feature development. But this is just a beginning: I'll revisit my notes when explaining how Track Dial and other column navigation commands work in the next article.
References:
1. Sinclair, Rob. Microsoft Active Accessibility architecture, Microsoft Developer Network, August 2000. https://msdn.microsoft.com/en-us/library/ms971310.aspx
2. UI Automation Overview, Microsoft Developer Network. https://msdn.microsoft.com/en-us/library/ms747327(v=vs.110).aspx
3. Java Access Bridge overview, Java SE Desktop Accessibility, Oracle. http://www.oracle.com/technetwork/articles/javase/index-jsp-136191.html
4. Introduction to OOP (Object-Oriented Programming) with Python, Voidspace. http://www.voidspace.org.uk/python/articles/OOP.shtml
5. Non-Programmer's Tutorial for Python 3/Intro to Object Oriented Programming in Python 3 - Wikibooks. https://en.wikibooks.org/wiki/Non-Programmer%27s_Tutorial_for_Python_3/Intro_to_Object_Oriented_Programming_in_Python_3
No comments:
Post a Comment