Syncing Braille Cell buttons with OfficeMath

A typical refreshable braille display has a row of braille cells with a button above each cell as illustrated in the figure of a HIMS Braille Edge 40. The purpose of a button is to move the insertion point (IP) to the character or abbreviation represented by cell below the button.

This post discusses how pressing a cell button associated with a math braille cell can move the text selection to the closest matching character position (cp) in the corresponding math zone. The algorithm accounts for the many-to-many relationship between braille cells and OfficeMath character positions (cp’s). A single math symbol may be represented by multiple braille cells (‘=’ is represented by the four braille cells “⠀⠨⠅⠀”) and characters appear in math zones that aren’t shown on a braille display (there’s a start delimiter for ????² that has no counterpart in the corresponding braille “⠁⠘⠆”. This is discussed further in Math Braille UI and below. And there are other cp differences such as hidden text as discussed in the post Using MathML-Based Speech to Edit Math in Different Math Models. In math braille, abbreviations are not used, a simplification that allows Nemeth math braille to be embedded in the braille for any language.

Synchronization algorithm

The algorithm given here to synchronize the math-zone insertion point with a braille-cell button is analogous to the algorithm for finding the cp from a mouse-button hit on a line of text. For the latter, you measure the text until its width equals or passes the hit position. For the braille-cell button press, you

• Get the braille for a whole math zone or some valid selection therein. Note that if you select the start delimiter of a math object, such as a fraction, the selection is automatically extended to include the entire fraction.
• Press a braille cell button and get its offset into the math braille string. I’ve found that Narrator reports the offset via ITextRangeProvider::Move() and Select() calls.
• Recreate the same math braille noting the math-zone cp where the length of the evolving braille string equals the braille offset for the braille-cell button.
• Adjust that cp so that it is a valid IP in the math zone.
• Set the math-zone IP equal to the adjusted cp.
• Once again recreate the math braille to update the braille cell IP (⣀) corresponding to the new math-zone IP.

Braille-cell buttons versus ←/→ key navigation

The left and right arrow keys can traverse every valid cp in an OfficeMath math zone, while the braille-cell buttons only have access to cp’s that have braille counterparts. Let’s consider the two cases mentioned in the introduction. The ‘=’ sign is represented by “⠀⠨⠅⠀”. If you press the button above the starting braille space (U+2800), the corresponding math-zone cp should be in front of the ‘=’. Remember that insertion points in rich text like math zones are in between characters, not on top of characters. If you press a button above any of the remaining three braille characters, the math-zone cp (IP) should follow the ‘=’. Meanwhile a single → key press in front of the ‘=’ moves past the ‘=’.

Now consider ????². In a math zone this is represented by {????|2}, where { stands for the superscript-object start delimiter, | is the argument separator, and } is the superscript-object end delimiter. In RichEdit these delimiters are given by the Unicode characters U+FDD0, U+FDEE, and U+FDEF, respectively. The start delimiter has no counterpart in the corresponding superscript braille string “⠁⠘⠆”. So, pressing the braille-cell button above the ‘⠁’ could set the IP in front of the superscript object or at the start of the superscript-object base (in front of the ????). Structured navigation that selects the whole superscript object or just the ‘⠁’ could be used to choose between these two possibilities. Meanwhile a single → key press in front of the superscript object moves to the start of the superscript-object base.

The subscript object ????₂ also appears as {????|2} in the OfficeMath backing store. The object type is specified in the character formatting of the start delimiter. But in Nemeth math notation, ????₂ is given simply by “⠁⠆”. The subscript operator ‘⠰’ is implied. In computer braille notation, this is “A2”. So, for such simple subscript objects, the start, middle, and end points are all ambiguous. For example, in the string “⠁⣀⠆”, with ‘⣀’ marking the insertion point, the next character typed could be at the end of the subscript base (first argument) or at the start of the subscript (second argument). Math Braille UI disambiguates the two by including a dot 8 (‘⠠’) in the braille-cell character(s) in the object argument that contains the insertion point. If the IP is at the start of the subscript, the math braille for ????₂ is “⠁⣀⢆”, whereas if it’s at the end of the base, the math braille is “⢁⣀⠆”.

Similar cases include the ends of integrands, summands, accents and math-function objects as well as optional arguments like missing integral limits and radical indices (a square root has a missing radical index).

A braille user should have at least three kinds of OfficeMath navigation: braille-cell buttons and vertical scrolling, structured navigation, and the ←/→ keys. Currently, I have to use a keyboard for structured navigation and the arrow keys. But we should be able to assign keys on refreshable braille displays to perform such navigation in addition to the braille-cell button navigation.