Gestures to pan screen with arrows

[Boumtchack]

Link to issue number:

Fixes #19471

Summary of the issue:

Users who doesn't use mouse were needing a way to move the screen

Description of user facing changes:

8 new gestures to learn for panning magnifier window

Description of developer facing changes:

N/A

Description of development approach:

• Creating a new class function that will move the magnifier window center based on the direction choosed
• Implementing list variable with xmin/xmax/ymin/ymax, that will update based on zoom level. This is used to know when window reached the edge.

Adding 8 new gestures:

• left, top, right, bottom of a customisable panning value
• left, top right, bottom to edge of the screen.

Testing strategy:

• Manual test of each gestures
• unit tests

Known issues with pull request:

Code Review Checklist:

• Documentation:
  • Change log entry
  • User Documentation
  • Developer / Technical Documentation
  • Context sensitive help for GUI changes
• Testing:
  • Unit tests
  • System (end to end) tests
  • Manual testing
• UX of all users considered:
  • Speech
  • Braille
  • Low Vision
  • Different web browsers
  • Localization in other languages / culture than English
• API is compatible with existing add-ons.
• Security precautions taken.

[Boumtchack]

I implemented NVDA+alt+arrows gestures for step by step panning, the values can be changed, there is a min/max/step variables that can be modified.
for the go to the edge the gestures are NVDA+shift+alt+arrows.

I choosed to force the mouse to stay in the middle of the magnifier window so if someone decides to move it, there won't be a sudden jump of the window.

I'm open to any modifications

[Copilot]

Pull request overview

This PR adds keyboard gestures for panning the magnifier screen, allowing users who don't use a mouse to navigate the magnified view. The implementation includes customizable pan values and edge detection for boundary awareness.

Changes:

• Added 8 new keyboard gestures (4 directional pans + 4 edge pans) with NVDA+Alt+Arrow combinations
• Introduced configurable pan value setting with range from 10-100 pixels
• Implemented pan margin calculation and boundary detection logic

Reviewed changes

Copilot reviewed 7 out of 7 changed files in this pull request and generated 9 comments.

Show a summary per file

File	Description
source/config/configSpec.py	Adds defaultPanValue configuration option with min/max constraints
source/_magnifier/config.py	Implements PanValue class with pan_range and pan_strings methods, plus getter/setter functions
source/_magnifier/magnifier.py	Adds _panValue field, setPanMarginBorder method, and _pan method for panning logic
source/_magnifier/utils/types.py	Defines 8 new MagnifierAction enum values for pan operations
source/_magnifier/commands.py	Implements 8 command functions for panning operations with user feedback
source/globalCommands.py	Registers 8 script handlers with keyboard gesture bindings
source/gui/settingsDialogs.py	Adds pan value dropdown to magnifier settings panel

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[seanbudd]

Please fill out the PR description in full and address CoPilot's review

[cary-rowen]

Do these really need four default gestures?

[CyrilleB79]

Do these really need four default gestures?

IMO yes. These commands are mandatory for any Magnifier user who does not use the mouse. IMO these gestures are much more useful than than gestures to toggle focus tracking mode or to cycle through filters.

What would you suggest instead? Leave them unassigned, or something else?

[seanbudd]

Thanks @Boumtchack - this is just a partial review. Would also like @CyrilleB79's thoughts around panning grain

[CyrilleB79]

I have done some tests on this PR.

1. The paning commands work until the first focus change or the first caret move after the Magnifier is started. After that, they do not work anymore. This critical bug make them unuseful in practice.

2. Being able to define the panning step size seems quite quite interesting, even if I do not remember to have seen this possibility in other magnifiers. I wonder though if it wouldn't be more interesting and useful to define it depending on how much the view is zoomed. E.g. step from 1% to 90% of the zoomed view; this way, I expect that the distance jumped on the magnified view be always the same, helping the eyes to jump at the right place (may be especially useful when user has difficulty to follow movements on screen due to narrow field of vision).

3. Since these commands may be used quite frequently / repeatedly, they are quite verbose. Saying "Panning right" when you pan right does not provide any useful information, since as a user, I already know that I have pressed right arrow key; a user who needs confirmation that they have pressed the correct key can already enable "Echo command keys". My recommendations regarding command feedback would be that when pressing any panning (or jump to edge) command:

• If the magnified view is actually being moved, just a slight beep confirms that the command has been executed
• if the view does not move when the command has been called due to the fact that it has reached the edge of the screen, a different beep (louder / longer) to inform the user that the view cannot be moved. Alternatively, reporting "Left edge" / "Top edge" / etc. is nice too.

4. Wording: What about using "Pan the magnified view" instead of "Pan the magnifier"? This seems clearer to me.

[Boumtchack]

1. The paning commands work until the first focus change or the first caret move after the Magnifier is started. After that, they do not work anymore. This critical bug make them unuseful in practice.

I have tried getting this problem but it works fine for me, have you some settings that are not default that could be the reason of this issue?

2. Being able to define the panning step size seems quite quite interesting, even if I do not remember to have seen this possibility in other magnifiers. I wonder though if it wouldn't be more interesting and useful to define it depending on how much the view is zoomed. E.g. step from 1% to 90% of the zoomed view; this way, I expect that the distance jumped on the magnified view be always the same, helping the eyes to jump at the right place (may be especially useful when user has difficulty to follow movements on screen due to narrow field of vision).

If I understand correctly, it we would change the pan steps value by a % step value so it feels the same for the user regardless of the zoom level?
so the settings would be replace by a % that allows me to calculate the real step value for the panning

3. Since these commands may be used quite frequently / repeatedly, they are quite verbose. Saying "Panning right" when you pan right does not provide any useful information, since as a user, I already know that I have pressed right arrow key; a user who needs confirmation that they have pressed the correct key can already enable "Echo command keys". My recommendations regarding command feedback would be that when pressing any panning (or jump to edge) command:

• If the magnified view is actually being moved, just a slight beep confirms that the command has been executed
• if the view does not move when the command has been called due to the fact that it has reached the edge of the screen, a different beep (louder / longer) to inform the user that the view cannot be moved. Alternatively, reporting "Left edge" / "Top edge" / etc. is nice too.

I guess there is a default bip sound for the good execution of a command? also I've tried to hold the command to pan at 1px and it sounds awfull, I don't know if having a bip would be better, but maybe no one will just hold down the panning command.

I would still keep the arrived to edge ones

4. Wording: What about using "Pan the magnified view" instead of "Pan the magnifier"? This seems clearer to me.

yes I don't mind that

[CyrilleB79]

1. The paning commands work until the first focus change or the first caret move after the Magnifier is started. After that, they do not work anymore. This critical bug make them unuseful in practice.

I have tried getting this problem but it works fine for me, have you some settings that are not default that could be the reason of this issue?

I do not think so: I have tested with a brand new NVDA config with only few changes.
But I seem to have found why you cannot reproduce. You probably use the mouse, while I do not use it at all. Moving the mouse allow to make panning commands work again. Unfortunately, panning command would be actually the most useful for people like myself who do not use the mouse at all.

2. Being able to define the panning step size seems quite quite interesting, even if I do not remember to have seen this possibility in other magnifiers. I wonder though if it wouldn't be more interesting and useful to define it depending on how much the view is zoomed. E.g. step from 1% to 90% of the zoomed view; this way, I expect that the distance jumped on the magnified view be always the same, helping the eyes to jump at the right place (may be especially useful when user has difficulty to follow movements on screen due to narrow field of vision).

If I understand correctly, it we would change the pan steps value by a % step value so it feels the same for the user regardless of the zoom level? so the settings would be replace by a % that allows me to calculate the real step value for the panning

Yes, exact.

3. Since these commands may be used quite frequently / repeatedly, they are quite verbose. Saying "Panning right" when you pan right does not provide any useful information, since as a user, I already know that I have pressed right arrow key; a user who needs confirmation that they have pressed the correct key can already enable "Echo command keys". My recommendations regarding command feedback would be that when pressing any panning (or jump to edge) command:

• If the magnified view is actually being moved, just a slight beep confirms that the command has been executed
• if the view does not move when the command has been called due to the fact that it has reached the edge of the screen, a different beep (louder / longer) to inform the user that the view cannot be moved. Alternatively, reporting "Left edge" / "Top edge" / etc. is nice too.

I guess there is a default bip sound for the good execution of a command?

Do you mean in NVDA in general? (the answer is no) Or just about panning commands? (I cannot hear them; can you point in the code where it is located).

Maybe we should first wait this PR to be merged and have people experimenting with it before we decide what the most desirable audio feedback upon panning commands would be.

also I've tried to hold the command to pan at 1px and it sounds awfull, I don't know if having a bip would be better, but maybe no one will just hold down the panning command.

Yes, when I set step to 1px and hold panning keys pressed, I just hear the beginning of the word "Panning" repeated again and again, what is not very desirable.

With windows Magnifier + Windows Magnifier add-on, I sometimes hold panning keys (control+alt+arrows) to go faster. So with Windows Magnifier, it was clearly a real use case. I have configured Windows Magnifier feedback to beep on panning commands. It gives useful information. But I agree that:

• the audio experience could probably be improved
• holding panning commands has the downside that due to how key repetition works in Windows, you have the first step somewhat isolated and then a series of more frequent steps. That's probably not the best UX.

Alternatively, one can imagine the Zoomtext panning command approach which is much nicer from UX point of view.

1. Press NVDA+alt+leftArrow and maintain NVDA+alt modifiers down.
   • The view pans left continuously (as long as you keep modifiers down)
2. Still keeping NVDA+alt modifiers down, press once again left arrow.
   • The view pans left continuously (as long as you keep modifiers down), with a panning speed twice the panning speed at step 1.
3. Still keeping NVDA+alt modifiers down, press right arrow.
   • The view pans left continuously (as long as you keep modifiers down), with a panning speed equal to the panning speed at step 1.
4. Release NVDA+alt modifiers
   • Panning stops

That's an example and steps 2 and 3 can be omitted, or can also be repeated to increase or decrease the panning speed.

Though, such key commands may not fit well easily with NVDA input gestures system. So if possible, this may be thought and discussed in a separate subsequent issue and PR.

I would still keep the arrived to edge ones

Do you mean that you would keep the spoken message?
It's OK for me. I'd prefer the shorter "Left edge", "Top edge", etc. It seems to me that NVDA often tries to be short and efficient in its feedback messages.

[Boumtchack]

I think the audio issue could be discussed more thoroughly in an other PR with a proper issue.

I came accross an other problem, in my opinion is that if we deal with percents, panning top and right won't have the same value as screens are not squares, so it might feel a little weird.
Maybe I could calculate the value only based on width or height of screen.

For the message I changed it to "at top edge" as suggested by Sean

[seanbudd]

@Boumtchack - could you ask the admins of France-Travail to set up pre-commit CI for your fork? that way it can fix things automatically when you push changes to your fork. I think it will save you a lot of headaches in the long term

[seanbudd]

another partial review - please make sure to review the diff and ensure everything is as expected before marking as ready

[seanbudd]

@Boumtchack - can you resolve merge conflicts from #19483 ?

[CyrilleB79]

Thinking again about edge messages, I think that there are two options about what should be reported.
For the "Jump to left edge" (i.e. "Pan to left edge") and "Pan left" commands, should we here the "Left edge" message:

1. as soon as the magnified view reaches the left edge, i.e. when the left edge is reached at the landing position of the paning command?
2. or when the magnified view is already at left edge and you type a "pan left" or "jump to left edge" command? I.e. when the command should reports that one cannot go left anymore?

Said otherwise, do these command need to report:

1. "You have just reached left edge"?
2. or "You are already at left edge; thus, the command you've just entered will have no effect"

Opting for 1. allows to provide a confirmation feedback that the user has used a jump command (instead of just a move command). On the opposite, opting for 2. provides additional useful information, i.e. indicates if the user was already on left edge or not; option 2. is the same approach as review cursor moving, e.g. shift+numpad7 or shift+numpad9.
I am uncertain of what the best solution is.

[Boumtchack]

What I will do is remove the verbose for the pannings so we can focus on the gesture on this PR, for any verbose or sound request we can open an other one, maybe it would be easier?

[Boumtchack]

@seanbudd

could you ask the admins of France-Travail to set up pre-commit CI for your fork? that way it can fix things automatically when you push changes to your fork. I think it will save you a lot of headaches in the long term

I've managed to install it but I have issues making it work in PR for your repository.
It seems it's the receiving repository that has the power to activate automatic commit.
I've tested from a branch to my France-travail/nvda and it works well, but as you can see in this PR it's not working.

the only way for me to make it work would be to push to my master branch and make PR from this, but it does not feel like a good way to do it.

[seanbudd]

Instead of making this a function, just save the text to a variable, and call ui.message(_ZOOM_LEVEL_MSG.format(zoomLevel=zoomLevel)

Suggested change

	"Zooming in with {zoomLevel} level",
	"Zooming to {zoomLevel}x",

[CyrilleB79]

Since you are modifying this code, I'd suggest something still shorter than @seanbudd's suggestion:

Suggested change

	"Zooming in with {zoomLevel} level",
	"{zoomLevel}x",

Indeed, this message is short, thus efficient, but remain understandable, i.e. we know that it deals with a level modification. The only information we lose is is the zoom direction (in or out), but that's not a problem IMO because:

1. the user probably knows which key they have pressed
2. a user who needs a confirmation of the pressed key can enable speaking of command keys (NVDA+4)
3. in case the direction was wrong, the user can zoom in the other direction without any side consequence.

Moreover, having the most useful information (zoom level value) at the beginning of the message is much more useful, especially when the command can be repeated various times in a row.

More generally, I think that NVDA command feedback need to be short, especially in the case of the magnifier where the visual channel can partially help the user to understand what's on.

Note: This is somewhat off-topic for this PR, but since you are modifying these messages here, I've decided to comment anyway.

[seanbudd]

I dont think we should frame this as "edgeMessage" instead as "hasMoved".
Let's only announce the message when bumping, not first contact.
i.e. let's only return True if the pan causes a move, otherwise False if the pan view doesn't move

[seanbudd]

same here with using a constant

[seanbudd]

f-strings dont work in pgettext

Suggested change

	f"{edge} edge",
	"{edge} edge",