20 GUI Automation

Question 1

1. How can you trigger PyAutoGUI’s fail-safe to stop a program? (2x)

Answer 1

1. Slam Mouse to Corner

2. Log out (ctrl+alt+del)

Question 2

2. What function returns the current resolution()?

Answer 2

> > > import pyautogui

|»_space;> w, h = pyautogui.size()

Question 3

3. What function returns the coordinates for the mouse cursor’s current position?

Answer 3

> > > pyautogui.position()

Question 4

4. a. What is the difference between pyautogui.moveTo() and pyautogui.move()?
   b. How can you specify the action to take 0.25 seconds?

Answer 4

1. MoveTo: moves TO coordinate
   »>pyautogui.moveTo(100, 100, duration=0.25)
2. moves from current point by the indicated pixel amount
   »>pyautogui.move(100, 0, duration=0.25)

Question 5

5. What functions can be used to drag the mouse?

Answer 5

1. pyautogui.dragTo(width, height, duration=X)

2. pyautogui.drag(x, y)

Question 6

6. a. What function call will type out the characters of “Hello, world!”?
   b. How do you write it into a text field at (100,200)

Answer 6

a.
»> pyautogui.write(‘Hello, world!’)

b.
»> pyautogui.click(100, 200); pyautogui.write(‘Hello, world!’)

Question 7

7. How can you do keypresses for special keys such as the keyboard’s left arrow key?

Answer 7

Single characters can use pyautogui.write() (e.g. ‘a’, ‘e’)

Other letters, (e.g. ‘Shift’, ‘F1’) can use:

1. pyautogui.press(‘XY’)
2. pyautogui.keyDown(‘XY’)
3. pyautogui.keyUp(‘XY’)

press() is a ‘wrapper’ of keyDown and Up, which simulate pressing a key and then releasing it.
That is handy, e.g. when you want to keep ‘shift’ pressed until you pressed ‘c’, and only release it afterwards.

Example prompts, XY =
'enter'
'f1'
'esc'
'volumemute'

Question 8

8. How can you save the current contents of the screen to an image file named screenshot.png?

Answer 8

> > > im = pyautogui.screenshot()

|»_space;> im.save(‘screenshot.png’)

Question 9

9. What code would set a two-second pause after a PyAutoGUI function call?

Answer 9

> > > pyautogui.sleep(2)

Question 10

10. If you want to automate clicks and keystrokes inside a web browser, should you use PyAutoGUI or Selenium?

Answer 10

If it is always the same browser which is in full-screen mode, on the same screen resolution etc., then PyAutoGui is likely to be faster (and yet more prone to error).

Question 11

11. What makes PyAutoGUI error-prone?

Answer 11

Blind to what it is clicking / typing

Question 12

12. How can you find the size of every window on the screen that includes the text Notepad in its title?

Answer 12

> > > notepadWindows= getWindowsWithTitle(‘Notepad’)
for win in notepadWindows:
print(win.size)

Question 13

13. How can you make, say, the Firefox browser active and in front of every other window on the screen? How can you print out its size=

Answer 13

Specific:
»> ff = pyautogui.getWindowsWithTitle(‘Firefox’)
»> ff[0].size
-> Careful: getWindowS returns a list. To access its content (even if only one match was found), .size or other functions cannot be used on ff.size, but on ff[i].size

in general, 4x functions:
pyautogui.getAllWindows() Returns a list of Window objects for every visible window on the screen.

pyautogui. getWindowsAt(x, y) Returns a list of Window objects for every visible window that includes the point (x, y).
pyautogui. getWindowsWithTitle(title) Returns a list of Window objects for every visible window that includes the string title in its title bar.
pyautogui. getActiveWindow() Returns the Window object for the window that is currently receiving keyboard focus.

Question 14

14. How can you find out certain mouse locations conveniently?

Answer 14

> > > pyautogui.mouseInfo()

Disable “3 seconds…”

then move your mouse to the desired pixel coordiantes and press f6

Question 15

• Almost all functions

Answer 15

moveTo(x, y) Moves the mouse cursor to the given x and y coordinates.

move(xOffset, yOffset) Moves the mouse cursor relative to its current position.

dragTo(x, y) Moves the mouse cursor while the left button is held down.

drag(xOffset, yOffset) Moves the mouse cursor relative to its current position while the left button is held down.

click(x, y, button) Simulates a click (left button by default).

rightClick() Simulates a right-button click.

middleClick() Simulates a middle-button click.

doubleClick() Simulates a double left-button click.

mouseDown(x, y, button) Simulates pressing down the given button at the position x, y.

mouseUp(x, y, button) Simulates releasing the given button at the position x, y.

scroll(units) Simulates the scroll wheel. A positive argument scrolls up; a negative argument scrolls down.

write(message) Types the characters in the given message string.

write([key1, key2, key3]) Types the given keyboard key strings.

press(key) Presses the given keyboard key string.

keyDown(key) Simulates pressing down the given keyboard key.

keyUp(key) Simulates releasing the given keyboard key.

hotkey([key1, key2, key3]) Simulates pressing the given keyboard key strings down in order and then releasing them in reverse order.

screenshot() Returns a screenshot as an Image object. (See Chapter 19 for information on Image objects.)

getActiveWindow(), getAllWindows(), getWindowsAt(), and getWindowsWithTitle() These functions return Window objects that can resize and reposition application windows on the desktop.

getAllTitles() Returns a list of strings of the title bar text of every window on the desktop.