Clicking a Flash object with Selenium

in

The problem: the internet connection back home keeps dropping. Stone the ISP or burn the router? The solution: write a Selenium script to run a speed test every 15 minutes and save the results as screenshots.

Should be done in 15 minutes, right? Wrong! Apparently, this is 2004, and my ISP's speed test page still uses Flash. Fkuc you, Ookla, fkuc you!

Approach #0

Desperately try to click through the entire Flash iframe thing.

iframe with Flash

iframes and Flash... this is gonna' suck...

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
import time

from selenium import webdriver
from selenium.webdriver.common.action_chains import ActionChains


# Firefox ain't got no Flash
browser = webdriver.Chrome()
actions = ActionChains(browser)
browser.get('http://www.rcs-rds.ro/internet-digi-net/testeaza-ti-viteza')
# wait for the page to load
time.sleep(10)
element = browser.find_element_by_css_selector('iframe')
# click all the things!
for i in range(1, 720, 30):
    for j in range(1, 800, 30):
        print('clicking at {}, {}'.format(i, j))
        # I quote: "move the mouse by an offset of the specified element"
        actions.move_to_element_with_offset(element, i, j).click().perform()

Obviously, this fails horribly, and I end up with 97 opened tabs in Chrome.

Approach #1

Navigate to the actual iframe and snoop around the code.

1
2
3
4
function toJava(jsmethod,args) {
 var e = document.getElementById('VoipApplet');
 e.fromJS(jsmethod,args);
}

Java and Applets... nope!

Approach #2

Google a lot and find out there's no way to interact with Flash from JavaScript. Cry. Turns out that Selenium uses the browser's JavaScript API behind the scenes, while Flash objects run in their own environment. Bummer.

Approach #3

Find a GUI automation library for Python and click the damn thing through the OS. Grab the first thing that pops up on Google and begin Ubuntu 14.04 dependency hell.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
mkdir speed-tests
cd speed-tests/
virtualenv -p $(which python3) .venv
source .venv/bin/activate
pip install pyautogui
# ImportError: No module named 'Xlib'
sudo apt-get install python3-xlib
# ImportError: No module named 'Xlib'

# become frustrated
# realize that the Xlib module is not visible from virtualenv
# give up on virtualenv and install everything globally
# never a good idea, but it's already 01:00 AM and I still have to go to work tomorrow

# Google "how to install python 3 packages globally"
sudo apt-get install python3-pip
sudo pip3 install pyautogui
# No module named '_tkinter', please install the python3-tk package
sudo apt-get install python3-tk
sudo pip3 install pyautogui
# ImportError: No module named 'PIL'
sudo pip3 install pillow
# ValueError: jpeg is required unless explicitly disabled using --disable-jpeg, aborting
sudo apt-get install libjpeg8-dev
sudo pip3 install pillow
sudo pip3 install pyautogui

Surprisingly, getting the whole thing to work on Windows was just a matter of pip install pillow pyautogui.

Finished product

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
import datetime
import time

import pyautogui
from selenium import webdriver


while True:
    print(datetime.datetime.now())
    try:
        # Firefox ain't got no Flash
        browser = webdriver.Chrome()
        browser.maximize_window()
        # drop the pleasantries, just open the Flash object
        browser.get('http://speedtest1.rcs-rds.ro/netgauge.swf?v=3.0')
        # wait for the page to load
        time.sleep(1 * 60)
        # make sure the browser is in focus
        browser.maximize_window()
        time.sleep(1 * 60)
        # click the start test button
        pyautogui.moveTo(683, 333)
        pyautogui.click()
        # wait for the test to finish
        time.sleep(8 * 60)
        browser.save_screenshot(datetime.datetime.now().strftime('rds__%Y-%m-%d__%H.%M.png'))
        browser.close()
        time.sleep(5 * 60)
    except Exception as e:
        print e.message

I really hope you enjoyed this, 'cause I sure didn't!