关于canvas控件定位的参考实现

[Amoyshmily (霍格沃兹-百里)]

关键词：selenium，canvas元素

实现思路：偏移坐标

• 利用常规定位方式获取canvas元素
• 利用坐标偏移计算出画布之上的内部元素的位置
  • move_to_element_with_offset(to_element, xoffset, yoffset)

参考代码

"""
__author__ = '霍格沃兹测试开发学社'
__desc__ = '更多测试开发技术探讨，请访问：https://ceshiren.com/t/topic/15860'
"""

from selenium import webdriver
from selenium.webdriver import ActionChains
from selenium.webdriver.common.by import By


class TestCanvas:

    def setup_class(self):
        self.driver = webdriver.Chrome()
        self.driver.get("https://kitchen.applitools.com/ingredients/canvas")
        self.driver.implicitly_wait(10)
        self.driver.maximize_window()

    def teardown_class(self):
        self.driver.quit()

    def test_canvas(self):

        # 获取canvas元素
        canvas = self.driver.find_element(By.ID, "burger_canvas")

        # 获取元素的尺寸
        canvas_dimensions = canvas.size  # {'height': 576, 'width': 864}

        # 计算偏移量（以canvas元素的左上角为原点）
        canvas_center_x = canvas_dimensions.get("width") / 2
        canvas_center_y = canvas_dimensions.get("height") / 2
        button_x = canvas_center_x + (canvas_center_x / 4) * 3
        button_y = canvas_center_y + (canvas_center_y / 4) * 3

        # 动作链
        ActionChains(self.driver)\
            .move_to_element_with_offset(canvas, button_x, button_y)\
            .click()\
            .perform()

偏移量计算示意图

[jhf917 (慧)]

老师，您好！
这个代码只是图片定位，但是没能获取值。
我要问的是，列表是一张canvas图片，想要断言列表中的字段值。如下图

[Amoyshmily (霍格沃兹-百里)]

帮忙确认下，页面源码中是否有标签中含有你关注的字段值，例如下面这段：

<script>
var canvas_el = document.getElementById('canvas');
var canvas = canvas_el.getContext('2d');
canvas.font = '60px Arial';
canvas.strokeText('Answer: 64803',90,112);
</script>

如果有的话，可以考虑参考以下方式解决：

"""
__author__ = '霍格沃兹测试开发学社'
__desc__ = '更多测试开发技术探讨，请访问：https://ceshiren.com/t/topic/15860'
"""

from selenium import webdriver
from selenium.webdriver.common.by import By


class TestCanvas:

    def setup_class(self):
        self.driver = webdriver.Chrome()
        self.driver.implicitly_wait(10)
        self.driver.maximize_window()

    def teardown_class(self):
        # self.driver.quit()
        pass

    def test_canvas(self):

        self.driver.get("https://the-internet.herokuapp.com/challenging_dom#edit")

        # 获取canvas元素
        elements = self.driver.find_elements(By.TAG_NAME, "script")

        content = ""
        for ele in elements:
            html = ele.get_attribute("innerHTML")
            if "canvas.strokeText" in html:
                content = html[html.index("Answer"):html.index("',")]
                break
        print(content)

另外，如果这招不管用，那么通过selenium可能无法实现目标了。

原因参见：How to Get Text inside a Canvas using Webdriver or Protractor - Stack Overflow

The basics are that you can’t with Selenium. the CANVAS tag is like an applet in the page. It doesn’t actually contain any HTML. There are a few options:

1. If you have access to the devs, you can have them expose an API for you so that you can access text, etc. using Javascript from your Selenium script. If it’s part of some library, etc. the library itself may provide an API that you can use. This is the most reliable option.
2. For executing actions, you can use coordinates. You can base all click, etc. actions off coordinates but this is highly dependent on the browser rendering, screen resolution, etc. This will not help you get text out of the CANVAS though.
3. For the text, you really don’t have any options to get the text directly. You could take screenshots and verify the text after the run is complete but that’s about your best option. If you wanted to get really fancy, depending on the text, etc. you may be able to find an OCR library that will be able to extract the text from the screenshot that you took.

也许要考虑借助OpenCV做图像识别了。