I am a big fan of Playwright, a tool for end-to-end testing that was originally built for Node.JS but is also available in Python and other languages.
Playwright 101
For example, here's a simplified test for the chat functionality of our open-source RAG solution:
def test_chat(page: Page, live_server_url: str):
page.goto(live_server_url)
expect(page).to_have_title("Azure OpenAI + AI Search")
expect(page.get_by_role("heading", name="Chat with your data")).to_be_visible()
page.get_by_placeholder("Type a new question").click()
page.get_by_placeholder("Type a new question").fill("Whats the dental plan?")
page.get_by_role("button", name="Submit question").click()
expect(page.get_by_text("Whats the dental plan?")).to_be_visible()
We then run that test using pytest and the pytest-playwright plugin on headless browsers, typically chromium, though other browsers are supported. We can run the tests locally and in our GitHub actions.
Viewport testing
We recently improved the responsiveness of our RAG solution, with different font sizing and margins in smaller viewports, plus a burger menu:
![Screenshot of RAG chat at a small viewport size](https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgkrX8jxwS1UUznS730bJbZ6dxOJlZxCGa65whpZPABZCvmdwovtlmVo10JDaaTMMRjf-VoHJFVA5FMWGB5zMAIXqIvanDb4r_p0By8u5q-RZRyM0hnbwUwCnu_8w5zjrk_ZcD1fVXgiHN46iwCWc_jB3rajWMZD9yr2gMz3Vtyk0oD5b4-QwgAl1KV0A/s320/Screenshot%202024-07-10%20at%209.21.18%E2%80%AFAM.png)
Fortunately, Playwright makes it easy to change the viewport of a browser window, via the set_viewport_size function:
page.set_viewport_size({"width": 600, "height": 1024})
I wanted to make sure that all the functionality was still usable at all supported viewport sizes. I didn't want to write a new test for every viewport size, however. So I wrote this parameterized pytest fixture:
@pytest.fixture(params=[(480, 800), (600, 1024), (768, 1024), (992, 1024), (1024, 768)])
def sized_page(page: Page, request):
size = request.param
page.set_viewport_size({"width": size[0], "height": size[1]})
yield page
Then I modified the most important tests to take the sized_page
fixture instead:
def test_chat(sized_page: Page, live_server_url: str):
page = sized_page
page.goto(live_server_url)
Since our website now has a burger menu at smaller viewport sizes, I also had to add an optional click()
on that menu:
if page.get_by_role("button", name="Toggle menu").is_visible():
page.get_by_role("button", name="Toggle menu").click()
Now we can confidently say that all our functionality works at the supported viewport sizes, and if we have any regressions, we can add additional tests or viewport sizes as needed. So cool!
No comments:
Post a Comment