I have a test where I do some soft assertions using pytest_check. The test continues to run even if assertions fail and finally fails at the end throwing assertion error.
Now in allure report, I observe that tests are shown as broken when there’s KeyError/TypeError. Tests are shown as failed when AssertionError happens.
I want to handle these soft assertions and raise KeyError instead to show them as broken in the allure report. How can I achieve it?
The reason for doing this is, I want to classify broken tests as tests that still complete the flow of steps with minor issues in math calculations. Whereas failed tests are tests that cannot proceed to the next step for eg, because a service that can do the calculation is not available.
Is this right? Is there a better way of doing this?
I can use a try except block to handle AssertionError and raise KeyError instead but it still shows as assertion error in report. And since the soft assertions need to be at different places, I can’t use try except, because the test stops at the first except block when I do this. I have also tried assertpy but I don’t want to import a new library just for the sake of doing this.
Here is my code,
import pytest
from pytest_check import check
def test_custom_assertions():
x = 25
y = 21
a = check.equal(x, 23, "x is not equal to 23") # False
b = check.greater(y, 20, "y is not greater than 20") # True
c = check.is_in(30, [10, 20, 25], "30 is not in the list") # False
if False in [a,b,c]:
raise KeyError("Test is broken")
This is how my report looks like for this code error report
