Are you Practicing E2E test? The evolution of test suites for web fronts and native apps, including Selenium, has been remarkable these days, and many projects have already adopted it. Also, with the advent of headless web test automation tools such as Puppeteer, it is becoming less difficult to perform E2E tests on CI servers. On the other hand, since it is the fate of the front end that UI improvements are frequently made and AB tests are performed, the E2E test created with much effort may not work, and the man-hours for repairing it may be troublesome. I think there are many. If the E2E test created just before the release fails, you have to check whether it is a specification change or a degraded. Stopping the release and fixing the test if necessary is something you don’t want to do too much. If the decision is made to tolerate test failures and release, the E2E test is likely to have a sad future that will be abolished in the near future.
By the way, it is a common tendency these days to eliminate the functional difference between PC-Web, SP-Web, and native apps, and to provide RESTful APIs that do not depend on the front end for the back end. The front end calls the same API to provide the same functionality on any device. In this case, the screen operation automated by the E2E test verifies that the backend API call issued along with it and the JSON content returned from the API are displayed correctly on the screen. I think it’s no exaggeration to say that this is something that can be guaranteed by front-end unit tests. The E2E test and the front-end unit test have overlapping coverage.
Therefore, by omitting the parts that can be guaranteed by the front-end unit test such as the display of screen items, and preparing and executing a series of API calls assumed from the use case as a test scenario, the quality of the function of the entire system It makes sense to secure. And you can make it less susceptible to front-end UI changes. Also, in this configuration, if unit tests are performed firmly for each area, it is unlikely that bugs that can only be detected by the E2E test that automates the operation of the front end will be mixed. Therefore, it is very convenient to have a test runner that issues only HTTP requests according to the test scenario created from the use case to the RESTful API group without automating the operation of the front end.
As many of you may know, the Karate test suite fits this need perfectly.
Karate runs a test scenario written in a format close to natural language called Gherkin. Karate is also a kind of so-called Cucumber, but it has the big advantage that it does not require programming for testing because the necessary step definitions are provided in advance.
With some HTTP knowledge, product owners, managers who accept and check quality, QA engineers, and young engineers in the first year can also create test scenarios. The specifications of the API to be tested will be shared within the project, so all you have to do is call the required API and focus on checking the consistency of the results.
For example ・log in ・find a product ・add a specific item to the cart ・settle the cart
In this use case, I think there are many cases where there are four consecutive API calls, By verifying whether the HTTP status code, response body, response header, etc. match the expected value, It is possible to carry out the same level of testing as the E2E test including the front end. It is also possible to include the value contained in the response header of the API call in the request body of the subsequent API call. Of course it is possible, so it is not difficult to adapt it to the actual use case.
However, creating a test scenario from a use case may be possible for non-developers as long as it is a member involved in the project, but writing it correctly as Gherkin for Karate (actually DSL) is a little until you get used to it. It takes time and effort.
Therefore, we have released “Karate Gherkin Visual Editor” to make this description as easy as possible.
You can write a full-scale test scenario just by entering the required items. Anyone can write test scenario because it can be used with a browser and no IDE is required.
Next time, I would like to introduce the functions of “Karate Gherkin Visual Editor” in detail. See you!
