Some insights on how to do interface testing step by step

Some insights on how to do interface testing step by step

Unknowingly, it has been nearly a month since I did interface testing in the company. Since I haven't done an interface test before, I took a lot of detours when I got started, and to be honest, I'm still taking detours, so I can only say that it is insight.

The purpose of interface testing

This is a clich , but I think that as long as I talk about the interface, I can't get around it. If there is no goal, there is no judgment standard, so the purpose of the test is still very important.

First move the English explanation on Wikipedia (Chinese is not found, even Baidu's...):

API testing is a type of software testing that involves testing application programming interfaces (APIs) directly and as part of integration testing to determine if they meet expectations for functionality, reliability, performance, and security.[1] Since APIs lack a GUI, API testing is performed at the message layer.[2] API testing is now considered critical for automating testing because APIs now serve as the primary interface to application logic and because GUI tests are difficult to maintain with the short release cycles and frequent changes commonly used with Agile software development and DevOps).[3][4]

The translation is:

API testing is a software testing activity that is part of the integration testing, which determines whether the functionality, reliability, performance, and security meet the expectations by directly controlling the interface (API) of the application under test. [1] Since APIs do not have GUI interfaces, API tests are all carried out at the communication layer. [2] API testing now has a very important position in automated testing, because APIs are generally the main interface of application logic, and GUI testing is difficult to maintain in the rapid iteration and frequent changes of agile development and DevOps. [3][4]

[1]:Testing APIs protects applications and reputations, by Amy Reichert, SearchSoftwareQuality March 2015

[2]:All About API Testing: An Interview with Jonathan Cooper, by Cameron Philipp-Edmonds, Stickyminds August 19, 2014

[3]: The Forrester Wave? Evaluation Of Functional Test Automation (FTA) Is Out And It's All About Going Beyond GUI Testing, by Diego Lo Giudice, Forrester April 23, 2015

[4]: Produce Better Software by Using a Layered Testing Strategy, by Sean Kenefick, Gartner January 7, 2014

Then I am currently at work Leader's expectations for me:

Find as many problems as possible. Let me talk about the background first. He is the main program of the app. His expectation for me to do interface testing is to find interface problems as soon as possible, and to reduce the problems found in the joint debugging of their app and server. He gave an example. For example, the server returned a {"width":30, "url":"http://xxx/a.jpg"}, then there may be an inconsistency between the actual width of the picture and the width field.

Communication with Dadong and other people who have done API testing:

Covering business, including normal/abnormal business conditions

My understanding from the beginning:

Found the problem, eh. That is, if the required fields mentioned in the document are missing, try only the required fields, and try using illegal values for the required fields. If the optional field is missing, try the illegal value of the optional field.

The result is: at least 6 use cases for an interface, or even more. And in many cases, the business may not be measured (for example, the interface only returns success and failure, but it may actually fail but the return is success, and you need to call other interfaces to verify)

Current understanding (it is estimated that it is still a bit off):

Combine business design use cases. If there is a page turning parameter, try the default value, valid value, last page, last page +1, invalid value. Design use cases for various parameters and business scenarios.

Implementation of interface testing

At this point I took two paths: Jmeter and pure Java code. Now it seems that neither of them is a good way, so I won't share them. Here I mainly talk about some recent interface testing tool posts on Testerhome, and briefly summarize their implementation methods:

Summary of the interface tests done so far

How to put it, I think although what I am doing now is also an interface test, but the use cases I design are more specific functions. For example, I will call the upload interface (check the uploaded file md5 by the way, the order is correct, and the related attribute fields are correct), send the circle interface (basically check the return value is enough, but there must be different types Text, including special symbols, length, etc.), check the circle interface (image md5, sequence, correctness of related attributes, correctness of the publisher, correctness of the data on replies and likes). Because there are three independent interfaces, each interface needs to have a certain encapsulation method (for sending a message, obtaining a specified field in the return value, etc.). Each packaging method averages about 10 lines of code, and a use case with 3 interfaces basically requires 3x10+10 lines of code (some instructions and parts that are not well packaged). A function roughly covers 10 use cases, so 10x40 lines of code are required, which is 400 lines of code. Excluding some code that can be shared, it basically needs about 300 lines of code (some complex functions will be more).

PS: I have counted my daily code volume through git, which is about 1000, but the number of use cases is still about 15 per day.

It can be seen that my speed is mainly slow because the amount of code is too large, time is spent on coding, and the code will take a certain amount of time to debug, so the efficiency is naturally low. So if you optimize the way of writing use cases, even if you vomit blood, you won t be able to write fast. Next, we have to summarize those places that can be extracted and completed by recording or automatic generation, and condensing the writing case into filling in forms or pure data.

Updated 2015.11.30

Today I communicated with Monkey and another colleague who has done api testing in the company, and found the most fundamental problem: There is a problem with my use case design

This concept is more difficult to explain clearly, let's take the posted circle of friends mentioned above as an example.

Suppose I want to verify the interface of posting Moments, its workflow is as follows:

Upload pictures, the server returns the url of these pictures

Publish Moments, including two main fields: text content of Moments and URL of each picture. The server will only return whether it is successful and the id of this circle of friends

The use case I designed for normal posting to Moments is as follows:

Use the upload picture interface to upload pictures and verify whether the pictures are uploaded successfully and whether the md5 of the file corresponding to each url is consistent with the md5 of the picture I uploaded locally.

Use the post local circle interface to post the local circle, where the picture url comes from the return value of the first step

Use the view local circle interface to view the sent local circle, and check whether its content and picture are correct.

At first glance, there seems to be no problem (I believe that the children's shoes who have done the api test have already seen the problem), but in fact, this use case itself has a serious problem: the interface has become a means, and the verification point is actually a function.

There should be two main types of api test cases I got after communication:

Single interface test. Calling an interface is a use case

Multi-interface business testing. Multiple interfaces will be called, and there may be data transfer between the interfaces.

The api test is the simplest, and each interface should have at least one use case should be to call a single interface with default parameters. The point is here: single. The use case like mine above is not to verify the interface of Moments, but to verify the function of Moments.

So what are the use cases for the interface in Moments? I rethinked the idea of the current test interface, there are mainly these:

There are pictures and texts, and it is expected to return and publish successfully

No picture, text, expected to return to publish successfully

There is a picture, no text, it is expected to return to the post successfully

No picture, no text, expected to return to publishing failure

There are pictures and texts, and the expected return is successful, and the records in the database are in line with expectations

One point is tested for each use case, and it is sufficient if the return value of successful publication is covered by a single use case.

If the use case is confirmed, it can be seen that there are actually many points that can be reused in the interface test. The essence of interface testing is to verify whether the return value/database change meets expectations through the permutation and combination of test parameters, so as to determine whether the interface-related code is correct. Thinking from a business perspective means that these permutations and combinations are not just thought out, but permutations and combinations that may appear in the business.

This is why many interface test cases are written in excel form, because it is the easiest and quickest to present the parameter arrangement and combination in a form.

There is also an example that the more concise the use case, the better.

For example, the list interface has a page turning function. It has an input parameter: lastTopicId, and there is an isMore field in the return value. lastTopicId indicates the topic from which to display, and isMore indicates whether there is a next page (1 means yes, 0 means no).

For one of the use cases of this interface, it is necessary to verify whether the isMore field is 0 on the last page. My initial idea was to keep turning pages like a function until it reached a large number of pages or reached the situation where isMore is 0. Since the number of pages is uncertain, even if this very large number of pages is set very large, there is no complete certainty that it will never exceed this number of pages. So there is a problem with the idea itself.

After talking with Monkey and my colleagues, he pointed out that the correct idea should be:

Through a trusted way (calculate directly from the server database, or have a port telling you what the lastTopicId of the last page is), use one step to know how to get to the last page at once.

Go directly to the last page

Verify the isMore parameter value

Some things can be easily developed for us, so we don't have to calculate it so hard by ourselves, but just let the development add an interface so that we can directly get the data. To make interface testing better, development assistance is inseparable.

Doing a good interface test is not as simple as previously imagined, but it is not as difficult as it was at the beginning. In any case, since it has started, then we must find a way to do it well. Next, I will use the new design use case ideas for the remaining interface test cases, and modify the Java code to support the use of excel as the use case. If there is something wrong in your insight, you are welcome to raise it in time.

For more information about interface testing, you can add QQ group: 747981058