Look at other people's Controller, it's so elegant!

preamble

In practice, we need to often deal with third-party platforms, may docking third-party platform Controller interface, or provide Controller interface to the third-party platform call.

So the question is, what if you design an elegant Controller interface that meets the needs of: security, re-callability, stability, good problem locating, and many other aspects?

Today, I'd like to talk with you about some of the things you need to pay attention to when designing Controller interfaces, and I hope you'll find them helpful.

1. Signature

In order to prevent the data in the Controller interface from being tampered with, very often we need to do asign (one's name with a pen etc)。

The interface requestor willRequest Parameters + timestamp + keysStitch into a string and then pass themd5etc. hash algorithm that generates a front sign.

Then in the request parameter or request header, add the sign parameter to be passed to the API interface.

The gateway service of the API interface gets the sign value, then splices it into a string with the same request parameters + timestamp + key, generates another sign using the same m5 algorithm, and compares the two sign values to see if they are equal.

If the two signs are equal, the request is considered valid and the gateway service of the API interface forwards the request to the corresponding business system.

If the two signs are not equal, the API interface's gateway service will simply return a signature error.

Here's the question: why is there a timestamp in the signature?

A: For security reasons, to prevent the same request from being utilized over and over again, increasing the likelihood of the key not being cracked, we must set a reasonable expiration time for each request, for example: 15 minutes.

Such a request is valid for 15 minutes, beyond which the gateway service of the API interface will return an exception indicating that the validity period has been exceeded.

There are currently two forms of generating keys in signatures:

One is for both parties to agree on a fixed value privateKey.

Another is that the API interface provider gives two values of AK/SK, and both parties agree to use SK as the key in the signature. the AK interface caller passes it to the API interface provider as the accessKey in the header, so that the API interface provider can get SK based on the AK, and generate a new sgin.

2. Encryption

In some cases, our Controller interface directly passes very important data, such as: the user's login password, bank card number, the amount of money transferred, the user's ID card, etc., if these parameters, directly in clear text, exposed to the public network is a very dangerous thing.

From this, we need to encrypt the data asymmetrically.

Currently used more often withRSA。

RSA contains a pair:public keycap (a poem)private key。

Let's take the user login password as an example.

After the user enters the password, the password needs to be encrypted on the front end using a public key.

The public key is kept in the front-end code, so it doesn't matter if it's leaked to someone else.

This is because passwords encrypted with a public key can only be decrypted using the corresponding private key in the back-end service.

And our private key is stored in the configuration of the back-end service, so no one else can get to it.

Therefore, encryption and decryption using RSA is secure.

We can use an online tool to generate key pairs: /rsa-key-pair-generator

3. ip whitelisting

To further strengthen the security of the API interface and prevent the signature or encryption of the interface from being broken, an attacker can request the interface on their own server.

Demand limitation requestsipIncreaseip whitelist。

Only ip addresses in the whitelist can successfully request the API interface, otherwise it directly returns no access.

The ip whitelist can also be added to the API gateway service.

However, it is also important to prevent the company's internal application servers from being breached, in which case requests for API interfaces can also be initiated from internal servers.

This is where you need to add a web firewall, e.g. ModSecurity, etc.

4. Flow limitation

If your API is called by a third-party platform, this means that the frequency of calls cannot be controlled.

When a third-party platform calls your API interface, if the concurrency is too high all of a sudden, it may cause your API service to be unavailable and the interface to hang directly.

From this, the API interface must be madecurrent limit。

There are three methods of current limiting:

1. Do a flow limit on the request ip: for example, the same ip, in one minute, to theTotal number of requests to the API interface, can't be more than 10,000 times.
2. Do a flow limit on the request interface: for example, for the same ip, in one minute, for theSpecified API Interface, the number of requests cannot exceed 2000.
3. Do flow limiting for requesting users: e.g. the sameAK/SK users, in one minute, the total number of requests to the API interface, can not exceed 10,000.

We can do this in practice bynginx，redisorgatewayRealize the function of current limiting.

5. Parameter validation

We need to do the API interfaceparameter calibrationFor example: checking required fields for null, checking field types, checking field lengths, checking enumerated values, and so on.

This will intercept some invalid requests.

For example, when adding new data, the field length exceeds the maximum length of the data field, and the database will report an error directly.

But this kind of abnormal request, we can completely in the API interface to identify the early stage, there is no need to go to the database to save data that step, wasting system resources.

There are some amount fields that are supposed to be positive, but if the user passes in a negative number, in case the interface doesn't do a checksum, it could lead to some unnecessary losses.

There are also some status fields that, without checksums, can lead to saved data exceptions if the user passes in enumeration values that don't exist in the system.

This shows that it is very necessary to do parameter calibration.

The most used checksum data in Java is thehiberate(used form a nominal expression)Validatorframework, which contains @Null, @NotEmpty, @Size, @Max, @Min annotations.

It is very easy to verify data with them.

Of course there are some date fields and enum fields that may require parameter validation by way of custom annotations.

6. Harmonization of return values

I've called someone else's API interface before and the normal return data is a json format, for example:

{
    "code":0,
    "message":null,
    "data":[{"id":123,"name":"abc"}]
},

The json format of the signature error return:

{
    "code":1001,
    "message": "Signature error".
    "data":null
}

No data permissions returned in json format:

{
    "rt":10,
    "errorMgt": "No permissions".
    "result":null
}

This is a rather pitiful approach, where there are multiple different formats of return data in the return value, which can make it difficult for the docking party to understand.

In this case, it is possible that the API gateway defines a return value structure all the time and the business system defines another return value structure. If it is a gateway exception, the return value structure defined by the gateway is returned, and if it is a business system exception, the return value structure of the business system is returned.

However, this will result in the API interface returning different return value structures when different exceptions occur, which is very detrimental to the maintenance of the interface.

In fact, this is a problem we can designAPI GatewayTime to settle.

The business system throws a business exception RuntimeException when an exception occurs, where there is a message field that defines the exception message.

All API interfaces must go through the API gateway, which catches the business exception and then converts it into a unified exception structure for return, which unifies the return value structure.

7. Harmonized encapsulation of exceptions

Our API interface needs to have a clear understanding of theexceptionsHarmonized treatment.

I don't know if you have ever encountered this scenario: sometimes in the API interface, you need to access the database, but the table doesn't exist, or the sql statement is abnormal, so you will directly return the sql information directly in the API interface.

The return value contains theException Stack Information、Database information、Error Codes and Linesand other information.

It would be a dangerous thing to expose this content directly to third-party platforms.

Some criminals, using the interface return value of this information, may be sql injection or directly off the library, and cause some damage to our system.

Therefore it is very necessary to do a uniform handling of exceptions in the API interface and convert them to this:

{
    "code":500,
    "message": "Internal server error".
    "data":null
}

return codecodebe500Return informationmessagebeInternal server anomalies。

This way third-party platforms know that there is an internal problem with the API interface, but don't know the exact cause, and they can come to us to troubleshoot the problem.

We can print out the stack information, database information, lines of error code, etc., in the internal log file.

We can find out more about this in thegatewayThe exception is intercepted in the middle of the process, a unified encapsulation is done, and then what is given to the third-party platform is an error message that is processed without sensitive information.

8. Request log

When a third-party platform requests your API interface, the interface request log is very important, through which you can quickly analyze and locate the problem.

We need to record the request url, request parameters, request header, request method, response data and response time of the API interface into the log file.

preferablytraceIdIt can be used to cascade the logs of the entire request and filter out redundant logs.

Of course, there are times when the request logs are not only for your company's developers to view, but also for users of third-party platforms who need to be able to view the interface's request logs.

This is where logs need to be landed in a database, for example:mongodborelastic search, and then make a UI page to enable viewing privileges for users of the third party platform. That way they can view the request logs on the extranet and they can locate part of the problem themselves.

9. Power design

It is very likely that the third-party platform will request our interface many times in a very short period of time, e.g., requesting twice in 1 second. It is possible that their business system has a bug, or in doing interface calls failed to retry, so our API interface needs to be madeidempotent design。

That is to say, to support in a very short period of time, the third-party platform with the same parameters to request the API interface many times, the first request database will add new data, but after the second request will not add new data, but will also return success.

This is done so that no erroneous data is generated.

We can do this on a daily basis by working oncomprehensive databaseaddunique indexor in theredissave (a file etc) (computing)requestIdand request parameters to ensure interface idempotency.

For those of you interested in butt power and other sensations, check out my other post,How to ensure interface idempotency under high concurrency?", which is described in great detail.

10. Limiting the number of records

For batch interfaces provided to me, be sure toLimit the number of records requested。

If too much data is requested, it's easy to cause the API tointerface timeoutand other issues that make API interfaces unstable.

Typically, it is recommended that up to 500 records are supported to be passed in as parameters in a single request.

If the user passes in more than 500 records, the interface gives a direct prompt.

It is recommended that this parameter be made configurable and negotiated with the third-party platform in advance to avoid unnecessary problems after going live.

11. Pressure testing

Before we go live, we must do a little bit of work on the API interfacestress test, knowing that each of the interfacesqpsSituation.

So that we can better predict how many server nodes need to be deployed, critical for the stability of the API interface.

Previously, although the API interface to do the flow restriction, but in fact the API interface can reach the limit of the threshold, this is a question mark, if you do not do the stress test, there is a great risk.

For example: your API interface limits the flow to only 50 requests per second, but the actual API interface can only handle 30 requests.

We can use it in our workjmeterorapache bencDo stress testing on API interfaces.

12. Asynchronous processing

The logic of the general API interface is handled synchronously, returning results immediately after the request.

But sometimes, the business logic inside our API interfaces is very complex, especially some batch interfaces, and if we synchronize the business processing, it will take a very long time.

In this case, to improve the performance of the API interface, we can change to theasynchronous processing。

In the API interface you can send amq messageand then returns success directly. After that, there's a specializedmq consumerGo ahead and consume the message asynchronously to do the business logic processing.

The interface for direct asynchronous processing is available to third-party platforms in two ways.

The first way is that wepull back (of a key (in music)interfaces of third-party platforms, informing them of the results of the processing of API interfaces, which is how many payment interfaces are played.

The second way is that third-party platforms throughpollingCalls our other API interface for querying the state at regular intervals, passing in a parameter that is a collection of ids from that previous API interface.

13. Data desensitization

Sometimes when the third-party platform calls our API interface, some of the data obtained is sensitive data, such as: user's cell phone number, bank card number and so on.

Such information, if retained directly to the extranet through the API interface, is very insecure and can easily lead to the problem of user privacy data leakage.

This requires that some of the data be madeData desensitizationUp.

We can use part of the returned data with theasterisksSubstitute.

The user's cell phone number has been used as an example:182****887。

This way, even if the data is compromised, only a portion of it is compromised, and there's not much use for an unscrupulous person to get their hands on this data.

14. Complete interface documentation

To be honest, a complete API interface documentation, when both sides do interface docking, can reduce a lot of communication costs, so that the other side to take a lot less detours.

The interface documentation needs to contain the following information:

1. interface address
2. Request method, e.g. post or get
3. Introduction to request parameters and fields
4. Introduction to return values and fields
5. Return codes and error messages
6. Encryption or Signature Example
7. Full request demo
8. Extra notes, e.g. Open ip whitelist.

It is a good idea to standardize the naming style of interface and field names in the interface documentation, e.g., both use thecamel-humped logoName.

You can add a version number v1 to the interface address, e.g.: v1/query/getCategory, so that if there is a big change in the interface in the future, it will be very easy to upgrade the version.

Harmonize the type and length of fields, e.g., id field is of type Long with a specified length of 20. status field is of type int with a fixed length of 2 and so on.

Unify the time format field, e.g.: time with String type, format: yyyy-MM-dd HH:mm:ss.

The interface documentation states AK/SK and domain name, look for so-and-so to provide it separately, etc.

 

One final note (ask for attention, don't patronize me)

If this article is helpful to you, or if you are inspired, help scan the QR code below to pay attention to it, your support is my biggest motivation to keep writing.
Ask for a one-click trifecta: like, retweet, and watch at.
Concerned about the public number: [Su San said technology], in the public number reply: into the big factory, you can get free access to my recently organized 100,000 words of the interview dictionary, a lot of partners rely on this dictionary to get a number of big factory offers.