Location>code7788 >text

An elegant way to integrate flow limiting, idempotence, and anti-theft brushes

Popularity：648 ℃/2024-09-01 14:04:08

We must have encountered in the work of the interface was wild brush experience, even if not experienced, in the process of interface development, we also need to be easy to brush the interface or and will consume the company's money related to the interface to increase the anti-theft brush function. For example, send SMS interface and send mail and other interfaces, I looked at a lot of domestic products SMS login interface, basically do the anti-theft brush, if not, overnight, perhaps the company are lost bar bar bar bar bar bar bar bar bar bar bar bar bar bar bar bar bar bar bar bar bar bar bar bar bar bar bar bar bar bar bar bar bar bar bar bar bar bar bar bar bar bar bar bar bar bar bar bar.

Suppose we are developing an interface to send SMS (domestic only), the process is as follows

  1. The interface is defined as/sendSms
  2. The only request parameters arephone
  3. When processing the request, we perform a legitimacy check on the request parameter phone
  4. If the cell phone number is legal, then call Tencent cloud and other service providers to send SMS Api, send SMS to the target cell phone number.
  5. End of process

The above is the simplest interface to send an SMS verification code to a cell phone number, without considering other business-related operations. Let's now analyze the problems with this interface (brushing the interface).

  1. Only verify the legitimacy of the cell phone number in the request parameter (11-digit cell phone number), and do not verify whether the cell phone number is empty or not, which can lead to others constructing a large number of legitimate but empty cell phone numbers
  2. No increase in single cell phone number, maximum number of sends per day
  3. Failure to control the sending interval for each cell phone number can result in a large number of SMS messages being sent to the same cell phone number at the same time.

Since we know the defects of the send SMS verification code interface, we will solve these problems one by one, can we avoid the interface to be stolen? The answer is only to a certain extent to prevent theft, because these malicious requests, cell phone numbers are generated through the program unlimited, can pass our regular checks, so the number of times the phone number is sent and send interval restrictions on them is not any effect. In addition, if you want to avoid sending SMS to empty cell phone numbers, you need to introduce additional third-party empty number verification Api, which adds new resource consumption.

Let's now move from the interface that sends SMS CAPTCHA to other interfaces to look for a solution that can be applied to all interfaces and implement flow limiting, idempotent, etc., anti-skimming features.

Public No.: Backend Essay

Personal Blog:/

Solve interface request parameters that are easily constructed

It is not really difficult to find out that the root cause of interface theft is that the request parameters are easily constructed by algorithmic construction, and these parameters generated through the program are legitimate in the eyes of our program.

{
"phone": "11-digit cell phone number"
}

Through the above two comparisons, it is not difficult to find that first for only one parameter phone send SMS interface, want to construct the parameters of Taobao send SMS, the difficulty rises directly by many steps.

Let's start from the perspective of solving the problem of interface request parameters being easily constructed, the only thing I can think of at the moment is to encrypt the request parameters, using asymmetric encryption. The specific idea is that before the client sends the request, it encrypts the request parameters using the public key provided by the server to make the request parameters look less easily constructed. After the server side obtains the request parameters, it uses the private key to decrypt them, and then carries out some subsequent verification operations.

So does this prevent the interface from being stolen? The answer is that it can only protect against a gentleman, not a villain. Especially for the Web side, if the person who initiated the swiping is also a developer, he can find the public key from the js file by directly F12. For the App, the way to get the source code will be a little more difficult, but eventually the public key should still be able to be found.

If we solve the problem of public keys being easily obtained, can we prevent the interface from being stolen in this way? If we can solve the problem of the public key being easily obtained, to a certain extent, it is indeed possible to solve the problem of the interface being stolen, but now that the problem has been shifted to the obtaining of the public key interface, we still need to solve the problem of the obtaining of the public key interface being stolen.

And if the obtained public key can not exist time-sensitive and can be used multiple times, then these interfaces that realize anti-theft swiping through encryption will still have the problem of being stolen if the public key is compromised. If you want to solve it, you can make the public key can only be used once, or can only be used for a very short period of time, and then can only be used by as many requests as you want. My final solution is similar to this, make the token only be used once.

And the use of public keys for encryption, which usually prevents man-in-the-middle attacks that occur during the request process, is meant to solve the problem of parameters being modified as well as leaked.

Ticket mechanism

I didn't end up preventing skimming by addressing the fact that the parameters are easily constructed, I did it by making a determination of whether the request is a bot or not, and if it's an illegal request, forcing it to have to go through a graphical CAPTCHA first, and only if it's a legitimate request will the server process it.

I based on the Ticket mechanism, the client must first request a Ticket from the server before sending a request, the server when processing the request for a Ticket request, the request is judged, the judgment includes whether the request is a malicious request and whether the need to limit the flow. When these two steps are passed, the server will generate an encrypted, time-limited and can only be used once the Ticket, the client sends a real request, you need to carry this Ticket. each Ticket can only be used once, and the client carries the Ticket each time, but also through the Ticket to achieve the idempotency of the request.

This scheme is not coupled to any interface, the Ticket is carried on the request header and does not pollute the request parameters.

Apply for Ticket

I ended up using Ticket to accomplish the three functions of flow limiting, anti-skimming, and idempotency, in order to make this function more generic and not coupled with any interface. When applying for a Ticket, the client needs to pass two parameters, serviceType and primaryKey. serviceType is used to control the type of the interface, and primaryKey will be used to limit the flow. The final combination of the configuration center makes it easy to perform independent flow limiting, UserAgent blacklisting and whitelisting, Ip flow limiting, etc. for any type of request.

The specific implementation process is (take sending SMS verification code as an example):

  1. The client invokes the interface to request a Ticket, passing the parameter{serviceType: sms, primaryKey: user cell phone number}

  2. Server-side validation of client requests

    1. Whether the UserAgent is in the blacklist (malicious requests have basically the same UserAgent), the UserAgent can also have a lot of play, such as similar to the Ip, to limit the flow of the UserAgent (which affects a portion of normal users)
    2. Initial identification of the user from the request header. It is possible to negotiate with the client to work on some of the request header values to help the server recognize the requestor's identity
    3. Identify the IP. Many of the malicious requests come from different Ip's, some of them come from the same network segment, we can restrict the Ip in combination with serviceType.

  3. If the server recognizes the request as malicious, it sets captchaStatus to true in the response body, indicating that graphical CAPTCHA validation is required on the client side

  4. In the next step, the server side gets the flow limiting rules from the configuration, via serviceType. With serviceType+primaryKey as key, see if it can pass the specified flow restriction.

  5. After passing the flow restriction, the server side encrypts {captchaStatus, primaryKey} using symmetric encryption to get the Ticket.The purpose of this step is to get the captchaStatus from the decrypted data when the Ticket is finally verified to avoid that the captchaStatus is passed by the client, thus solving the request Bypassing the graphical CAPTCHA validation problem, the client determines whether the Ticket requires the user to pass the graphical CAPTCHA based on the captchaStatus to perform the subsequent operations.

  6. The server puts the Ticket into Redis and sets the expiration time, then returns {ticket, captchaStatus} to the client.

The Ticket returned by the server is an encrypted ciphertext with an expiration time, stored in Redis, and can only be used once and cannot be constructed by the client. Even if the encryption algorithm is accidentally leaked, the server can't query Redis to find the "legitimate Ticket", so the Ticket is safe enough.

CAPTCHA

The response to a call to the Request Ticket interface contains two parameters: captchaStatus, ticket. captchaStatus indicates whether the ticket requires the client to pass a graphical CAPTCHA.

When captchaStatus is true, the client calls another interface to load the graphical CAPTCHA, when calling the interface, you need to carry the Ticket obtained in the previous step, the server side will eventually bind the graphical CAPTCHA and the Ticket this time, and ultimately realize the verification results of the graphical CAPTCHA obtained through the Ticket in the next step, the specific steps are:

  1. The client carries the requested Ticket to load the graphical CAPTCHA data.
  2. The server gets the Ticket from the request header, queries the Db for the number of times the Ticket has been loaded with the graphical CAPTCHA, and if it exceeds the maximum number of times it has been loaded, then it directly notifies the client to re-apply for a new Ticket and deletes the data associated with the old Ticket.
  3. After the verification passes, generate the graphical verification code data, get the key of the graphical verification code, and then put the key and ticket into the Db to store it, the purpose is to save the graphical verification code verification results
  4. The client receives the graphical CAPTCHA data and loads the

Among the anti-skimming features, the most effective has to be the CAPTCHA feature

Server-side validation Ticket

Once the client has completed the above two, the client now starts calling the real interface (Send SMS). When calling Send SMS Captcha, the client needs to carry the requested Ticket and the graphical captcha Key (if captchaStatus is true).

After the server receives the request, the specific processing steps are as follows:

  1. Get the Ticket from the request, decrypt the Ticket, and query Redis to see if the Ticket exists.

    Even though our anti-skimming logic is known, they are not free to construct Ticket

  2. Get the value of the captchaStatus field from the decrypted data, if it is true, it means that the Ticket needs to perform the graphical CAPTCHA verification. The server queries the DB for the result of the last graphical CAPTCHA Key bound to the Ticket, and ends the process directly if there is no validation or the result is a failure.

  3. The idempotency validation of a Ticket is done mainly by determining whether the Ticket has been used before or not, and if the previous request has already been completed, then get the execution result directly from Redis and return the

  4. When the above are no problem, only now began to implement the final business logic, here is the implementation of sending SMS verification code. Because this function is not coupled with any interface, if we need more detailed anti-theft brush, but also in the specific interface inside the article.

  5. After the execution, you need to delete all the data related to the Ticket.

The above is the specific process I realized the interface anti-theft brush, now let's verify that the anti-theft brush can really prevent (or to send SMS verification code)?

  1. Constructing a large number of legitimate but empty cell phone numbers

    For each request, you need to request a Ticket first, and the primaryKey is the cell phone number. Because these malicious requests have the same UserAgent, if we receive alerts in advance and put the UserAgent in the blacklist, these requests will be blocked directly.

    Even if UserAgent is every intercepted, there are other flow limiting measures like Ip. If all pass, we can also directly force every request for graphical CAPTCHA verification, because the graphical CAPTCHA is much more difficult to crack, basically has dissuaded a lot of people, to force graphical CAPTCHA verification, for normal users, but also will only reduce the use of experience.

    For a cell phone number that is null, if this user does pass all these measures above, then it is basically guaranteed that he is a real user, so the verification of whether the cell phone number is null or not is redundant, in my opinion, unless the resources to send the SMS are really very valuable.

  2. Ticket was compromised, forged.

    In the absence of a mole in the company, the Ticket cannot be forged, and even if it is, the Ticket is not saved to the Db, and if the Ticket's captchaStatus is false and it is leaked, they can only use the Ticket for a specified period of time, and they can only use the Ticket once. It is not possible to have an infinite leak of Tickets.

In the above process, the server side verifies whether the request is a robot or not, and also verifies the real request when it is sent, and if the verification fails, the client performs the corresponding action based on the response body, and then carries the Ticket to resend the request.

The above logic doesn't cache the validation results for normal users, which would result in, for every call to these interfaces, a normal user would need to go through a graphical CAPTCHA.

Other measures

There are other measures that can also increase interface skimming. These include making it more difficult for anti-skimming logic to be cracked and preventing interfaces from being skimmed.

First of all, to prevent the interface from being stolen, essentially to prevent the interface from being leaked. For the App, someone who wants to know our interface information, you must decompile the App, I do not know much about the App decompile, you can try those measures to increase the decompile, even if you do not decompile, the use of the Fiddler tool is also able to see the request information. For the Web side, the user only needs to press F12 to see the JavaScript code, as well as the parameters of each request, the response body and so on. We can disable F12 as well as the right-click (which reduces the user experience), as well as add an automatic entry into infinite Debug mode when the user presses F12 in the production environment. These two operations can increase the risk of our interface being exposed, thus playing a certain degree of "anti-theft brush" purpose.

For increasing the anti-brush logic to be cracked difficulty, there are many apps on the market to limit the flow and other rules have been broken, I personally think it will be broken, in addition to the reasons for the design of the interface, there is another is the interface's response body suggests a very obvious error message. For example, we visit a certain increase in the anti-scrubbing function of the interface, the interface prompts the UserAgent is invalid, the current Ip has been limited to flow, Ticket is invalid, the graphic authentication code has not been verified and other very obvious information. This information has in fact indirectly prompted to make the request to become legitimate steps are, which can help developers debugging, but also indirectly help those who send malicious requests. So in order to make it more difficult to break the anti-skimming logic, we don't need to return these obvious hints, we can return "Illegal Request" no matter what the reason is, for the developers of the company, they will check the meaning of each code from the development document by the code themselves.

These are some of my insights into preventing interface theft, there may be better options, but this is really all I can think of at the moment. Alternatively, you can use existing services, such as CAPTCHA from providers like Tencent Cloud and Aliyun. The graphical CAPTCHA I use is open source and comes from thedromaraBig Brother Open SourceJava Line CAPTCHA, it's very easy to use and supports slider, rotate, swipe, and text tapping, thanks a lot big guy. In addition, because the Ticket requested on each request is encrypted, the performance consumption during encryption and decryption is also a point that can be optimized, depending on what algorithm one chooses.

<Previous Next>