It broadens your horizons for friends who are learning technology, provides reference for technology learning directions, and can choose one or several of the directions you are interested in; the express logistics companies that look very low on the surface need thousands of internal IT teams.
1. Massive data from express delivery companies
Massive scan data: The express delivery company needs to process 100 million packages in one day during its peak business. These packages need to be scanned at each link, and more than 1 billion scan data may be generated in a day. The processing of these core business data requires a huge server cluster, and the data needs to be distributed to major business systems.
Massive image data: Every express parcel needs to take pictures when it is transferred to each transfer center. When the express parcel is damaged and lost, which link is wrong, who lost it, who needs to compensate, etc., all require evidence of these picture records.
Massive vehicle trajectory data: tens of thousands of trucks, where they drove, whether they arrived on time, whether they were detours, whether they were punished, subsidies, and the analysis of traffic stories, etc., all require the analysis of the vehicle's trajectory data, an estimation point in a few seconds, or even more trajectory point data. These data require efficient storage and rapid reading to provide display and analysis.
Massive surveillance video data: surveillance video of the operating site, vehicle loading and unloading video, driver driving video, automatic sorting line working video, and security video surveillance data of various factories across the country require huge network bandwidth and flexible technical control of video surveillance.
2. The amount of data processed by daily business
The headquarters is several thousand people, and each branch has thousands of people. There are also tens of thousands of franchise companies in various places that need to work together and need to process data in the business system, such as user order data, customer complaint data, various business approval data, personnel entry, resignation, promotion, and various financial settlement data. There are generally dozens of business systems to hundreds of business systems. The information system of a huge national company requires huge backend technical support.
If purchased from outside, although it is easy to use, it cannot be effectively integrated with dozens of internal systems. The information systems sold to large companies are often very expensive and are prone to being kidnapped in various ways. Moreover, subsequent modifications and optimizations are far away. Therefore, large companies with good performance need to use independent development, which may not be the most advanced but most in line with the company's development needs.
3. The scanning record of each express delivery is displayed quickly
When each user querys the dynamics of express delivery on his or her mobile phone or computer, he or she can quickly display the trajectory data, which requires the optimization display of massive big data from non-database technology, such as HBase database, single number reversal distributed storage, etc. Otherwise, it is very difficult to quickly display the scan trajectory data under high concurrency according to traditional databases.
4. Express scanning data push partners
The express scanning trajectory needs to be pushed to major partners, such as major e-commerce partners, national regulatory departments, and e-commerce platforms also assess the integrity and timeliness of information, which require strong data push capabilities, push failures, push status monitoring capabilities, etc., and various data push requirements of various partners are compatible.
5. Asynchronous push data by message queue
Massive data needs to be pushed to the subsystem of each other, and it also needs to be pushed to each partner. Message queue technology needs to be used to start working in a multi-channel non-interference mode. The huge message queue requires various monitoring, expiration strategies, etc.; the message queue can ensure that each system does not interfere with each other, the crash of a certain subsystem will not affect the normal operation of other subsystems, and the optimization and improvement of a certain subsystem will not affect the stable operation of other subsystems. Message queue is the core middleware for the coordinated operation of multiple systems.
6. Information and data security
Customer's order data, user's mobile phone number, express delivery picture data, and product data purchased by users are all confidential data of the company and cannot be obtained by telecom fraudsters. Various information leakage needs to be prevented, and various information security measures are required. Picture servers, file servers, database servers, application servers, etc. all need to have good information security guarantees.
7. System stability assurance and video surveillance system
Tens of thousands of virtual machines, various servers, and various applications need to operate stably, hard disk storage, network equipment, and strong monitoring capabilities, and even 24-hour uninterrupted on-site personnel monitoring. If the system is paralyzed for half an hour, it will be a huge blow to companies with national business.
8. Big data platform extracts data from each subsystem
After years of optimization and construction, some systems may be barely able to use and run, but may not be able to withstand the needs of various massive data processing. Incremental business data needs to be extracted into the big data platform to perform large-scale massive data calculations on the big data platform.
9. Statistical summary of various massive data
For example, there are 1 billion scan data every day, 30,000 franchise outlets, and hundreds of thousands of internal employees; there are hundreds of transshipment centers distributed throughout the country. These people need to conduct cost-effective analysis, various performance data analysis, various business data summary, and order data of nearly 100 million per day.
For example:
Which center has the most serious decline in business volume? quantity? percentage?
Which express delivery outlet has the most customers lost? Is the order volume declined significantly?
Which courier has the most complaints?
Which branch has the highest delivery efficiency?
Which customer has the most complaints?
These all require various calculations from hundreds of billions of business data. It may take several hours to go on a SQL statement. After the cluster operation of dozens of servers, you can draw conclusions: ordinary small database servers have no ability to store, let alone calculate.
10. Purchase cloud services and build your own systemAt that time, the technical strength was not enough, so it could be achieved by purchasing various cloud services, but it was often kidnapped by service providers. Various fees may be bottomless. The scale was large enough and it could be relied on the company's own strength to control the huge information system by itself; it depends on its own technical capabilities and cost-effective analysis.